AI Panic!

Imagine, someday, we’ll lose the control over our computers to some sentient being, unable to stop it or to unplug the millions of home pcs, office workstations or lab mainframes affected.

Well, there is no need to imagine, this very thing has been happening for a long time now, in the form of Zombie Computers. These machines have been compromised by hackers, viruses or trojans, and are remote controllable. Often they are grouped in a network of zombie computers, called a botnet.

The RSA 2008 conference in San Francisco earlier this month had a panel discussion on this topic, and the tone was “a mix of resignation, indignation and post-9/11 rhetoric”, according to Threat Level. According to the panelists, botnets are the biggest threat in the internet today and a danger to national security.

However, little is done to combat the use of zombie computers. Although some botnet operators are caught once in a while, the spam and attacks they create does not seem to decrease. I agree with Ira Winkler, a security consultant who said at the conference:

“The problem is no one is doing anything. Guess what? If your system has a bot on it, you don’t get on the internet. [...] We need to hold people responsible when they present an imminent threat to other people.”

However, ISPs are reluctant to cut off offenders, as they often have no clue what’s going on and respond with confusion and angry phone calls, which drive up the customer support costs.

The inability to quickly and efficiently stop these bot nets could prove very helpful for a malevolent AI. Once released on the internet (or on its successor, maybe The Grid), there is virtually no way to stop it again. The infrastructure to spread uncontrollably is already given by the botnets. Unless there is some fundamental change on the network level and software level (hello Microsoft!), the only way to stop such an outbreak would be the complete shut-down of the net. Of course, this would cause quite a disturbance to the world economy, as most financial transactions and international trading are entirely done online. Most nations will be hesitant to risk that, just because some strange program has been set free from a university lab somewhere.

Therefore I think the current infrastructure of the web eases the spread of an AI (Panic Level: +3%), once it discovers the weaknesses of connected hosts and “understands” how botnets work. But that’s not yet an immediate threat; after all we have to develop the artificial general intelligence first, which, mind you, is a little tricky.

The game of Go has been one of the last big board games where artificial intelligence strength was nowhere near experienced players, let alone master grade professionals. During a recent Go tournament in Paris however, a professional 5th Dan grade player, Catalin Taranu, was beaten by a computer program (diagram and picture of the match below).

The catch is, it was played only on a 9×9 field, which is smaller than the usual 19×19 field. This no doubt had part in the victory, as the overwhelming state-space is reduced from 10¹⁷⁰ to 10³⁸ possible game positions (chess: about 10⁵²).

According to Earth News, that was the first ever recorded victory of a machine over a Go master (in a sanctioned non-blitz game). The culprit is the MoGo artificial intelligence engine. It has been developed by INRIA - the French National Institute for Research in Computer Science and Control - running on a Bull NovaScale supercomputer. During the 3-game match on the 22nd March 2008, MoGo managed to win the second game. On a 19×19 board it was weaker, losing with against Taranu, who had a handicap of 9 stones.

About ten years ago, the then strongest Go program, The Many Faces of Go, lost against a master level player although he had a 29 stone handicap.

To be honest, this is about the speed I expect research to have here, sooner or later processing power and better algorithms will make computers unbeatable in any game with logical rules, and Go is argued to be the most complex of all games. If you happen to know a complexity theorist, she will be happy to certify you that the complexity of Go is nasty EXPTIME-complete, which is not where you want to be (assuming you are a computer program and want to be fast when you’re grown up), really. That’s why it is nigh to impossible to ever “solve” Go, but it shouldn’t be too hard to win against us puny earthlings - even on a proper-sized board.

We have been trying to model our successors robot servants to resemble human beings physically and psychically for a while now. While the psychical development is taking its first steps out of its literal infancy, the physical resemblance has still a long way to go. It has to cross the uncanny valley:

The uncanny valley is a hypothesis that when robots and other facsimiles of humans look and act almost, but not entirely, like actual humans, it causes a response of revulsion among human observers. The “valley” in question is a dip in a proposed graph of the positivity of human reaction as a function of a robot’s lifelikeness.

This valley is the major hurdle towards believable and likeable humanoid model, be it robots, 3D animations or game avatars.

A common bail-out here is to depict characteristics generally considered humanoid by transmuting or exaggerating these features. Comics and the typical Pixar CGI movies are a very good example. The suspension of disbelief is the key factor here: We accept that comic-like depictions are there to interpret reality and show us a filtered, deliberately unrealistic image of the plot. It is very easy for our imagination to fill in the gaps and to ignore physical inaccuracies.

This gets much harder when we try to be as realistic as possible by using correct proportions, complex shading and detailed textures. The movie Beowulf almost perfectly falls into this category: It is completely CGI, but uses real actors and tries to simulate reality. While still-images still look alright, it is the animations where it goes downhill. The trailer has some examples in it, especially the horse animations look wrong, which is much more obvious in the actual movie (while technically not directly a humanoid animation, the galloping horses still add to the feeling that something is wrong with the depicted “reality”).

While not directly concerned with the uncanny valley, some military robots are quite creepy in my opinion. For example the Boston Dynamics “Big Dog”, which somehow looks like two soldiers bending over and pushing against each other. In a sense, it is the opposite to Beowulf: The movement, especially when it tries to regain its balance or slips on ice, looks quite natural, but the visual appearance (and sound) is really disturbing.

Scientists from the MIT are trying - but not quite managing - to avoid the valley with a robot using the comic-approach. They have developed the Nexi, a what they call MDS robot - mobile, dexterous, social. Both the pictures of this robot and the video of Nexi talking and gesticulating look a bit creepy, the robot reminds me of Jigsaw from the movie Saw.

They say that the “purpose of this platform is to support research and education goals in human-robot interaction, teaming, and social learning.”

The antipathy the uncanny valley introduces leads to a potential problem when regarding this social aspect of human-robot interaction: Once humans start to enhance themselves with transhuman enhancements, for example improved eye-sight, nanobots that increase the IQ or prostheses that work better than their natural counterparts, there may be a sudden rift between enhanced and “normal” humans, caused by the antipathy mentioned above. This hypothesis was introduced by Jamais Cascio, who further states that once such technologies gain further distance from human norms, “transhuman” individuals would cease to be judged on human levels and instead be regarded as separate entities altogether (this point is what has been dubbed “posthuman“), and it is here that acceptance would rise once again out of the uncanny valley.

I think the idea of the uncanny valley makes it harder for malicious AIs to undermine human civilisation, as fear of robots is more easily sparked with the help of uncanny-valley-antipathy (AI Panic -1%). However, the concept might be known to an intelligence, and thus we might see robots that does not look like human beings at all - or worse, like our childhood Disney-heroes!

London-based charity Landmine Action wants autonomous robots capable of killing people banned under the same kind of treaty that has outlawed landmines in over 150 countries. According to the New Scientist it is the first time a high profile non-governmental organisation has campaigned against such a technology. This campaign follows the reasoning of Noel Sharkey, who condemned these automation plans earlier this year.

As I’ve written before, the robots in use by the military nowadays (and the next years) are almost fully automatic, but so far the trigger has still to be pulled by a human soldier. However, it is only a question of time until the software is strong enough so that this decision will be made entirely by the machine. And once the software is in place, there will be no ethical opposition - at least in the US Department of Defence, who wants them in future to work without supervision.

A reaction to this news article comes from Wired, where the idea of danger through war robots is dismissed:

But to argue as if this is in the here or now, or even in the next decade, is just plain silly. The Pentagon has not only never advocated taking the man-out-the-loop of targeting decisions for drones or robots, its current policies and procedures would prohibit such a move (some might argue that international law already prohibits autonomous armed drones). [...] Unless and until those policies are drastically altered, it’s safe to say we are safe from renegade Terminators.

To completely ignore the threat by robots with weapons and justifying this ignorance by saying that these robots are still science fiction and it will take decades until these robots appear seems a bit strange to me, especially as Wired itself has reported about existing armed robots before. It is exactly this ignorance and belief that everything is so far away that allows organisations like the military to push these developments without any opposition.

Furthermore, I think it is a little blue-eyed to think that the control of weapons mounted on otherwise fully automated robots will remain in human hands just because the Pentagon does not admit of having plans that say otherwise. By arguing that international law might already prohibit autonomous armed drones and at the same time clearly seeing that these very drones are being used by the U.S. Army in Iraq right now, Wired maneuvers itself on very thin ice. I don’t think it can support the arguments that promises, laws and policies are sufficient enough to protect us from a - what Wired deems nonexistant - danger. And heck, what is so bad about protesting against future dangers as opposed to only trying to fight seeing the effects of existing weapons like landmines in hindsight?

I think it is about time that more voices are raised against automating war machinery, and the Landmine Action has taken a step in the right direction. If it will be heard by the military - and the U.S. Army is without doubt on the forefront of research towards this automation - is a completely different matter. So far, I think the chances are still slim, to specify I say it lowers the AI Panic Level by -0.1%.

I’ve been home and relaxing a bit over Easter the last days, and didn’t get around to write much on AI Panic (partly because I wanted to have Wikipedia on my MP3-Player which took a while to program). Although most of the stuff that happened is already yesterdays news now, I feel I should mention it anyways, as it is relevant or interesting to AI. 

An elderly man has killed himself by programming a robot to shoot him in the head after building the machine from plans downloaded from the internet. Apparently, he used a jigsaw power tool that was connected to a .22 semi-automatic to create a “robot” that could fire multiple shots once triggered remotely (i guess by connecting the plug of the jigsaw tool to the power supply). This simple contraption (althoug remarkably complex for an 81 year old who’s tired of life) is nowhere near a modern, autonomous robot. It is just way too much hassle to put a sophisticated AI into a machine whose single purpose is killing its creator. Remember, we are talking about suicide machines here, so the “user” of that machine actually cooperates and works towards its goal. So I see no danger from the AI side in these robots. Military robots are more likely to turn into suicide robots — unintentionally.

Wired has a very well written and long article about Ray Kurzweil. It covers his early inventions that earned him his first little fortune while still an undergraduate, his reading machines for the blind, his pills to extend his life and of course his thoughts on the technological singularity. Nothing ground-breaking, but if you want to have glimpse into Ray Kurzweils life it is a good read.

There is an interesting paper on Utilitarian Essays about Friendly AI. The author analyses an approach called CEV (Coherent Extrapolated Volition) and his concerns with this approach. The essay does not deal at all with an AI turing evil, but only with the problems of extrapolating volition. There are three concerns the author rises: whose volitions to extrapolate, what about wild animals and lab universes and possible ramifications for religion. I’m not yet familiar enough with CEV to feel competent to criticise the raised points extensively (Michael Anissimov has some more interesting thoughts on the essay). Some quick questions that crossed my mind would be: How big is the role of primal urges and drives in general volition? Will we be able to do anything at all if we decide to always decrease suffering in every creature that can suffer? How can lab universes physically exist at all? What if we don’t like the extrapolated volition (”Trust me, it’s all for your best!”)?

Pioneering science fiction writer, visionary and futurist Arthur C. Clarke, best known for his work on the movie 2001: A Space Odyssey (where he created the prime example of an AI turned bad, HAL 9000), has died in his adopted home of Sri Lanka at the age of 90 on the 19th of March. He was also known for “inventing” the idea of geostationary satellites and was the last living of the “big three” science fiction writers. The two others were the Russian-born Isaac Asimov, who died in 1992, and Robert A. Heinlein, a Missouri native who died in 1988. Although he didn’t believe in an afterlife (or religion), he does live on through his brilliant works and ideas.

Also, I’m happy that my blog has been mentioned by Michael Anissimov during a Radio Interview and my post on COSPAL has been picked up by io9, which gave me a new visitor record of whopping 120 visitors on one day. I know I know, but you have to cherish the small things in life … and I don’t think thats too bad for a blog only 2 months old. AI Panic was also the Site Of The Week on scifi.com a while back, which is cool! I am quite surprised how quick I got addicted to media attention once the blog was up, sometimes I catch myself just watching the visitor counter go up and checking where all the visitors come from… Thanks for reading, dear visitor

With the military building scary armed robots already, it was about time for robots that are scared of these monsters. Phobot, a Lego Mindstorms Robot built by students from the University of Amsterdam, is exactly that: A robot that is scared of bigger robots.

Now here comes the pedagogical value: It can “learn” to lose its fear by conditioning it with increasingly big and evil looking things. The process, shown in the video below, is meant to help children with phobias to learn that not everything is as scary and bad as it seems. Especially not the autonomous sentry gun protecting the kindergarten. Enjoy!

(Via Spiegel Online)

The march towards learning, fully functional AI systems has taken another step. A cognitive AI system with the intelligence level of a puppy has been created to learn and perform simple tasks through a robotic arm.

Sure, a puppy is (often) quite stupid, but that achievement should not be underestimated, especially as the researchers at the EU-funded COSPAL project claim that this level of cognition and learning goes beyond the current state of the art.  They combine artificial neural networks (ANNs) to control low-level motor actions and a classical rule-based system to combine these actions to form a plan.

Although this advancement of AI potentially is a step towards a fully aware system, i.e., an AI system that is able to reason and learn from all its surrounding world information (for example through a common sense knowledge database), it is not an unexpected development nor something close to self-awareness, as Michael Felsberg from the research team at Linköping University explains:

In human terms, our robot is probably like a two or three year old child, and it will take a long time for the technology to progress into the equivalent of adulthood. I don’t think we will see it in our lifetimes.

His pessimism shows that Artificial General Intelligence is still quite far out, and while I think he might be overestimating the time-span for it (alas, the Singularity is scheduled earlier than after our lifetimes), he’s still one of the researchers on the forefront of AI research, and thus I think it is safe to say that we will still have to wait quite a while before smart enough AIs appear that could be dangerous (AI Panic -2%).

(Via PhysOrg.com)

It sounds like the sermon Noel Sharkey preached last month about the ethics of autonomous systems has been really necessary. The time-frame we are talking about here is smaller than one might think - I suspect just 10-15 more years and we will see autonomously shooting robots. If there is a war at that time and morality debates fail, that is.

A reporter from Popular Mechanics visited an U.S. Army test area in Texas recently and wrote about the demonstration of the “MULE”, a six-wheeled utility vehicle, which is currently remote controlled by an Xbox 360 controller, but the developers at Lockheed Martin plan to include autonomous movement by the scheduled release date 2014. Already 1700 of these Humvee sized beasts have been ordered by the U.S. Army, and while they say that the use of the mounted weapons (4 antitank missiles and a machine gun) in this generation of robots will always have to be initiated by a human controller, I suspect that this will be “optional” in future versions. Maybe in the form of a bAutoFire = false; that can easily be overridden in desperate situations after shouting a key phrase like It’s coming right for us!

The article questions the readiness of the robots for battle, and comes to the conclusion that there’s still a long way to go and the robots are still clumsy and slow. However, the hardware is completely ready for battle (and probably has been for a while). It is just the software that is still missing. Just some little intelligent pathfinding, friend/foe distinction, an aimbot and a wallhack and we’re all set!

Well, of course it is not that easy. Indeed, the artificial intelligence is by far the most difficult problem these robots face, and as mentioned, it’ll take another couple of years until we see reliable algorithms that prevent friendly fire (My guess would be that they come up with a hardware based solution first, an active transmitter or reflector that indicates a friend. However, that could become problematic when soldiers are captured.)

Eventually, the software side will be solved, and only the ethical issue will remain. War ethics are a difficult terrain and probably won’t count for a lot when an officer sees his soldiers at risk and decides to push the button. The huge budget of the U.S. Army lures researchers and universities like Carnegie Mellon to put their brainpower to use on modern AI, AI that will be used to control weapons to kill people.

Now I’ve rambled along way too much again, time to finish up. I’ll increase the AI Panic Level by +1% (not more because I’ve already credited the general risk of autonomous army robots in the post about Noel Sharkey).

 Addition (March 22nd, 2008): There are also flying prototypes, like that bat spy plane shown below. The U.S. Army awarded the University of Michigan College of Engineering a five-year, $10-million grant to construct and build it. While it sure is more difficult to build a flying object than an earthbound robot, the challenges lie more in the physical realization than in a better AI.

Unmanned flying drones are in use by the military already and have been used in Iraq to fire missiles and collect terrain information.

Thanks Igor Gabrielan and Erk Subasi for letting me know!
If you see something in the net that would make a good AI Panic article, feel free to contact me!

A new molecule structure is able to control 8 nano-machines simultaneously by switching the states of the involved molecules. This chemical “brain” has been developed at the International Center for Young Scientists, Tsukuba, Japan, reports Engadget and BBC.

The scientists use 17 duroquinone molecules that could (one day) act as a central control for other nanobots and are themselves controlled via a scanning tunneling microscope. They have already built bigger machines capable of 256 simultaneous operations, but using these machines in a computer - while faster than a conventional pc - is very difficult at the moment because of the required scanning tunnel microscope.

Nanorobotics is, alongside artificial general intelligence, one of the most likely technologies to radically change the way human beings live and develop in the next decades, according to most futurologists. While many applications are very promising, especially when concerned with body-enhancing functions (longevity, intelligence boosters), some people believe that these robots may run out of control and, for example, replicate uncontrollably and devour every living organism on earth (grey goo scenario). I am still undecided which variant I find more likely, I haven’t read enough material on nanorobotics yet. At first sight, the grey goo scenario seems less likely to happen than the evil AI scenario, because replication appears to be a very complicated and sensitive process to me.

However, assuming nanobots become reality, they will affect the creation of strong AI: If they are used to improve the IQ of researchers, say by suppressing the need for sleep or increasing the brain capacity, the AI development will accelerate. On the other hand, if everything is turned into goo, obviously there will be no AI that enslaves human beings (other than swarm AI maybe). The impact on the probability of an evil AI thus depends on how likely the different nanorobotics scenarios are.

What if the people you talk to, play with and interact with in Second Life weren’t human at all? Granted, none of them is literally human anyway, but almost all of them are actually controlled by someone sitting in front of a computer. Not all are.

You might run into a little, 4 year old child that appears to be reasonably smart and is able to reason about its surroundings. This child is controlled by a program. A program that is able to reason about its own beliefs to draw conclusions in a manner that matches human children its age.

The program is the result of research conducted by a group around Selmer Bringsjord at the Rensselaer Polytechnic Institute. According to ScienceDaily, Bringsjord says:

Truly convincing autonomous synthetic characters must possess memories; believe things, want things, remember things. Such characters can only be engineered by coupling logic-based artificial intelligence and computational cognitive modeling techniques with the processing power of a supercomputer. [...] The logico-mathematical theory will include rigorous, declarative definitions of all of the concepts central to a theory of the mind, including lying, betrayal, and even evil.

Using Second Life as a platform for this makes sense. Reducing the communication bandwidth is a common trick to lower the barrier of believability. Even the good ol’ Alan Turing limited the interaction for his famous Turing test to test if a computer can demonstrate intelligence to a text chat. Second Life is reasonably close to that, avatars can easily be controled by keyboard commands.

Building agents based on the theory of mind (which describes the ability to attribute mental states to oneself and others and to understand that others have beliefs, desires and intentions that are different from one’s own) is a step towards an artificial general intelligence, and thus towards a real threat (or a great opportunity, depends on who you ask). And indeed, this project was presented a couple of days ago at the Artificial General Intelligence conference held at the University of Memphis.

I believe there is a high risk connected to the creation of such an intelligence, it is easy to think of hypothetical scenarios in which a reasoning system concludes that humans are a risk and need to be wiped out. Although testing such an agent in a virtual world is a good idea to realise what it is capable of and evaluating the actual risk of it, once the tools are there the step into the real life will be taken quickly. And sure enough, Selmer Bringsjord has plans to apply the technlogy to education and homeland defense. Good thing the system has no sense of lying, betrayal and isn’t evil…

The researchers want to build an interface with a certain resemblance to the holodecks in Star Trek for direct interaction of agents with human beings. While this definitely sounds like science fiction, it has to be taken seriously. They have the support of strong industrial partners and want to use the worlds most powerful university-based supercomputing system at the CCNI.

The research is still in an early stage and the Second Life results are far away from being strong AI, but they might well bring us closer to an artificial general intelligence, thus raising the AI Panic Level by +1%. I don’t know whether the system they are working on will achieve a level of strong AI (most likely not), and even if it does, we don’t know if it will be good or bad. I’ll finish with a quote from the group’s research homepage, which interestingly enough seems to answer that thought:

An advanced synthetic character, however, can literally be evil, because it has the requisite desires, beliefs, and cognitive powers.