Having a couple of raw drafts in wordpress, I never really got around to finish a proper post during the summer. But now that Michael Anissimov ended his summer break with a post on the non-storyness of the future, and I just had a nice post on Wall-E in my queue, I felt I had to waffle a bit about that topic, too. Damn peer pressure!

Last week I saw the new Pixar animated movie Wall-E, and I must say, it’s one of the best movies I’ve seen in a while. Although I have to admit that I’m a sucker for animated and movies loaded with special effects, critics agree with me here, and it’s already on #26 of the all time best movie list on imdb (Warning: minor spoilers ahead).

It’s an interesting story with lots of (heavily) anthropomorphised robots in it, which is already a gem for its slapstick and brilliant display of robotic “emotions” alone. However, it also picks up the classic idea of machines disobeying humans and robots acting on their own judgement, with an interesting twist: Mankind has to be saved by robots gone rogue that act against other command-obeying robots. Now if this conveys the right ideas to the younger audience, I don’t know. However, it makes the younger generation think about “what could happen if robots were ubiquitous and we relied entirely on this autopilot?”.

Wall-E draws an exaggerated but hilarious picture of our future selves of what I’d consider American unlimited consumerism. In an almost matrix-esque way humans are reduced to stupid meatballs without any sense of reality. But I guess that’s necessary in order for the audience to sympathise even more with the robotic protagonists. And, by the way, from a “real AI” point-of-view the programming of the robots in Wall-E isn’t very sound. They possess all kinds of human characteristics that make absolutely no sense to be programmed into specialised robots (such as trembling because of fear) while they lack others (a robot with a full-blown personality, but no proper sound output?). But of course, that’s not the point of a movie. The content was made interesting on a human level, as Michael phrased it.

This cunning bridge brings us right to his the future is not a story-post. Micheal argues that only stories that humans can relate to are interesting:

For a story to be interesting to humans, it has to feature interesting content occurring at the human level. [...] Conversely, humans cannot write meaningful stories about content above the human level, because we lack the cognitive complexity to imagine such things.

Now, ignoring that the “human level” doesn’t seem to be a fixed barrier to me that cannot be moved, this ironically reminds a bit of religious beliefs into a superiour being: God moves in mysterious ways. And, staying in this limping analogy, many people find god quite interesting, although his motives of letting millions of children starve are indeed mysterious.

Of course, I could have just fallen in this very trap of being unable to imagine anything above the human level, but I just don’t think transhuman (AI) actions will be that much more incomprehensible than, say, a superpower declaring war on a small country to get their natural resources under some false pretenses (WMDs, for example).

Total annihilation is also what Michael has in mind when thinking about unfriendly AIs:

More likely, when confronted by a recursively self-improving unFriendly AI with abstract mathematical goals unrelated to human concerns, the simple outcome is death.

Obviousy, this is not content above the human level, because we just imagined it. You could even call it interesting, as it is definitely relating to human concerns. Hey - maybe we should make a movie out of this!

I agree with Michael on the “interestingness bias” (authors make up showy stories to get attention), especially when fiction is sold as science and scentific authors get carried away by stories that start with “no, it we will survive, because…” and then go on with some fancy explanation, that reduce to “or not” when we apply logic or - God forbid! - occams razor to it. However, I don’t really see a worrisome problem with all that. Of course, the “true” threat might be waved aside as fiction, but that may be true for every futuristic scenario. The more we talk about it, the better. And to be honest, most of the futuristic movies nowadays assume rogue robots anyway, so we’re well prepared!

The world is very different ever since the robotic uprising of the mid-nineties!

I think we were wrong … there doesn’t seem to be a whole lot of AI required for the robotic uprising.

I’ve written about AI in video games and their real-life impact a while ago, with the conclusion that it is quite basic and does not really contain “intelligence”. Now with all the hype around the newest version of the Grand Theft Auto series, GTA IV, and a development budget of about $100 million, lets take a closer look if anything has changed.

GTA is basically a sandbox game with a good story tucked on. As a gangster in a virtual city, this iteration it is Liberty City, modelled after New York, you are free to roam around in all kinds of vehicles, from bikes, lots of cars up to military helicopters and do all kinds of things - most of them illegal, of course. Everything that is forbidden in the real world becomes not only possible, but a goal in GTA, from assassination tasks, drug trafficking, gang wars and - of course - car theft there’s everything there what the villain’s heart desires (I wont go into the psychological aspects here, but it is a real concern for parents and politics, especially the German government is pressing hard to ban Killerspiele).

There is a wide variety of actions in Liberty City, especially when it comes to driving and destroying things. Also, such a city simulation requires believable and complicated (artificial) agent behaviour, to allow the player to suspend his disbelief. And in fact, it is an interesting and often funny pastime to just watch non-player character (NPC) interactions, their discussions and police activity.

The general pedestrian behaviour and driving AI has improved considerably over GTA: San Andreas, but there are still the odd situations where agents don’t react properly. Some mention that the cover seeking behaviour of the police has some glitches, so that policemen in search for a new cover run around the side of the police car facing you instead of the back, or that they fall off buildings when you hide just behind the edge. However, these situations are rare and are often more funny than annoying. The AI, while not perfect, does it’s job alright. All the situations where you say “Wow, I didn’t think it would be that smart” are most likely scripted though, meaning that they have been predefined by the game designers and are not (fully) dynamically generated. Another indicator for the limitations of the AI system is the density of pedestrians, or rather, the lack of it. Similar to the previous incarnations, there are not nearly enough people or cars in the streets. Admittedly, this increase in crowd density is hard to implement correctly and is probably an effect of limited CPU power.

GTA4 is a good example of fake AI, artificial intelligence that only pretends to reason. This is very common in video games, as their primary goal is not to plan realistically and beat the player, but to provide enjoyment and make (smart) mistakes. This becomes apparent in how the game handles the appearance of police units. When your wanted-level increases, there are suddenly more police cars and cops around (especially for the higher levels), that come pouring out from behind street corners or are just there when you turn around. This behaviour, while perfectly legit for a game, would be useless in real life. All the steps which are most important for navigation and reasoning in robots and “proper” AI are cropped away in video games. Therefore there is -still- no possibility that non-player villain behaviour (would that be the police then, in GTA IV?) accidentally or even on purpose crosses over into the real world from a game AI.

Side note: As you may have noticed, I didn’t have a lot time to update AI Panic very often over the last weeks. At the moment I’m quite busy at the university, I have to hand in a kind-of-important internal report in about a month of what I’m planning to do in the next 2-3 years. Once this is out of the way, I’ll have more time for the blog again. Sorry about that!

The University of Pennsylvania has built a new self-reconfigurable robot, which can, as the name suggests, assemble itself after it has been kicked apart. Of course it is not much more than three robots with a smart communication, orientation and interfacing system, but as the atomic units get smaller these compound robots might actually get useful for, say, building a liquid metal Terminator.

There is also a high-resolution version available at the youtube page of the video.

(Via NewScientist)

About 9 years ago, on June 10th 1999, the first humans fell victim to a computer control system failure. Due to a chain of unfortunate circumstances, a the pressure on a 16-inch steel pipe line increased uncontrollably and caused it to rupture, flooding two small creeks near Bellingham in the process. The petrol ignited and killed two 10-year-old boys and an 18-year-old adolescent. Although it was later revealed that the accident was in part caused by human failure, it is still considered as the first cyber-event that killed humans.

Now, you might think that things have changed since then, but you’d be wrong. Joe Weiss from Applied Control Solutions says:

Until eight years ago, my whole life was making control systems usable and efficient, and, by the way, very vulnerable. It is exactly what you will find today in many, many industrial applications. This isn’t just 1999. No, this is June 2008.

And it is true. The computer systems that control complex technical facilities, like the power grid, traffic control infrastructure, or pipeline systems have historically only been optimized with respect to usability, efficiency and effectivity. Security has almost never been a concern in the design of these systems. Many of them have been developed before the rise of the internet, so that attacks had to be carried out physically. But eventually some systems responsible for the control of important infrastructure have been connected to a network, often in an effort to improve control or just because of company policies, without any evaluation of possible risks.

There are more examples. The 787 Dreamliner from Boeing, which is scheduled to come out later this year, may be vulnerable to hacker attacks. This is because the flight control network is separated from the passenger network only by software firewalls, and thus might be attacked from inside the plane (or even from the outside, with the help of a trojan horse maybe).

When military control systems fail, it gets nasty quite quickly. Last year, a robot anti-aircraft cannon killed 9 and wounded 14 soldiers when it went berserk during a training of the South African National Defence Force. However, it is not yet clear if the accident was caused by a mechanical malfunction or software failure.

And it is not only the security of control systems that poses a risk, the control systems themselves are feared by researchers to remove human control from key decision-making processes. Tom Rodden, Professor of Interactive Systems at the University of Nottingham concluded at the Human-Computer Interaction conference that this loss of control might not directly be a risk for our lives, but surrenders “basic human values and concepts such as personal space, society, identity, independence, perception, intelligence and privacy.”

This lack of security must become a higher priority to control system designers, especially as there is an unavoidable trend towards more autonomous, intelligent systems that can cope with the size and complexity of modern infrastructure. We have to take care that we do not hand over all control of critical systems and uncritical but socially important concepts to machines. Even if a worst case scenario - such as malicious AI breaking loose - does not happen, programs are likely to be susceptible to bugs and misinterpreted signals, and therefore have to be very carefully designed. As the examples in the past have shown, these issues are a real threat to human beings and the lack of security against tampering from the outside increases the AI Panic level by +1%.

Imagine, someday, we’ll lose the control over our computers to some sentient being, unable to stop it or to unplug the millions of home pcs, office workstations or lab mainframes affected.

Well, there is no need to imagine, this very thing has been happening for a long time now, in the form of Zombie Computers. These machines have been compromised by hackers, viruses or trojans, and are remote controllable. Often they are grouped in a network of zombie computers, called a botnet.

The RSA 2008 conference in San Francisco earlier this month had a panel discussion on this topic, and the tone was “a mix of resignation, indignation and post-9/11 rhetoric”, according to Threat Level. According to the panelists, botnets are the biggest threat in the internet today and a danger to national security.

However, little is done to combat the use of zombie computers. Although some botnet operators are caught once in a while, the spam and attacks they create does not seem to decrease. I agree with Ira Winkler, a security consultant who said at the conference:

“The problem is no one is doing anything. Guess what? If your system has a bot on it, you don’t get on the internet. [...] We need to hold people responsible when they present an imminent threat to other people.”

However, ISPs are reluctant to cut off offenders, as they often have no clue what’s going on and respond with confusion and angry phone calls, which drive up the customer support costs.

The inability to quickly and efficiently stop these bot nets could prove very helpful for a malevolent AI. Once released on the internet (or on its successor, maybe The Grid), there is virtually no way to stop it again. The infrastructure to spread uncontrollably is already given by the botnets. Unless there is some fundamental change on the network level and software level (hello Microsoft!), the only way to stop such an outbreak would be the complete shut-down of the net. Of course, this would cause quite a disturbance to the world economy, as most financial transactions and international trading are entirely done online. Most nations will be hesitant to risk that, just because some strange program has been set free from a university lab somewhere.

Therefore I think the current infrastructure of the web eases the spread of an AI (Panic Level: +3%), once it discovers the weaknesses of connected hosts and “understands” how botnets work. But that’s not yet an immediate threat; after all we have to develop the artificial general intelligence first, which, mind you, is a little tricky.

The game of Go has been one of the last big board games where artificial intelligence strength was nowhere near experienced players, let alone master grade professionals. During a recent Go tournament in Paris however, a professional 5th Dan grade player, Catalin Taranu, was beaten by a computer program (diagram and picture of the match below).

The catch is, it was played only on a 9×9 field, which is smaller than the usual 19×19 field. This no doubt had part in the victory, as the overwhelming state-space is reduced from 10¹⁷⁰ to 10³⁸ possible game positions (chess: about 10⁵²).

According to Earth News, that was the first ever recorded victory of a machine over a Go master (in a sanctioned non-blitz game). The culprit is the MoGo artificial intelligence engine. It has been developed by INRIA - the French National Institute for Research in Computer Science and Control - running on a Bull NovaScale supercomputer. During the 3-game match on the 22nd March 2008, MoGo managed to win the second game. On a 19×19 board it was weaker, losing with against Taranu, who had a handicap of 9 stones.

About ten years ago, the then strongest Go program, The Many Faces of Go, lost against a master level player although he had a 29 stone handicap.

To be honest, this is about the speed I expect research to have here, sooner or later processing power and better algorithms will make computers unbeatable in any game with logical rules, and Go is argued to be the most complex of all games. If you happen to know a complexity theorist, she will be happy to certify you that the complexity of Go is nasty EXPTIME-complete, which is not where you want to be (assuming you are a computer program and want to be fast when you’re grown up), really. That’s why it is nigh to impossible to ever “solve” Go, but it shouldn’t be too hard to win against us puny earthlings - even on a proper-sized board.

We have been trying to model our successors robot servants to resemble human beings physically and psychically for a while now. While the psychical development is taking its first steps out of its literal infancy, the physical resemblance has still a long way to go. It has to cross the uncanny valley:

The uncanny valley is a hypothesis that when robots and other facsimiles of humans look and act almost, but not entirely, like actual humans, it causes a response of revulsion among human observers. The “valley” in question is a dip in a proposed graph of the positivity of human reaction as a function of a robot’s lifelikeness.

This valley is the major hurdle towards believable and likeable humanoid model, be it robots, 3D animations or game avatars.

A common bail-out here is to depict characteristics generally considered humanoid by transmuting or exaggerating these features. Comics and the typical Pixar CGI movies are a very good example. The suspension of disbelief is the key factor here: We accept that comic-like depictions are there to interpret reality and show us a filtered, deliberately unrealistic image of the plot. It is very easy for our imagination to fill in the gaps and to ignore physical inaccuracies.

This gets much harder when we try to be as realistic as possible by using correct proportions, complex shading and detailed textures. The movie Beowulf almost perfectly falls into this category: It is completely CGI, but uses real actors and tries to simulate reality. While still-images still look alright, it is the animations where it goes downhill. The trailer has some examples in it, especially the horse animations look wrong, which is much more obvious in the actual movie (while technically not directly a humanoid animation, the galloping horses still add to the feeling that something is wrong with the depicted “reality”).

While not directly concerned with the uncanny valley, some military robots are quite creepy in my opinion. For example the Boston Dynamics “Big Dog”, which somehow looks like two soldiers bending over and pushing against each other. In a sense, it is the opposite to Beowulf: The movement, especially when it tries to regain its balance or slips on ice, looks quite natural, but the visual appearance (and sound) is really disturbing.

Scientists from the MIT are trying - but not quite managing - to avoid the valley with a robot using the comic-approach. They have developed the Nexi, a what they call MDS robot - mobile, dexterous, social. Both the pictures of this robot and the video of Nexi talking and gesticulating look a bit creepy, the robot reminds me of Jigsaw from the movie Saw.

They say that the “purpose of this platform is to support research and education goals in human-robot interaction, teaming, and social learning.”

The antipathy the uncanny valley introduces leads to a potential problem when regarding this social aspect of human-robot interaction: Once humans start to enhance themselves with transhuman enhancements, for example improved eye-sight, nanobots that increase the IQ or prostheses that work better than their natural counterparts, there may be a sudden rift between enhanced and “normal” humans, caused by the antipathy mentioned above. This hypothesis was introduced by Jamais Cascio, who further states that once such technologies gain further distance from human norms, “transhuman” individuals would cease to be judged on human levels and instead be regarded as separate entities altogether (this point is what has been dubbed “posthuman“), and it is here that acceptance would rise once again out of the uncanny valley.

I think the idea of the uncanny valley makes it harder for malicious AIs to undermine human civilisation, as fear of robots is more easily sparked with the help of uncanny-valley-antipathy (AI Panic -1%). However, the concept might be known to an intelligence, and thus we might see robots that does not look like human beings at all - or worse, like our childhood Disney-heroes!

London-based charity Landmine Action wants autonomous robots capable of killing people banned under the same kind of treaty that has outlawed landmines in over 150 countries. According to the New Scientist it is the first time a high profile non-governmental organisation has campaigned against such a technology. This campaign follows the reasoning of Noel Sharkey, who condemned these automation plans earlier this year.

As I’ve written before, the robots in use by the military nowadays (and the next years) are almost fully automatic, but so far the trigger has still to be pulled by a human soldier. However, it is only a question of time until the software is strong enough so that this decision will be made entirely by the machine. And once the software is in place, there will be no ethical opposition - at least in the US Department of Defence, who wants them in future to work without supervision.

A reaction to this news article comes from Wired, where the idea of danger through war robots is dismissed:

But to argue as if this is in the here or now, or even in the next decade, is just plain silly. The Pentagon has not only never advocated taking the man-out-the-loop of targeting decisions for drones or robots, its current policies and procedures would prohibit such a move (some might argue that international law already prohibits autonomous armed drones). [...] Unless and until those policies are drastically altered, it’s safe to say we are safe from renegade Terminators.

To completely ignore the threat by robots with weapons and justifying this ignorance by saying that these robots are still science fiction and it will take decades until these robots appear seems a bit strange to me, especially as Wired itself has reported about existing armed robots before. It is exactly this ignorance and belief that everything is so far away that allows organisations like the military to push these developments without any opposition.

Furthermore, I think it is a little blue-eyed to think that the control of weapons mounted on otherwise fully automated robots will remain in human hands just because the Pentagon does not admit of having plans that say otherwise. By arguing that international law might already prohibit autonomous armed drones and at the same time clearly seeing that these very drones are being used by the U.S. Army in Iraq right now, Wired maneuvers itself on very thin ice. I don’t think it can support the arguments that promises, laws and policies are sufficient enough to protect us from a - what Wired deems nonexistant - danger. And heck, what is so bad about protesting against future dangers as opposed to only trying to fight seeing the effects of existing weapons like landmines in hindsight?

I think it is about time that more voices are raised against automating war machinery, and the Landmine Action has taken a step in the right direction. If it will be heard by the military - and the U.S. Army is without doubt on the forefront of research towards this automation - is a completely different matter. So far, I think the chances are still slim, to specify I say it lowers the AI Panic Level by -0.1%.