AI Panic!

The University of Pennsylvania has built a new self-reconfigurable robot, which can, as the name suggests, assemble itself after it has been kicked apart. Of course it is not much more than three robots with a smart communication, orientation and interfacing system, but as the atomic units get smaller these compound robots might actually get useful for, say, building a liquid metal Terminator.

There is also a high-resolution version available at the youtube page of the video.

(Via NewScientist)

About 9 years ago, on June 10th 1999, the first humans fell victim to a computer control system failure. Due to a chain of unfortunate circumstances, a the pressure on a 16-inch steel pipe line increased uncontrollably and caused it to rupture, flooding two small creeks near Bellingham in the process. The petrol ignited and killed two 10-year-old boys and an 18-year-old adolescent. Although it was later revealed that the accident was in part caused by human failure, it is still considered as the first cyber-event that killed humans.

Now, you might think that things have changed since then, but you’d be wrong. Joe Weiss from Applied Control Solutions says:

Until eight years ago, my whole life was making control systems usable and efficient, and, by the way, very vulnerable. It is exactly what you will find today in many, many industrial applications. This isn’t just 1999. No, this is June 2008.

And it is true. The computer systems that control complex technical facilities, like the power grid, traffic control infrastructure, or pipeline systems have historically only been optimized with respect to usability, efficiency and effectivity. Security has almost never been a concern in the design of these systems. Many of them have been developed before the rise of the internet, so that attacks had to be carried out physically. But eventually some systems responsible for the control of important infrastructure have been connected to a network, often in an effort to improve control or just because of company policies, without any evaluation of possible risks.

There are more examples. The 787 Dreamliner from Boeing, which is scheduled to come out later this year, may be vulnerable to hacker attacks. This is because the flight control network is separated from the passenger network only by software firewalls, and thus might be attacked from inside the plane (or even from the outside, with the help of a trojan horse maybe).

When military control systems fail, it gets nasty quite quickly. Last year, a robot anti-aircraft cannon killed 9 and wounded 14 soldiers when it went berserk during a training of the South African National Defence Force. However, it is not yet clear if the accident was caused by a mechanical malfunction or software failure.

And it is not only the security of control systems that poses a risk, the control systems themselves are feared by researchers to remove human control from key decision-making processes. Tom Rodden, Professor of Interactive Systems at the University of Nottingham concluded at the Human-Computer Interaction conference that this loss of control might not directly be a risk for our lives, but surrenders “basic human values and concepts such as personal space, society, identity, independence, perception, intelligence and privacy.”

This lack of security must become a higher priority to control system designers, especially as there is an unavoidable trend towards more autonomous, intelligent systems that can cope with the size and complexity of modern infrastructure. We have to take care that we do not hand over all control of critical systems and uncritical but socially important concepts to machines. Even if a worst case scenario - such as malicious AI breaking loose - does not happen, programs are likely to be susceptible to bugs and misinterpreted signals, and therefore have to be very carefully designed. As the examples in the past have shown, these issues are a real threat to human beings and the lack of security against tampering from the outside increases the AI Panic level by +1%.

Imagine, someday, we’ll lose the control over our computers to some sentient being, unable to stop it or to unplug the millions of home pcs, office workstations or lab mainframes affected.

Well, there is no need to imagine, this very thing has been happening for a long time now, in the form of Zombie Computers. These machines have been compromised by hackers, viruses or trojans, and are remote controllable. Often they are grouped in a network of zombie computers, called a botnet.

The RSA 2008 conference in San Francisco earlier this month had a panel discussion on this topic, and the tone was “a mix of resignation, indignation and post-9/11 rhetoric”, according to Threat Level. According to the panelists, botnets are the biggest threat in the internet today and a danger to national security.

However, little is done to combat the use of zombie computers. Although some botnet operators are caught once in a while, the spam and attacks they create does not seem to decrease. I agree with Ira Winkler, a security consultant who said at the conference:

“The problem is no one is doing anything. Guess what? If your system has a bot on it, you don’t get on the internet. [...] We need to hold people responsible when they present an imminent threat to other people.”

However, ISPs are reluctant to cut off offenders, as they often have no clue what’s going on and respond with confusion and angry phone calls, which drive up the customer support costs.

The inability to quickly and efficiently stop these bot nets could prove very helpful for a malevolent AI. Once released on the internet (or on its successor, maybe The Grid), there is virtually no way to stop it again. The infrastructure to spread uncontrollably is already given by the botnets. Unless there is some fundamental change on the network level and software level (hello Microsoft!), the only way to stop such an outbreak would be the complete shut-down of the net. Of course, this would cause quite a disturbance to the world economy, as most financial transactions and international trading are entirely done online. Most nations will be hesitant to risk that, just because some strange program has been set free from a university lab somewhere.

Therefore I think the current infrastructure of the web eases the spread of an AI (Panic Level: +3%), once it discovers the weaknesses of connected hosts and “understands” how botnets work. But that’s not yet an immediate threat; after all we have to develop the artificial general intelligence first, which, mind you, is a little tricky.

The game of Go has been one of the last big board games where artificial intelligence strength was nowhere near experienced players, let alone master grade professionals. During a recent Go tournament in Paris however, a professional 5th Dan grade player, Catalin Taranu, was beaten by a computer program (diagram and picture of the match below).

The catch is, it was played only on a 9×9 field, which is smaller than the usual 19×19 field. This no doubt had part in the victory, as the overwhelming state-space is reduced from 10¹⁷⁰ to 10³⁸ possible game positions (chess: about 10⁵²).

According to Earth News, that was the first ever recorded victory of a machine over a Go master (in a sanctioned non-blitz game). The culprit is the MoGo artificial intelligence engine. It has been developed by INRIA - the French National Institute for Research in Computer Science and Control - running on a Bull NovaScale supercomputer. During the 3-game match on the 22nd March 2008, MoGo managed to win the second game. On a 19×19 board it was weaker, losing with against Taranu, who had a handicap of 9 stones.

About ten years ago, the then strongest Go program, The Many Faces of Go, lost against a master level player although he had a 29 stone handicap.

To be honest, this is about the speed I expect research to have here, sooner or later processing power and better algorithms will make computers unbeatable in any game with logical rules, and Go is argued to be the most complex of all games. If you happen to know a complexity theorist, she will be happy to certify you that the complexity of Go is nasty EXPTIME-complete, which is not where you want to be (assuming you are a computer program and want to be fast when you’re grown up), really. That’s why it is nigh to impossible to ever “solve” Go, but it shouldn’t be too hard to win against us puny earthlings - even on a proper-sized board.

We have been trying to model our successors robot servants to resemble human beings physically and psychically for a while now. While the psychical development is taking its first steps out of its literal infancy, the physical resemblance has still a long way to go. It has to cross the uncanny valley:

The uncanny valley is a hypothesis that when robots and other facsimiles of humans look and act almost, but not entirely, like actual humans, it causes a response of revulsion among human observers. The “valley” in question is a dip in a proposed graph of the positivity of human reaction as a function of a robot’s lifelikeness.

This valley is the major hurdle towards believable and likeable humanoid model, be it robots, 3D animations or game avatars.

A common bail-out here is to depict characteristics generally considered humanoid by transmuting or exaggerating these features. Comics and the typical Pixar CGI movies are a very good example. The suspension of disbelief is the key factor here: We accept that comic-like depictions are there to interpret reality and show us a filtered, deliberately unrealistic image of the plot. It is very easy for our imagination to fill in the gaps and to ignore physical inaccuracies.

This gets much harder when we try to be as realistic as possible by using correct proportions, complex shading and detailed textures. The movie Beowulf almost perfectly falls into this category: It is completely CGI, but uses real actors and tries to simulate reality. While still-images still look alright, it is the animations where it goes downhill. The trailer has some examples in it, especially the horse animations look wrong, which is much more obvious in the actual movie (while technically not directly a humanoid animation, the galloping horses still add to the feeling that something is wrong with the depicted “reality”).

While not directly concerned with the uncanny valley, some military robots are quite creepy in my opinion. For example the Boston Dynamics “Big Dog”, which somehow looks like two soldiers bending over and pushing against each other. In a sense, it is the opposite to Beowulf: The movement, especially when it tries to regain its balance or slips on ice, looks quite natural, but the visual appearance (and sound) is really disturbing.

Scientists from the MIT are trying - but not quite managing - to avoid the valley with a robot using the comic-approach. They have developed the Nexi, a what they call MDS robot - mobile, dexterous, social. Both the pictures of this robot and the video of Nexi talking and gesticulating look a bit creepy, the robot reminds me of Jigsaw from the movie Saw.

They say that the “purpose of this platform is to support research and education goals in human-robot interaction, teaming, and social learning.”

The antipathy the uncanny valley introduces leads to a potential problem when regarding this social aspect of human-robot interaction: Once humans start to enhance themselves with transhuman enhancements, for example improved eye-sight, nanobots that increase the IQ or prostheses that work better than their natural counterparts, there may be a sudden rift between enhanced and “normal” humans, caused by the antipathy mentioned above. This hypothesis was introduced by Jamais Cascio, who further states that once such technologies gain further distance from human norms, “transhuman” individuals would cease to be judged on human levels and instead be regarded as separate entities altogether (this point is what has been dubbed “posthuman“), and it is here that acceptance would rise once again out of the uncanny valley.

I think the idea of the uncanny valley makes it harder for malicious AIs to undermine human civilisation, as fear of robots is more easily sparked with the help of uncanny-valley-antipathy (AI Panic -1%). However, the concept might be known to an intelligence, and thus we might see robots that does not look like human beings at all - or worse, like our childhood Disney-heroes!