AI Panic!

London-based charity Landmine Action wants autonomous robots capable of killing people banned under the same kind of treaty that has outlawed landmines in over 150 countries. According to the New Scientist it is the first time a high profile non-governmental organisation has campaigned against such a technology. This campaign follows the reasoning of Noel Sharkey, who condemned these automation plans earlier this year.

As I’ve written before, the robots in use by the military nowadays (and the next years) are almost fully automatic, but so far the trigger has still to be pulled by a human soldier. However, it is only a question of time until the software is strong enough so that this decision will be made entirely by the machine. And once the software is in place, there will be no ethical opposition - at least in the US Department of Defence, who wants them in future to work without supervision.

A reaction to this news article comes from Wired, where the idea of danger through war robots is dismissed:

But to argue as if this is in the here or now, or even in the next decade, is just plain silly. The Pentagon has not only never advocated taking the man-out-the-loop of targeting decisions for drones or robots, its current policies and procedures would prohibit such a move (some might argue that international law already prohibits autonomous armed drones). [...] Unless and until those policies are drastically altered, it’s safe to say we are safe from renegade Terminators.

To completely ignore the threat by robots with weapons and justifying this ignorance by saying that these robots are still science fiction and it will take decades until these robots appear seems a bit strange to me, especially as Wired itself has reported about existing armed robots before. It is exactly this ignorance and belief that everything is so far away that allows organisations like the military to push these developments without any opposition.

Furthermore, I think it is a little blue-eyed to think that the control of weapons mounted on otherwise fully automated robots will remain in human hands just because the Pentagon does not admit of having plans that say otherwise. By arguing that international law might already prohibit autonomous armed drones and at the same time clearly seeing that these very drones are being used by the U.S. Army in Iraq right now, Wired maneuvers itself on very thin ice. I don’t think it can support the arguments that promises, laws and policies are sufficient enough to protect us from a - what Wired deems nonexistant - danger. And heck, what is so bad about protesting against future dangers as opposed to only trying to fight seeing the effects of existing weapons like landmines in hindsight?

I think it is about time that more voices are raised against automating war machinery, and the Landmine Action has taken a step in the right direction. If it will be heard by the military - and the U.S. Army is without doubt on the forefront of research towards this automation - is a completely different matter. So far, I think the chances are still slim, to specify I say it lowers the AI Panic Level by -0.1%.

I’ve been home and relaxing a bit over Easter the last days, and didn’t get around to write much on AI Panic (partly because I wanted to have Wikipedia on my MP3-Player which took a while to program). Although most of the stuff that happened is already yesterdays news now, I feel I should mention it anyways, as it is relevant or interesting to AI. 

An elderly man has killed himself by programming a robot to shoot him in the head after building the machine from plans downloaded from the internet. Apparently, he used a jigsaw power tool that was connected to a .22 semi-automatic to create a “robot” that could fire multiple shots once triggered remotely (i guess by connecting the plug of the jigsaw tool to the power supply). This simple contraption (althoug remarkably complex for an 81 year old who’s tired of life) is nowhere near a modern, autonomous robot. It is just way too much hassle to put a sophisticated AI into a machine whose single purpose is killing its creator. Remember, we are talking about suicide machines here, so the “user” of that machine actually cooperates and works towards its goal. So I see no danger from the AI side in these robots. Military robots are more likely to turn into suicide robots — unintentionally.

Wired has a very well written and long article about Ray Kurzweil. It covers his early inventions that earned him his first little fortune while still an undergraduate, his reading machines for the blind, his pills to extend his life and of course his thoughts on the technological singularity. Nothing ground-breaking, but if you want to have glimpse into Ray Kurzweils life it is a good read.

There is an interesting paper on Utilitarian Essays about Friendly AI. The author analyses an approach called CEV (Coherent Extrapolated Volition) and his concerns with this approach. The essay does not deal at all with an AI turing evil, but only with the problems of extrapolating volition. There are three concerns the author rises: whose volitions to extrapolate, what about wild animals and lab universes and possible ramifications for religion. I’m not yet familiar enough with CEV to feel competent to criticise the raised points extensively (Michael Anissimov has some more interesting thoughts on the essay). Some quick questions that crossed my mind would be: How big is the role of primal urges and drives in general volition? Will we be able to do anything at all if we decide to always decrease suffering in every creature that can suffer? How can lab universes physically exist at all? What if we don’t like the extrapolated volition (”Trust me, it’s all for your best!”)?

Pioneering science fiction writer, visionary and futurist Arthur C. Clarke, best known for his work on the movie 2001: A Space Odyssey (where he created the prime example of an AI turned bad, HAL 9000), has died in his adopted home of Sri Lanka at the age of 90 on the 19th of March. He was also known for “inventing” the idea of geostationary satellites and was the last living of the “big three” science fiction writers. The two others were the Russian-born Isaac Asimov, who died in 1992, and Robert A. Heinlein, a Missouri native who died in 1988. Although he didn’t believe in an afterlife (or religion), he does live on through his brilliant works and ideas.

Also, I’m happy that my blog has been mentioned by Michael Anissimov during a Radio Interview and my post on COSPAL has been picked up by io9, which gave me a new visitor record of whopping 120 visitors on one day. I know I know, but you have to cherish the small things in life … and I don’t think thats too bad for a blog only 2 months old. AI Panic was also the Site Of The Week on scifi.com a while back, which is cool! I am quite surprised how quick I got addicted to media attention once the blog was up, sometimes I catch myself just watching the visitor counter go up and checking where all the visitors come from… Thanks for reading, dear visitor

With the military building scary armed robots already, it was about time for robots that are scared of these monsters. Phobot, a Lego Mindstorms Robot built by students from the University of Amsterdam, is exactly that: A robot that is scared of bigger robots.

Now here comes the pedagogical value: It can “learn” to lose its fear by conditioning it with increasingly big and evil looking things. The process, shown in the video below, is meant to help children with phobias to learn that not everything is as scary and bad as it seems. Especially not the autonomous sentry gun protecting the kindergarten. Enjoy!

(Via Spiegel Online)

The march towards learning, fully functional AI systems has taken another step. A cognitive AI system with the intelligence level of a puppy has been created to learn and perform simple tasks through a robotic arm.

Sure, a puppy is (often) quite stupid, but that achievement should not be underestimated, especially as the researchers at the EU-funded COSPAL project claim that this level of cognition and learning goes beyond the current state of the art.  They combine artificial neural networks (ANNs) to control low-level motor actions and a classical rule-based system to combine these actions to form a plan.

Although this advancement of AI potentially is a step towards a fully aware system, i.e., an AI system that is able to reason and learn from all its surrounding world information (for example through a common sense knowledge database), it is not an unexpected development nor something close to self-awareness, as Michael Felsberg from the research team at Linköping University explains:

In human terms, our robot is probably like a two or three year old child, and it will take a long time for the technology to progress into the equivalent of adulthood. I don’t think we will see it in our lifetimes.

His pessimism shows that Artificial General Intelligence is still quite far out, and while I think he might be overestimating the time-span for it (alas, the Singularity is scheduled earlier than after our lifetimes), he’s still one of the researchers on the forefront of AI research, and thus I think it is safe to say that we will still have to wait quite a while before smart enough AIs appear that could be dangerous (AI Panic -2%).

(Via PhysOrg.com)

It sounds like the sermon Noel Sharkey preached last month about the ethics of autonomous systems has been really necessary. The time-frame we are talking about here is smaller than one might think - I suspect just 10-15 more years and we will see autonomously shooting robots. If there is a war at that time and morality debates fail, that is.

A reporter from Popular Mechanics visited an U.S. Army test area in Texas recently and wrote about the demonstration of the “MULE”, a six-wheeled utility vehicle, which is currently remote controlled by an Xbox 360 controller, but the developers at Lockheed Martin plan to include autonomous movement by the scheduled release date 2014. Already 1700 of these Humvee sized beasts have been ordered by the U.S. Army, and while they say that the use of the mounted weapons (4 antitank missiles and a machine gun) in this generation of robots will always have to be initiated by a human controller, I suspect that this will be “optional” in future versions. Maybe in the form of a bAutoFire = false; that can easily be overridden in desperate situations after shouting a key phrase like It’s coming right for us!

The article questions the readiness of the robots for battle, and comes to the conclusion that there’s still a long way to go and the robots are still clumsy and slow. However, the hardware is completely ready for battle (and probably has been for a while). It is just the software that is still missing. Just some little intelligent pathfinding, friend/foe distinction, an aimbot and a wallhack and we’re all set!

Well, of course it is not that easy. Indeed, the artificial intelligence is by far the most difficult problem these robots face, and as mentioned, it’ll take another couple of years until we see reliable algorithms that prevent friendly fire (My guess would be that they come up with a hardware based solution first, an active transmitter or reflector that indicates a friend. However, that could become problematic when soldiers are captured.)

Eventually, the software side will be solved, and only the ethical issue will remain. War ethics are a difficult terrain and probably won’t count for a lot when an officer sees his soldiers at risk and decides to push the button. The huge budget of the U.S. Army lures researchers and universities like Carnegie Mellon to put their brainpower to use on modern AI, AI that will be used to control weapons to kill people.

Now I’ve rambled along way too much again, time to finish up. I’ll increase the AI Panic Level by +1% (not more because I’ve already credited the general risk of autonomous army robots in the post about Noel Sharkey).

 Addition (March 22nd, 2008): There are also flying prototypes, like that bat spy plane shown below. The U.S. Army awarded the University of Michigan College of Engineering a five-year, $10-million grant to construct and build it. While it sure is more difficult to build a flying object than an earthbound robot, the challenges lie more in the physical realization than in a better AI.

Unmanned flying drones are in use by the military already and have been used in Iraq to fire missiles and collect terrain information.

Thanks Igor Gabrielan and Erk Subasi for letting me know!
If you see something in the net that would make a good AI Panic article, feel free to contact me!

A new molecule structure is able to control 8 nano-machines simultaneously by switching the states of the involved molecules. This chemical “brain” has been developed at the International Center for Young Scientists, Tsukuba, Japan, reports Engadget and BBC.

The scientists use 17 duroquinone molecules that could (one day) act as a central control for other nanobots and are themselves controlled via a scanning tunneling microscope. They have already built bigger machines capable of 256 simultaneous operations, but using these machines in a computer - while faster than a conventional pc - is very difficult at the moment because of the required scanning tunnel microscope.

Nanorobotics is, alongside artificial general intelligence, one of the most likely technologies to radically change the way human beings live and develop in the next decades, according to most futurologists. While many applications are very promising, especially when concerned with body-enhancing functions (longevity, intelligence boosters), some people believe that these robots may run out of control and, for example, replicate uncontrollably and devour every living organism on earth (grey goo scenario). I am still undecided which variant I find more likely, I haven’t read enough material on nanorobotics yet. At first sight, the grey goo scenario seems less likely to happen than the evil AI scenario, because replication appears to be a very complicated and sensitive process to me.

However, assuming nanobots become reality, they will affect the creation of strong AI: If they are used to improve the IQ of researchers, say by suppressing the need for sleep or increasing the brain capacity, the AI development will accelerate. On the other hand, if everything is turned into goo, obviously there will be no AI that enslaves human beings (other than swarm AI maybe). The impact on the probability of an evil AI thus depends on how likely the different nanorobotics scenarios are.

What if the people you talk to, play with and interact with in Second Life weren’t human at all? Granted, none of them is literally human anyway, but almost all of them are actually controlled by someone sitting in front of a computer. Not all are.

You might run into a little, 4 year old child that appears to be reasonably smart and is able to reason about its surroundings. This child is controlled by a program. A program that is able to reason about its own beliefs to draw conclusions in a manner that matches human children its age.

The program is the result of research conducted by a group around Selmer Bringsjord at the Rensselaer Polytechnic Institute. According to ScienceDaily, Bringsjord says:

Truly convincing autonomous synthetic characters must possess memories; believe things, want things, remember things. Such characters can only be engineered by coupling logic-based artificial intelligence and computational cognitive modeling techniques with the processing power of a supercomputer. [...] The logico-mathematical theory will include rigorous, declarative definitions of all of the concepts central to a theory of the mind, including lying, betrayal, and even evil.

Using Second Life as a platform for this makes sense. Reducing the communication bandwidth is a common trick to lower the barrier of believability. Even the good ol’ Alan Turing limited the interaction for his famous Turing test to test if a computer can demonstrate intelligence to a text chat. Second Life is reasonably close to that, avatars can easily be controled by keyboard commands.

Building agents based on the theory of mind (which describes the ability to attribute mental states to oneself and others and to understand that others have beliefs, desires and intentions that are different from one’s own) is a step towards an artificial general intelligence, and thus towards a real threat (or a great opportunity, depends on who you ask). And indeed, this project was presented a couple of days ago at the Artificial General Intelligence conference held at the University of Memphis.

I believe there is a high risk connected to the creation of such an intelligence, it is easy to think of hypothetical scenarios in which a reasoning system concludes that humans are a risk and need to be wiped out. Although testing such an agent in a virtual world is a good idea to realise what it is capable of and evaluating the actual risk of it, once the tools are there the step into the real life will be taken quickly. And sure enough, Selmer Bringsjord has plans to apply the technlogy to education and homeland defense. Good thing the system has no sense of lying, betrayal and isn’t evil…

The researchers want to build an interface with a certain resemblance to the holodecks in Star Trek for direct interaction of agents with human beings. While this definitely sounds like science fiction, it has to be taken seriously. They have the support of strong industrial partners and want to use the worlds most powerful university-based supercomputing system at the CCNI.

The research is still in an early stage and the Second Life results are far away from being strong AI, but they might well bring us closer to an artificial general intelligence, thus raising the AI Panic Level by +1%. I don’t know whether the system they are working on will achieve a level of strong AI (most likely not), and even if it does, we don’t know if it will be good or bad. I’ll finish with a quote from the group’s research homepage, which interestingly enough seems to answer that thought:

An advanced synthetic character, however, can literally be evil, because it has the requisite desires, beliefs, and cognitive powers.

Our death is our last service for the world: If we didn’t clear the path, following generations would not have to recreate human culture. It would become stiff, changeless, thus dead. And with the death of culture everything human would perish, too.

– Joseph Weizenbaum in one of his last e-mails (translated from German)

Joseph Weizenbaum died yesterday at the age of 85 in Berlin from a stroke.

Born in Berlin, Germany to Jewish parents, he escaped Nazi Germany in 1936, emigrating with his family to the United States. He started studying mathematics in 1941 in the US, but his studies were interrupted by the war, during which he served in the military. Around 1950 he worked on analog computers, and helped create a digital computer for Wayne State University. In 1955 he worked for General Electric on the first computer used for banking, and in 1963 took a position at MIT.

In 1966, he published a comparatively simple program called ELIZA which demonstrated natural language processing by engaging humans into a conversation resembling that with an empathic psychologist. The program applied pattern matching rules to the human’s statements to figure out its replies. (Programs like this are now called chatterbots.) Weizenbaum was shocked that his program was taken seriously by many users, who would open their hearts to it. He started to think philosophically about the implications of Artificial Intelligence and later became one of its leading critics. His influential 1976 book Computer Power and Human Reason displays his ambivalence towards computer technology and lays out his case: while Artificial Intelligence may be possible, we should never allow computers to make important decisions because computers will always lack human qualities such as compassion and wisdom. This he saw as a consequence of their not having been raised in the emotional environment of a human family.

I feel honoured to have met Joseph Weizenbaum in person about three years ago, when he gave a lecture at the Westfälische Wilhelms-Universität Münster, Germany. He was a very charismatic and gifted speaker with important messages that not enough people take to heart, even I did not, at that time.

Farewell, Joseph Weizenbaum.

(Paragraphs about Weizenbaums life taken from Wikipedia, quotation found at heise.de)

If you think that artificial intelligences that actively attack and try to kill its human counterparts are just a hypothetical idea that might never come to reality, you might be mistaken.

The gaming industry has been training gruesome robots, mighty automated armies and deadly opponents for years now. So one might wonder, how dangerous are these algorithms really?

I was inspired to write a bit about this topic by a comment from Horus Aha on last weeks article about Ray Kurzweil’s talk on the Future in Gaming at the Games Developer Conference:

Lots of investment in making game AI’s that are basically sociopathic killers explicitly designed to effectively compete against humans. This applies in 1st person shooters, but also to other genres. The games are networked and it is possible that the AI’s in these games may learn how to game-the-game by finding ways to compete with players outside of game environments. Why not? I don’t think anybody would have taken warnings about “gold-farming” seriously until it was already a reality.

Now, is this fear justified? Are we really endangered by computer game AI?

No.

AI in the entertainment industry is incredibly dumb. The are a couple of reasons for that. Probably the most important is that the game developers do not want an intelligence that can out-reason humans and win all the time: The goal is making an AI that plays to lose, not to win. It is a well-known psychological fact that an opponent that almost beats but eventually loses against the player is most enjoyable for him (There was a good presentation about that from Soren Johnson at this year’s GDC called Playing to Lose: AI and “CIVILIZATION”).

But don’t we need a proper AI to be able to figure out how to achieve this? When you look at the concepts of AI in games and talk to game developers it quickly becomes clear that they don’t even try to make a learning or guessing AI: They cheat! Faking intelligence is a lot easier than actually achieving proper reasoning. In almost all games with incomplete information, i.e., a fog of war or hidden objects, the AI is given full information and only pretends to not know where you are and what you do. And if you see a really smart manoeuvre from the enemy troops (or your virtual team-mates) it is hard-coded or pre-scripted very often. I attended the AI roundtable talks at the GDC in San Francisco two weeks ago and almost none of the attending developers (who worked on Bioshock, Halo3, The Sims, Command & Conquer etc.) has used any of the “proper” academic AI algorithms like neural networks or genetic algorithms.

There are some exceptions, for example Black & White uses a simple reinforcement learning algorithm to make the creature learn from your actions. F.E.A.R. is an often cited example of real planning in a game, however, the used algorithm is really simple and some argue that the resulting behaviour could have been achieved with pre-written scripts just as well.

So there’s no real, potentially dangerous, intelligence in games at all. This is why the AI Panic Level drops by -5%, the “sociopathic killers” will not be able to escape the virtual world of computer games for a long time. If at all, they will become smart only by developments outside the games industry, for example academic research.

This post has nothing directly to do with Artificial Intelligence, but astronomy is one of my other big interests and I thought I’d share the interesting topic of the most powerful explosions in the universe with you.

When someone asks what the biggest, most energetic events in the universe are, Gamma Ray Bursts are a good bet. These incredibly powerful explosions happen when the core of a very massive, fast rotating star collapses to form a black hole and surrounding matter of the outer hulls of the dying star fall towards the black hole. Due to the fast rotation and the acceleration towards the hole, the matter heats up and powers a powerful jet of gamma radiation that blasts outward through the “poles” of the rotating system (see illustration below). At least, that is what the most popular theory of GRBs says, the whole thing is still poorly understood.

The power these bursts have is incredible: If one would happen 6000 light years away from earth, it might cause a mass extinction (according to the History Channel, so take it with a grain of salt!). Also, just some seconds within the beam might cause a lot (some say 30-50%) of earth’s ozone layer to disappear. The whole ozone layer damage we saw in the last decades had only about 5% of the layer disappearing! So, we’d rather not have a GRB happen near us.

The likelihood of such a burst striking earth very slim. GRBs are very rare events, and if a burst happens, the jets are very concentrated and thin. However, the odds are not zero.

Now, there is a binary star system called WR 104 about 8000 light years away from us near the center of the milkyway. After all the introduction about gamma ray bursts and lethal explosions and stuff, it is no coincidence I bring this system up:

The brighter of the two stars might, just maybe kinda possibly, be ready to go GRB on us. It’s not at all clear if it can, and there is reason to believe it can’t (young stars like this one tend to have characteristics that make it very hard for them to form an actual GRB). Also, even if it does blow up that way, the beams are a double-edged sword; yes, they pack an unbelievable punch, but they’re narrow. A GRB would have to be aimed precisely at us to damage us, and the odds of that are pretty low.

Except that for WR 104, it’s possible the star does have us in its sights.

That’s an excerpt from an article Phil Plait wrote about WR 104 in his excellent blog BadAstronomy.com. Phil goes on to say that there are a lot of other variables like the distance, matter between us and the star, the time it will explode and so on which decrease the probability of the star being a direct danger to us substantially. So most likely, nothing is going to happen, and if something happens, it probably won’t affect us at all.

Alas, a Gamma Ray Burst that hits us would most likely mean the end of most life on the planet, and with it the development of an universal AI (thus justifying this astronomy article here, phew!). Therefore such an catastrophic astronomical event (and that includes hits by asteroids, attacks by extraterrestrial lifeforms and so on) decreases the AI Panic Level by -0.1%. Actually I think the danger of these events is still lower, but I don’t want to go into hundredths of percents, and therefore rounded it up.

I hope you bore with me here and don’t find astronomy too boring, I certainly don’t!