AI Panic!

Our death is our last service for the world: If we didn’t clear the path, following generations would not have to recreate human culture. It would become stiff, changeless, thus dead. And with the death of culture everything human would perish, too.

– Joseph Weizenbaum in one of his last e-mails (translated from German)

Joseph Weizenbaum died yesterday at the age of 85 in Berlin from a stroke.

Born in Berlin, Germany to Jewish parents, he escaped Nazi Germany in 1936, emigrating with his family to the United States. He started studying mathematics in 1941 in the US, but his studies were interrupted by the war, during which he served in the military. Around 1950 he worked on analog computers, and helped create a digital computer for Wayne State University. In 1955 he worked for General Electric on the first computer used for banking, and in 1963 took a position at MIT.

In 1966, he published a comparatively simple program called ELIZA which demonstrated natural language processing by engaging humans into a conversation resembling that with an empathic psychologist. The program applied pattern matching rules to the human’s statements to figure out its replies. (Programs like this are now called chatterbots.) Weizenbaum was shocked that his program was taken seriously by many users, who would open their hearts to it. He started to think philosophically about the implications of Artificial Intelligence and later became one of its leading critics. His influential 1976 book Computer Power and Human Reason displays his ambivalence towards computer technology and lays out his case: while Artificial Intelligence may be possible, we should never allow computers to make important decisions because computers will always lack human qualities such as compassion and wisdom. This he saw as a consequence of their not having been raised in the emotional environment of a human family.

I feel honoured to have met Joseph Weizenbaum in person about three years ago, when he gave a lecture at the Westfälische Wilhelms-Universität Münster, Germany. He was a very charismatic and gifted speaker with important messages that not enough people take to heart, even I did not, at that time.

Farewell, Joseph Weizenbaum.

(Paragraphs about Weizenbaums life taken from Wikipedia, quotation found at heise.de)

If you think that artificial intelligences that actively attack and try to kill its human counterparts are just a hypothetical idea that might never come to reality, you might be mistaken.

The gaming industry has been training gruesome robots, mighty automated armies and deadly opponents for years now. So one might wonder, how dangerous are these algorithms really?

I was inspired to write a bit about this topic by a comment from Horus Aha on last weeks article about Ray Kurzweil’s talk on the Future in Gaming at the Games Developer Conference:

Lots of investment in making game AI’s that are basically sociopathic killers explicitly designed to effectively compete against humans. This applies in 1st person shooters, but also to other genres. The games are networked and it is possible that the AI’s in these games may learn how to game-the-game by finding ways to compete with players outside of game environments. Why not? I don’t think anybody would have taken warnings about “gold-farming” seriously until it was already a reality.

Now, is this fear justified? Are we really endangered by computer game AI?

No.

AI in the entertainment industry is incredibly dumb. The are a couple of reasons for that. Probably the most important is that the game developers do not want an intelligence that can out-reason humans and win all the time: The goal is making an AI that plays to lose, not to win. It is a well-known psychological fact that an opponent that almost beats but eventually loses against the player is most enjoyable for him (There was a good presentation about that from Soren Johnson at this year’s GDC called Playing to Lose: AI and “CIVILIZATION”).

But don’t we need a proper AI to be able to figure out how to achieve this? When you look at the concepts of AI in games and talk to game developers it quickly becomes clear that they don’t even try to make a learning or guessing AI: They cheat! Faking intelligence is a lot easier than actually achieving proper reasoning. In almost all games with incomplete information, i.e., a fog of war or hidden objects, the AI is given full information and only pretends to not know where you are and what you do. And if you see a really smart manoeuvre from the enemy troops (or your virtual team-mates) it is hard-coded or pre-scripted very often. I attended the AI roundtable talks at the GDC in San Francisco two weeks ago and almost none of the attending developers (who worked on Bioshock, Halo3, The Sims, Command & Conquer etc.) has used any of the “proper” academic AI algorithms like neural networks or genetic algorithms.

There are some exceptions, for example Black & White uses a simple reinforcement learning algorithm to make the creature learn from your actions. F.E.A.R. is an often cited example of real planning in a game, however, the used algorithm is really simple and some argue that the resulting behaviour could have been achieved with pre-written scripts just as well.

So there’s no real, potentially dangerous, intelligence in games at all. This is why the AI Panic Level drops by -5%, the “sociopathic killers” will not be able to escape the virtual world of computer games for a long time. If at all, they will become smart only by developments outside the games industry, for example academic research.

This post has nothing directly to do with Artificial Intelligence, but astronomy is one of my other big interests and I thought I’d share the interesting topic of the most powerful explosions in the universe with you.

When someone asks what the biggest, most energetic events in the universe are, Gamma Ray Bursts are a good bet. These incredibly powerful explosions happen when the core of a very massive, fast rotating star collapses to form a black hole and surrounding matter of the outer hulls of the dying star fall towards the black hole. Due to the fast rotation and the acceleration towards the hole, the matter heats up and powers a powerful jet of gamma radiation that blasts outward through the “poles” of the rotating system (see illustration below). At least, that is what the most popular theory of GRBs says, the whole thing is still poorly understood.

The power these bursts have is incredible: If one would happen 6000 light years away from earth, it might cause a mass extinction (according to the History Channel, so take it with a grain of salt!). Also, just some seconds within the beam might cause a lot (some say 30-50%) of earth’s ozone layer to disappear. The whole ozone layer damage we saw in the last decades had only about 5% of the layer disappearing! So, we’d rather not have a GRB happen near us.

The likelihood of such a burst striking earth very slim. GRBs are very rare events, and if a burst happens, the jets are very concentrated and thin. However, the odds are not zero.

Now, there is a binary star system called WR 104 about 8000 light years away from us near the center of the milkyway. After all the introduction about gamma ray bursts and lethal explosions and stuff, it is no coincidence I bring this system up:

The brighter of the two stars might, just maybe kinda possibly, be ready to go GRB on us. It’s not at all clear if it can, and there is reason to believe it can’t (young stars like this one tend to have characteristics that make it very hard for them to form an actual GRB). Also, even if it does blow up that way, the beams are a double-edged sword; yes, they pack an unbelievable punch, but they’re narrow. A GRB would have to be aimed precisely at us to damage us, and the odds of that are pretty low.

Except that for WR 104, it’s possible the star does have us in its sights.

That’s an excerpt from an article Phil Plait wrote about WR 104 in his excellent blog BadAstronomy.com. Phil goes on to say that there are a lot of other variables like the distance, matter between us and the star, the time it will explode and so on which decrease the probability of the star being a direct danger to us substantially. So most likely, nothing is going to happen, and if something happens, it probably won’t affect us at all.

Alas, a Gamma Ray Burst that hits us would most likely mean the end of most life on the planet, and with it the development of an universal AI (thus justifying this astronomy article here, phew!). Therefore such an catastrophic astronomical event (and that includes hits by asteroids, attacks by extraterrestrial lifeforms and so on) decreases the AI Panic Level by -0.1%. Actually I think the danger of these events is still lower, but I don’t want to go into hundredths of percents, and therefore rounded it up.

I hope you bore with me here and don’t find astronomy too boring, I certainly don’t!

Remember those little spider-things from Minority Report? Well, version 0.1 of these critters has just arrived:

Interestingly enough, the phone is actually real. Engadget reports that it is the Toshiba 815T PB on Softbank and is to be released at the launch of a strange robot-phones TV show called Ketai Sousakan 7.

Luckily, the appendixes don’t look as if they were powered, so it is just a gimmick for the über-cool japanese high-school kids. For now. And the phones seem to be angry already …

Noel Sharkey, a professor at the University of Sheffield, warned conference attendees today about the danger of an arms race in robotic warfare.

He gave a keynote speech at the “Ethics of Autonomous Military Systems” event hold at the Royal United Services Institute (RUSI) for Defence and Security Studies, warning the audience about terroristic aspects of war robots. He argues that terrorists will use these robots for their purposes:

“Once the new weapons are out there, they will be fairly easy to copy. How long is it go­ing to be be­fore the ter­ror­ists get in on the ac­t? [...] With the cur­rent prices of ro­bot con­struc­tion fall­ing dra­mat­ic­ally and the avail­abil­ity of read­y-made com­po­nents for the am­a­teur mar­ket, it would­n’t re­quire a lot of skill to make au­ton­o­mous ro­bot weapons.” Sharkey said a small GPS-guided drone with au­to­pi­lot can be made for about $500.

Interestingly, he accounts the dumbness of these robots for part of the potential danger, because the robots might not be able to distinguish between friend or foe.

Military technology expert James Canton compared the development of robot war technologies with Moore’s Law, but “on steroids.” It is only a matter of time before the human factor is completely removed from the control of war robots. For now, the trigger has still to be pulled manually, as the control logic is not yet good enough and ethical questions still have to be answered.

However, discussing the morality and ethicality of warfare is hypocritical anyway, and has rarely stopped the use and development of military equipment (with the notable exception of biological and chemical weapons). Now as semi-automatic robots are used in real-”life” already, I doubt that a noticeable opposition will be formed, especially as the effects and danger of these weapons are not directly visible — on the contrary, the accuracy will improve and friendly fire is likely to reduce. I think only a mayor accident will be enough to change the perception of these robots, but then it is already too late. As soon as the technology is out there, it is available to be used by terrorist organisations.

While terrorist attacks are not directly related to AI, the development of autonomous robots is. Many research institutes and universities are striving towards automated image recognition, which is the major hurdle for the development of fully autonomous war robots, as the Geneva Convention requires the reliable distinction between combatants and civilians. Warnings from Experts like Noel Sharkey indicate the reality of the danger of autonomous war robots, which is why the AI Panic Level will be increased by +5%.

[Sources: AFP, World Science, The Register]

Ray Kurzweil gave a keynote speech at this years GDC in which he adressed the future of gaming (a bit) and the future of mankind (a lot). Most of the readers of this blog are probably familiar with Ray Kurzweil and his theories. In short, he proposed the law of accelerating returns which will lead to a technological singularity in about 20-30 years. He extends Moore’s Law to the future and uses it (among other indicators) to predict future values for chip densities, prices, computing power, etc. The exponential acceleration of these values will lead to massive and fundamental changes in the way we live and work, culminating in self-replicating and -improving machines, which triggers an explosion in technological advancement, as the human as innovating factor is eliminated.

As far as I can tell, the keynote speech last Thursday did not contain any results that have not been discussed in one of his previous talks about that matter, Kurzweil just added some game related stuff at the beginning, saying how “game” was an unfortunate name, because it stressed the disconnection from reality, but games are increasingly interwoven with reality (just like talking to someone over the telephone is not virtual). More info on the keynote can be found at 1up and the Escapist.

The law of accelerating returns and the technological singularity pretty much state that a strong AI will be created in the very near future. The singularity, if it so happens, will most likely produce an intelligence superiour to our own. While Ray Kurzweil sees a positive future here, talking about eternal life and friendly AI, this vision seems unlikely to me. Especially considering the inherent malevolence of generic AI algorithms, the probability of a hostile AI is high, I rise the AI Panic Level by +10%. This rise is not an effect of the keynote speech from Ray Kurzweil, but more a result of the discussion of the singularity and accelerating return theories.

The theories are compelling. The statistical foundation Kurzweil provides is reasonable and I can’t see many reasons why Kurzweils predictions should be wrong (a nuclear war would be one, but we surely do not want that!).

Alas, we’re up for some interesting times in the next 25 years, lets try to make the best of it! By which I mean, proactively promote the need for AI safety regulations (for example in the form of technological frameworks or social infrastructures)!

I’ll fly to America tomorrow! Woohoo! That’s going to be my first trip out of Europe, and I’m quite excited to see if the US can live up to its self-declared awesomeness (see? I’m already learning the vocabulary).

I’m going to the Games Developer Conference in San Francisco to hear talks and join round table sessions about anything related to AI, so maybe there will be stuff for an article or two here on AI Panic. Ray Kurzweil will give the keynote speech, so there will definitely be some future-talk there.

I’ll be going there for 11 days, so you’ll have to live with less posts here than usual while I enjoy Yosemite National Park and Alcatraz (and all the other typical tourist attractions).

Scientists from the Ural University of Physics and Technology have developed components required for artificial consciousness according to the “consciousness electromagnetic information field” theory of Johnjoe McFadden. This theory states that the electromagnetic field induced by the neurons in the brain carries human consciousness.

Russia IC reports in somewhat broken English that the Russian think tank “created a model of neural network on neurons with additional channels for information exchange via electromagnetic field.” Apparently the artificial neurons they created are very similar to the EM neuron concept from McFadden.

It is not really clear from the article exactly how powerful or useful their hardware is, however, for the Russian scientists it seemed good enough to patent it. So far, I haven’t seen a single convincing use of artificial neural networks apart from academic toy problems. There are implementations here and there that use them, but almost everywhere it is easier and faster to just use other, more predictable and understandable techniques. Universities hype them in AI lectures (which is ok, they are relatively easy and have a nice theory behind them), and I believe that’s why many try to apply them everywhere they can (which is also ok, but only as long as they have good reasons to do so — which they usually don’t).

Therefore I doubt they will achieve their goals of an artificial consciousness, especially as they themselves don’t know how or if spontaneous generation of consciousness in this kind of networks works at all. It is not impossible though, therefore I’ll rise the AI Panic Level by +0.1%. After all, the Flying Spaghetti Monster must have had some motivation to endow us with a natural neural network …

The DARPA has announced that it will be using a Generalized Integrated Learning Architecture (GILA) system from Lockheed Martin to manage crowded airspace. It is intended to help the Air Force to cope with increased air traffic, especially as unmanned aerial vehicles and other airborne weapons get more common.

In effect, the DARPA is planning to hand over the control of missiles to an AI system which is able to learn and reason on its own. Brilliant! According to DARPA, the software will combine reasoning systems, general knowledge and by asking what-if questions.

“The integrated learner also will have explicit learning goals, keep track of what it does not know, what it needs to know, as well as track and reason about its uncertainties. The software will attempt to figure things out, as well as tolerate errors and missing information by using whatever information or reasoning is available. Integrated learning software must be able to manipulate many different forms of information and even trade off different types of information and reasoning, as well as interact with humans to fill in information gaps.”

Let’s just hope it does not learn from the wrong examples, or find out that it can work much more efficiently when it completely shuts off human influence from its controls. Many researchers believe that such reasoning systems are not unlikely to come to wrong solutions that result for example because of poor problem domain description (e.g., the goal might be to minimise the probability of a crash at a given future date. A solution is to crash all planes now, then there are no planes in the future, i.e., the probability of a crash will be zero …). Also, a learner that learns from positive and negative examples is prone to misclassifications. Therefore I’ll rise the AI Panic Level by +1%.

I believe that it might be hard to spot errors in the rule base and inconsistencies in the examples used to train the software. A system able to predict outcomes based on examples has to have some kind of bias, a restriction to possible world states. This bias must be chosen carefully to avoid empty solution sets (too strict restrictions) or full sets (too lax restrictions, the system is unable to make useful predictions).

As long as this system is used in training and double-checking the flight operators decisions only, there is no direct danger. But if DARPA decides to go live with this system (which might never happen though), it is of course critical that no errors occur. This depends crucially on the proper set-up of the background knowledge and underlying rule bases. Lets hope they get it right and make no precipitous moves to replace those error-prone humans!

The Gulf Cooperation Council might have to rely on artificial intelligence technologies to fight unemployment, GulfNews.com reports. Joachim Diedrich, a professor of artificial intelligence at the Uni of Queensland, Australia, embraces this dependency:

“For decades now, there has been a misconception that artificial intelligence will replace human resource. Rather, it is a job creator. [...] I see this as a positive opportunity. We should be optimistic about the availability of new jobs through the development of artificial intelligence.”

Probably we should indeed be optimistic, as there might not be a way around a tighter integration of AI into our daily life and products anyway. This integration has long started, it is being used in predicting weather and the stock market, so there is no way (and use) of denying its importance to the job market.

Although Diedrich tries to allay security doubts by saying that he does not think that “the industry will come to depend on artificial intelligence,” he lowers the credibility of his own statement with his next: “The use of artificial intelligence is not public knowledge, we don’t know how or where it is being used in the region.”

This far-reaching integration of black-box AI into important economical factors like the stock exchange and a huge number of jobs creates a dangerous dependency with regards to the worst case scenario of a hostile AI (it increases the AI Panic Level by +2%).

While current AI methods, used for example to predict the weather or stock values, are quite simple and pose no threat whatsoever, it will get more and more difficult to say that of future techniques. The accelerating technological advancement will enable very complex systems that incorporate extensive background knowledge from news articles, common sense databases and historical data to optimise the accuracy of predictions. This accurate world model might enable AIs to reason about everyday life and is a requirement for general artificial intelligence.

Thus a high job dependency on AI in this area will feed into the advancement and research of this area and brings us closer to a possibly dangerous strong AI.