FAQ Annex

(an extension of FAQs)

Q: I am a University Student at ___ . I am part of an honors seminar that will debate whether or not AI is a threat, or could become a threat to mankind and why.
A: Response by Patrick J. Hayes:

Well, Im tempted to ask what KIND of a threat is being considered here, and a threat to WHAT exactly? Because the answer must depend on having a little more detail.

To attempt to answer your question at face value, I'd say that AI is certainly not a threat in the usual science-fiction sense. This is the imaginary scenario in which machines become (or are made) intelligent, and then as a result decide to take over, and (since once one is as smart as a human, to get much smarter is only a matter of buying some extra RAM) the machines will have the upper hand and we humans will all become slaves before we know what has hit us. Some variation of this is a widespread idea in current fiction and even among AI folk, but it is utter nonsense. It rests on a simple mistake: the identification of intelligence with ambition. Human beings evolved, and so, like all animals, we are imbued with a passionate will to dominate or succeed in competition with other living things, a will which might be paraphrased as the will to survive or the will to live. (I don't mean to imply that mankind is red in tooth and claw, only that it is unusual to find a human being who is totally disinterested in their own survival.) However, this will to live isn't identical with intelligence, and one can imagine a very intelligent artifact with a deep intellectual capacity which is totally uninterested in surviving or taking over or controlling human life or destiny. In fact all the AI systems that have ever been built have no particular interest in social or biological domination, yet some of them are undeniably very smart at what they do. Chess-players play intelligent chess, AI diagnostic systems do intelligent diagnosis, etc. etc... Just making them smarter isnt going to make them more ambitious. Intelligence, itself, is not synonymous with fitness for survival and still less is it anything particular to do with having a will to live, let along a will to dominate or control human society. So unless people deliberately set out to make something like an artificial Ghengis Kahn, I see no rational reason to suppose that our artifacts will have any particular desire to take us over, any more than the automobiles that we make are likely to start running around by themselves.

Now, this answer might seem naive, if what you mean by "AI" is the creation of something like an artifical superman, since of course human beings do have these ambitions. But thats a basic mistake about AI. The field isnt concerned with making Golems, but creating intelligent artifacts. Some AI people talk about the field as having an ultimate ambition of creating "human-level intelligence" (John McCarthy's phrase). I disagree, myself, but in any case, notice that he says human-level *intelligence*, not human-level ambition or greed or will to conquer or craftiness.

Just in terms of the likely effects of technology, I think that AI is much more likely to be a boon than a threat to humans. In many ways one can best describe AI technology as the development of what my colleague Ken Ford calls 'cognitive prostheses': systems that people can use to amplify their own intellectual capacities; that will do some of their thinking for them, just as our eyeglasses help us see things and our clothes keep us warm. We have developed this theme in some publications in more detail, but the essential point is that such tools actually *empower* people and aid in removing social barriers. To dramatize the point, its worth noting that about a hundred years ago, the ability to do rapid mental arithmetic was considered an impressive intellectual talent, and people who could do it received academic honors. Nowadays a high-school dropout at a supermarket checkout can tell the customer the total charge in a fraction of a second. A barcode scanner and a computer read-out act as a kind of mental amplifier enabling someone to perform a task that, without it, would require greater mental capacity than he could deploy unaided. True, we don't usually say that the supermarket checkout clerk is using this machinery to think with; but ask yourself: who is earning the wages, the human or the computer?

Now, I think that one could say that AI has some threatening aspects, but of a much more subtle and intellectual sort. In some ways, the scientific goal of AI might be described as understanding intelligence (including human intelligence, ultimately) in computational terms. If we do ever achieve such an understanding (we are only at the beginning of this task right now) then we will, I think, as a society, be forced to re-think our view of ourselves and our place in nature. There are some who might find this prospect threatening, as I suspect it will call into question some of our long-held views of ourselves. But this is a very tenuous kind of 'threat'.