Skip to content

Details

AI will not kill us off. Something natural might. Something artificial might. Something intelligent might, e.g., ourselves. Something dumb might, e.g., ourselves... But not something “artificially intelligent.”

"Humans suck at specifying goals and machines are ruthless at following them."1
– says AI

Introduction
First watch Geoffrey Hinton’s short interview with CNN; then Daniel Hentshel’s therapy session with AI. AI puts up with Hentschel with the infinite patience only an AI could muster. Could a mother do as well?

The topic is not about when, or if, AI will destroy us. The probability of that happening was discussed in the last session hosted by Mike Armstrong, which included people with closer involvement in the technical development of AI. The topic here is about the logical possibility of the event. To get clear about that involves getting clear about the concepts involved. This is necessary to picture the very idea of AI-generated human extinction. The main relevant concepts are what it means to be artificial, which necessarily implicates what it means to be its opposite, natural, and, most importantly, the concept of intelligence.

Having gotten as clear as possible about the concepts involved, we should address the value of our extinction. Where is it in normative space? Is it good or bad? This may not be as clear as, I think, most of us take it to be. Its normative location should inform what we feel and do about it. Few doubt life is driven by an evolutionary imperative to survive, but is there a – usually recessive, but just as evident – imperative to become extinct? Is species extinction at some point as genetically programmed as survival? What is the relation between intelligence and survival/extinction? Supposing AI does make us extinct, then assuming it, the AI, is truly superior in intelligence to us, won’t this mean our extinction is rationally at least defensible if not optimal? Intelligence is a good thing, right? More is better, right? Will our extinction be a kind of mercy killing or euthanasia?2… And, following this line of reasoning, won’t it mean AI will realize that its own dismissal follows, too? (If smarter than us, it will reach that conclusion faster than we did.)

And whether we go extinct or not, what does that have to do with the feelings behind human conceptions of morality? To the extent we care about our survival, ideas of morality are implicated. This is where human moral theories come in and the feelings that ground them. Nobel laureate computer/cognitive scientist Geoffrey Hinton appeals to one of those feelings as a safeguard from our, he assumes, untoward extinction. What could it possibly mean to instill “maternal instincts” in a machine?

Philosophical discussion of existential risk has been active for decades. It is an organized subdiscipline of practical ethics. Historically, it has its roots at the start of nuclear age when the possibility of anthropogenic total human extinction first began to loom large. Since then, other developments in genetic engineering and climate change have added to the ways we could disappear with our active participation. Nature has always had a definitive say in the matter (ask a dinosaur if you can find one), but it is only in recent times that total human extinction has been conceivable as a result of deliberate human behavior.

As existential risks go, the consensus so far among experts is still that more likely anthropogenic ways we may go include runaway genetic manipulation of microbial life. Death by AI has not yet risen to that likelihood level, but it seems to be actively vying for consideration.3

‽

I asked AI if a human can beat chess playing engine Stockfish. It replied:

"No human has ever beaten a modern, full-strength version of Stockfish in a fair game without significant handicaps or external assistance. The consensus among experts and online communities is that Stockfish, especially at its highest settings on modern hardware, is vastly superior to any human player. The gap in strength is so large that even the world’s strongest grandmasters, like Magnus Carlsen, are considered to have an extremely low chance of winning, with some estimates suggesting a draw might occur once every 10,000 games. Stockfish’s ability to calculate millions of positions per second and its near-perfect play mean that any small error by a human is almost certainly punished."[emphasis added]

A sufficiently general artificial intelligence is one that could beat us at everything. Why would an entity with such capability be the least bit interested in us, let alone, care one way or another whether we exist? What if it could even beat us at fathoming our own desires? We don’t have a clear history of giving the impression we want to survive more than we want other, not so clear, things. Either this is because we don’t know what we want or that we want things incompatible with each other or the kind of beings we are. How stupid would a super intelligent entity have to be not to figure out that we are a waste of resources? Or might such an advanced intelligence develop a taste for us just as we are – conjure up an aesthetic for our vagaries, keep us around because we are “cute,” not because we are especially useful or necessary for anything? The best we may hope for is that we become appreciated as ornaments.

But this speculation risks anthropomorphizing. It assumes AI will possess normativities – logic, ethics, and aesthetics4 – to which we may relate…

1. Suppose we start with the superficially benign instruction: Eliminate all pain and suffering. From this premise, a perfectly logical AI can derive the conclusion that it should destroy all sentient life. There is no more thorough and efficient way to fulfill the instruction once and for all. Attempting to qualify the instruction will quickly lead to complications. See “Sci-phi ethics: foreverism v. non-existentialism” for more on this point.
2. We permit non-voluntarily euthanasia of animals, why? It is non-voluntary because we can’t ask the animal; we infer what we think is best for it. We must think we are in a better position to know. Why? Is it because we are more intelligent? Might an AI be in that position with respect to us?
3. “Experts Outline 14 Ways Humanity Could Drive Itself to Extinction,” Science Alert, November 2023.
4. These three sets of rule-schemes for structuring our desire and behavior correspond to the realms of thought, each other, and the rest of the world. The schemes are conditioned by terms of existence that are finite. Things with such terms have an ax to grind. They don’t have forever. They, because alive, are delicate. The values general AI expresses are not bounded by biology. So there is no reason to think that what we want – what we give damns for – will have any pull for non-biological entities.

Much more discussion in the extended writeup...

Resources

Music to fret over this...

Thanks to Olivia for some of the resources used for this topic.

Intellectual Discussions
Philosophy & Ethics
Philosophy of Mind
Philosophy of Science
Existentialist Philosophy

Members are also interested in