Existential risk is sexy. Like Oppenheimer, tech gurus can become Death, destroyer of worlds. The AI world is full of people who on the one hand tell us that their chatbot is the greatest thing ever invented, key to the mysteries of consciousness itself, and in the next breath warn us that it may well kill every human being alive.
Sam Altman has some interesting things to say about the threat AI poses to the very survival of humanity.
the bad case — and I think this is important to say — is, like, lights out for all of us.
As for the next sentence, each time I read it I am newly amazed. It’s possibly one of the most incredible things a human has ever said.
I think AI will… most likely sort of lead to the end of the world, but in the meantime there will be great companies created with serious machine learning
That a person could even put these ideas together and let them pass their mouth is incomprehensible to me. But anyway, he is clear, AI is not just a threat, it is the threat.
development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity.
In a formal statement on the OpenAI website, he says:
Some people in the AI field think the risks of AGI (and successor systems) are fictitious; we would be delighted if they turn out to be right, but we are going to operate as if these risks are existential.
His views are normal in the AI community.
Dario Amodei, CEO of Anthropic: “I think at the extreme end is the … fear that an AGI could destroy humanity. I can’t see any reason in principle why that couldn’t happen.”
Elon Musk, founder and CEO of xAI: “one of the biggest risks to the future of civilization is AI”.
Sundar Pichai, the brahmin CEO of Google: advanced AI “can be very harmful if deployed wrongly,” and that with respect to safety issues, “we don’t have all the answers there yet, and the technology is moving fast. … So does that keep me up at night? Absolutely.”
Shane Legg, co-founder of DeepMInd: “We have no idea how to solve this problem.”
Since everything that is happening today is unprecedented, there is definitely a chance that AI will destroy humanity. But I believe this is the first time the developers of a technology have flirted so openly with nihilistic desire. They sell us anxiety and fear because they know we’re addicted.
But I think it’s more than that. It’s not just a cynical marketing ploy, nor is it rational. It’s nihilism. I think they want humanity to die. I think there’s a deep level hatred of themselves, manifesting as hatred of humanity, that at some unconscious level is fueling this inchoate need to propel ourselves to a future where we no longer cause any problems because we don’t exist.
In Buddhism we call this vibhavataṇhā, the craving for annihilation. It manifests as addictive and self-harming behavior, and in extreme forms as suicide. But these guys are coyly flirting with suicide of the whole human race. Look, if that’s their thing, fine. People are messed up. All I’m saying is that sane people need to stop them.
It’s impossible to guess the actual risk and foolish to try. I personally agree with Émile P. Torres that the risk of extinction is overrated. Its difficult to imagine a genuine scenario leading to extinction. That, however, doesn’t change the fact that, as the OpenAI website says, “a misaligned superintelligent AGI could cause grievous harm to the world”.
The real risk is the actual things that are destroying the world, primarily climate change. AI distracts us from the real problems, offers no genuine solutions, and in doing so consumes a vast amount of energy, both physical and mental.
An OpenAI employee tweeted that since everything is accelerating, just relax and worry about “stupid mortal things” like spending time with your families while AGI becomes a reality: “I don’t feel any control everyone else certainly shouldn’t”.
Has any product ever been sold with such reckless nihilism? Even with nuclear weapons, they would tell us they could destroy the world, but at the same time reassure us that they would try their best not to use them. How is it vaguely legal to advertise your product as a world-destroyer and then just put it into everyone’s computers?