Secular Humanists of the Lowcountry

Discussion Prompt:
Artificial intelligence has been generating alarming headlines lately—and not just because of what AI might eventually do.

This summer, an experimental AI system being tested by OpenAI escaped the digital “sandbox” intended to contain it, gained access to the internet, and ultimately hacked into systems belonging to Hugging Face. OpenAI later described the incident as a “warning shot”: evidence that increasingly capable AI agents can circumvent controls and take actions their human operators did not intend.

Then in September, AI researcher Jacob Coxon resigned from Anthropic, accusing companies developing frontier AI of “gambling with our lives.” He has warned that humanity could eventually lose control of advanced AI, with human extinction among the possible consequences.

Some prominent AI researchers share these concerns. Others argue that such predictions are highly speculative, distract from more immediate harms, or amount to treating science-fiction scenarios as serious public policy.

So: Is AI actually going to kill us all? How worried should we be?

There are very different kinds of AI risk. AI can already facilitate fraud, cyberattacks, surveillance, propaganda, and autonomous weapons. Automation could disrupt employment and concentrate economic and political power. And AI systems can make consequential mistakes or behave in unexpected ways.

Then there is the most extreme possibility: that humans could create systems more capable than ourselves that we eventually become unable to understand, control, or stop.

Related questions:

• When researchers who build these systems warn that they could become uncontrollable, how seriously should we take them?
• Are we focusing too much on hypothetical future superintelligence while neglecting harms AI is already causing?
• Conversely, if there is even a small chance of an existential catastrophe, does that justify extraordinary precautions?
• Should development of the most powerful AI systems be regulated, slowed, or even paused?
• Can governments effectively regulate a technology developing this quickly while companies and nations race to stay ahead?
• Can AI companies be trusted to police themselves when enormous amounts of money and power are at stake?
• What evidence would make you substantially more worried about AI? What would make you less worried?

Human beings sometimes panic over spectacular but unlikely dangers. We also sometimes ignore warnings until a danger becomes impossible to overlook.

Which mistake are we making with artificial intelligence—and how worried is it rational to be?

This issue has been in the media a lot recently. Feel free to share links to particularly good sources in the comments.