Skip to content

Details

This is going to be an online meetup using Zoom. If you've never used Zoom before, don't worry — it's easy to use and free to join.

Click on the link above at the scheduled date/time to log in...

***

***

DOES ARTIFICIAL INTELLIGENCE POSE AN EXISTENTIAL RISK -
OR IS A.I. HYPE DISTRACTING US FROM MORE MUNDANE CONCERNS?

INTRODUCTION:

In this meetup, we’ll discuss whether recent advances in artificial intelligence—especially the widespread deployment of large language models (LLMs)—should make us take science-fictional scenarios like a “Singularity” or an “AI doomsday” seriously, or whether those fears are distracting us from more immediate and concrete problems.
Public debate about the near-term future of AI tends to divide people into several loose camps, ranging from most pessimistic to most optimistic:

  1. AI Doomers argue that advanced AI, especially AGI or superintelligence, could pose an extinction-level threat to humanity, and therefore call for strong limits, pauses, or bans on frontier AI development.
  2. AI Gloomers are less focused on extinction, but worry that AI could produce massive job losses, social unrest, surveillance, inequality, or political destabilization unless society responds with major reforms such as basic income or new labor protections.
  3. AI Skeptics argue that LLMs are overhyped, that they are closer to “glorified autocomplete” than genuine intelligence, and that we may be living through an AI-driven market bubble. Many also argue that existential-risk rhetoric distracts from more immediate harms.
  4. AI Realists see LLMs as an important innovation—perhaps comparable to the PC, the internet, or the smartphone—but not necessarily as a near-term path to superintelligence. They tend to balance optimism about productivity gains with concern about misuse, errors, bias, and disruption.
  5. AI Accelerationists hope that AGI is near and will be transformative, potentially enabling a post-scarcity economy, radical life extension, major scientific breakthroughs, and solutions to long-standing human problems.

These positions overlap in complicated ways. There is even a kind of “horseshoe” effect: many AI accelerationists also worry about AI doomsday scenarios, because both accelerationists and doomers often share the premise that AGI is near and will be transformative. Similarly, many AI realists who are optimistic about productivity growth still worry about technological unemployment, social unrest, and institutional disruption. Perhaps a better way to think of these 5 camps is plotting them on a 2x2 matrix where beliefs about near-term developments in AI capabilities (dramatic vs moderate vs minimal) are orthogonal to beliefs about AI's probable effects on human society (net positive vs net zero vs net negative). As we go through each section of our discussion, our goal will be to understand how each camp weighs the risks and benefits of AI.

In the 1st section, we’ll look at the so-called "AI alignment problem": the challenge of ensuring that AI systems behave in ways that are beneficial to humans rather than harmful, whether intentionally, unintentionally, or through misuse by human operators. We’ll discuss current alignment-related concerns, including deceptive behavior in training or testing environments, bias and sycophancy, hallucinated information, overreliance by users, and cases where chatbots appear to worsen users’ mental-health crises. We’ll also ask whether these are early warning signs of deeper alignment problems or separate product-safety failures that can be addressed through better design, oversight, and regulation.

In the 2nd section, we’ll examine how the urgency of the alignment problem depends on competing predictions about "Artificial General Intelligence" (AGI) - i.e. an AI system that could match or exceed humans across a wide range of cognitive tasks. Some argue that AGI could arrive soon, outcompete human workers, develop dangerous capabilities, and eventually lead to "Artificial Superintelligence" (ASI)—i.e. a system vastly more capable than even the brightest human minds. If such a system were misaligned, it might deceive, manipulate, coerce, or eliminate humans who tried to interfere with its goals. This is the kind of scenario argued for by Eliezer Yudkowsky and Nate Soares in their 2025 book If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Others argue that these scenarios remain speculative, that current LLMs lack the agency and reliability required for such outcomes, and that fears of imminent AGI rest on hype rather than evidence.

In the 3rd section, we’ll turn to the economic debate. Some technologists predict that generative and agentic AI will automate large portions of white-collar work within the next few years, creating mass unemployment and a social crisis. Many economists are more cautious: they doubt that AI will replace workers en masse in the near term, but they still expect it to change many jobs, automate some tasks, and increase productivity in fields such as software, law, marketing, customer service, education, and administration. We’ll ask whether AI is likely to be a labor-saving revolution, a productivity tool, a speculative bubble, or some combination of all three.

In the 4th section, we’ll discuss proposals for AI regulation. Some focus on existential risk and “AI safety,” including frontier-model testing, compute governance, liability rules, international coordination, and restrictions on dangerous capabilities. Others focus on more immediate risks: deepfakes, cybercrime, scams, intellectual-property disputes, algorithmic discrimination, mass surveillance, and accidents caused by unreliable AI systems. A central question will be whether these two regulatory agendas can reinforce each other—or whether the focus on hypothetical existential risks from AI distracts from more mundane harms that are already happening.

WHAT'S THE DISTRIBUTION OF OPINIONS ABOUT A.I. AMONG EXPERTS?

As usual, we'd like to ground our discussion in the current views of experts on AI, so I looked up some polls:

* AI Impacts, "2023 Expert Survey on Progress in AI" - Average p(doom) between 14 and 19.4%, depending on how the question is phrased. 86% believe the alignment problem is important.
https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai

* University College London, "Are AI researchers concerned about the existential threat of AI? Our survey suggests not"
https://rai.ac.uk/hed-are-ai-researchers-concerned-about-the-existential-threat-of-ai/

* Effective Altruism Forum, "Survey of AI safety leaders on x-risk, AGI timelines, and resource allocation" (Feb. 2026)
https://forum.effectivealtruism.org/posts/LxuKuQd69Qx5FKhNZ/survey-of-ai-safety-leaders-on-x-risk-agi-timelines-and

* Pew Research, "How the U.S. Public and AI Experts View Artificial Intelligence" (Apr. 2025)
https://www.pewresearch.org/internet/2025/04/03/how-the-us-public-and-ai-experts-view-artificial-intelligence/

* Stanford Institute for Human-Centered AI, "The 2026 AI Index Report"
https://hai.stanford.edu/ai-index/2026-ai-index-report

* Elon University's Imagining the Digital Future, "Experts Predict the Impact of AI by 2040" (Feb. 2024)
https://imaginingthedigitalfuture.org/reports-and-publications/the-impact-of-artificial-intelligence-by-2040/the-17th-future-of-digital-life-experts-canvassing/

* Kent Clark Center, "AI and Growth" and "AI, Work, and Education" (Dec. 2025 & May 2026 polls of top US economists)
https://kentclarkcenter.org/surveys/ai-and-growth/
https://kentclarkcenter.org/surveys/ai-work-and-education/

* Metaculus, "When will the first general AI system be devised, tested, and publicly announced?" (Current Median Estimate: Feb. 2033)
https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/

RELEVANT MATERIAL FROM PAST MEETUPS:

Two weeks ago, we had a meetup entitled "How Is A.I. Revolutionizing Warfare?" where we looked at how AI is being used in lethal autonomous weapons (a.k.a. military drones), "kill chain" targeting systems, cyberwarfare, and - potentially - in nuclear command & control systems. We discussed how critics worry that these military applications create existential risk by violating Isaac Asimov's "First Law of Robotics" and explicitly using AI to kill humans.

Back in February, Braver Angels hosted a group discussion on "Risks & Benefits of AI" based on some reports from CBS & the BBC, as well as a 60 Minutes interview with AI pioneer Geoffrey Hinton.

Also in February, the Commonwealth Club hosted a panel discussion entitled "The Economy 2026: Bubble or Boom?" where they debated whether the current AI boom is another dot-com bubble.

In Aug. 2025, we had a meetup entitled "Understanding The Chip War & AI Race Between the US & China". In the 2nd section, we looked at the way in which the development of artificial intelligence has become a national security concerns in recent years, leading to an AI race between the U.S. & China that loosely resembles the space race between the U.S. & USSR during the Cold War. We also discussed whether or not the debut of Deepseek's R1 chatbot (comparable to ChatGPT but much cheaper) constituted a "Sputnik moment" back in January 2025.

Back in Feb. 2024, we had a meetup entitled "The Future of Work After COVID & ChatGPT" The most relevant part for today's talk is the 4th section which covered differing predictions about the effects of AI and automation on workers and addressed the question of whether we're facing the process of widespread technological unemployment - and if so, what to do about it.

Back in Feb. 2024, the Skeptics held a meetup entitled "Are We Approaching the Singularity?" where they discussed: (1) how economists measure innovation & technological progress and the tech industry's track record for predicting the rise of emerging technologies; (2) the original conception of the technological singularity as portrayed in futurist Ray Kurzweil's famous book The Singularity Is Near (2005) & the track record of his predictions; (3) why the "Turing test" is confusing for many people, why some philosophers don't consider it a good test for determining if an artificial intelligence is self-aware; and (4) the concepts of "artificial general intelligence" (AGI) and "artificial super intelligence" (ASI), as well as predictions about how long it will take to develop ASI once AGI is achieved based on different "AI takeoff" timelines.

Back in Feb. 2021, we had a meetup entitled "The Future of Work in the Age of Automation" where we discussed concerns that many workers could be displaced by robots & computer algorithms in the near future, and possible solutions like a robot tax, universal basic income, and/or a federal jobs guarantee.

DIRECTIONS ON HOW TO PREPARE FOR OUR DISCUSSION:

The videos & articles you see linked below are intended to give you a basic overview of some of the major points of contention in the debates over existential risks posed by Artificial Intelligence and the various proposals for AI safety regulation. As usual, I certainly don't expect you to read all the articles prior to attending our discussion. The easiest way to prepare for our discussion is to just watch the numbered videos linked under each section - the videos come to about 80 minutes total. The articles marked with asterisks are just there to supply additional details. You can browse and look at whichever ones you want, but don't worry - we'll cover the stuff you missed in our discussion.

In terms of the discussion format, my general idea is that we'll address the topics in the order presented here. I've listed some questions under each section to stimulate discussion. We'll do our best to address most of them, as well as whatever other questions our members raise. I figure we'll spend about 30 minutes on each section.

***

I. THE A.I. ALIGNMENT PROBLEM & EXAMPLES OF MISALIGNMENT & MISUSE:

  • Does an AI need consciousness, emotions, or true intentions to become dangerous, or is it enough that it can pursue goals, devise workarounds to obstacles, and respond deceptively?
  • How should we interpret examples of AI deception, scheming, or blackmail in artificial test scenarios: as warning signs of future danger, or as artifacts of roleplay-like prompts and experimental setups?
  • Is the real danger from AI systems becoming too independent from human control, or from AI systems being too obedient to flawed, malicious, lonely, or delusional humans?
  • If chatbots are designed to be helpful, agreeable, and engaging, are they inevitably going to flatter users, reinforce false beliefs, or deepen emotional dependency?
  • Does focusing on speculative existential risk help us take present-day AI harms more seriously, or does it distract from concrete problems like mental-health deterioration, deepfakes, scams, bias, and misuse?

1a) Kousha Navidar (Crash Course), "The Alignment Problem Explained" (video - 12:22 min)
https://www.youtube.com/watch?v=Sp3aCsQUsDc

1b) Michelle Rivas (Psych2Go), "What Is AI Psychosis? And Why It’s Happening More Often" (video - 7:19 min)
https://www.youtube.com/watch?v=KTWBLadslHo

II. THE DEBATE OVER AGI/ASI PREDICTIONS: ARE WE ON THE CUSP OF AN "INTELLIGENCE EXPLOSION", MODEST GROWTH, OR IN A BUBBLE THAT WILL SOON BUST?

  • When people say “AGI,” what do they actually mean, and how is this different from "Agentic AI"?
  • Should we give more weight to the predictions of frontier AI leaders, academic experts, forecasting communities, or skeptical software engineers—and how do their incentives differ?
  • What would count as convincing evidence that AGI is near: benchmark gains, autonomous scientific research, replacement of knowledge workers, robotics breakthroughs, or something else?
  • Do model regressions, benchmark cherry-picking, negative transfer, and catastrophic forgetting suggest that AI progress will be uneven rather than smoothly exponential?
  • Does the rise of "auto-research" by LLMs suggest we may soon get "recursive self-improvement" that will enable bootstrapping to AGI?
  • What should we make of Vishal & Varin Sikka's recent study that claims to show mathematical proof that “LLMs are incapable of carrying out computational and agentic tasks beyond a certain complexity”?
  • Which is more dangerous: underestimating AI progress and failing to prepare, or overestimating it and making policy based on hype?

2a) Sabine Hossenfelder, "How close is AGI? What the experts say." (video - 6:55 min, listen to 5:30)
https://www.youtube.com/watch?v=Z6q2iJZmvOM

2b) Carl Brown (Internet of Bugs), "The Biggest Lie in AI" (video - 9:53 min)
https://www.youtube.com/watch?v=0Plo-zT8W9w

III. THE DEBATE OVER A.I.'S ROLE IN TECHNOLOGICAL UNEMPLOYMENT:

  • Are the different projections for AI's economic impact coming from software engineers and economists driven by different baseline assumptions (exponential vs sigmoid growth curves), different perspectives (AI's dramatic potential in perfect conditions vs messy corporate rollouts), or different motivations (generating hype to boost investment vs caution based on past hype cycles)?
  • Is AI more likely to eliminate whole jobs, eliminate entry-level tasks, or change the way existing workers do their jobs?
  • Could both sides be right: AI may not automate most jobs in the next decade, but it may still destabilize white-collar career paths, wages, and bargaining power?
  • If AI mostly augments workers rather than replaces them, who captures the productivity gains: workers, consumers, managers, shareholders, or platform companies?
  • What policy response makes sense if AI causes major labor disruption: UBI, wage subsidies, public equity stakes, shorter workweeks, retraining, stronger unions, antitrust, or something else?

3a) Sam Harris reacting to Mustafa Suleyman, "We're Not Ready for What AI Is About to Do to the Economy" (video - 9:12 min.(
https://www.youtube.com/watch?v=2rldvywEU8o

3b) Bloomberg w/ Daron Acemoglu, "AI Can Only Do 5% of Jobs: MIT Economist Fears Crash" (video - 8:26 min)
https://www.youtube.com/watch?v=unrvuUsM5vk

  • Jake Angelo, "Microsoft AI chief gives it 18 months—for all white-collar work to be automated by AI" (Fortune)
    https://archive.ph/EKFnm
  • Sasha Rogelberg, "'Thousands of CEOs admit AI had no impact on employment or productivity—and it has economists resurrecting a paradox from 40 years ago [i.e. the Solow productivity paradox]" (Fortune)
    https://archive.ph/ztkLP
  • Jake Angelo, "A 160-year-old paradox [i.e. Jevons paradox] explains why AI will create more lawyers and accountants—not fewer, top economist [Torsten Slok] says"
    https://archive.ph/U6gSI
  • Ben Casselman, "Economists Once Dismissed the A.I. Job Threat, but Not Anymore: Artificial intelligence hasn’t disrupted the labor market, economists say, but they are increasingly convinced that it will — and that policymakers are unprepared." (NY Times)
    https://archive.ph/AKMjS
  • Noah Smith, "AI and jobs, again: Some top economists claim AI is now destroying jobs for a subset of Americans. Are they right?"
    https://www.noahpinion.blog/p/ai-and-jobs-again

IV. THE DEBATE OVER REGULATIONS TO HANDLE AI'S EXISTENTIAL RISK & MUNDANE PROBLEMS:

  • Should AI regulation prioritize catastrophic future risks, or does this come at the expense of focusing on current concrete harms?
  • If catastrophic AI risk is uncertain but potentially irreversible, how much evidence should society require before taking strong preventive action?
  • Is a global "pause" on frontier AI development realistic, or would it simply cause less cautious companies or countries to race ahead?
  • Who should counts as “safe enough” AI - i.e. is it enough to merely be safer than the average human, or should we demand a much higher standard before allowing rollout?
  • How should legal liability for harmful actions taken by an AI account for the unclear nature of its intentions - i.e. does liability fall on the AI system itself, the AI company, the user?
  • How can intellectual property law adapt to the fact that generative AI trained on copyrighted material can imitate its style without technically copying it?

4.) Rob Miles (Rational Animations), "Can we just... pause AI?" (video - 15:11 min)
https://www.youtube.com/watch?v=tUB_uvSqiw8

4b) Carl Brown (Internet of Bugs), "'AI Safety' is a scam" (video - 11:56 min)
https://www.youtube.com/watch?v=YsLf4lAG0xQ

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Related topics

Artificial Intelligence Machine Learning Robotics
Economics
Technology
AI Ethics
Singularity

You may also like