Bi-Weekly Discussion - Preventing an AI Apocalypse

Hosted by Brian B.

Super Organizer

Philadelphia Political Agnostics

Details

This is going to be an online meetup using Zoom. If you've never used Zoom before, don't worry — it's easy to use and free to join.

Click on the link above at the scheduled date/time to log in...

***

DOES ARTIFICIAL INTELLIGENCE POSE AN EXISTENTIAL RISK -
AND IF SO, HOW SHOULD WE PREVENT THIS?

INTRODUCTION:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

WHAT'S THE DISTRIBUTION OF OPINIONS ABOUT A.I. AMONG EXPERTS?

As usual, we'd like to ground our discussion in the current views of experts on AI, so I looked up some polls:

* AI Impacts, "2023 Expert Survey on Progress in AI" - Average p(doom) between 14 and 19.4%, depending on how the question is phrased. 86% believe the alignment problem is important.
https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai

* University College London, "Are AI researchers concerned about the existential threat of AI? Our survey suggests not"
https://rai.ac.uk/hed-are-ai-researchers-concerned-about-the-existential-threat-of-ai/

* Effective Altruism Forum, "Survey of AI safety leaders on x-risk, AGI timelines, and resource allocation (Feb 2026)"
https://forum.effectivealtruism.org/posts/LxuKuQd69Qx5FKhNZ/survey-of-ai-safety-leaders-on-x-risk-agi-timelines-and

* Pew Research, "How the U.S. Public and AI Experts View Artificial Intelligence"
https://www.pewresearch.org/internet/2025/04/03/how-the-us-public-and-ai-experts-view-artificial-intelligence/

* Stanford Institute for Human-Centered AI, "The 2026 AI Index Report"
https://hai.stanford.edu/ai-index/2026-ai-index-report

* Elon University's Imagining the Digital Future, "Experts Predict the Impact of AI by 2040"
https://imaginingthedigitalfuture.org/reports-and-publications/the-impact-of-artificial-intelligence-by-2040/the-17th-future-of-digital-life-experts-canvassing/

* Kent Clark Center, "AI and Growth" (poll of US & EU economists)
https://kentclarkcenter.org/surveys/ai-and-growth/
https://kentclarkcenter.org/surveys/ai-growth-and-jobs/

* Metaculus, "When will the first general AI system be devised, tested, and publicly announced?" (Mean Estimate: Apr. 2033)
https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/

RELEVANT MATERIAL FROM PAST MEETUPS:

Back in February, Braver Angels hosted a group discussion on "Risks & Benefits of AI" based on some reports from CBS & the BBC, as well as a 60 Minutes interview with AI pioneer Geoffrey Hinton.

Also in February, the Commonwealth Club hosted a panel discussion entitled "The Economy 2026: Bubble or Boom?" where they debated whether the current AI boom is another dot-com bubble.

Back in Feb. 2024, the Skeptics held a meetup entitled "Are We Approaching the Singularity?" where they discussed: (1) how economists measure innovation & technological progress and the tech industry's track record for predicting the rise of emerging technologies; (2) he idea of the technological singularity in futurist Ray Kurzweil's famous book The Singularity Is Near (2005) & the track record of his predictions; (3) why the "Turing test" is confusing for many people, why some philosophers don't consider it a good test for determining if an artificial intelligence is self-aware; and (4) the concepts of "artificial general intelligence" (AGI) and "artificial super intelligence" (ASI), as well as predictions about how long it will take to develop ASI once AGI is achieved based on different "AI takeoff" timelines.

DIRECTIONS ON HOW TO PREPARE FOR OUR DISCUSSION:

The videos & articles you see linked below are intended to give you a basic overview of some of the major points of contention in the debates over existential risks posed by Artificial Intelligence and the various proposals for AI safety regulation. As usual, I certainly don't expect you to read all the articles prior to attending our discussion. The easiest way to prepare for our discussion is to just watch the numbered videos linked under each section - the videos come to about XX minutes total. The articles marked with asterisks are just there to supply additional details. You can browse and look at whichever ones you want, but don't worry - we'll cover the stuff you missed in our discussion.

In terms of the discussion format, my general idea is that we'll address the topics in the order presented here. I've listed some questions under each section to stimulate discussion. We'll do our best to address most of them, as well as whatever other questions our members raise. I figure we'll spend about 30 minutes on each section.

***

I. THE A.I. ALIGNMENT PROBLEM & SIGNS OF MISALIGNMENT IN LLMs:

SECTION 1, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 1, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 1, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 1, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 1, QUESTION 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

1a) Fred Sala, "What is AI alignment?" (video - 3:50 min)
https://www.youtube.com/watch?v=7kPqKk87WIw

1b) VIDEO 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (video - XX:XX min)

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Wikipedia, "AI alignment"
https://en.wikipedia.org/wiki/AI_alignment
ARTICLE 2 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 3 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 4 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

II. THE DEBATE OVER AGI/ASI PREDICTIONS: ARE WE ON THE CUSP OF AN "INTELLIGENCE EXPLOSION", MODEST GROWTH, OR IN A BUBBLE THAT WILL SOON BUST?

SECTION 2, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 2, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 2, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 2, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 2, QUESTION 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

2a) Sabine Hossenfelder, "How close is AGI? What the experts say." (video - 6:55 min, listen to 5:30)
https://www.youtube.com/watch?v=Z6q2iJZmvOM

2b) VIDEO 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (video - XX:XX min)

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Less Wrong, "[Eliezer] Yudkowsky vs [Robin] Hanson on FOOM: Whose Predictions Were Better?"
https://www.lesswrong.com/posts/gGSvwd62TJAxxhcGh/yudkowsky-vs-hanson-on-foom-whose-predictions-were-better
ARTICLE 2 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 3 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 4 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

III. THE DEBATE OVER A.I. TAKEOVER & DOOMSDAY SCENARIOS:

SECTION 3, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 3, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 3, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 3, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 3, QUESTION 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

3a) Conscious Code, "{Max Tegmark] Reveal the Most Likely Endings for AI" (video - 15:05 min)
https://www.youtube.com/watch?v=TA6cfGsF4Sw

3b) VIDEO 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (video - XX:XX min)

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Wikipedia, "AI aftermath scenarios"
https://en.wikipedia.org/wiki/AI_aftermath_scenarios
Wikipedia, "Existential risk from artificial intelligence"
https://en.wikipedia.org/wiki/Existential_risk_from_artificial_intelligence
ARTICLE 2 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 3 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 4 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

IV. THE DEBATE OVER THE FEASIBILITY OF A GLOBAL A.I. PAUSE OR BAN:

SECTION 4, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 4, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 4, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 4, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SECTION 4, QUESTION 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

4a) ABC News, "Elon Musk calls for artificial intelligence pause" (video - 2:13 min)
https://www.youtube.com/watch?v=jJbKdue5vdc

4b) VIDEO 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (video - XX:XX min)

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

ARTICLE 1 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 2 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 3 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ARTICLE 4 - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Philadelphia Political Agnostics

Bi-Weekly Discussion - Preventing an AI Apocalypse

Philadelphia Political Agnostics

Details

Related topics

You may also like