Bi-Weekly Discussion - How Much Should We Trust the Experts?

This is going to be an online meetup using Zoom. If you've never used Zoom before, don't worry — it's easy to use and free to join.

Click on the link below at the scheduled date/time to join the Zoom event...

***

***

HOW MUCH SHOULD WE TRUST THE EXPERT CONSENSUS ON POLITICAL ISSUES?

(AND HOW DO WE EVEN DETERMINE WHO/WHERE "EXPERTS" ARE & WHAT MOST OF THEM THINK?)

INTRODUCTION:

As many of our meetup's regular know, my wife is having a baby in early April, so I'm going to be very busy over the next several months. I'll be handing off some of the hosting duties for individual meetups to some of our regular members on topics they know fairly well. But before I do that, I want to lay some ground rules and impart some things I've learned from running a political discussion meetup for the past 8+ years.

In this discussion, we'll address a very important topic - how should we source our knowledge on complex & controversial political issues? The title of this meetup - "How Much Should We Trust Experts?" - is intentional, i.e. it presupposes that it's obvious to most of our members that we all have to place *some* trust in *some* experts. I'm making this the starting point of our discussion because I think it forces people to be more intellectually honesty. Although we might be an expert in one domain of knowledge (and perhaps a fairly proficient hobbyist in a couple others), none of us have the time, money or background training to independently do research and come to an informed conclusion on the wide range of complex topics that often become part of our political debates.

When people say they "do their own research", what they're almost always referring to is not direct empirical research but rather reviews of other people's studies - so even in that case, you're relying on the data collected by experts who conducted the research, although you may dispute their conclusions. I've met a few really smart people with strong statistical training and a relatively unbiased mindset who appear to be able to do that in multiple domains, but it's still quite time-consuming and I think this ability is rather rare. So that means a certain degree of qualified trust or deterrence to experts is necessary in many cases, but we want to distinguish credible experts from crackpots and also make sure we're hearing from credible experts on all sides of an issue.

In the 1st section of this discussion, we'll discuss the merits and drawbacks to looking for the expert consensus among scholars in universities, focusing on how the leftward shift in the political views of academics since the 1990s has led most social science departments to have a 10-to-1 (or worse) progressive-to-conservative ratio. This has led most conservatives & also many moderates to suspect they can't trust the scholarly consensus anymore, and in response many have begun to look to conservative & libertarian think tanks for advice, but of course the experts in those institutions also have their own biases. We'll discuss several ways to account for these biases among experts, like looking for what lawyers call "admissions against interest" and using some heuristics that the rationalist community calls "bounded distrust" and the "general factor of correctness" (a.k.a. correct contrarian cluster).

In the 2nd section, we'll discuss the current expert consensus in macroeconomics - known as the "neoclassical-Keynesian synthesis" - and the merits and drawbacks to judging the expert consensus on various issues based on polls of top US & EU economists conducted by the University of Chicago's Clark Center Forum (formerly called the Initiative on Global Markets or "IGM"), the surveys of US economists conducted by the National Association of Business Economists (NABE), and the survey of EU economists done by the Center for Macroeconomics (CFM). We'll also consider some argument about the laypeople's distrust for the economic consensus that the economist Bryan Caplan made in his 2007 book The Myth of the Rational Voter: Why Democracies Choose Bad Policies (summarized here), as well as some heuristics for distinguishing good & bad critiques of mainstream economics.

In the 3rd section, we'll discuss the current expert consensus in political science, focusing on the trend towards "experimental political science" that incorporates insights from the behavioral sciences and bases its conclusions on empirical research rather than the older traditions that leaned more on armchair theorizing. We'll discuss whether the expert consensus in political science can be assessed based on a couple widely-discussed works - Hans Noel's 2010 paper "Ten Things Political Scientists Know That You Don't" and Larry Bartels & Christopher Achen's 2016 book Democracy for Realists: Why Elections Do Not Produce Responsive Government (summarized here). We'll also discuss the surveys of political scientists conducted by the nonprofit Bright Line Watch, which are intended to “monitor the status of democratic practices and highlight potential threats to American democracy."

In the 4th section, we'll discuss the current expert consensus in the field of international relations (IR), focusing mostly on the two major schools of thought known as neorealism (a.k.a. structural realism) & neoliberalism (a.k.a. institutional liberalism) - both are "positivist" schools in the sense that their scholars believe in the efficacy of empirical research to understanding the behaviors of nation-states. We'll also discuss the "SNAP polls" of IR scholars conducted by the College of William & Mary's TRIP Program, as well as the Council on Foreign Relations' Preventative Priorities Survey, and the extent to which they can give us some useful insight into how foreign policy experts think. We'll also consider some problems with the accuracy of predictions made by geopolitical experts raised in Philip Tetlock's 2005 book Expert Political Judgment: How Good Is It? How Can We Know? (summarized here), as well as his follow-up research that indicated some smart laypeople could be trained to generate geopolitical predictions that were better than most experts, described in his 2015 book Superforecasting: The Art & Science of Prediction (summarized here).

RELEVANT MATERIAL FROM PAST MEETUPS:

Right before today's discussion, the Skeptics are having a meetup entitled "Should Skeptics Defer to the Expert Consensus?" In Part I, they'll go over some common mistakes people make when they dismiss the expert consensus (e.g. calling it "argument from authority", invoking the "shill gambit", talking about "lived experience"). In Part II, they'll address the "demarcation problem" which determines what qualifies as "science," as well as how many of your opponent's arguments you should address before concluding it's all bunk. In Part III, they'll address how we might be able to check the strength of the expert consensus and compare the views of experts from different fields.

Back in Jan. 2024, we had a meetup entitled "Intellectual Virtues for Political Discussions", and in the 1st section we discussed the value of intellectual honesty (i.e. not lying to others - or to yourself) and the need to balance one's intellectual autonomy (thinking for yourself) with intellectual humility (acknowledging the limits of one's knowledge).

In Jan. 2021, we hosted a discussion entitled "Can & Should We Be 'Free Thinkers'?" After comparing two different methods of "free thinking" - i.e. reasoning from first principles vs constructing hybrid models from other people's theories - we looked at how this might apply to thinking about debates over public health responses to the COVID pandemic, racial disparities in policing & the criminal justice system, and contested elections. One of the major takeaways was the while it's conceivably possible to be a free thinker in multiple domains, it's also easy to end up becoming a crank without even realizing it if you don't have enough intellectual humility & awareness of your own tendencies towards motivated reasoning. And when many people decide to disregard the experts and take matters into their own hands, the societal costs can be high.

Back in 2016, we had a meetup where we looked at how the issue of expert consensus is treated in the rationalist community, Eliezer Yudkowsky's about why the majority is often wrong & when thinking for yourself is the better option, and some suggestions from Scott Alexander about trying to find the "correct contrarian cluster".

NOTE: Several past meetups have discussed the expert consensus in various academic fields in greater depth:
* "Expert Consensus & Economics" (Nov. 2016)

* "IR Scholars & The Foreign Policy Consensus" (Sept. 2017)

* "Political Science & Political Myths" (Feb. 2018)

***

DIRECTIONS ON HOW TO PREPARE FOR OUR DISCUSSION:

The videos & articles you see linked below are intended to give you a basic overview of some of the major debates over expertise in the social sciences and how much laypeople should defer to social scientists. As usual, I certainly don't expect you to read all the articles prior to attending our discussion. The easiest way to prepare for our discussion is to just watch the numbered videos linked under each section - the videos come to about about 60 minutes total. The articles marked with asterisks are just there to supply additional details. You can browse and look at whichever ones you want, but don't worry - we'll cover the stuff you missed in our discussion.

In terms of the discussion format, my general idea is that we'll address the topics in the order presented here. I've listed some questions under each section to stimulate discussion. We'll do our best to address most of them, as well as whatever other questions our members raise. I figure we'll spend about 30 minutes on each section.

***

I. WHO/WHERE ARE THE EXPERTS ON SOCIAL SCIENTIFIC ISSUES & HOW MUCH SHOULD WE TRUST THEM?

• SECTION 1, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 1, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 1, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 1, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

1a) Tom Nichols, "The problem with thinking you know more than the experts" (video - 3:10 min)
.
1b) Brett Weinstein & Heather Heying, "When to Listen to Experts" (video - 7:25 min)
.

• Lars Harhoff Andersen, "Should [Social] Science be Ideological?" (Substack)

• Cory Clark, "Biased [Social] Science Makes Bad Policy" (Psych. Today)

• Miles Corak, "How to think about 'think' tanks" (personal blog)

• Scott Alexander, "The General Factor of Correctness" (SSC)

• Avi Moshowitz, "On Bounded Distrust" (Substack)

II. WHAT IS THE EXPERT CONSENSUS ON ECONOMIC ISSUES & HOW MUCH SHOULD WE TRUST IT?

• SECTION 2, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 2, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 2, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 2, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

2a) Bryan Caplan, "Myth of the Rational Voter: Laymen/Expert Comparison, Survey of Econ. Beliefs, Laymen & Expert Biases" (video - 50:07 min, start at 16:50 & listen to 32:50)
.
2b) Erik Angner, "Economics: Science vs Ideology" (video - 1:27 min)
.

• Kevin Drum, "Economists Are Almost Inhumanly Impartial" (Mother Jones)

• The Economist, "Room with a view: If economists agree on something, the public will almost certainly think differently"

• Roger Gordon & Gordon B. Dahl, "What Do Economists think about major Public Policy issues? Views among Economists: Professional Consensus or Point-Counterpoint?" (Econweb)

• Jason Collins, "Six signs you’re reading good criticism of economics" (personal blog)

• Alex Marsh, "On signs you’re reading bad criticism of economics" (personal blog)

III. WHAT IS THE EXPERT CONSENSUS ON THE POLITICAL PROCESS - AND HOW MUCH SHOULD WE TRUST IT?

• SECTION 3, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 3, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 3, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 3, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

3a) Steven Gonzales, "Hans Noel - Ten Things Political Scientists Know that You Don't Video Lecture" (video - 20:17 min)
.
3b) Susan Stokes, "Democratic Erosion and How to Prevent It" (video - 59:44 min, start at 8:55 & listen to 13:30)
.

• Jason Brennan, "Political Science and the Obvious" (BHL blog)

• Julia Azari, "The Roots of Political Science Frustration" (Wash. Monthly)

• Andrew Taylor, "Political Science Needs Intellectual Diversity, But Few Realize It" (JGM Center)

• Matt Grossman w/ Brendan Nyhan, "How Much Did Trump Undermine U.S. Democracy [according to Bright Line Watch]?" (Niskanen)

IV. WHAT IS THE EXPERT CONSENSUS ON FOREIGN POLICY ISSUES & HOW MUCH SHOULD WE TRUST IT?

• SECTION 4, QUESTION 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 4, QUESTION 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 4, QUESTION 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
• SECTION 4, QUESTION 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

4a) College of William & Mary, "Steve Hanson on the SNAP Poll" (video - 1:39 min.)
.
4b) Andreas Bieler, "Positivist vs Post-positivist IR theory" (video - 4:33 min)
.
4c) Philip Tetlock, "His Groundbreaking Research About 'Expert Judgment' [on Geopolitics]" (video - 3:23 min)
.

• Jacob Poushter, "American public, foreign policy experts sharply disagree over involvement in global economy" (Pew Research)

• Henry E. Brady, et al., "Informed Public Opinion About Foreign Policy: The uses of deliberative polling" (Brookings)

• Stephen M. Walt, "America’s IR Schools Are Broken: There’s a lot of innovation on the surface, but the rot runs deep. Here’s how to fix it." (FP)

• Richard Hanania, "‘Just Trust the [Foreign Policy] Experts,’ We’re Told. We Shouldn’t." (NY Times)

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX