Group Discussion: Skepticism and "Expert Consensus"

Those who've been involved in the skeptic movement and have seen some debates between skeptics and conspiracy theorists or pseudoscience believers probably know how often the "expert consensus" among scientists or scholars becomes a key issue. Skeptics tend to come down on the side of the expert consensus, while their opponents often argue the experts are paid off or engaging in groupthink or otherwise compromised. This group discussion will focus on a series of short entries from RationalWiki (a wiki designed for the skeptic community) as well as a series of essays from some of the well-known skeptic celebrities & upcoming skeptic bloggers that deal with issues of expert consensus & contrarianism.

Although the "appeal to authority" and "appeal to majority" are normally considered to be informal logical fallacies which cannot prove a case definitively, we'll discuss cases where looking at what the majority of authorities on a subject believe can still give laymen an idea of what is more likely to be true. The essays also address the difference between skepticism and other forms of disbelief like denialism & cynicism that aren't justified. We'll also look at why disbelieving the expert consensus in non-scientific fields is justified but tricky and depends on a philosophical problem called the "demarcation problem". Lastly, we'll examine four debates in the skeptic community over whether laypeople would ever be justified in disputing the expert consensus in empirical fields.

Each of the following articles is fairly short. I don't expect people to read all of the essays, but hopefully some of you can at least skim them. For those who don't have the time to read the essays, I've added some notes for each essay to give you a general sense of the points they're making.

• NOTE: This meetup is held in cooperation with the Philly Political Agnostics meetup and the Philadelphia Philosophy Meetup.

WHEN ARGUMENT FROM AUTHORITY & MAJORITY AREN'T LOGICAL FALLACIES, AND WHY THEY WORK BETTER WHEN COMBINED:

1. Rationalwiki, "Argument from Authority"

RW clarifies that an argument from authority refers to two kinds of logical arguments:

(1) A logically valid (http://rationalwiki.org/wiki/Validity) argument from authority grounds a claim in the beliefs of one or more authoritative source(s) (http://rationalwiki.org/wiki/Credentials), whose opinions are likely to be true on the relevant issue. Notably, this is a Bayesian (http://rationalwiki.org/wiki/Bayesian) statement -- it is likely to be true, rather than necessarily true. As such, an argument from authority can only strongly suggest what is true -- not prove it.

(2) A logically fallacious (http://rationalwiki.org/wiki/Logical_fallacy) argument from authority grounds a claim in the beliefs of a source that is not authoritative. Sources could be non-authoritative because of their personal bias, their disagreement with consensus on the issue, their non-expertise in the relevant issue, or a number of other issues. (Often, this is called an appeal to authority, rather than argument from authority.)

http://rationalwiki.org/wiki/Argument_from_authority

2. Rationalwiki, "Argumentum ad populum"

In its entry for "argumentum ad populum" (argument from popularity), RW has a section on legitimate use that pertains to the scientific consensus:

What's the difference between most people believe X and scientific consensus (http://rationalwiki.org/wiki/Scientific_consensus) which is, at the end of the day, most scientists believe X? Doesn't this make out scientists to be somehow superior (http://rationalwiki.org/wiki/Elitism) to the rest of the population?

There are two significant differences:

(1) Scientific consensus doesn't claim to be true, it claims to be our best understanding currently held by those who study the matter. Scientific claims for truth are always tentative rather than final (http://rationalwiki.org/wiki/Falsifiability), even if they are often very impressive tentative claims for truth.

(2) Scientific consensus is built upon a foundation of logic (http://rationalwiki.org/wiki/Logic) and systematic evidence (http://rationalwiki.org/wiki/Evidence) - the scientific method (http://rationalwiki.org/wiki/Scientific_method) - rather than popular prejudice. The consensus comes not from blindly agreeing with those in authority (http://rationalwiki.org/wiki/Argument_from_authority), but from having their claims to be thoroughly reviewed and criticised (http://rationalwiki.org/wiki/Peer_review) by their peers. (Note that even long-established scientific consensus can be overthrown by better logic and better evidence typically preceded by anomalous research findings.)

http://rationalwiki.org/wiki/Argumentum_ad_populum#Legitimate_use

3. Rationalwiki, "Internet Laws - Shank's Law"

The rationalist Scott Alexander coined the term "Lizardman's Constant" to refer to the small percentage of people who will give crazy responses in polls for a variety of reasons, e.g. saying the president is a lizardman. Rationalwiki notes that we can also find a similar percentage of academics who occasionally endorse irrational beliefs like creationism, ancient astronauts, or various conspiracy theories.

The realization that there is a small percentage of people with advanced academic degrees that can be found to endorse almost any irrational belief is known as "Shank's Law": "The imaginative powers of the human mind have yet to rise to the challenge of concocting a conspiracy theory so batshit insane that one cannot find at least one PhD-holding scientist to support it." Shank's Law is a good argument for favoring expert consensus rather than just allowing people to cherrypick the experts that agree with their pet position.

http://rationalwiki.org/wiki/Internet_law#Shank.27s_Law

DIFFERENTIATING SKEPTICISM FROM DENIALISM OR CYNICISM:

4. Rationalwiki, "Denialism: Denialism vs. Skepticism"

RW clarifies the different between denialism & skepticism:

While both have a negative or critical tone, the positions are different in how they view and acquire and interpret data. Skepticism is a method while denialism is a position. The opposite of "skeptic" is not "believer (http://rationalwiki.org/wiki/Belief)," and it is possible to embrace something while remaining skeptical. This is an essential part of the ethos of science (http://rationalwiki.org/wiki/Philosophy_of_science) as it suggests new experiments (http://rationalwiki.org/wiki/Experiment) to strengthen or falsify a proposition. Skeptics look at experiments to ensure that they were performed properly with the appropriate controls, proper data analysis and so on. The skeptical method involves examining all data and coming to a conclusion that it produces.

Denialists, on the other hand, view data slightly differently, as a means to a predetermined end – minimizing its importance if it goes against their opinion, highlighting it if it supports them, or just plain misrepresenting it for their own purposes. Skeptics keep an open mind (http://rationalwiki.org/wiki/Open_mind) until data shows that a hypothesis (http://rationalwiki.org/wiki/Hypothesis) is invalid, while denialists start with the conclusion and look for support (http://rationalwiki.org/wiki/Texas_sharpshooter_fallacy). To put it another way, denialism embraces confirmation bias (http://rationalwiki.org/wiki/Confirmation_bias) while skepticism seeks to avoid it.

http://rationalwiki.org/wiki/Denialism#Denialism_vs._skepticism

5. Kyle Hill, "Skepticism: I Don't Think It Means What You Think It Means"

Hill argues there's a different between scientific skepticism & the type of media-savvy cynicism that's become common in the age of internet hoaxes & corporate PR spin doctors. Hills explains that cynicism is a general distrust of other’s motivations, a jaded worldview. Cynicism works for describing media and social engagement, but not for why scientists and skeptics tend to dismiss things like UFO sightings. Hill says that cynicism is akin to a refusal to believe anything extraordinary whereas skepticism is a refusal to believe anything without sufficient evidence -- in the case of extraordinary claims, that requires extraordinary evidence (as Carl Sagan put it).

http://archive.randi.org/site/index.php/swift-blog/2101-skepticism-i-dont-think-it-means-what-you-think-it-means.html

ARGUMENTS FOR RESPECTING EXPERT CONSENSUS & DOUBTING EXPERT CONSPIRACIES:

6. Daniel Loxton, "What, If Anything, Can Skeptics Say About Science?"

Loxton, a skeptic who blogs over at Skepticblog, has a very similar article to Chris Hallquist's post on Less Wrong, and suggests a similar tier of responses by amateur in light of expert consensus:

• Where both scientific domain expertise and expert consensus exist, skeptics are (at best) straight science journalists. We can report the consensus, communicate findings in their proper context — and that’s it.

• Where scientific domain expertise exists, but not consensus, we can report that a controversy exists — but we cannot resolve it.

• Where scientific domain expertise and consensus exist, but also a denier movement or pseudoscientific fringe, skeptics can finally roll up their sleeves and get to work. This is where skeptics can sometimes do even better than scientists, due to their familiarity with popular science rhetoric (i.e. shorter "inferential distance" from the audience) and experience with debunking pseudoscience.

• Where a paranormal or pseudoscientific topic has enthusiasts but no legitimate experts, skeptics may perform original research, advance new theories, and publish in the skeptical press. In this shadowy, fringe realm, skeptics can indeed critique working scientists. There is no mainstream of consensus science on, say, ghosts; skeptics are the relevant domain experts.

http://www.skepticblog.org/2009/12/22/what-if-anything-can-skeptics-say-about-science/

*For a good follow-up, check out Daniel Loxton's "Due Diligence: Never Say Anything That Isn't Correct". The title gives you the general gist, but he lays out a specific series of questions skeptics should ask themselves before making public pronouncement on scientific matters.

http://www.skepticblog.org/2010/02/16/due-diligence/

7. Michael Shermer, "Consilience and Consensus"

Shermer explains the value of expert consensus, and that it's strongest when we see "consilience" - i.e. a convergence of evidence from various scientific fields that all points to the same phenomenon. He explains that consilience is more important than a poll that shows the majority of experts favor a certain view, because when we see convergence of evidence from multiple lines of inquiry that all converge to a singular conclusion it provides a very high probability that the conclusion is correct (unless we assume a secret mass conspiracy, which is a low probability explanation - see Article #8 below).

Shermer explains that "AGW doubters point to the occasional anomaly in a particular data set, as if one incongruity gainsays all the other lines of evidence. But that is not how consilience science works. For AGW skeptics to overturn the consensus, they would need to find flaws with all the lines of supportive evidence and show a consistent convergence of evidence toward a different theory that explains the data. (Creationists have the same problem overturning evolutionary theory.) This they have not done."

http://www.michaelshermer.com/2015/12/consilience-and-consensus/

8. Natalie Shoemaker, "What's The Probability that the Moon Landing Was All a Hoax? One Man Has Done the Math"

Occasionally, denialists will claim the scientific consensus is an elite conspiracy. The mathematician David Robert Grimes has shown why conspiracies are increasingly unlikely the more people that are involved and the more time that elapses. This is based off the multiplayer version of the Iterated Prisoner's Dilemma, which shows that the chance of defection increases logarithmically over time: "When the amount of people involved in a conspiracy increases, the rate at which the conspiracy will be revealed increases (whether by some sort of whistleblower or by an accidental slip of the tongue). For a conspiracy of even only a few thousand actors, intrinsic failure would arise within decades... For hundreds of thousands, such failure would be assured within less than half a decade.”

http://bigthink.com/natalie-shoemaker/whats-the-probability-that-the-moon-landing-was-all-a-hoax-one-man-has-done-the-math

CAN YOU CAN SAFELY IGNORE THE "EXPERT CONSENSUS" IN SOME FIELDS? THE COURTIER'S REPLY & THE DEMARCATION PROBLEM

9. Rationalwiki, "Courtier's Reply"

The Courtier's Reply is a term popularized by biologist/blogger PZ Myers (http://rationalwiki.org/wiki/PZ_Myers) to describe an informal logical fallacy (http://rationalwiki.org/wiki/Logical_fallacy) that boils down to: "But you haven't read enough on it!" His answer to the fallacy is to say that telling a non-believer that he should study theology (http://rationalwiki.org/wiki/Theology) before he can properly discuss whether a god exists is like scolding the child in the fable "The Emperor's New Clothes" who cries out that the Emperor is naked. It's as if a courtier of the Emperor argued that that anyone claiming the Emperor's naked must first study a library full of texts that describe the intricate details of the Emperor's clothes. Essentially, it's a particularly ham-handed argument from authority (http://rationalwiki.org/wiki/Argument_from_authority) where the position's proponent attempts to bury the opponent under a pile of detail which is largely irrelevant to the opponent's argument.

RW notes that denunciation of this particular fallacy is quite easy to misuse. Whenever one is told to read more about a subject that he disagrees on, it is easy to accuse one's contradictors of giving a "Courtier's Reply". The element of the Courtier's Reply that is being forgotten here is that it asks the questioner to "read more" about a subject that begs the question (http://rationalwiki.org/wiki/Begs_the_question). Therefore it cannot be used, for instance, by people not liking that they are being asked to read more about global warming (http://rationalwiki.org/wiki/Global_warming) if they deny it. In addition, it is not fallacious to tell, for example, a creationist (http://rationalwiki.org/wiki/Creationism) to read more on evolution (http://rationalwiki.org/wiki/Evolution), if they clearly do not understand what they are talking about, and are basing their evidence on false premises (such as, for example, believing that abiogenesis (http://rationalwiki.org/wiki/Abiogenesis) = evolution).

http://rationalwiki.org/wiki/Courtier%27s_Reply

• For a fairly detailed critique of PZ Myers' use of the "Courtier's Reply" to shut down Christians who tell him he needs to read & understand the major tracts in theology, check out Scott Alexander's essay, "The Courtier's Reply and the Myers Shuffle". Scott makes the point that

http://squid314.livejournal.com/324594.html

10. Rationalwiki, "Demarcation Problem"

So if the Courtier's Reply should only be directed at non-science & pseudo-science, how do we clearly distinguish it from real science? That's called the "demarcation problem". This is one of the central topics of the philosophy of science (http://rationalwiki.org/wiki/Philosophy_of_science), and it has never been fully resolved. In general, though, a hypothesis must be falsifiable (http://rationalwiki.org/wiki/Falsifiability), parsimonious (http://rationalwiki.org/wiki/Occam%27s_Razor), consistent, and reproducible (http://rationalwiki.org/wiki/Reproducibility) to be scientific.

http://rationalwiki.org/wiki/Demarcation_problem

• For a more in-depth treatment of the demarcation problems within skepticism, check out Massimo Pigliucci & Maarten Boudry's article, "Philosophy of Pseudoscience: reconsidering the demarcation problem".

http://rationallyspeaking.blogspot.com/2013/08/philosophy-of-pseudoscience.html

FOUR DEBATES OVER WHETHER SKEPTICS SHOULD EVER DISPUTE THE EXPERT CONSENSUS IN A FIELD THAT'S EMPIRICAL:

11. Julia Galef, "Should Non-Experts Shut Up? A Skeptic's Catch-22"

Julia Galef, the founder of the Center For Applied Rationality (CFAR) and host of the "Rationally Speaking" podcast, thinks intelligent laypeople trained in logic can occasionally spot problems with the expert consensus, especially in the social sciences. The Catch-22 situation Julia mentions is as follows: "The only people who are qualified to evaluate the validity of a complex field are the ones who have studied that field in depth - in other words, experts. Yet the experts are also the people who have the strongest incentives not to reject the foundational assumptions of the field, and the ones who have self-selected for believing those assumptions."

Julia suggests 2 possible ways around this Catch-22: (1) find people who are experts outside of the field in question but are experts in the particular methodology used by that field and see how they judge whether that field is applying the methodology soundly (e.g. have statisticians judge stat-based sociology research), and (2) see if the field make testable predictions and has a good track record of correct predictions.

Julia has a interesting debate with Massimo Pigliucci, a philosophy professor at CUNY, and several others in the comments section. Julia thinks there may be a way for non-experts to judge whether the foundations of an academic field are solid, whereas Massimo doubts that non-experts can ever judge the consensus without spending several years mastering the basics of the field.

http://rationallyspeaking.blogspot.com/2010/07/should-non-experts-shut-up-skeptics_14.html

• NOTE: For a look at a more extended debate between Julia Galef & Massimo Pigliucci on this issue, check out Episode #16 of the Rationally Speaking podcast entitled "Deferring to Experts".

http://rationallyspeakingpodcast.org/show/rs16-deferring-to-experts.html

-----------------------------------------------------------------------

12. Neuroskeptic, "I Just Don't Believe Those Results"

Neuroskeptic (NS) is an anonymous neuroscientist who blogs at Discover Magazine and is often critical of the finding in his field. NS points out that he occasionally judges the validity of studies by their results and asks if his skepticism about this result justifiable: "If I’m free to decide that a result is just unbelievable, how am I any different from (say) a creationist who maintains that it’s just too incredible that natural selection produced humans and other life? To put it another way, how can I call myself a scientist if I sometimes reject scientific evidence that conflicts with my intuitions?".

He notes that disbelieving strange results can be justified in terms of Bayesian probability: "We would say that my 'prior probability' of a [a particular result] is very low. If my prior is low, it is perfectly rational for me to remain unconvinced after seeing one study in favor of [a highly unexpected result] - it might take ten such studies to convince me."

http://blogs.discovermagazine.com/neuroskeptic/2016/08/07/i-just-dont-believe-those-results/#.WBuWnYWcGUk

13. Neuroskeptic, "Is Science Broken? Let's Ask Karl Popper"

NS claims that in many fields of science today, the system by which scientists publish their results is broken and this creates perverse incentives that lead us from the ideal of perfectly objective science. He mentions 3 problems: (1) Journals often don't publish negative results that disconfirm a hypothesis;

(2) Instead of rejecting inadequate hypotheses, p-hacking, selective reporting, and publication bias mean that scientists can unconsciously ignore contrary evidence, or allow it morph into positive conclusions;

(3) hypotheses should make predictions, and then the evidence should be collected to test those predictions, but in science today, hypotheses are often formed after the results of the experiment are known but before publication - this is "post-hoc storytelling".

http://blogs.discovermagazine.com/neuroskeptic/2015/03/15/is-science-broken-lets-ask-karl-popper/

• NOTE: For Neuroskeptic's proposed solution to the above problems in science, see his post "Fixing Science - Systems and Politics".

http://blogs.discovermagazine.com/neuroskeptic/2012/04/14/fixing-science-systems-and-politics/#.WCDXkIWcGUl

14. Patrick Watson, "Neuroskeptic: Science Hipster"

Patrick Watson compares Neuroskeptic to a hipster who's always interested in the cutting edge and has such discerning taste that even what most of us would consider good still isn't good enough. He says he respects NS's skepticism on neuroscience, but points out that NS is a "trustworthy expert selling himself as an outsider. Like all experts, he sells himself as having special access to the truth. He doesn’t, but we should still trust his ideas (mostly), because he has proved reliable in the past, and has done a good job of explaining his reasoning. If he has concerns about peer-review and about science journalism we should listen. His predictions aren’t good because they’re doubtful. They’re good because his rat-smeller is finely honed."

However, Watson doesn't think this type of skepticism can be respected when it comes from people without professional credentials like anti-vaxxers, climate-change deniers, and creationists - "These groups need less critical thinking, and more credulity".

https://medium.com/@patrickdkwatson/neuroskeptic-science-hipster-50a9ff1c1dca#.8ms4ejgu8

----------------------------------------------------------------------

15. Richard Carrier, "On Evaluating Arguments from Consensus"

Richard Carrier is a classical historian that argues that Jesus Christ was a myth rather than a historical figure, which is a contrarian position within the field of biblical history. Carrier argues that intelligent, rational laypeople trained in logic can evaluate the value of both an individual expert's opinion as well as any expert consensus. He notes that both laypeople & experts "need to make these evaluations without themselves having to re-do all the research and study that that consensus is based on, as otherwise we would be demanding an absurd scale of inefficiency in the expert group, by nixing the ability to divide their labor, and instead requiring every expert to reproduce all the work of every other expert, a patent impossibility."

Carriers argues that "a consensus has zero argumentative value when the individual scholars comprising that consensus have neither (a) examined the strongest case against that consensus nor (b) examined enough of it to be able to identify and articulate significant errors of fact or logic in it. So it is fallacious (indeed, a conspicuously unreliable practice) to just cite the consensus on anything, without first ascertaining whose opinions within that consensus actually count... The second cull comes from eliminating from the pool of experts to count, those who articulate their reasons for their conclusion and those reasons are self-evidently illogical (you can directly observe their conclusion is arrived at by a fallacious step of reasoning) or false (you can reliably confirm that a statement of fact they made is false)." Carrier says that this type of evaluation can help determine which experts should be counted when trying to ascertain the consensus, as well as the relative strength (or weakness) of the expert consensus.

http://www.richardcarrier.info/archives/5553

16. Aaron Adair, "Critical Thinking and Expert Consensus"

Aaron Adair criticizes Richard Carrier & others who think they can evaluate the expert consensus outside their own field. He points out the examples Carrier gives of classical historicans making fallacious arguments are valid, but "if you are not a historian, let alone a classicist, how would you know to even evaluate this argument? Or even prior to that, why would you think it was suspect? It’s only when you have the background knowledge that you realize how bad the position is. Thus even a critical thinker would read past [most logical errors] without any red flags popping up. On the other hand, if you are suspicious of every statement by [a given expert], then you will have to effectively redo his research, and in that process you will need to become an expert, the very thing you hoped to avoid in order to save time."

Adair raises several other problems with non-experts trying to evaluate experts:

(1) "You can’t just look it up. First, you have to know what to look up. What would be a relevant fact, what would not? You may not even realize what are the sorts of things you need to know in order to evaluate a claim." Adair references a concept from constructivist educational theory, the "zone of proximal development" (ZPD), which is the difference between what a learner can do with and without help - i.e. if a question is within a person's ZPD, it essentially means they can only grasp it with the help of someone more qualified, and that they'll probably go astray if they try to handle it alone.

https://en.wikipedia.org/wiki/Zone_of_proximal_development

(NOTE: Adair doesn't mention this, but a similar concept to ZPD is "inferential distance" - the gap between the background knowledge and epistemology of a person trying to explain an idea, and the background knowledge and epistemology of the person trying to understand it. If the inferential gap between the experts who normally handle a topic and the interested layperson is too great, it means there's too many layers of knowledge for the layperson to traverse by themselves - https://wiki.lesswrong.com/wiki/Inferential_distance )

(2) "You also need to know if your source is reliable or not. Unless you have background knowledge about what are good sources for your subject of interest, you will have a hard time. If you are ignorant, you can’t really tell the difference between bad and good sources; you don’t have the background knowledge to say something seems fishy."

(3) "You also have the problem of finite working memory: you can only have a few ideas in your conscious memory at any given moment. If you are looking up fact after fact you will forget things. You will also have a very hard time remembering and evaluating facts and arguments if it’s all new and sudden."

(4) "Given the research from political science... more information can polarize people. Worse still, the more educated/knowledgeable/rational people become the most polarized by the same data."

https://gilgamesh42.wordpress.com/2014/07/05/critical-thinking-and-expert-consensus/

• For a more in-depth look at Richard Carrier's arguments for allowing informed, rational laymen to scrutinize the expert consensus, check out the "Inspiring Doubt" podcast where Greg Breahe interviews Carrier:

https://www.youtube.com/watch?v=upi9W__FoCc

---------------------------------------------------------------------

17. John Horgan, "Dear 'Skeptics,' Bash Homeopathy and Bigfoot Less, Mammograms and War More. A science journalist takes a skeptical look at capital-S Skepticism"

John Horgan is a science journalist that often criticizes certain trends within the scientific establishment. Horgan takes the skeptic movement to task for going after "soft targets" for debunking like Bigfoot and homeopathy but ignoring "hard targets" where mainstream science has become corrupted by corporate funding & vested interests. He says skeptics should address issues like over-testing & over-medication in the medical industry, and the way the "deep roots theory of war" (i.e. war as a natural product of human tribal aggression) enables the military-industrial complex. Horgan attributes this lack of skeptics criticizing bad tendencies in mainstream science to what he calls the "Science Delusion," i.e. blind faith in the goodness & inerrancy of the scientific establishment.

https://blogs.scientificamerican.com/cross-check/dear-skeptics-bash-homeopathy-and-bigfoot-less-mammograms-and-war-more/

18. David Gorski, "John Horgan is 'skeptical of skeptics' or: Homeopathy and Bigfoot versus the Quest for World Peace"

David Gorski starts by noting that Horgan starts by assuming skeptics are highly tribal & engaged in groupthink, but doesn't realize there have been fierce debates within the skeptic movement over diverse issues like AGW, social justice & the science of morality, and this has included criticism of skeptic celebrities like Richard Dawkins, Lawrence Krauss & Michael Shermer.

Gorski points out that Hogan seems identify things other people care about as "soft targets" while calling what he cares about "hard targets". He cites Daniel Loxton who argues skeptics should focus not on areas that they merely care about but rather on issues in which they have some level of expertise.

Gorski also points out that Horgan engages in the "fallacy of relative privation" by minimizing some common concerns of skeptics in light of the problem of war. Gorski points out "there is always something more important" but that doesn't mean what skeptics are doing now is worthless.

http://scienceblogs.com/insolence/2016/05/18/john-horgan-is-skeptical-of-skeptics-or-homeopathy-and-bigfoot-versus-the-quest-for-world-peace/