BAYES' THEOREM 101: WHAT'S THE BIG DEAL?

https://a248.e.akamai.net/secure.meetupstatic.com/photos/event/6/a/a/600_453001706.jpeg

A basic overview of Bayes (https://www.youtube.com/watch?v=OqmJhPQYRc8)

A visual explanation of Bayes (https://www.youtube.com/watch?v=BrK7X_XlGB8)

Bayes’ Theorem (alternatively called Bayes' law or Bayes' rule), named after its inventor, the 18th-century Presbyterian minister Thomas Bayes, is a method for calculating the relative validity of a given belief (hypotheses, claims, propositions) based on the best available evidence (observations, data, information). It describes the probability of an event occuring, based on conditions that might be related to that event. So powerful, and indeed so useful is it, that its advocates insist that if more of us routinely adopted conscious Bayesian Reasoning when evaluating any claim that's made, the world would be a much better place. The question we therefore need to ask ourselves is this: what are we as Skeptics to make of this particular claim?

Why does a mathematical concept generate such enthusiasm amongst its users? What is the so-called Bayesian Revolution now sweeping through the sciences which claims to subsume even the experimental method itself which is now seen as nothing more a special case of reasoning? What is the secret that the adherents of Bayes know? What is the light that they have seen?

Modern philosophers assert that science as a whole can be viewed as a Bayesian process, and that Bayes' Theorem enables us to distinguish science from pseudoscience more precisely than falsification, the method popularized by Karl Popper, ever could. It's been used to try to clarify the relationship between theory and evidence. As a consequence, Quantum Physicists have proposed Bayesian interpretations of quantum phenomena in order to explain that which they observe as well as Bayesian defences of string and multiverse theories.

The use of Bayesian statistics can be found in everything from physics to cancer research, and ecology to psychology. Artificial-intelligence researchers, including the designers of Google’s self-driving cars, employ Bayesian software to help machines recognize patterns and make decisions regarding how to respond when faced with dangers. Bayesian-based software sorts spam from bona fide e-mail, assesses medical data and homeland security risks, and decodes DNA amongst other things. Our understanding of these phenomena can be made more precise, and sometimes extended or corrected, by using Bayes' Theorem.

Bayes' Theorem provides us with many important insights:

1. One should not focus on tests alone to determine the veracity of a given claim. In other words, tests are not in fact the event we need to focus on. We may have a test available that enables us to determine if we have cancer, but the test itself is separate from the event of actually having cancer.

2. More often than not, tests are flawed. It's extremely rare to find a test that is 100% accurate 100% of the time. Almost all tests have an inherrent error rate (human error notwithstanding) Tests in fact detect things that don’t exist (false positives), and miss things that do exist (false negatives). These factors need to be taken into account when evaluating evidence and it's only if we do so, that we can properly evaluate the veracity of any claim.

3. Tests results only ever give us test probabilities, not the real probabilities of the thing we're testing for being present. People often consider the test results directly, without considering the errors in tests. They are wrong to do so. Ignoring the the later, inevitably leads to erroneous conclusions.

4. False positives skew results. Suppose you are searching for some really rare medical condition (it's occurence being 1 in a 10 million say). Even with a good test avilable, it’s likely that a positive result is really a false positive on somebody within the 9,999,999. Only by properly understanding probability can we accout for this.

5. Even science itself is a test. At a philosophical level, scientific experiments can be considered “potentially flawed tests” and need to be treated accordingly. There is a test for the existence of a chemical, or a particular phenomenon, and there is the event of the phenomenon itself. The two are different. Our tests and measuring equipment have some inherent rate of error. Both must be taken into account when determining how likely it is that your test has in fact found the thing you're searching for.

6. Bayes’ Theorem converts the results from your test into the real probability of the event. For example, you can correct for measurement errors. If you know the real probabilities and the chance of a false positive and false negative occurring, you can correct for measurement errors.

For example:

Suppose a drug test designed to detect heroin usage is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people in the population are users of the drug. If a randomly selected individual tests positive, what is the probability that they are a user?

Start with the simplest form of the theorem in the form of a mathematical equation:

P(A│B)= (P(B│A) P(A))/(P(B)) where A and B are events and P(B) ≠ 0.

• P(A) and P(B) are the probabilities of observing A and B without regard to each other.

• P(A | B), a conditional probability, is the probability of observing event A given that B is true.

• P(B | A) is the probability of observing event B given that A is true.

Or, as in our case:

P(User│+)=(P(+┤| User) P (User))/(P(+┤| User)P(User) + P(+Non-user)P(Non-user)) = (0.99 x 0.005)/(0.99 x 0.005 + 0.01 x 0.995) ≈33.2%

This surprising result arises because the number of non-users is very large compared to the number of users; thus the number of false positives (0.995%) outweighs the number of true positives (0.495%).

To use concrete numbers, if 1000 individuals are tested, there are expected to be 995 non-users and 5 users. From the 995 non-users, 0.01 × 995 ≃ 10 false positives are expected.

From the 5 users, 0.99 × 5 ≃ 5 true positives are expected. Out of 15 positive results, only 5, about 33%, are genuine.

Despite the apparent accuracy of the test, if an individual tests positive, it is more likely that they do not use the drug than that they do. This again illustrates the importance of base rates, and how the formation of policy can be egregiously misguided if base rates are neglected.

Embedded in Bayes’ Theorem is an important message:

If you aren’t scrupulous in seeking alternative explanations for your evidence, the evidence will just confirm what you already believe which may in fact be wrong.

Scientists often fail to heed this dictum, which helps explains why so many scientific claims turn out to be erroneous. Bayesians claim that their methods can help scientists overcome confirmation bias and produce more reliable results, but one should have one's doubts.

And as mentioned above, some string and multiverse enthusiasts are embracing Bayesian analysis. Why? Because the enthusiasts are tired of hearing that string and multiverse theories are unfalsifiable and hence unscientific, and Bayes’ theorem allows them to present their theories in a more favourable light. In this case, Bayes’ Theorem, far from counteracting confirmation bias, enables it.

The problem is that whilst Bayes Theorem is extremely useful, it can also be used to promote superstition and pseudoscience if used unethically. Bayesian statistics can’t save us from bad science. Bayes’ Theorem is a powerful all-purpose tool that can be used to serve any cause. The prominent Bayesian statistician Donald Rubin of Harvard has served as a consultant for tobacco companies facing lawsuits for damages from smoking for example and has used to Bayes's Theorem to help establish in the minds of jurors the veracity of law suits brought against tobacco firms.

As Skeptics, we should be nonetheless fascinated by Bayes’ Theorem. It's reminiscent of the theory of evolution; another idea that seems tautologically simple or dauntingly deep, depending on how you view it, and that has inspired abundant nonsense as well as profound insights.

The purpose then of this month's meeting is, in the first instance, to have opportunity to learn about what exactly Bayes' Theorem is by having it explained to us by one of the country's leading experts, and secondly, to evaluate its utility. Come along and explore some of its uses, the issues surrounding its application and apply it to some practical real-world examples. In doing so, joining in on a discussion about how we might use it ourselves in our every day lives.

ABOUT NORMAN FENTON:

Professor Norman Fenton is Professor of Risk Information Management at Queen Mary London University and is also a Director of Agena, a company that specialises in risk management for critical systems. Norman, who is a mathematician by training, works on quantitative risk assessment. This typically involves analysing and predicting the probabilities of unknown events using Bayesian statistical methods including especially causal, probabilistic models (Bayesian networks). This type of reasoning enables improved assessment by taking account of both statistical data and also expert judgment.

In April 2014 Norman was awarded one of the prestigious European Research Council Advanced Grants (BAYES-KNOWLEDGE) to focus on these issues. Norman's experience in risk assessment covers a wide range of application domains such as legal reasoning (he has been an expert witness in major criminal and civil cases), medical analytics, vehicle reliability, embedded software, transport systems, financial services, and football prediction. Norman has a special interest in raising public awareness of the importance of probability theory and Bayesian reasoning in everyday life (including how to present such reasoning in simple lay terms) and he maintains a website dedicated to this and also a blog focusing on probability and the law.

In March 2015 Norman presented the award-winning BBC documentary Climate Change by Numbers.

A few bits of information regarding the Salvage Cafe:

The cafe is small. The maximum number of people that can be accommodated is 40. This is why we have been forced to set a limit on the number of people who can attend alongside a waiting list.

BSitC is extremely fortunate in that it has been able to secure the use of the Salvage Cafe effectively for free. That is, the cafe owners have not seen fit to levy a fee for letting us use it. However, in order that it be made worth their while to do this, there is an expectation that members attending events hosted by BSitC buy drinks and cakes.

Please note that there is also a £3 charge levied for this event.