Bayesian inference for A/B testing

Please bring a valid ID in order to access the building.

Agenda:

6:30 – 7 Networking and pizza
7 – 7:05 Christine Hurtubise, Vice President, Head of Data Science, Marketing. Acquisition A/B Testing
7:05 – 7:10 Zhijiao Chen, Data Scientist, Predicting In-App User Conversion with Bayesian Techniques
7:10 – 8:10 Andrew Gelman
7:10 – 8:30: Networking

Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University
Lauren Kennedy, Columbia Population Research Center, Columbia University

Abstract: Suppose we want to use empirical data to compare two or more decisions or treatment options. Classical statistical methods based on statistical significance and p-values break down in the context of incremental improvement: that is, when there is a stream of innovations, each only slightly better (or possibly slightly worse) than what came before. In contrast, a Bayesian approach is ideally suited to decision making under uncertainty. We discuss the implications for applied statistics and code up some of these models in R and Stan, based on a case study by Bob Carpenter.