How to Test & Roll

Marketers often use A/B testing as a tool to compare marketing treatments in a test stage and then deploy the better-performing treatment to the remainder of the consumer population. These tests have traditionally been analyzed using hypothesis testing, which Elea covered in a previous R Ladies workshop. In this workshop, Elea and Ron will explain a new approach to A/B testing that they developed called "Test & Roll" that focuses on the profit earned during and after the test. It is based on a Bayesian decision theory and the (very technical) Test & Roll paper explains why marketers should use this approach. This workshop will focus on how to Test & Roll using hands-on examples in R. Elea and Ron will cover:

• When you should use "Test & Roll" versus traditional hypothesis test
• How to compute the Test & Roll sample size that maximizes profits
• How to estimate priors from your own data to help plan your Test & Roll
• How to analyze a Test & Roll experiment (It's easy!)

All you need to participate are basic R skills; we'll explain the rest. All code and data will be available via Github. Participants should install R, the rstan package and RStudio prior to the workshop.

Speakers:

• Elea McDonnell Feit is an Assistant Professor of Marketing at the LeBow College of Business at Drexel University. She develops data analysis methods to inform marketing decisions including designing new products as well as planning advertising campaigns.

• Ron Berman is an Assistant Professor of Marketing at the Wharton School at University of Pennsylvania. He focuses his research on online marketing, marketing analytics and the marketing actions of startup firms.

Getting there:
This event will be virtual and accessed via Zoom.