A/B Testing in the Wild

This month, Emily Robinson (aka ERobs), tackles A/B Testing and its related challenges.

About the Talk:

A/B Testing has become the gold standard for measuring the effectiveness of changes to a company’s website or app, ranging from a change in the color of a button to a whole new advertising model. It’s used extensively at large technology companies such as Amazon, Facebook, and Google, as well as at small startups and non-tech companies.

For companies with a lot of traffic and a data engineering pipeline already set-up, A/B Testing can appear to be exceedingly simple. But whether it’s trying to sequence experiments, work with non-technical partners, or deciding what to do when there is a bug, there are a number of issues that can complicate an analysis. We’ll cover some of these business and statistical challenges, using some recent experiments at Etsy as examples.

About Emily:

Emily is a Data Analyst at Etsy (https://www.etsy.com/) where she works with the search team to design, implement, and analyze experiments on the ranking algorithm, UI changes, and new features. Emily earned her masters in Organizational Behavior from INSEAD (https://www.insead.edu/) in 2016 and her bachelor’s in Decision Sciences from Rice University (http://www.rice.edu/) (where she took classes from Hadley Wickham). Follow her on twitter at @robinson_es (http://twitter.com/robinson_es).

Pizza (http://bit.ly/pizzapoll) begins at 6:30, the talk starts at 7, then after we head to the local bar.

https://secure.meetupstatic.com/photos/event/a/8/0/0/600_462943008.jpeg

Members of the meetup receive a 20% discount with code nyhackr (https://www.eventbrite.com/e/learn-bayes-mcmc-and-stan-2017-with-andrew-gelman-jonah-gabry-michael-betancourt-tickets-36284546054?discount=nyhackr) to the Stan Master Class (https://www.eventbrite.com/e/learn-bayes-mcmc-and-stan-2017-with-andrew-gelman-jonah-gabry-michael-betancourt-tickets-36284546054) featuring Andrew Gelman (http://andrewgelman.com/) and Stan developers Jonah Gabry (http://jgabry.github.io/about/) and Michael Betancourt (https://betanalpha.github.io/).