addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

Share a nice 20-min video about user testing

From: Yingying
Sent on: Tuesday, April 24, 2012 12:19 PM
Hi guys,

I saw a very nice 20-min video about user testing and would like to share with you :
Title: Practical Guide to Controlled Experiments on the Web: Listen to Your Customers not to the HiPPO

I love the summary notes for user testing too - you will see this in the end of the video, but it's great that you can view the details of each point in the video.
  1. Listen to customers because our intuition at assessing new ideas is poor.
  2. Figure out the optimal solution based on data
  3. Compute the statistics carefully
  4. Experiment Often (triple your experiment rate and you triple your success rate & failure rate. Fail fast & often in order to succeed.)
  5. Create a trustworthy system to accelerate innovation

A/B testing
  • Novice experimenters run 1% experiments
  • To detect an effect, you need to expose a certain number of users to the treatment (based on power calculations)
  • Fastest way to achieve that exposure is to run equal-probability variants (e.g. 50/50% for A/B)
  • But don't start an experiment at 50/50% from the beginning: that's too much risk. Ramp-up over a short period



Yingying Zhang

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy