Mining Member Feedback to Improve the Customer Experience- Nishant Hegde Netflix

Abstract:-

At Netflix we strive to go beyond the user’s expectations for a streaming experience. A key component to achieving this is ensuring that the best quality of digital assets is made available. These assets are video, audio, subtitle and closed captioning files that collectively contribute towards the viewing experience. Having a rich catalog with the freshest content becomes inconsequential if a user experiences issues like the timing of the audio and video being off, or if the subtitles are positioned poorly. Moreover, asset quality can have a direct effect on member satisfaction and ultimately retention.

Netflix sets a high bar on content quality, and has a thorough Quality Control (QC) process in place to ensure that this bar is met. Our recent global launch has necessitated having a broad catalog with a wide variety of audio and subtitle languages across countries. In order to retain a lean operation as we scale, we based our QC process on a supervised model that predicts the likelihood of an asset having an issue. We then perform a QC only on assets that are predicted with a likelihood beyond a threshold.

Since we intelligently select what to QC and do not check every asset that goes on our service, there may be instances when a bad asset slips through. We needed a facility on the back end that could catch these undetected issues. Our member feedback channels were an obvious area to tap. Member feedback comes in two forms: explicit and implicit. Explicit feedback is received from sources like the “Report a Problem” section on the site, social media (Twitter, Facebook) and customer service calls. Implicit feedback can be derived from user viewing behavior, such as sharp drop-offs at certain points during the playback.

This talk will focus on

• How we mine explicit member feedback, in particular from the “Report a Problem” section on our site

• The challenges posed with identifying what is relevant due to the variety of context in the feedback obtained

• The analysis framework to monitor feedback and manage workflow

• Areas for improvement / future work

Bio:-

Nishant Hegde is a Senior Analytics Engineer at Netflix. He focuses on data engineering, analysis and data visualization in the Digital Supply Chain Analytics team. Nishant was previously an analyst and managed teams in the Forensic Analytics practices at Deloitte and Price Waterhouse Coopers.

Parking & Other Info

• Enter from Hannum Ave into the parking structure for the 200-300 buildings • Once you park, exit the parking structure on P4 near the elevators.

• Head across the courtyard veering slightly to the right and you will see the 200 building. If you find the 300 or 100 buildings you are in the wrong place. Take elevator to second floor

• Parking is free in the lot, after 6:30 PM the gates are open so people can just leave

Sign Up Link for Brief Interview

Datascience is interested in doing a five minute video interview (more like 2 minutes) for our Voice of the Data Scientist series.

The link to sign up is here: bit.ly/voice_of_data

Participants get free t-shirt and chance to win $100.