Machine learning measurement and validation in the real world

Link to register:
https://dataapplab.zoom.us/webinar/register/3015669274034/WN_eCwLBXVnRUaDG8_Z-IXncw

Description:
Data science and machine learning practitioners are waking up to the stark reality that a “99% accurate” AI model isn’t what most of us can expect in the real world. What you build in the lab might not generalize to the field. The data you used in development isn’t accessible to your model in production. The fundamental shape of your ML model drivers changes over time, leading to predictive degradation. And as your model degrades, your stakeholders will ask you to determine why that’s happening, to give them a timeframe for upgrading your model release, and to specify your expectations for avoiding this outcome in the future. (Have fun with that!)

The financial services industry is the domain where probability and risk were coined in the 1700s. (Sorry for the pun. Fully intentional.). In this industry, when data science practitioners make these mistakes, tens to hundreds of millions of dollars are at stake, consumers’ credit records can be affected, and heads roll. For these reasons, the financial services industry (and banking in particular) has put in place repeatable practices to measure model effectiveness and to validate all aspects of these models in ways that is transparent, auditable and explainable.

In this deeply technical and practical discussion, you will learn the practices specific to the best-in-class model performance and validation practices used by financial services practitioners. These practices are completely transferable to virtually every other industry, and the majority (by volume) of actionable business problems, that draw on machine learning approaches. By better understanding how to measure the performance of your models, how to diagnose why they’re working well or poorly, and what you need to do to fix it, you will be able to more effectively set the expectations of your stakeholders, and save yourself a ton of headaches.

Speaker’s Profile:
Robin Way is the founder and President of the decision optimization specialist firm Corios and is a Faculty Member at the International Institute of Analytics. He has over 30 years of experience in the design, development, execution, and improvement of applied analytics models for clients in the banking, insurance and energy industries. Robin wrote the book Skate Where The Puck’s Headed: A Playbook for Scoring Big with Predictive Analytics. He lives in Portland, Oregon with his wife and two sons. In his spare time, Robin plays and coaches soccer and lacrosse, and holds a black belt in taekwondo.