Data Drift: Uncovering Changes in Your ML Model's Input Data

Welcome back to PyData Edinburgh!

We're delighted to say we will be starting in person events again on the anniversary of our 4th birthday, at one of traditional venues, Wood Mackenzie.

We're really looking forward to welcoming Emeli Dral, CTO and Co-founder of Evidently AI to speak about data drift in machine learning.

As usual, we'll have lightning talks, nibbles and refreshments and plenty of chat!

# MAIN TALK

Data Drift: How to Uncover Changes in Your ML Model's Input Data

Once you put an ML model in production, you need to keep tabs on its performance. If you have ground truth labels or actual values, you can directly evaluate the model quality, such as accuracy or error rates.

But in real-world scenarios, it's only sometimes possible. The labels often come with a delay, and you need to find a way to estimate how well your model is doing using proxies. One way of doing that is by evaluating distribution shifts in the model inputs and predictions.

But how exactly can you calculate it? How big of a drift is important? Which data to compare?

In this talk, I will present different approaches to evaluating data drift as a proxy for model performance decay.

• I will introduce different drift detection methods, including statistical tests, distance metrics, and domain classifier models.
• Show how one can visualize and interpret the results of drift detection analysis.
• Share practical tips on how to incorporate it in the production ML workflow; when the goal is not to select the most statistically precise method but to get an actionable alert on time.

# LIGHTNING TALKS

Share your stories! Share your successes & failures! Share your hints & tips! Whether you are a beginner or an expert, we want to hear from you. If you want to give a lightning talk at this event, send us a message either here on Meetup, through Twitter @PyDataEdinburgh or any other way you know how to find us!
Just 5 minutes, no need for slides unless you want them — go on, give it a go!

# LOGISTICS

Please remember to RSVP to come to this event :)

1815: Doors open, drinks & networking
1845: Talks start - welcome & community announcements followed by our main speaker, then lightning talks.
We'll end the evening with food and more refreshments, wrapping up by 2100.

# SPONSORS

As always, we couldn't do this without our sponsors help to provide a venue and drink & food for this event — Cathcart Associates and Wood Mackenzie.

# CODE OF CONDUCT

The PyData Code of Conduct governs this meetup. To discuss any issues or concerns relating to the code of conduct or the behavior of anyone at a PyData meetup, please contact the local group organizers (message us on the meetup page). Please also submit a report of any potential Code of Conduct violation directly to NumFOCUS using this form: https://numfocus.org/code-of-conduct.

Thank you for helping us to maintain a welcoming and friendly PyData community!