PyData Zurich: ML Inference and Deployment

After a long time of hibernation, the PyData Zurich meetup is finally back! We would like to rekindle the community and provide regular meetups with talks revolving around all things data in the Python ecosystem. This will be our second edition.

Schedule:
6:00 Entry
6:15 Start
6:20 Talk #1
6:50 Q&A
7:00 Talk #2
7:30 Q&A
7:40 End

Talk #1: Train in Python, deploy anywhere and anyhow you like! - Christian Bourjau
Why is it that even small models are often much more difficult to deploy than "traditional software"? We will find that the process of packaging the state of a pipeline after the training phase is one of the main culprits. We will find that these problems can be solved with a standardized file format for trained models - ONNX. However, it turns out that ONNX is able to solve many more problems than deployment headaches!

In this talk I will introduce ONNX, which we use successfully at QuantCo since two years, to make our deployment orders of magnitude more efficient, more stable, and easier all at the same time. In the course of our journey with this technology we have invested heavily into Open Source tooling to make it more approachable. After this talk you'll know the capabilities of ONNX, what it is really good at, what its current shortcomings are, and how to get started on using it in your project.

Talk #2: Modern End-to-end Open Source Machine Learning - Omar Sanseviro
In this talk, Omar will talk about how to run inference and training of state-of-the-art Open Source Large Language Models. Omar will talk about technologies such as text generation inference for serving and bitsandbytes for 4-bit inference. Then, the talk will be about training models with parameter-efficient fine-tuning techniques (such as LoRA and QLoRA) and how they've been used for models such as Whisper, Stable Diffusion, and Falcon.