Evaluating LLM based applications
Details
🙋♀️ Join us for the joint event with PyLadies Hamburg. This event is online for PyLadies Amsterdam members and hybrid for PyLadies Hamburg members, as the in-person part will take place in Hamburg, Germany.
Workshop
It is so easy and quick to build a shiny PoC using LLMs and it is so hard to turn it into a production-grade LLM application. To succeed you need a robust evaluation framework, which you are going to use during the development and post-deployment of your LLM based app.
This workshop focuses on understanding evaluation-driven development and architecture of a LLM based app, building an evaluation framework for a LLM based app, establishing a test suite with evals and laying the monitoring foundations for it. All of it by leveraging Python OSS libraries.
Speaker
Una Galyeva
With over 19 years of experience in Data and AI, Una Galyeva held various positions, from hands-on Data and AI development to leading Data and AI teams and departments. As a driving force behind PyLadies Amsterdam, a Microsoft MVP, Women in AI Benelux Advisory board member, and the owner of AI MLOps Agency, Una is passionate about challenging perspectives and inspiring others to see things differently.
📆 Agenda
18:00 - PyLadies Hamburg and PyLadies Amsterdam greetings
18:15 - Workshop stream start
GitHub Repo
https://github.com/pyladiesams/eval-llm-based-apps-jan2025
Stream
YouTube Stream
📧 Contact
Are you interested in speaking at one of our events? Have a good idea for a Meetup? Get in touch with us at [amsterdam@pyladies.com](mailto:amsterdam@pyladies.com)
💬 Find us on the PyLadies Global workspace:
- https://slackin.pyladies.com enter your email address.
Accept the email invitation - Go to workspace https://pyladies.slack.com
- Join channel #city-amsterdam
Evaluating LLM based applications