[hybrid] Evaluating LLM based applications

🙋‍♀️Join us for an exclusive workshop with Una Galyeva, visiting us all the way from PyLadies Amsterdam!

It is so easy and quick to build a shiny PoC using LLMs and it is so hard to turn it into a production-grade LLM application. To succeed you need a robust evaluation framework, which you are going to use during the development and post-deployment of your LLM based app.

This workshop focuses on understanding evaluation-driven development and architecture of a LLM based app, building an evaluation framework for a LLM based app, establishing a test suite with evals and laying the monitoring foundations for it. All of it by leveraging Python OSS libraries.

With over 19 years of experience in Data and AI, Una Galyeva held various positions, from hands-on Data and AI development to leading Data and AI teams and departments. As a driving force behind PyLadies Amsterdam, a Microsoft MVP, Women in AI Benelux Advisory board member, and the owner of AI MLOps Agency, Una is passionate about challenging perspectives and inspiring others to see things differently.

🤔 Before the session
-- check out the requirements and make sure you have everything installed: https://github.com/pyladiesams/eval-llm-based-apps-jan2025?tab=readme-ov-file#requirements

***
📆 Agenda

• 18h00 Doors open & Community Announcements
• 18h15 Workshop Kickoff
• 20h15 Networking

***

*Snacks and drinks will be provided by our sponsor Moia, we will accommodate folks' needs but if you require something specific you are also welcome to bring along your own food and drinks.

​❓ Can men attend ❓
Everyone is welcome. :)
If you identify as someone well-represented in open source and in tech, please be mindful of the space and privileges you have, and use it to support others.
​
​
🗣️ To include a multitude of perspectives, the spoken language of this meetup will be English.
​
🤝 By attending our event, you agree to the PyLadies Code of Conduct: https://www.pyladies.com/CodeOfConduct/ and the LadyNerds Code of Conduct: http://bit.ly/LadyNerds-CoC

​📧 Contact
Are you interested in speaking at one of our events? Have a good idea for a Meetup? Get in touch with us at [hamburg@pyladies.com](mailto:hamburg@pyladies.com)

​💬 Find us on the PyLadies Global workspace:

1. ​https://slackin.pyladies.com enter your email address.
   Accept the email invitation
2. ​Go to workspace https://pyladies.slack.com
3. ​Join channel #city-hamburg, #germany, #jobs-europe, #city-amsterdam