PyData Prague #33 - The Root of All Eval

Hello Python magicians and A.I. operators!

The 33rd Prague PyData meetup will take place at Pure Storage offices. As usual, the talks will start at 18:30 but we encourage you to come as soon as 18:00 to enjoy the opportunity to socialize and refresh yourselves (which you can continue doing during the break and after the talks). Note that this time we will want you to sign a non-disclosure agreement (NDA) when entering (only to protect the Pure Storage hardware in development; the talks themselves are public of course!)

Our main goal is to build the community around Python and data and make it welcoming to people of various skills and experience levels.

⚡ If you are interested in giving a lightning talk (up to 5 minutes to present an idea, tool or results related at least to some degree to Python and/or data), please contact us before the event or at its beginning.

📢 Evals, Benchmarks, and Guardrails: A Pythonista's Guide to Not Mixing Them Up
(Šimon Podhajský, Waypoint)

"I'll just write pytest tests for my LLM"—but should you? This talk untangles benchmarks, evals, and guardrails: three concepts that sound similar but map to different Python patterns. Learn why pytest CAN work for evals (with the right mindset), why guardrails aren't tests at all, and a grounded theory approach to defining what "good" actually means for your task.

📢 Orchestration Beyond the Schedule: Real-Time Integrations with Prefect
(Ondřej Hlaváč, Pure Storage)

As Python data workflows grow in complexity, relying on simple scripts and cron jobs often leads to "silent failures" and a lack of visibility. Enter Prefect—a modern orchestration framework that empowers developers to build, observe, and manage robust pipelines using standard Python code. While Prefect is rapidly gaining traction for scheduled batch processing, we took a different path: using it to power real-time event integrations for Master Data Management.

In this talk, I will introduce what makes Prefect unique compared to legacy tools and demonstrate how we adapted it to handle immediate, event-driven flows. We will explore how its built-in resilience
capabilities—like automatic retries, state management, and detailed observability—can be repurposed to make real-time integrations as reliable as nightly ETL jobs.