Build & Learn: From Data Science to AI Engineering Week 6

📅 Week 6: Monitor and evaluate your LLM application

This event is part of our seven-week AI Engineering program, where we build an end-to-end AI application step by step and present our finished projects during Demo Day in Week 7.

This week, we’ll focus on understanding how our LLM application behaves after deployment and how to measure whether its outputs are actually useful.

• Monitor and debug LLM calls with Langfuse tracing.
• Manage and version prompts with Langfuse Prompt Management.
• Introduce the main concepts behind LLM evaluation.
• Write focused unit tests for predictable application behavior.
• Build evaluation datasets from representative examples.
• Add human annotations and feedback using Langfuse scores.
• Build an LLM-as-a-judge evaluator

By the end of the 7 week cycle, you should have a working AI application in your own GitHub repository and practical experience with AI pipelines, APIs, databases, retrieval, evaluation and deployment.
You are welcome to follow the shared project or apply the weekly concepts to your own project.

✨ Who is this for?
This cycle is designed for data scientists, analysts, software developers and Python users interested in moving toward AI engineering.
No previous LLM or AI engineering experience is required. However, you should already understand basic Python or be willing to complete some preparation independently.

✨ Who’s hosting?
I’m Lindsey, a senior data scientist working on AI systems, causal inference and data products.
I’ve worked on machine learning, uplift modelling, fraud detection and production LLM systems. I care about learning through building—and moving beyond AI hype toward applications that actually work.

💻 Bring: Your laptop, your project and your curiosity.
☕ Please grab a coffee or drink from the Octopus Bar to support the business and thank them for providing the space.

And as usual, we will be hosting our presentation in the first half of our meeting on google meet "

AI engineer session 6
Saturday, September 19 · 10:00 – 11:30am
Time zone: Europe/Berlin
Google Meet joining info
Video call link: https://meet.google.com/kkj-mwvb-whb