LLMOps: Test-Driven Development for Large Language Model Applications


Details
Thank you to our host Pulze.ai!
Co-founder and CEO Fabian Baier will introduce **Pulze.ai**.
Thank you to our sponsor Airbyte for food, drinks, and recording support!
Sponsor introduction by Michel Tricot, Airbyte CEO.
NOTE: you have to register on Eventbrite to get in!
Josh Tobin (right) is the founder and CEO of Gantry. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning (fullstackdeeplearning.com), the first course focused on the emerging engineering discipline of production machine learning and LLM applications. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel.
Large language models are a powerful primitive for building applications quickly and easily. However, when it comes to robustness, reliability, and production readiness, they leave something to be desired.
If you've built applications with LLMs, you may have wondered, "isn't it a bit generous to call this prompt engineering?", "how do I know if this thing is actually working", or "is it even possible to test these things"?
In this talk, we will present a more principled way to develop LLM applications using an approach that is analogous to test-driven development. We'll also show you how to get started with this approach in minutes using Gantry.
Airbyte is the leading open-source data integration platform that seamlessly syncs data from the largest catalog of APIs, databases, and files to various destinations. Airbyte differentiates itself by its open-source extensibility, deployment options - cloud-hosted or self-managed and transparent and predictable pricing. Airbyte empowers AI-driven organizations with leveraging all their data, whatever the tools they use.

LLMOps: Test-Driven Development for Large Language Model Applications