Ragas: Evals for RAG & Memory - AI Build & Learn #10
Details
Welcome to AI Build & Learn a weekly AI engineering stream where we pick a new topic and learn by building together.
This event is about evaluating LLM and RAG applications with Ragas, an open-source Python toolkit for objective, automated evals instead of vibes-based testing.
We'll explore Ragas metrics for retrieval and generation quality, automated test data generation, and how to wire evals into a feedback loop so production data drives continuous improvement.
Some things to look up to get started:
- Ragas GitHub: https://github.com/vibrantlabsai/ragas
- Ragas docs: https://docs.ragas.io/
Resources
- GitHub: https://github.com/sagecodes/ai-build-and-learn
- Events Calendar: https://luma.com/ai-builders-and-learners
- Slack (Discuss during the week): https://slack.flyte.org/
- Hosted by Sage Elliott: https://www.linkedin.com/in/sageelliott/
In this stream
- Intro to topic
- Community Discussion
- Practical examples
Community challenge (optional)
Try spending 30–90 minutes during the week learning or building something related to the topic, then share what you’re working on in Slack.
Note on Flyte / Union
You may see Flyte used in some demos. Flyte is an open-source AI orchestration platform maintained by Union (where I work) for building scalable, durable, and observable AI workflows. You do not need to use Flyte to participate.
- Union: https://www.union.ai/
- Flyte: https://flyte.org/
Drop a comment with ideas for future topics (agents, RAG, MLOps, robotics, frameworks, and more).