AI Red Teaming: Hands-on Demo
Details
Think your LLM application is secure and reliable? Let's put it to the test!
Join us for a live hands-on demo and retrospective on AI Red Teaming, where we'll explore how to identify vulnerabilities, stress-test LLM applications, and strengthen their security using open-source tools like Giskard and Garak.
### In this session, you'll learn:
- How to perform red teaming on an LLM application from scratch.
- How to use Giskard and Garak to stress-test LLM applications.
- Why you can't always blindly trust an LLM judge when evaluating model outputs.
- How to harden prompts against role-play exploits and potential vulnerabilities.
- How to automate local regression testing to improve the reliability and security of LLM applications.
- Practical lessons learned and challenges encountered while testing and securing an LLM-powered bot.
This is a practical, hands-on session featuring a live demonstration and real-world lessons learned. You'll gain insights into identifying LLM vulnerabilities, improving prompt security, and building more robust AI applications.
š¹ Format: Live Hands-on Demo + Retrospective + Q&A
š¹ Presenter: Sai Thummala
š¹ Tools Covered: Giskard and Garak
š¹ Who's it for: AI engineers, developers, ML engineers, security enthusiasts, and anyone interested in building secure and reliable LLM applications.
š¹ Takeaway: A practical understanding of LLM red teaming, prompt hardening, vulnerability testing, and automated regression testing using open-source tools.

