AWS Canberra User Group

Join us for our monthly meetup.

Agenda
5:30 Doors Open
6:00 Welcome and announcements
6:15 Panel - AWS Summit Sydney Community Re:Cap
7:00 Break - Food and networking
7:30 Not Everything Needs an LLM, Dave Hall, Principal Consultant, Dave Hall Consulting
8:00 Networking
8:30 Close

AWS Summit Sydney Community Re:Cap
If you couldn't make it to the AWS Summit in Sydney, that's ok. We've got your covered. Our panelists all went. They'll share their experiences. What they liked, what they didn't like, why they think you should go next year. If you have a question they'll be happy to answer it.

Panel Members
Details coming soon...

Not Everyting Needs an LLM
My support tickets kept getting routed to the wrong team, and every misroute added a day to resolution. The obvious 2024 fix was to throw an LLM at it, but I knew that wouldn't work: too much overlap between teams to write a concise prompt, unpredictable accuracy, and a fragile system overall. So I built Gata instead, an open source ticket router for Zendesk. The core routing uses a fine-tuned BERT model trained on your organisation's own ticket history. There are LLMs in the stack, but only for the jobs they're genuinely good at: priority classification and ticket summarisation.

This talk is about how I made that call, and how you can make similar ones. When does a problem actually need an LLM, and when will a fine-tuned classifier do better? What does it take to build something that gets more accurate over time instead of drifting every time you tweak a prompt? You'll leave with a practical framework for matching automation problems to the right tool, plus a concrete case study of what that looks like in production.

Dave Hall
Dave is the Principal Consultant and Founder of Dave Hall Consulting, a Canberra-based technology consultancy with over two decades of experience delivering automation, cloud, and DevOps solutions to clients ranging from early-stage startups to Fortune 50 enterprises.