HKPUG #100 - Trace the Clue, Find What’s True: An Opik LLM Debugging Night

Date: 11 July 2026 (Sat)
Time: 16:00 - 19:30 *HKT
Coordinator: Alex Au

Light Refreshments & Pizza Night will be available!

Due to University's Entrance Policy, please register at the University counter (at Exit C) for Entrance QR Code.

LLM apps are easy to demo, but hard to debug.
A chatbot gives a confident but wrong answer.
A RAG pipeline retrieves the wrong context.
An agent calls the right tool with the wrong argument.

An evaluator says “pass”, but everyone in the room knows something is off.
So where did things go wrong?

In this HKPUG meetup, Tarun will introduce Opik and show how traces, spans, evaluations, datasets, and feedback loops can help developers debug LLM applications more systematically.

After the talk, we will run a short mini workshop. Participants will inspect simplified Opik-style traces, discuss the clues, find the failing span, and decide what evaluation should be added to prevent the same issue from coming back.

Finally, we will launch a 30-day Opik prompt experiment challenge:

• 100 Questions.
• 30 Days.
• Beat the Baseline.

Participants will receive a context pack, a starter Python LLM workflow, a baseline prompt, and a 100-question evaluation set. The goal is to use Opik to run experiments, inspect traces, improve prompts, compare results, and submit better final answers.

This is not a model fine-tuning competition.

This is an evidence-driven prompt improvement challenge.

Speaker

Tarun Jain

Tarun is a Founding Engineer, and Content Creator at AI with Tarun. He is also a Google Developer Expert AI. He was a maintainer to open source AI projects like OpenAGI and BeyondLLM and has spoken at international conferences. Tarun enjoys building intelligent systems, knowledge graphs, and long term memory solutions for AI agents.

Linkedin: https://www.linkedin.com/in/jaintarun75/

What you will learn

• How to debug LLM apps beyond simply saying “the model is bad”
• How Python LLM workflows can be traced and inspected with Opik
• How traces and spans reveal what happened inside a RAG or agent workflow
• How prompt versions can be compared using experiments and evaluation results Common failure modes: bad retrieval, weak prompts, wrong tool calls, hallucination, evaluator bugs, and privacy/safety issues
• How to turn bad answers into useful evaluation cases
• How to use Opik to improve prompts and beat a baseline over a 100-question challenge
• How the mini workshop connects to the upcoming 30-day Kaggle-style challenge

Format
This meetup is part talk, part debugging game.
Tarun will first introduce Opik and the core ideas behind LLM observability and evaluation.
Then we will work through a few “broken bot” cases together. Each group will inspect the trace, identify the suspicious span, and explain what they think went wrong.
No deep machine learning background is required. Curiosity and debugging instinct are enough.

Mini Workshop: Trace the Clue
Each group will receive a few simplified LLM application traces.
For each trace, we will discuss:

• What went wrong?
• Which span looks suspicious?
• Is it a retrieval issue, prompt issue, tool-call issue, hallucination, evaluator issue, or privacy/safety issue?
• What evaluation or metric should be added?
• Should this app pass or fail the release gate?

The bad answer is the symptom.
The trace is the evidence.
Your job is to find what’s true.

Capacity: 100

Venue Info:
City University of Hong Kong, Kowloon Tong (Exact Location TBC)

Rundown:

4:00–4:20 pm — Arrival / registration / casual networking
4:20–4:30 pm — HKPUG intro: tonight’s debugging case file
4:30–5:20 pm — Talk: Debugging LLM Apps with Opik
5:20–5:35 pm — Q&A
5:35–5:50 pm — Short break
5:50–6:30 pm — Mini workshop: trace the clue, find the failing span
6:30–6:55 pm — Group discussion and case walkthrough
6:55–7:15 pm — One-month Kaggle-style challenge launch
7:15 pm onwards — Pizza, networking, and team forming

Audience pre-requisite:

• Skills: Recommended having basic level knowledge of Python

How to join?

1. Click "Attend" on this page