Inference on Kubernetes / Building a Multi-Avatar System with Agentic AI

For the November meetup the first preso will be from Hrittik Roy, a Platform Advocate at vCluster Labs and CNCF Ambassador, “From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?”. The presentation covers GPU scheduling, batching, KV cache and inference observability, using vLLM and NVIDIA Dynamo as examples,

For the second presentation Arsen Yeremen returns to discuss agentic AI development of a multi avatar system.

See the Presentation Details section below for more info.

Sponsor and LA DevOps Meetup:
Sponsoring this or a future meetup is great way to promote your service or product and to give back to the local DevOps Community,'

Reach out to one of the hosts for details.

Event Details:
6:00 - 6:30 Attendees arrive. Networking/Mingle
6:30 - 6:45 LA DevOps Welcome,Attendee Introductions
6:45 - 7:15 Hrittik Roy - From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?
7:15 - 8:00 Arsen Yeremin - Building a Multi-Avatar System with Agentic AI
8:00 - 9:00 Networking/Mingle

Presentation Details:

Title: From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?

Description:

• Running an LLM locally is a great starting point. But what happens when one user becomes hundreds, requests compete for GPU memory, and response times start climbing?
  This talk follows an inference request from prompt to generated tokens, explaining prefill, decode, and the KV cache. We’ll explore how batching, model caching, and GPU scheduling affect latency and throughput, then introduce disaggregated inference: running prefill and decode on separate workers, including when the added complexity makes sense.
  Using vLLM and NVIDIA Dynamo as examples, we’ll connect these concepts to serving models on Kubernetes and measuring what matters: time to first token, token generation speed, and GPU utilization.
  Key takeaways:

Understand what happens inside an LLM inference request.
- Identify bottlenecks as you move beyond local serving.
- Learn when to consider disaggregated inference.
- Understand the Kubernetes building blocks for GPU workloads.
*
About Hrittick:

Hrittik Roy, a Platform Advocate at vCluster Labs and CNCF Ambassador, he helps engineering teams and open source communities get more out of products through demos, documentation, talks, and deep dives into AI Infrastructure, developer workflows, and platform engineering. The goal is always the same: take something complex and make it click.

Title: Building a Multi-Avatar System with Agentic AI: From GPU Cluster to Photoreal Characters
Description:
'
What does it take to get several AI avatars talking, reacting and showing emotion in real time, without handing the whole thing to a closed API?

This talk walks through building a multi-avatar system where each character is driven by its own agent. The system runs on a self-hosted 5-GPU Kubernetes cluster. We’ll cover the full pipeline: building 3D environments from 360° panoramas, rigging photoreal Gaussian Splat characters to skeletons, pulling motion-capture bones from ordinary video, and driving facial expressions from sentiment. The expressions are calibrated against an emotion reference sheet generated by a diffusion model.

We’ll also look at how agents coordinate when several avatars share a scene. That covers turn-taking, keeping each character’s state separate, and splitting work between on-device LLM and TTS and the GPU cluster. Throughout, we’ll look at the DevOps side: scheduling mixed GPU workloads, keeping latency low enough for conversation, and watching a pipeline that has a lot of moving parts. There will be live demos.

Key takeaways:
How to design an agent architecture for multiple interacting avatars

• What runs on the device and what runs on the cluster in a real-time generative pipeline
• Practical patterns for running mixed GPU workloads on self-hosted Kubernetes

About Arsen:
Arsen Yeremin is a CTO and solutions architect with 25+ years of software engineering experience, including 8+ years at the intersection of AI/ML and blockchain. He’s currently leading AI infrastructure and agentic systems at Tygher AI, and has previously served as Fractional CTO of Lilypad Network (decentralized compute for AI) and CTO of Blockfreight (blockchain-based global logistics). Arsen specializes in building production systems that combine autonomous agents, distributed consensus, and enterprise-grade reliability—with hands-on expertise in containerized LLM deployment, on-chain state anchoring, and distributed system design. When he’s not architecting the future of autonomous organizations, he’s exploring how to make decentralized AI genuinely usable for enterprises.

About O'Brien's Pub on Wilshire:
The covered patio at Obrien's Pub has been a great place to host LA DevOps meetups for our post covid revival. The food and drink are great and the staff are very friendly. O'brien's is also the LA home of Manchester United Football Club Supporters.
You may want to bring a jacket for the cooler months.

Regarding parking, there are meters on Wilshire and the side streets that expire at 6.

Present at an Upcoming LA DevOps Meetup:
If you would like to present at an upcoming LA DevOps meetup please send a message in meetup to the organizers with a proposed title and a sentence or two about the talk.

Sponsor DevOpsDayLA at SCALE24x April 2nd 2027
Scale24x runs from April 1 - 4 at the Pasadena Conventrion Center, and DevOpsDayLA is on the Friday.

Download the DevOpsDayLA Prospectus for more info.

Subscribe to the LA DevOps Meetup Calendar at lu.ma
The LA DevOps events are also listed on lu.ma which has an interesting interface. Subscribe to LA DevOps on lu.ma at https://lu.ma/calendar/cal-F7BPqBLAe80T5nS

##