Study Group: LLM Inference
Join Applied AI Collective for a practical study group on LLM inference and vLLM — how modern language models are served efficiently, what happens after a prompt reaches a model, and how inference systems balance latency, throughput, memory, and cost.
As LLM applications move from prototypes to real-world products, running the model efficiently becomes an important part of the AI stack. It is not enough to choose a capable model — builders also need to understand how that model is deployed, how requests are processed, how GPU resources are used, and how inference performance changes as traffic grows.
This session is designed as a builder-friendly discussion for engineers, founders, researchers, product builders, and AI enthusiasts who want to better understand the infrastructure behind serving LLMs in production.
What we’ll cover:
• What LLM inference is and what happens during model serving
• Prefill vs. decoding and why they have different performance characteristics
• Latency, throughput, tokens per second, and other important inference metrics
• How batching and continuous batching improve GPU utilization
• KV caching and why it matters for inference performance
• An introduction to vLLM and how it enables high-throughput LLM serving
• PagedAttention and the problem it was designed to solve
• Quantization and other techniques for reducing inference cost and memory usage
• Serving open-source models and thinking about GPUs, model size, and concurrency
• Open discussion around inference frameworks, architectures, benchmarks, and real-world deployment challenges
Who should attend:
AI builders, software engineers, ML engineers, founders, product managers, researchers, students, and anyone interested in understanding how LLMs are served in real-world systems.
No prior expertise in inference infrastructure is required. Bring your questions, experiences with tools like vLLM, models you have deployed, or AI systems you are currently building.
This is a study group, not a formal lecture. The goal is to learn together, discuss practical approaches, and build a better understanding of what happens between an LLM application and the GPUs actually running the model.
Let’s build a stronger community around practical, scalable, real-world AI systems.
