Skip to content

Details

Monday, November 16, 2026 · 9:00–10:00 AM PST · Online
Zoom link: https://us02web.zoom.us/j/82308186562
Join us for two deep dives into hidden AI performance costs: surprise PyTorch recompilations during inference and compute, memory, and communication bottlenecks in Jev training.
Talk #0: Introductions and Meetup Updates
by Chris Fregly and Antje Barth
Talk #1: Overcoming the PyTorch Recompilation Dilemma in Real-Time Inference Workloads
by Chaim Rand
`torch.compile` can accelerate steady-state inference, yet a guard failure on a new input shape or layout can trigger an expensive recompilation in the request path. Drawing on his latest post, Chaim will compare six approaches using a dynamic graph-neural-network example: cache preloading, bounded recompilation with eager fallback, Dynamo and input-layout tuning, warmup seeding, background compilation, and ahead-of-time compilation with `torch.export` and AOTInductor. He’ll explain their tradeoffs and how the right choice depends on the model, input distribution, runtime environment, and latency requirements.
Bio: Chaim Rand is a machine-learning algorithm developer who writes about PyTorch and practical AI/ML performance optimization.
Based on: Overcoming the PyTorch Recompilation Dilemma in Real-Time Inference Workloads
Talk #2: From Profiles to Throughput: How TypeSafe Optimized Jev with Zymtrace
by Israel Ogbole, Co-founder and CEO of Zymtrace
Israel will explore the latest Zymtrace/TypeSafe case study on training and serving Jev. He’ll discuss how continuous CPU/GPU profiling and MCP-accessible profile data supported an iterative optimization loop across compute, memory, communication, and inference synchronization. The published case study reports approximately 46% higher training throughput on the same hardware after three successive changes, plus approximately 10% better inference performance after removing a hidden device-to-host synchronization. The talk will focus on how the team found each bottleneck, measured each change, and identified the next constraint.
Bio: Israel Ogbole is co-founder and CEO of Zymtrace, which develops continuous CPU/GPU profiling tools for performance optimization.
Based on: How TypeSafe achieved 46% higher GPU throughput training Jev — with the help of Zymtrace
Zoom link: https://us02web.zoom.us/j/82308186562
Related Links
Github Repo: http://github.com/cfregly/ai-performance-engineering/
O’Reilly Book: https://www.amazon.com/Systems-Performance-Engineering-Optimizing-Algorithms/dp/B0F47689K8/
YouTube: https://www.youtube.com/@AIPerformanceEngineering
Generative AI Free Course on DeepLearning.ai: https://bit.ly/gllm

You may also like