DIY LLMs: Hosting your own LLM inference, from silicon to service

Join us for the USF Data Science Speaker Series featuring Dr Charles Frye, Modal Developer Advocate!

Charles Frye is a passionate educator who specializes in teaching people to build AI applications. After publishing research in psychopharmacology and neurobiology, he got his Ph.D. at the University of California, Berkeley, for dissertation work on neural network optimization. He has taught thousands about the full stack of AI application development—from foundational linear algebra to advanced GPU techniques and creating defensible AI-driven businesses.

Charles will explore the essential components for running your own large language model (LLM) inference service. This talk will delve into:
• Compute options: CPUs, GPUs, TPUs, and LPUs.
• Model options: Qwen, LLaMA, and others.
• Inference server options: TensorRT-LLM, vLLM, and SGLang.
• Observability tools: OTel stack, LangSmith, W&B Weave, and Braintrust.

Don’t miss this opportunity to gain practical knowledge on building and hosting your own LLM services from a leading AI educator and expert!

#USFDataScienceSpeakerSeries #DataScience #MSDS #LLMs #AI #MachineLearning #AIApplications