Thu, Jul 16 · 6:30 PM CEST
A lot of tools wrap an inference engine like llama.cpp to let you run LLMs locally. Few of them actually benchmark on your hardware to tell you which models you can realistically run.
Months ago I found myself with tens of gigabytes of models sitting on my hard drive, so I decided to figure out which configuration was actually the fastest - at reading prompts and generating tokens. I wrote a PowerShell script to measure exactly that.
It quickly grew into something more structured, once I realized it could help other people figure out what their hardware can actually do. That became calibr: a benchmark and recommender tool that downloads models from Hugging Face and runs them locally under different configurations. When it's done, you get stats on every model and configuration it tried: prompt and generation speed, VRAM and RAM usage, VRAM-to-RAM spillover, and load time.
In this talk I'll walk through what calibr is, where it's heading, and how it works under the hood - along with some of the challenges I ran into while building it.
Agenda
---
Speaker
Federico Piana
Senior Software Engineer with 10+ years building Angular/TypeScript applications on SaaS products. Currently extending into local LLM tooling: running llama.cpp with Qwen on consumer hardware, building RAG pipelines with FastAPI + ChromaDB, and integrating LLMs into Unreal Engine.
Hosted By
Marco Gomiero, Android Engineer
Marco is an Android engineer, currently working at Airalo. He is a Google Developer Expert for Kotlin, he loves Kotlin and he has experience with native Android and native iOS development, as well as cross-platform development with Flutter and Kotlin Multiplatform.
In his spare time, he writes and maintains open-source code, he shares his dev experience by writing on his blog, speaking at confs and organizing events with the Google Developer Group Venezia and he plays basketball.
Andrea Maglie, Organizer
Fabio Catinella, Android Engineer
Simone Formica, Organizer
Android & AOSP Dev
Co-organizer of GdG Venezia - Mentor at Coderdojo ZeroBranco (TV)
Jessica Marini, Social Media Manager
Hi, I’m Jessica Marini, also known as Nessie, and I’m an Italian illustrator.
Omar Al Bukhari, Organizer
Experienced Mobile Developer with a track record of optimizing development lifecycles and enhancing client satisfaction.Key highlights include:
-Proven expertise in Android, Outsystems development, and ongoing learning in Kotlin Multiplatform.
Nicolò Giaccone, iOS Engineer
---
Partner
Var Group (https://www.vargroup.com/it-IT)
---
Complete your event RSVP here: https://gdg.community.dev/events/details/google-gdg-venezia-presents-calibr-an-open-source-tool-to-benchmark-local-llms-on-your-hardware/.
