Local AI and AI Harnesses by Jaime Pardo

Building a Local AI Companion – From Hardware to Autonomous Agents

The era of relying entirely on cloud-based AI is shifting, opening the door for users to build and run their own powerful models at home. This presentation explores the transition from renting cloud space to constructing a personal, localized AI assistant—think less along the lines of a corporate chatbot and more like building a custom, cinematic "Jarvis." For beginners, we will demystify the core benefits of running AI locally: absolute data privacy, zero recurring subscription fees, and the freedom from server outages or internet latency. It is about taking back ownership of your data and your digital tools.

To bridge the gap between concept and reality, we will look under the hood at the hardware that makes this possible. We will break down exactly what an enthusiast needs to get started without needing an enterprise budget, highlighting the importance of a solid foundation—like an AM5 motherboard and plenty of fast DDR5 RAM—and the crucial role of GPU VRAM. Using a dual-GPU setup as a real-world example, the audience will learn how concepts like quantization compress massive open-source models so they can run efficiently on consumer hardware.

Once the hardware is understood, we will dive into the software ecosystem that brings the machine to life. We will explore user-friendly inference servers like Ollama and LM Studio that make loading up models as easy as installing an app. For the enthusiasts looking to go deeper, we will touch on Unsloth Studio for fine-tuning models and creating custom LoRAs. The goal is to show the audience how to navigate platforms like Hugging Face, select the right model weights, and understand the metrics that matter, such as tokens per second and prompt processing times.

Finally, we will look at the true potential of a local AI setup: autonomous agents. We will explore how to give a local model "hands" by connecting it to tools, local CLI environments, and secure Docker sandboxes. By setting up gateways to platforms like Telegram and giving the AI access to read and write local Markdown wikis, it transitions from a passive chat window into a self-improving, continuous digital assistant. The presentation leaves the audience with a practical blueprint for a living project, demonstrating that setting up local AI is an evolving story well worth writing.

LOGISTICS AND PARKING:

The talk starts at 7:00 PM. The first half hour is reserved for everyone to get set up and mingle. Free pizza and drinks!

The cheapest parking option is to find street parking, which will only cost you a few bucks. Otherwise, park in the nearby veteran's museum lot for $8. It's highly recommended you avoid the nearby $15 garage parking.