🧠 RAG Proxy: I Built a Universal Brain for Local AI
Details
Wednesday, 7pm. Dress rehearsal for my TechKnowFile 2026 talk at U of T. You get the unfiltered version.
🔥 THE STORY
I spent months trying to teach an AI to memorize 94 dental policy documents. Six training runs. Six failures. The AI learned to sound like an expert while making up email addresses, inventing forms, and citing policies that don't exist. Confidently wrong is worse than obviously wrong, especially when someone just got stuck with a needle.
Then I read the research. Fine-tuning doesn't teach AI what to say. It teaches it how to sound. The actual accuracy gap: 87.5% (RAG) vs 50.4% (fine-tuning). Barely better than a coin flip.
So I pivoted. And ended up building something I haven't seen anyone else do.
🧩 THE UNIVERSAL BRAIN
Most RAG systems are locked inside one chat app. OpenWebUI has its own. ChatGPT has Custom GPTs. Claude has Projects. Switch tools and the brain is gone.
I built RAG Proxy: a single layer that sits between any AI tool and the model. Every tool you already use, instantly RAG-aware. Your terminal, your IDE, your chat app, a script, a future mobile app. Same brain, every client. Change one setting and the tool gets smarter. No cloud. No API keys. No data leaves the building. $0/month. Runs on one Mac Studio in a server closet.
📋 WHAT WE'LL COVER
- The six fine-tuning failures and what each one taught me
- The peer-reviewed research on why fine-tuning can't inject knowledge
- The trick that makes RAG Proxy universal (this is the novel bit, and I'll show it live)
- Live demo on real hardware. Ask anything about the policies. Then ask it something it shouldn't know and watch it refuse instead of making something up. Same brain, three different clients.
- Why local AI matters: privacy by architecture, not policy memo
🎯 WHO THIS IS FOR
AI folks, self-hosters, Linux nerds, privacy people, or anyone who likes seeing one person out-build an enterprise vendor with open-source tools and a weekend. No experience required. Questions encouraged. Bring your skepticism.
⏱️ FORMAT
About 30 minutes of talk, then open Q&A.