Tue, Apr 21 · 6:00 PM CEST
Presenter: Mathias Magnusson, ACE Pro
Running locally means consistent access. No throttling like Anthropic has done due to increased demand they cannot meet.
I also hear people say they don't want to use it as long as there is a cost associated.
Everything in this presentation is of course directly applicable to using public AI services, there are just a little more setup required to run locally.
So let's look at how it works to run the inference engine (the software that exposes LLM to you) on your own hardware and see it work with:
Getting an LLM running on your machine
Cline in Visual Code
Separate chat window
Vibe tool
APEX
Database
Access and awareness of your database (MCP)
Using defined skills to achieve things in your database
We will focus on what it takes to set up and show it working. The hour will run fast and getting everyone set up or digging in to each topic in more detail would have to be their own workshop.
We will see how far we get on the listed topics, this may need to get a followup part II timeslot.
I will use an EU-based AI for this. There is so much talk about "there is nothing in EU" looking at it is worthwhile even when we keep the option to use any LLM on the planet.
