Building Local LLM Applications with llama.cpp: Modern C++ in Action

Hello all!

Welcome to our final meetup before the summer break. This session will introduce llama.cpp, a lightweight, open-source C++ library designed for efficient large language model (LLM) inference on a wide range of hardware—including CPUs and GPUs from Nvidia, AMD, and Apple. llama.cpp is the engine behind popular projects like Ollama and GPT4All, and it enables developers to run state-of-the-art LLMs locally, even on laptops and resource-constrained devices.

We are kindly hosted by Consat this evening.

Abstract
This talk will take a practical, application-driven approach to leveraging llama.cpp in modern C++ projects. We will cover:

• A high-level introduction to LLMs, focusing on practical applications rather than mathematical theory.
• Where to find open-source LLM models and how to convert them to the GGUF format for use with llama.cpp.
• How to integrate llama.cpp into your C++ projects using #include "llama.h", with live demonstrations of running inference on a laptop.
• An overview of Retrieval Augmented Generation (RAG) systems from an application perspective—what they are, the benefits for custom data, and how to build a simple RAG pipeline with llama.cpp as the inference engine.

Hopefully, by the end of the session, you’ll understand how to use llama.cpp to build efficient, private, and customizable LLM-powered applications in modern C++, and how to set up a RAG system for your own data and business needs.

Explore the library: https://github.com/ggml-org/llama.cpp

AGENDA
18:00 - Doors open, with fika/snacks/food and mingling
18:30 - Event starts, welcome from Consat
18:40 - Info about GbgCpp
18:45 - Presentation starts
19:50 - Wrap-up and closing
20:00 - Event finishes

We look forward to seeing you and diving into hands-on LLM development with modern C++!