How to setup your private ChatGPT-Like Tool (LLM) with RAG

ChatGPT is currently everywhere, but it has some flaws. All data goes to the US, lacking proper security or privacy, and adding personal data (PDFs, CSVs, etc.) is not straightforward.

These issues can be solved by a private Ollama (llama2) with RAG setup. Do you want to input your data and run LLM within your infrastructure? In this meet-up, we will present success stories and use cases for LLM based solutions, and in the second part, how to set it up for yourself.

Part 1: Use Cases and Success Stories (30 Minutes)

Use cases for a private ChatGPT-like tool and success stories from our customers. In this part, we give a short introduction to Large Language Models and what problems it can solve for your company. We will explain what the differences are compared to ChatGPT and why most probably your business will need Retrieval-Augmented Generation (RAG). At the end of this part, we will make a live demo to show you how your setup could look like, and there will be room for questions.

Target Group
• Business professionals who are curious about leveraging AI for their business.
• Non-technical persons
• Commercial Owners
• Sellers

Level of complexity:
• Beginner/ Overview

Part 2: How to set it up (30 Minutes)

After the business part we go over to the technical part. In this part we explain how to set it up on the OpenTelekomCloud. We start with Docker and then go over step-by-step till we reach a full solution with RAG, Ollama (llama2), Kubernetes and GPU Node Pools.

Target Group
• Tech enthusiasts eager to learn how to set up their own LLM.
• Developers looking to integrate AI tools into their projects.

Level of complexity:
• Advanced

About the Speakers:

Dr. Tim Delbrügger is the Head of the AI Department of iits-consulting. He was working with AI Technologies long before it became mainstream and has now over 10 years of AI Architecture and Development experience. He supervises all AI projects of iits and therefore has gathered a lot of success stories.

Victor Getz is the CTO of iits-consulting. He builds and supervises dozens of cloud architecture each year and therefore has seen all kinds of cloud setups from petabyte of data till millions of users.

Don't miss this opportunity to dive into the world of AI with your own ChatGPT-like tool. Whether you're a beginner or an expert, there's something for everyone. We look forward to seeing you there!