Cloud Native Innovations: Securing LLMs and Streamlining Platforms

Join us this May for our cloud native meetup, "Cloud Native Innovations: Securing LLMs and Streamlining Platforms," featuring two highly anticipated talks from industry experts.

If this event is fully booked, head over to CNCF platform for more seats.

## Agenda

16:30 Door open, finding seat, enjoying snacks and networking
17:00 Welcome by the Cloud Native Copenhagen
17: 05 Welcome by our host Visma e-conomic
17:15 Talk 1: Securing custom LLMs using HashiCorp Vault
18:00 Break with food and drinks
18:45 Talk 2: The 3 Layers of our Platform and Helmping Ourselves before helping others
19:30 Break with snacks, networking
20:00 Thank you for today

### Talk 1: Securing custom LLMs using HashiCorp Vault

Building custom Large Language Models (LLM) using private data dramatically benefits your business by allowing the model to understand specific documents and data unavailable in public datasets. However, there are issues with this approach; one problem is how you secure the data used to train the dataset, and another is how you ensure that this data remains private, specifically when using the public cloud. Secondly, how do you ensure that your compiled models are tamper-proof and have not been manipulated by a hostile actor? This talk will show you how to solve both problems using HashiCorp Vault, and how to do it using common Kubernetes patterns that you are used to. You will learn how to install Vault onto your own server using the Helm, how to install the operator, and all the configuration you need to secure and configure Vault. We will wrap up all this in a healthy layer of fun, and as a takeaway, you can try these techniques on your own using the interactive workbook provided at the end of the session.

Speaker Bio: Nic Jackson is a developer advocate at HashiCorp and has over 25 years of industry experience working with and creating open-source software and distributed systems. One of his core interests is system connectivity and reliability. Nic is the CNCF TAG Network co-chair and a published author working on his second book, Service Mesh Patterns for O'Reilly Media.

### Talk 2: The 3 Layers of our Platform and Helmping Ourselves before helping others

At Banking Circle, we've got an Internal Developer Platform which our developer teams use to run their software on, the easy and secure way.
We've designed that Platform with 3 layers in mind:

1. At the core of it, we have our Kubernetes clusters, together with different technologies running on it, such as ArgoCD, Trivy, Kyverno or Cilium
2. On top of the core layer, we build the "foundation", the area where a developer team can work
3. On top of that, we provide mechanisms to enable developers to deliver their software to the Platform

The tooling in the core layer is mostly handled with Helm charts. To help ourselves before helping others, we've also built an internal tool called Helmper. Helmper facilitates importing all the images used in any Helm chart to our own registries. It assists in deploying arbitrary Helm Charts in regulated and/or air-gapped environments.
After using Helmper to import images, we just edit the image references on our values file to point to our own registries. At Banking Circle, we use it in combination with Trivy and Copacetic, to ensure we remove fixable vulnerabilities from images before using them.

Speaker Bio:
Christoffer Nissen is a Senior Platform Engineer at Banking Circle, working on providing an Internal Developer Platform with a focus on developer experience, security and reliability. Before that he has helped several companies as a DevOps consultant and has been a Teaching Assistant at IT University in Copenhagen.

Juan Herreros Elorza is a Tech Lead for Platform Engineering at Banking Circle, working on providing an Internal Developer Platform with a focus on developer experience, security and reliability. Before that he has been working on different domains of Tech, mostly on DevOps and Platform Engineering, in different financial institutions.