Running Kafka on Kubernetes with Strimzi


Kafka is traditionally well suited to run on statically defined clusters, but running it on container orchestrated clusters like Kubernetes is becoming more common. The best way to run stateful services on Kubernetes that have complex operational needs (such as Kafka) is to use the Operator pattern. This talk will review a popular new open source multi-operator-based Apache Kafka implementation on Kubernetes called the Strimzi Kafka Operator.

Strimzi makes many Kafka operational tasks easy, such as the initial deployment and updates of a Kafka and ZooKeeper cluster. Kafka resources such as brokers, ZooKeeper, topics, users, and mirror-maker configurations can all be managed using Kubernetes Custom Resources. Transient and persistent volume storage options may be used. Common operational tasks are made easier, such as rolling cluster updates and upgrades, scaling brokers, moving brokers, monitoring, and running multiple Kafka clusters on the same Kubernetes cluster.

Strimzi is the leading open source project to run Apache Kafka and ZooKeeper on Kubernetes and is an integral component of the Lightbend Platform.