Scale your streaming data pipelines efficiently with Kubernetes

A system architecture for streaming data pipelines that scales automatically while minimizing compute costs.

Applications deployed as Kubernetes Deployments (or StatefulSets) can be scaled manually or automatically either through the API or with the HorizontalPodAutoscaler (HPA). This can be a great solution for scaling relatively steady workloads with short-lived task durations, such as a REST API backing a web application.

With volatile workloads, back pressure can become a problem. If volume increases faster than resources can scale up, requests will time out.

Agenda
6:00 - 6:10 PM Introductions and Announcements
6:10 - 6:50 PM Presentation by Richard Wood from SaasGlue - https://www.saasglue.com/
6:50 - 7:00 PM Q & A

Here is the zoom link:
https://zoom.us/j/98056432054