Kicking Your Database to the Curb

Host: Carahsoft
Address: Carahsoft Technology Corporation, 11493 SUNSET HILLS ROAD, WEST, 5th Floor, SUITE 100, RESTON, VA 20190

Abstract:
In typical microservice applications, you’d need to send the state of that microservice to a relational database -- i.e., manage remote state -- then use another app to view the results as the microservice updates the database. Unfortunately, this common approach has few downsides: increased complexity and code management among them. But there is an alternative for your microservices: The Streams API of Apache Kafka is a state-of-the-art stream processing library to write large-scale, fault-tolerant, elastic, scalable applications. It includes powerful local state management functionality, which eliminates the need for remote state storage in many use cases.

Kafka Streams eases that burden by providing a mechanism, Interactive Queries or IQ, to view the state across all streams instances. What’s compelling about Interactive Queries is that you don’t need to know which application instance contains the information required. You query one Kafka Streams instance, and if it does not hold the desired record(s), it asks the other Kafka Streams instances on your behalf.

So with Interactive Queries, you can build, for example, dashboard functionality without the need of adding another storage technology, you can directly query Kafka Streams itself and retrieve all of the state you need directly as records are arriving.

In this talk, Bill will give the audience some background context on event streams and update streams, and how update streams use state. From there, the audience will learn how to enable interactive queries, how to implement IQ, and some best practices and the tradeoffs of using IQ.

6:00pm: Doors Open, Networking, Snacks and Drinks
6:30 pm - 7:15pm: Bill Bejeck, Confluent, Kicking Your Database to the Curb

7:15 pm-7:30pm: Q&A
7:30-8:00 pm: Additional networking

Speaker Bill Bejeck:
Bill has been a software engineer for over 15 years. Currently, he is working at Confluent as an engineer on the streams team. He regularly contributed to Kafka Streams before starting with Confluent and continues to do so in his work today. Before Confluent, he worked on various ingest applications as a U.S. Government contractor using distributed software such as Apache Kafka, Spark and Hadoop. He has also written a book about Kafka Streams titled “Kafka Streams in Action,” published in August 2018, and writes at https://www.confluent.io/blog/author/bill-bejeck/