Kafka Tuning for Low-Latency/Building a Dynamic Rules Engine with Kafka Streams

Hello streamers! Please join us for an IN-PERSON Apache Kafka® meetup on Tuesday, July 25th from 5:30pm venue hosted by our friends at SmartThings and food and beverage sponsored by our friends at Kinetic Edge.

📍Venue:
Riverplace Building Complex
1 SE Main Street, Minneapolis, MN 55414
Meetup Location Details:
Kafka Meetup will be held in the Garamond Conference Room located inside the Main Hall Building within the Riverplace Complex.

*Parking is available on the street or in the Riverplace Parking Ramp on 2nd St SE

***
🗓 Agenda:

• 5:30pm: Doors open
• 5:30pm - 6:00pm: Snack, drinks and networking
• 6:00pm - 6:45pm: Kafka Tuning for Low-Latency Experiences, Jeff Blaisdell, SmartThings
• 6:45pm - 7:30pm: Building a Dynamic Rules Engine with Kafka Streams, Will LaForest, Confluent
• 7:30pm-8:00pm: Additional Q&A & Networking

***
💡 Speaker One:
Jeff Blaisdell, SmartThings

Title of Talk:
Kafka Tuning for Low-Latency Experiences

Abstract:
SmartThings, a consumer IoT company launched in 2012 through a successful Kickstarter campaign, has evolved into a distributed architecture composed of many systems following its acquisition by Samsung. Designing an IoT Platform comes with some unique considerations - including strong needs to provide near instantaneous experiences. During this talk we will explore some of SmartThings’ low latency experiences, and some of the optimizations we’ve made around Kafka to optimize low latency.

Bio:
Jeff is a software engineer with over a decade of experience across many industries. He has been an Engineer @ SmartThings for the last 8 years, and has worked across a variety of areas of the platform. His current focus is the Event Pipeline. Professionally, you’ll find him happiest when hunched over a laptop puzzling through a scaling / performance problem. Outside of tech he loves adventures with his family, and all things soccer.

***
💡 Speaker Two:
Will LaForest, Confluent

Title of Talk:
Building a Dynamic Rules Engine with Kafka Streams

Abstract:
The benefit of real-time data can be measured not only by how frequently the data in question changes but also how quickly you can change the way in which you handle and react to it. One area where this is highly apparent is in threat detection. Responding to an ever changing landscape of attacks and exploits requires a system that can not only handle the scale and dynamic nature of the data but also a dynamically changing set of detection rules. We developed Confluent SIGMA, an open source project built on Kafka Streams for the open SIGMA DSL, to handle real-time rule additions and modifications. In this talk we will cover:

* The architecture of our Kafka Streams layer that makes it possible to use external data feeds as rule input
* How sets of Sigma Rules are mapped to Kafka Streams topology
* Considerations for creating a Kafka Streams based DSL implementation
* Why Kafka Streams was chosen as opposed to other technologies.

Although Confluent SIGMA focuses on cyber threat detection this same pattern can also be applied to any DSL (domain specific language) that would benefit from real-time stream processing. After attending you will have a recipe and starting point to drive dynamic rules through Kafka Streams for any use case that might require it.

Bio:
As Field CTO for Confluent, Mr. LaForest works with customers across a broad spectrum of industries and government enabling them to realize the benefits of data in motion and event driven architecture. He is passionate about innovation in data technology and has spent 26 years helping customers wrangle data at massive scale. His technical career spans diverse areas from software engineering, NoSQL, data science, cloud computing, machine learning, and building data visualization software but began with code slinging at DARPA as a teenager. Mr. LaForest holds degrees in mathematics and physics from the University of Virginia.

***
DISCLAIMER
BY ATTENDING THIS EVENT IN PERSON, you acknowledge that risk includes possible exposure to and illness from infectious diseases including COVID-19, and accept responsibility for this, if it occurs.
NOTE: We are unable to cater for any attendees under the age of 21.

***
If you would like to speak or host our next event please let us know! community@confluent.io