Design Patterns for working with Fast Data in Kafka

Apache Kafka is an open-source message broker project that provides a platform for storing and processing real-time data feeds. In this presentation Ian Downard will describe the concepts that are important to understand in order to effectively use the Kafka API. You will see how to prepare a development environment from scratch, how to write a basic publish/subscribe application, and how to run it on a variety of cluster types, including simple single-node clusters, multi-node clusters using Heroku’s “Kafka as a Service”, and enterprise-grade multi-node clusters using MapR’s Converged Data Platform.

Ian will also discuss strategies for working with "fast data" and how to maximize the throughput of your Kafka pipeline. He'll describe which Kafka configurations and data types have the largest impact on performance and provide some useful JUnit tests, combined with statistical analysis in R, that can help quantify how various configurations effect throughput.

The code and presentation for this talk will be available at https://github.com/iandow/design-patterns-for-fast-data .

Speaker Ian Downard:

Ian Downard is a technical evangelist for MapR where he is focused on creating developer-friendly ways to use the MapR Converged Data Platform (https://www.mapr.com/products/mapr-converged-data-platform).

Personal Blog: http://www.bigendiandata.com

Twitter: https://twitter.com/iandownard

GitHub: https://github.com/iandow

LinkedIn: https://www.linkedin.com/in/iandownard

Venue Notes:

Doors open at 6 for pizza provided by TEKSystems. The presentation will begin at 6:30pm.