Adobe: Exactly Once Semantics in Kafka & Confluent: Speaks Kafka

This is a past event

146 people went

Location image of event venue


Join us for an Apache Kafka meetup on July 27th from 6:30pm, hosted by Cloudfare in San Francisco. The address is 101 Townsend Street. The agenda and speaker information can be found below. See you there!


6:30pm: Doors open
6:30pm - 6:45pm: Networking, Pizza and Drinks
6:45pm - 7:15pm: Presentation #1: Kafka in Adobe Ad Cloud's Analytics Platform: Building systems with exactly-once end-to-end semantics, Michael Schiff & Vikram Patankar, Adobe
7:15pm - 7:45pm: Presentation #2: Exactly-once Stream Processing with Kafka Streams, Guozhang Wang, Confluent
7:45pm - 8:15pm: Additional Q&A and Networking


First Talk

Michael Schiff & Vikram Patankar


Michael Schiff
He is the lead software engineer on the data analytics team at Adobe Ad Cloud (Formerly TubeMogul). He architected and built the real-time analytics infrastructure that empowers the end-to-end platform for managing advertising across traditional TV and digital formats. He actively contributes to open source projects like Druid and Apache Kafka. Michael holds a bachelor's degree from UC Berkeley.

Vikram Patankar
He is the Senior Engineering Manager of the data analytics team at Adobe Ad Cloud (Formerly TubeMogul). He owns the evolution of data architecture on this platform to support billions of ad events every day and responsible for data infrastructure including scalability, robustness, and performance of backend data systems. Previously he helped build the big data analytics platform and a suite of SaaS applications that provides advanced analytics for insurers, re-insurers and capital markets at Risk Management Solutions (RMS). Prior to that, he built the key infrastructure pieces at Myspace like advanced NoSQL Store and a custom graph database that empowered the most demanding features on the site like music, video & stream to handle monumental loads. Vikram holds a master's degree in Computer Science from Texas Tech University and a bachelor's degree in Computer Engineering from University of Pune, India.


Kafka in Adobe Ad Cloud's Analytics Platform: Building systems with exactly-once end-to-end semantics

Kafka has become the cornerstone of many stream processing applications. Until recently, Kafka gave you the choice between at-least and at-most once delivery, but lacked built-in support for exactly-once semantics. Exactly-once semantics guarantee the assumption that each event you send will take effect once, and only once. This is critical for any system in which events represent incremental changes to some state. Without support for exactly-once semantics out of the box, such applications required additional work to guarantee correctness. In this session, we will cover how Adobe Ad Cloud uses Kafka in its data analytics platform, what is necessary to uphold exactly-once semantics, and how we implemented this in our streaming platform.


Second Talk

Guozhang Wang

Guozhang is a an engineer at Confluent, building a stream data platform on top of Apache Kafka. He receives his PhD from Cornell University database group where he worked on scaling iterative data-driven applications. Prior to Confluent, Guozhang was a senior software engineer at LinkedIn, developing and maintaining its backbone streaming infrastructure on Apache Kafka and Apache Samza.

Exactly-once Stream Processing with Kafka Streams

In this talk, we present the recent additions to Kafka to achieve exactly-once semantics within its Streams API for stream processing use cases. This is achieved by leveraging the underlying idempotent and transactional client features. The main focus will be the specific semantics that Kafka distributed transactions enable in Streams and the underlying mechanics to let Streams scale efficiently.