What we're about

This Meetup Group will focus on the Apache Kafka® technology and its ecosystem, also allowing its members to interact and share their use cases, do and don't, how to's, etc.

We will also hear about the Confluent Platform and topics like Kafka's Connect API and streaming data pipelines, Kafka’s Streams API and stream processing, Security, Microservices and anything else related to Apache Kafka.

If you want to hear about a particular topic please let us know and we will try to find the best possible speaker. We are also very open to this being an active platform for the community, so if you would like to participate in other ways like speaking, hosting, writing a blog, etc, please let us know on community@confluent.io

Upcoming events (1)

Post-Summit Meetup @ Confluent's SF Office! Airbnb, Bloomberg and Confluent

We would love to have some events in East Bay, but need your help! If you're interested in speaking/hosting please get in touch at [masked] _ Join us for a post-summit Apache Kafka® meetup on October 1st at 6:30pm, at Confluent's new office in San Francisco! 6:30pm F+B & Networking 7:00pm 1) Frank Greco, Confluent 7:40pm 2) Lei Chen, Bloomberg 8:20pm 3) Qianqian Zhong, Xu Zhang, & Zuofei Wang, Airbnb 9:00pm 9:30pm: More Q&A & Networking -- Talks: 1) How to Build a Cloud From proof-of-concept to best in class, Confluent Cloud has come a long way in 2.5 years! In this talk we'll take a journey through the various iterations the product has taken. We'll explore the lessons we learned along the way, architectures that served different orders of magnitude as we scaled, useful optimizations made along the way, and the exciting future of our cloud product! We'll also discover how we scaled the developer experience internally from only a handful of engineers to hundreds! Speaker bio: Frank is an extremely passionate tech engineer, developer, and architect from San Jose. His current passions lie in highly available and scalable infrastructure, containerization, serverless architecture, automation, artificial intelligence, web development, API management and algorithm theory. He is also very passionate about open source software and contributes to it regularly. --- 2) Run Kafka Streams beyond Kafka Kafka Streams, together with KSQL, have formed a pretty complete ecosystem to build data processing pipelines around Kafka clusters. However, there are occasions where your data is not in a Kafka topic or you might want to keep a golden copy of your data for regression testing. Other times, you may want to tweak the data manually before feeding it into your Kafka Streams pipeline or you may want to have your Kafka Streams output dumped to a file directly instead of to Kafka. For those use cases, there is no out of the box solution, since Kafka Streams is tightly dependent on Kafka, and Kafka is designed to serve as short-term, append-only storage. This talk will describe how to use a plugable client supplier in Kafka Streams to allow it to read data from non-Kafka topics and output to non-Kafka topics (e.g., to a local or HDFS file), without changing any of your Kafka Streams library semantics. Speaker bio: Lei Chen currently leads the data pipeline platform effort in the Derivatives Data Group in Bloomberg's San Francisco Engineering Office. He has built streaming analytics pipelines using different streaming engines, including Kafka Streams, Flink, etc., in order to process/analyze hundreds of millions of market data ticks in real-time each day. -- 3) Apache Kafka at Airbnb Airbnb runs one of the largest Kafka deployments in the cloud. We use Apache Kafka as a message bus to transport data and to power real-time streaming services. This talk will briefly talk about two major systems we built @ Airbnb. SpinalTap is a general-purpose reliable Change Data Capture (CDC) service, capable of detecting data mutations with low-latency across different data sources, and propagating them as standardized events to downstream consumers. Mussel is a scalable Key/Value store integrated with Airbnb ecosystem for configuring offline datasets to be loaded for access from online services. It models data as Primary key, Secondary key, Timestamp and Value. We will talk about how Airbnb uses Kafka to power the system. Speakers' bio: Qianqian Zhong, Xu Zhang, and Zuofei Wang are software engineers at Airbnb from Cloud Infrastructure. As part of Infrastructure, the Cloud Infrastructure team at Airbnb is responsible for providing the core services and systems that allow product engineers to efficiently build and operate scalable and reliable applications ---- KAFKA SUMMIT SF 2019: 9/30 - 10/1 For 25% off, use the code "KS19Meetup" at bit.ly/KSummitMeetupInvite Community slack: https://launchpass.com/confluentcommunity

Photos (1)