Next Meetup

Kafka & Elasticsearch: Introduction, Best Practices & User Stories
Join us for our next Apache Kafka® meetup on September 24th from 6:00pm, hosted by Lyft. We are delighted to join forces with our friends from the ElasticSearch Meetup group for this, so please make sure you only RSVP in only 1 of the 2 meetup pages. ( The agenda, venue and speaker information can be found below. See you there! ----- Agenda: 6:00pm: Doors open 6:00pm - 6:30pm: Pizza, Drinks and Networking 6:30pm - 7:15pm: Streaming DynamoDB changelogs to Elasticsearch using Apache Kafka and Flink by Ying Xu and Dan Fan 7:15pm - 7:45pm: Integrating Kafka into your Elasticsearch use case by Andrew Selden 7:45pm - 8:00pm: Microservices Integration Patterns with Kafka by Kasun Indrasiri 8:00pm - Additional Q&A & Networking ------ Abstract Information: Microservices Integration Patterns with Kafka Microservice composition or integration is probably the hardest thing in microservices architecture. Unlike conventional centralized ESB based integration, we need to leverage the smart-endpoints and dumb pipes terminology when it comes to integrating microservices. There two main microservices integration patterns; service orchestration (active integrations) and service choreography (reactive integration). In this talk, we will explore on, Microservice Orchestration, Microservice Choreography, Event Sourcing, CQRS and how Kafka can be leveraged to implement microservices composition Kasun Indrasiri is the director of Integration Architecture at WSO2 and an architect with over nine years of experience in Enterprise Integration and Microservice. He is an author and an evangelist on Microservices Architecture. He has authored ‘Microservices for Enterprise’ (Apress: 2018 Q4) and ‘Beginning WSO2 ESB’ (Apress - Released in 2017) books. He was also an architect and the product lead of WSO2 ESB and an Apache committer. Integrating Kafka into your Elasticsearch use case Andrew will be providing an introduction on how users are integrating Kafka into their Elasticsearch use cases. This will include best practices, architecture overviews and more. Andrew is a Senior Solutions Architect at Elastic Streaming DynamoDB changelogs to Elasticsearch using Apache Kafka and Flink In this talk, we will present the architecture of Lyft’s changelog data ingestion pipeline which allows for real-time ingestion of DynamoDB changelogs into Elasticsearch. Our system uses Apache Kafka as a core pub-sub component storing all the changelog data. Apache Flink jobs are employed as connectors linking data source(s) and destination(s). By virtue of state-of-the-art streaming technology, the whole data pipeline achieves low latency, strong message durability and ordering guarantee, with scalability and extensibility built into the design. -------- Don't forget to join our Community Slack Team ! If you would like to speak or host our next event please let us know! [masked] NOTE: We are unable to cater for any attendees under the age of 18. Please do not sign up for this event if you are under 18.

Lyft HQ

185 Berry St #5000 · San Francisco, CA


    Past Meetups (30)

    What we're about

    This is a group interested in Apache Kafka® -


    **** Join our Slack Channel for live, global interaction! **** (

    This Meetup Group will focus on the Apache Kafka® technology and its ecosystem, also allowing its members to interact and share their use cases, do and don't, how to's, etc.

    We will also hear about the Confluent Platform and topics like Kafka's Connect API and streaming data pipelines, Kafka’s Streams API and stream processing, Security, Microservices and anything else related to Apache Kafka.

    If you want to hear about a particular topic please let us know and we will try to find the best possible speaker. We are also very open to this being an interactive platform for the community, so if you would like to participate in other ways like speaking, hosting, writing a blog, etc, please let us know on

    Members (2,803)

    Photos (10)