Processing Changes from Couchbase in Kafka at PayPal

Speakers: Matt Ingenthron, (https://twitter.com/ingenthr) Shibi Sudhakaran

Description: Running a site like PayPal requires both a huge scale and a lot of processing. Couchbase is the perfect fit for the scale and to additional processing of the millions of updates flowing a day on a site like PayPal, Apache Kafka’s high throughput capabilities are in order.

At this month’s meetup, Matt Ingenthron of Couchbase and Shibi Sudhakaran of PayPal will introduce the recently introduced, experimental paypal/couchbasekafka project. Shibi and Matt will give an overview of the problem domain. In a word, it’s “scale”. Apache Kafka has a unique approach allowing it to keep up with the huge data feed coming out of a Couchbase deployment. Shibi will talk about what makes Kafka unique in it’s ability to handle high throughput.

Matt will then talk about how this integration has been achieved currently, and where Couchbase is going with future data integration. Couchbase has long had a number of methods for both intra-cluster replication and cross-cluster replication, as each deployment topology has its own needs. Matt will talk about how this is handled internally in Couchbase today, how PayPal’s experimental Kafka adapter receives data, and how this will likely be even more robust in the future using the Database Change Protocol recently introduced in Couchbase 3.0.

Schedule:

6:30 - 7:00p Meet and discuss over drinks and appetizers
7:00 - 7:40p PayPal on Couchbase and Kafka for Processing Changes
7:40 - 7:50p Short break
7:50 - 8:30p Couchbase Data Streaming for Kafka and other Integrations