LinkedIn’s Big Data Pipeline with Kafka, Hadoop and Couchbase

Tracking user events as they happen with a rate up to millions of operations per second can challenge any business providingreal time user interaction. The solution requires both huge scale and a lot of processing to support dynamic adjustment to targeting products and services.

Couchbase accommodates fast access to user profiles at scale while leveraging Kafka to stream data to Hadoop for deepanalytics. As the operational data store Couchbase data services are easily capable of processing tens of millions of updates a day, or more. Streaming through Kafka into Hadoop, these key events are turned into business insight.

Will GardellaDirector of Product Management for Analytics, Connectors, and Search at Couchbase.

Previously, Will was a product manager in the big data platform team at HP, a Senior Director of Product Management for SAP HANA, and the Senior Director of SAP Research's global Big Data program focused on big data and machine learning.

Michael KehoeEngineer at LinkedIn

Michael specialises in maintaining large system infrastructures as demonstrated by work at LinkedIn (applications) and at The University of Queensland (networks). He possesses high-level skills in maintaining Linux and Windows servers and their respective infrastructure services. His interpersonal skills allow him to interact with clients and colleagues in a professional manner using exemplary communication skills.