Building Streaming Data Pipelines with Kafka, Elastic and kSQL

Details

Join us for a meetup co-hosted by Confluent & Hotstar Tech on Wednesday at January 9th from 6:00 PM at Hotstar Bangalore office. The address, agenda and speaker information can be found below:

Schedule
6:00 PM: Registration & Networking
6:30 PM: Building Streaming Data Pipelines with Kafka, Elastic and kSQL by Mark Teehan
7:30 PM: Bifrost - Data Management Service at Hotstar by Bhavesh Raheja & Navdeep Agarwal

---

Building Streaming Data Pipelines with Kafka, Elastic and kSQL

Mark Teehan is a Sales Engineer from Confluent. Mark will be speaking about the basics of Apache Kafka, data streaming, the core principles of streaming platforms (streams, tables etc.), and how the Elastic stack interacts with Kafka.
In his day-to-day work, Mark engages with customers to evaluate business and technical requirements, helps guide technical teams while conducting POCs and acts as a liaison between customers and the engineering teams in Confluent. Prior to Confluent, Mark lead the HANA proof of concept team in SAP in Singapore, where he lives. His first role in Singapore was on the tech team implementing servers, databases, cards and devices for EZ-Link, the national public transit smartcard ticketing system

---

Bifrost - Data Management Service at Hotstar
Bifrost is an attempt at democratizing and abstracting production and consumption of Data at Hotstar. It is an opinionated platform that encourages best practices for ingesting data, complying with necessary data governance, and sharing data with internal and external services.

It’s a service that makes it easy to ingest, process, store, and share data. It abstracts underlying complexities in producing, consuming, and processing data. It uses Knol (Hotstar’s Realtime Stream Ingestion platform) as the single platform for ingesting all kinds of incoming data and uses Hotstar's Data Lake to store all kinds of stationary data.

Bhavesh Raheja and Navdeep Agarwal will talk about the journey of building Bifrost and the challenges faced in buiding a platform that can ingest at the rate of 1M RPS and provide enriched data to serve the business and analytical needs of Hotstar.