Twitter Heron in Practice

http://photos3.meetupstatic.com/photos/event/6/0/4/a/600_453324650.jpeg

Twitter generates billions and billions of events per day. Analyzing these events in real-time presents a massive challenge. To meet our scaling and operational needs, Twitter designed and deployed a new streaming system called Heron, our next generation streaming system.

In this talk, we will provide an introduction to Heron, how it is being used at Twitter and share our operating experiences and challenges of running Heron at scale. We recently announced our open sourcing of Heron under the permissive Apache v2.0 license.

Heron has been in production nearly 2 years and is widely used by several teams for diverse use cases. Prior to Heron, Twitter used Apache Storm, which we open sourced in 2011.

Heron features a wide array of architectural improvements and is backward compatible with the Storm ecosystem for seamless adoption.

Following the talk, we will provide a hands-on experience with Heron. Bring your laptops and we will show you how to install, use, and operate Heron.

Speaker:

Karthik Ramasamy (https://www.linkedin.com/in/kramasamy)

https://a248.e.akamai.net/secure.meetupstatic.com/photos/event/5/f/9/c/600_453324476.jpeg

Karthik is the engineering manager for Real Time Analytics at Twitter and co-creator of Heron. He has two decades of experience working in parallel databases, big data infrastructure and networking. He cofounded Locomatix, a company that specializes in realtime streaming processing on Hadoop and Cassandra using SQL that was acquired by Twitter. Before Locomatix, Karthik was at Juniper Networks where he designed and delivered platforms, protocols, databases and high availability solutions for network routers that are widely deployed in the Internet.

Before Juniper, at University of Wisconsin, he worked extensively in parallel database systems, query processing, scale out technologies, storage engine and online analytical systems. Several of these research were spun as a companies later acquired by Teradata. He is the author of several
publications, patents and one of the best selling book "Network Routing: Algorithms, Protocols and Architectures."

He has a Ph.D. in Computer Science from UW Madison.

AGENDA:

6:30 - 7:00 - Registration and networking

7:00 - 7:45 - Heron Introduction - Karthik Ramasamy

7:45 - 8:30 - Hands on Heron