Scaling Apache Storm pipeline and building real-time Market Data Platform

Dear friends, Happy Thanksgivings!

This time we will have two interesting talks during our meetup:

1. How Spotify scales Apache Storm pipeline.

Kinshuk Mishra, Tech Lead at Spotify NYC

Abstract:

Spotify has built several real-time pipelines using Apache Storm for use cases like ad targeting, music recommendation, data visualization, and notifications. Each of these real-time pipelines have Apache Storm wired to different systems like Apache Kafka, Apache Cassandra, Apache Zookeeper, and other sources and sinks. In this talk the speaker, Kinshuk Mishra, will share his experiences of scaling Apache Storm pipelines at Spotify. The talk will cover the topics such as Spotify's data architecture, best practices, caching, tuning event sources and sinks, monitoring pipeline health and miscellaneous optimizations to make Apache Storm pipelines more robust.

Speaker: Kinshuk Mishra (http://www.linkedin.com/in/kinshukmishra), @_kinshukmishra (https://twitter.com/_kinshukmishra), is a Tech Lead at Spotify NYC. He has been wrangling data for Spotify since 2011 and currently works on user personalization and ads optimization problems.

2. Market Data Platform using Apache Storm.

Infant Rosario Vasanth, Software Engineer @BlackRock and Arjun Rao ,Software Engineer @BlackRock

Abstract:

There has been a paradigm shift in the way security and market data is being created and consumed. Market data delivery mechanisms are moving towards stream based systems, there's been an exponential increase in the velocity/volume/variety of streaming market data. Traders, portfolio managers, analysts increasingly need to analyze this data to know the market better and find their liquidity options when executing orders. Investors need visibility to the latest prices and historical trends for instruments like equities, fixed-income products, derivatives etc. to make sound investment decisions. We are designing and building the next generation market data platform at BlackRock using Apache Storm.

In this meetup, Infant Rosario Vasanth and Arjun Rao , will discuss some of the use cases, lessons learned while designing and deploying Apache Storm at BlackRock.

Speakers: Infant Rosario Vasanth, Software Engineer @BlackRock (https://www.blackrock.com/)

Arjun Rao ,Software Engineer @BlackRock

Agenda:

6 :30 pm: Guest arrival & networking.

7:00 pm: How Spotify scales Apache Storm pipeline

7: 30 pm: Market Data Platform using Apache Storm.

8.00 pm: Event ends.

Thanks

Eugene Dvorkin

20% off Strata/Hadoop world with this link (http://www.jdoqocy.com/click-7592525-12000083) and code AFF20

http://photos3.meetupstatic.com/photos/event/8/f/1/3/600_431916627.jpeg