Nathan Marz - The inherent complexity of stream processing

The inherent complexity of stream processing.

Nathan Marz, creator of Storm project.

Abstract: In the beginning of this talk, a seemingly straightforward data problem will be presented. However, as will become readily apparent, the solution to this problem is not so easy – at least, not when you shackle yourself to traditional techniques to building data systems. Fully incremental approaches, whether using a relational database or a modern stream processing system coupled with NoSQL databases, are inherently problematic and complex.

You will see how to solve this data problem by making use of different techniques. The solution is easy to understand and the pattern behind it can be readily applied to any other data problem. The key is to introduce fundamental simplicity into the data architecture – that is, to make separate functions that were previously inexorably intertwined. By making these functions independent, you avoid the inevitable conflicts which make solutions difficult or impossible to build.

About speaker:

http://photos3.meetupstatic.com/photos/event/6/e/2/3/600_434728195.jpeg

Nathan Marz is currently working on a new startup. Previously, he was the lead engineer at BackType before being acquired by Twitter in 2011. At Twitter, he started the streaming compute team which provides and develops shared infrastructure to support many critical realtime applications throughout the company.

Nathan is the creator of Storm, open-source project which are relied upon by over 50 companies around the world, including Yahoo!, Twitter, Groupon, The Weather Channel, WebMD and many more companies.

Nathan is also working on a book for Manning publications entitled “Big Data: principles and best practices of scalable realtime data systems (http://www.amazon.com/gp/product/1617290343/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=1617290343&linkCode=as2&tag=seasartandcra-20&linkId=HKIBV7AXPSDOPIEK)” due out for publication this month.

http://photos2.meetupstatic.com/photos/event/b/4/6/7/600_434806183.jpeg

Agenda:

6 :30 pm: Guest arrival & networking.

7:00 pm: The inherent complexity of stream processing.

8.30pm: Event ends.

P.S. There will be two Big Data events in New York in March. One is Structure Data. This year Structure Data will cover and discuss everything big data from machine learning, AI, analytics and virtualization to healthcare, new media, security and more. Speakers include AWS, Cloudera, Google, Facebook, Spotify, the FTC, NASA, and more. We will also be featuring some of the best big data startups on stage. Get $100 off ticket prices by using this link: http://bit.ly/SD15MEETUP

Second is Spark summit East. For those of you who is interested in Spark in addition to Storm, we were able to secure 20% discount for our group for upcoming Spark East conference. Please use this link (http://goo.gl/wCOF8C) and discount code: SSESTORM20.

http://spark-summit.org/wp-content/uploads/2014/08/logo-2015e.jpg