A view on the world of streaming: use cases, challenges and solutions

Agenda:

• 6:30 – 6:40 Welcome

• 6:40 – 7:10 Talk 1: Flink in Production - Lessons Learned. Presenters: Maximilian Bode, Konstantin Gregor

• 7:10 – 7:40 Talk 2: SQL on Streams powered by Apache Flink and Apache Calcite. Presenter: Dr. Radu Tudoran

• 7:40 – 8:10 Talk 3: Practical considerations of implementing streaming use-cases on Fast-Data Architectures. Presenter: Matthias Hofschen

• 8:10 – 8:40 Talk 4: The ‘A’ in Smack - Akka for Fast Data Presenter: Lutz Huehnken

• 8:40 – 9:00 Open discussions & Networking (drinks and cookies)

Abstracts:

Talk 1: Flink in Production - Lessons Learned.

In this short talk we want to point out some insights we gained using Apache Flink in production. After having used it for quite some time now, we share some strategies we found helpful. Two of the points we are going to cover are resilience and monitoring. Furthermore, we touch on delivery guarantees and the integration of Flink with other Big Data frameworks.

Talk 2: SQL on Streams powered by Apache Flink and Apache Calcite.

Streaming becomes nowadays an increasingly popular paradigm to handle the massive amounts of data that are being continuously produce from a plethora of sources. Although that the stream data analytics paradigm offers a rich semantic for doing business intelligence, SQL remains from historical reasons, the universal language for data analytics, query and data minding. Although there exists conceptual differences between stream API and SQL, there are ongoing efforts in the community to bring these together and enable stream analytics via SQL. In this talk we are going to discuss these efforts and the architectural solutions through which this can be achieved. As a case study, we will exemplify with the Apache Flink platform.

Talk 3: Practical considerations of implementing streaming use-cases on Fast-Data Architectures.

This talk shares my team's journey of the last 2,5 years: from building a by-the-book lambda architecture implementation to moving towards an increasingly stream-based Fast-Data architecture. We'll talk about how we solved the technical challenges of implementing customer use-cases within this architecture (while processing billions of events per day) and end with a comparison between storm and flink, from our point of view.

Talk 4: The ‘A’ in Smack - Akka for Fast Data.

In this talk we will explore which role the actor toolkit for the JVM, Akka (http://akka.io (http://akka.io/) ), can play in your Fast Data architecture. We will have closer look at Akka Streams ( http://doc.akka.io/docs/akka/current/scala/stream/#streams-scala ), an implementation of the Reactive Streams specification (http://www.reactive-streams.org (http://www.reactive-streams.org/) ), and how they can be used to complement Spark. What are Akka Streams, and just as importantly, what are they not? We will discuss some use cases and how to combine them with Spark Streaming.

Presenters:

Maximilian Bode - Max is an IT consultant working for TNG Technology Consulting in Munich. He helps our customers build better software and is currently focusing on big data applications.

Konstantin Gregor - Konstantin works for TNG Technology Consulting in Munich where he builds big data applications. He is interested in machine learning and image processing.

Radu Tudoran - Radu is acting as the Big Data expert at HUAWEI’s European Research Center with a focus on advanced technologies and innovation in the area of Big Data processing (stream analytics, low latency processing, high performance data management). Prior to joining Huawei, Radu has worked as doctoral researcher at INRIA Rennes and ENS Rennes, France, in the area of high performance Big Data processing across cloud data centers.

Matthias Hofschen - Matthias is a software engineer at Teradata Marketing Applications. He has implemented and maintained Java-based Saas solutions in digital marketing for 15 years. He is currently working on a data platform for processing billions of events to power advanced use-cases in digital marketing.

Lutz Huehnken - Lutz ( http://www.huehnken.de ) is a Solutions Architect at Lightbend. He has worked in professional software development since 1997 and successfully deployed major web applications for clients in different fields (retail, logistics, hospitality, finance). His current focus is on the development of reactive applications – responsive, scalable, resilient systems – with Scala, Akka and Play.

Location:

The meetup will be hosted in Huawei ERC officeon Riesstr. 25, Munich. The meeting room is located in tower D at the 3rd floor

You can reach the office by public transportation by U1 or U3 lines (Olympia-Einkaufszentrum station) and walk from there for 5 minutes

The meetup is announce and jointly organized between Apache Flink meetup group and Big Data Stream Analytics meetup group. One registration is sufficient