Spark Streaming - Adventures by AppsFlyer

Target audience:
Data Developers, Data architects, Data Scientists and everyone who like Spark Streaming

Abstract:

Despite being a relatively new technology, Apache Spark has been adopted by thousands of companies and educational institutions not only for it's performance and API richness, but also for the powerful machine learning library it provides. Spark Streaming, the streaming component of Apache Spark, is however much newer and more mysterious, even when compared with established stream processing giants such as Storm and Flink.

In this session we'll go beyond the typical "world count" streaming example to cover different event processing scenarios, including "exactly once" semantics. We'll also talk about configuration aspects that should be taken into consideration when deploying a streaming application.

Stream processing is inevitable in any data driven company aiming to provide its customers with immediate insights and AppsFlyer is no exception. In the second part we'll present our Spark driven real-time dashboard application running in production, and discuss the challenges we've been through during its development

What you will learn:
• Spark Streaming technology - its usages, alternatives and more
• Go beyond the simple use case and understand when and how did we chose to use it for our product

Presenters names :
Michael Spector, Data tech lead, https://il.linkedin.com/in/spektom
Avner Schwartz, Software Developer, https://il.linkedin.com/in/avners