Reference Architecture for In-Stream Processing Service using Spark Streaming

Speaker Bio

Victoria Livschitz is a founder and CTO of Grid Dynamics, the engineering services company that specializes, amongst other things, in the design of big data, real-time analytics and machine learning applications using open source technologies running in a cloud-portable immutable infrastructure. Victoria will present the reference architecture for In-Stream Processing Service that designed with 100% open source components and runs on any cloud. Reference Architecture for In-Stream Processing Service, Illustrated by Real-time Twitter Sentiment Analytics application (Victoria Livschitz, Founder/CTO of GridDynamics)

Stream processing architectures are rapidly emerging in various business domains to offer real-time ML and processing applications such as fraud detection, risk management, sentiment analytics and many more.

Topics presented will include:

• Introduction to In-Stream Processing

• Introduction to Real-time sentiment analysis of Twitter streams applications

• Overview of Reference Architecture (RA) for ISP using Kafka/Spark Streaming/ Cassandra/Redis/HDFS

• Overview of Reference Implementation (RI) and devops stack for portable cloud deployment using Docker, Mesos/Marathon, and Ansible • Demonstration of all technologies at work