Storm, Spark Streaming. Prometheus Monitoring. Spark/Akka for Data Generation

HUG Ireland has the pleasure of announcing its final event of 2015, which is hosted by Accenture, where we will enjoy three great talks from accomplished Academics and Industry Practitioners. On the agenda is a fine blend of experience sharing by Wu on Storm/Spark Streaming, Brian on network and cluster monitoring with Prometheus along with Floran on using Spark, Akka and Kafka to build a country level telecoms data synthesis platform for real time analytics.

The agenda is as follows:

Real time analytics with Storm and Spark Streaming by Guangyu Wu, Post Doctoral Researcher with CeADAR

High volume, high velocity streaming data is now commonplace in many industries and real-time analytics over these types of live data streams is gaining increased interest. Apache Spark Streaming and Apache Storm are distributed stream processing frameworks which can be applied for these tasks. CeADAR (Centre for Applied Data Analytics Research) has developed several industrial research projects using Spark and Storm, such as fraud detection in financial transactions and spam identification in SMS traffic which showcase these frameworks.

Monitoring Hadoop et al with Prometheus by Brian Brazil, Founder of Robust Perception

The Prometheus monitoring system was publicly announced at the start of this year, and offers a new perspective on gaining insight into and managing your systems. In this talk we shall take a look at the Prometheus perspective on monitoring and alerting on batch jobs such as those common in Hadoop. These techniques will let you reduce email spam (aka notifications), get actionable alerts and help you sleep better at night.

Building a country-scale Telecom simulator using Akka, Spark and Kafka by Floran Hachez

Real Impact Analytics, creates products which capture the value in telecom data. To build great products, we need data to benchmark, test and stress test our solutions, but in the telecom industry, data are private and hard to get. A solution is to simulate a real world telecommunication provider at a country-scale. Our approach is to “simulator simulate” a network of millions of users in order to generate synthetic data. Let's learn more about Akka and see how we built such a simulator using Akka, Spark and Kafka.

As usual, our event hashtag is #HUGIreland and we look forward to seeing you all at Accenture on 14th of December for our end of year event.