Apache Spark Streaming with Apache Flume

Hello everyone and happy new year! I hope 2015 is off to a great start for you all.

While putting together the calendar for the upcoming months, I tried to keep in mind the topics you've mentioned wanting to focus on. One request that's come up a few times is to have meetups focused on architectures for processing large volumes of data.

I connected with a number of individuals who've given similar presentations in the Boston area -- there are so many great and talented people in the Manchester/Boston area -- and I am excited to have Abhinav Garg making the trip up to Manchester to give this presentation.

Apache Spark Streaming with Flume

In this meetup, Abhinav Garg will be discussing Apache Spark Streaming with Apache Flume.

Apache Flume (https://flume.apache.org/) is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. Here we explain how to configure Flume and Spark Streaming to receive data from Flume.

Apache Spark (https://spark.apache.org/docs/latest/index.html) is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala and Python, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL (https://spark.apache.org/docs/latest/sql-programming-guide.html) for SQL and structured data processing, MLlib (https://spark.apache.org/docs/latest/mllib-guide.html) for machine learning, GraphX (https://spark.apache.org/docs/latest/graphx-programming-guide.html) for graph processing, and Spark Streaming (https://spark.apache.org/docs/latest/streaming-programming-guide.html).

Abhinav Garg, Senior Manager, Trading and Risk Management

Abhinav currently leads a globally-distributed development team on a long-term consulting engagement with a global asset management firm. He oversees the delivery of a reference data platform, which includes integration with external data providers and internal consumers. In past,Abhinav has been involved in the development of backoffice trade settlement and confirmation systems, and has prior experience in implementing mid-office solutions for Europe’s largest online travel firm.