Julia and Big Data - Aviva/Spark/Hadoop/Hive

So autumn has well and truly arrived, winter is surely not far behind. Our meetup in November will hopefully warm our hearts and minds. We will discuss how easy it is to interface Julia with traditional big data tools. There will be a talk discussing Aviva's large scale Julia+Spark implementation, followed by a hands on demonstration of using Julia with Spark, Hadoop and Hive.

1. Large scale credit analytics with Julia and Spark

In this session, John Adcock will speak about the experiences of his team in building Aviva's portfolio credit risk system using Julia and Spark. He will discuss their reasons for choosing Julia for the compute and Spark for data transfer and distribution. He'll also talk about the lessons learned in developing, testing and deploying this system.

John joined Aviva as Head of Credit Calibration in 2016. Before Aviva John had been building systems for over 20 years, mostly improving risk management systems for banks and insurance companies. John enjoys contributing to open source projects particularly video processing and home automation as well as working with start-ups on image recognition and machine learning.

2. Using Julia with Spark, Hadoop, Hive and friends

A hands on demo by Avik Sengupta, exploring the use of Julia based compute on the Spark and Hadoop ecosystem. We will show how to set up a Spark cluster with Julia support, and demonstrate the Julia interface libraries to Spark, Hadoop and Hive.

Avik is the VP of Engineering at Julia Computing, and a long time contributor to various part of the Julia ecosystem.

Schedule:
18:00 - Registration, Pizza & Networking
18:30 - Large scale credit analytics with Julia and Spark
19:15 - Break
19:30 - Using Julia with Spark, Hadoop, Hive and friends
20:15 - End / Retire to Pub

Image Credit: Generative Art with Julia, by The Amazing @cormullion. https://flic.kr/p/Q12r1y