Spark in Production at IBM

A deep dive into core Apache Spark performance

Speakers:

Tim Ellison : Technical Leader, IBM Runtimes

Adam Roberts : Spark Team Lead, IBM Runtimes

Bio: Adam leads the development of Apache Spark within IBM's Runtimes organisation, specialising in improving both the Spark and Java runtime by bringing a knowledge of hardware acceleration techniques, IBM systems, a deep understanding of Java internals, profiling for performance and contributing to open source projects. He has presented at a variety of conferences and most enjoys helping people to achieve their own goals. Adam's current priorities are with improving Apache Spark performance while ensuring all tests run clean on IBM systems. As well as helping large IBM customers use Spark for the first time, he is regularly praised for his enthusiasm and drive in getting jobs done and has been involved in several high profile demos featuring Spark. Adam has voluntarily taught Year Six children since he started at IBM in 2013 upon completing his degree -- his current group's topic being to teach machine learning concepts using IBM's Watson services on Bluemix.

Apache Spark in production at Wimbledon Tournament 2016

Speaker:

Adelina Balasa : Technical Consultant, IBM Cloud

Abstract: The Summer of 2016 was unusual in sporting terms, with several events taking place simultaneously (Euro 2016 Football Championships, 2016 Copa América, the Tour de France, two F1 Grand Prix and the build up to the Rio Olympics). The packed schedule presented a challenge to Wimbledon, in vying for sports fans attention, but also an opportunity to draw in fans from other sports. In order to make their own editorial content relevant, Wimbledon needed to understand what tennis and general sports fans were talking about and where their interests lay.

The Cognitive Social Command Centre (CSCC) used IBM's Watson technology & the Spark platform to identify, in real-time, emerging topics of conversation around Wimbledon and the other sports events. It highlighted which of those topics had the potential to perform best and what content was being engaged with now.

By using a Cognitive solution that was trained not programmed, the system was continually improving throughout the Championships, learning what was being said about Wimbledon and the other sporting events.

Bio: Adelina has been with IBM for 3 years and during this time she’s developed a very broad set of skills, from technical expertise to project management to creative work.

After finishing a degree in Computer Science & AI, she spent her first 2 years at IBM as a Technical Consultant in GBS with a focus on the financial sector, performing both technical roles such as Design Architect for RBS and creative roles such as UX/UI Designer for RBS and Barclays. Adelina left the consultancy work in summer 2015 and decided to make better use of her AI degree by joining the big data & analytics team in IBM's Emerging Technologies department; ever since she’s been undertaking the Data Scientist role in Big Data projects for a varied number of customers, using mostly Big Data tools such as BigInsights, Spark, Hadoop, Hive and BigSQL and a variety of languages, R, Python, Scala and SQL.