Running Spark Clusters in Containers with Docker

This session will examine the many options that data scientists have for Spark-as-a-Service, whether in a public cloud environment or on-premises. We will discuss various environments and options – including Amazon Web Services and BlueData EPIC software – with the benefits and challenges of each approach. In particular, we will discuss how to spin up instant Spark clusters using Docker containers.

About the Speaker:

Thomas Phelan

Tom has spent the last 25 years as a senior architect, developer, and team lead in the computer software industry in Silicon Valley. Prior to co-founding BlueData, Tom spent 10 years at VMware as a senior architect and team lead in the core R&D Storage and Availability group. Most recently, Tom led one of the key projects – vFlash, focusing on integration of server-based Flash into the vSphere core hypervisor. Prior to VMware, Tom was part of the early team at Silicon Graphics that developed XFS, one of the most successful open source file systems. Earlier in his career, he was a key member of the Stratus team that ported the Unix operating system to their highly available computing platform

Kartik Mathur

Kartik is software engineer at BlueData with more than 3 years of industry experience working with Hadoop ecosystem , Spark, Cassandra and related Big Data technologies.. At BlueData, Kartik is responsible for integrating Spark and various Big data applications with the EPIC product. Prior to BlueData, Kartik worked at Indaco Inc where he architected and owned their data and ETL pipelines. He contributed to Indaco's open source cassandra spark JDBC (CSJB) efforts for querying Cassandra databases and running real time analytics with Spark . Before Inadco, Kartik worked for the AMD Grid Compute Team where he was introduced to the Hadoop ecosystem while working on data center real time reporting tool.

Agenda:

6:30 - 7:00pm Check in & social (Pizza will be provided)

7:00 - 8:00pm Talk and Q&A

8:00 - 8:30pm Social