Keeping Spark on Track: Best practices using Apache Spark in Production

http://photos2.meetupstatic.com/photos/event/6/b/1/f/600_457947423.jpeg

This event is catered thanks to the good people @databricks

Miklos Christine is a solutions architect for Databricks where he helps customers deploy and use Apache Spark to build batch and streaming applications. Miklos was previously a systems engineer at Cloudera where he helped strategic customers deploy and use the Apache Hadoop ecosystem in production. He has contributed to several projects in the open source community and holds a BS in electrical engineering and computer sciences from the University of California-Berkeley.

The purpose of this talk is to share best practices learned while developing Apache Spark workflows in production across various industries. Apache Spark is a popular distributed processing framework that allows organizations to analyze multiple streams of data for machine learning and exploratory SQL workloads. I'll discuss debugging tips, best practices for transforming datasets, and discuss integrating with existing libraries such as numpy and pandas. Code examples will be shared in python.