Understanding Spark DataFrames and SparkSQL

Gil Zhaiek is a Vancouver-based developer, working with Databricks and NewCircle to deliver public and private training for Spark. In this Meetup presentation, he will touch on a wide range of Spark topics:

• Introduction to DataFrames

• The Catalyst Optimizer

• DataFrames vs. RDDs

• Spark SQL

• Transformations, Actions, Laziness

• Schemas

• UDFs and UDAFs

• Window Functions

• Limitations

Gil with provide examples and a demonstration using PySpark for this presentation. This will be a great session for both new and experienced Spark users.

Schedule:

6:00-6:30 Networking

6:30-7:30 Presentation

7:30-8:00 Wrap and wind down.