Introduction to Spark

Apache Spark (https://spark.apache.org/) is a fast and general engine for large-scale data processing. In contrast to Hadoop (https://en.wikipedia.org/wiki/Hadoop)'s two-stage disk-based MapReduce (https://en.wikipedia.org/wiki/MapReduce) paradigm, Spark's in-memory primitives provide performance up to 100 times faster for certain applications.

The Spark software stack includes a core data-proccessing engine, an interface for interactive querying, Sparkstreaming for streaming data analysis, and growing libraries for machine-learning and graph analysis. Spark is quickly establishing itself as a leading environment for doing fast, iterative in-memory and streaming analysis.

This talk will give an introduction the Spark stack, explain how Spark has lighting fast results, and how it complements Apache Hadoop.

Presented by Carol McDonald, MapR Technologies