How to Share State Across Multiple Spark Jobs using Apache Ignite

This session will demonstrate how to easily share state in-memory across multiple Spark jobs, either within the same application or between different Spark applications using an implementation of the Spark RDD abstraction provided in Apache Ignite (https://ignite.apache.org/).

During the talk, attendees will learn in detail how IgniteRDD (https://ignite.apache.org/features/igniterdd.html) - an implementation of native Spark RDD and DataFrame APIs - shares the state of the RDD across other Spark jobs, applications and workers. Examples will show how IgniteRDD allows execution of SQL queries many times faster than native Spark RDDs or Data Frames due to its advanced in-memory indexing capabilities.

Presenter:
Akmal Chaudhri, PhD., is a Technology Evangelist for GridGain Systems (https://www.gridgain.com/). His role is to help build the global Apache Ignite community and raise awareness through presentations and technical writing. Akmal has over 25 years experience in IT and has previously held roles as a developer, consultant, product strategist and technical trainer. He has worked for several blue-chip companies such as Reuters and IBM, and also the Big Data startups Hortonworks (Hadoop) and DataStax (Cassandra NoSQL Database).