Name: Toronto Apache Spark #4
Start: 2015-12-14T19:00:00-05:00
End: 2015-12-14T21:00:00-05:00
Location: Paytm Labs

Title:
Spark After Dark 1.5: Real-time, Advanced Analytics with Spark 1.5, Kafka, Cassandra, ElasticSearch, Zeppelin, and Docker by Chris Fregly (https://www.linkedin.com/in/cfregly)

NOTE: We will broadcast the session via hangout;

https://plus.google.com/u/1/events/ccit823ovv7vutg5mkpo3k7rm30

Agenda:

6:30PM to 6:50PM - Refreshment & Opening
6:50PM to 7:45PM - Spark After Dark 1.5 - Part 1
7:45PM to 8:00PM - Break (Refreshment provided by GuruLink)
8:00PM to 8:45PM - Spark After Dark 1.5 - Part 2
8:45PM to 9:00PM - QA

9:00PM to 9:05PM - 5 Bullet Points

Speaker:
Chris Fregly (https://www.linkedin.com/in/cfregly) is a Principal Data Solutions Engineer for the newly-formed IBM Spark Technology Center, an Apache Spark Contributor, a Netflix Open Source Committer, as well as the Organizer of the global Advanced Apache Spark Meetup and Author of the Upcoming Book, Advanced Spark. Previously, Chris was a Data Solutions Engineer at Databricks and a Streaming Data Engineer at Netflix.

When Chris isn’t contributing to Spark and other open source projects, he’s creating book chapters, slides, and demos to share knowledge with his peers at meetups and conferences throughout the world.

Topic:
Combining the most popular and technically-deep material from his wildly popular Advanced Apache Spark Meetup, Chris Fregly will provide code-level deep dives into the latest performance and scalability advancements within the Apache Spark Ecosystem by exploring the following:

1) Building a Scalable and Performant Spark SQL/DataFrames Data Source Connector such as Spark-CSV, Spark-Cassandra, Spark-ElasticSearch, and Spark-Redshift

2) Speeding Up Spark SQL Queries using Partition Pruning and Predicate Pushdowns with CSV, JSON, Parquet, Avro, and ORC

3) Tuning Spark Streaming Performance and Fault Tolerance with KafkaRDD and KinesisRDD

4) Maintaining Stability during High Scale Streaming Ingestion using Approximations and Probabilistic Data Structures from Spark, Redis, and Twitter's Algebird

5) Building Effective Machine Learning Models using Feature Engineering, Dimension Reduction, and Natural Language Processing with MLlib/GraphX, ML Pipelines, DIMSUM, Locality Sensitive Hashing, and Stanford's CoreNLP

6) Tuning Core Spark Performance by Acknowledging Mechanical Sympathy for the Physical Limitations of OS and Hardware Resources such as CPU, Memory, Network, and Disk with Project Tungsten, Asynchronous Netty, and Linux epoll

Resources:
This talk features many interesting and audience-interactive demos - as well as code-level deep dives into many of the projects listed above.

All demo code is available on Github at the following link:

https://github.com/fluxcapacitor/pipeline/wiki

In addition, the entire demo environment has been Dockerized and made available for download on Docker Hub at the following link: https://hub.docker.com/r/fluxcapacitor/pipeline/

5 Bullets:

Found something that blows you away? Submit it, so the rest of the community can revel in its awesomeness: http://goo.gl/forms/5K9Ua4kli2

------------------------------------------------------------
Sponsors:
Food & Drinks provided by GuruLink

http://gurulink.ca/img/logos/gurulinkbanner2_jpg.jpg

The event is hosted by PaytmLabs

http://paytmlabs.com/wp-content/themes/PaytmLabsLayout/img/paytm_labs_logo_inline.png

Mehrdad Pazooki

PipelineAI Advanced Spark and TensorFlow Meetup (Toronto)

Technology

High Scalability Computing

Data Visualization

Python

Scala

Hadoop

TensorFlow

Deep Learning

Machine Learning

Data Analytics

Data Science

Big Data

Apache Spark

Artificial Intelligence

Apache Kafka

Toronto Apache Spark #4

Paytm Labs

Share

PipelineAI Advanced Spark and TensorFlow Meetup (Toronto)

Toronto Apache Spark #4

PipelineAI Advanced Spark and TensorFlow Meetup (Toronto)

Details

Members are also interested in