Past Meetup

xPatterns on Spark, Shark, Mesos, & Tachyon

This Meetup is past

104 people went

Every 2nd Wednesday of the month

Location image of event venue

Details

Due to the popularity, we have moved the xPatterns on Spark, Shark, Mesos, and Tachyon session from Atigeo offices to the larger Expedia offices - thanks Expedia!

We will be in Expedia Building,[masked]th Ave NE, Bellevue, WA - in Conf Room Bellevue 1N-110. Parking is a bit limited in the area but there is street parking nearby.

Claudiu Barbura will be presenting on his and his team's experience for building the Atigeo data platform - called xPatterns - making use of Spark, Shark, Mesos, and Tachyon.

xPatterns is a big data analytics platform that enables rapid development of enterprise-grade analytical applications through built in apis and tools, driven from a management console with data, application and system monitoring. We will showcase the tools and APIs used for building multiple big data apps for largest production customer (20 billion healthcare records, 200 TB of compressed hdfs data) while evolving our infrastructure from Hadoop/Hive to Spark, Shark, Tachyon and Mesos. We will provide detailed ELT pipeline stats with performance comparison between Hive, Shark and Shark w/ Tachyon, live demos of: Jaws, our http Shark Server and GUI for exploring the data warehouse through Shark queries, Mesos providing resource management for multiple workloads, (Hadoop/Hive, Spark Job Server, Jaws, Aurora) the Export to NoSql API console (generates geo-replicated, instrumented and resilient apis for real-time access to Cassandra data exported from the warehouse through instrumented Spark jobs), monitoring and instrumentation (Graphite, Nagios) Lessons learned through our Spark journey, from 0.8.0 to 0.9.1, challenges (OOM!), advanced performance tuning and our own patches for the BDAS.

Agenda:

6:30pm: Come in and have some pizza

7:00pm: xPatterns Presentation

8:30pm: we should be done!

See all Meetups from Seattle Spark Meetup