First Tachyon Meetup

Welcome to the first Tachyon meetup! This will be a chance to learn about Tachyon from the developers, hear about other peoples’ experiences with Tachyon, network, and get to know future development plans.

Start time: 6:30pm (food and drinks thanks to Yahoo), talks start at 7pm.

Abstract: 

Memory is the key to fast Big Data processing. This has been realized by many, and frameworks such as Spark and Shark already leverage memory performance. As data sets continue to grow, storage is increasingly becoming a critical bottleneck in many workloads.

To address this need, we have developed Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce. The result of over two years of research, Tachyon achieves memory-speed and fault-tolerance by using memory aggressively and leveraging lineage information. Tachyon caches working set files in memory, and enables different jobs/queries and frameworks to access cached files at memory speed. Thus, Tachyon avoids going to disk to load datasets that are frequently read.

Tachyon is Hadoop compatible. Existing Spark and MapReduce programs can run on top of it without any code changes. Tachyon is the default off-heap option in Spark, which means that RDDs can automatically be stored inside Tachyon to make Spark more resilient and avoid GC overheads. The project is open source and is already deployed at multiple companies. In addition, Tachyon has more than 40 contributors from over 15 institutions, including Yahoo, Intel, Redhat, and Pivotal. The project is the storage layer of the Berkeley Data Analytics Stack (BDAS) and also part of the Fedora distribution.

In this meetup, Haoyuan Li will give a overview of the project, including motivation, current status, and its roadmap. In addition, we will have a Tachyon tutorial.

Bio:

Haoyuan Li is a Computer Science Ph.D. candidate in AMPLab at UC Berkeley, and he works with Prof. Scott Shenker and Prof. Ion Stoica on big data and cloud computing. He leads Tachyon, an open source memory-centric distributed file system enabling reliable file sharing at memory-speed across cluster frameworks. He is a founding committer of Apache Spark and a co-creator of Spark Streaming. Before Berkeley, he worked at Conviva and Google, where he co-created PFP-Growth algorithm, which is included in Apache Mahout. Haoyuan has a M.S. from Cornell University and a B.S. from Peking University, both in Computer Science.

Future Meetups: The meetup will rotate among locations in San Francisco, Silicon Valley and Berkeley.

Join or login to comment.

  • Henry S.

    Very good introduction of Tachyon to start the meet up

    August 26

  • Burt P.

    Fantastic first meetup on Tachyon! Very excited about learning and using it!

    Excellent presentation by Haoyuan. Also, he was super knowledgeable and patient with our questions.

    Many thanks to Yahoo for hosting and the great food!

    Can't wait for the next meetup!

    August 26

  • tom p.

    yes

    August 26

  • Anoop D.

    Is there going to be live streaming?

    2 · August 24

    • Haoyuan L.

      Unfortunately, we didn't have a live streaming. Will try to have it in the future meetups.

      August 26

  • Haoyuan L.

    Thanks everyone for coming! I've posted my slides at http://files.meetup.com/14452042/Tachyon_Meetup_2014_8.pdf Also, we would appreciate it if you could take a moment to complete a short survey: https://docs.google.com/forms/d/1qSAcpRWytpRuek2MTTrifnFXhx71v0iJuhXSem3Mn1s/viewform?usp=send_form

    Look forward to seeing you at our future meetups!

    1 · August 26

  • Pengfei X.

    Wish I was over there.

    August 25

  • Qifan P.

    It was amazing (food, talk, etc.). Thanks all organizers for the work. Looking forward to the next one.

    1 · August 25

  • Zia S.

    What is the correct address for this? I am at Building E, 700, and there is no public access and security here isn't aware of the event either

    August 25

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Henry

I decided to start Reno Motorcycle Riders Group because I wanted to be part of a group of people who enjoyed my passion... I was excited and nervous. Our group has grown by leaps and bounds. I never thought it would be this big.

Henry, started Reno Motorcycle Riders

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy