Next Meetup

How Tivo Uses Presto
What is Presto? Presto is an open source distributed SQL engine for running fast analytic queries against various data sources ranging in size from gigabytes to petabytes. In this talk, I describe how we use the open source Apache-licensed Presto project in TiVo's new Targeted Audience Delivery platform. First I explore the use cases specific to our platform that we evaluated Presto on and compare them to other similar technologies. In particular I describe the benchmarks we used and how they compare to Amazon's Redshift Spectrum and Amazon Athena. Having chosen Presto, I go over the benefits of using Presto and give a brief overview of Presto's architecture. From there I explore a few important technical challenges we faced as new users of Presto. I first show how we chose our instance types on EC2 for our clusters using Pareto efficiency. I then show how we integrated Presto with our existing infrastructure. Finally, I highlight some of the unforeseen benefits and challenges of picking Presto that eluded our initial evaluation of the technology including key open technical issues with the project that impacted our use of it. After an exploration of the Presto technology, I talk about how we use Qubole to manage the deployment, scaling, auditing, and configuration of our Presto clusters. In particular I focus on how we integrated our existing code with Qubole's API and how we adjusted our usage of Qubole for certain performance-critical workloads. In conclusion, I share some of the wisdom gained from this experience for others that may be considering using Presto. About the Speaker: Lucas Waye is a Principal Engineer at TiVo. He currently maintains and enhances TiVo's data analytics architecture. Previously he worked on the backend for Timeful (acquired by Google) and on Hulu's analytics platform. He has also consulted for various companies, helping them improve their software engineering practices and received DARPA's Rising Star Award for some of his consulting work. His academic research focuses on language-based solutions to problems in distributed systems and security. He received a Ph.D. and Master's degree from Harvard and a Bachelor's degree from Cornell.


300 A Street Suite 500 · Boston, ma

Upcoming Meetups

What we're about

Public Group

This meet up is for developers, analysts, architects, data scientists, data artists, data hackers and anyone else using the tools within the Hadoop ecosystem or interested in learning about it.

Members (3,204)

Photos (127)