Past Meetup

October 2015 Meetup

This Meetup is past

200 people went

Location image of event venue

Details

Agenda:

• 6-630pm -- Networking

•[masked]pm -- Presentation: Resolving Transactional Access/Analytic Performance Trade-offs in Apache Hadoop

•[masked]pm -- Breakout sign-ups

•[masked]pm -- Breakouts

Thank you to the General Assembly team for hosting us again. We will have one main presentation and then move into Unconference sessions (community-proposed breakouts).

_____________________________________________________

Presentation: Resolving Transactional Access/Analytic Performance Trade-offs in Hadoop with Kudu

Over the past several years, the Hadoop ecosystem has made great strides in its real-time access capabilities, narrowing the gap compared to traditional database technologies. With systems such as Impala and Spark, analysts can now run complex queries or jobs over large datasets within a matter of seconds. With systems such as Apache HBase and Apache Phoenix, applications can achieve millisecond-scale random access to arbitrarily-sized datasets.

Despite these advances, some important gaps remain that prevent many applications from transitioning to Hadoop-based architectures. Users are often caught between a rock and a hard place: columnar formats such as Apache Parquet offer extremely fast scan rates for analytics, but little to no ability for real-time modification or row-by-row indexed access. Online systems such as HBase offer very fast random access, but scan rates that are too slow for large scale data warehousing workloads.

This talk will investigate the trade-offs between real-time transactional access and fast analytic performance from the perspective of storage engine internals. It will also describe Kudu (http://blog.cloudera.com/blog/2015/09/kudu-new-apache-hadoop-storage-for-fast-analytics-on-fast-data/), the new addition to the open source Hadoop ecosystem that fills the gap described above, complementing HDFS and HBase to provide a new option to achieve fast scans and fast random access from a single API.

Speaker:
Todd Lipcon is a Software Engineer at Cloudera and a PMC member of the Apache Hadoop and Apache HBase projects. He holds a Sc.B in computer science from Brown University, where he completed an honors thesis developing a new collaborative filtering algorithm for the Netflix Prize Competition. Todd interned at Google, where he developed machine learning methods to detect credit card fraud on AdWords and Google Checkout.

Additional Information:

PLEASE pre-register with security for quicker access to the venue: https://generalassemb.ly/education/sf-hadoop-users-meetup/san-francisco/17447 (https://generalassemb.ly/education/sf-hadoop-users-meetup/san-francisco/15046). The closest BART station is Montgomery. See you soon!