ALOJA: an open source framework for Big Data benchmarking

ALOJA (http://aloja.bsc.es) , is an open research initiative from the Barcelona Supercomputing Center (BSC) and Microsoft Research to explore new cost-effective hardware architectures and applications for Big Data. The project has created and open source platform to benchmark Big Data systems and to analyze performance details.

The talk/workshop will DEMO how to use the benchmarking tools and explore the main results of the project including over 50k Hadoop job runs and the results that can be extracted from them. Including:

• How to speedup Hadoop over 3x from the defaults by changing its config and the system.

• Understand main bottlenecks and scalability and the gains with new hardware.

• The main considerations with Hadoop in the cloud, such as what type of VMs to use and cluster sizes.

• An overview of using Predictive Analytics to extract performance knowledge

The talk will end with discussion over pizza and beer.

Agenda:

19:00 - Arrive at Itnig and meet other members
19:15 - Main talk
19:45 - Q&A and discussion of topics
20:00 - Networking, pizza and beers

About the presenter:

Nico Poggi (@ni_po (https://twitter.com/ni_po)) is an IT professional with focus on performance and scalability of Web and Data intensive applications. He is currently leading a new research project on upcoming architectures for data processing at the Barcelona Super Computing (BSC) and Microsoft Research joint center ( http://www.bscmsrc.eu/ ). Nicolas received his PhD at the BarcelonaTech university (UPC) and combines both a pragmatic approach to performance and scalability with Machine Learning techniques. His publications can be found at: http://personals.ac.upc.edu/npoggi/