Past Meetup

Hadoop User Group: Facebook London

This Meetup is past

121 people went

Location image of event venue


Our next Hadoop User Group meeting is going to be at Facebooks new office in London. We've got three speakers from and Facebook covering a range of topics. Plus the usual food and drinks for the evening.

Details of the three talks:

1. Tools and approaches for migrating big datasets to the cloud
- Adrian Woodhead and Elliot West from

This presentation describes the journey taken by the big data platform team when tasked with migrating big data sets and pipelines from on-premises clusters to cloud based platforms. We present two open source tools that we built to overcome the unexpected challenges we faced.

2. Facebook disaggregated storage for Data Warehouse
- Pavel Zakharov from Facebook

This talk will give details on the Warm Storage service that we use to replace HDFS. We will go into details about the design choices for this from recent changes in hardware available. This will also summarise how this fits into our data centre networking setups to achieve the storage and compute tradeoffs.

3. Scribe, the Globally Distributed Message Bus at Facebook
- Benjamin Leonhardi from Facebook

This talk will give an overview of Scribe, the global distributed message bus which is the backbone for most log aggregation and realtime usecases at Facebook. We will describe Scribe and its tradeoffs compared to similar open source systems build on Kafka and how Scribe solved some of the most common issues like auto scaling, multi tenancy, and failover.