Bay Area Hadoop User Group (HUG) September Meetup

Sep 2011 HUG Agenda:

6:00 - 6:30 - Socialize over food and beer(s) 6:30 - 7:00 - Architecture of an Enterprise MapReduce Engine for Hadoop Deployments 7:00 - 7:30 - Gateway: Cluster Virtualization Framework 7:30 - 8:00 - Apache Sqoop - A Data Transfer Tool for Hadoop

Architecture of an Enterprise MapReduce Engine for Hadoop Deployments: Enterprise customers expect a solution that is easy to deploy and manage, integrates with IT security and management tools, guarantees high reliability and availability, and supports multiple lines of business and applications as well as multiple distributed file systems.

In this talk we will explore the architecture of the Platform MapReduce distributed runtime engine, which is capable of achieving these enterprise-class needs today.

Presenter: Scott Campbell, Platform

Gateway: Cluster Virtualization Framework: Access to Hadoop clusters in production environment is restricted by corporate firewalls. Users access Hadoop clusters via dedicated portal nodes. Portals located behind firewalls perform user authentication and authorization. On clusters with many users portals become shared multitenant resources, which create contention among users and increase maintenance overhead for administrators.
We present the Gateway Project developed at eBay. Gateway is a cluster virtualization framework, which provides

1. Seamless access to multiple clusters from users’ workplace computers through corporate firewalls.
2. Service availability: failover to active clusters when one has scheduled/unscheduled downtime.
3. Flexible cluster upgrades: redirect traffic to other clusters when one is upgrading.
4. Versioning: access to clusters running different versions of Hadoop.Presenter: Konstantin V. Shvachko, eBay

Apache Sqoop - A Data Transfer Tool for Hadoop: Apache Sqoop is a tool designed for efficiently transferring bulk data
between Hadoop and structured datastores such as relational databases.This talk aims at familiarizing the user with Sqoop and how to effectively use it in real deployments.

Presenter: Arvind, Cloudera

Yahoo Campus Map:

Detail map (http://photos4.meetupstatic.com/photos/event/2/8/e/d/600_21370477.jpeg)

Location on Wikimapia:

http://www.wikimapia.org/#lat=37.4181633&lon=-122.0250607&z=18&l=0&m=b&search=yahoo