37th Bay Area Hadoop User Group (HUG) Monthly Meetup

Detailed agenda and summaries to follow. General agenda:

 

  • 6:00 - 6:30 - Socialize over food and beer(s)
  • 6:30 - 7:00 - HCatalog/Hive Data Out
  • 7:00 - 7:30 - Apache Sqoop 2 - A next generation of data transfer tools
  • 7:30 - 8:00 - Building common denominator of Hadoop distributions with Bigtop

 

Session I (6:30 – 7:00 PM): HCatalog/Hive Data Out

Yahoo! Hadoop grid makes use of a managed service to get the data pulled into the clusters. However, when it comes to getting the data-out of the clusters, the choices are limited to proxies such as HDFSProxy and HTTPProxy. With the introduction of HCatalog services, customers of the grid now have their data represented in a central metadata repository. HCatalog abstracts out file locations and underlying storage format of data for the users, along with several other advantages such as sharing of data among MapReduce, Pig, and Hive. In this talk, we will focus on how the ODBC/JDBC interface of HiveServer2 accomplished the use case of getting data out of the clusters when HCatalog is in use and users no longer want to worry about the files, partitions and their location. We will also demo the data out capabilities, and go through other nice properties of the data out feature.

Presenter(s): Sumeet Singh, Director, Product Management, Yahoo!

Chris Drome, Technical Yahoo!

Session II (7:00 – 7:30 PM): Apache Sqoop 2 - A next generation of data transfer tools

Apache Sqoop 2 is the next generation of the massively successful open source tool designed to transfer data between traditional SQL databases and warehouses into Apache Hadoop. Sqoop 2 is designed as a client-server system with a repository which stores connection and job information. Sqoop 2 is designed to support secure job submission and multiple different roles for users. In this talk, we will discuss the issues users faced in Sqoop 1, and the design of Sqoop 2 and how the issues faced in Sqoop 1 are being handled in Sqoop 2.

Presenter(s): Hari Shreedharan, Software Engineer, Cloudera

 

Session III (7:30 – 8:00 PM): Building common denominator of Hadoop distributions with Bigtop

What it takes to get to Hadoop2 GA?

Bigtop is stepping up in its role as the foundation of a standard Hadoop-based data analytics stack, essentially bringing most of the commercial offering to the standard footing. 6 out of 7 commercial vendors using Bigtop framework to power their distributions based on ASF Hadoop.

Bigtop is also the must have stabilization tool for Hadoop platform where's any downstream application or system developer can make sure that their software would work with the next version of Hadoop.

Presenter(s): Dr. Konstantin Boudnik, ASF Hadoop committer, Bigtop PMC; Director of Engineering, WANdisco

Roman Shaposhnik, VP, Apache Bigtop, IPMC member at ASF; Software engineer, Cloudera inc.

 

Yahoo Campus Map:

Detail map

 

Location on Wikimapia:

http://www.wikimapia.org/#lat=[masked]&lon=[masked]&z=18&l=0&m=b&search=yahoo

 

Join or login to comment.

  • Laura U.

    HUG organizer - Accenture would llike to be a speaker then. Room for speakers at July event? can I speak with someone live?

    June 24, 2013

  • Laura U.

    HUG organizer - Accenture would like to sponsor the July Meet Up. Still room for sponsors?

    June 23, 2013

    • Yahoo! HUG O.

      We do not need or accept sponsorships. Thank you.

      June 23, 2013

  • varun s.

    Would you share the HCatalog ppt, presented by Sumeet/Chris??

    May 17, 2013

  • Yahoo! HUG O.

    Slides are posted on slideshare.net/ydn, and videos to youtube.com/ydntheater. Thank you for your patience. All the past meetups can also be looked up with the tag HUG.

    May 20, 2013

  • Eugene V.

    Yahooooooooo!

    May 15, 2013

  • Leo

    HCatalog / Sqoop2 was alright.
    BigTop lost a lot of audience.
    I would suggest practical example of a sets of distro build from jenkins-bigtp and how it is put together / validated / reconstructed

    2 · May 15, 2013

  • Sunil V.

    Good to see you

    May 15, 2013

  • Sunil V.

    Good

    May 15, 2013

  • Kalyan K.

    Will have to drop out for today's session. Sorry on last minute update. Hope it might serve some one on wait list. Request slides on big top!

    May 15, 2013

  • Arvind

    whats the protocol for the ~134 wait listed people ? just keep waiting and not show up or cause a big data problem :)

    May 15, 2013

    • Yahoo! HUG O.

      We have always had space historically, please drop by

      1 · May 15, 2013

  • sabahat A.

    Hii , attending the meeting for first time and really excited to know more about Hadoop.

    May 13, 2013

  • KRS

    my first HUG confirmed.... super excited

    May 13, 2013

  • Peter L.

    Traveling for work

    May 10, 2013

  • Eden

    Brian

    May 10, 2013

  • Harish S.

    is there anyone driving from palo alto or mountain view and came take me along? :)

    May 7, 2013

  • shyam

    What is spoken and who is speaking? ;)

    May 6, 2013

  • Harish S.

    anyone going to the meet up from palo alto or Mountain view? I can get a ride?

    April 29, 2013

  • Maria Y.

    I'm excited to join my first Big Data meet up!

    April 28, 2013

  • Joe B.

    Hey Guys
    I am looking for a QA/Qe hadoop core, ASAP, you must be experienced with Hadoop, strong in Linux, Automation, work independently and finally Positive attitude :)

    send me an email jbounour_a_T_ddn_com

    April 26, 2013

  • nischitha p.

    Interested in learning hadoop

    1 · April 26, 2013

  • Jian L.

    Thanks.

    April 25, 2013

Our Sponsors

  • Yahoo! Inc.

    Meeting space, pizza and drinks are sponsored by the Yahoo! Hadoop team.

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Rafaël

We just grab a coffee and speak French. Some people have been coming every week for months... it creates a kind of warmth to the group.

Rafaël, started French Conversation Group

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy