addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1light-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Introducing Apache Weave by Jonathan Gray of Continuuity

YARN is a new framework for writing distributed frameworks and applications. YARN stands for “Yet Another Resource Negotiator” and began as a re-architecting effort for solving issues observed with the previous Hadoop Map-Reduce framework on large distributed clusters – particularly with JobTracker (popularly known as JT). The old version of JobTracker was responsible for two major functions on a distributed cluster:

Cluster Resource Management and

Application Lifecycle Management for all applications running on the cluster

Because JT was responsible for all applications, the cluster was inherently limited, resulting in wasted resources or limited scalability.

The Challenge with YARN

YARN is powerful but not always easy to use. Setup is simple but YARN is not easy to build new frameworks or applications with. Building directly against the YARN API is difficult as it is intentionally low-level and verbose. Out of the box, YARN lacks application developer friendly APIs, application state management, service discovery, request routing, application monitoring, application logging and a simple testing framework.

Weave – The Vision

The vision for Weave is to enable developers to easily harness the power of YARN, with a simple programming model and reusable components for building distributed frameworks or applications. Weave hides the complexity of YARN with a programming model that closely resembles the Java thread model. Weave supports running and managing applications with robust application lifecycle management support, support for long running processes, centralization of logs and metrics on the client, service registration and discovery capabilities.

Co-founder & CEO, Continuuity & HBase Committer

Jon is an early adopter of Hadooop and HBase and long-time contributor and committer in the open source community. After founding Streamy.com, Jon was at Facebook where he helped drive HBase engineering efforts for massive projects like Facebook Messages. Jon loves to dance to hip-hop and run on the beach with his dog.


Join or login to comment.

  • Subash D.

    Deck has been posted in the Files section

    September 13, 2013

  • A former member
    A former member

    Great presentation on YARN/Weave and HBase

    August 30, 2013

  • Subash D.

    The live stream URL for those of you who can't attend
    http://www.ustream.tv/channel/l-a-hadoop-user-group

    August 29, 2013

  • A former member
    A former member

    Will attend remotely.

    August 27, 2013

  • Tracy M.

    I'm coming

    August 16, 2013

  • A former member
    A former member

    will attend remotely if there is a streaming setup

    August 5, 2013

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy