Network Design Considerations and Challenges for Hadoop 'Big Data' Environments

Business decision processes have increasingly relied on frameworks that involve managing large data sets of unprecedented volume for business analytics, such as Hadoop and other distributed computing platforms.  Hadoop and the related Hadoop Distributed File System (HDFS) has been the popularly adopted open source framework of choice for mining business intelligence for its ability to process massively intensive data workloads over clusters of commodity server hardware and storage nodes.  The power of this distributed architecture, however, is directly proportional to the network infrastructure that these nodes communicate over, syncing with CPU and I/O performance. Organizations therefore have realized that the key to unlocking highly effective, optimally performing Hadoop investments is a network designed around massive scale, efficient operational manageability, and stability. 

Join us as Arista Networks, designers and implementors of some of the largest, most mission critical Hadoop clusters in the world, takes you under the covers of the network behind Hadoop with a hearty discussion on best practice network architectures for these demanding workloads, the power of EOS - Arista’s programmable Extensible Operating System.  We’ll dive deep into EOS and its uniquely integrated feature sets specifically focused on Hadoop/HDFS including MapReduce Tracer, RAIL, and Zero Touch Provisioning (ZTP).  Get answers to questions such as “how much visibility do we have into these Hadoop workloads?”, “how much data is each job sending over the network?”, “what are the current map and reduce tasks?”, and “how can I find and troubleshoot a worrisome node behind a particular interface?”

Guest speakers will include Benoit Sigoure of Arista Networks.  Benoit is the original author of OpenTSDB, the distributed monitoring system built on top of HBase. He also wrote AsyncHBase, the alternative HBase client that is fully asynchronous, non-blocking, and thread-safe. Benoit is currently working on building distributed systems for next-generation datacenter networks at Arista Networks. He also works on network extensibility, Hadoop integration, APIs, and network programmability in general.

Join or login to comment.

  • Sol

    Could make it. I hope slides and videos will be made available sometime soon.

    June 25, 2014

  • Denis G.

    Unfortunately I will not be able to attend.
    Will the Meetup be recorded?

    June 25, 2014

  • HonYin K.

    Looks like the usual meeting location has been moved from Penn Plaza to Tribeca. Is this correct?

    June 24, 2014

    • JI

      Yes we are changing for this week only, this is actually a combined meetup with a Hadoop users group. Should be quite fun!

      June 24, 2014

  • Steve K.

    New to the group, would like to share and understand
    the Next Generation Data Centers, private and public Cloud with Arista and SolidFire

    June 4, 2014

40 went

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Allison

Meetup has allowed me to meet people I wouldn't have met naturally - they're totally different than me.

Allison, started Women's Adventure Travel

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy