Network Design Considerations and Challenges for Hadoop 'Big Data' Environments

Business decision processes have increasingly relied on frameworks that involve managing large data sets of unprecedented volume for business analytics, such as Hadoop and other distributed computing platforms. Hadoop and the related Hadoop Distributed File System (HDFS) has been the popularly adopted open source framework of choice for mining business intelligence for its ability to process massively intensive data workloads over clusters of commodity server hardware and storage nodes. The power of this distributed architecture, however, is directly proportional to the network infrastructure that these nodes communicate over, syncing with CPU and I/O performance. Organizations therefore have realized that the key to unlocking highly effective, optimally performing Hadoop investments is a network designed around massive scale, efficient operational manageability, and stability.

Join us as Arista Networks, designers and implementors of some of the largest, most mission critical Hadoop clusters in the world, takes you under the covers of the network behind Hadoop with a hearty discussion on best practice network architectures for these demanding workloads, the power of EOS - Arista’s programmable Extensible Operating System. We’ll dive deep into EOS and its uniquely integrated feature sets specifically focused on Hadoop/HDFS including MapReduce Tracer, RAIL, and Zero Touch Provisioning (ZTP). Get answers to questions such as “how much visibility do we have into these Hadoop workloads?”, “how much data is each job sending over the network?”, “what are the current map and reduce tasks?”, and “how can I find and troubleshoot a worrisome node behind a particular interface?”

Guest speakers will include Benoit Sigoure of Arista Networks. Benoit is the original author of OpenTSDB, the distributed monitoring system built on top of HBase. He also wrote AsyncHBase, the alternative HBase client that is fully asynchronous, non-blocking, and thread-safe. Benoit is currently working on building distributed systems for next-generation datacenter networks at Arista Networks. He also works on network extensibility, Hadoop integration, APIs, and network programmability in general.