Long Running Services on YARN using Slider

AGENDA:

6:30 pm – 6:45 pm Networking & Refreshments
6:45 pm - 6:50 Introduction/Announcements
6:50 pm - 7:20 Speaker 1

7:20 pm - 7:30 pm Q & A
7:30 pm – 8: 00pm Speaker 2
8:00 pm – 8:10 pm Q & A

Topic 1 – Running Dockerized Applications on YARN via Slider (Thomas(Yu) Liu from Hortonworks)

Abstract: Docker provides a very natural extension of YARN container for software isolation for deployment ease and consistency. Apache™ Slider can be used to on-board dockerized applications that aren’t YARN-aware can now participate in the YARN ecosystem – with no code modification. Dockerized applications can take advantage of complex component orchestration along with node affinity, resource and co-location requirement or hadoop security model apart from being truly distributed. Slider framework also provides these application lifecycle management which is also tightly integrated Hadoop’s only open source management tool - Ambari without any additional integration effort.

Bios:

Yu Liu(Thomas) is a Software Engineer at Hortonworks working Apache Slider, framework for on-boarding Distributed applications on YARN. Yu joined Hortonworks from Amazon. He was one of the founders of AWS CloudTrail service, which makes use of Hadoop to analyze all customer activities on AWS public APIs for security purpose. He also built web services and applications to manage delivery, finance and employees in all Amazon delivery stations. He earned a Master of Science in Computer Science degree from Illinois Institute of Technology.

Topic 2 – KOYA: Apache Kafka on YARN via Slider (Thomas Weise from Datatorrent)

Abstract: A while ago, DataTorrent announced a new initiative to integrate Kafka and YARN under the KOYA project. KOYA was proposed as KAFKA-1754 and well received by the community. Considering our goals with KOYA and that Kafka was built with fault tolerance in mind and already provides most of the HA features, we evaluated Slider. Slider was built to enable long running services on YARN without making changes to the services themselves. We found it sufficient to bring Kafka to YARN using Slider as it provides much of the infrastructure required for KOYA.

Bios:

Thomas is a principal architect at DataTorrent. Thomas has extensive experience architecting and developing real-time streaming applications based on Hadoop. Before joining Datatorrent Thomas was part of the Hadoop engineering team at Yahoo

PARKING INSTRUCTIONS:

When you arrive, drive along the driveway, past the garage, past the Dell building, to the large parking lot. To the far right, along the grass bank, are parking slots reserved for “HW” (Hortonworks). Look for an open “HW” spot and park you car. Or ask the valet in red jacket who will guide you where to park. Tell a valet you're here for a Meetup.

From the parking lot, walk back to the Tech Stadium Café, up the short stairs, and adjacent to the Tech Café Stadium, is an outside entrance to Mojave Conference Learning Center. (You’ll notice “Hortonworks Shuttle Pick Up and Drop” sign at the bottom of the short stairs.)