Hadoop Engineering Best Practices Birds of Feather (BoF) Session

Pramod Thangali, senior director of engineering at Hortonworks, will host the BoF session.

Theme:

Best Practices in Hadoop Engineering: operations, quality and releases

We want to share with the Hadoop community our approach on how we manage the challenges of operations, quality assurance, and release engineering. As well as sharing our processes and approaches to these challenges, we want to hear from the community how you address these issues in your engineering organizations.

We will cover engineering operations, quality and release engineering, at Hortonworks. The discussion will provide insight into the infrastructure, processes, and procedures we have in place to compose a long list of open source Apache projects into a production-ready stack that is certified through a rigorous set of functional, system and performance tests on a complex test matrix.

Speakers

Raja Aluri, Release Engineering Manager

Deepesh Khandelwal, Quality Engineering Manager

Ramya Sunil, Quality Engineering Manager

Agenda

Part 1: Release Engineering (30 Mins)

• Hortonworks Open source philosophy

• How we do Apache first development

• How do we incorporate fixes or features that did not make it into apache yet.

• How we integrate our partner contributions to the source code

• Bookkeeping of the delta’s between apache and hortonworks

• How we respond to customers issues and patch workflow

Part 2: Quality Engineering (30 - 45 mins) - Deepesh/Ramya

• HDP stack then and now (the journey from 1.0 to 2.3)

• HDP Major & Minor Releases

• Continuous Integration with Apache Branches

• Provision, Deploy, Test

• Support Matrix and Cloud Infrastructure

• Overall Test Coverage

• Limitations of unit testing

• Why system tests are important

• Types of Testing

• Test Plans Development

Q&A