Embracing collaborative chaos - running Chaos Days on large platforms

Today’s systems are inherently complex, with some component parts often operating in or close to suboptimal or failure modes. Left unchecked, as complexity increases, the compounding of failure modes will inevitably lead to catastrophic system failure. Chaos Days help us address this risk by spending time deliberately inducing failures, then analysing the response. This session summarises our experience of running Chaos Days on a large scale platform. We’ll explore the what, why, how and when of running a Chaos Day.
Chaos Engineering addresses the risks inherent in distributed systems that stem from unexpected component failure. It does so by running experiments that explore the impact of sub-system failures by deliberately inducing different types of failure in different components. Outcomes are then analysed and learnings applied to improve the system’s resilience. These learnings deepen our understanding of the system and its failure modes, which aids the identification of new failure scenarios. This feedback loop informs subsequent rounds of experimentation, and thus the cycle repeats. In addition, planned failures provide a safe environment for teams to improve their incident response and how they conduct subsequent postmortems.

This session shares our experience of running Chaos Days over the last year, with one of our clients – a major Government department that hosts around 60 distributed, digital delivery teams. These teams design, deliver and support hundreds of microservices that serve online content to the department’s varied customers.

Duration of Talk: 1 hour followed by 30 minutes of networking/Q&A.

Who should attend?
Developers, DevOps engineers, QA's, Architects, Delivery Lead's and anyone interested to know how Netflix stays online despite all the chaos.

About the Speaker
Lyndsay Prewer is an agile delivery consultant with over 20 years' experience of helping teams deliver and operate better services, pipelines and platforms at speed and at scale. He is currently working with Equal Experts, at a variety of public and private sector clients

LinkedIn: https://www.linkedin.com/in/lyndsp/