Everything Is Broken

Everything Is Broken
Everything Is Broken
Public group

Snap Inc.

2025 1st Avenue · Seattle, wa

How to find us


Location image of event venue


This month we are at Snap and have two amazing speakers lined up for everyone:

"Building Resilient Systems Using Chaos Engineering" - Tammy Butow

Tammy Butow explains how to build resilient systems by focusing on the detection, mitigation, resolution and prevention of incidents using chaos engineering. Tammy Butow is a principal SRE at Gremlin, where she works on chaos engineering—the facilitation of controlled experiments to identify systemic weaknesses. Gremlin helps engineers build resilient systems using their control plane and API. Tammy is the cofounder of Girl Geek Academy, a global movement to teach one million women technical skills by 2025. Tammy is an Australian and enjoys riding bikes, skateboarding, snowboarding, and surfing. She also loves mosh pits, crowd surfing, metal, and hardcore punk.

"Observability-driven development" - Charity Majors
TDD is table stakes for any good engineering team, but TDD stops at the edge of your laptop. That's not good enough. In an era of complex distributed systems with emergent patterns and unpredictable user behaviors, what you need is observability-driven design and development. You need to stop thinking of your environments as binary (first you stage, then flip a switch and deploy to production) and start thinking of it like baking: shipping to production is just one step in the process of gaining confidence in your code. Tests are great but your code isn’t baked til it has been running with real services, real data, real network hops, and real users over a period of time. Most of the failures in complex systems are unpredictable and can't be caught by tests. So we need to embrace an ethos of testing in production -- building guard rails and checkpoints into our pipelines and deploying a robust event-level observability solution so we can explore real results, not rely on fragile, post hoc staging systems.

Charity is the CEO/cofounder of honeycomb.io. Previously ran operations at Parse/Facebook, managing a massive fleet of MongoDB replica sets as well as Redis, Cassandra, and MySQL. Worked closely with the RocksDB team at Facebook to develop and roll out the world's first Mongo+Rocks deployment using the pluggable storage engine API. Likes single malt scotch; kind of a snob about it.