Property Based Testing: Avoid Accidental Quality (Mourjo Sen)

Automated tests are the cornerstone of modern software development. It ensures that every time we build new functionalities, we do not break existing features our users rely on.

Traditionally, we tackle this with example-based tests. We list down specific scenarios (or test cases) that verify the expected behaviour.

However, example-based tests have a critical flaw. The quality of our software depends only on the examples in our test suites. This leaves out a class of scenarios that the authors of the test did not envision – the “unknown unknowns”.

While example-based tests focus on reproducibility of known bugs, property based tests focus on discoverability of unknown bugs. It shifts our focus from enumerating examples to verifying the fundamental properties of our system.

Outline of the session:

- Example vs property based tests: Verifying predetermined scenarios vs finding bugs
- Thinking about fundamental properties of systems
- Why the property of associativity does not hold for floating point numbers
- Sampled test cases lead to accidental quality
- Property based tests find bugs in unknown unknowns
- Shrunk failing cases are key feedback to fixing bugs
- Introduction to a Meeting application for the demo
- Demo
- Bugs found were not coded in the tests

ABOUT MOURJO

Mourjo Sen is a senior software engineer at Booking.com. He has ten years of experience in building resilient well tested microservices. He is originally from India and currently lives in Amsterdam with his wife.

- LinkedIn: https://www.linkedin.com/in/mourjo
- Twitter: https://x.com/mourjo_sen
- GitHub: https://github.com/mourjo
- Website: https://mourjo.me/