Benchmarking with Benchbase

Benchmarking is an important part of developing database systems like Postgres. For businesses, benchmarking can guide important decisions around selecting a database management system, or evaluate configuration changes. However, while there are many standard benchmarks out there, setting up, configuring, running, and collecting results was left as an exercise to the reader.

Benchbase is an open-source benchmarking framework from Carnegie Melon University’s Database Group (https://github.com/cmu-db/benchbase). It helps you configure, run, and collect results from various standard benchmarks. It also provides a framework for authoring your own performance benchmarks, and wiring it up to the batteries-included configuration, running, and results collection provided by it.

Paul is going to walk us through Benchbase, and some recent work he has embarked on around creating a benchmark. If you are a veteran of benchmarking, or have never done a benchmark before, this talk is for everyone.