addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

How to Compare NoSQL Databases: Determining True Performance and Recoverability

One of the major challenges in choosing an appropriate NoSQL solution is finding reliable information as to how a particular database performs for a given use case. Stories of high profile systems failures abound, and tossed around with widely varying benchmark numbers that seem to have no bearing on how tuned systems behave out of the lab. Many of the profiling tools and studies out there use deeply flawed tools or methodologies. Getting meaningful data out of published benchmark studies is difficult, and running internal benchmarks even more so.

In this presentation, Will Keung, Senior Software Engineer at adMarketplace, will talk about choosing the best NoSQL engine and avoiding vendor lock with a smart NoSQL abstraction layer.

Ben Engber, CEO and founder of Thumbtack Technology, will discuss how to perform tuned benchmarking across a number of NoSQL solutions (Couchbase, Aerospike, MongoDB, Cassandra, HBase, others) and to do so in a way that does not artificially distort the data in favor of a particular database or storage paradigm. This includes hardware and software configurations, as well as ways of measuring to ensure repeatable results.

We also discuss how to extend benchmarking tests to simulate different kinds of failure scenarios to help evaluate the maintainablility and recoverability of different systems. This requires carefully constructed tests and significant knowledge of the underlying databases — the talk will help evaluators overcome the common pitfalls and time sinks involved in trying to measure this.

Lastly we discuss the YCSB benchmarking tool, its significant limitations, and the significant extensions and supplementary tools Thumbtack has created to provide distributed load generation and failure simulation.


WILL KEUNG - senior software engineer, adMarketplace

William has been a software engineer for more than ten years, during which time he has designed and implemented enterprise frameworks for various Fortune 500 companies. He has worked in different industries such as telecomm, banking and finance, publishing, biotech and the US government. His specialty is Java with a hint of security and networking. In his spare time, he tracks the startup and the local culinary scene while dabbling on Ruby on Rails and natural language processing.

BEN ENGBER - CEO and Founder of Thumbtack Technology

Ben has made a career around building highly reliable internet-scale applications. Prior to founding Thumbtack Technology in 2003 he was VP of Operations Technology at, where he built the back-office and e-commerce architectures needed to support millions of digital and physical transactions per day. During his tenure he became acutely aware of the need for outsourced labor that could both understand core business concerns and drive high-quality and robust software infrastructure projects. His inability to find such a partner led him to establish Thumbtack.

Ben holds a BS in Mathematics from the University of Chicago and a MBA from Boston University.


Join or login to comment.

  • Victor M.

    Thank you for organizing the presentation.

    March 28, 2013

  • Mehboob A.

    Enjoyed the detailed testing results from Thumbtack.. would definitely be interested in a follow up to that.

    March 28, 2013

  • A former member
    A former member

    Very impressive meetup. I learned a lot and had great conversation afterwards with some of the adMarketplace and Aerospike guys. Thanks!

    March 28, 2013

  • Ajay

    Thanks Ben & Will, well put together presentations, very useful, saved me tons of work. Thanks to Michael & ad as well, for the hospitality.

    March 27, 2013

  • Philip M.

    need to drop out, I need to stay at work late.

    March 27, 2013

  • Anton Y.

    I work a lot with different NoSQL solutions and would like to know more

    March 15, 2013

  • Mehboob A.

    Hmm.. the meetup logo changed to AeroSpike? I'd like to see the AeroSpike architect speak, I missed the last presentation :/

    March 14, 2013

49 went

Our Sponsors

  • Internap

    Collaboration and discussions with good food and beverages.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy