007 - Databases and Dog Tracks; Simple Minds and Abstraction

Hi All,

We'll be back at BashoWest this month for the meetup, and the date is set for February 27th. (Sorry for the late announcement.)

We have two great talks queued up: Andy Pavlo (http://cs.brown.edu/~pavlo/) from Brown will be on-hand to make wild claims and interesting analogies about some of the DBMS technology on which he's working. Following Andy's talk, Ted Nyman (http://ted.io/) returns to BashoChats to melt minds with a talk about abstractions and their interplay with infrastructure problems. It should be a good night.

Full details are below the break. As usual, Basho is taking care of pizza and refreshments. And seats are limited for this one, so make your move early.

Mark

------

7:15

Everything I Know About Fast Databases I Learned at the Dog Track

Andy Pavlo, Brown University

An emerging class of distributed database management systems (DBMS), known as NewSQL, provide the same scalable performance of NoSQL systems whilemaintaining the consistency guarantees of a traditional, single-node DBMS. These NewSQL systems achieve high throughput rates for data-intensive applications by storing their databases in a cluster of main memory partitions. This partitioning enables them to eschew much of the legacy, disk-oriented architecture that slows down traditional systems, such as heavy-weight concurrency control algorithms, thereby allowing for the efficient execution of single-node transactions. But many applications cannot be partitioned such that all of their transactions execute in this manner; these multi-node transactions require expensive coordination that inhibits performance. Thus, without intelligent methods to overcome these impediments, a NewSQL DBMS will scale no better than a traditional DBMS.

In this talk, I will present our research on integrating machine learning techniques to improve the performance of fast database systems that is inspired by my adventures at greyhound racing tracks. In particular, I will discuss my work on the H-Store parallel, main memory transaction processing system. I will first describe the Houdini framework that uses Markov models to predict transactions’ behaviors to allow a DBMS to selectively enable runtime optimizations. I will then present Hermes, a method for the deterministicexecution of speculative transactions whenever a DBMS stalls because ofdistributed transactions. Together, these projects enable H-Store to support transactional workloads that are beyond what single-node systems can handle.

8:00

Simple Minds and Abstraction

Ted Nyman, GitHub

This is a talk about three issues related to abstraction, and the infrastructure problems they invariably create.

Problem one is that fixation on programming languages and programming patterns leads us to think about problems incorrectly.

Problem two is the belief that complex distributed systems can be reasoned about in totality, when, in a very strong sense, they cannot be at all.

Problem three is that we overestimate our ability to incorporate disparate components and dependencies into a working whole.

The net result is that we inevitably build flawed systems, despite our best intentions and best work.

This may be fixable. Or, we may just be out of luck.