007 - Databases and Dog Tracks; Simple Minds and Abstraction

Hi All,

We'll be back at BashoWest this month for the meetup, and the date is set for February 27th. (Sorry for the late announcement.)

We have two great talks queued up: Andy Pavlo from Brown will be on-hand to make wild claims and interesting analogies about some of the DBMS technology on which he's working. Following Andy's talk, Ted Nyman returns to BashoChats to melt minds with a talk about abstractions and their interplay with infrastructure problems. It should be a good night.

Full details are below the break. As usual, Basho is taking care of pizza and refreshments. And seats are limited for this one, so make your move early.

Mark

------

 

7:15

Everything I Know About Fast Databases I Learned at the Dog Track

Andy Pavlo, Brown University

An emerging class of distributed database management systems (DBMS), known as NewSQL, provide the same scalable performance of NoSQL systems whilemaintaining the consistency guarantees of a traditional, single-node DBMS. These NewSQL systems achieve high throughput rates for data-intensive applications by storing their databases in a cluster of main memory partitions. This partitioning enables them to eschew much of the legacy, disk-oriented architecture that slows down traditional systems, such as heavy-weight concurrency control algorithms, thereby allowing for the efficient execution of single-node transactions. But many applications cannot be partitioned such that all of their transactions execute in this manner; these multi-node transactions require expensive coordination that inhibits performance. Thus, without intelligent methods to overcome these impediments, a NewSQL DBMS will scale no better than a traditional DBMS.

In this talk, I will present our research on integrating machine learning techniques to improve the performance of fast database systems that is inspired by my adventures at greyhound racing tracks. In particular, I will discuss my work on the H-Store parallel, main memory transaction processing system. I will first describe the Houdini framework that uses Markov models to predict transactions’ behaviors to allow a DBMS to selectively enable runtime optimizations. I will then present Hermes, a method for the deterministicexecution of speculative transactions whenever a DBMS stalls because ofdistributed transactions. Together, these projects enable H-Store to support transactional workloads that are beyond what single-node systems can handle.

 

8:00

Simple Minds and Abstraction

Ted Nyman, GitHub

This is a talk about three issues related to abstraction, and the infrastructure problems they invariably create.

Problem one is that fixation on programming languages and programming patterns leads us to think about problems incorrectly.

Problem two is the belief that complex distributed systems can be reasoned about in totality, when, in a very strong sense, they cannot be at all.

Problem three is that we overestimate our ability to incorporate disparate components and dependencies into a working whole.

The net result is that we inevitably build flawed systems, despite our best intentions and best work.

This may be fixable. Or, we may just be out of luck.

Join or login to comment.

  • A former member
    A former member

    Databases and Dog Tracks was a great comprehensive overview of the trials and tribulations of an in-memory DB, distributed partitions, optimizations, trade-offs, research, etc.
    Simple Minds and Abstraction was unfortunately a completely un-actionable stream-of-consciousness ramble. The presenter should have focused more on his lesson's learned, tips, techniques, do's and don't's, etc to elicit ideas and engage in a dialog with his audience rather than what amounted to a 30min rant.

    February 28, 2013

People in this
Meetup are also in:

You don't have to be an expert to start your own Meetup G roup

Get started Learn more
Katie

I'm surprised by the level of growth I've seen since becoming an organizer, it's given me more confidence in my abilities.

Katie, started NYC ICO

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy