Building a High-Performance Behavioral Database

Databases are built to exploit the unique properties of the data they're meant to store. For example, document databases excel at nested data while relational databases work well at relating entities.

An increasingly important type of data we work with is behavioral data. This can be an action that a person performs (e.g. signup, checkout) or a change in the person (e.g. became a customer). Unfortunately, there are few databases built to handle behavioral data.

Sky is an open source, behavioral analytics database written in Go. It's goal is to store user events in a way that is fast to query and scales out well.

In this talk, we'll discuss what Sky is, the many design (and redesign) choices made along the way, and dive into how it all works internally. We'll talk about low level database choices such as LevelDB, LMDB, and Bolt DB. We will also talk about building and compiling query languages using LuaJIT and LLVM.

Speaker

Ben Johnson is an open source software developer specializing in customer behavior analytics and data visualization. His behavioral analytics database, Sky, backs the reporting systems at companies such as Shopify. Ben's Raft implementation in Go powers commercial distributed computing applications such as CoreOS' etcd project.

Join or login to comment.

People in this
Meetup are also in:

Create your own Meetup Group

Get started Learn more
Bill

I started the group because there wasn't any other type of group like this. I've met some great folks in the group who have become close friends and have also met some amazing business owners.

Bill, started New York City Gay Craft Beer Lovers

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy