Databases are built to exploit the unique properties of the data they're meant to store. For example, document databases excel at nested data while relational databases work well at relating entities.
An increasingly important type of data we work with is behavioral data. This can be an action that a person performs (e.g. signup, checkout) or a change in the person (e.g. became a customer). Unfortunately, there are few databases built to handle behavioral data.
Sky is an open source, behavioral analytics database written in Go. It's goal is to store user events in a way that is fast to query and scales out well.
In this talk, we'll discuss what Sky is, the many design (and redesign) choices made along the way, and dive into how it all works internally. We'll talk about low level database choices such as LevelDB, LMDB, and Bolt DB. We will also talk about building and compiling query languages using LuaJIT and LLVM.
Ben Johnson is an open source software developer specializing in customer behavior analytics and data visualization. His behavioral analytics database, Sky, backs the reporting systems at companies such as Shopify. Ben's Raft implementation in Go powers commercial distributed computing applications such as CoreOS' etcd project.