Featurebase DB Use case: how we scaled to millions of events per second

This is a live event in Tremor International offices and it will also be broadcasted live!
You can watch it here:
https://zoom.us/j/95314644441?pwd=NmxjcC8zUjAreWlNMzAzYjdKWUdKZz09

Agenda:
18:00-18:30 Mingling

18:30-19:00 Bitmaps as the underlying format for an analytical database / Pat O’Keeffe, SVP of Technology @ FeatureBase

19:00-19:15 Break - Beer, Pizza and Mingling

19:15-19:45 Scaling to 6M Events Per Second with Featurebase + Kafka / Shuli Hakim, Director Of Engineering (DevOps) @ Tremor International

19:45-20:30 Mingling

Pat O’Keeffe:
The simple idea of using bitmaps as the underlying format for an analytical database can unlock unimaginable performance. Bitmaps are amazing, and so much more powerful than they’re given credit for. It’s easy to forget that the phone, computer, or other devices you’re using to read this content is processing tens or even hundreds of millions of bitwise computations per processor
per second.
FeatureBase has been (maybe irrationally) obsessed with bitmaps, and they have built on top of breakthrough innovations like roaring bitmaps. Pat will showcase how the simple idea of denormalizing data to its most basic representation reduces the I/O constraints of traditional analytical formats by at least 10x, sometimes more. The end result is an order of magnitude reduction in compute needs for analytical workloads and orders of magnitude performance improvements.

Shuli Hakim:
Shuli will show you a real world example of how FeatureBase is powering Tremor’s data management platform. A year ago, after challenging the FeatureBase team to demonstrate that it could ingest over 1m events per second, Shuli embarked on replacing a very large Hadoop cluster with a fraction of the nodes of FeatureBase servers. Since then, Shuli has become an expert at managing and pushing FeatureBase to its limits, recently achieving over 6m events per second and now pushing to process over 1T events per day.

The talks are in English.