April Meetup

This month we'll have Danil Zburivsky of Pythian presenting about his team's port of OpenTSDB to Google's BigTable cloud service:

Data comes in different shapes. One of the these shapes is time series data. Time series is a very important abstraction since it can be used to describe multiple different processes. You can discover patterns in your website users behavior, capture sensor metrics from industrial equipment or track movement of celestial bodies using time series. The real power of this abstraction lies in providing a simple mechanism for different types of aggregations and analytics. It is easy to calculate minimum and maximum values over a given period of time, calculate average, sums and other statistics.

OpenTSDB is a popular open source project that provides a unified way to ingest high-volumes of time series data. OpenTSDB relies on HBase to provide scalable and reliable storage, but implements it’s own logic layer for storing and retrieving data on top of it.

One of the challenges in using OpenTSDB at scale is a need to deploy and maintain large HBase cluster. With a public release of Google Cloud Bigtable it is now possible to utilize flexibility of the cloud for HBase-like deployments. Since HBase is based on original Google’s work for Bigtable both systems are compatible on API level. Cloud Bigtable is a managed service and thus it requires minimum maintenance efforts even for large installations.

We would like to introduce a result of collaboration between Pythian and Google — an open source add-on to OpenTSDB which enables integration between OpenTSDB and Google Cloud BigTable. We will demonstrate how to setup a powerful and scalable time series collections system in several minutes. We believe that this project will be attractive to anyone dealing with monitoring systems, sensor data capturing or any other time series systems.