Druid can be the perfect tool … in some cases

As our traffic continues to grow and our business continues to demand more value, we have the luxury of using new technologies when creating new systems. In the last few months, we have been using Druid for our projects. Druid is very interesting database that came to replace much more expensive solutions that we found useful for different situations. This is an opportunity to share our experience and explain not only when Druid was the right solution, but also when it wasn't a perfect match.

Agenda -

17:30 - 18:00 - meet, drink beers etc.

18:00 - 18:45 - "Sometimes, Druid is not the best solution for a business use case", by Yulia Trakhtenberg, Data Team Leader
The AppsFlyer dashboard provides our customers with all the analytics that they need - in real-time. As our traffic grows rapidly, we are now required to aggregate 8 billion daily events, which is a very interesting challenge. The previous solution was built to support a few million events per day, and became less reliable with the growth. Just a few months ago we started to look for a new solution. We tried different solutions including Cassandra, Redis and Druid. This is the story of why we didn’t choose Druid and what solution we did decide to use.

18:45 - 19:00 - Pizza break

19:00 - 19:30 “And sometimes it is perfect", by Ido Barkan, Senior Developer Real-Time Attribution team
Analyzing the web access log for 8 billion events per day becomes a big data problem, for which conventional log analysis tools either fail or cost a lot of money. In this session, we will present how we used Druid to parse these logs and gain an analytical point of view on what is happening in our web endpoints both historically and in real-time. It helps us identify fraud and analyze wrong requests coming from the client and how to help them fix their usage and much more.