This is an excellent meetup with a partner group, you can check out the event and RSVP here.
One of the major issues that Spark batch jobs have to contend with at AppsFlyer is that our data is inherently skewed. For instance a couple of apps account for the vast majority of our traffic. Data skew wreaks havoc on naively written data jobs by making them perform and scale very poorly as the amount of data they need to process increases. Recently one of our central data aggregations -- the process that prepares data for the overview dashboard -- stopped working and we had essentially reached the limit where we could no longer devote more Ram to the process to help it. Using a technique called "Salting" to overcome the data skew that was killing this job we were able to get the job working again and make the entire process much more scalable. I'll go over Salting in depth to explain how it works and how we are starting to use it here at AppsFlyer.
Join AppsFlyer Engineering for one of the most requested sessions from DevOps Days Tel Aviv!
Please RSVP through the AppsFlyer Engineering Meetup Group: (https://www.meetup.com/AppsFlyer/events/257831523/?isFirstPublish=true)
Avi Kivity, father of KVM, and now ScyllaDB will (FINALLY!) be gracing us with the talk that didn't pan out at DevOps Days: "How to Write a Database" - you won't want to miss this talk from one of the only people in our community who can give it!
18:00 - Mingling, Pizzas & Beers
18:30 - Main Session: How to Write a Database
19:10 - AMA / How to Choose the NoSQL Database that's right for you, Avi Kivity & AppsFlyer Engineering
Title: How to Write a Database
Application developers take it for granted that the database will always be there for them, tirelessly serving their reads and writes (hopefully with the same data!). But how does it work?
This talk explores the database internals: commit logs, memory allocation, I/O strategies, and more.
Technical - Advanced