Polyglot Persistence At Netflix

Netflix’s architecture involves thousands of microservices built to serve unique business needs. As this architecture grew, it became clear that the data storage and query needs were unique to each area; there is no one silver bullet which fits the data needs for all microservices. CDE (Cloud Database Engineering team) offers polyglot persistence, which promises to offer ideal matches between problem spaces and persistence solutions. In this meetup you will get a deep dive into the Self service platform, our solution to repairing Cassandra data reliably across different datacenters, Memcached Flash and cross region replication and Graph database evolution at Netflix.

Agenda :
6:00 - 7:00 Registration, Food/Drink & Networking
7:00 - 8:30 Talks
8:30 - 9:00 Q&A

Talk Details:

1. CDE Service & Data Explorer (20 mins)
   Join us to learn about CDE Service, a central hub for managing our large-scale fleet of polyglot datastores, and how it empowers Netflix engineers across the company to get onboarded and access operational insights that they care about. We'll also demo the Netflix Data Explorer for Cassandra and Dynomite -- see how Netflix engineering explores data in our persistent stores with tools that encourage best practices.

2. Repair Service (20 mins)
   Anti-entropy repair in C* is and has been one of the most painful operational overheads in providing C* as a service. To solve this pain, we built a fully decentralized, self-schedulable, self-healable and self-monitoring repair service to keep data consistent across nodes and data centers which solves this problem once and for all. In this meetup, we will share the design internals and production wins our repair service brought to hundreds of C* clusters and thousands of C* nodes.

3. Memcached Flash and Cross region replication (20 mins)
   Memcached Flash is the next gen storage solution used by EVCache which uses SSD (Flash) to store data. We are going to talk about how we were able to scale the storage from GB’s to TB’s without compromising speed or throughput and at a significantly reduced cost.
   Coherency in a distributed cache is a tough problem to solve. Doing this at scale across multiple aws regions is challenging. We are going to talk about our approach and our solution

4. What’s Next? Graph Database (10 mins)
   Graph databases optimize for use cases driven by many-to-many relationships and the need for fast, flexible, interactive traversals of those relationships. At Netflix we have identified use cases that require flexible fine-grained data model, and decided to leverage a framework based on our Cassandra deployment. Therefore, we have integrated JanusGraph with the Netflix ecosystem. In this talk, we are going to go over a few of the use cases that leverage JanusGraph, how we got motivated to use Janusgraph and the migration path from TitanDB.