Cassandra rocks! Instagram's migration to C3.0 and the new storage engine.

Apache Cassandra Bay Area
Apache Cassandra Bay Area
Public group
Location image of event venue


Join us at the Instagram HQ and learn more about the largest takeaway of Instagram migration to Cassandra 3.0 and the observed performance of the RocksDB storage engine.

We will start the event with a series of lightning talk (5mins). Please reach out to a host if you would like to present something.

Register at the building 24 lobby and mention "Cassandra meetup". Self parking is available. Using lyft/uber is recommended. Those who have provided name/email/company information, will have their badges ready to pick up at the lobby reception (Thank you!). If you haven't pre-registered, arrive earlier to account for the checkin process.

Agenda (more details to come):

* 6pm: Doors open
* 6pm-7pm: Socialize around some food and drinks
* 7pm-7.20pm: Lightning talks
* 7.20pm-7.40pm: "Getting Instagram on C3.0, when is it your turn?"
* 7.40pm-8.00pm: "Cassandra on RocksDB"
* 8.00pm-8.30pm: Saying goodbye

Getting Instagram on C3.0

We're all culprits of snoozing software upgrades - 'Remind me tomorrow' for so many days, or until we're faced with mandatory upgrade. The hesitation may stem from the risks of upgrading. Will it lose my data? Will it be incompatible with dependent systems? Will it be slower? However, when the risks are mitigated and the benefits are significant, the decision to upgrade is a no-brainer.

Cassandra 3.0 introduces major changes - storage engine rewrite, materialized views, file-based hint storage, to name a few. As Instagram serves much of its core functionality on Cassandra, the risks of upgrading to this major version weighed heavily from the start. In this talk we'll cover the testing process we employed to detect regressions and data inconsistency, the patches we deployed to address performance issues, and the lessons we learned from upgrading our production clusters to 3.0.

Andrew Whang is a software engineer at Instagram. He enjoys working on infrastructure challenges at work, almost as much as he enjoys the croissant section in the cafeterias.

Cassandra on RocksDB

Instagram is running one of the largest Cassandra deployments. In this year, the Cassandra team in Instagram has been working on a very interesting project to make Apache Cassandra’s storage engine to be pluggable, and implement a new RocksDB based storage engine into Cassandra. The new storage engine can improve the performance of Apache Cassandra significantly.

In this talk, we will describe the motivation and different approaches we have considered, the high-level design of the solution we choose, also the performance metrics in benchmark and production environments.

Pengchao Wang (aka @wpc) is a software engineer from Instagram Cassandra Team. Before Instagram, he worked for a consulting company helping build high-quality software products/services for 11 years. Father of two kids, an open source enthusiast. Hobbies are coding and tea.

Attendees (140)