[ONLINE] Kafka as Real Queue + Dropping ACID
Details
LAST MINUTE CHANGE: This event was changed to ONLINE instead real-life-event due to the rising COVID situation in Israel. We prefer to keep every one safe as top priority. We'll do this event and next ones online, as we've done successfully in the last 1.5 years.
This second meetup in July, will host 2 incredibly interesting lectures. The first will showcase how Draftkings managed to add real queue semantics to Kafka at scale, while the second will explain how to drop ACID (DB Semantics) and still save you sanity.
AGENDA
20:15 - 20:30 Virtual huddle
20:30 - 21:15 How we Built a Highly Scalable User Engagement Platform / Idan Fridman - Draftkings
21:15 - 21:25 A Short break
21:25 - 22:10 Dropping ACID / Avishay Ish Shalom - Scylla
=] How we Built a Highly Scalable User Engagement Platform / Idan Fridman
At Draftkings we were faced with a challenge of implementing a complicated workflow in an existing micro service architecture, with demands to hundreds of milliseconds latency, zero data loss and topping it off with horizontal scalability. This was all needed to build our real-time user-engagement platform. In this lecture we'll go through the architecture we chose for it (queue based), the different technologies we considered for it and then explain why we eventually chose Kafka + Redis to turn Kafka into a real queue semantics. We'll wrap up with the caveats we encountered and how this can be further improved.
== Idan Fridman / Draft Kings
A software addict with over a decade of experience in various technology fields: software development, team management, software architecture and product. I have focused on best practices and architecture for more than 10 years. Over the last 7 years, I have been focusing on building high performing teams in startups and bigger organizations. I’ve taken a big part in building software teams and infrastructure. I was the CTO in my last company (Blueribbon) which has been acquired by a public company DraftKings (NASDAQ: DKNG). Currently I Am director of engineering at Draftkings
=] Dropping ACID / Avishai Ish Shalom - Scylla
Traditionally, ACID is at the core of database usage. Like the JVM memory model it represents the fundamental promises made by databases to programmers, ensuring data sanity in an insane world. Unfortunately, ACID as imagined by programmers exists only in fantasy - in reality the ANSI standard is poorly defined and understood and databases often deviate from it, with possible data anomalies. If that’s not bad enough, ACID requires a specific use pattern which is rarely implemented as prescribed due to performance concerns, thus the programmers themselves often break ACID guarantees. Given this poor state of affairs, we must ask: how do we restore data sanity to the world? Is ACID the right tool? What else is available?
== Avishai Ish Shalom - Scylla
״In a world where anything has an API, everything is a software problem״ this insight has guided Avishai Ish-Shalom throughout his diverse career working on improving the complex socio-technical systems that create and operate modern software and promoting the use of Mathematics in system design and operations. Spending 15 years in various software fields and capacities, Avishai has served as Engineer in Residence in Aleph VC, engineering manager at Wix.com, co-founded Fewbytes and consulted many other companies on software operations, reliability, design and culture. Currently Avishai is a Developer Advocate for ScyllaDB (The boring database ;-).
FOOD & DRINKS
Bring your own food - it's an online event
