Resilient events-handling & Kappa Architecture
Details
18:00 - 18:30 - Mingling
18:30 - 19:15 - When the going gets tough - your system needs to get tougher + Guarding the gates! (of hell) - Artyom Shein @ AppsFlyer
19:15 - 20:00 - Why you need to know Kappa Architecture - Yoni Iny @ Upsolver
Talk Title:
When the going gets tough - your system needs to get tougher + Guarding the gates! (of hell)
Abstract:
Your clients’ data is precious. As soon as it enters your system - it’s your responsibility to process it safely, create backups and restore frameworks.
Processing traffic of 10+ Billion events per day isn’t easy.
Even if you write the best and most robust code - the going will get tough: AWS instances that suddenly decide to disappear, networking issues or just huge spike in traffic because business guys managed to sign Pokemon Go without telling you.
And what happens when a problem occurs at the entrance gates?
Traffic handlers aren’t independent and linked with cache systems and messaging busses - and these connections aren’t invulnerable.
Most of our clients’ data comes from the SDKs integrated in real users’ applications, meaning its unique and can’t be reproduced.
In this session Artyom will show how AppsFlyer is trying to prevent data loss which can potentially occur at the earliest stages of the flow, creating resilient events-handling system.
Bio:
Artyom Shein (https://il.linkedin.com/in/artyom-shein-71917723) is a Software Engineer @ AppsFlyer
Talk Title: Why you need to know Kappa Architecture
Abstract:
Kappa architecture is a new software architecture pattern, first described in 2014 by Jay Kreps at LinkedIn.
The key concepts of Kappa architecture are that raw data is saved in an append-only immutable log and processes are performed directly on the raw data.
Kappa architecture evolved from Lambda architecture which utilized two code paths – one for real-time and another for batch processing. In Kappa, we only have a single code path which reduces system complexity dramatically.
In my talk I’ll review Lambda architecture and its pitfalls, Kappa architecture and its advantages and showcase it for a machine learning application.
Bio:
Yoni Iny (https://www.linkedin.com/in/yoni-iny-7496173) is a technologist specializing in big data and predictive analytics algorithms. He’s a co-founder and CTO at Upsolver, a Big Data startup. Before founding Upsolver, he performed several technology roles including CTO of a data science department at IDF’s elite technology intelligence unit. Yoni holds a BA in Mathematics and Computer Science which he started before he was 16.
Resilient events-handling & Kappa Architecture