Behind the Scenes of Scaling Up with Oribi

Our friends at Oribi are hosting this cool online event, focus on typical challenges with transitioning from a startup to a scaleup and the impact on engineering.

----------------------------
How to Ensure Data Integrity in a Complex Software Architecture, Asaf Halili, Senior Software Engineer

Ensuring data integrity in small scale systems is relatively simple, but it becomes exponentially harder when a system scales up.
One of the go-to solutions is to simply “Retry”. However, in a complex system, a simple retrying solution is not sufficient.

In this talk, I’ll describe the scalable & comprehensive solution we implemented to ensure data integrity in our large-scale microservice-based system.

Monitoring Streaming Pipelines at Scale, Ehud Winograd

When our systems are small and simple, it’s not too hard to figure out where our bottlenecks are, but when we grow and expand, and especially when the system is composed of a series of asynchronous operations, it gets much more challenging to find and monitor those bottlenecks.

In this talk I’ll walk you through what worked for us, and what you should consider when monitoring your microservice-based ingestion pipeline.

Evaluating a Big Data Technology for Our Use-Case at Scale, Natasha Chernyavsky

When designing your system for high scale and big data, choosing the right technology can be crucial. Choosing well, will make you prosperous. Choosing badly will probably have a very costly effect on your company (both in human and tech resources). Stakes become even higher if you want to start using an unfamiliar technology, with which you still have zero experience.

And so, correctly weighing a technology’s capabilities (be it a database or a hybrid of solutions working together), its use-cases, performance and scale is a task worth mastering as a Software Engineer, though not always an easy one.

In this talk, I’ll share our journey of adding Spark and Presto to our tech-stack and architecture for one of our features, which points and evaluation criterias we took into account, which conflicts we encountered and how eventually we were able to go live with this solution with high confidence.

----------------------------

Speaker Bios:
Asaf is a senior software engineer. He’s passionate about solving problems by using code, software design & architecture and is a technology enthusiast in general. He started writing code at elementary school applying his passion to almost every aspect in life. Among his hobbies, he’s an hobbyist photographer with a growing collection of cameras.

Ehud has been writing varied code for over a decade now, he loves front-end and user interfaces, but also back-end development, creating beautiful APIs and resilient systems.
He enjoys traveling, diving, and photography, and is happiest when he gets to combine all three.

Natasha is a Sr. Backend and Big Data Software Engineer with over 12 years of experience. She loves solving complex problems and even more so when the solutions are beautiful. Natasha finds almost everything interesting and worth learning (even when it’s below Binging Netflix on her priorities list) and is especially fascinated by data and everything that handling it entails whether its modelling, processing, storing and querying up to its performance tuning.

----------------------------

The event will be streamed live in:
https://tlvcommunity.dev/live