This is the fourth instalment of our series of Data Engineering Meetups.
For this edition, we have an amazing lineup of speakers, both from Zalando and outside. We have plenty of time for discussions before, during, and after the talks. Interested in data engineering? Join us, and share your story!
18:00 - 18:30 Doors Open: drinks, and discussions
18:30 - 19:00 Francesco Mucio, Zalando: Scaling data infrastructure in the fashion world; or, “What is this? Business intelligence for ants?”
19:00 - 19:30 Aarni Koskela, Valohai: Say no to notebooks!
19:30 - 20:00 Break: food, drinks, and discussions
20:00 - 20:30 Sergii Kamenskyi, Zalando: How to build a data streaming self-service and not get killed?
20:30 - 21:00 Viacheslav Inozemtsev, Zalando: Serverless Ingestion of Event Data
21:00 - 21:45 Networking until end.
For more details on topics and speakers, please read below.
Title: Say no to notebooks!
Speaker: Aarni Koskela, Valohai
Abstract: Say no to notebooks!
… or why you probably shouldn’t be using notebooks after initial data exploration. A quick look at some of the common pitfalls that make using notebooks troublesome in the longer run.
Title: Scaling data infrastructure in the fashion world; or, “What is this? Business intelligence for ants?”
Speaker: Francesco Mucio, BI architect, Zalando
How does your BI team scale when you go from a startup to €4.5 billion per year? Or when you decide to embrace microservices? Is your data infrastructure ready to do data science, machine learning, and AI, or will you be squashed by the weight of the next buzzword?
Title: How to build a data streaming self-service and not get killed?
Speaker: Sergii Kamenskyi, Software Engineer, Zalando
As your organization grows, your data streaming platform grows as well. It faces several challenges, and not all of them are pure technical problems. In many cases, social aspects and user interaction are as important as the code.
In this talk, we will discuss the challenges we faced with the growing number of teams and developers that uses our data streaming service. We will discuss how we drive cross-teams collaboration, promote best practices, enforce security and compliance requirements, but at the same time build trust and developers satisfaction without killing our team. I will demonstrate the web interface of our service that encapsulates all our experience and integrates many tools to create a complete self-service data streaming solution for all Zalando developers.
By the way, It is open source, so you can use it too.
Title: Serverless Ingestion of Event Data
Speaker: Viacheslav Inozemtsev, Data Engineer, Zalando
In the Data Lake project at Zalando one of the major challenges we have is to ingest all the data from Zalando's messaging bus Nakadi, and make this data available for the company in a timely manner. To achieve this we have gone through a journey of building ingestion pipelines with various architectures - from a manually managed monolith, to a fully serverless implementation. In this talk we will present major evolutionary steps we have experienced, and share our most valuable learnings for building data pipelines in the cloud.
Our Data Engineering Meetup is an event by engineers, for engineers. We aim for short, practical talks about experiences with data engineering at scale. We don’t do sales pitches, and instead focus on sharing experiences and lessons learned the hard way.
If you like to talk about your experience at our next Meetup, get in touch, we’d love to hear from you!