Data Pipelines - From Dev to Production

Are you facing challenges while creating a development environment for your data pipelines? Have you ever found yourself in a situation where you had to deploy your pipeline into production without proper testing? If so, you're not alone. Developing data pipelines presents many challenges and questions, such as whether to use dev or production data, whether to run the pipeline in staging all the time, and how to handle the differences in data behavior between dev and production environments.
Let’s discuss the different practices for the data pipelines development life cycle.

Topic suggester: Keren Kapach

How does it work?
There is no speaker and no presentation, we will just gather around the table and brainstorm, anyone can ask questions and share his ideas.
Just come with an open mind, and share your experiences and challenges.

Food
Thanks to Datorios for hosting us and providing drinks and light snacks.

Agenda
18:00 - 18:30 Gathering together, food and drinks
18:30 - 20:30 Open discussion