Batch Data Processing at Spotify with Luigi

Hello everyone,

Luigi is an open-source Python framework that helps you build complex pipelines of batch jobs, handle dependency resolution, and create visualizations to help manage multiple workflows. Meetup member and software engineer Andy Sloane from Spotify is giving the talk.

Luigi also comes with Hadoop support built in (and that’s where really where its strength becomes clear). Luigi provides an infrastructure that powers several Spotify features including recommendations, top lists, A/B test analysis, external reports, internal dashboards, and many more. Here is the repo link: https://github.com/spotify/luigi
Hope to see everyone there!

Cheers,
Pitt