Batch Data Processing at Spotify with Luigi

Hosted By
Pitt F.

Details
Hello everyone,
Luigi is an open-source Python framework that helps you build complex pipelines of batch jobs, handle dependency resolution, and create visualizations to help manage multiple workflows. Meetup member and software engineer Andy Sloane from Spotify is giving the talk.
Luigi also comes with Hadoop support built in (and that’s where really where its strength becomes clear). Luigi provides an infrastructure that powers several Spotify features including recommendations, top lists, A/B test analysis, external reports, internal dashboards, and many more. Here is the repo link: https://github.com/spotify/luigi
Hope to see everyone there!
Cheers,
Pitt

Big Data Madison
See more events
Madison Central Public Library
201 West Mifflin St. Room 302 · Madison, WI
Sponsors
Batch Data Processing at Spotify with Luigi