A few weeks ago, we held a BigData.be hackathon on a bunch of new open source technologies: Amundsen by Lyft, Delta lake by Databricks and Kedro by Quantum Black (McKinsey).
We not only had a lot of fun at @Dataminded's new offices in the heart of Leuven's historic city centre (#awesome), but all participants learned an awful lot.
During this meetup, we'll walk you through the most prominent lessons learned. Our typical format for the evening:
20h15 Lyft's Amundsen (Nick De Bock, Lineas)
20h45 Databricks' Delta Lake (Wim Van Leuven, BigData.be)
21h15 QuantumBlack's Kedro (Hakim Elakhrass, Prophecy Labs)
A big thanks to Design is Dead to host us for this meetup!
Amundsen by Lyft is a metadata-driven data-discovery application for improving the productivity of data natives by indexing data resources and powering a page-rank style search based on usage patterns. Think of it as Google search for data.
Delta Lake by Databricks is an open-source storage layer that brings ACID
transactions to Apache Spark and big data workloads.
Kedro by Quantum Black (McKinsey) is a workflow development tool that helps you build data pipelines that are robust, scalable, deployable, reproducible and versioned.