DataOps in action with Nessie, Iceberg and Great Expectations

We are back again with another meet-up in which we will explore modern and innovative data engineering concepts and technologies.

What is DataOps? Why do we need it and how to put in into action?
With the spotlight on Nessie, Iceberg, and Great Expectations, Antonio Murgia will lead us to conquer the great expectations(pun intended) of DataOps: velocity, reliability, quality.

Project Nessie enables version control on data lakes (iceberg tables, views, and delta lake tables). It is a transactional catalog for data lakes with git-like semantics.
Apache Iceberg is a high-performance format for huge analytic tables. It supports flexible SQL commands and schema evolution, it hides partition management and supports time travel.
Great Expectations is an open standard for data quality providing data testing as well as automated documentation and profiling.

Antonio Murgia is a big data architect at Agile Lab, he is also an Apache Spark contributor.
https://www.linkedin.com/in/antonio-murgia/

Grab your drink of choice and join us November 15 at 18:00 on teams.