Data Engineering and the Death of Visual ETL

Details

This meetup features two presentations. First, Craig Warman is going to demonstrate how Snowflake can seamlessly ingest JSON data and then make it available to end users without the need for complex ETL processing. Then Stewart Bryson from Red Pill Analytics will deliver the featured presentation on Data Engineering and the Death of Visual ETL.

Agenda:
6:00 - 6:30 PM Networking, Food and Beverages
6:30 - 6:45 PM Automated JSON Data Handling with Snowflake (Craig W)
6:45 - 7:45 PM Data Engineering and the Death of Visual ETL (Stewart B)
7:45 - 8:00 PM Q&A

Data Engineering and the Death of Visual ETL - Summary:

A funny thing happened on the way to modern, data-driven analytics: We ditched the GUI for ETL development. There are drawbacks to moving data using code instead of clicks, especially for organizations heavily invested in visual tools. But writing code has an upside as well.

In this presentation, we’ll explore the data engineering discipline, and how it compares to conventional ETL. Using Databricks and Snowflake together, we’ll look at classic batch and modern streaming use cases, and consider DataOps requirements around testing and delivery where code really shines. And we’ll do it all inside of an interactive notebook.

Presenter: Stewart Bryson, CEO of Red Pill Analytics