Data Workflow Orchestration with Prefect

As data pipelines become increasingly complex and interconnected, workflow management systems are being used to schedule and monitor tasks. Prefect is an open-source workflow management system designed for large-scale data processes. We'll show how to get started with Prefect and also cover how to run Prefect on top of Dask on the cloud to parallelize workflows.

Speaker:

Kevin Kho is an Open Source Community Engineer at Prefect, an open-source workflow orchestration management system. Previously, he was a data scientist at Paylocity, where he worked on adding machine learning features to their Human Capital Management (HCM) Suite. Outside of work, he is a contributor for Fugue, an abstraction layer for distributed compute.