DataOpsDC Monthly Meetup

Join us for our monthly DataOpsDC Meetup

This event will be Online only, Zoom link will be provided

DataOps is more than just DevOps for Data, come learn about how technologists are building a data-centric culture to gather value from data.

Agenda:
6:00- 6:15- Mingle
6:15- 7:00 - An Intro to Fugue an Abstraction Layer for Distributed Compute

Abstract: Data practitioners use distributed computing frameworks such as Apache Spark to work with big data. One of the major pain points of Apache Spark is its testability. In order to run tests on simple code changes, users have to spin up a local PySpark instance, which takes a few minutes. Even worse, libraries such as databricks-connect forward all of the local Spark code to be executed on a cluster. This leads to very expensive projects, considering both developer time wasted, and unneeded cluster usage. In this talk, we’ll introduce Fugue, an abstraction layer for distributed compute built to speed up the development cycles of Spark projects and address this problem.

Speaker Bio: Megan Yow is a Data Scientist at Sobeys, one of Canada's largest grocery retailers. Her work includes developing machine learning features for Sobeys' personalization engine so as to make customer interactions more meaningful. She is a contributor for Fugue, an abstraction layer that keeps your code and computation native to Python yet easily portable to Spark clusters.