Skip to content

ETL in BigQuery using Airflow & Creating a Secure Knowledge Repository

Photo of Patrick Dinneen
Hosted By
Patrick D.
ETL in BigQuery using Airflow & Creating a Secure Knowledge Repository

Details

In this daytime session, we'll have the following talks:

  1. ETL in BigQuery using Airflow - Steven Cohen

Airflow is a versatile tool which can be used to build basic ETL processes to load data into BigQuery. In this talk, you will learn how Airflow can be used to load and transform data in BigQuery (or anything that supports sql). This includes backfills, retries, complex workflows and Jinja Templating of SQL statements.

  1. Creating a Secure Knowledge Repository using Hugo, Blogdown and OAuth2 - Patrick Dinneen

So you've started sharing your analyses in Jupyter Notebooks and RMarkdown docs via email and in Slack, but now referring back to previous analyses is cumbersome. Getting new starters up to speed on the current state of knowledge is also challenging.

In this talk Patrick will present a solution to this problem by using Hugo and Blogdown to create a static site for sharing analyses. Additionally, some options for securing access to the site using bit.ly's OAuth2 proxy will be presented.

Parking instructions:
There is quite a bit of street parking available. If you would like to park on the Grand Parade please send your license plate number to Gordon.Inggs@capetown.gov.za

Photo of Cape Town Data Science Meetup group
Cape Town Data Science Meetup
See more events
Cape Town City Hall
Darling St · Cape Town, WC