ETL In Azure Made Easy With Integration Pipeline Data Flows - Paul Andrew

Microsoft Data Platform Group Birmingham - Thursday 6th May 2021 @ 1:00pm

Session: ETL in Azure Made Easy with Integration Pipeline Data Flows with Paul Andrew

Abstract:

ETL in Azure Made Easy with Integration Pipeline Data Flows

What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a set of feature rich, graphical, scalable data transformation capabilities to rival the ETL tech we’ve previously had available in Azure.

In this session we will look at Azure Data Factory/Synapse Analytics Integration Pipelines and how they offer a powerful abstraction over the Apache Spark analytics ecosystem. If you have ever transformed datasets using SQL Server Integration Services (SSIS) packages or via Power BI’s Power Query tool, this is the session for you.

Now we can transform data in Azure using our favourite pipeline interfaces but with the support of Spark compute to do the heavy lifting. In this session you will get a quick introduction to the core Integration Pipeline components before we go deeper into the resource Data Flow features. Start using cloud native technology with distributed compute in a convenient, easy to use and rich set of graphical interfaces.

Speaker Bio:

Group Manager & Analytics Architect specialising in big data solutions on the Microsoft Azure cloud platform.

Data engineering competencies include Azure Synapse Analytics, Data Factory, Data Lake, Databricks, Stream Analytics, Event Hub, IoT Hub, Functions, Automation, Logic Apps and of course the complete SQL Server business intelligence stack.

Many years’ experience working within healthcare, retail and gaming verticals delivering analytics using industry leading methods and technical design patterns.

STEM ambassador and very active member of the data platform community delivering training and technical sessions at conferences both nationally and internationally.

Father, husband, swimmer, cyclist, runner, blood donor, geek, Lego and Star Wars fan!

Agenda:

12.30pm – 1:00pm Setup and Join

1:00pm - 1.05pm Introductions

1:05pm - 2:15pm ETL in Azure Made Easy with Integration Pipeline Data Flows with Paul Andrew

Venue:

Wherever you have access to Computer or Smart Device! This session will be online only!

Other Details:

*** Please note registration on MeetUp is required to gain access to the Teams remote link!***

You MUST RSVP to this event to gain access to the remote link! Please contact us on Twitter or via email if you are having any issues joining and we'll do everything we can to help.

Event Organiser Contact Details:

If you need any further details or have any requests for this or future Data Platform Group events, please get in touch.

Nick Edwards
Email: nick.edwards@purplefrogsystems.com

Reiss McSporran
Twitter: @PurpleFrogReiss (https://twitter.com/PurpleFrogReiss)
Email: Reiss@purplefrogsystems.com

Microsoft Data Platform Group - Birmingham
Twitter: @MSDataGroupBrum (https://twitter.com/MSDataGroupBrum)

Sponsors:
Purple Frog Systems
PowerBI Sentinel