April Meeting: Exploring Design Patterns For Data Integration - with John Miner
Details
The Albuquerque SQL Server User Group (ABQSQL) will be having its next meeting on Friday, April 8th from 11:30 AM to 1:00 PM online via Microsoft Teams.
This month John Miner, Microsoft Data Platform MVP and Senior Data Architect at Insight Digital Innovation, will be examining different methods for building a robust data integration pipeline. Please join us for his presentation “Exploring Design Patterns For Data Integration Pipelines". Ask John your data questions, and get to know your peers!
Presentation description: "Both Azure Data Factory and Azure Synapse Analytics support the use of data integration pipelines. Within this data integration framework, there are many ways a business problem can be solved. How do you pick the best solution for your client?
The talk will start with reviewing three ways to solve data integration problems: traditional pipelines, mapping data flows and wrangling data flows. It is my opinion that traditional pipelines allow the developer to write a flexible meta data driven ELT framework.
Data might exist on premise or in the cloud. Therefore, it is important to know how to set up and configure a self-hosted integration engine on a virtual machine for a hybrid cloud design.
The talk will finish with a comparison between hard coded and dynamic pipelines. The former is only good for a specific business use case and will cause a maintenance nightmare over time. The later may uses parameters at the linked service, data set and pipeline levels to promote re-use of the objects.
At the end of this talk, the developer will be very knowledgeable at the different ways to solve business problems with data integration pipelines."
Attend this FREE data platform training event!
Microsoft Teams meeting:
