What we're about

This meetup is all about data. From small to big, SQL to NoSQL, BI, reports, development, design, and more. If you're a developer, DBA, BI expert, data analyst, data scientist, or data engineer, then this is a great place to meet and learn.

If you would like to present a session in one of our meetups, then please submit your session here: https://sessionize.com/israeli-data-platform-meetup/

Upcoming events (4+)

Azure Data Engineer - Part 13: Orchestrate Data Movement and Transformation

Join us in this weekly series and learn how to become an Azure Data Engineer, how to integrate, transform, and consolidate data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.

Responsibilities for this role include helping stakeholders understand the data through exploration, building, and maintaining secure and compliant data processing pipelines by using different tools and techniques. You will learn how to use various Azure data services and languages to store and produce cleansed and enhanced datasets for analysis.

An Azure Data Engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints. This professional deals with unanticipated issues swiftly and minimizes data loss. An Azure Data Engineer also designs, implements, monitors, and optimizes data platforms to meet the data pipeline needs.

In each week, we will cover a different module towards the full learning path, which will prepare you for the Azure Data Engineer Associate certification (https://docs.microsoft.com/en-us/learn/certifications/azure-data-engineer/) as well as for the real world.

A candidate for this certification must have a solid knowledge of data processing languages, such as SQL, Python, or Scala, and they need to understand parallel processing and data architecture patterns. Specifically, you should have the knowledge equivalent to the Azure Data Fundamentals certification.

All sessions are recorded, and the entire series can be found here: https://bit.ly/Azure-Data-Engineer-certificate.

This is part 13: Orchestrate Data Movement and Transformation in Azure Data Factory or Azure Synapse Pipeline. In this module, we will learn how Azure Data Factory can orchestrate large scale data movement by using other Azure Data Platform and Machine Learning technologies.

In this module you will:

• Understand the data factory control flow
• Work with data factory pipelines
• Debug data factory pipelines
• Add parameters to data factory components
• Execute data factory packages

Link to the relevant module in Microsoft Learn: https://docs.microsoft.com/en-us/learn/modules/orchestrate-data-movement-transformation-azure-data-factory/

Agenda:

• 19:00-19:15 – Opening, Announcements, and More...
• 19:15-20:15 – Session
• 20:15-20:30 – Q&A and Open Discussion

Azure Data Engineer - Part 14: Execute Existing SSIS Packages in ADF

Join us in this weekly series and learn how to become an Azure Data Engineer, how to integrate, transform, and consolidate data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.

Responsibilities for this role include helping stakeholders understand the data through exploration, building, and maintaining secure and compliant data processing pipelines by using different tools and techniques. You will learn how to use various Azure data services and languages to store and produce cleansed and enhanced datasets for analysis.

An Azure Data Engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints. This professional deals with unanticipated issues swiftly and minimizes data loss. An Azure Data Engineer also designs, implements, monitors, and optimizes data platforms to meet the data pipeline needs.

In each week, we will cover a different module towards the full learning path, which will prepare you for the Azure Data Engineer Associate certification (https://docs.microsoft.com/en-us/learn/certifications/azure-data-engineer/) as well as for the real world.

A candidate for this certification must have a solid knowledge of data processing languages, such as SQL, Python, or Scala, and they need to understand parallel processing and data architecture patterns. Specifically, you should have the knowledge equivalent to the Azure Data Fundamentals certification.

All sessions are recorded, and the entire series can be found here: https://bit.ly/Azure-Data-Engineer-certificate.

This is part 14: Execute Existing SSIS Packages in ADF. In this module, we will see how we can integrate SQL Server Integration Services packages into an Azure Data Factory solution.

In this module you will:

• Describe SQL Server Integration Services
• Explain the Azure-SSIS integration runtime
• Set up the Azure-SSIS integration runtime
• Run SSIS package in Azure
• Migrate SSIS packages to Azure

Link to the relevant module in Microsoft Learn: https://docs.microsoft.com/en-us/learn/modules/execute-existing-ssis-packages-azure-data-factory/

Agenda:

• 19:30-19:45 – Opening, Announcements, and More...
• 19:45-20:45 – Session
• 20:45-21:00 – Q&A and Open Discussion

Analyze Large Amounts of Near-Real-Time Events with Azure Data Explorer

Agenda

• 10:00-10:15 - Opening, Announcements, and More...

• 10:15-10:45 - Analyze Large Amounts of Near-Real-Time Events with Azure Data Explorer

• 10:45-11:00 - Q&A and Open Discussion

Session: Analyze Large Amounts of Near-Real-Time Events with Azure Data Explorer – Alex Peles (30 Minutes, Hebrew)

Analyzing large amounts of events, with near-real-time latency, is a common requirement for online and IoT applications. We will see how it can be done using Azure Data Explorer, a scalable and flexible database best suited for this use case.

Azure Data Engineer - Part 15: Operationalize your Azure Data Factory

Join us in this weekly series and learn how to become an Azure Data Engineer, how to integrate, transform, and consolidate data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.

Responsibilities for this role include helping stakeholders understand the data through exploration, building, and maintaining secure and compliant data processing pipelines by using different tools and techniques. You will learn how to use various Azure data services and languages to store and produce cleansed and enhanced datasets for analysis.

An Azure Data Engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints. This professional deals with unanticipated issues swiftly and minimizes data loss. An Azure Data Engineer also designs, implements, monitors, and optimizes data platforms to meet the data pipeline needs.

In each week, we will cover a different module towards the full learning path, which will prepare you for the Azure Data Engineer Associate certification (https://docs.microsoft.com/en-us/learn/certifications/azure-data-engineer/) as well as for the real world.

A candidate for this certification must have a solid knowledge of data processing languages, such as SQL, Python, or Scala, and they need to understand parallel processing and data architecture patterns. Specifically, you should have the knowledge equivalent to the Azure Data Fundamentals certification.

All sessions are recorded, and the entire series can be found here: https://bit.ly/Azure-Data-Engineer-certificate.

This is part 15: Operationalize your Azure Data Factory. In this module, we will learn how we can publish our Azure Data Factory work between different environments.

In this module you will:

• Configure a git repository with a development factory
• Create and merge a feature branch
• Deploy a release pipeline
• Visually monitor pipeline runs
• Integrate with Azure Monitor
• Set up alerts
• Rerun pipeline runs
• Learn about Azure Data Factory security

Link to the relevant module in Microsoft Learn: https://docs.microsoft.com/en-us/learn/modules/operationalize-azure-data-factory-pipelines/

Agenda:

• 19:00-19:15 – Opening, Announcements, and More...
• 19:15-20:15 – Session
• 20:15-20:30 – Q&A and Open Discussion

Photos (160)