Data ingestion As A Service using Azure Service Bus, Storage and Azure SQL DB

Join us for an informative and hands-on workshop focused on building scalable web applications using Serverless Architecture for data ingestion on Microsoft Azure. In this workshop, you will learn how to leverage powerful Azure services like Azure Functions, Service Bus, Azure SQL Database and Azure NoSQL Table Storage to develop near real-time ETL with Endpoint solutions for immediate searching and reporting using PBI or EXCEL

We will delve into various topics including , data integration techniques, secure API development, performance monitoring, and log analysis. Discover how to take advantage of Azure's PaaS offerings to streamline your application development process and leverage the power of cloud computing.

Whether you are a seasoned professional or just getting started with Microsoft Azure, this workshop will equip you with valuable knowledge and practical skills to design and develop high-performance serverless applications. Don't miss this opportunity to network with experts in the field and gain insights into the latest developments in Serverless Architecture on Azure. Join us for an exciting day of learning, collaboration, and innovation.

Description for Demo
Show off Azure Cloud Features for the following:

* **Azure Function App**
* Exposes REST endpoints for **search**, **reporting**, and **data access**
* Supports **traditional keyword search** and **Natural Language Search**
* Integrates with **ChatGPT-4o** to translate user questions into structured queries
* Acts as the orchestration layer between data stores and analytics services
* **Azure Service Bus**
* Receives ingestion messages triggered by file arrivals or upstream systems
* Decouples producers from consumers for reliable, scalable ingestion
* Supports retry, dead-lettering, and ordered message processing
* **Azure SQL Database**
* Stores ingested data using a **predefined relational schema**
* **Azure Table Storage**
* Stores ingested data in a **schema-less (NoSQL) format**
* Used for raw, semi-structured, or evolving datasets
* **Azure Storage Account (File Share)**
* Receives inbound files via **SFTP**
* Serves as the landing zone for batch and external partner data
* Triggers ingestion workflows via storage events
* **Event-Driven Ingestion**
* Storage events (file create/update) trigger Azure Functions
* Functions publish messages to Service Bus for downstream processing
* Enables scalable, asynchronous, near-real-time ingestion
* **Azure Application Insights**
* Captures API request counts, response times, and failure rates
* Tracks Function execution health and dependency performance
* Enables fast troubleshooting and operational visibility
* **Natural Language Search (ChatGPT-4o Integration)**
* Users submit free-form questions via API
* ChatGPT-4o interprets intent and generates structured query logic
* Results returned from SQL or Table Storage through the same API
* **Azure Container Instances (ACI)**
* Hosts a **secure SFTP server**
* Isolated, ephemeral compute with no VM management overhead
* Integrates with Azure Storage File Shares for persistence
* **Azure Log Analytics**
* Centralizes ingestion, processing, and pipeline telemetry
* Displays near-real-time **ingestion progress charts**
* Supports KQL queries for operational dashboards and alerts