Data API Builder & Community Talk

Agenda:

5-Min Community Talk

Time: 5.15pm-5.20pm
Title: From Object Storage to Browser: Virtualising 1TB of Geospatial Data
Large scientific datasets are often treated as something that must be copied, from
storage to analytics platform before they can be visualised. For weather and
geospatial data, this can mean working on terabytes of archival file formats stored in
object storage, creating significant data access challenges and cloud egress cost. In
my talk, I will demonstrate how I used open-source tools from the Zarr ecosystem,
including Kerchunk and VirtualiZarr to build a data virtualisation pipeline for 1TB of
weather data. My pipeline does not duplicate data, but generates lightweight
reference metadata that maps analytical requests directly to the byte ranges in
object storage.
With the created virtual data layer, downstream applications can request for the
exact byte ranges they need rather than downloading entire files. This approach
enables a purely client-side, serverless architecture, where interactive visualisations
can be delivered directly from object store to a web browser without the need for a
backend proxy or dedicated data service.

Speaker: John Chew
I'm a Physics/Data Science graduate currently building my career in data analytics
and data engineering. I’m passionate about designing efficient data systems that turn
large, noisy datasets into actionable insights.
Recently, I've been exploring modern data platforms like Databricks, where I built a
scalable pipeline to process over multi-terabyte weather data. I enjoy bridging the
gap between technical data work and real-world decision-making.

Time: 5.25pm-6.15pm
Title: Data API Builder
Imagine taking your databases—SQL Server, Postgres, or MySQL—and instantly
transforming them into secure, production-ready REST and GraphQL endpoints
without writing a single line of backend code. That is the power of Microsoft’s Data
API Builder (DAB). This open-source, configuration-driven engine uses a single
JSON file to deliver out-of-the-box CRUD operations, advanced filtering, and
enterprise-grade security via Microsoft Entra ID. Deployed seamlessly via
containers, DAB is backed by a robust Microsoft roadmap well. Let's dive into a live
demo to see how you can go from a raw database to a fully functioning API in just
minutes.

Speaker: Victor Saraiva
Victor is a Senior DevOps and former Senior Data Engineer at the Department of
Health, bringing over two decades of enterprise experience in Government, Finance
and Oil & Gas. With an extensive background in data engineering, automated CI/CD
pipelines, and an impressive suite of active Microsoft Azure certifications including
Azure Administrator, Azure DevOps and Security Engineering, Victor specializes in
building secure, scalable, and highly automated cloud environments.