Databricks as Data: SQL-Native Provisioning and Agentic Operations
Details
For our next Melbourne Databricks Community Group session, Jeff Aven from Aven Solutions will take us under the hood of StackQL and explore a different way of working with the Databricks platform: treating its APIs as data that can be queried and managed using SQL.
StackQL is an open-source project that exposes cloud and SaaS providers as SQL-accessible data sources. That means standard SQL operations such as `SELECT`, `INSERT`, `UPDATE` and `DELETE` can be used to query, audit and provision infrastructure and services.
Jeff will demonstrate how this works with Databricks, including:
- Querying Databricks account and workspace APIs using StackQL
- Exploring inventory, governance and security posture using SQL
- Using StackQL from the command line, Python, Node.js and PostgreSQL-compatible tools
- Provisioning a Databricks Lakehouse on AWS with Unity Catalog using a state-file-less alternative to Terraform
- Building agentic FinOps and SRE workflows using the StackQL MCP server
- Using human approval controls to protect write operations in agentic workflows
This is an engineering-focused session and will be particularly useful for people working with Databricks platform engineering, infrastructure, DevOps/SRE, governance, FinOps and automation.
There’ll be plenty of opportunity to ask questions, dig into the demos and discuss where this approach makes sense in real Databricks environments.
Looking forward to seeing you there
- Luke
Connect. Share. Learn.