Refactoring Data Vaults with Ontologies
Details
Have you joined a Data Vault project that looks like its source systems rather than a proper Data Vault 2.0 warehouse?
A well-intentioned team has pushed ahead and developed quickly, leaning on existing data architectures and entity relationship diagrams without embracing many aspects of the Data Vault Method.
One of the many advantages of the Data Vault 2.0 approach is that warehouses can be refactored quickly and easily, but it can be challenging to know where to begin without upsetting technical and business stakeholders. An ontological approach might be a good place to start. By putting business processes and concepts into a semantic framework, you can translate business users’ experiences into a skeleton of your core data model.
But this ontological approach can go further than supporting a refactor. The patterns in the connections between your semantic entities can help guide your delivery of new and novel information to the business. Areas of dense connection between departments or entities are perfect places for exploration links. These can feed Business Intelligence and Artificial Intelligence teams new information, growing the Warehouse’s value.
With a robust and tested semantic framework, you can go even further. The framework can translate directly into an architecture for graph databases or graph networks in the Information Delivery Layer. These tools can handle deep-join queries and novel AI with considerable gains in performance and flexibility.
Speaker Bio:
Richard Strange is a Doctoral Researcher at the Department of Earth Sciences at the University of Oxford, where he is creating AI models to better forecast volcanic eruptions, and create the unified datasets that feed them. He is also a Certified Practitioner in Data Vault.
He is responsible for platform and data management for the Frontier Development Lab initiative (http://fdl.ai/), a collection of NASA and ESA state-of-the-art AI research projects, and have authored several papers including a NEURIPS Best AI Paper in Climate publication through the organisation.