Skip to content

Details

This is the fifth in a short series of presentations on Uber's Dragon data integration toolkit (https://eng.uber.com/dragon-schema-integration-at-uber-scale). This session will return to the graph construction use case from Part 2 (ETLing a data catalog into an RDF triple store) and illustrate the end-to-end flow of schemas and data in detail, examining each of the major steps in the transformation pipeline. Subtopics may include, but are not limited to:

  • Dragon's core data model
  • Monads for schema and data transformation
  • Data access paths and transformation steps
  • Schema-level transformations
  • Data-level transformations
  • Validation rules
  • Language descriptions and options
  • Type constraints, annotations, and type promotion
  • Global operations on schemas
  • Lexical operations
  • Specific schema transformers, including YAML, Protobuf, Thrift, Avro, OWL, SHACL, Java, and Haskell
  • The command-line interface

Videos from previous sessions:

Other links:

Members are also interested in