Cambridge Semantic Web Monthly Meetup




David Booth, Independent Consultant: The RDF Pipeline Framework: Automating Distributed, Dependency-Driven Data Pipelines

Semantic web technology is well suited for large-scale information integration problems involving multiple diverse data sources and sinks, each with its own data format, vocabulary and information requirements. The resulting data production processes often require a number of steps that must be repeated when source data changes -- often wastefully if only certain portions of the data changed.  This presentation explains how distributed data production processes can be conveniently described in RDF as executable dependency graphs, using the RDF Pipeline Framework.  Nodes in the graph can perform arbitrary processing and are cached automatically, thus avoiding unnecessary data regeneration.  The framework is loosely coupled, using native protocols for efficient node-to-node communication when possible, while falling back to RESTful HTTP when necessary. It is data and programming language agnostic, using framework-supplied wrappers to allow pipeline developers to use their favorite languages and tools for node-specific processing.

A live demo of a simple data pipeline will be included.

The RDF Pipeline Framework is open source software available under an Apache 2.0 license.



Join or login to comment.

  • Jim M.

    Oops, I can't make it today, but I just wanted to join the group. Hopefully I'll see you next month.

    August 13, 2013

    • Ted S.

      Excellent! I'll buy you a beer afterwards. :-)

      August 13, 2013

  • David W.

    Hi all, I'm in Cambridge again today and look forward to seeing you all tonight.

    August 13, 2013

  • Justin L.

    Missing this talk with sincere regret (no babysitter for baby this week!). Really wanted to learn this topic!!

    August 12, 2013

  • John C.

    Humanitarian building open data ecosystem between institutions in disaster response.

    August 12, 2013

  • Bernadette H.

    Looking forward to my first SemWeb meetup in Cambridge!

    August 9, 2013

  • Ari D.

    On another subject, would love to discuss the idea of how to best supplement book metadata using RDF--who is using LOD or similar to make books more discoverable?

    August 7, 2013

  • David M. B.

    ... of US Treasury's Office of Financial Research is interested in applications of data standards and semantic technologies generally, and particularly for understanding financial contracts, markets, and systemic interconnectedness.

    July 31, 2013

  • A former member
    A former member

    Will be on vacation in West Sussex.

    July 8, 2013

Sometimes the best Meetup Group is the one you start

Get started Learn more

I'm surprised by the level of growth I've seen since becoming an organizer, it's given me more confidence in my abilities.

Katie, started NYC ICO

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy