Data Operations: a New Discipline for Data in Motion

6:00pm Food, drinks, networking

6:30pm Data Operations: a New Discipline for Data in Motion - Pat Patterson (StreamSets)

Traditional data integration had a waterfall model; we imagined that you could take inventory of your data sources, integrate them into a data warehouse, and walk away, basking in the glow of a job well done. Unfortunately, reality turns out to be somewhat more challenging. New data sources coming online and changing business requirements mean that many projects never even complete their initial discovery phase. The rise in data volume, evolving schema, and the need to continuously process data streams soon overwhelm even those enterprises that do manage to deploy a data warehouse.

The data operations discipline, in contrast, considers change an inevitable, but manageable issue, promotes an agile approach, and iterates around a build-execute-operate cycle. Companies such as GlaxoSmithKline and Cox Automotive are already reaping the benefits of this more flexible approach to managing data in motion. Employing a strategy of continuous iteration, they integrate new and changing data sources without the headaches of the traditional model.

StreamSets technical director Pat Patterson will explain the need for data operations, just what it entails, and how enterprises are benefiting from this agile approach to working with data in motion. This is a great opportunity for everyone involved in data engineering, from developers to architects and CTO's, to learn about this emerging area.

7:30pm Hang out and chat some more

8:00pm Finish