Ingest at Intuit and an Intro to StreamSets Dataflow Performance Manager

This time we will take a look at the work of Intuit's Data Engineering & Analytics team, and get an introduction to StreamSets' new product, Dataflow Performance Manager.

This event is kindly hosted by Intuit (https://www.intuit.com/).

Please RSVP by 12pm noon on Nov 3, as we need to give an attendee list to Intuit for security badges.

6 - 6:30 pm - Food and networking.

6:30 - 7:15 pm - Cynan De Leon (https://www.linkedin.com/in/cynandeleon), Engineering Manager at Intuit (https://www.intuit.com/): "Big Data Ingest at Intuit"

Building robust pipelines or systems from scratch for ingesting data from across your enterprise can be difficult for a number of reasons: metadata management, compliance regulations, heterogeneous source systems that are out of your control, lack of robust metrics collection and monitoring all along the pipeline, lack of automated operations and recovery. There are also unique problems that pop up when replacing an existing pipeline or system for enterprise-wide data ingestion at scale. Learn about some of the challenges Intuit’s Data Engineering and Analytics team encountered in just this situation, what we’ve done so far on our journey, and what’s next on our plate.

Cynan is an engineering manager of Intuit’s Data Engineering and Analytics group (“IDEA”), which is responsible for Intuit’s central big data platform that leverages the power of the collective data of Intuit’s 45 million customers. This analytics platform allows users to create unique data-driven insights and product experiences that help customers simplify the business of life.

7:15 - 8 pm - Pat Patterson (https://about.me/patpatterson), Community Champion at StreamSets (https://streamsets.com/): "Map, Measure, Master - Introducing StreamSets Dataflow Performance Manager"

Since its initial release in September 2015, StreamSets Data Collector has made its mark as an open source big data ingest platform, enabling data scientists to build dataflows robust in the face of data drift. Announced in mid-September, StreamSets Dataflow Performance Manager provides an end-to-end view of your enterprise's data topology, providing release management, data flow orchestration, intelligent monitoring of data flows, and a central view of related pipelines. Learn more about this addition to the big data ingest landscape, and see Dataflow Performance Manager in action!

Pat has been working with Internet technologies since 1997, building software and working with communities at Sun Microsystems, Huawei, Salesforce and StreamSets. At Sun, Pat was the community lead for the OpenSSO open source project, while at Huawei he developed cloud storage infrastructure software. Part of the developer evangelism team at Salesforce, Pat focused on identity, integration and the Internet of Things. Now community champion at StreamSets, Pat is responsible for the care and feeding of the StreamSets open source community.