addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Data Integration and CDC in a Hadoop world: A working session

  • Mar 26, 2013 · 7:00 PM
  • T4G

Ideally I would like this to be a sort of panel where people share their large Data Integration and Change Data Capture problems. I would like to hear the larger or unique integration challenges regardless of whether or not you are using Hadoop.

Bring your:
1. Data Integration (ETL and otherwise) challenges
2. Change Data Capture (CDC) challenges
3. Hadoop, NoSQL, MPP Data Integration challenges

If you can, please submit 1-3 slides describing the issue (A diagram and some points) and I will put the whole deck together for all to consume. Alternatively you can whiteboard the concept but then no one can take it away after. The goal here is for people with relevant experience to come together and be aware of different approaches, how Hadoop may or may not fit in, and how other newer technologies are handling it.

I will cover some common Hadoop Integration scenarios that my colleagues and I have come across. I will also summarize LinkedIn's recently Open Sourced DataBus as an example of new technologies available (if anyone else has concrete experience with this or would like to summarize it then please feel free to take over that aspect of the discussion). http://engineering.linkedin.com/data-replication/open-sourcing-databus-linkedins-low-latency-change-data-capture-system

To that end, I would rather have more participants than observers if possible, but I suspect there will be many that would benefit from listening to the conversion. Please consider what you will bring to the discussion (questions or answers are welcome) before you RSVP.

Join or login to comment.

  • Adam M.

    Here is the deck. I still need to look over the audio recording.

    http://www.slideshare.net/adammuise/2013-march-26thugetlcdctalkingpoints

    March 27, 2013

  • A former member
    A former member

    Nice hadoop ecosystem video
    http://vimeo.com/m/29844999

    March 26, 2013

  • Adam M.

    On my way....

    March 26, 2013

  • David B.

    Our product development at InfoTrellis (AllSight Platform) is all around Big Data integration so happy to talk about what we are doing.

    March 26, 2013

    • Adam M.

      You are in the agenda already Dave. :)

      March 26, 2013

  • Raghu S.

    It seems this meet is more for experienced big data operators. I am still a beginer

    March 26, 2013

    • Adam M.

      This is for experienced architects. You don't need to be a Hadoop expert at all.

      March 26, 2013

  • Sridip

    Stuck at work with some production issues. See you guys next time

    March 26, 2013

  • Tri N.

    Oh sorry I forgot to mention about the parking. Starting 6PM parking on both sides of the street is free. On the portion of Broadview Ave between Queen and Eastern Ave.
    http://maps.google.ca/maps?q=43.658108,-79.349617&hl=en&ll=43.658108,-79.349613&spn=0.01096,0.022552&sll=43.6581,-79.349653&sspn=0.01096,0.022552&t=m&z=16

    I'll post signs of "Toronto Hadoop User Group" at the entrance and along the wall to guide you to the meeting room.

    March 26, 2013

  • Tim C.

    Something came up, seems to happen a lot with these meetups which is a shame I really want to get out to this.

    March 26, 2013

  • Raghu S.

    I will participate more as a learner

    1 · March 25, 2013

  • David T.

    I'm claiming observer status! I'll drop out if space becomes tight.

    March 8, 2013

  • Jorge E.

    I'll be attending with the Canopy Labs development team. Their contact information is below:

    - Jean Saad ([masked])
    - Peter Zhang ([masked])

    March 4, 2013

  • Ali

    Sorry, but I will be just an observer.

    March 4, 2013

  • Adam M.

    If you are on that RSVP list I *will* be polling you for your data integration questions and challenges before this meetup.

    March 2, 2013

37 went

Our Sponsors

  • IBM

    Meeting facilities, expert speakers, free product, books and education.

  • Big Data University

    Free on-line courses in Hadoop and big data related technologies.

  • Cloudera

    10% off training for Toronto Hadoop User Group members.

  • Hortonworks

    Food, speakers, beverages

  • T4G

    Hosting Meeting locations and providing relevant speakers

  • Paytm Labs

    Paytm Labs offers a venue for the THUG.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy