Princeton Tech Meetup #27 w/ Gilt Groupe (Big Data)

  • June 18, 2014 · 7:00 PM
  • This location is shown only to members

Topic: Big Data - How Gilt Manages Real-time Data Capturing with Kafka, Avro and Hadoop/Hive

Speaker:

 Michael Hansen - Principal Data Engineer Gilt Groupe

Agenda: 

7:00 - Arrival - Snacks, Pizza and Networking.

7:30 - Introduction / Announcements by the organizers.

7:35 - 7:50 - Demos / Pitches

Demo 1 - SuperDealyo - Sach Kangovi

Demo 2 - Gruberie - Sven Hermann

Demo 3 - Outdoor Exchange (OX) - Dariusz Jamiolkowski

7:50 - How Gilt Manages Real-time Data Capturing with Kafka, Avro and Hadoop/Hive - Michael Hansen - Principal Data Engineer - Gilt Groupe 

8:45 - Open-mic to quickly promote your business or broadcast a need that someone in the group might be able to fill. 

8:55 - Wrap-Up, discussion of Meetup, feedback and opportunities for improvement or future topics.

8:59 - End of formal part of meeting.

9:00 - Exit Venue and head to After Hours Party - Location: TBA

More about this Event:

SuperDealyo: Presentation and Demo by SuperDealyo team of a unique location based, shopping list driven platform bringing lowest prices to YOUR fingertips.

Gruberie: Your one stop gateway to great food deals.

Outdoor Exchange (OX): A trusted community based platform where the supply and demand for rental of outdoor gear is met.

Michael Hansen - Principal Data Engineer - Gilt Groupe

Large-scale, real-time (or near real-time) data capture of various clickstream and messaging events has become much more practical with the combination of Kafka and Hadoop. However, without some sort of backward compatible data structure for these data events, a lot of unnecessary transformation and formatting work is left to the data consumers.  This is where protocol buffers, a data serialization system like Apache Avro, or frameworks like Apache Thrift can come to the rescue. This talk will focus on how Gilt uses the trio of Kafka, Avro, and Hadoop/Hive to manage and control data structure for real-time events passed into HDFS/Hive and/or consumed by other web services. 

Join or login to comment.

  • David S.

    Tremendous technical content.

    July 10

  • Michael H.

    Thank you all for attending and for your great feedback. Also thanks to Venu and Chris for doing an excellent job organizing this!

    It was certainly a lot to cover in under an hour, but I'm glad we made it beyond the buzzwords, as that was the intend. For anyone interested, I've posted the entire presentation at: http://prezi.com/gxm3ily-zovb/?utm_campaign=share&utm_medium=copy&rc=ex0share

    4 · June 20

    • JP P.

      Michael thanks for posting the presentation, I will definitely review this again as it is full on insightful information on managing a big data environment.

      June 20

  • Chris T.

    I agree -- it was heavily technical, and I only picked up about 50% of it, but it was worth the time and one of the first talks I've seen to really go beyond buzzwords in "big data".

    1 · June 20

  • Mike S.

    A great technical talk. There were some jeers in the back about it being too technical but when it comes to big data, anything less quickly degrades to buzzwords and hype. I expect more of the same quality from Princeton Tech in the future. Thanks.

    1 · June 20

  • Santosh K.

    It was good turnout and very nice and interesting talk by Michael Hansen. As we introduced in open-mic...in the meet-up..@Veenu - Thanks for the time slot.

    Xenolytics is a Services and Solution providing Big Data company. We can help in data modeling, data analytic, infrastructure setup and software development in Big data space. We can help to migrate/integrate existing Dataware house systems/applications into Big Data platform. We have expertise in the complete Big Data echo system and we work with you to develop analytic and Proof of Concept projects. If you have any need for Big Data initiatives please get in touch with us.

    contact - [masked]
    Website - http://www.xeno-lytics.com/
    Follow us on @Xenolytics
    @SantoshKumarKN

    June 19

  • Shravan (Sean) K P.

    Good Talk by Michael Hansen. Thanks

    June 19

  • Marc M.

    Always glad to be apart of the Princeton Tech Meetup. By any chance, does anyone remember who is looking for Program Managers towards the last 5 minutes of the meetup? Thanks

    June 19

  • Dominic F.

    A little too much info and too much slide content.

    June 19

  • Mohan V.

    This was my first session. Great work in organizing this and a great subject to present.

    1 · June 19

  • Arun S.

    Great turnup, good balance between tech and business

    June 19

  • Sam

    Great Meetup guys - Kudos to the organizers !! missed the Winberie's part ... a round of beer on me next time !! Cheers !!

    1 · June 19

  • Pamela

    Interesting. A lot to digest.

    June 18

  • George

    good discussion of big data collection and management

    June 18

  • Khürt W.

    Got my dates confused. Niece's graduation is tomorrow. I'll be there.

    June 18

    • Venu M.

      Can you bring In
      Video gear this time?

      June 18

  • Mehul

    I'm interested in networking with Data Science/Data Mining Enthusiasts to build models that foster Data Driven Decisions. The idea is one of creating a solution or two that can lay the foundation for a startup. Hacking skills, passion and time would be essential prerequisites. Drop me a line and we can connect at the event on Wednesday!

    June 9

    • Faber

      I'll keep an eye out for you. I'm looking for a data project or two to hack on.

      June 9

    • Sabala K.

      Hi Mehul, I have time and enthu for data science. But can't make it to meetup tomorrow. Please connect on

      June 17

  • A former member
    A former member

    I wish I could be there, but a company outing got rescheduled for the same day. :( I look forward to recaps on the talk. It looks like it's going to be a good one!

    June 16

  • Lauren

    The Sparkway team is excited to be there for the big data talk. We're looking forward to chatting with folks who have great data ideas and need help with the execution.

    1 · June 11

  • Sebastien B.

    hi - are there recommended/useful reads (technical and non technical) on the topic before the presentation?

    2 · May 23, 2014

  • Pamela

    I am a student studying in the field of cyber security. I really am into technology and how to apply defense-in-depth protocols implementing security from attackers. I am open to all information concerning technology.

    June 1

  • Khürt W.

    My nieces's graduation from 8th grade middle school to 9th grade high school.

    May 29

Your organizer's refund policy for Princeton Tech Meetup #27 w/ Gilt Groupe (Big Data)

Refunds are not offered for this Meetup.

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Henry

I decided to start Reno Motorcycle Riders Group because I wanted to be part of a group of people who enjoyed my passion... I was excited and nervous. Our group has grown by leaps and bounds. I never thought it would be this big.

Henry, started Reno Motorcycle Riders

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy