addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1light-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

AN EXAMPLE OF A MAP/REDUCE ALGORITHM USING R AND HADOOP

In this session, Anette Bergo talk will give us a tutorial on conducting statistical analysis on large data sets.

R has been described as 'a DSL for statistical analysis'. Hadoop is for LARGE scale computing. Between them, they can take on a number of interesting problems - once you get them to play together. Which is actually both easier and more accessible than you might think. In this demo I will solve a simple map/reduce problem in R, and run it on an Amazon EMR cluster.

Anette is a consultant for ThoughtWorks where she builds people, teams, projects and occasionally a bit of code. She has worked in a number of different countries, industries and development stacks to solve all sorts of problems, but lately it has been R and EMR and big piles of data that has been taking up her time.

This is a hands on session so please bring a laptop with R stats analysis package installed.

Don't forget to sign-up on the Skills Matter site after you RSVP on meetup http://skillsmatter.com/podcast/home/an-example-of-a-mapreduce-algorithm-using-r-and-hadoop

Join or login to comment.

  • Kuan

    I agree, it was really interesting and helpful, thanks again Anette and Yodit!

    Helen I couldn't bring a laptop so I took extensive notes, I'll write them up on a blog when I can (it won't be for a while though).

    Anette is it OK please if I mention the URL of your repo in my write-up? (or might the slides be made available please?)

    August 23, 2013

    • Helen L.

      Thank you Kuan. Everyone else - this is a really helpful blog. Read it!

      September 25, 2013

    • Kuan

      Helen thank you for your kind words, I'm glad you found it helpful!

      September 25, 2013

  • Yodit S.

    Just to let you know that the video from the session is now available, http://skillsmatter.com/podcast/java-jee/an-example-of-a-mapreduce-algorithm-using-r-and-hadoop

    The code can be found on github, https://github.com/anettebgo/wid.demo

    1 · August 24, 2013

  • Sarah G

    An interesting session, thanks! :)

    August 23, 2013

  • Shruti T.

    Good session, indeed :). Got a nice kick-start on integrating R with hadoop and scaling with amazon EMR. Thanks Anette and Yodit!

    August 23, 2013

  • A former member
    A former member

    I like the meeting and I have seen some nice example in R Language.
    I am just supprised by extreme sensitivity of Kass regarding my few comments that I made. I do not think I know everything about R that is why I visited the meeting.

    August 23, 2013

    • Vanessa

      Ladislav next time let the presenter answer the questions, rather than taking over, i don't think Kass was being sensitive at all, but was only expressing what many of us felt.

      1 · August 23, 2013

  • Nicola H.

    Great to meet such cool like-minded women. Brilliant that we're not frightened to take on really quite imposing topics like Hadoop. Learnt that AWS has terrible UX. If I can make a call out for a mentor I'm looking for a Pythonista with experience using NumPy and SciPy to show me the ropes. All help appreciated, will clean your toilets, organize your DVDs into alphabetical order, etc!

    August 23, 2013

    • Yodit S.

      Give me a shout, always happy to do a brain dump yodit at atomicdatalabs dot com. Won't make you clean my toilet you can just buy me a drink at the next meetup :0D

      August 23, 2013

  • Helen L.

    I agree with Kass, I got more than I expected out of the session even though I'm an R newbie

    Will there be any notes from the session for anyone who wants to work through it again at home?

    1 · August 23, 2013

  • Kass S.

    A great session, let down only by a) the flakey network and b) the dude in the front who seemed anxious to demonstrate that he knew more than everyone else in the room. Seriously, that's exactly the kind of behaviour I'm seeking to avoid when I attend women-oriented tech events. Very impressed with Annette's professional handling of both issues. Also, session was advertised as ending at 20:00, and I scheduled around that, hence reluctantly had to miss the end.

    1 · August 23, 2013

  • A former member
    A former member

    Sorry, I can't make it tonight.

    August 22, 2013

  • Ela

    Sorry, I cannot make it this time. I will be there in September for the Confidence in Statistics event though!

    August 22, 2013

  • Aida

    Sorry I can not attend :( Will be any video later available?

    August 22, 2013

  • Thidawan K.

    Sorry I could not attend.

    August 21, 2013

  • Galiya W.

    Unfortunately can't make it, hope to see you all next time.

    August 21, 2013

  • Susie M.

    I might be around 15 minutes late - hope that's ok!

    August 18, 2013

  • Cecilia L.

    Sadly conflicting diary :(

    August 15, 2013

  • Jessica L.

    Oh no! I can't go :-(

    July 30, 2013

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy