Session IV: Hive Deep Down

Hi all,
In this session we look in more detail on how to extend hive with
1. custom mappers and reducers
2. Writing user defined functions (UDF) and aggregation functions (UDAF)

Prerequisite:
it will greatly help if you try to calculate number of male and female raters per genre in the movielens dataset before coming to this meetup. While you can easily do this for a single genre, try to write a single query that generates count of male and female raters for all the genres. Try not to write too many if else conditions.

Join or login to comment.

  • Ritesh A.

    Hi all,

    I just completed writing the exercise for tomorrow's session. Its over here: https://docs.google.com/document/d/1MuHUoPCIZPEfOQI5CwJmR0IlEzKQeZMrL29CPLVzWkw/edit?usp=sharing. Tomorrow we will tackle an interesting problem of determining whether there is a significant gender bias in movie rating. Enjoy.

    September 14, 2013

    • A former member
      A former member

      Hey Ritesh. I have missed the sessions after the 2nd session as I am not in US any more. Is the exercise for the 4th session all about just installing? The doc does not have anything else. If the doc was not supposed to have everything, can you point me to something to catch up? Thanks :)

      October 16, 2013

    • Ritesh A.

      Hi Vaidik. Follow my blog post on how to write UDF and evaluating if there is a gender bias in rating

      October 16, 2013

  • A former member
    A former member

    Great, as as always !

    September 15, 2013

  • Nirav

    Ritesh, Can't make it tomorrow morning

    September 14, 2013

    • Ritesh A.

      Hi Nirav,

      September 15, 2013

    • Ritesh A.

      missed you. But anyways you can easily catchup. Let me know if you have any problems with the exercise. This time I didn't a great job of documenting the steps as I wanted everyone to come with their own ideas. But I will try to do it now.

      September 15, 2013

  • Sergey

    super.. going deeper and deeper.. nice diving!

    September 15, 2013

  • Nirav

    Anyone interested in carpool from San Rafael/Marin, Berkeley?

    September 9, 2013

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Allison

Meetup has allowed me to meet people I wouldn't have met naturally - they're totally different than me.

Allison, started Women's Adventure Travel

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy