addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

EXPERTALKS: July 2013 - Big Data with Hadoop (Part 1)

So, if you haven’t heard of Big Data and Hadoop in some time, you must be living in the Lenyadri caves!

Hadoop is a family of tools and the associated libraries for use from multiple programming environments that lets you create and manage a data cluster (using commodity hardware), so you can then run MapReduce jobs on data in the cluster.

MapReduce refers to a class of algorithms, where one arrives at an output only using ‘map’ and ‘reduce’ operations on data that is accessed as key-value pairs

The little yellow elephant (Hadoop’s mascot) also comes with its own little zoo of associated projects including Pig, Oozie and more...

--------------------------------------------------------------------------------------------

WORKSHOP:

We will walk you through getting a basic Hadoop cluster up and running, show you how to get a sample “large” dataset into it, followed by writing our own MapReduce jobs to get some insight from the dataset. We’ll talk about some of the other tools and jargon around Hadoop.

So bring your laptops with the installations below.

--------------------------------------------------------------------------------------------

PRE-REQUSITIES:

Anyone with a Windows laptop will be charged a penalty at the door**.

While there are VMs available for Hadoop you could download and run, we’ll probably just setup and use instances on AWS.

You are encouraged to sign up for your own AWS account. With the AWS Free Usage Tier now available (see http://aws.amazon.com/free/), you incur no cost, but will need a valid credit card to sign up. [...And No, we can’t lend you our credit card... :-)...]

--------------------------------------------------------------------------------------------

LEARNINGS:

The motivation for the talks is to de-mystify the world of Big Data (!!!). We will probably need a couple of sessions to cover the ground, and it is hoped we can schedule these in quick succession (spaced over a few weeks)

--------------------------------------------------------------------------------------------

PRESENTER:

Krishnan Mani currently works on a Hadoop cluster that feasts on usage data from a popular mobile app, and writes applications in Scala that help organise and make sense of all the terabytes.

He claims to be a ‘JaiKishen’ of all trades, and is currently working on a library he wants to open-source called ‘maa.kasam’ (we have no idea yet what it does)

 

** - just kidding...

Join or login to comment.

  • Mayank S.

    Presenting the EXPERTALKS YouTube channel !!!

    Check out - http://www.youtube.com/watch?v=j8yIwcc12uI&list=SPMqXoQWiY8w7cysKOTCgo6nmdsEIjFmYV

    This was our 1st attempt at recording and therefore the video is pretty raw. We apologize for the same. Will ensure good quality videos for future events.

    September 5, 2013

  • Jagdish M.

    Hi Mayank, can you upload the video for the coming session or is it possible if I can attend the session online ? any of this will be very helpful as don't want to miss it and i am not in town

    August 22, 2013

  • Akshay J.

    hey hi can u help me on that day to setup the Amazon AWS. And wat about the missed part of PART-1. are we going to go through that?

    August 21, 2013

  • Mayank S.

    PART 2 of this meetup is announced : Go to http://www.meetup.com/expertalks/events/132508222/

    August 2, 2013

  • Mohammad A.

    It was really a fantastic presentation by Krishnan. Very informative.
    Looking forward to know about hadoop internals how map reduce jobs work internally and may be we can begin some small project.
    Thanks a lot Krishnan, Mayank and all organizers :)

    July 29, 2013

    • Krishnan

      Hi Mohd Adnan:

      Please write in to Mayank and we can collaborate, and this is also an invite for anyone on the meetup group. Thanks!

      August 2, 2013

  • Krishnan

    More than one of the participants is also working on efforts with Hadoop and there were offers to collaborate and/or present at Expertalks. Please write in to Mayank and we are keen to do this. We welcome people not at Equal Experts currently (!) to also present here.

    August 2, 2013

  • Ujjwal L.

    Perfect Talk which unleashed facts for beginners. Would like to go extra mile in the Part 2, talking about Hadoop internals/design and outstanding issues currently being addressed by hadoop such as backup and archival.

    July 29, 2013

    • Krishnan

      Hello Ujjwal: This (Hadoop backup and archival) is not an area that i have any understanding of. Perhaps we can get together with anyone else that is interested, and put together something around this, but it might be for after the meetup, since not everyone might be interested. Feedback is welcome

      August 2, 2013

  • Sandeep

    Wonderful Introduction to Hadoop and MapReduce..Couldn't be better.Waiting for the next meetup to learn more details...

    July 29, 2013

  • Mayank S.

    Here are some areas we will improve upon in the next EXPERTALKS:

    1) more seating (!)

    2) more mics to hand out (so everyone can hear questions from the audience)

    3) bring the role-plays to the front of the audience so everyone can see it

    4) more interactive

    5) we switched the installation instructions from what was published earlier on EXPERTALKS (...apologies...). Will make sure to have more accurate instructions for future sessions

    6) more internet bandwidth.

    7) make it shorter and sweeter. we don't want the sandwiches to go soggy again because Krishnan wanted "15 more mins" [...read 60 more mins.... :-)...]

    3 · July 29, 2013

  • Mayank S.

    The presentation for this EXPERTALKS can be found here:
    http://bit.ly/16e9qLB

    The code can be found here:
    https://github.com/krishnan-mani/mr.git

    July 29, 2013

  • Mayank S.

    Thanks all for the encouragement and appreciation. It was a fabulous experience.
    The session became engrossing because of active participation from everyone. So kudos to all of you.... :-)

    If you would like to share more feedback, nominate a workshop you'd like to present at EXPERTALKS, or suggest more topics, please feel free to contact me at:

    [masked]
    [masked]

    July 29, 2013

  • Kaustubh P.

    Thanks Kirshnan and all the EE team for arranging this.
    I guess the slides/presentation which had the github links and the few instructions will soon be posted here..

    July 29, 2013

  • Chetan K.

    Thanks a lot EE team to let us know about the insight of Hadoop... :) Hope to see next part of it and more expert talks on technology.... :)

    July 28, 2013

  • Amol

    Thanks Equal Expert Team, I really appreciate your initiative in such innovative and highly growing technology. Great presentation and the one I personally most like the #HadoopGame really interesting and easy to understand with things around.

    I would like to see you again with more detailed and handson session on Hadoop and Bigdata. Keep in touch.

    Thanks again Krishnan, Mayank and EqualExperts team.

    Regards,
    Amol

    1 · July 28, 2013

  • Pramod S.

    Great session and great presenter! It was a very good introduction to what big data and hadoop is. Unfortunately couldn't get the VM running because of my 32-bit OS :(. But will intstall 64-bit OS and try it out. It would be great if in the session we have some significant exercise or use case to implement and get good hands on.

    Thanks to Krishnan, Mayank and EqualExperts team.

    July 28, 2013

  • Anirudha

    Thank you krishnan,mayank and team for providing us with such an adorable environment to share and extend our knowledge...Great job !!

    July 28, 2013

  • Girish K.

    Thank you Krishnan and also Mayank who gave me this opportunity to attend this workshop.I learned so many things from this workshop.

    Waiting for next meetup.......!

    July 28, 2013

  • Monark

    Great event with interesting people, thumbs up for Krishnan and organiser. Lookin forward for future meetups and collaboration from all members to make the community stronger

    July 27, 2013

  • Shreyas P.

    Thank you Krishnan, Mayank and the entire team. The Meetup was well organized and structured. The 'role-plays' indeed played an important role to give a better understanding of the Nodes.
    And of course, the sandwiches and soft-drinks were good too! :) Looking forward to seeing you guys again.

    1 · July 27, 2013

  • PARAS S.

    The whole hadoop thing was beautifully explained by krishnan..thnx for that and thnx to all Equalexperts team...

    July 27, 2013

  • Anand C M.

    Couldn't have been better than this to start with to get an understanding of Big Data with Hadoop . Thanks a lot to Krishnan M , Mayank S and Team for initiating this with such a overwhelming response in the very first part of this series.

    July 27, 2013

  • Lalit B.

    Thanks Krishnan for the excellent introduction to Hadoop and thanks to the team for wonderful grilled sandwiches :)

    July 27, 2013

  • Manish

    Excellent intro by Krishnan Mani. Thanks for your time, and effort.

    July 27, 2013

  • Aaditya G.

    Attended the talk today. It was good and very informative. Thank you all and I will be looking forward for the next meet up.

    July 27, 2013

  • Deepesh C.

    Would you please make one spot for me?

    July 27, 2013

    • Mayank S.

      Yes Deepesh Chaudhari. You can come down.

      July 27, 2013

  • Prakash W.

    Sorry,will not be able attend.

    July 26, 2013

  • Chetan C.

    Due to medical emergency need to travel to native place

    July 26, 2013

  • Nilesh P.

    Due to some urgent work i will not attend. I have just cancelled my RSVP.

    July 26, 2013

  • Mayank S.

    Hi All,

    One last thing. You do not need to bring any print outs. We will check you in at the venue.

    Let's save paper.

    July 26, 2013

  • Girish

    Good workshop. . . Any empty space ?

    July 22, 2013

    • Girish K.

      thanks dear how they will identify me pls tell me because i am going to in place of you...........>>>­>!

      July 26, 2013

    • Mayank S.

      @Girish Korde - you can come for this event. I will let you in... :-)

      July 26, 2013

  • Mayank S.

    Details for tomorrow:

    PARKING - Tomorrow being Saturday, most offices are closed so there will be ample parking available in both basements of Cerebrum IT Park - B3

    SECURITY CLEARANCE - Security at the building checks your car boot. They don't check your IDs usually. But still, carry them

    REPORTING TIME - On or before 10 am. And PLEASE be there on time

    LAPTOP POWER ACCESS - We have close to 150 power points. There won't be an issue. But still, bring your laptops charged

    INTERNET ACCESS - There are wired and wireless connections in the office. You will have access

    AWS INSTANCES - You don't need to worry about AWS. The installations will be provided in multiple ways

    FOOD - We'll order grilled sandwiches for lunch. No gravy dishes. This will keep the office from getting messy. Also, it will help us interact more freely during lunch

    DIRECTIONS - There is only one "D-Mart" in Kalyani Nagar.
    Its on the ground floor of Cerebrum IT Park - B3. We are on the 2nd floor above D-Mart

    1 · July 26, 2013

  • Amitesh R.

    Thanks Mayank for organizing the workshop.

    Appreciate if following info could be shared -
    1. parking availability
    2. security clearance to get inside the premises
    3. whats the reporting time?
    4. access to power for laptops
    5. How many AWS EC2 instances should we have ready?
    6. we bring our own internet access?

    July 26, 2013

  • Mayank S.

    Hi All.... 24 hours to go for this workshop !!!
    Looking forward to see you all. Please reach the venue on or before 10 am sharp. Don't get late.

    Many people have insisted for extra registrations. Accordingly, I have added 5 more tickets for this event.

    The venue for this event (...our office...) is fairly small & cannot accommodate more than 60 people. However, because there are back outs at the event, we're OK releasing 70 tickets for the same.

    1 · July 26, 2013

    • Shreyas P.

      Awesome, Looking forward to it. Any directions or landmarks for the venue? I don't often visit Kalyani Nagar, hence a little unfamiliar with the location.

      July 26, 2013

    • Mohammad A.

      Thank you Mayank :)

      July 26, 2013

  • Girish

    Thank you Mayank :)

    July 26, 2013

  • Deepak C.

    Hey Mayank, today I got the invite for this meetup. I'm really looking forward to attend this. Do we still have any registrations available. plz let me know

    July 25, 2013

  • Jayant

    Would like to attend, but no spots left.

    July 25, 2013

  • Ajit W.

    I am too really excited about this event

    July 25, 2013

  • Monark

    Really excited about this.

    July 9, 2013

    • Girish K.

      Sir pls try to do something i dont want to miss this chance in case any one cancel pls let me know

      July 25, 2013

    • Girish K.

      khade rehne ke liye to jagaha milegi to bhi bas hua.....!

      July 25, 2013

  • akash

    i want to attend it but no seats left

    July 14, 2013

    • Mayank S.

      There are 4 spots left. You can register if you'd like.

      July 23, 2013

  • Sandy

    Due to some urgent personal work im changing my RSVP and giving opportunity to others ..

    July 23, 2013

    • Mayank S.

      Thanks Sandy. That's very considerate of you.

      July 23, 2013

  • Kailash V.

    I am out of City on 27th July

    July 23, 2013

  • Gaurav S.

    Have some urgent personal work. Will really miss the workshop!

    July 20, 2013

  • Chetan C.

    Good to Attend

    July 16, 2013

  • Aaditya G.

    Thank you !

    July 15, 2013

  • Mayank S.

    I've added 5 more tickets... this maxes out our capacity.

    July 15, 2013

  • Ratnakar

    any cancellations ?

    July 15, 2013

  • Aaditya G.

    Was looking forward to join this meetup but no spot is left. So please give an extra entry or please arrange a live telecast or a webinar kind of meetup so that everyone else remaining can join , if it is feasible for you.

    Thank You !

    July 14, 2013

  • Pankaj G.

    Love to measure data cluster scalability & performance ..

    July 13, 2013

  • Anand C M.

    YES

    July 11, 2013

  • Nilesh P.

    PG student

    July 11, 2013

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy