addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwchatcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgoogleimageimagesinstagramlinklocation-pinmagnifying-glassmailminusmoremuplabelShape 3 + Rectangle 1outlookpersonplusprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

Energy Usage Insights with Hadoop & HBase; Cloudera Support of Accumulo

Energy Usage Insights with Hadoop and HBase

Oren Benjamin, Opower

Scott Kuehn, Opower

Opower software has already helped millions of people collectively save
over 2 terrawatt hours of energy and $300 million in energy bills.
Opower's expanding energy usage dataset includes meter and smart meter data streams from around the globe.  The engineering team is leveraging Hadoop, HBase, and Kiji to build a scalable energy usage insight engine capable of ingesting, maintaining, and analyzing these data streams to provide actionable insights that help people manage their energy use.  In this talk we'll cover the system architecture, data schema, and the generation of energy usage insights with M/R.  We'll also discuss how Hadoop and HBase integrate with Opower's substantial relational database backed software.

Bio:

Oren is a Senior Software Engineer working on the Opower data platform. He lead the development of the Opower social product, using Hadoop and Hive along with Opower's nationwide dataset to make a home energy use comparison freely available to anyone in the US, while encouraging users to save energy via social interaction and game mechanics.  Prior to Opower, he lead engineering for AddThis analytics where he designed and implemented the analytics API, share counter, and the open source stream-lib for realtime data stream summarization of web traffic from AddThis' 1B unique monthly users.

Scott is a Data Architect at Opower.  As a member of the Data Services Team, Scott creates Hadoop and HBase services that provide a scalable core to Opower's various products. Scott's career has mirrored his passion for working with data and distributed systems.  As a research consultant at the University of Washington, he created the first high-throughput DNA analysis pipeline capable of processing billions of genomic data reads, resulting in publications of multiple novel insights into the nature of genetic diseases.  More recently, Scott was a Lead Associate at Booz Allen, where he developed a commercial cyber data analytics platform that used Hadoop, Storm, and Accumulo to successfully detect malicious cyber activity from streaming log data.

 

Cloudera support for Accumulo

Joey Echeverria, Cloudera

Apache Accumulo is a vibrant and important member of the Apache Hadoop Ecosystem. Come to hear how Cloudera, the Platform for Big Data, is building out support for Accumulo in our distribution and ensuring tight integration with the rest of the Ecosystem.

Bio:

Joey Echeverria is a Principal Solutions Architect at Cloudera where he oversees the technical success of projects in the Federal and Mid-Atlantic regions. Joey also works directly with customers to deploy production Hadoop clusters and solve a diverse range of business and technical problems. Joey joined Cloudera from the NSA where he worked on data mining, network security, and clustered data processing using Hadoop. Prior to working full time for NSA, Joey attended Carnegie Mellon University where he attained an M.S. and a B.S. in Electrical and Computer Engineering.

Schedule

6:30-7:15 - Snacks and Networking

7:15-7:20 - Announcements

7:20-8:05 - Oren Benjamin and Scott Kuehn

8:05-8:15 - Break

8:15-9:00 - Joey Echeverria

 

 

Join or login to comment.

  • Joey E.

    Here are the Cloudera slides: http://www.slideshare.net/JoeyEcheverria/apache-accumulo-and-cloudera

    Thanks for hosting and the great questions!

    July 26, 2013

  • Scott K.

    Here are the Opower slides: http://www.slideshare.net/OpowerSlides/energy-usage-insightswithhadoopandhbase-24658869

    Thanks for the great questions. We had fun and look forward to seeing all of you at the next meetup.

    July 26, 2013

  • Omar A.

    Amazing job by all those involved especially the presenters. Thanks to Comcast for hosting as well.

    July 26, 2013

  • Vaibhav K.

    Great speakers and insightful content presented. Thanks organizers and speakers. A request for the speakers, please share the link towards the content.

    Thanks Again.

    July 26, 2013

  • Desire

    Great talks, great speakers. Thank you.

    July 26, 2013

  • Won C.

    Great. I loved the real life problem solving by Hadoop technology and the insightful latest info about Cloudera and Accumulo. Thank you.

    July 26, 2013

  • Clint G.

    Great location, great host, great talks. Thank you to BAH, Comcast, Cloudera and Opower for putting this together.

    July 26, 2013

  • Aaron C.

    Awesome talks by Scott and Oren and Joey. Please post slides, please.

    1 · July 26, 2013

  • A former member
    A former member

    Thank you all for attending the meetup. Special thanks to the speakers and the cosponsors. It was a full house event with lots of brilliant people exchanging ideas. Keep in touch!

    3 · July 25, 2013

    • Helen

      Thanks for organizing !

      July 26, 2013

  • chris a

    Running a bit late @#wmata. Hope Ill still have my spot.

    July 25, 2013

  • Krishna C.

    Krishna Chakka. I RSVP'd weeks ago. can I still attend?

    July 25, 2013

  • A former member
    A former member

    Any tips on nearby parking?

    July 25, 2013

  • A former member
    A former member

    Please read this

    Security has informed me that they can only print 100 security passes hence we will be limiting the attendees to first 100 who send in their Firstname and Lastname info. Count crossed 50 today so please hurry up and send me the info.

    Apologies for the double rsvp req but its only because of security folks. thanks

    July 22, 2013

    • John

      John Stocker, thanks.

      July 25, 2013

    • A former member
      A former member

      Jeff field though I also emailed

      July 25, 2013

  • Helen

    Sid, I RSVP weeks ago. Am I still on the list? If not, is there still a seat for me? First name : Yuhan, Last name: Li. Thanks!

    July 25, 2013

  • Erin F.

    Sid
    I'm with Opower and didn't realize until just now that I needed to RSVP. Can I still attend? Erin Fristoe

    July 25, 2013

  • A former member
    A former member

    Repeat - Everyone who sent in firstname/lastname are IN. see you tomorrow! we still have about 20 spots open so drop in if you can.
    text/call me at[masked] if you have questions

    July 24, 2013

    • John L.

      I sent you a text, I would like to get John Lemp and Erik Nienaber added if there is still room. Thanks!

      July 25, 2013

    • A former member
      A former member

      done

      July 25, 2013

  • vincent y.

    I sent my details via text... Just wanted to confirm I am in.

    July 25, 2013

    • A former member
      A former member

      done

      July 25, 2013

  • vincent y.

    What is the closest metro rail stop to event?

    July 25, 2013

    • John H.

      It looks like the McPherson square metro is closest (orange, blue lines)

      July 25, 2013

  • A former member
    A former member

    Thank you all who sent in the firstname,lastname. Security has been informed , just bring in a photo ID and you are all set. Volunteers will be waiting outside.

    Though we will cross 100, we can easily accomodate 10-15 more folks so txt me at[masked] if you wish to attend. Doesnt matter if you are on Waiting List.

    Its going to be exciting!

    July 24, 2013

  • A former member
    A former member

    Everyone who sent in firstname/lastname are IN. see you tomorrow! we still have about 20 spots open so drop in if you can.
    text/call me at[masked] if you have questions

    July 24, 2013

  • Raviteja V.

    Hi.. I am planning to attend tomorrow's event but wondering whether I am on waiting list or the list of attendes.Organizer please help me.

    July 24, 2013

    • A former member
      A former member

      you are in :) see ya tomorrow

      July 24, 2013

  • A former member
    A former member

    Where do we stand on knowing if we are in the extra special first 100 who can attend? I have to make commuting plans around whether I am really in the 100 who can attend. Thanks.

    1 · July 24, 2013

    • A former member
      A former member

      you are:)

      July 24, 2013

  • Samia

    Sorry I won't be able to attend, something just came up.

    July 23, 2013

  • Lance H.

    Lance Howden

    July 23, 2013

  • Peter G.

    Peter Guerra

    July 22, 2013

  • A former member
    A former member

    Important!
    I’ll need a list of names from you to add to the building visitor list (preferably by 4pm on the 24th)…they like to pre-enter & print the stickers when they’re expecting a bunch of visitors to save time (all visitors do still need to show a picture ID whether they’re on the list or not though)

    Please send me First and Last names if you are absolutely sure you are attending directly to [masked] (cc [masked])

    Subject: RSVP

    July 19, 2013

  • Ely K.

    Looking forward to it

    July 15, 2013

  • vincent y.

    I will try. It would really help if someone says hi and that they are interested in doing mobile with their project.

    July 12, 2013

  • A former member
    A former member

    Looking to learn, total noob.

    July 9, 2013

  • Ivan T.

    with Jason Romano

    1 · June 25, 2013

  • A former member
    A former member

    Will be there

    June 17, 2013

    • A former member
      A former member

      See ya there :)

      June 18, 2013

  • A former member
    A former member

    The meetup is Datascience MD. They have folks from NIST and, HHS. Hope to see you there :)

    1 · June 13, 2013

  • A former member
    A former member

    Mr. Kahn, please consider teaming up with Archer. It is a win-win. We are a govt contractor. I have 3 BD folks working for me. We are an 8A, stars II, and IT70 contract holders. We might be getting on Eagle II soon. Come to the shady grove meetup on June 18th if you are not already tied up please.

    1 · June 13, 2013

Our Sponsors

  • Tetra Concepts

    Thank you to Tetra Concepts for sponsoring this meetup.

  • BAE Systems

    Thank you to BAE Systems for sponsoring this meetup.

  • Booz Allen

    Thank you to Booz Allen for sponsoring this meetup.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy