Using Lenses to Structure State and Conquering Hadoop with Haskell

7pm -- Using Lenses to Structure State: A Sure Bet
8pm -- Hadron: Conquering Hadoop with Haskell

----

7pm -- Using Lenses to Structure State: A Sure Bet
Nathan Bouscal

This introductory talk will present a simple Haskell program capturing the game of poker. Using this example, we will walk through tactics for program design in Haskell, from constructing appropriate datatypes to using monadic state. Along the way, we'll see how basic uses of the "lens" library can tie things together neatly, letting the underlying domain logic shine through.

----

8pm -- Hadron: Conquering Hadoop with Haskell
Ozgun Ataman


Hadoop provides a powerful tool for batch-analyzing large amounts of multi-structured data. It can be made to tap into various sources of raw data in their dormant state and parallelize computations onto hundreds of nodes.

Yet Hadoop's ecosystem leaves something to be desired for Haskell programmers. Complex data formats such as custom-format logs, proprietary data stores and deep JSON files must be decoded into something the Hive/Pig ecosystem will understand, introducing many redundancies along the way. The alternative is to drop down to Hadoop Streaming and use the available Java/Python/Ruby/Scala toolchain, abandoning the advantages of Haskell.

In this talk, we will introduce Hadron: A Haskell library/toolkit that makes it possible to construct Map-Reduce programs in Haskell and run them on Hadoop as smoothly as possible. We will discuss what motivated Hadron, how its design materialized within a very limited timeframe, how a "free monad" was used to kill two birds with one stone, and some of the API pain points that remain as future work. 

Hadron has been developed by Soostone originally for use in client projects and will be formally open sourced following this month's NY Haskell Meetup. 


Join or login to comment.

  • Sami B.

    Hadron the new Haskell Hadoop lib is very impressive.
    Especially considering the short development time.

    1 · January 23, 2014

  • E\/IE B.

    now we can all secretly do Texas Holdem' while pretending to work in the console.

    1 · January 23, 2014

  • Alim S. G.

    Other obligations :-(

    January 15, 2014

  • Alim S. G.

    Can't make it. CB committee meeting. Will this be recorded and posted?

    1 · January 15, 2014

  • timmy_tofu

    Oh, also the map shows 6th Ave in Brooklyn, not Manhattan, because the map link uses zipcode[masked]

    January 9, 2014

  • timmy_tofu

    +1 for more meetups on not-Monday, looking forward to it.

    1 · January 9, 2014

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Bill

I started the group because there wasn't any other type of group like this. I've met some great folks in the group who have become close friends and have also met some amazing business owners.

Bill, started New York City Gay Craft Beer Lovers

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy