Spark Summit 2014 & Spark at Meetup: The First Year (July 8)

Pizza & Refreshments and room opens at 6:30. Talks will start at 7.

Pre-Talk: Spark Summit 2014 - a recap of some of the more interesting announcements and items from Spark Summit 2014

Talk: Spark at Meetup: The First Year (July 8)

Last summer Meetup was in the enviable position of being able to choose a distributed data processing framework, without any prior commitment to legacy (or perhaps, venerable) frameworks. We decided to start with Spark and Hadoop MapReduce, sharing the same HDFS storage. Most custom jobs would be written for Spark, while tooling like Hive could be used where it fit the problem set.

How did it all shake out? Some joy, some tears, and plenty of growing pains. We’ll  share the story from two perspectives, Evan’s in Hadoop and big data, Erik’s in Scala and performance optimization.

Evan is a Machine Learning Engineer at Meetup, where he is responsible for building intelligent systems that directly affect the user experience. Evan owns the recommendation engine at Meetup from data collection to production. Previously, Evan was on the Machine Learning Team at Orbitz Worldwide and he got his start in the Information Retrieval Lab at the Illinois Institute of Technology.

Erik is one of the creators of Spire, a Typelevel project designed to support fast, generic, and precise numerical programming in Scala. He is a contributor to the Scala compiler and standard library, and uses Scala to push the limits of JVM performance and expressiveness. Erik works at Meetup, where he collects, processes, and analyzes data to learn how to more effectively create community.

Join or login to comment.

  • Erik O.

    I'm not sure how coherent these slides are without the talk itself, but here is the presentation on how Meetup uses Spark: https://docs.google.com/presentation/d/1kyj3KYRX36pkw70qQXBhfAkMuQGoh9G_4balv8_MGYM/edit#slide=id.p

    1 · July 14

  • Prafulla W.

    Are the presentations going to be shared?

    July 9

  • François Le L.

    Thank you everyone for coming and good interactions!

    July 9

  • vijay v.

    Very helpful. both talks were insightful and enjoyable. Hope materials are shared, really love to follow up with some of the folks.

    July 8

  • Nathan H.

    A note from your hosts at Meetup HQ:

    When you arrive at 632 Broadway between 6:30 and 7pm, just tell the lobby staff that you're here for the Spark-NYC meetup on the 9th floor. You don't need to show ID, the sign that says so is for daytime visitors.

    But don't come before 6:30 because they probably won't let you in! Rules and such.

    3 · July 8

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Allison

Meetup has allowed me to meet people I wouldn't have met naturally - they're totally different than me.

Allison, started Women's Adventure Travel

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy