Python in handling eCrime and massive web requests in production

On Oct 8, learn about how a giant gaming company uses Python to build an eCrime investigation system and how our long-time sponsor Yelp uses MapReduce and Storm to analyze massive amounts of web requests.

Talk 1: Electronic Crime Investigation in Python

Abstract:

If you allow user-generated content, some of that content will be evil. Your "eCrime" team will need tools to fight evil, both to keep users happy and to satisfy three-letter government agencies. In this talk, you'll find out how one of the biggest gaming companies used the power of Python to build an eCrime investigation system that runs reliably, autonomously, and economically.

Bio:

David Schachter studied Electrical Engineering and Computer Science at Princeton University and has over 30 years of industry experience at Fortune 500 companies (Oracle and HP) and numerous startups (two were successful). He enjoys designing and implementing fast, scalable systems for managing large data flows in real time. Most recently, he designed and managed the implementation of a web crawler in pure Python that out-performs a commercial crawler by 30 times.

Talk 2: Performance Profiling in Production: Analyzing Web Requests at Scale Using MapReduce and Storm

Abstract:

Code profiling gives a rich, detailed view of runtime performance. In production, huge challenges in scalability, access, and ease-of-use appear when profiling even a small fraction of web requests. Yelp tackles these challenges to profile a non-trivial fraction of its traffic by combining Python, Amazon EC2, Elastic MapReduce, and S3. We enable developers to search, sort, filter, and combine interesting profiles. Some of our analyses run nightly, while others run in real-time. We present our use cases for code profiling, its benefits, and our implementation of handlers and analysis flows. We'll also touch on issues such as concurrent logging, cross-data center replication, job scheduling, and API definitions. The entire stack is written in Python.

Bio:

Zach Musgrave is a software engineer on the infrastructure team at Yelp. He specializes in building services that ensure the performance, reliability, and security of Internet-scale Web systems. Zach is from Pittsburgh, got schooled at Clemson and Michigan, and claims to have been an English major in a past life. To his credit, he believes that rewriting systems to handle 1,000x data throughput is a lot more fun than reading James Joyce.

Please sign-up here (https://docs.google.com/forms/d/1LtV839ktupRboMUSXlXqoJ9lLFvpe-TZtLCf2q6jUpY/viewform) to give a lightning talk.

Lighting Talks:

Matthew Sundquist -- Plotting on the Web: IPython, D3.js, and matplotlib

Agenda:

6:15p - Check-in and mingle, with Pizza and Beer provided by our generous sponsor Yelp!

7:05p - Welcome

7:10p - Talk 1 and Q&A

7:50p - Lightning Talks and Announcements while lightning speakers set up

8:10p - Talk 2 and Q&A

9:00p - More mingling

9:30p - Doors close

Please take note of the important check-in details at Yelp

1. Doors open at 6:15pm to allow enough time for the check-in process. Before 6:15pm, please wait outside without blocking the building entrance. Wait list will be admitted beginning at 6:45pm. Doors close at 7:30pm.

2. Please update the name on your account to reflect your FIRST NAME and LAST NAME. Hy will be greeting downstairs and Yelp Security will be checking IDs upstairs. If your name on Meetup.com is not the name on your ID, then please enter your full name here (https://docs.google.com/forms/d/1d_oPoxjcAQzOJqozHIzVuFNnOYi7CDrzouywq4U9SUo/edit).

3. Since alcohol will be served at the event, we ask that any underage attendees RSVP directly to the meet up host.

4. Waiting list folks will be allowed into the event AFTER we admit all confirmed attendees.

Yelp is generously providing pizza, drinks, and beer in addition to their venue space.