addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

real-time search infrastructure and architecture at craigslist

Handling hundreds of millions of search queries per day against a data set that normally lives in MySQL and is constantly changing and growing presents a number of interesting challenges.  Things get even more fun when you'd like to move from "batch updates" of your search indexes to real-time.  That's what we've done at craigslist in the last year using the open source Sphinx search engine along with MySQL and some custom Perl code to tie it all together.

This presentation will include:
• a brief overview of the sphinx search engine and why we use it
• a snapshot of our previous search architecture(s)
• our new search architecture
• lessons learned and pain points along the way
• new sphinx features that make real-time indexing more flexible
• ideas for future enhancements

Jeremy Zawodny (@jzawodn / blog) hacks on Sphinx, MySQL, Redis, MongoDB, Perl, and various back-end bits of the craigslist infrastructure. He flies both small airplanes and gliders in his spare time.

6:30 - 7:00 -- Doors open/general socializing & food provided by craigslist 
7:00 - 7:15 -- Intro/announcements 
7:15 - 8:15(ish) -- Main presentation 
8:15 - 8:45 -- General discussion/Q&A/networking/etc.

Percona has made MySQL faster and more reliable for over 2,000 consulting and support customers worldwide since 2006. Percona provides enterprise-grade MySQL SupportConsultingTrainingRemote DBA, and Server Development services to companies including Cisco Systems, Alcatel-Lucent, Groupon and the BBC. Percona's founders authored the definitive book “High Performance MySQL” from O'Reilly Press and the widely read MySQL Performance Blog. Percona also develops software for MySQL users, including Percona ServerPercona XtraBackupPercona XtraDB Cluster, and Percona Toolkit. The popular Percona Live conferences draw attendees and acclaimed speakers from around the world. For more information, visit

craigslist is a place to buy, sell, rent, hire, share, meet, swap, discuss, find, serve, connect, give away, announce, work, collect, care, perform, learn, marvel, mentor, befriend, fall in love and/or save the world.

Lithium helps companies unlock the passion of their customers. Lithium software powers amazing Social Customer Experiences for more than 400 iconic brands including AT&T, BT, Best Buy, Indosat, Sephora, and Telstra. Lithium helps companies grow brand advocacy, drive sales, reduce costs and accelerate innovation to create social communities that redefine the customer experience. 

O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, research, and conferences. Since 1978, O'Reilly has been a chronicler and catalyst of leading-edge development, homing in on the technology trends that really matter and galvanizing their adoption by amplifying "faint signals" from the alpha geeks who are creating the future. An active participant in the technology community, the company has a long history of advocacy, meme-making, and evangelism.

Join or login to comment.

  • A former member
    A former member

    The food here is amazing!

    1 · August 26, 2014

  • Mike L.


    August 26, 2014

  • Lee M.

    I just now changed my meetup profile name to exactly match my photo id (Lee Melton). Prior to 5 mins ago my meetup profile name showed the nickname I go by (Buck Melton). Hopefully this is not a problem.

    August 26, 2014

  • Huzefa S.

    I cannot make up for this event. Will there be a recording of this event

    August 26, 2014

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy