Streaming similarity search over one billion tweets

Come for Table discussions, Member Self-Intro, What's New, Application Showcase, and Advanced Application Development Techniques! Exchange ideas, meet experts, share code... all HPC & GPU, all practical, all cutting-edge.

General Discussion:  6:15-6:50pm: What’s new and first-time attendee intros 

Main Program: 

7:00-7:50pm: Streaming similarity search over one billion tweets (Dr. Narayanan Sundaram, Intel Research)


Abstract: 

In recent years, adding support to databases to identify similar objects or finding nearest neighbors has become an important operation on databases, with applications to text search, multimedia indexing, and many other areas. One popular algorithm for similarity search, especially for high dimensional data (where spatial indexes like kd-trees do not perform well) is Locality Sensitive Hashing (LSH), an approximation algorithm for finding similar objects.

We show that on a workload where we perform similarity search on a dataset of > 1 Billion tweets, with hundreds of millions of new tweets per day, we can achieve query times of 1–2.5 ms. We show that this is an order of magnitude faster than existing indexing schemes, such as inverted indexes. To the best of our knowledge, this is the fastest implementation of LSH, with table construction times up to 3.7× faster and query times that are 8.3× faster than a basic implementation.  


Location:

Open space; 
Carnegie Mellon Silicon Valley; 
NASA Research Park Bldg 23; 
Mountain View, CA 94043; 

Directions to Carnegie Mellon Silicon Valley; 

Google Map showing parking, check point, and building entrance; 

NOTE: You will need a government issued ID (e.g. Driver's License) to enter NASA Research Park


Join or login to comment.

Our Sponsors

  • Baidu Research

    Location and Refreshments, 9/29/14

  • Cirrascale

    Sponsor of 8/11/14 Meetup

  • Acceleware

    Sponsor of 4/21 & 7/14 Meetup

  • NVIDIA

    Ongoing; Sponsor of 5/19/14 and 6/16/14 Meetups

  • AMAX

    Sponsor of 2/24/14 Meetup

  • Carnegie Mellon - Silicon Valley

    Host of the Meetup location

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Rafaël

We just grab a coffee and speak French. Some people have been coming every week for months... it creates a kind of warmth to the group.

Rafaël, started French Conversation Group

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy