Improving Search Relevance Using CrossEncoders and LLMs & Emergent Facets

We're back with another Search & AI Meetup! The event will start at 6pm sharp and finish at 8pm, kindly hosted by Just Eat who will also provide food and drinks. This will be an in-person event although we hope to record the talks and share the videos later.

Our first talk is from Kapil Kumar of Roku on Improving Search Relevance Using CrossEncoders and LLMs:
Search relevance is a critical component of any content discovery system. While traditional semantic retrieval using bi-encoders offers efficient search capabilities, it often suffers from false positives due to its inability to capture complex query-document relationships. Alternatively, Large Language Models (LLMs) can provide superior relevance understanding, but their computational costs and high latency make them impractical for production search systems.

This talk explores how we achieved such results by fine-tuning a cross-encoder model, combining the best of both worlds: we demonstrate how our approach maintains the efficiency of semantic retrieval while significantly reducing false positives through cross-attention mechanisms. Through model distillation techniques, we achieved remarkable improvements in search quality. Our results show that cross-encoders can effectively bridge the gap between lightweight but imprecise bi-encoders and powerful LLMs.

The second talk comes from Mark Harwood, ex-Elastic and an Apache Committer, on Emergent Facets: Generating Specialized Filters from Search Results:
Faceted search typically relies on predefined, indexed fields — product categories, tags — chosen in advance and exposed selectively at query time. But in exploratory domains, the most useful structure may not exist in the index at all.

This talk introduces the idea of emergent facets: tailored groupings and labels that are generated after the query, directly from the search results themselves. I’ll show two techniques that enable this: both operate client-side, require no index changes, work with multiple databases and adapt to each new dataset and query. I’ll also show how these facets can be turned into Boolean queries using a graphical query builder — bridging unstructured exploration with structured refinement.

The goal is to rethink how structure in search can form: not from what we predefine, but from what we discover.

We'll also have time for Q&A on all things search and AI and general networking. Set yourselves up for the summer at our friendly and informative Search Meetup, organised by The Search Juggler, OpenSearch and Eliatra.

Please provide your full name and email address when registering for the event as we will need a list of attendees for security.