Elastic Berlin Meetup @ Zalando: September Edition

Join us for a meetup on September 14th at 18.30 at Zalando, Berlin!

18:30: Join us for a drink

Please have your full and real name in your profile description because the security team will check if you're registered when you arrive.

18:45: Rankquest: Benchmarking Search API Ranking with Elasticsearch (Jilles van Gurp)

Search Ranking is something that many companies that use Elasticsearch struggle with.

Something we noticed while helping various clients is that many companies never evolve to having a systematic approach for testing their search ranking quality. It's too abstract for them; they don't know where to start with this, and they don't really get how this should be done or even why this is important.

In this presentation we present and unveil our new ranking tool, Rankquest Studio, which aims to address some of these issues.
Rankquest emerged out of our frustration with existing tools and approaches in this space and we'll reflect a bit on the requirements we have for this before diving into a demo. Rankquest Studio, is open source, web-based, easy to use, and it can be used to to build out test benchmarks for evaulating
your search solutions.

19:15: Generative Black-Box Testing for Evaluating Search Quality (Oliver Trosien @ Zalando)

We present a novel generative Black-Box Testing approach that uses semantically equivalent queries (e.g. “rote Kleider”, “Kleid rot”) for introspecting the quality of a search engine. At Zalando, we developed a tool for finding search quality problems at scale with the help of mass-generating semantically equivalent query variants. This is a novel way to find relevance problems that complements other approaches that use customer metrics or ground truth data. The tool allows easy extension with new test scenarios and languages by non-technical native language speakers. We will show how it was used to continuously monitor search relevance, to find regressions, and how you can implement such a tool yourself. Here are some questions that we’ll discuss in the session:

• How can you track search quality at scale?
• What kind of quality issues does generative black-box testing (not) uncover?
• Can it be used to test semantic or vector search quality?

20.00: Pizza

Special thanks to our hosts, Zalando!