Skip to content

SEA: Search Engines Amsterdam - July

Photo of Samarth Bhargav
Hosted By
Samarth B. and Maarten de R.
SEA: Search Engines Amsterdam - July

Details

Location: L3.33, Lab 42, Science Park Amsterdam
The Zoom link, in case you want to join online, will be available when you RSVP 1 day prior to the event.

Speaker #1:
Title: PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation
Abstract: Traditional offline evaluation methods for recommender systems struggle to capture the complexity of modern platforms due to sparse behavioural signals, noisy data, and limited modelling of user personality traits. While simulation frameworks can generate synthetic data to address these gaps, existing methods fail to replicate behavioural diversity, limiting their effectiveness. To overcome these challenges, we propose the Personality-driven User Behaviour Simulator (PUB), an LLM-based simulation framework that integrates the Big Five personality traits to model personalised user behaviour. PUB dynamically infers user personality from behavioural logs (e.g., ratings, reviews) and item metadata, then generates synthetic interactions that preserve statistical fidelity to real-world data. Experiments on the Amazon review datasets show that logs generated by PUB closely align with real user behaviour and reveal meaningful associations between personality traits and recommendation outcomes. These results highlight the potential of the personality-driven simulator to advance recommender system evaluation, offering scalable, controllable, high-fidelity alternatives to resource-intensive real-world experiments.
Bio: Chenglong Ma is a Research Fellow at the RMIT University node of the ARC Centre of Excellence for Automated Decision-Making & Society (ADM+S). He received his Ph.D. in Computer Science from RMIT University in 2024, where his dissertation focused on the role of user conformity in recommender systems. His research interests include recommender systems, user modeling, social influence, information retrieval, computer vision, and generative simulations. His work combines psychological and social theories with large language model-based multi-agent systems to evaluate recommender performance, fairness, and influence. He has presented his work at top-tier venues such as TheWebConf, SIGIR and CIKM.

Speaker #2: Nuha Abu Onq
Title: Classifying Term Variants in Query Formulation
Abstract: Formulating queries is a challenging stage of the search process. This study investigates how crowd workers formulate an initial query for a common information need described in a backstory, resulting in diverse query variations. Using the UQV100 dataset of information need backstories and corresponding queries, we analyze the variations. Our findings show that 70% of the query terms used in crowd worker queries did not appear in the backstory text. Examining such terms we developed a taxonomy of search strategies, with the most common being semantic variations of backstory terms, followed by information type specifications. Additionally, we categorized the backstories by cognitive complexity, showing that higher complexity led to greater diversity in query variations and a wider range of term variant categories. This study highlights the importance of accounting for query variations, term variants, user strategies, and cognitive complexity in designing search systems and test collections to better align with users' information needs, influenced by the cognitive demands of a task, and enhance system performance and usability.
Bio: I am a PhD candidate conducting research in the field of information retrieval, with a particular focus on how human and contextual factors shape the way users formulate search queries. Although people may seek the same information, they often express their queries in remarkably different ways, and even slight wording variations can produce significantly different search results. This highlights the importance of understanding the factors and underlying reasons behind these differences. My research examines how individual characteristics, the components of a query, and the task-related factors influence query formulation. By uncovering what contributes to query variation, this work aims to improve both fairness and relevance in information retrieval systems. The ultimate goal is to support the design of more effective and personalised search tools that address diverse user needs and behaviours.
This research provides insights to guide the development of future query variant test collections. Drawing on the data gathered about how individuals formulate queries, researchers can create more realistic and nuanced persona profiles for generating queries, whether using crowdworkers or large language models (LLMs). These enriched personas can help ensure that test collections better capture the diversity of real-world search behaviours, ultimately enhancing the accuracy, fairness, and relevance of search systems. In addition, the study explores the practical implications for search technologies, including improvements to query suggestions, auto-completion, and query handling, contributing to better user experiences and satisfaction with search systems.

Counter: SEA Talks #287 and #288.

Photo of SEA: Search Engines Amsterdam group
SEA: Search Engines Amsterdam
See more events

Every last Friday of the month

Online event
This event has passed