Child-friendly Search and Result Lists Refinement and Quality

This Friday, we'll have two talks followed by drinks.

Thijs Westerveld (WiseNoze) -- Child-friendly access to age-specific content
WizeNoze is an Amsterdam based start-up that develops technology to make the Internet a more suitable place for children. The amount of online information that is targeted at children is currently small and the content that does exist is hard to find. In the main web search engines, this information gets overpowered by the plethora of information that is available for adults. Moreover, the information that is aimed at children often targets them as one homogeneous group, failing to differentiate between children of different ages and comprehension levels.
WizeNoze provides a child-friendly technology platform that increases the amount of content available to children, that improves the access to this information, and that targets each child at their own comprehension level.
In this talk we focus on our child-friendly search engine. We demonstrate the engine and discuss the underlying classifier that given a text, determines the comprehension level required to understand it. This classifier needs to work with a highly heterogeneous set of training data with a mix of fine-grained and coarser labels that sometimes cover overlapping comprehension level ranges.

Thijs Westerveld is an Information retrieval specialist with an interest in both scientific and practical work. As Chief Science Officer at WizeNoze, he is responsible for identifying, inventing and assessing algorithms to solve key technical questions to give customers access to the latest state of the art in kids technology. Thijs holds a PhD in Computer Science from the University of Twente and he has over 15 years experience in various areas of information retrieval, both in academic and industry settings.

Jiyin He (CWI) -- Untangling Result List Refinement and Ranking Quality: a Framework for Evaluation and Prediction
Traditional batch evaluation metrics assume that user interaction with search results is limited to scanning down a ranked list. However, modern search interfaces come with additional elements supporting result list refinement (RLR) through facets and filters, making user search behavior increasingly dynamic. We develop an evaluation framework that takes a step beyond the interaction assumption of traditional evaluation metrics and allows for batch evaluation of systems with and without RLR elements. In our framework we model user interaction as switching between different sublists. This provides a measure of user effort based on the joint effect of user interaction with RLR elements and result quality.
We validate our framework by conducting a user study and comparing model predictions with real user performance. Our model predictions show significant positive correlation with real user effort. Further, in contrast to traditional evaluation metrics, the predictions using our framework, of when users stand to benefit from RLR elements, reflect findings from our user study.
Finally, we use the framework to investigate under what conditions systems with and without RLR elements are likely to be effective. We simulate varying conditions concerning ranking quality, users, task and interface properties demonstrating a cost-effective way to study whole system performance.

Jiyin He received her PhD from the University of Amsterdam (2011) in Information Retrieval. Since then she has spent time at CWI, UvA, FXPal, and currently she works as a researcher at CWI and a visiting researcher at University College London. Her research interest is interactive information retrieval, including retrieval models, users, and interaction between users and search systems. In 2014 she has been awarded a VENI grant from NWO to carry out research in search evaluation focused on capturing variability of user behaviour with respect to diverse types of advanced search interfaces that are well beyond the traditional "10 blue links".