• Virtual RecSysNL on News and Media

    Online event

    [UPDATE: new schedule and zoom link]

    Dear RecSysNL members, we're happy to announce our second (and last?) virtual meetup! On Tuesday June 8th, we will host a meetup focused on news and media, both in terms of evaluation and production implementation. As usual we have one academic guest speaker, and one speaker from industry....details below. Hope to see you there!

    ------------ SCHEDULE ------------
    19:00-19:05 Meetup starts
    19:05-19:20 Community section: 🇳🇱 ACM RecSys 2021 Announcements
    19:20-19:50 Talk 1: News recommenders with a (democratic) mission - Sanne Vrijenhoek (UvA)
    19:50-20:20 Talk 2: Serverless Data Processing - Aron Hammond (DPG Media)

    ------------ ABSTRACTS ------------

    Abstract: News recommenders with a (democratic) mission - Sanne Vrijenhoek, Project Researcher, University of Amsterdam
    News recommenders have the potential to help users filter the enormous amount of news that is available online, and as such may play an important role in determining what information users do and do not get to see. However, current approaches to evaluating recommender systems are often focused on measuring an increase in user clicks and short-term engagement, rather than measuring the user's and society’s longer term interest in diverse and important recommendations. In this talk we aim to bridge the gap between so-called normative notions of news diversity, as it is known in social sciences and specifically democratic theory, and quantitative metrics necessary for evaluating the recommender system. We discuss a number of democratic missions a recommender system could have, together with a set of evaluation metrics stemming from these missions, and suggest ways for practical implementations of these metrics.

    Abstract: Serverless Data Processing at DPG Media - Aron Hammond - Machine Learning Engineer
    The talk will be about practical considerations that our team has had to make in order to bring a recommender system into production. I’ll cover the “default” tools with which we started (Batch processing in Spark) and follow that up with more recent tools like AWS Lambda and Spark Streaming.

  • Virtual RecSysNL on Bias & on Blendle

    Online event

    We have been holding back on new meetups in the hope we could see you all again in real life, but no longer! On Tuesday November 26th, we will host our first ever virtual RecSysNL meetup. We have two exciting speakers and look forward to checking in with you all!

    - Talk 1: Toward Natural Language Mitigation Strategies for Cognitive Biases in Recommender Systems - Alisa Rieger, PhD Student, TU Delft
    - Talk 2: Blendle: Diverse content recommendation from a vast collection - Jasper Oosterman - Data Scientist

    Second abstract and link to be announced soon.

    Abstract: Toward Natural Language Mitigation Strategies for Cognitive Biases in Recommender Systems - Alisa Rieger, PhD Student, TU Delft
    Decision-making at individual, business, and societal levels is influenced by online news and social media. Filtering and ranking algorithms such as recommender systems are used to support these decisions. Further, individual cognitive selection strategies and homogeneous networks can amplify bias in customized recommendations, and influence which information we are exposed to. A first step in the direction of bias mitigation would consequently be to raise users' awareness of filtering mechanisms and potential cognitive biases, e.g., through explanations and interactive interfaces.
    We review studies in the field of recommender systems focused on interface-based mitigation of various cognitive biases. Analysis and comparison of previous studies provides an overview of effective implementation and methodological evaluation approaches for these biases.

  • 15th Recommender Systems Netherlands meetup @ Bol.com

    Papendorpseweg 100

    We look forward to welcoming you to the 15th RecSysNL meetup, hosted by bol.com in Utrecht. We'll have our usual program with one talk from academia by Sandy Manolios (TU Delft), and one from industry, by Barrie Kersbergen (bol.com).

    17:30 - 18:30 Registration + Pasta Buffet
    18:30 - 19:00 Intro
    19:00 - 20:00 Technical Talks
    20:00 - 21:00 Networking and Drinks

    Talk 1:
    "The Influence of Personal Values on Music Taste: Can Personal Values be used to improve Recommendations ?"

    The field of psychology can bring a lot to the field of recommender systems by adding some understanding of what drives human preference. In particular, the psychology of music highlighted the potential role of demographics, personality, social relationship and past experiences in musical preference. However, very few works investigated the potential role of personal values (what is most important for people in life) in musical preferences and its possible benefits for music recommendations. This talk will introduce a first qualitative experiment that exposes the link between those personal values and music taste.

    Sandy Manolios is a Ph.D. student at TU Delft under the joint supervision of Dr. Cynthia C. S. Liem, Prof. Dr. Alan Hanjalic and Prof. Dr. Catholijn Jonker. Her main research focuses on personal values-aware music recommendations. She holds an MSc. and a BA in cognitive science and aims to bridge psychology and computer science together.

    Talk 2:
    "The journey of building the Recommender systems at bol.com"

    In this presentation, I will discuss the item-to-item recommender system at bol.com. How we started in 2010 and how this is maturing into a system that is now doing 2000 predictions per second on a catalog of over 70mio items. What does our architecture look like, what are the practical challenges that we faced, how we solve them and what we working on right now.
    Keywords: PySpark, item-to-item, learning-to-rank, Tensorflow

    Barrie Kersbergen is a senior data scientist at bol.com since 2010. He worked on the different personalization and recommender systems at bol.com.

    Looking forward to seeing you all there!

  • 14th Recommender Systems Netherlands meetup @ RTL

    Barend en Van Dorpweg 2

    After a long break, RecSys is back!

    We look forward to welcoming you to the 14th RecSysNL meetup, hosted by RTL in Hilversum. We'll have our usual program with one talk from academia by Jay Kim (TU Delft), and one from industry, by Anca Dumitrache (FD Mediagroep). In addition, Marijn Koolen will give us a quick flashback and summary of last October's RecSys2019 conference.

    Doors open at 18:00, talks start at 18:30.
    Drinks and Pizza will be provided.

    Talk 1:
    "Towards Seed-Free Music Playlist Generation: Enhancing Collaborative Filtering with Playlist Title Information" by Jay Kim (TU Delft).

    In this presentation, Jay proposes a hybrid Neural Collaborative Filtering (NCF) model trained with a multi-objective function to achieve a music playlist generation system. The proposed approach focuses particularly on the cold-start problem (playlists with no seed tracks) and uses a text encoder employing a Recurrent Neural Network (RNN) to exploit textual information given by the playlist title. To accelerate the training, we first apply Weighted Regularized Matrix Factorization (WRMF) as the basic recommendation model to pre-learn latent factors of playlists and tracks. These factors then feed into the proposed multi-objective optimization that also involves embeddings of playlist titles. The experimental study indicates that the proposed approach can effectively suggest suitable music tracks for a given playlist title, compensating poor original recommendation results made on empty playlists by the WRMF model.

    Jaehun(Jay) Kim currently is seeking his Ph.D. under the supervision of Dr. Cynthia C. S. Liem and Prof. Dr. Alan Hanjalic at TU Delft. The main research interest of him is the interpretability of the complex recommender systems. Specifically, highly non-linear systems such as deep learning-based models are the main challenge for his projects. He is an almost-retired freelancer music writer, which fuels and motivates his research more oriented mostly on the music recommendation. He received his MSc in 2015 from Seoul National University and BA in 2013.

    Talk 2:
    Intermezzo by Marijn Koolen (Royal Netherlands Academy of Arts and Sciences), who gives a summary of the ACM RecSys 2019 conference that was held in September in Copenhagen, Denmark.

    Talk 3:
    "Experiments with Recommending Financial News" by Anca Dumitrache from FD Mediagroep.

    This talk will discuss a series of experiments to build a recommender system for news articles in the financial domain. Recommending news means a continuous cold-start problem, that can be tackled by issuing content-based recommendations. Other interesting challenges for this use case are learning from implicit feedback, and handling a large number of user interactions. The proposed approach uses Gradient Boosted Decision Trees (GBDT) to learn from a diverse set of features related to the article content, aggregated user reading behavior, and composite user-article features (e.g. the set overlap between the article's tags and the user's most read tags). The talk will discuss a selection of experiments used to select what model to use, what is the trade-off between data volume and recency, and feature ablation for the model. Finally, a description will be given of how the data pipeline was implemented to bring this model to production.

    Anca Dumitrache is a Senior Data Scientist at FD Mediagroep, working on a Google DNI project on personalized news feeds. She graduated from her PhD at Vrije Universiteit Amsterdam, where she studied how to capture and interpret inter-annotator disagreement in crowdsourcing, and how to use this disagreement to get better training data for natural language processing models.

    Looking forward to seeing you all there!

  • 13th Recommender Systems Netherlands (RecSysNL) meetup

    Prins Bernhardplein 150

    We look forward to welcoming you to the 13th RecSysNL meetup. Hosted by Bookarang, the meetup is held at the Volkshotel, Wibautstraat 150, Amsterdam with two interesting talks, one from industry and one from academia.

    Doors open at 18:00h, talks start at 18:30h.
    Drinks and snacks/pizza will be provided.

    - 1st talk by Niels Bogaards and Isaac Sijaranamual from Bookarang, "Content based recommendations for books"

    Choosing a book is a difficult task: books take a significant amount of time to read, come in vast numbers and differ on very subtle qualities. Book taste is highly personal: for the same book there can be as many readers that love as that hate it. By modelling the content of books and comparing them on aspects that matter to readers, Bookarang can provide recommendations that are explainable, personalised and tuned to a shop, library or user's preferences.

    Niels Bogaards is an expert in the field of Artificial Intelligence and worked at the world-famous IRCAM in Paris.

    Isaac Sijaranamual studied Informatics and worked as a scientific programmer. He is an expert in the field of Machine Learning, Natural Language Processing and Information Retrieval.


    - 2nd talk by Marijn Koolen from Royal Netherlands Academy of Arts and Sciences, Netherlands, "Narrative-driven Recommendation for Casual-Leisure Needs"

    Recommender systems typically generate recommendations for a user based on their profile, or for an item given its user interactions, but there are many scenarios especially in leisure domains such as books, movies, games and music, where users have specific recommendation needs, where they want to steer the recommendation process towards certain aspects they find relevant. Currently, there are few recommender or search systems that can deal with the complexity of such directed needs, nor do we know well which data types (metadata, user ratings and reviews, item content) are useful to match against different aspects of recommendation needs. There are many discussion forums where users describe their needs and their frustration with current search and recommender systems. In this talk I will summarize our work on analyzing relevance aspects for these needs and describe experiments on dealing with these.

    Marijn Koolen is researcher and developer at the Royal Academy of Arts and Sciences, and works on bestseller prediction, reading impact analysis and transparent recommender systems. He has a background in information retrieval and web search and got his PhD at the University of Amsterdam for work on hyperlink structure in information retrieval.

  • 12th Recommender Systems Netherlands (RecSysNL) meetup

    We look forward to welcoming you to the 12th RecSysNL meetup. Hosted by Elsevier, Amsterdam with two interesting talks, one from industry and one from academia.

    Doors open at 18:00h, talks start at 18:30h.
    Drinks and pizza will be provided.

    "Mixed-initiative Recommender Systems: Towards a Next Generation of Recommender Systems through User Involvement"
    Katrien Verbert, KU Leuven

    Researchers have become more aware of the fact that effectiveness of recommender systems goes beyond recommendation accuracy. Thus, research on these human factors has gained increased interest, for instance by combining interactive visualization techniques with recommendation techniques to support transparency and controllability of the recommendation process. I will present work on interactive visualizations to enable end-users to interact with recommender systems. The objectives are: 1) to explain the rationale of recommendations as a basis to increase user trust and acceptance of recommendations, and 2) to incorporate user feedback and input into the recommendation process and to help steer it. In addition, I will present several user studies that investigate how such user controllability interacts with personal characteristics such as expertise and visual working memory.

    Bio: Katrien Verbert is an Associate Professor at the HCI research group of KU Leuven. She obtained a doctoral degree in Computer Science in 2008 at KU Leuven, Belgium. She was a post-doctoral researcher of the Research Foundation – Flanders (FWO) at KU Leuven. She was an Assistant Professor at TU Eindhoven, the Netherlands and Vrije Universiteit Brussel, Belgium. Her research interests include visualisation techniques, recommender systems, visual analytics, and digital humanities. She has been involved in European projects on these topics, including the EU ROLE, STELLAR, STELA, ABLE, LALA and BigDataGrapes projects. She is also involved in the organisation of conferences and workshops.

    "Using heterogenous data to recommend scientific articles and funding opportunities"
    Finne Boonen and Minh Le, Elsevier

    How do you know if a research article is relevant to you without reading it? What makes a grant a good match for your research needs? Oftentimes, the answer depends on a variety of factors: the objective quality of the paper, the match between a researcher’s interest and the topic of the paper, the career stage of a researcher compared to what is expected by a grant, current trends of the field and many other things. Elsevier, as a global information analytics company, drives solutions that approach such problems for the scientific community using various big data sources and technologies including machine learning. We, for instance, combine click logs, reading history, full-text, and citations using a mixture of recommender system techniques, including learning-to-rank, graph-based keyword extraction, and random walk. In this presentation, we will walk through the techniques we have used and their impact in improving researchers’ experience.


    Finne has been a Data Scientist at Elsevier for the last three years and currently is working on the recommenders team. Prior to Elsevier, she held a variety of roles in different companies. Finne Boonen has an M.Sc. in ICT in Business from Leiden University and a bachelor’s in computer science from the Vrije Universiteit Brussel. Finne is interested in mastering the end-to-end development of data products.

    Minh Le is a Data Scientist in the Recommenders team at Elsevier. He did a master in Cognitive Sciences and, next to his job at Elsevier, is finishing his PhD at the Vrije Universiteit Amsterdam specializing in Natural Language Processing. He is interested in applied research and general artificial intelligence.

  • 11th Recommender systems Netherlands (RecSysNL) meetup

    Bedrijvencentrum Rudolf Magnus

    We look forward to welcoming you to the 11th RecSys Amsterdam meetup. Hosted by SPARQUE, Utrecht with two interesting talks from industry and academia.

    Doors open at 18:00h, talks start at 18:30h.
    Drinks and pizza will be provided.

    1. Chi Shing Chang, SPARQUE, "Managing evolving and sprawling recommendations models"

    What are good recommendations? Data, domain experts and users may not necessarily be in agreement. Recommendations may best be based on methods that integrate the important aspects of all three sources. Warmteservice, a wholesaler of installation materials with physical branches and a webshop, needed high quality recommendations for complementary products. By presenting these recommendations, customers would not forget to order essential materials while Warmteservice sells more items. SPARQUE developed a model for Warmteservice that combines aspects from existing data, from experts and user behavior. I’ll talk about how the recommendations evolved and the tooling we used to manage the model and to give Warmteservice a performant, interactive and personalized recommendation engine.

    Chi’s Bio: Chi Shing Chang is an ICT-entrepreneur. Co-founder of SPARQUE; a technology scale-up specialized in personalized E-commerce. Chi Shing is a graduate of the IMD MBA program and the Industrial Engineering program from University of Twente. He worked for several employers: Deli Maatschappij, Shell, Orange and McKinsey, before taking the leap to start a company. A generalist and an omnivore, he takes interest in a broad range of topics, including business, economics and human and organizational psychology.

    2. Martijn Willemsen, Eindhoven University of Technology (TU/e), "Recommender systems to help people move forward"

    Many real-life recommender systems are evaluated mostly on (implicit) behavioral data such as clicks streams and viewing times. However, such an approach has limitations and I will show how a user-centric approach can help better understand why users are satisfied or not, for example why users prefer diversification over prediction accuracy as it reduces choice difficulty. The behaviorist approach to evaluation also misses that users’ short term goals (i.e. their current behavior) might not be representative of the goals they want to attain (i.e. their desired behavior). What we like today might not be what we like tomorrow, because our tastes evolve or because our prior preferences and past behavior might not represent our current needs and goals. This is especially relevant in health and sustainability domains where people are in need of support while changing their current behavior. I will elaborate on an example from the energy recommendation domain, and show how a different type of recommender approach and interface might help users to save more energy. I will also discuss our recent work using Spotify to help people explore new tastes for unknown genres.

    Martijn's bio:
    Dr. Martijn Willemsen is an expert on human decision making in interactive systems. He is working as an associate professor in the Human-Technology Interaction group of Eindhoven University of Technology (The Netherlands) and is the principle investigator of the recommender LAB at JADS Den Bosch (www.jads.nl). His primary interests lie in the understanding of cognitive processes of decision making by means of process tracing and in the application of decision making theory in interactive systems such as recommender systems. He is also an expert on user-centric evaluation of adaptive systems. He is part of the core team of the Customer Journey Research Program in the Data Science Center Eindhoven (DSC/e) and is teaching in the joint BSc and MSc data science programs of the Jheronimus Academy of Data Science (jads.nl). www.martijnwillemsen.nl

  • 10th Recommender systems Amsterdam meetup

    Persgroep Nederland

    We look forward to welcoming you to the 10th RecSys Amsterdam meetup. Hosted by Persgroep with two interesting talks from industry and academia.

    Doors open at 18:00h, talks start at 18:30h.
    Drinks and pizza will be provided.

    1. Daan Odijk, RTL, "Searching to be Entertained"

    As the largest commercial broadcaster in a declining Dutch TV market, RTL is making a transition from a traditional TV company to a consumer-focused media company. RTL is embracing a closer relationship and more direct interaction with its viewers, followers and visitors. In this talk, I will share how we are using AI, recommender systems and search technology to help our users find the right content for them, ranging from the 1M daily visitors on our news website to the over 2B video plays we had in 2017, most of these on our rapidly growing video-on-demand platform Videoland.

    Bio: Daan Odijk is the lead data scientist at RTL. In 2016, he obtained his PhD from the University of Amsterdam, researching search algorithms for news. Subsequently, he joined journalism start-up Blendle to lead the personalization team. At RTL since 2018, Daan leads a team of a dozen data scientists and engineers, delivering data-powered products across RTL, including personalization for RTL Nieuws and Videoland.

    2. Julián Urbano, TU Delft. "The measure dilemma: which dataset-based measures are better to predict end-user satisfaction?"

    The main goal of an evaluation experiment is to determine which systems perform well and which systems perform poorly on a task like retrieval or recommendation. However, there has been little systematic analysis regarding how well these evaluation results predict end-user satisfaction. For many researchers, reaching statistical significance is usually the objective, but not enough attention is paid to the real implications of the observed improvements. In this talk I'll present empirical results on the correlation between dataset-based results and user satisfaction in the task of music retrieval for recommendation. In particular, we'll discuss the effect that different annotation scales and measure formulations have on this correlation, and the implications for researchers, developers or reviewers.

    Bio: Julián Urbano is an Assistant Professor at Delft University of Technology. His research is primarily concerned with evaluation in IR, working in both the music and text domains. Current topics of interest are the application of statistical methods for the construction of datasets, the reliability of evaluation experiments, statistical significance testing for IR, low-cost evaluation and stochastic simulation for evaluation. He has published over 50 research papers in related venues like Foundations and Trends in IR, the IR Journal, the Journal of Multimedia IR, SIGIR, ISMIR, CIKM, ICTIR and ECIR, winning two best paper awards and a best reviewer award.

  • 9th Recommender Systems Amsterdam meetup

    Booking.com HQ, Herengracht 597

    We look forward to welcoming you to another RecSys Amsterdam meetup! Kindly hosted by Booking.com, we'll have three speakers in a mix from industry & academia. Doors open at 18:00, talks start at 18:30!
    As usual, drinks and snacks will be provided.


    1. Christophe van Gysel (UvA) will talk about work he's done at Bing: "Reply With: Proactive Recommendation of Email Attachments" (see: https://arxiv.org/abs/1710.06061)

    2. Dung Chu (FD Mediagroep): "Recommendation systems at the FD"
    Recommendation system is a popular data-driven tool used in many different business sectors. Within het FD we use an in-house recommendation engine in personalized email campaigns. In this talk, I will share our experience in: why we decided to develop our own recommendation engine; challenges that we encountered from business and technological points of view; first A/B testing results; and our future plan.

    3. Themis Mavridis (Booking.com): "Learning To Match"
    Booking.com is a virtual two-sided marketplace where guests and accommodation providers are the two distinct stakeholders. They meet to satisfy their respective and different goals. Guests want to be able to choose accommodations from a huge and diverse inventory, fast and reliably within their requirements and constraints. Accommodation providers desire to reach a reliable and large market that maximises their revenue. Finding the best accommodation for the guests, a problem typically addressed by the recommender systems community, and finding the best audience for the accommodation providers, are key pieces of a good platform. This work describes how Booking.com extends such approach, enabling the guests themselves to find the best accommodation by helping them to discover their needs and restrictions, what the market can actually offer, reinforcing good decisions, discouraging bad ones, etc. turning the platform into a decision process advisor, as opposed to a decision maker. Booking.com implements this idea with hundreds of Machine Learned Models, all of them validated through rigorous Randomised Controlled Experiments. We further elaborate on model types, techniques, methodological issues and challenges that we have faced. (see: https://export.arxiv.org/pdf/1802.03102)

  • 8th Recommender Systems Amsterdam meetup

    FD Mediagroep

    We look forward to welcoming you to the 8th RecSys Amsterdam meetup. Hosted by FD Mediagroep, this media-themed meetup features four speakers from industry.

    Doors open at 18:00h, talks start at 18:30h.
    Drinks and snacks will be provided.

    1. Ghida Ibrahim (Senior Data Scientist, (formerly at) Liberty Global (http://www.libertyglobal.com/)): "Recommender systems for video and TV products"

    Ghida will give a talk about building recommender systems for video and TV products. Ghida's talk is inspired by her past experience working as a data scientist for Liberty Global. She will explain how recommender systems can be built using users viewership data and many variations of collaborative filtering, and how these can be assessed using A/B testing.

    2. Bouke Huurnink and Roman Ivanov (XITE (https://www.xite.tv/)): "Music Video Recommendation@XITE"

    Remember watching television and waiting for hours for that one music video to come on? Those days are long gone. With advanced music recommendation, smart TVs and ubiquitous screens, music television is transforming into an interactive, on-demand service. Giving you the music video you want, when and where you want it.

    In this talk we discuss the evolution of music video recommendation at XITE. We will cover challenges that we have faced, our first recommender setup, and lessons learned/plans for the future.

    3. Robbert van der Pluijm (Head of Bibblio Labs, Bibblio (http://www.bibblio.org/)): "Scaling a recommendation service - a threefold story"

    Robbert works for Bibblio, which offers a content recommendation service for publishers. In this talk he'll share how they 1) built a local popularity recommender, 2) solved the challenges of scale and catalogue updating (for now) and 3) started to venture into deep learning.

    PS: Take the entrance of the FD Building on the Wibautstraat side (https://www.google.nl/maps/@52.3488042,4.917869,3a,75y,238.4h,95.88t/data=!3m6!1e1!3m4!1sIM7stUICPvldtSkhendu6w!2e0!7i13312!8i6656?hl=en) (There's another entrance at the Prins Bernhardplein, but it's closed after 6pm...).