Name: [UCL WI Talk]: Scaling Transformer-based models for sequential recommendation
Start: 2023-11-10T14:00:00Z
End: 2023-11-10T15:00:00Z
Location: 90 High Holborn

Sequential recommendation seeks to forecast the next choice in a user’s interaction history with items. This task shares similarities with language modelling, which also involves learning from sequences. Therefore, the Transformer architecture—widely adopted in the realm of language processing—has achieved the state-of-the-art in sequential recommendation as well. However, training Transformers for recommendation purposes presents unique challenges. Pre-training and fine-tuning, a method highly successful for language models, is less applicable in recommendations because most datasets contain specific sets of items. Additionally, recommender systems often have to deal with millions of items, a scale much larger than the vocabularies of typical language models.

In this talk, I will explore how the Transformer training can be modified for sequential recommendation and present techniques that can help address the training challenges. In particular, I will cover the gSASRec model, which allows for effective and efficient application of negative sampling to avoid unnecessary computations (published at RecSys’23; best paper award), as well as the RecJPQ technique to reduce the size of the item embedding tensor (accepted by WSDM ‘24).

**Bio:**
Aleksandr V. Petrov is a PhD candidate at the University of Glasgow’s Information Retrieval Group, specialising in the usage of Transformer models for recommender systems with large catalogues. His scholarly contributions include multiple papers in premier recommender system venues, such as RecSys, WSDM, and ACM TORS. His work earned him a Best Paper award at RecSys 2023 and a Best Paper nomination at RecSys 2022. Before his PhD, Aleksandr accumulated over ten years of industry experience, with senior roles at large tech companies such as Amazon and Yandex, as well as a co-founder of the recommender systems startup E-Contenta.

Xi Wang

UCL-WI

Hossein A. (Saeed) Rahmani

UCL Web Intelligence Meetup Group

Technology

Artificial Intelligence

Seminars

Big Data

Machine Learning

Data Analytics

Data Visualization

Predictive Analytics

Data Science

Natural Language Processing

Search, Information Retrieval

Researchers

Information Retrieval

[UCL WI Talk]: Scaling Transformer-based models for sequential recommendation

90 High Holborn

Share

UCL Web Intelligence Meetup Group

[UCL WI Talk]: Scaling Transformer-based models for sequential recommendation

UCL Web Intelligence Meetup Group

Details

Members are also interested in