Training New Multilingual Foundation Models for Search & Classification

Nils Reimers, who is director of machine learning at cohere.ai, presents online on March 28 on Training New Multilingual Foundation Models for Search & Classification.

Abstract: Training Large Language Models / Transformer Models in a multilingual setting poses several challenges: How should the text be tokenized? How to balance across languages? How to perform data cleaning across 100 languages? How to achieve a strong cross-lingual transfer performance? How to deal with the curse of multilinguality?

In this talk, I will give an overview of how we trained large transformer models specifically for the task of search, content recommendation and cross-lingual text classification. We started with the creation of a large multilingual dataset with over 1.5B training pairs, which needed to be carefully cleaned and augmented, which poses several challenges when this needs to be done across 100+ languages.

Bio: Nils Reimers is an expert on training transformer networks specifically for text understanding tasks. In 2018, he authored and open-sourced the popular sentence-transformers library, which is the most popular framework to design semantic search applications. He authored various research papers and published many state-of-the-art models that significantly advanced the field of semantic search.

In 2021, he joined Hugging Face to start the research group on Neural Search. Recently, he joined cohere.ai as director of machine learning to provide semantic search as a service.

--
By responding here, you acknowledge and consent to our Code of Conduct: We seek to provide a respectful, friendly, professional experience for everyone regardless of gender, sexual orientation, physical appearance, disability, age, race, and religion. We do not tolerate behavior that is harassing or degrading to any individual, in any form. Participants are responsible for knowing and abiding by these standards. We encourage all attendees to assist in creating a welcoming, safe, and respectful experience.