NLP IL @Booking.com

18:00- 18:30- Gathering, food, and drinks
18:30- 18:45 - Booking introduction
18:45-19:15 - Text2topic- Leverage reviews data for topics multi label classification- Moran Beladev & Elina Frayerman
19:15 - 19:45 Text classification in production- key takeaways (engineering wise)- Ofri Kleinfeld & Gil Amsalem
19:45 - 20:00 Break
20:00 - 20:30 Testing NLP data and models - Nir Hutnik

Abstracts:
**Text2topic- Leverage reviews data for multi label topics classification in Booking.com**
Abstract:
Having millions of customers' reviews, we would like to better understand them and leverage this data for different use cases. For example, finding popular activities per destination, detecting popular facilities per property, allowing the users to filter reviews by specific topics, detecting violence in reviews and summarizing most discussed topics per property.
In this talk, we will present how we build a multilingual multi-label topic classification model that supports zero-shot, to match reviews with unseen users’ search topics.
We will show how fine-tuning BERT-like models on the tourism domain with a small dataset can outperform other pre-trained models and will share experiments results of different architectures.
Furthermore, we will present how we collected the data using an active learning approach and AWS Sagemaker ground truth tool, and we will show a short demo of the model with explainability using Streamlit.

—-----------------------------------------------------------------------------------------------
Text classification in production- key takeaways

Abstract:
For the last two years, the Content Intelligence team has been generating a variety of enrichments for content uploaded to Booking.com by our partners and guests. The platform hosts ML models that generate image tags, text tags, image captions, and more.
While deploying and maintaining machine learning models in production is a challenging task on its own, deploying deep learning models can be even more challenging. That's because deep learning models are often computationally intensive compared to classic machine learning models and usually require dedicated hardware to run (GPU).
In this session, we will share how we optimized the inference of our text classification model in production. We will start by sharing how changing only a few lines of code can improve the model's throughput and latency. Then, we will discuss different optimization methods for the service calling the model depending on the infrastructure in use (CPU vs. GPU).
Finally, we will share how these methods helped us cut the costs of running our text classification model on AWS Sagemaker by half, while keeping the same throughput.

—-----------------------------------------------------------------------------------------------
Validating NLP data and models - Nir Hutnik

Abstract:
NLP data, and unstructured data in general, is very hard to validate. Validating NLP data is a real challenge, as actions such as statistical analysis and segmentation, which are pretty straightforward on structured data, are not so easy to undertake. In this talk, we will look at common issues in NLP data and models, such as data and prediction drift, sample outliers and error analysis, discuss the ways they can impact our model performance, and show how we can detect these issues using the deepchecks open source testing package.