An Effective Label Noise Model for DNN Text Classification

Abstract:
Because large, human-annotated datasets suffer from labeling errors, it is crucial to be able to train deep neural networks in the presence of label noise. While training image classification models with label noise have received much attention, training text classification models have not. In this paper, we propose an approach to training deep networks that is robust to label noise. This approach introduces a non-linear processing layer (noise model) that models the statistics of the label noise into a convolutional neural network (CNN) architecture. The noise model and the CNN weights are learned jointly from noisy training data, which prevents the model from overfitting to erroneous labels. Through extensive experiments on several text classification datasets, we show that this approach enables the CNN to learn better sentence representations and is robust even to extreme label noise. We find that proper initialization and regularization of this noise model is critical. Further, by contrast to results focusing on large batch sizes for mitigating label noise for image classification, we find that altering the batch size does not have much effect on classification performance.

About Ishan Jindal:
Ishan Jindal, PhD received his Doctorate from Wayne State University in Detroit, MI in 2019. He completed his M.Tech degree with distinction from the Indian Institute of Technology Roorkee, India in 2014. He was awarded a DAAD Fellowship for completing his Masters at TU Berlin, Germany. He joined the Indian Institute of Technology Gandhinagar, India as a Junior Research Fellow from 2014 to 2015. His research interests include machine learning and data mining, deep neural networks and signal processing. He now works as an AI Engineer at Interactions Digital Roots.