NOTE: the venue does not have drinks for sale. If you want drinks, you can buy them across the street/nearby and bring them to the venue.
Talk 1: Data augmentation instead of explicit regularization
Speaker: Alex Hernández-García (40 min + 10 min Q&A)
Abstract: Explicit regularization techniques, such as weight decay and dropout, are the standard and most popular ways of improving the generalization of CNNs. However, these techniques blindly reduce the effective capacity of the model and, importantly, have very sensitive hyper-parameters that require specific fine-tuning. Furthermore, they are used, unquestioned, in combination with other techniques from the "machine learning toolbox", such as SGD, normalization, convolutional layers or data augmentation, which also provide implicit regularization. Little is known about the interactions among these techniques. In this talk, I will present the results of systematically contrasting data augmentation and explicit regularization on different architectures and object recognition data sets. Data augmentation, unlike explicit regularization, does not reduce the capacity of the model and does not require fine-tuning of hyper-parameters. Besides, we have recently shown that models trained with heavier data augmentation learn more similar representations to those measured in the human visual cortex. In sum, I will show how replacing weight decay and dropout by data augmentation can safely free us from the hassle of fine-tuning sensitive hyper-parameters, potentially achieve better performance and learn more biologically plausible representations.
Bio: Alex Hernández-García is a last-year PhD candidate at the Institute of Cognitive Science of the University of Osnabrück. After completing his M.Sc. at the University Carlos III of Madrid, Spain, he moved in 2016 to Berlin to start a PhD on biologically-inspired machine learning, with a Marie Sklodowska-Curie ITN grant. Although his main background is on machine learning and computer vision, he has an interdisciplinary profile and interests in other fields such as computational neuroscience as reflected by his internships at the Spinoza Centre for Neuroimaging in Amsterdam and the Cognition and Brain Sciences Unit of the University of Cambridge. His paper "Further advantages of data augmentation on convolutional neural networks" recently won the Best Paper Award at the International Conference on Artificial Neural Networks, ICANN.
