Big thanks to SAP for hosting us tonight!
Talk 1: Continual Learning in Language and Vision
Speaker: Claudio Greco
Abstract: Language and vision are two fundamental modalities through which humans acquire knowledge about the world. Recent work aimed to bridge computer vision and natural language processing through language learning tasks based on natural images. Neural networks have lately brought steep advances to visually-grounded conversational agents. However, they are hugely affected by catastrophic forgetting, which means that they are incapable of learning new tasks without forgetting the previously-learned ones. This is in stark contrast to how humans learn. Indeed, they build on previous experience, incrementally refine their skills during their lifetime, and typically learn from easier contexts to more complex ones. The capability of machine learning models to continuously learn new tasks without forgetting is called continual learning. In this talk, I will provide some background on language and vision tasks and models and on continual learning. Then, I will present our ACL paper on continual learning in visual question answering, which involves to answer natural language questions about images. Results show that dramatic forgetting is at play and that task difficulty and order matter. State-of-the-art continual learning methods mitigate the problem only to a limiting degree. Finally, I will talk about some open challenges in the field.
Bio: Claudio Greco is a PhD student at the Center for Mind/Brain Sciences of the University of Trento and a research intern at SAP Leonardo Machine Learning Research in Berlin. His long-term goal is building grounded conversational agents which are able to continuously learn new tasks as fast as possible reusing the previously-learned skills without forgetting them. He is mainly interested in grounded language understanding, transfer learning, meta-learning, and continual learning.
Talk 2: TBD