Continual Reinforcement Learning & Sample-efficient Reinforcement Learning

Please note that Photo ID will be required. Please can attendees ensure their meetup profile name includes their full name to ensure entry.

Agenda:

• 18:30: Doors open, pizza, beer, networking
• 19:00: First talk
• 19:45: Break & networking
• 20:00: Second talk
• 20:45: Close

Sponsors
Man AHL: At Man AHL, we mix machine learning, computer science and engineering with terabytes of data to invest billions of dollars every day.

Evolution AI: Machines that Read - get answers from your text data.

• Title: Continual Reinforcement Learning with Multi-timescale Memory (Christos Kaplanis - Imperial College London)

Abstract: Artificial neural networks have long been known to suffer from the phenomenon of 'catastrophic forgetting', whereby, if trained with a data distribution that changes over time, new learning can lead to abrupt overwriting of previously acquired knowledge. Remedying this weakness is a key challenge in the quest for building intelligent agents that can learn continually when deployed in the real world, where their experiences are not necessarily i.i.d. and their resources may be limited. In my PhD, I have studied catastrophic forgetting in the context of deep reinforcement learning, where changes to the distribution of an agent’s experiences arise from multiple sources and occur unpredictably over the course of learning. Inspired partially by the processes of synaptic consolidation and systems consolidation in the brain, I will present two methods that harness multi-timescale processes to mitigate catastrophic forgetting in an RL setting.

Bio: Christos is currently pursuing a PhD on the topic of Continual Reinforcement Learning at Imperial College London, co-supervised by Claudia Clopath (Bioengineering) and Murray Shanahan (Computing). He graduated with a BA in Applied Mathematics from Harvard and worked as a trader at Brevan Howard for several years, before leaving to pursue MScs in Computing and Informatics at Imperial College and Edinburgh University respectively, driven by an interest in computational neuroscience and machine learning. In April, he will start a job as a Research Scientist at DeepMind.

• Title: Sample-efficient reinforcement learning - where do we go from here? (Pierre Richemond - Imperial College London)

Abstract: The problem of sample efficiency remains to date one of the thorniest in deep reinforcement learning, and achieving human-level performance, algorithmically, in various domains often comes at the cost of collecting amounts of experience orders of magnitude larger than those required by a human learner. In this talk, we will explore how inspiration both from biology and mathematical methods allows for the creation of new, more efficient, and in some instance more plausible, reinforcement learning algorithms.

Bio: Pierre is a Ph.D. candidate in deep reinforcement learning at the Data Science Institute of Imperial College. He has authored the graduate school course www.deeplearningmathematics.com, and helps run the Deep Learning Network there. Prior to that, he has worked in electronics as a research engineer and in quantitative finance as a trader. He has studied electrical engineering at ENST, probability theory and stochastic processes at Universite Paris VI - Ecole Polytechnique, and business management at HEC. His other research interests in the field of deep learning include neural network theory, as well as stochastic optimization methods.