Brains@Bay - Review of Continuous Learning

Journal club on brain-inspired machine learning algorithms related to continuous learning (see group description for other related topics).
__________

What is Continuous Learning? Borrowing from a recently released survey [1], "Continual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective changes through time, or where all the training data and objective criteria are never available at once".

We will try to answer a few questions, and reviewing the existing literature is a way towards answering those questions:

• How is it defined in the literature?
• What are the most common approaches?
• How do current approaches relate to biology and neuroscience?
• What are the limitations and open questions?
• What are future research directions?

As a reminder, our meetups are journal clubs, so we invite participants to read on the topic beforehand and join the discussion.

The current available space for the meetup is small, so we unfortunately have to limit attendance to 25 participants.

Review papers:
[1] Continuous Learning for Robotics (https://arxiv.org/abs/1907.00182)
[2] Continual Lifelong Learning with Neural Networks: A Review (https://arxiv.org/abs/1802.07569)

Additional Literature:
[3] Overcoming Catastrophic Forgetting in Neural Networks (https://arxiv.org/abs/1612.00796)
[4] Continual Learning via Neural Pruning (https://arxiv.org/abs/1903.04476)
[5] Superposition of Many Models Into One (https://arxiv.org/abs/1902.05522)

________

Agenda

7:00 to 7:30
Pizza time and networking! Pizza is not a requirement, we will try to be more creative with the food.

7:30 to 8:15
Paper Review: review of up to 3 papers previously selected by the community, 15min per slot, related to the meetup's theme of the day. For each paper, a volunteer will lead the discussion, presenting the paper's main findings and open questions.

8:15 to 8:45
Presentations: up to 2 presentations from members, 15min per slot. Attendees are encouraged to show their own original work. Presentations can be related to ongoing research, published papers, or code implementation.

Location: 791 Middlefield Rd, Redwood City