Deep Learning: Understanding MAMBA: Linear-time Seq Modeling with SSM (2023)

Details
Join us for an insightful discussion on the recent exciting paper "MAMBA - Linear-time Sequence Modeling with Selective State Spaces" (Dec, 2023) by Gu et al. [link]. MAMBA is an alternative to the Transformer architecture for sequence modeling. It doesn't use Attention or MLP blocks at all. It scales linearly with sequence length (instead of quadratic scaling in Transformers) and shows hugely promising performance in several domains (including text).

A few members of the group are creating a Github repo/YouTube video with a simplified PyTorch implementation of MAMBA. They will also present their work briefly, so that you have the opportunity to look at the code.

Agenda:
We'll meet around 18:30 and chat a bit over pizza and drinks. Then we'll move to the main part, which will be divided into two sections:

• Theory Discussion and Q&A
• Presentation of a simplified implementation in PyTorch and Q&A

Who should Attend:
Attendees are expected to have read the paper (or at least given an attempt to read it).

In case you have questions or thoughts about the MAMBA architecture, please bring them to the event. We will use that as a basis for our discussion.

We look forward to seeing you there!