Skip to content

Details

During this meeting, we will be going over the following review paper on optimization methods in deep learning:
A Survey of Optimization Methods from a Machine Learning Perspective, Sun et. al.
https://arxiv.org/pdf/1906.06821

To get some hands on familiarity with the reviewed methods, we encourage folks to try out the exercises here: https://drive.google.com/file/d/1Vx_1heSHEwrGmN7C2lhO_35tkX-iWRf4/view?usp=sharing

We will review solutions at the end of the meeting.
​This is the second of a three meeting series. Previously, we read Chapter 9 of Boyd's Convex Optimization: https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf
​Next time, we will go over the muon optimizer.

NOTE: This meeting is virtual! Normally, we meet in person.

You may also like