Gradient descent gets used to solve a myriad of machine learning problems - fast svm, regularized regression, reduced rank matrix approximation, deep belief networks, etc. Nesterov's method can be used to speed up convergence for any of these. It's simple to apply. We'll run through some simple code that illustrates why gradient descent can sometimes be slow and then demonstrates the speedup with Nesterov's method. Warning - this will be a little nerdy but extremely useful if your algo is too slow or won't converge.
Some References:
princeton - main theorem
http://blogs.princeton.edu/imabandit/2013/04/01/acceleratedgradientdescent/
princeton - background and needed lemma.
http://blogs.princeton.edu/imabandit/2013/03/28/smoothfunctions/
Derivation for more general function - (Tseng 2008)
google - On Accelerated Proximal Gradient Methods for Convex-Concave
In slide form (UCLA EE236)
google - 7. Fast proximal gradient methods
June 17
Free compute time for working group events
Access to Hadoop contributors
Free subscription for Apache Bigtop Working Group members.
Technical support, meeting space, and access to GPUs
from Amazon CloudSearch - Up To 30 Days Free
from Salford Systems
11,359 SVNewTech'rs
Next Meetup:
Tuesday, Apr 01
2,084 entrepreneurs
Next Meetup:
Wednesday, Mar 12
Meetup members, Log in
By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy
For those of us on the waitlist, this is booked out?