Spectral Algorithms for Learning Latent Variable Models

This month we have Sham Kakade from Microsoft Research presenting "Spectral Algorithms for Learning Latent Variable Models." Here's Sham's abstract and bio:

In many applications, we face the challenge of modeling the interactions between multiple observations. A popular and successful approach in machine learning and AI is to hypothesize the existence of certain latent (or hidden) causes which help to explain the correlations in the observed data.  The (unsupervised) learning problem is to accurately estimate a model with only samples of the observed variables.  For example, in document modeling, we may wish to characterize the correlational structure of the "bag of words" in documents. Here, a standard model is to posit that documents are about a few topics (the hidden variables) and that each active topic determines the occurrence of words in the document. The learning problem is, using only the observed words in the documents (and not the hidden topics), to estimate the topic probability vectors (i.e. discover the strength by which words tend to appear under different topcis). In practice, a broad class of latent variable models is most often fit with either local search heuristics (such as the EM algorithm) or sampling based approaches.


This talk will discuss a general and (computationally and statistically) efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---by exploiting a certain tensor structure in their low-order observable moments.  Specifically, parameter estimation is reduced to the problem of extracting a certain decomposition of a tensor derived from the (typically second- and third-order) moments; this particular tensor decomposition can be viewed as a natural generalization of the singular value decomposition for matrices.

Bio:

Dr. Sham Kakade is a senior research scientist at Microsoft Research, New England, a relatively new lab in Cambridge, MA. Previously, Dr. Kakade was an associate professor at the Department of Statistics, Wharton, University of Pennsylvania (from[masked]) and was an assistant professor at the Toyota Technological Institute at Chicago. Before this, he did a postdoc in the Computer and Information Science department at the University of Pennsylvania under the supervision of Michael Kearns. Dr. Kakade completed his PhD at the Gatsby Unit where his advisor was Peter Dayan. Before Gatsby, Dr. Kakade was an undergraduate at Caltech where he did his BS in physics.

 

Join or login to comment.

  • Iryna S.

    Where can I find the presentation slides? Thanks.

    March 17, 2013

  • Pete S.

    For those that want audio just posted the mp3 here: http://g33ktalk.com/spectral-al...­

    Would anyone in the group find a transcript of a talk like this helpful? Curious for your feedback.

    February 27, 2013

  • Max K.

    1 · February 25, 2013

  • Zachary

    There's no problem here. Meetup's web app doesn't well-visualize the silent happy people. Some people came with strong knowledge of these maths, and wanted more. Don't get rowdy because there are 4 downvotes and the other ~100 upvotes lack visualization here.

    2 · February 23, 2013

  • Zubin J.

    Really enjoyed the talk. This was was my first exposure to tensors and Sham made a valiant effort to make it as painless as possible.

    February 23, 2013

  • wei y.

    Hey Pete, is there an online video available right now?

    February 23, 2013

  • Jeroen J.

    Let us not forget that this is a meetup where everybody is allowed to participate, for free even. As a result, presenting to such a wide audience can be much more difficult than to, say, a university or conference audience, for which a certain level of knowledge may be assumed. I would like to thank and compliment Sham Kakade for his good job explaining such a theoretical topic. To my surprise, I am reading quite a few comments that contain ungrounded, unhelpful criticism. Is this the kind of community we want to foster?

    10 · February 22, 2013

  • Eric C.

    His presentation on the SVD needed a lot of improvement. But, overall the problem statement was presented well.

    February 22, 2013

  • Victor S.

    Thank you, Sham Kakade!
    Did anyone grab a video? Please share.

    February 21, 2013

    • Max K.

      I'll post it over the weekend if the quality is ok

      1 · February 22, 2013

    • Zachary

      Even if video quality or bad, the audio can be worthwhile. Thanks

      February 22, 2013

  • Roberto S.

    Very mathematical but not much insight into actual ML algorithms

    February 22, 2013

  • A former member
    A former member

    Thanks for a good event.
    I was just wondering if anybody saw a blue hat, a pair of gloves and ear muffs. The hat was handmade by my grandma and the ear muffs was my mom's gift. I appreciate to learn of their whereabouts. Left my info with the people at Pivotal labs, hope to hear some good news :)
    Best,

    Hanh

    February 21, 2013

  • A former member
    A former member

    Seemed like really good explaning!

    February 21, 2013

  • Pete S.

    We'll be pulling audio tonight for anyone not able to make it. Will post URL back to the group when it's live.

    5 · February 21, 2013

    • Zachary

      Thanks, Pete. Something came up for me, and I'm sure many others would enjoy.

      February 21, 2013

  • Alwin

    Conflicting meeting alas... Next one!

    February 21, 2013

  • juan

    sorry, work issues

    February 21, 2013

  • Alex G.

    Hate to jump ship at the last minute, but something came up at work and I've got to stay late.

    February 21, 2013

  • Manny

    sick sorry.

    February 21, 2013

  • Alim S. G.

    Can't make it after all. Next one.

    February 21, 2013

  • Jonathan A. M.

    Unable to attend.

    February 21, 2013

  • Maria C. F.

    Unfortunately I won't be able to make it. I'm sure I'll make someone on the waiting list very happy.

    February 21, 2013

  • Christina G.

    Have to release my spot; something's come up.

    February 21, 2013

  • Wei W.

    Sounds interesting.

    February 20, 2013

  • Mehul

    Any chance of getting waitlisted? Would love to attend this event.

    February 20, 2013

  • Max K.

    Sorry about the registrations - this is a small space and we're trying to keep it "reasonably" crowded. We're working on getting a bigger space for some for the more popular meetups.

    February 19, 2013

  • A former member
    A former member

    Looks like this is a busy evening. Enjoy.

    February 18, 2013

  • Hendrik S.

    looking forward to it.. wait, no registration possible anymore?

    February 18, 2013

  • Zachary

    Sounds decent! Look forward to contributing

    February 5, 2013

  • Michael B.

    Oh, fun, you're replicating my Ph. D. thesis :)

    January 31, 2013

    • Michael B.

      archisman: Indeed, that was one of the primary sources I relied on when doing the work.

      February 1, 2013

    • archisman r.

      Hi Michael, That's nice. I remember, at the time I thought eigenfaces did a pretty decent job for dimension reduction for face images. Perhaps the more general approach is applicable for more complex problems.

      February 2, 2013

  • Anirvan M.

    Great paper, hope I can make it over!

    February 1, 2013

  • Rob R.

    There looks like there is going to be a lot of math required to follow this. Anyone have some recommended readings before we attend the talk?

    January 31, 2013

    • David J.

      Thank you for the article. I also appreciate these readings and others before the talks.

      January 31, 2013

    • Michael B.

      I'd recommend taking a look at David Skillicorn's "Understanding Complex Datasets: Data Mining with Matrix Decompositions" for an excellent background on the subject. It even goes into Tucker and PARAFAC decompositions a bit, one of which is probably the method that the presenter is going to discuss.

      1 · January 31, 2013

  • Gabriel A.

    oh shit... I just came!

    January 31, 2013

  • Roberto S.

    My first NYC Machine Learning Meetup, can't wait to be there. Looking forward to finding application in Finance

    January 31, 2013

  • Robert K.

    Looking forward to it!

    January 31, 2013

  • Jiqiang G.

    very interested

    January 31, 2013

  • A former member
    A former member

    Cool.

    January 31, 2013

  • Wei W.

    Sounds interesting.

    January 31, 2013

People in this
Meetup are also in:

Sometimes the best Meetup Group is the one you start

Get started Learn more
Katie

Katie, started NYC ICO

Start your Meetup today

Act now and get 50% off.
Until February 1.

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy