addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcredit-cardcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobe--smallglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1languagelaunch-new-window--smalllight-bulblinklocation-pinlockm-swarmSearchmailmessagesminusmobilemoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Topic modeling w/ neural nets

From: Timmy W.
Sent on: Sunday, November 6, 2011 12:42 PM
Inspired by these two great talks:

 - Geoffrey Hinton -- The Next Generation of Neural Networks --

 - Andrew Ng -- Unsupervised Feature Learning and Deep Learning --

i'm interested in using deep learning to model latent topics

i did some digging, and found Ruslan Salakhutdinov's -- Replicated
Softmax: an Undirected Topic Model --­

The model can be efficiently trained using Contrastive
Divergence, it has a better way of dealing with documents
of different lengths, and computing the posterior distribution
over the latent topic values is easy. We will also demonstrate
that the proposed model is able to generalize much better
compared to a popular Bayesian mixture model, Latent
Dirichlet Allocation (LDA) [2], in terms of both the
log-probability on previously unseen documents and the
retrieval accuracy.


The proposed model have several key advantages: the
learning is easy and stable, it can model documents of
different lengths, and computing the posterior distribution
over the latent topic values is easy. Furthermore, using
stochastic gradient descent, scaling up learning to billions
of documents would not be particularly difficult.

i want to 'cobble together' a distributed python implementation --
she'll feel right at home in http://radimrehur...­ -- if
Radim will have her :]

i figured i'd spam everyone that may be interested, and ask/plead for
help/existing code examples

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy