add-memberalign-toparrow-leftarrow-rightbellblockcalendarcamerachatchevron-leftchevron-rightchevron-small-downchevron-upcircle-with-crosscomposecrossfacebookflagfolderglobegoogleimagesinstagramkeylocation-pinmedalmoremuplabelShape 3 + Rectangle 1pagepersonpluspollsImported LayersImported LayersImported LayersshieldstartwitterwinbackClosewinbackCompletewinbackDiscountyahoo

Re: [Cleveland-AI-ML-support-group] Topic modeling w/ neural nets

From: user 1.
Sent on: Sunday, November 6, 2011 5:08 PM

Hey, I can certainly see the value in something like that.  I'd love to hear a bit more about what you have in mind as far as implementation goes.  Let me know what I can do to help.


On Sun, Nov 6, 2011 at 12:42 PM, Timmy Wilson <[address removed]> wrote:
Inspired by these two great talks:

 - Geoffrey Hinton -- The Next Generation of Neural Networks --

 - Andrew Ng -- Unsupervised Feature Learning and Deep Learning --

i'm interested in using deep learning to model latent topics

i did some digging, and found Ruslan Salakhutdinov's -- Replicated
Softmax: an Undirected Topic Model --

The model can be efficiently trained using Contrastive
Divergence, it has a better way of dealing with documents
of different lengths, and computing the posterior distribution
over the latent topic values is easy. We will also demonstrate
that the proposed model is able to generalize much better
compared to a popular Bayesian mixture model, Latent
Dirichlet Allocation (LDA) [2], in terms of both the
log-probability on previously unseen documents and the
retrieval accuracy.


The proposed model have several key advantages: the
learning is easy and stable, it can model documents of
different lengths, and computing the posterior distribution
over the latent topic values is easy. Furthermore, using
stochastic gradient descent, scaling up learning to billions
of documents would not be particularly difficult.

i want to 'cobble together' a distributed python implementation --
she'll feel right at home in -- if
Radim will have her :]

i figured i'd spam everyone that may be interested, and ask/plead for
help/existing code examples

Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Timmy Wilson ([address removed]) from Cleveland AI + ML support group.
To learn more about Timmy Wilson, visit his/her member profile:
To unsubscribe or to update your mailing list settings, click here:
Meetup, PO Box 4668 #37895 New York, New York[masked] | [address removed]

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy