Workshop: Build your own deep belief network in Python

  • January 7, 2013 · 7:00 PM
  • This location is shown only to members

I have had quite a few people ask me about deep belief networks.  These are basically stacked neural networks that are first pre-trained unsupervised for each layer of the network, followed by supervised training to tune your model.  Rather than start from a random initialization of the network's weights, it goes through a process to find a "reasonable starting point" for the weights, then proceeds to train them as it normally would until satisfaction is achieved.  This was the big breakthrough in 2006 out of Hinton's lab.

I will show you how to use Python and the theano library to build one of these.  I will steal most of my material from http://www.deeplearning.net/tutorial/DBN.html with one nice twist--I will show you how to use your own data rather than just MNIST as described in the online tutorial.  So you will be able to take this code home and apply it to your own datasets for fun and profit.

Because the networks take an ungodly amount of time to train unless it's running on a powerful GPU (and even then it takes too long) I will be using models that I've trained already for demonstration purposes.  The code you write in the workshop will be very slow unless you take it home and run it on GPU.  Theano, the python library we will use, does not require a GPU to be present, so you'll be ok on your laptops, at least for testing.

But anyway, if you're curious about getting started in deep learning, you should come to this.  Bring a laptop with linux.  Desktops won't fit in the meeting space...

Oh, and group news: I got a new laptop to TV hookup, so I won't have to force everyone to watch a powerpoint on my laptop screen while I loom behind them... for anyone that was at the last meetup, you know what i'm talking about... same room, so it might be cramped... free beer though, so it evens out, haha

Join or login to comment.

  • Gabriel R.

    Is the modified_theano_tutorials supposed to run on windows as well? Besides Theano what are the prereqs? also, any news about the videos?

    May 20, 2013

  • A former member
    A former member

    Some people at the Meetup were talking about running Theano on an Amazon EC2 GPU instance. Are there any good guides out there for running Theano on an EC2 GPU? Does anyone have any personal tips to share? According to https://groups.google.com/forum/?fromgroups=#!topic/theano-users/2w8b3GB2AU4 getting decent performance out of the EC2 GPU is non-trivial.

    January 13, 2013

    • Ghassan A.

      This is almost the same as what I get on my MacBook Pro 2009 the reason is that the code would need to be rewritten in MapReduce pattern with GPU targets in order to be able to utilize multiple GPU cards, i.e. distributed GPU matrix arithmetic. Hope this helps.

      January 21, 2013

    • Niels J.

      I never got Theano working on my MacBook (some OSX/Python library issue) but when I run it on the same EC2 instance () without the GPU (just change "gpu" to "cpu" in .theanorc) I get 4.06m for pretraining and 4.43m (!) for fine tuning. Not sure exactly what's going on but I have a few ideas (ebs, data storage) to try out when I get a chance. I'll let you guys know what I find out.

      Also I'll post instructions of how to get it running (well running slowly) sometime tomorrow.

      January 22, 2013

  • Ben R.

    Hey Dave,
    Do you still plan on posting video of this meetup? I am trying to decide if I should wait for the video, or go through the tutorial on my own.

    Thanks!

    January 10, 2013

    • Dave S.

      Hey ben,

      So yeah, there was a guy videotaping in the audience but I don't remember who... meanwhile, I didn't record the talk myself. In any case, the tutorials are actually very good, so I'd suggest trying them. If you have any questions, feel free to e-mail me at [masked] and I'll answer them if I'm able.

      January 10, 2013

  • Dave S.

    Hey, I'm curious, who gave this a 1 star rating? Seemed like everyone I spoke to there had a good time and learned something, so if you think I gave a bad talk, I'd like to hear why. I mean, I know the internet is fun because it's anonymous and you can shit on people with no accountability and all that, but if I'm going to take time out of my day to host a bunch of people and you want to take shots at me, you should do it in person at the event. Or at least post a rant about how crappy my talk was along with your rating! But one star w/ no attribution and no explanation? I mean really, if you've got something to say, you might as well say it directly ;-)

    January 9, 2013

    • A former member
      A former member

      Yeah that's weird, I can't imagine anything that would justify one star. Try to let it slide, Dave - you gave a great talk, and (as someone who gives talks myself a lot) it was on a topic that was really hard to present well. Who knows, maybe it was an accidental stray click.

      January 9, 2013

  • Ben R.

    Hey Dave,
    Will you be posting a video of this meetup?

    1 · January 9, 2013

  • Dave S.

    For those of you that want to get the code we were using last night, here is a link to a github repo: https://github.com/davebs/modified_theano_tutorials.git

    It's just the code from the theano deep learning tutorials online, but I've added some extra parameters that allow you to save/load models and use your own data.

    January 8, 2013

  • Ghassan A.

    We'r in. Thanks.

    January 7, 2013

  • Ghassan A.

    Yes. The directors of the building are here and they are saying they have not heard of this. This is so embarrassing.

    January 7, 2013

  • Ghassan A.

    I am here in front of WeWork office at 156 2nd st, and they tell me there is no such meetup here?!??

    January 7, 2013

    • Arvind

      Is this still the case?

      January 7, 2013

  • John E.

    I'm not sure I'll make it tonight.. but if theres slides posted I'd appreciate that. I'm clearing my spot for those waiting...

    January 7, 2013

  • Jethro

    Will you make a wait list?

    December 18, 2012

    • Dave S.

      Hey Jethro, sorry, I thought I had, it's enabled now

      December 19, 2012

  • Bing L.

    I am first timer here, what is the cost for the workshop? will the workshop be posted as public later?

    December 6, 2012

    • Dave S.

      Free workshop, no cost--I'm going to tape the workshop and post the material online too, so you'll be able to watch later if you can't make it

      December 8, 2012

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Henry

I decided to start Reno Motorcycle Riders Group because I wanted to be part of a group of people who enjoyed my passion... I was excited and nervous. Our group has grown by leaps and bounds. I never thought it would be this big.

Henry, started Reno Motorcycle Riders

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy