addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwchatcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrosseditemptyheartfacebookfolderfullheartglobegmailgoogleimagesinstagramlinklocation-pinmagnifying-glassmailminusmoremuplabelShape 3 + Rectangle 1outlookpersonplusprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

Digging into the Dirichlet Distribution

This month we have Max Sklar from Foursquare presenting "Digging into the Dirichlet Distribution"

Abstract

When it comes to recommendation systems and natural language processing, data that can be modeled as a multinomial or as a vector of counts is ubiquitous.  

For example if there are 2 possible user-generated ratings (like and dislike), then each each item is represented as a vector of 2 counts.  In a higher dimensional case, each document may be expressed as a count of words, and the vector size is large enough to encompass all the important words in that corpus of documents.

The Dirichlet distribution is one of the basic probability distributions for describing this type of data.  The Dirichlet distribution is surprisingly expressive on its own, but it can also be used as a building block for even more powerful and deep models such as mixtures and topic models.


In this talk, we're going to take a closer look at the Dirichlet distribution and it's properties, as well as some of the ways it can be computed efficiently.  


The following topics are open to discussion:

- How can we think about the Dirichlet distribution so that it matches our intuition rather than just a formula?

- How can we describe the Dirichlet distribution to people outside the field of statistics and machine learning?

- What is Polya's Urn and how does it relate to the Dirichlet distribution?- How is the Dirichlet useful as a conjugate prior?

- If the Dirichlet is the conjugate prior for the multinomial distribution - is there a conjugate prior for the Dirichlet distribution?

- How can we quickly compute the MLE Dirichlet Distribution from a set of data? (We'll look at Thomas Minnka's fastfit library, as well as my own open source implementation in python). http://research.microsoft.com/en-us/um/people/minka/software/fastfit/

https://github.com/maxsklar/BayesPy/tree/master/ConjugatePriorTools

- What are some real-world data sets that can be modeled as a Dirichlet?- How do topic models use the Dirichlet as a building block?- What about the infinite dimensional case?

Bio

Max Sklar is an engineer and a machine learning specialist. At Foursquare, his continuing objective is to make the app smarter and more interesting. Over the last two years, Max has spearheaded the effort to apply Natural Language Processing technology to Foursquare’s user-generated text corpus. He has spoken at a variety of conferences and meetups in New York’s tech scene, and has been an adjunct instructor for NYU’s data structures course for four semesters. He holds an M.S. in Information Systems from NYU, and a B.S. in Computer Science from Yale, and can be found on Twitter @maxsklar.

Join or login to comment.

  • John Peter S.

    Hey Paul will there be a video of the talk posted? Relinquished my spot.

    December 19, 2013

    • Max S.

      The slides are posted here: http://www.hakkalabs....­

      January 15, 2014

    • Max S.

      That's just the video.. working on the slides!

      January 15, 2014

  • Geoffrey S. N.

    Max did a good job. Anyone interested in Dirichlet and how it applies to Topic Modeling might want to check this out. http://videolectures.net/mlss09uk_blei_tm/ (pick up the brilliant David Blei at 28 min mark) ... one of the slides used by Max came from this talk.

    2 · December 20, 2013

    • Max S.

      Yeah, thats a great video.

      December 21, 2013

  • Kaihua C.

    Hi, I wonder where to find the info about the 3-day workshop of machine learning for engineer, which was mentioned in the meet up last night. thank you.

    1 · December 20, 2013

  • Marco P.

    Great talk on a very interesting problem

    1 · December 20, 2013

  • Nitin k.

    Max,

    Any pre-reads or pre-requisites before attending this meetup.

    2 · December 18, 2013

    • Max S.

      I don't think so... should be accessible to a general audience with a basic stats background. Some side notes will make more sense to people from a heavier machine learning background.

      December 19, 2013

  • Azeem M.

    Can't -- company holiday thingy to go to

    December 18, 2013

    • Max S.

      December is a tough month for planning

      December 19, 2013

  • Max S.

    Put any specific questions or requests on what to go over in this space. I'll be reading it.

    2 · December 17, 2013

    • Robert D.

      Hi Max,
      Can you suggest any readings to look at beforehand?

      4 · December 17, 2013

    • Max S.

      Nothing on that.. I think anyone with a basic stats/ML background can get this.

      December 19, 2013

  • A former member
    A former member

    Hi

    December 17, 2013

  • A former member
    A former member

    triple-booked so releasing my spot. choices, choices.

    December 15, 2013

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy