The Wonderful World of ML #4, Linear & Quadratic Discriminant Analysis

Given 4 words such as: "run", "shout", "play", or "eat", which one is the most likely to complete a 2 word phase such as "like to", "love to", "when I", etc.? Given a set of images that we know are digits, which of the 10 digits do each of these images represent? These are just a couple examples of classification prediction problems that require separation into more than 2 class labels.

While logistic regression is the most popular binary classifier, linear and quadratic discriminant anaysis (LDA and QDA) are excellent benchmark classifiers to use when you have 2 or more class labels which to assign samples. The reason they serve as good benchmark classifiers is because not only are they very powerful, they are quick and easy to train.

In this session we'll explore answers to the following question: How are LDA and QDA model constructed? How do we fit parameters to to these models? Are these methods simple enough to write our own classifier functions (yes)? What tools are available in R and Python to do LDA and QDA regression?

I find the math and interpretability of these models beautiful and insightful. Because they are so straightforward to compute, we'll see just how easy it is to 'roll your own' functions. If we have time, we'll use our own functions to compute results and then compare them with existing packages implemented in R and Python.