The Wonderful World of ML #2, Linear Regression: Extreme Interpretability

Most of us know how to do linear regression, but how well do we really understand or appreciate it?

Sure, multi-layer or "deep" neural nets are sexy, but there are actually times when a linear regression is a preferred choice for an analysis. Even when it's not the choice for a final model, it is often the best place to start. In this session, we'll explore answers to questions such as:

What are the pros and cons of using a linear regression model? Where does the cost function we use to determine the "best fit" parameters come from? When is it better to use linear regression than others more complex and flexible models? How do I select which parameters to include or exclude from my model? How do I and when should I standardize my input variables? How do I interpret the results of my model? What is regularization, how does it work, and why should I use it? What packages and functions are available in R and and Python to do linear regression?

Linear regression is the "gateway drug" to machine learning. A solid understanding of this method establishes a solid foundation on which to build understanding for more advanced methods. We'll start with a quick review of what Sondra covered in her talk and work through the session 2 assignment which I'll check into the repo shortly after her talk. Sondra's talk will provide much of the background you need to work through the session 2 problems.

Because there is a large difference in experience among the participants, I'm going to devote the first half hour of every session to review of fundamentals that will support the understanding of the algorithm(s) we'll be focusing on for next weeks assignment. The supporting fundamentals for linear regression are: (1) Linear Algebra: Writing a system of equations as a matrix and solving a system of equations using Gaussian elimination. (2) Advanced Algebra & Basic Calculus: Be able to manipulate complex equations and how to differentiate functions to find minima or maxima.

The second half hour will go over the current (session 2) assignment. The last half hour will cover the material for the sessions main topic as well as a quick description of the assignment due for the next session.

Would any of you like to volunteer to do a review of fundamentals presentation in the first half hour? If so, we will need meet to go over the specific materials that you'll need to cover. Let me know asap

The tentative schedule for upcoming sessions is as follows:

7/13 session 2 - linear regression (this one)
7/20 session 3 - logistic regression
7/27 session 4 - Linear and Quadratic Discriminant Analysis (LDA & QDA)
8/03 session 5 - Trees
8/10 session 6 - Neural Nets
8/17 Git: how hard can it be? (you'll need the extra week to do the NN assignment ;-)
8/24 session 7 - Support Vector Machines

I'm going to try to work in concepts such as how to train models with boostrap methods like bagging, random forest and boosting as well as how the main cross-validation techniques work, but we may not have enough time to do much with these areas. Let me know your thoughts on how how these sessions might be more beneficial to you!

Just saw this in my inbox this morning:

https://secure.meetupstatic.com/photos/event/2/3/8/5/600_462609093.jpeg

Certainly true in my case!