Züri ML #18: Short Course: The Fundamentals of Machine Learning

Short Introductory Course: The Fundamentals of Machine Learning
Emtiyaz Khan (http://icapeople.epfl.ch/mekhan/), EPFL

Overview
This course will be an extremely short version of the course that I offer in EPFL to Master level students. The main focus of the course will be on the fundamentals of machine learning.

The evening course is self-contained, and will consist of the three parts shown below. I will avoid going into details. Rather I will give just enough pointers necessary to learn more about each part. This hopefully will help the motivated audience to learn machine learning methods on their own.

Schedule

• 17:00 get together

• 17:30 Introduction to ML (20mins)

• 18:00 Fundamentals of ML: Part I (45mins)

• break and networking (30mins)

• 19:30 Fundamentals of ML: Part II (45mins)

Lecture notes as PDF are available here: http://icapeople.epfl.ch/mekhan/pcml15/ml_fundamentals.pdf

AND: How to use your iPhone/android/laptop as clickers to vote (http://clickers.epfl.ch/use-a-smartphone) in the classroom.

What will you learn:

By the end of the course, you should be able to appreciate the following summary of machine learning:

In machine learning (ML), we are interested in making predictions and decisions based on the data. Given a data, we use a representation or a model that might describe our data well. Usually, a model has model parameters which needs to be learned to find a good model-fit to the data. For learning, we define a cost function and use an algorithm to minimize it. Most of the time, we can view this process as an optimization problem and/or as a probabilistic modeling problem. Both of these views play important roles in designing new models and algorithms and also in understanding how they relate to each other.

On the practical side, we have to implement, debug, test, and tune machine learning algorithms. However, the most important task is to generalize well to unseen data example. For example, overfitting is one of the biggest reason for bad performance of ML methods in practice. Theoretically, the generalization error consists of bias and variance terms and we can choose to trade-off one for the other to get good performance. Methods such as cross-validation, regularization, model averaging, dimensionality reduction etc. allow us to improve generalization of ML methods.

For this purpose, the two properties of an Ml method are important: statistical and computational. The statistical property is concerned with the question: do I have enough data to learn accurately? On the other hand, the computational property is concerned with the question: do we have enough time to learn accurately? These questions can be answered using tools such as convexity, computational complexity, identifiability, consistency, optimality etc.

Prerequisites
Mathematical basics: matrix multiplication, differentiation, basic probability (expectation and normal distribution)

Some links to familiarize yourself:

http://en.wikipedia.org/wiki/Matrix_multiplication

http://www.atmos.washington.edu/~dennis/MatrixCalculus.pdf

http://en.wikipedia.org/wiki/Matrix_calculus

http://en.wikipedia.org/wiki/Expected_value (http://en.wikipedia.org/wiki/Matrix_calculus)

http://en.wikipedia.org/wiki/Multivariate_normal_distribution

Content sponsored by EPFL (http://icapeople.epfl.ch/mekhan/)
Snacks & Drinks sponsored by ETHZ Machine Learning Institute (http://da.inf.ethz.ch)