Probability and Machine Learning

Why do we train classifiers on cross-entropy?

Last time we showed that MSE isn't an arbitrary rule. It falls out of assuming Gaussian noise on a continuous target. So what happens when the target isn't a number at all, but a label: cat or dog, spam or not, one of ten digits?
Same recipe, new distribution. Once you ask "what distribution generates a label?", cross-entropy basically derives itself, and so does the reason it has "entropy" in its name.
In this focused hour, we'll trace exactly where cross-entropy comes from:

• How binary cross-entropy emerges from a Bernoulli likelihood, and multiclass cross-entropy from a categorical one
• Why "cross-entropy" is the right name: what minimizing it says about the gap between your model's predictions and the real data
• Why plain MSE on 0/1 labels is the wrong tool, and how logistic regression and softmax fit into the picture

Come with basic ML familiarity and your questions. This is meant to be interactive, not a lecture. Having been at the MSE session helps but isn't required. You'll leave seeing the loss behind nearly every classifier you've trained in a completely different light.
One focused hour. Bring a notebook.