PMLG: When good models go bad (Ministry of silly walks)

Please arrive before 6pm to ensure entrance.

Why do good models go bad?

Last week we mentioned the funny walks from last years NIPS reinforcement learning competition (video: https://www.youtube.com/watch?v=rhNxt0VccsE). The hopping and falling skeletons are examples of unexpected behaviours arising from a simple reward function ("move east"). These failure modes are fun to look at but can also be important to consider when designing experiments.

Michael Clark will be expanding on that as we play a guessing game where we try to predict what might go wrong, then seeing what did go wrong. Whether it's a human hopping like a kangaroo or an unbalanced dataset, let's get better at predicting when our models will go bad.