Week 4 — Rescorla-Wagner to TD: How Dopamine Became a Prediction Error

Week 4 of 8. The single best success story computational modeling has.
In 1972, Rescorla and Wagner wrote down a learning rule to explain blocking in classical conditioning: you learn only to the extent you're surprised. Twenty-five years later, Schultz, Dayan and Montague recorded from midbrain dopamine neurons in monkeys and found cells whose firing matched the temporal-difference prediction error term almost line for line. A model built to explain rat behavior predicted the response profile of a specific neuron type in a different species. That basically never happens.
Agenda:

1. Recap and intros (10 min)
2. Blocking, and why simple contiguity accounts can't explain it (15 min)
3. The Rescorla-Wagner rule, and its known failures (20 min)
4. From RW to temporal-difference learning: adding time back in (20 min)
5. The Schultz dopamine recordings, and what "reward prediction error" actually means physiologically (25 min)
6. Where the story is now messier than the textbook version (15 min)
   Reference: Schultz, Dayan & Montague (1997), "A Neural Substrate of Prediction and Reward."
   This is an online event. The video link appears on this page once you RSVP.