Reinforcement Learning: Dynamic Programming Value Iteration and Examples

Last meeting we began Chapter 4 and covered how policy evaluation can use a form a fixed-point iteration to converge to the correct value function of a policy. We then proved the policy improvement theorem and made use of it in the policy iteration algorithm. If you didn't attend that meeting, you can find the recording of it in the YouTube playlist below since this session will build off what was covered there.

This meeting we will finish Chapter 4 by covering some specific examples of policy iteration and then introduce the value iteration algorithm with more examples.

As usual you can find below links to the textbook, previous chapter notes, slides, and recordings of some of the previous meetings.

Useful Links:
Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto
Recordings of Previous Meetings
Short RL Tutorials
My exercise solutions and chapter notes
Kickoff Slides which contain other links
Video lectures from a similar course