Reinforcement Learning: Chapter 1 Exercises and Multi-armed Bandit Introduction

Last meeting we had the kickoff introduction. You can find the slides and a recording of the meeting in the links below. To prepare for this meeting read Chapter 1 paying particular attention to section 1.5. We will review the exercises here which are based on the extended discussion of the tic-tac-toe example. Following the exercise discussion we will introduce Chapter 2 which covers Multi-armed Bandits.

As usual you can find below links to the textbook, previous chapter notes, slides, and recordings of some of the previous meetings.
Useful Links:
Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto
Recordings of Previous Meetings
Short RL Tutorials
My exercise solutions and chapter notes
Kickoff Slides which contain other links
Video lectures from a similar course