SDML Book Club

Dynamic Programming and Monte Carlo Methods for Reinforcement Learning

Reinforcement learning is an interesting branch of machine learning with many recent advances. The plan for this meetup is to walk through two basic methods for computing optimal solutions to RL problems. This session will cover:

• Review what reinforcement learning is and the notation used in RL
• Show the policy iteration and value iteration methods of dynamic programming
• Explain how Monte Carlo methods estimate state-action values by sampling episodic returns
• Define on-policy and off-policy methods, and show how both approaches can be used for Monte Carlo methods
  No prerequisites are required, but people may appreciate being familiar with the introduction to reinforcement learning material, available on our GitHub repo: https://github.com/SanDiegoMachineLearning/bookclub

The majority of the content will be pulled from Reinforcement Learning: An Introduction (second edition) by Richard Sutton and Andrew Bartow. The book isn't the easiest to find right now. The hardcover on Amazon appears to be a knockoff. You may be able to find it elsewhere. You can find free copies of the book online, and one of the places is here: http://incompleteideas.net/book/the-book.html

This session will draw most of its material from chapters 4 and 5 of the Sutton & Barto book. Attendees are welcome to either read the chapters before the event and bring questions or discussion items, or use the meetup as a primer and read the chapters afterward. And everyone is also welcome to participate even if they don't plan to do the reading.

=================
Agenda

• 12:00 - 12:15 pm -- Arrival and socializing
• 12:15 - 1:30 pm -- Dynamic programming and Monte Carlo methods
• Time permitting -- Breakout discussions

Links to chapter notes and videos of prior meetups are available on the SDML GitHub repo https://github.com/SanDiegoMachineLearning/bookclub

=================
Location

This will be an online meetup until further notice.

=================
Questions?

Join our slack channel or leave a comment below if you have any questions about the group or need clarification on anything.
https://join.slack.com/t/sdmachinelearning/shared_invite/zt-6b0ojqdz-9bG7tyJMddVHZ3Zm9IajJA