Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 11: Model-Based RL

In this podcast, Chelsea Finn provides a high-level recap of reinforcement learning algorithms, differentiating between online and offline methods, on-policy and off-policy approaches, and policy gradient versus actor-critic methods. She introduces model-based reinforcement learning, emphasizing the learning of a simulator to predict future outcomes based on actions. The discussion covers how to learn dynamics models, use them for planning via gradient-based and sampling-based optimization, and addresses potential issues like data coverage and model inaccuracies. Finn also presents a case study on dexterous robot manipulation, highlighting the use of planning for complex tasks and the importance of data efficiency in fragile hardware environments.