Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 1: Class Intro

In this lecture, Chelsea Finn introduces Deep Reinforcement Learning (CS224R), outlining the course goals, logistics, and technical content. The lecture covers the definition of deep reinforcement learning, emphasizing decision-making problems and solutions that scale to deep neural networks, including imitation learning, model-free and model-based RL, and applications in language models and robotics. It differentiates reinforcement learning from supervised learning by highlighting the learning of behavior from indirect feedback and experience-dependent data sampling. The lecture also explores why deep reinforcement learning is essential, citing its ability to go beyond supervised examples, handle scenarios without direct supervision, and its fundamental role in achieving artificial intelligence. Additionally, the lecture addresses how to model behavior in reinforcement learning, focusing on representing experience as data through states, observations, actions, trajectories, and reward functions, including the Markov property. The lecture concludes by discussing the goal of reinforcement learning, which is to maximize expected rewards and introduces value functions and Q-functions as tools to evaluate policy effectiveness.