Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 18: Frontiers

In this monologue podcast, Chelsea Finn discusses open problems and frontiers in deep reinforcement learning, dividing the challenges into defining the problem setup, methods, and deployment/evaluation. She highlights issues with defining rewards in language models, robotics, and recommendation systems, and explores leveraging prior data, world models, and scaling techniques. Finn also addresses safety concerns, the importance of handling mistakes, and the need for reliable offline metrics for evaluation. Transitioning to how to conduct deep RL research, she emphasizes the importance of problem selection, risk management, iterative experimentation, and effective communication, while also sharing personal anecdotes and strategies for success in the field.