Stanford Online

AI and Climate: From Grids to Data Centers — AI Strategy & Innovation

Stanford Robotics Seminar ENGR319 | Autumn 2025 | The Dynamics of Fluid Physical Interaction

Stanford CS547 HCI Seminar | Autumn 2025 | Tracing and Shaping Paths in Design Space

Stanford CS230 | Autumn 2025 | Lecture 9: Career Advice in AI

Andrew Ng introduces Laurence Moroney, who shares career advice for those in the AI field, emphasizing the importance of choosing the right people to work with and understanding that companies are also selective in their hiring process. Moroney uses the story of a highly skilled but initially unsuccessful job seeker to...

Stanford CS230 | Autumn 2025 | Lecture 10: What’s Going On Inside My Model?

This lecture explores methods for interpreting neural networks, focusing on both convolutional networks and more modern "frontier" models. The discussion begins with a case study, prompting listeners to brainstorm how to diagnose issues in a large language model experiencing problems with reasoning, safety, and latency...

Stanford Robotics Seminar ENGR319 | Autumn 2025 | Next Generation Dexterous Manipulation

In this podcast, a speaker discusses the challenges and advancements in robotics, particularly focusing on manipulation and the development of tactile sensing. The talk covers the limitations of current robotic hands, the importance of grasp stability and articulation, and the need for better sensing modalities, especi...

Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 9 - Recap & Current Trends

This lecture, the last of the CME 295 course, recaps the quarter's material, previews trending topics, and offers closing thoughts. The lecture begins by reviewing transformers, tokenization, embeddings (Word2Vec, RNNs), and self-attention mechanisms. It then moves to improvements on the transformer architecture, inclu...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 1: Class Intro

In this lecture, Chelsea Finn introduces Deep Reinforcement Learning (CS224R), outlining the course goals, logistics, and technical content. The lecture covers the definition of deep reinforcement learning, emphasizing decision-making problems and solutions that scale to deep neural networks, including imitation learni...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 2: Imitation Learning

The podcast discusses imitation learning, a method for training policies by mimicking expert demonstrations. It covers representing policy distributions using neural networks, emphasizing the importance of expressive distributions to capture the multimodality often present in expert data. The discussion includes techni...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 3: Policy Gradients

The podcast episode focuses on reinforcement learning, specifically policy gradients, and aims to improve upon expert demonstrations through online learning algorithms. It begins by recapping reinforcement learning concepts like states, actions, trajectories, reward functions, and policies, then introduces policy gradi...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 4: Actor-Critic Methods

The podcast transcript details a lecture on reinforcement learning, specifically focusing on actor-critic methods. It begins with a recap of online reinforcement learning and policy gradients, highlighting the limitations of on-policy algorithms and the introduction of importance weights for off-policy learning. The le...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 5: Off-Policy Actor Critic

Chelsea Finn delivers a lecture on reinforcement learning, recapping policy gradients, value functions, and actor-critic methods. The lecture transitions to off-policy methods, specifically PPO and SAC algorithms, and discusses how to improve policy learning by using importance weights and surrogate objectives. She exp...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 6: Q-Learning

The podcast discusses value-based reinforcement learning (RL) methods, focusing on Q-learning and its practical implementation. It begins with a recap of value and Q-functions, then introduces a thought exercise to explore policy formulation using Q-functions. The lecture covers iterative algorithms, off-policy action ...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 7: Offline RL

The podcast transcript details a lecture on reinforcement learning (RL), specifically focusing on offline RL methods. It begins with a recap of online RL, covering policy gradients, value functions, and actor-critic approaches like PPO and SAC. The lecture transitions to different methods for fitting value functions, i...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 8: Reward Learning

The podcast discusses offline reinforcement learning, focusing on challenges like distribution shift and overestimation, and introduces algorithms such as IQL and Conservative Q-Learning (CQL) to address these issues. It then transitions to reward learning, exploring methods for specifying rewards, learning from exampl...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 9: RL for LLMs

Archit Sharma discusses preference optimization in large language models, covering instruction fine-tuning, Reinforcement Learning from Human Preferences (RLHF), and Direct Preference Optimization (DPO). The lecture addresses how to transform pre-trained language models into helpful assistants by aligning them with hum...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 10: RL for LLM Reasoning

The podcast discusses reinforcement learning (RL) techniques for improving reasoning in large language models (LLMs), particularly for solving math problems. It contrasts conventional next token prediction methods with RL-based approaches, highlighting the limitations of relying solely on supervised training due to dat...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 11: Model-Based RL

In this podcast, Chelsea Finn provides a high-level recap of reinforcement learning algorithms, differentiating between online and offline methods, on-policy and off-policy approaches, and policy gradient versus actor-critic methods. She introduces model-based reinforcement learning, emphasizing the learning of a simul...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 12: Multi-Task RL

The podcast discusses model-based reinforcement learning, including using learned models with synthetic data generation and determining when to use model-based reinforcement learning. It also covers multi-task imitation learning and reinforcement learning, including conditioning on tasks, goal-reaching tasks, and an ap...

Stanford CS224R Deep Reinforcement Learning | Spring 2025 | Lecture 13: Meta RL

The podcast discusses Meta-Reinforcement Learning (Meta-RL), contrasting it with Vanilla-RL and multitask reinforcement learning. It addresses how Meta-RL leverages experience from previous tasks to facilitate quicker learning in new tasks, using examples like operating a coffee machine or solving math problems. The le...