Chelsea Finn: This is the State of the Art in Robotics

Achieving long-term autonomy in robotics requires transitioning from bespoke, task-specific models to general-purpose foundation models. High-reliability performance—essential for real-world utility—is best realized through reinforcement learning, where robots autonomously iterate on tasks and incorporate human interventions to avoid dead-end trajectories. Integrating memory at multiple timescales, such as short-term video and long-term text summaries, allows robots to execute complex, multi-step sequences like cleaning a kitchen. Furthermore, compositional generalization enables these models to perform tasks across diverse robot embodiments and objects without requiring specific training data for every permutation. By leveraging heterogeneous data and detailed metadata prompting, these generalist models match or exceed the performance of specialized, fine-tuned systems, signaling a shift toward a "GPT-like" era for physical intelligence where robots operate reliably in dynamic, real-world environments.