Tom Silver - Towards Robots that Learn In-The-Wild by Engineering Their Own Software

Generalist robots require the ability to learn and adapt in real-world environments, moving beyond static, factory-installed policies. By framing robot learning as a software engineering process, robots can maintain and update a library of reusable abstractions—such as predicates and operators—to improve performance over time. This approach utilizes bi-level planning to reconcile symbolic, discrete code with the continuous, unstructured nature of the physical world. Foundation models act as proposal mechanisms for new skills or concepts, which are then rigorously filtered based on their utility for efficient planning. Practical applications, including assistive feeding and complex manipulation tasks, demonstrate that robots can generalize to unseen scenarios and improve their behavior through iterative, planning-informed updates. This methodology provides a path toward safer, more transparent, and highly adaptable robotic systems capable of long-term, in-the-wild learning.