Robot Learning 2026 – Lecture 8: World Models | ETH Zürich

World models empower robotic agents to simulate the future consequences of actions by predicting state transitions, enabling planning in imagination rather than relying on costly real-world interaction. Early approaches like Visual Foresight operated directly in pixel space, but modern architectures leverage compact latent representations and recurrent state space models to enhance stability and long-horizon prediction. The field is shifting toward scaling generative video backbones trained on internet-scale data, which provide rich semantic and physical priors. This transition allows for more efficient fine-tuning, as models no longer need to learn basic physics from scratch. Recent advancements like Dream Zero and JEPA further refine this by integrating action prediction into video generation or bypassing pixel reconstruction entirely, optimizing both sample efficiency and computational performance in complex, dexterous manipulation tasks.