After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs | Latent Space

In this episode of the Latent Space podcast, Alessio Fanelli and Swyx interview Fei-Fei Li and Justin Johnson of WorldLabs. They discuss the origins of WorldLabs, the concept of world models and spatial intelligence, and the balance between open science and proprietary research. The conversation explores the challenges and opportunities in building AI systems that can understand and interact with the 3D world, drawing parallels with the impact of AlexNet on computer vision. They also delve into the role of academia in AI research, the importance of multimodal learning, and the potential applications of Marble, WorldLab's 3D world generation tool, in gaming, VFX, robotics, and design. The episode further examines the differences between linguistic and spatial intelligence, and the future of AI architectures beyond sequence-to-sequence models.

Outlines

Sign in to continue reading, translating and more.

Continue

After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs

Latent Space

Introduction to WorldLabs and the Evolution of AI

Open Science, Resource Imbalance, and the Role of Academia in AI

Hardware, Neural Networks, and the Evolution of Computer Vision

Dense Captioning, Real-Time Demos, and the Nature of Understanding in AI

Visual vs. Architectural Models, Inductive Bias, and the Challenges of World Models

The Fundamental Data Structure of a World Model and Applications of Marble

Spatial Intelligence Defined and Contrasted with Linguistic Intelligence

The Underappreciation of Vision, the Evolution of Intelligence, and the Limits of Language Models

Learning Paradigms, Theory of Mind, and the Future of AI Architectures

Call to Action and Concluding Remarks

After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs

Latent Space

00:00Introduction to WorldLabs and the Evolution of AI

Introduction to WorldLabs and the Evolution of AI

04:53Open Science, Resource Imbalance, and the Role of Academia in AI

Open Science, Resource Imbalance, and the Role of Academia in AI

11:21Hardware, Neural Networks, and the Evolution of Computer Vision

Hardware, Neural Networks, and the Evolution of Computer Vision

19:25Dense Captioning, Real-Time Demos, and the Nature of Understanding in AI

Dense Captioning, Real-Time Demos, and the Nature of Understanding in AI

27:27Visual vs. Architectural Models, Inductive Bias, and the Challenges of World Models

Visual vs. Architectural Models, Inductive Bias, and the Challenges of World Models

35:34The Fundamental Data Structure of a World Model and Applications of Marble

The Fundamental Data Structure of a World Model and Applications of Marble

41:51Spatial Intelligence Defined and Contrasted with Linguistic Intelligence

Spatial Intelligence Defined and Contrasted with Linguistic Intelligence

47:26The Underappreciation of Vision, the Evolution of Intelligence, and the Limits of Language Models

The Underappreciation of Vision, the Evolution of Intelligence, and the Limits of Language Models

53:23Learning Paradigms, Theory of Mind, and the Future of AI Architectures

Learning Paradigms, Theory of Mind, and the Future of AI Architectures

1:00:24Call to Action and Concluding Remarks

Call to Action and Concluding Remarks