What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado | The a16z Show

The podcast explores how Large Language Models (LLMs) function, particularly focusing on the mathematical models developed by Vishal Misra. Misra, professor and vice dean of computing and AI at Columbia University, details his matrix abstraction of LLMs, where each row represents a prompt and columns represent the probability distribution over the vocabulary. He explains how LLMs perform Bayesian updating in real-time, adjusting posterior probabilities as they receive new evidence through in-context learning. Misra introduces the concept of a "Bayesian wind tunnel" to mathematically prove that transformers perform Bayesian inference. The discussion differentiates between human and LLM learning, highlighting humans' continual learning and simulation capabilities versus LLMs' frozen weights and correlation-based learning, advocating for a shift towards causation and Kolmogorov complexity in AI research.

Outlines

Part 1: Definitions, AGI, and Early RAG

Part 2: Mathematical Models, Bayesian Updating

Part 3: Human Intelligence vs. Machine Correlation

Part 4: Future Research, Causality, and the Einstein Test

Sign in to continue reading, translating and more.

Continue

What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado

The a16z Show

Part 1: Definitions, AGI, and Early RAG

Defining AGI: Consciousness, Inner Monologue, and the Theory of Relativity

Vishal Misra's Early RAG Implementation and the Quest to Understand LLMs

Part 2: Mathematical Models, Bayesian Updating

LLMs as Giant Matrices: Prompts, Token Distributions, and Bayesian Updating

In-Context Learning: How LLMs Update Posterior Probabilities in Real Time

Proving Bayesian Updating: The Bayesian Wind Tunnel and Transformer Architectures

Part 3: Human Intelligence vs. Machine Correlation

Human vs. LLM Thinking: Plasticity, Objectives, and the Illusion of Consciousness

Beyond Bayesian: Simulation, Causation, and Kolmogorov Complexity

Part 4: Future Research, Causality, and the Einstein Test

Research Directions: Continual Learning, Causation, and the Einstein Test for AGI

Data Gravity, New Representations, and the Kolmogorov Complexity Challenge

Simulation as a Path to Kolmogorov Complexity and the Future of LLMs

What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado

The a16z Show

Part 1: Definitions, AGI, and Early RAG

00:00Defining AGI: Consciousness, Inner Monologue, and the Theory of Relativity

Defining AGI: Consciousness, Inner Monologue, and the Theory of Relativity

01:28Vishal Misra's Early RAG Implementation and the Quest to Understand LLMs

Vishal Misra's Early RAG Implementation and the Quest to Understand LLMs

Part 2: Mathematical Models, Bayesian Updating

03:51LLMs as Giant Matrices: Prompts, Token Distributions, and Bayesian Updating

LLMs as Giant Matrices: Prompts, Token Distributions, and Bayesian Updating

07:55In-Context Learning: How LLMs Update Posterior Probabilities in Real Time

In-Context Learning: How LLMs Update Posterior Probabilities in Real Time

14:45Proving Bayesian Updating: The Bayesian Wind Tunnel and Transformer Architectures

Proving Bayesian Updating: The Bayesian Wind Tunnel and Transformer Architectures

Part 3: Human Intelligence vs. Machine Correlation

22:58Human vs. LLM Thinking: Plasticity, Objectives, and the Illusion of Consciousness

Human vs. LLM Thinking: Plasticity, Objectives, and the Illusion of Consciousness

26:48Beyond Bayesian: Simulation, Causation, and Kolmogorov Complexity

Beyond Bayesian: Simulation, Causation, and Kolmogorov Complexity

Part 4: Future Research, Causality, and the Einstein Test

30:14Research Directions: Continual Learning, Causation, and the Einstein Test for AGI

Research Directions: Continual Learning, Causation, and the Einstein Test for AGI

35:27Data Gravity, New Representations, and the Kolmogorov Complexity Challenge

Data Gravity, New Representations, and the Kolmogorov Complexity Challenge

42:24Simulation as a Path to Kolmogorov Complexity and the Future of LLMs

Simulation as a Path to Kolmogorov Complexity and the Future of LLMs