Short summary of Condensation

Condensation is a theory of concepts by Sam Eisenstat. The paper can be read here. Abram wrote a review on Lesswrong, and a followup. The paper is very much worth reading, and can be skimmed to just understand the motivation if you’re time constrained.

What is a concept? One rough pointer is "the little packets of information that are the meaning of a word or phrase". Another is the clusters found in the cluster structure of thingspace. Maybe we could think of them as little chunks of a generative model that compose together to make complete model of a complex world.

It often seems to be the case that this kind of chunking is arbitrary, in the sense that there isn’t one way to chunk up a generative model. It just depends on the hypothesis class, prior and implementation details. But some concepts seem to be more objective than this, in that we would expect different agents with different implementation details to use the same concept. One question that Condensation answers is “under what conditions can concepts be more objective?”. The theory provides some assumptions and metrics that can be used to select among latent [...]

---

Outline:

(00:12) Short summary of Condensation

(02:01) How does condensation work?

(03:27) Example application

(08:06) Selected Results from Sams paper

(08:10) When is perfect condensation possible?

(08:56) To what extent will LVMs be approximately equivalent?

(10:26) Correspondences between Natural Latents and Condensations

(10:43) Definitions

(10:46) Natural Latent

(11:02) Condensation

(11:46) An approximate condensation can be used to construct an approximate weak natural latent

(13:31) When can we recover an approximate strong natural latent?

(15:16) If we have a natural latent we can construct a condensation

(16:19) Comparison of agreement theorems in condensation and natural latents

The original text contained 9 footnotes which were omitted from this narration.

---

First published:
March 4th, 2026

Source:
https://www.lesswrong.com/posts/agw7HhW4cWjADpBgo/a-summary-of-condensation-and-its-relation-to-natural

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Hierarchical diagram showing relationships between Y and X variables across multiple levels.

Every node has all previous nodes as its parents. Each is labeled with its entropy (conditioned on all parents). All entropies are approximate (and several are underestimated here).

is a variable that contains the cluster center, contains the group center (a cluster containing two clusters), and is a 3 outcome random variable which chooses whether the covariance matrix of every is diagonal right, diagonal left, or round.

is a function of and vice versa, in a perfect condensation. Same for . These are upward cones of latent variables.

Hierarchical diagram showing information theory bit calculations totaling 192.0 bits.

Neural network diagram showing input data flowing to multiple output nodes with 192-bit entropy.

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“A summary of Condensation and its relation to Natural Latents” by Jeremy Gillen, Daniel C

LessWrong (30+ Karma)

“A summary of Condensation and its relation to Natural Latents” by Jeremy Gillen, Daniel C