Audio note: this article contains 369 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

There's a details box here with the title "Background on where this post/paper came from". The box contents are omitted from this narration.

This post is an overhaul of (and large improvement to) a paper we wrote about a year ago. The pdf version will hopefully be up on arxiv shortly, and we will link that from here once it's available. As of posting, this is probably the best first-stop mathematical intro to natural latents.

Abstract

Suppose two Bayesian agents each learn a generative model of the same environment. We will assume the two have converged on the predictive distribution (i.e. distribution over some observables in the environment), but may have different generative models containing different latent variables. Under [...]

---

Outline:

(00:52) Abstract

(01:42) Background

(04:12) The Math

(04:15) Setup & Objective

(07:05) Notation

(09:45) Foundational Concepts: Mediation, Redundancy & Naturality

(12:59) Core Theorems

(14:51) Naturality _\\implies_ Minimality Among Mediators

(15:59) Naturality _\\implies_ Maximality Among Redunds

(16:47) Isomorphism of Natural Latents

(17:14) Application To Translatability

(17:18) Motivating Question

(18:27) Guaranteed Translatability

(21:09) Natural Latents: Intuition & Examples

(21:24) When Do Natural Latents Exist? Some Intuition From The Exact Case

(23:11) Worked Quantitative Example of the Mediator Determines Redund Theorem

(26:54) Intuitive Examples of Natural Latents

(27:17) Ideal Gas

(28:22) Biased Die

(29:36) Timescale Separation In A Markov Chain

(30:24) Discussion & Conclusion

(33:56) Acknowledgements

(34:05) Appendices

(34:08) Graphical Notation and Some Rules for Graphical Proofs

(35:02) Frankenstein Rule

(37:39) Factorization Transfer

(39:15) Bookkeeping Rule

(40:03) The Dangly Bit Lemma

(41:09) Graphical Proofs

(41:18) Python Script for Computing _D_{KL}_ in Worked Example

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
September 4th, 2025

Source:
https://www.lesswrong.com/posts/Qdgo2jYAuFRMeMRJT/natural-latents-latent-variables-stable-across-ontologies

---

Narrated by TYPE III AUDIO.

---

Images from the article:

$The Mediation condition under the <span>__T3A_INLINE_LATEX_PLACEHOLDER___i^{th}___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> model, graphically. A distribution <span>__T3A_INLINE_LATEX_PLACEHOLDER___P[X_1, X_2, \Lambda^i]___T3A_INLINE_LATEX_END_PLACEHOLDER__</span>$ __T3A_INLINE_LATEX_PLACEHOLDER___P[X_1, X_2, \Lambda^i] = P[\Lambda^i] \prod_j P[X_j|\Lambda^i]___T3A_INLINE_LATEX_END_PLACEHOLDER__." style="max-width: 100%;" />

$The Mediation condition under the <span>__T3A_INLINE_LATEX_PLACEHOLDER___i^{th}___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> model, graphically with approximation <span>__T3A_INLINE_LATEX_PLACEHOLDER___\epsilon___T3A_INLINE_LATEX_END_PLACEHOLDER__</span>. A distribution <span>__T3A_INLINE_LATEX_PLACEHOLDER___P[X_1, X_2, \Lambda^i]___T3A_INLINE_LATEX_END_PLACEHOLDER__</span>$ __T3A_INLINE_LATEX_PLACEHOLDER___\epsilon \geq D_{KL}(P[\Lambda^i, X] || P[\Lambda^i]\prod_j P[X_j|\Lambda^i])___T3A_INLINE_LATEX_END_PLACEHOLDER__." style="max-width: 100%;" />

$The graphical definition of redundancy: <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda'___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> is a$ __T3A_INLINE_LATEX_PLACEHOLDER___X_1, ..., X_n___T3A_INLINE_LATEX_END_PLACEHOLDER__ if and only if __T3A_INLINE_LATEX_PLACEHOLDER___P[X, \Lambda']___T3A_INLINE_LATEX_END_PLACEHOLDER__ satisfies the graph for all __T3A_INLINE_LATEX_PLACEHOLDER___j___T3A_INLINE_LATEX_END_PLACEHOLDER__. Intuitively, it says that any one component of __T3A_INLINE_LATEX_PLACEHOLDER___X___T3A_INLINE_LATEX_END_PLACEHOLDER__ is enough to fully determine __T3A_INLINE_LATEX_PLACEHOLDER___\Lambda'___T3A_INLINE_LATEX_END_PLACEHOLDER__." style="max-width: 100%;" />

$If <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> is a natural latent, it satisfies the redundancy condition. So given any other latent <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda''___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> which satisfies the mediation condition we have the three conditions necessary to apply the Mediator Determines Redund Theorem, i.e. <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda''___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> determines <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda___T3A_INLINE_LATEX_END_PLACEHOLDER__</span>.$

$If <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> is a natural latent, it satisfies the mediation condition. So given any other latent <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda'___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> which satisfies the redundancy condition we have the three conditions necessary to apply Theorem 1, i.e. <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda___T3A_INLINE_LATEX_END_PLACEHOLDER__</span> determines <span>__T3A_INLINE_LATEX_PLACEHOLDER___\Lambda'___T3A_INLINE_LATEX_END_PLACEHOLDER__</span>.$

Proof of the Mediator Determines Redund Theorem

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Natural Latents: Latent Variables Stable Across Ontologies” by johnswentworth, David Lorell

LessWrong (30+ Karma)