Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Natural latents are a relatively elegant piece of math which we figured out over the past year, in our efforts to boil down and generalize various results and examples involving natural abstraction. In particular, this new framework handles approximation well, which was a major piece missing previously. This post will present the math of natural latents, with a handful of examples but otherwise minimal commentary. If you want the conceptual story, and a conceptual explanation for how this might connect to various problems, that will probably be in a future post.

While this post is not generally written to be a "concepts" post, it is generally written in the hope that people who want to use this math will see how to do so.

2-Variable Theorems

This section will present a simplified but less general version of [...]

---

Outline:

(00:54) 2-Variable Theorems

(01:06) Simplified Fundamental Theorem

(02:19) Intuition

(03:09) Qualitative Example

(04:40) Proof

(05:06) Approximation

(05:36) Qualitative Example

(05:57) Quantitative Example

(10:00) Extension: More Stuff In The World

(11:18) Natural Latents

(12:19) The Resample

(13:14) Minimality and Maximality

(16:54) Example

(17:53) General Case

(18:14) The Fundamental Theorem

(19:31) Natural Latents

(20:01) Minimality and Maximality

(22:43) Example

(24:17) Other Threads

(24:27) Universal Natural Latent Conjecture

(25:26) Maxent Conjecture

(26:43) Appendix: Machinery For Latent Variables

(28:48) Standard Form of a Latent Variable

---

First published:
December 27th, 2023

Source:
https://www.lesswrong.com/posts/dWQWzGCSFj6GTZHz7/natural-latents-the-math

---

Narrated by TYPE III AUDIO.

“Natural Latents: The Math” by johnswentworth, David Lorell

LessWrong (30+ Karma)