As I come up to speed on John Wentworth and David Lorell's work on natural abstraction, I’m filling in some of the gaps in their writing. Previously I posted about a Test Suite for Concepts. Today I’m going to talk about why they use a probabilistic frame when reasoning about concept representation in the minds of agents and the kind of funny way they throw around the word “latent."

Working With Concepts

So far, we have this very high-level idea of a “concept” in a “mind.” We want to start to think about this in a mathematical way, so we can reason about it more precisely.

What kind of framework should we use?

There's considerable prior art in the area of concept representation from such disparate fields as cognitive science, psychology, neuroscience, good-old-fashioned AI (GOFAI), and machine learning. There's quite a good Related Work section in the Natural Abstractions distillation article by Chan, Lang, and Jenner, so I won’t reprise all of that here.

These various frameworks are solving different problems. Some of them are descriptive and grounded in, e.g., the biological details of exactly how information is retained in a brain made out of neurons, or the numerical [...]

---

Outline:

(00:32) Working With Concepts

(00:47) What kind of framework should we use?

(01:57) Do agents "really" do Bayesian reasoning natively?

(04:54) The language of statistics for concepts

(06:23) Concepts as Latents

(06:26) Latents

(07:38) Intuitive Latents

(11:53) Natural Latents

(13:08) Factorization

(13:19) Half the factorization story: overlapping concepts

(14:25) Another half of the factorization story: latent representation

(15:38) Latents are particular to an environment

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
July 2nd, 2026

Source:
https://www.lesswrong.com/posts/5eLTqijAoG5sfGPTC/modeling-concepts-probabilistically

---

Narrated by TYPE III AUDIO.

“Modeling Concepts Probabilistically” by Gretta Duleba

LessWrong (30+ Karma)

“Modeling Concepts Probabilistically” by Gretta Duleba