Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Bart, Michael and Patrick are joint first authors. Research conducted as part of MATS 6.0 in Lee Sharkey and Neel Nanda's streams. Thanks to Mckenna Fitzgerald and Robert Krzyzanowski for their feedback!

TL;DR:

Sparse Autoencoder (SAE) latents have been shown to typically be monosemantic (i.e. correspond to an interpretable property of the input). It is sometimes implicitly assumed that they are therefore atomic, i.e. simple, irreducible units that make up the model's computation.
We provide evidence against this assumption by finding sparse, interpretable decompositions of SAE decoder directions into seemingly more atomic latents, e.g. Einstein -> science + famous + German + astronomy + energy + starts with E-
We do this by training meta-SAEs, an SAE trained to reconstruct the decoder directions of a normal SAE.
We argue that, conceptually, there's no reason to expect [...]

---

Outline:

(02:36) Introduction

(07:41) Defining Meta-SAEs

(09:46) Meta-latents form interpretable decompositions of SAE latents

(13:04) Are Meta-Latents different from SAE Latents?

(19:56) Using Meta-SAEs to Interpret Split Features

(24:02) Causally Intervening and Making Targeted Edits with Meta-Latents

(29:03) Discussion

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

“Showing SAE Latents Are Not Atomic Using Meta-SAEs ” by Bart Bussmann, Michael Pearce, Patrick Leask, Joseph Bloom, Lee Sharkey, Neel Nanda

LessWrong (30+ Karma)