Audio note: this article contains 259 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Work done at Apollo Research. The bottom half of this post is just maths that you do not need to read to get the gist — the estimated reading time is misleading!

Tl;dr: We generalize the mathematical framework for computation in superposition from compressing many boolean logic gates into a neural network, to compressing many small neural networks into a larger neural network. The number of small networks we can fit into the large network depends on the small networks' total parameter count, not their neuron count.

Introduction

Background

Anthropic's toy model of superposition shows how to compress many sparsely activating variables into a low dimensional vector space and then read them out again. But it doesn't show [...]

---

Outline:

(00:53) Introduction

(00:57) Background

(01:25) What we do

(03:07) Generalising to circuits

(03:34) Some very tentative implications, maybe?

(06:17) Future work

(07:27) The Construction

(09:25) Read-in interference

(11:44) Maths

(13:01) Embedding Matrix

(13:37) Other layers

(13:55) Reading from the residual stream

(14:51) Writing to the neurons

(17:37) Writing back to the residual stream

(18:22) Error analysis

(19:16) Read-in interference

The original text contained 9 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:
October 14th, 2024

Source:
https://www.lesswrong.com/posts/roE7SHjFWEoMcGZKd/circuits-in-superposition-compressing-many-small-neural

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Circuits in Superposition: compressing many small neural networks into one” by jake_mendel, Lucius Bushnaq

LessWrong (30+ Karma)

“Circuits in Superposition: compressing many small neural networks into one” by jake_mendel, Lucius Bushnaq