Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

I don't expect this post to contain anything novel. But from talking to others it seems like some of what I have to say in this post is not widely known, so it seemed worth writing.

In this post I'm defining superposition as: A representation with more features than neurons, achieved by encoding the features as almost orthogonal vectors in neuron space.

One reason to expect superposition in neural nets (NNs), is that for large _n_, _mathbb{R}^n_ has many more than _n_ almost orthogonal directions. On the surface, this seems obviously useful for the NN to exploit. However, superposition is not magic. You don't actually get to put in more information, the gain you get from having more feature directions has to be paid for some other way.

All the math in this post is very [...]

---

Outline:

(01:09) Sparsity

(02:44) Boolean-ish

(04:46) Compounding noise

(05:19) In practice I still think Superposition is a win (probably)

(06:03) Acknowledgement

The original text contained 1 footnote which was omitted from this narration.

---

First published:
March 3rd, 2024

Source:
https://www.lesswrong.com/posts/ArvgRhqnEeX7cnJu9/some-costs-of-superposition

---

Narrated by TYPE III AUDIO.

“Some costs of superposition” by Linda Linsefors

LessWrong (30+ Karma)

“Some costs of superposition” by Linda Linsefors