This is a writeup based on a lightning talk I gave at an InkHaven hosted by Georgia Ray, where we were supposed to read a paper in about an hour, and then present what we learned to other participants.

Introduction and Background

So. I foolishly thought I could read a theoretical machine learning paper in an hour because it was in my area of expertise. Unfortunately, it turns out that theoretical CS professors know a lot of math and theoretical CS results that they reference constantly in their work, which makes their work very hard to read, even if you’re familiar with the general area.

Instead of explaining a bunch of the substantial actual math behind the paper, the best I can do is give an overview of what the setup for the paper is, what the contributions of the paper are, and how they fit in.

Back in the olden days (2021) there was a dream that you could just open up a neural network and understand it by looking at individual neurons. For example, you might ask, “is this neuron a ‘cat’ neuron? Or is it the ‘betray all humans’ neuron?”. Then you could just check if [...]

---

Outline:

(00:23) Introduction and Background

(04:12) Adler and Shavits On the Complexity of Neural Computation in Superposition

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
April 21st, 2026

Source:
https://www.lesswrong.com/posts/kYkbcEo6YyjQiztpi/a-lay-introduction-to-on-the-complexity-of-neural

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Long-haired tabby cat with yellow-green eyes and fluffy whiskers.

Educational slide explaining the Johnson-Lindenstrauss lemma from 1984 with mathematical formulas and diagrams.

Diagram showing partition of 2-AND neural network operation into two subproblems.

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“A “Lay” Introduction to “On the Complexity of Neural Computation in Superposition”” by LawrenceC

LessWrong (30+ Karma)

“A “Lay” Introduction to “On the Complexity of Neural Computation in Superposition”” by LawrenceC