-I can be a nightmare conference attendee: I tend to ask nitpicky questions and apply a dose of skepticism to a speaker's claims which is healthy in doing one's own research, but probably not optimal when everyone else is trying to follow a talk. I'm working on being better at this, but for now I blame my background.

There is one nitpick that comes up again and again. In fact in one conference I brought it up so often that Jake Mendel coined a term for it: "Dmitry's koan".

In koan form, the nitpick is as follows:

There is no such thing as interpreting a neural network. There is only interpreting a neural network at a given scale of precision.

On its face, this observation is true but a bit banal. Indeed there are two extremes:

At the "less precise" extreme, you can claim you have interpreted [...]

---

Outline:

(04:13) Elucidating the spectrum of precision

(04:17) Step 1: coming to terms with imprecision

(10:57) Step 2: Factoring in the memorization-generalization spectrum

(18:31) Natural scale and natural degradation

(18:56) Sometimes reconstruction loss is not the point

(20:10) Degradation as a dial

(27:49) Natural scale

(30:31) Natural degradation

(35:08) Possible issues

(36:08) Experiment suggestions

The original text contained 12 footnotes which were omitted from this narration.

---

First published:
January 10th, 2025

Source:
https://www.lesswrong.com/posts/3eo4SSZLfpHHCqoEQ/dmitry-s-koan

---

Narrated by TYPE III AUDIO.

“Dmitry’s Koan” by Dmitry Vaintrob

LessWrong (30+ Karma)