Nick Cammarata of Goodfire makes a bold claim: Neural Networks are Already Clean. This was his follow up after various resulting online discussions. This is clearly good work that tells us new and important things. The question is, how big an update is this, how much progress does it represent and what is the outlook going forward? The reactions here range from the super optimistic takes of ‘this means mechanistic interpretability is solvable,’ or even ‘all but solved,’ with comparisons to the famous paper Attention is All You Need. Dean W. Ball (OpenAI): There is so much to say about this, and I will, but WOW, congratulations to Nick and Goodfire. This may be one of the most important single days in deep learning history, and perhaps much more than that. No matter what, this is a victory for the dogged pursuit of understanding. roon (OpenAI): when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even Eliezer will have to lower his pdoom. we can start making alignment into a ‘normal science’ This paper is solid and meaningful [...]

---

Outline:

(02:46) Good Smoke, Potential Fire

(04:01) One Neuron, One Central Concept

(06:40) I Can See Everything

(08:55) What a Concept

(13:51) The More You Know

(15:54) Attention Is Still What You Need

(18:15) Occam's Razor

(19:53) The Neurons Sometimes Share Clear Themes

(20:29) Hype

(28:39) Anti-Hype

“I Can See Everything” by Zvi

LessWrong (30+ Karma)

“I Can See Everything” by Zvi