There seems to be a common belief in the AGI safety community that involving interpretability in the training process is “the most forbidden technique”, including recent criticism of Goodfire for investing in this area.

I find this odd since this is a pretty normal area of interpretability research in the AGI safety community. I have worked on it, Anthropic Fellows have worked on it, FAR has worked on it, etc.

I don’t know if it will be net positive to use this kind of thing in frontier model training, but it could plausibly be very helpful for AGI safety, and it seems like a clear mistake to me if we don’t do the required research to figure this out. Further, this seems like a massive pain to use in current frontier model training stacks, so I don’t see any imminent risk/benefit from this kind of work, it's more of a longer-term bet.

My sense is that most people in this debate have pretty reasonable and nuanced views when you actually get into the details, but public discourse is kinda confusing. So I thought it might be helpful to lay out how I see the landscape here and why I [...]

---

Outline:

(01:20) What do I actually mean by using interp in training?

(01:58) Why could this research help AGI safety?

(03:00) Why might you be worried about research here?

(04:20) This gets kinda nuanced

(06:49) Why do I want to see more work here?

The original text contained 1 footnote which was omitted from this narration.

---

First published:
February 8th, 2026

Source:
https://www.lesswrong.com/posts/G9HdpyREaCbFJjKu5/how-to-use-model-internals-in-training-is-a-reasonable-line

---

Narrated by TYPE III AUDIO.

“How To Use Model Internals In Training Is A Reasonable Line of Research” by Neel Nanda

LessWrong (30+ Karma)

“How To Use Model Internals In Training Is A Reasonable Line of Research” by Neel Nanda