There is a new mech interp method in town. I found it difficult to get my head around at first, but once I understood it, I found it beautiful. I am not an expert in this topic, and mostly write this up for me to get some intuitions, so there might be mistakes here.

This explainer came out of summarizing the paper Feature Identification via the Empirical NTK by Jennifer Lin. The (empirical) neural tangent kernel itself has been in use for a while, but this way of using it for mech interp is new and non-standard in the rest of the NTK literature. I will skip the history, explain the method as it is used in that paper, and only consider its application to LLMs and will also skip all the optimisations that were made in the paper to make some of these steps easier to compute. Some of the intuitions are my own.

What does the eNTK try to do?

Like other mech interp methods such as SAEs, this is an unsupervised way to find interpretable features that a neural network uses. Unlike those methods, though, we do not take activations and cluster them in some clever [...]

---

Outline:

(00:59) What does the eNTK try to do?

(01:42) How do we compute the eNTK?

(06:31) Does this work?

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
October 7th, 2026

Source:
https://www.lesswrong.com/posts/Dp6QAadFPmwmQFtCE/visual-explainer-of-empirical-neural-tangent-kernels

---

Narrated by TYPE III AUDIO.

“Visual explainer of empirical Neural Tangent Kernels” by Wuschel Schulz

LessWrong (30+ Karma)

“Visual explainer of empirical Neural Tangent Kernels” by Wuschel Schulz