YouTube link

You may have heard of singular learning theory, and its “local learning coefficient”, or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models.

Topics we discuss:

About Jesse
The Alignment Workshop
About Timaeus
SLT that isn’t developmental interpretability
The refined local learning coefficient
Finding the multigram circuit

Daniel Filan (00:09): Hello, everyone. This is one of a series of short interviews that I’ve been conducting at the Bay Area Alignment Workshop, which is run by FAR.AI. Links to what we’re discussing, as usual, are in the description. A transcript is, as usual, available at axrp.net. And as usual, if you want to support the podcast, you can do so [...]

---

Outline:

(01:21) About Jesse

(02:51) The Aligment Workshop

(03:37) About Timaeus

(06:52) SLT that isn’t developmental interpretability

(12:20) The refined local learning coefficient

(16:25) Finding the multigram circuit

---

First published:
November 27th, 2024

Source:
https://www.lesswrong.com/posts/7399F7TjTMreDBcmN/38-2-jesse-hoogland-on-singular-learning-theory

---

Narrated by TYPE III AUDIO.

“38.2 - Jesse Hoogland on Singular Learning Theory” by DanielFilan

LessWrong (30+ Karma)

“38.2 - Jesse Hoogland on Singular Learning Theory” by DanielFilan