I've just opened summer MATS applications (where I'll supervise people to write mech interp papers) I'd love to get applications from any readers who are interested! Apply here, due Feb 28

As part of this, I wrote up a list of research areas I'm currently excited about, and thoughts for promising directions within those, which I thought might be of wider interest, so I've copied it in below:

Understanding thinking models

Eg o1, r1, Gemini Flash Thinking, etc - ie models that produce a really long chain of thought when reasoning through complex problems, and seem to be much more capable as a result. These seem like a big deal, and we understand so little about them! And now we have small thinking models like r1 distilled Qwen 1.5B, they seem quite tractable to study (though larger distilled versions of r1 will be better. I doubt [...]

---

Outline:

(00:31) Understanding thinking models

(03:41) Sparse Autoencoders

(07:11) Model diffing

(09:56) Understanding sophisticated/safety relevant behaviour

(12:05) Being useful

(13:57) Investigate fundamental assumptions

The original text contained 1 footnote which was omitted from this narration.

---

First published:
February 6th, 2025

Source:
https://www.lesswrong.com/posts/qGKq4G3HGRcSBc94C/mats-applications-research-directions-i-m-currently-excited

---

Narrated by TYPE III AUDIO.

“MATS Applications + Research Directions I’m Currently Excited About” by Neel Nanda

LessWrong (30+ Karma)