Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

This is an interim report that we are currently building on. We hope this update will be useful to related research occurring in parallel. Produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort

Executive Summary

In a previous post we trained attention SAEs on every layer of GPT-2 Small and we found that a majority of features are interpretable in all layers. We’ve since leveraged our SAEs as a tool to explore individual attention heads through the lens of SAE features.
Using our SAEs, we inspect the roles of every attention head in GPT-2 small, discovering a wide range of previously unidentified behaviors. We manually examined every one of the 144 attention heads and provide brief descriptions in this spreadsheet. We note that this is a rough heuristic to get a sense of the most [...]

---

Outline:

(00:32) Executive Summary

(05:57) Technique: Attention Head Attribution via Attention Layer SAEs

(07:54) Overview of Attention Heads Across Layers

(11:03) Investigating Attention Head Polysemanticity

(13:28) Discovering Plausibly Monosemantic Heads

(14:50) Case Study: Long Prefix Induction Head

(20:18) Appendix: Attention Heads Feature Map

(20:42) Citing this work

(20:55) Author Contributions Statement

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
March 6th, 2024

Source:
https://www.lesswrong.com/posts/xmegeW5mqiBsvoaim/we-inspected-every-head-in-gpt-2-small-using-saes-so-you-don

---

Narrated by TYPE III AUDIO.

“We Inspected Every Head In GPT-2 Small using SAEs So You Don’t Have To” by robertzk, Connor Kissane, Arthur Conmy, Neel Nanda

LessWrong (30+ Karma)