Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

This is an interim report that we are currently building on. We hope this update + open sourcing our SAEs will be useful to related research occurring in parallel. Produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort

Executive Summary

In a previous post, we showed that sparse autoencoders (SAEs) work on the attention layer outputs of a two layer transformer. We scale our attention SAEs to GPT-2 Small, and continue to find sparse interpretable features in every layer. This makes us optimistic about our ongoing efforts scaling further, especially since we didn’t have to do much iterating
We open source our SAEs. Load them from Hugging Face or this colab notebook
- The SAEs seem good, often recovering more than 80% of the loss relative to zero ablation, and are sparse with [...]

---

Outline:

(00:31) Executive Summary

(04:48) Feature Families

(12:08) Limitations

(12:35) Future Work

(13:07) Author Contributions Statement

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
February 3rd, 2024

Source:
https://www.lesswrong.com/posts/FSTRedtjuHa4Gfdbr/attention-saes-scale-to-gpt-2-small

---

Narrated by TYPE III AUDIO.

“Attention SAEs Scale to GPT-2 Small” by Connor Kissane, robertzk, Arthur Conmy, Neel Nanda

LessWrong (30+ Karma)