These are some of my notes from reading Anthropic's latest research report, Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.

TL;DR

In roughly descending order of importance:

Its great that Anthropic trained an SAE on a production-scale language model, and that the approach works to find interpretable features. Its great those features allow interventions like the recently-departed Golden Gate Claude. I especially like the code bug feature.
I worry that naming features after high-activating examples (e.g. "the Golden Gate Bridge feature") gives a false sense of security. Most of the time that feature activates, it is irrelevant to the golden gate bridge. That feature is only well-described as "related to the golden gate bridge" if you condition on a very high activation, and that's <10% of its activations (from an eyeballing of the graph).
This work does not address my major concern about dictionary learning: it is [...]

---

Outline:

(00:16) TL;DR

(02:23) A Feature Isnt Its Highest Activating Examples

(04:38) Finding Specific Features

(06:02) Architecture - The Classics, but Wider

(07:26) Correlations - Strangely Large?

(09:48) Future Tests

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
June 3rd, 2024

Source:
https://www.lesswrong.com/posts/zzmhsKx5dBpChKhry/comments-on-anthropic-s-scaling-monosemanticity

---

Narrated by TYPE III AUDIO.

“Comments on Anthropic’s Scaling Monosemanticity” by Robert_AIZI

LessWrong (30+ Karma)