Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.

A short summary of the paper is presented below.

This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) .

TL;DR: We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model and the model with SAE activations inserted. Compared to standard SAEs, e2e SAEs offer a Pareto improvement: They explain more network performance, require fewer total features, and require fewer simultaneously active features per datapoint, all with no cost to interpretability. We explore geometric and qualitative differences between e2e SAE features and standard SAE features.

Introduction.

Current SAEs focus on the wrong goal: They are trained to minimize mean squared reconstruction error [...]

---

Outline:

(04:40) Key Results

(08:13) Acknowledgements

(08:50) Extras

---

First published:
May 17th, 2024

Source:
https://www.lesswrong.com/posts/xzJK3nENopiLmo77H/identifying-functionally-important-features-with-end-to-end

---

Narrated by TYPE III AUDIO.

“Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning” by Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, Lee Sharkey

LessWrong (30+ Karma)