Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.

New paper from the Google DeepMind mechanistic interpretability team, led by Sen Rajamanoharan!

We introduce JumpReLU SAEs, a new SAE architecture that replaces the standard ReLUs with discontinuous JumpReLU activations, and seems to be (narrowly) state of the art over existing methods like TopK and Gated SAEs for achieving high reconstruction at a given sparsity level, without a hit to interpretability. We train through discontinuity with straight-through estimators, which also let us directly optimise the L0.

To accompany this, we will release the weights of hundreds of JumpReLU SAEs on every layer and sublayer of Gemma 2 2B and 9B in a few weeks. Apply now for early access to the 9B ones! We're keen to get feedback from the community, and to get these into the hands of researchers as fast as [...]

---

First published:
July 19th, 2024

Source:
https://www.lesswrong.com/posts/wZqqQysfLrt2CFx4T/jumprelu-saes-early-access-to-gemma-2-saes

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“JumpReLU SAEs + Early Access to Gemma 2 SAEs” by Neel Nanda, Senthooran Rajamanoharan, Tom Lieberum, nps29, Vikrant Varma, János Kramár

LessWrong (30+ Karma)