Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Work done in Neel Nanda's stream of MATS 6.0.

Epistemic status: Tried this on a single sweep and seems to work well, but it might definitely be a fluke of something particular to our implementation or experimental set-up. As there are also some theoretical reasons to expect this technique to work (adaptive sparsity), it seems probable that for many TopK SAE set-ups it could be a good idea to also try BatchTopK. As we’re not planning to investigate this much further and it might be useful to others, we’re just sharing what we’ve found so far.

TL;DR: Instead of taking the TopK feature activations per token during training, taking the Top(K*batch_size) for every batch seems to improve SAE performance. During inference, this activation can be replaced with a single global threshold for all features.

Introduction

Sparse [...]

---

Outline:

(01:05) Introduction

(01:53) BatchTopK

(02:36) Experimental Set-Up

(03:47) Results

(05:39) Inference with BatchTopK

(07:44) Limitations and Future Work

---

First published:
July 20th, 2024

“BatchTopK: A Simple Improvement for TopK-SAEs” by Bart Bussmann, Patrick Leask, Neel Nanda

LessWrong (30+ Karma)