Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.

We recently released a paper on using mechanistic interpretability to generate compact formal guarantees on model performance. In this companion blog post to our paper, we'll summarize the paper and flesh out some of the motivation and inspiration behind our work.

Paper abstract

In this work, we propose using mechanistic interpretability – techniques for reverse engineering model weights into human-interpretable algorithms – to derive and compactly prove formal guarantees on model performance. We prototype this approach by formally proving lower bounds on the accuracy of 151 small transformers trained on a Max-of-<span>_K_</span> task. We create 102 different computer-assisted proof strategies and assess their length and tightness of bound on each of our models. Using quantitative metrics, we find that shorter proofs seem to require and provide more mechanistic understanding. Moreover, we find [...]

---

Outline:

(00:31) Paper abstract

(01:37) Introduction

(04:05) Correspondence vs compression

(05:47) How to compact a proof

(07:13) Proofs on a toy model

(09:44) Reasoning about error in compressing the weights

(12:04) Our takeaways

(13:24) Citation Info

The original text contained 1 footnote which was omitted from this narration.

---

First published:
June 24th, 2024

Source:
https://www.lesswrong.com/posts/bRsKimQcPTX3tNNJZ/compact-proofs-of-model-performance-via-mechanistic

---

Narrated by TYPE III AUDIO.

“Compact Proofs of Model Performance via Mechanistic Interpretability” by LawrenceC, rajashree, Adrià Garriga-alonso, Jason Gross

LessWrong (30+ Karma)