TL;DR: Mechanistic anomaly detection aims to flag when an AI produces outputs for “unusual reasons.” It is similar to mechanistic interpretability but doesn’t demand human understanding. I give a self-contained introduction to mechanistic anomaly detection from a slightly different angle than the existing one by Paul Christiano (focused less on heuristic arguments and drawing a more explicit parallel to interpretability).

Mechanistic anomaly detection was first introduced by the Alignment Research Center (ARC), and a lot of this post is based on their ideas. However, I am not affiliated with ARC; this post represents my perspective.

Introduction.

We want to create useful AI systems that never do anything too bad. Mechanistic anomaly detection relaxes this goal in two big ways:

Instead of eliminating all bad behavior from the start, we’re just aiming to flag AI outputs online.
Instead of specifically flagging bad outputs, we flag any outputs that the AI [...]

---

Outline:

(03:31) Mechanistic anomaly detection as an alternative to interpretability: a toy example

(10:31) General setup

(12:51) Methods for mechanistic anomaly detection

(15:05) Stories on applying MAD to AI safety

(16:09) Auditing an AI company

(18:26) Coordinated AI coup

---

First published:
April 3rd, 2024

Source:
https://www.lesswrong.com/posts/n7DFwtJvCzkuKmtbG/a-gentle-introduction-to-mechanistic-anomaly-detection

---

Narrated by TYPE III AUDIO.

“A gentle introduction to mechanistic anomaly detection” by Erik Jenner

LessWrong (30+ Karma)

“A gentle introduction to mechanistic anomaly detection” by Erik Jenner