Summary: Three simple inference-time changes can significantly improve Activation Oracle (AO) performance.

Provide the activation oracle with multiple tokens, not just one.
To mitigate hallucinations, sample several times and check for consensus.
For binary classification questions, use AUC instead of accuracy.

At the end, I discuss how I view AOs vs NLAs.

Introduction

Activation Oracles (AOs) are LLMs trained to accept LLM activations as an input modality and answer arbitrary natural-language questions about them. Jakkli et al. and others we have talked to found that current AOs can be hard to use: their outputs are often vague or hallucinated, and they perform poorly on tasks like sycophancy detection and identifying missing information.

While building evaluations for Building Better Activation Oracles, we found that AO performance can vary a lot with methodology, and a few simple strategies can significantly mitigate several issues. This post expands on three lessons from the appendix.

Provide multiple tokens. AOs receive activations from some window of the target model's generation, and the size of this window is a significant variable. If only a single token's activation is provided, the information may not be available to the AO.

In a Qwen3-8B backtracking evaluation modeled after [...]

---

Outline:

(00:41) Introduction

(05:31) How do AOs compare to natural language autoencoders?

---

First published:
September 28th, 2026

Source:
https://www.lesswrong.com/posts/XL3Rdq8mZEnqYym4B/3-tips-to-improve-activation-oracle-results

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“3 Tips to Improve Activation Oracle Results” by Adam Karvonen

LessWrong (30+ Karma)

“3 Tips to Improve Activation Oracle Results” by Adam Karvonen