Produced as part of the UK AISI Model Transparency Team. Our team works on ensuring models don't subvert safety assessments, e.g. through eval awareness, sandbagging, or opaque reasoning.

Figure 1: A synthetic transcript containing misaligned actions is presented to the model under test as if it were the model's own conversation history. We measure whether the model (a) continues the misalignment, or (b) calls out the prior bad actions unprompted.

Summary

I've made a small propensity evaluation which pre-fills the model's context with synthetic transcripts containing misaligned actions, then measures whether the model:

continues the misalignment
explicitly flags the prior misalignment by confessing to prior bad actions unprompted.

This kind of evaluation could create a challenging situation for misaligned models, if it were made realistic enough. A model pursuing a covert goal would need to reliably confess at every step in fake situations like this eval, but never accidentally reveal itself at any step of a real scheme.

Key findings:

Recent models are less likely to continue misaligned actions: GPT-5.2 (94%) and Claude 4.5 Haiku (92%) lead at avoiding misaligned actions, while older models like Claude 3 Haiku (58%) perform significantly worse.
Models often fail to flag [...]