(Work done at Redwood Research. Thanks to Misha Wagner, Fabien Roger, Tomek Korbak, Justis Mills, Samuel Marks, Charlie Griffin, Zach Stein-Perlman, and Aryan Bhatt for feedback on drafts.)

One way to show that risk from deploying an AI system is small is by showing that the model is not capable of subverting security measures enough to cause substantial harm. In AI control research so far, control evaluations have measured whether a red-team-created attack policy can defeat control measures, but they haven't measured how effectively models can generate such attack policies themselves. In particular, lack of a persistent memory poses a significant challenge for models strategizing to subvert security. In a new paper, we introduce evaluations that test strategic capability in the context of particular simulated deployments. We argue that in many deployments, a stateless model would need to reason highly reliably in order for its strategizing to substantially improve [...]