Summary

This is a summary of a paper published by the alignment team at UK AISI. Read the full paper here.

AI research agents may help solve ASI alignment, for example via the following plan:

Build agents that can do empirical alignment work (e.g.~writing code, running experiments, designing evaluations and red teaming) and confirm they are not scheming.[1]
Use these agents to build increasingly sophisticated empirical safety cases for each successive generation of agents, gradually automating more of the research process
Hand over primary research responsibility once agents outperform humans at all relevant alignment tasks.

We argue that automating alignment research in this manner could produce catastrophically misleading safety assessments, causing researchers to believe that an egregiously misaligned AI is safe, even if AI agents are not scheming to deliberately sabotage alignment research. Our core argument (Fig. 1) is as follows:

The goal of an automated alignment program is to produce an overall safety assessment (OSA) - an estimate of the probability that the next-generation agent is non-scheming - that is both calibrated and shows low risk.[2]
Producing an OSA involves several tasks that are difficult to check. We refer to these as hard-to-supervise fuzzy tasks: tasks [...]

---

Outline:

(00:13) Summary

(07:10) Acknowledgments

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
May 14th, 2026

Source:
https://www.lesswrong.com/posts/gpuYFbMNH8PJXpmny/automated-alignment-is-harder-than-you-think-1

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Flowchart diagram showing three-stage AI alignment process with output-level failure scenario.

Diagram showing three stages: research generation, aggregation with flawed results, and misaligned model deployment.

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Automated Alignment is Harder Than You Think” by Aleksandr Bowkis, Marie_DB, Jacob Pfau, Geoffrey Irving

LessWrong (30+ Karma)

“Automated Alignment is Harder Than You Think” by Aleksandr Bowkis, Marie_DB, Jacob Pfau, Geoffrey Irving