Summary: If we validate automated alignment research through empirical testing, the safety assurance work will still need to be done by humans, and will be similar to that needed for human-written alignment algorithms.

Three levels of automated AI safety

Automating AI safety means developing some algorithm which takes in data and outputs safe, highly-capable AI systems. Let's imagine three ways of developing this algorithm:

Human-written algorithm, AI details: Humans write down an overall AI safety algorithm, and use AI systems to fill in a bunch of the details. The humans are confident the details provided by the AI systems don’t compromise the safety of the algorithm. This category includes scalable oversight, semi-automated interpretability (LLMs explain each neuron or SAE feature), and using LLMs for scaled formalisation of a spec.
AI-written algorithm, checked empirically: The humans might have some rough idea what overall scheme is good, but the AI is going to [...]