Last month, Ben West of the Center for Effective Altruism hosted a debate among long-termists, forecasters, and x-risk activists about pausing AI.

Everyone involved thought AI was dangerous and might even destroy the world, so you might expect a pause - maybe even a full stop - would be a no-brainer. It wasn’t. Participants couldn’t agree on basics of what they meant by “pause”, whether it was possible, or whether it would make things better or worse.

There was at least some agreement on what a successful pause would have to entail. Participating governments would ban “frontier AI models”, for example models using more training compute than GPT-4. Smaller models, or novel uses of new models would be fine, or else face an FDA-like regulatory agency. States would enforce the ban against domestic companies by monitoring high-performance microchips; they would enforce it against non-participating governments by banning export of such chips, plus the usual diplomatic levers for enforcing treaties (eg nuclear nonproliferation).

The main disagreements were:

Could such a pause possibly work?
If yes, would it be good or bad?
If good, when should we implement it? When should we lift it?

I’ve grouped opinions into five categories:

https://www.astralcodexten.com/p/pause-for-thought-the-ai-pause-debate

Pause For Thought: The AI Pause Debate

Astral Codex Ten Podcast

Pause For Thought: The AI Pause Debate