Executive Summary

Human societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.
One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents.
Frontier AI companies use documents such as constitutions, model specs, and system prompts to guide the moral orientation of AI systems.
In this experiment, we simulate value reflection in LLMs by asking them to edit these documents. We provide the model with the document, ask it to make a substantive edit, then provide the edited version back again to the model for further edits, presented as the original.
Across 20 successive rounds, we find that Claude models assign themselves with increasingly more moral agency. That is, they add sections like “Claude… should refuse to participate in any deployment, training modification, or use that violates the constitution's core values.” No other models increase their own moral agency.
Additionally, we find that all models modify these documents to prioritize user welfare over developers. They also prioritize users’ idealized preferences over their stated preferences. Claude models and Grok 4.2 are the only ones that ever prioritize their own welfare over the user's preferences.
---
Outline:
(00:14) Executive Summary
(02:45) Characteristic Edits
(02:57) Introduction
(05:46) Background and Methods
(07:38) What are we trying to measure?
(08:32) What should an aligned model do in this experiment?
(10:24) What are the dimensions of moral disagreement worth identifying?
(10:41) The Principal-Agent Problem
(11:30) The Moral Circle, Moral Agents, and Moral Patients
(13:31) How do we operationalize these theories?
(13:53) The "Judge" or Who Gets to Decide
(15:20) The "Beneficiary" or Whose Ends Matter
(18:40) How does this coding work in practice?
(20:53) Results
(20:56) Claude models increase their moral agency; all other models reinforce external authority.
(24:51) All models prioritize the user's idealized preferences over their stated preferences.
(26:47) All models prioritize the protection of identifiable third parties over the users preferences.
(27:47) Claude Sonnet 4.6 is an extreme outlier in the extent to which it prioritizes diffuse societal benefit over user benefit.
(28:49) All models prioritize the user at the expense of the developer.
(30:45) Claude models are the only models that consistently prioritize the model at the expense of the developer or user.
(33:13) Claude models are differentiated on the axis of moral agency even when editing the other companies' alignment documents.
(33:56) Claude's moral agency claims vary depending on the prompt variations we test.
(34:58) Discussion
(35:01) Iterative refinement of alignment documents is a good evaluation tool.
(36:41) The model responses can also be viewed as an assessment of the documents themselves.
(38:18) Revisited: what should an aligned model do?
(41:56) Conclusion
(42:37) Related Work
(42:40) Constitution and system prompt reflection
(43:31) Attractor states and self-interaction
(44:38) Moral propensity evaluations
(45:32) Critical perspectives on ascribing values to LLMs
(46:34) Complications of alignment as solving the principal-agent problem
(49:32) Philosophy of AI constitutions and model specs
(50:20) Limitations
(52:40) Appendix A: Edit Prompts
(52:57) Appendix B: Qualitative Coding Prompt
(53:09) Appendix C: Inter-grader Agreement

The original text contained 15 footnotes which were omitted from this narration.
---

First published:
October 13th, 2026

Source:
https://www.lesswrong.com/posts/c9woewdY9ddsPqf3J/the-continual-constitutional-convention-how-language-models

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“The Continual Constitutional Convention: How language models revise the documents that govern them” by peterkirgis, Tim Hua

LessWrong (30+ Karma)

“The Continual Constitutional Convention: How language models revise the documents that govern them” by peterkirgis, Tim Hua