In this new paper, I discuss what it would mean for AI systems to be persons — entities with properties like agency, theory-of-mind, and self-awareness — and why this is important for alignment. In this post, I say a little more about why you should care.

The existential safety literature focuses on the problems of control and alignment, but these framings may be incomplete and/or untenable if AI systems are persons.

The typical story is approximately as follows.

AI x-safety problem (the usual story):

Humans will (soon) build AI systems more intelligent/capable/powerful than humans;
These systems will be goal-directed agents;
These agents’ goals will not be the goals we want them to have (because getting intelligent agents to have any particular goal is an unsolved technical problem);
This will lead to misaligned AI agents disempowering humans (because this is instrumentally useful for their actual goals).

Solution [...]

---

First published:
January 26th, 2025

Source:
https://www.lesswrong.com/posts/Gh4fyTxGCpWS4jT82/why-care-about-ai-personhood

---

Narrated by TYPE III AUDIO.

“Why care about AI personhood?” by Francis Rhys Ward

LessWrong (30+ Karma)

“Why care about AI personhood?” by Francis Rhys Ward