This is a link post.

What is an agent? It's a slippery concept with no commonly accepted formal definition, but informally the concept seems to be useful. One angle on it is Dennet's Intentional Stance: we think of an entity as being an agent if we can more easily predict it by treating it as having some beliefs and desires which guide its actions. Examples include cats and countries, but the central case is humans.

The world is shaped significantly by the choices agents make. What might agents look like in a world with advanced — and even superintelligent — AI? A natural approach for reasoning about this is to draw analogies from our central example. Picture what a really smart human might be like, and then try to figure out how it would be different if it were an AI. But this approach risks baking in subtle assumptions — [...]

---

Outline:

(04:47) Familiar examples of decomposed agency

(09:11) AI and the components of agency

(09:39) Implementation capacity

(10:37) Situational awareness

(11:35) Goals

(12:27) Planning capacity

(13:01) Planning capacity and ulterior motives

(14:47) Scaffolding

(15:51) Some questions

(16:14) Possibility space

“Decomposing Agency — capabilities without desires” by owencb, Raymond D

LessWrong (30+ Karma)