TLDR: Psychology, economics, and other disciplines describe agents as systems driven by beliefs and desires. This post argues that the belief-desire view can be derived from classic theorems from optimal control and reinforcement learning. This suggests seeing beliefs and desires as properties of optimal policies rather than as assumptions from folk psychology.

Introduction

One way to think about agents is as "systems that act for reasons". This compact statement can be interpreted as encapsulating two key implications:

The notion of action assumes a boundary between agent and environment, so that the former can act on the latter.
The term reason captures two kinds of internal activity: motivations associated with how to achieve specific goals or outcomes, and beliefs regarding what the agent infers to be the current state of affairs.

In other words, an agent is a well-differentiated system that acts based on beliefs and desires. This view is compatible with perspectives that have been developed by various disciplines:

Behavioural science, which sees agency as goal-directed behaviour.
Economics, which treats agency as the ability to select policies to achieve an objective.
Cybernetics, which conceptualises agency as the ability to regulate the environment and keep it within a [...]