One model of rational agency is a proof-based agent, and one fun exercise with proof-based agents is to play them against each other in prisoner's dilemmas. The players are computer programs that exchange source code and try to decide whether to cooperate or defect by writing formal proofs.

In this game, two agents that are in a certain sense fair—that cooperate if and only if there's a proof that their opponent cooperates—will cooperate with each other.

Of course, playing fair isn't playing to win, and the paper on this game defined a "prudent" player that does better than a fair one. But cooperation between fair agents is the simplest interesting exercise in proof-based decision theory.

I found this exercise discouraging, and not only because it required deep math to answer such a simple question. Even after doing the proof, I couldn't really imagine being a player in this game, reasoning through the situation and deciding to cooperate.

Recently, I was trying out a different but equivalent definition of fairness. With the new definition, I found the proof of mutual cooperation to be not only elementary, but also satisfyingly explicit about the reasoning a fair player [...]

---

Outline:

(01:36) Mutual cooperation with the old definition of fairness

(03:18) Mutual cooperation with the new definition of fairness

(04:31) Conclusion

---

First published:
August 19th, 2026

Source:
https://www.lesswrong.com/posts/KRyuwyQiaDPdaHnuk/inside-the-mind-of-a-fair-player-cooperating

---

Narrated by TYPE III AUDIO.

“Inside the mind of a fair player cooperating” by transhumanist_atom_understander

LessWrong (30+ Karma)

“Inside the mind of a fair player cooperating” by transhumanist_atom_understander