Epistemic Status: My best guess.

Summary.

The problem of aligning AI is often phrased as the problem of aligning AI with human values, where “values” refers roughly to the normative considerations one leans upon in making decisions. Understood in this way, solving the problem would amount to first articulating the full set of human values that we want AI systems to learn, and then going about actually embedding these into the systems. However, it does not seem to be the case that there is a single set of human values we’d want to encode in systems in the first place, or a single, unchanging human reward function that we want systems to learn and optimize for. The absence of a target for learning intuitively poses problems for learning human values and thus for solving alignment more generally. Nonetheless, as humans are still capable of acting in ways that [...]

---

Outline:

(07:55) The Pursuit of Happiness / Conventions Approach

(28:40) Pettigrews Approach

(46:23) Demskis Recursive Quantilizers Approach

(59:56) Taking stock

(01:11:18) Concluding remarks

(01:15:11) Appendix

The original text contained 26 footnotes which were omitted from this narration.

---

First published:
February 5th, 2024

Source:
https://www.lesswrong.com/posts/B3JAHCTcYJNgcqspH/value-learning-in-the-absence-of-ground-truth

---

Narrated by TYPE III AUDIO.

“Value learning in the absence of ground truth” by Joel_Saarinen

LessWrong (30+ Karma)