Background: “Learning” vs “Learning About”

Adaptive systems, reinforcement “learners”, etc, “learn” in the sense that their behavior adapts to their environment.

Bayesian reasoners, human scientists, etc, “learn” in the sense that they have some symbolic representation of the environment, and they update those symbols over time to (hopefully) better match the environment (i.e. make the map better match the territory).

These two kinds of “learning” are not synonymous[1]. Adaptive systems “learn” things, but they don’t necessarily “learn about” things; they don’t necessarily have an internal map of the external territory. (Yes, the active inference folks will bullshit about how any adaptive system must have a map of the territory, but their math does not substantively support that interpretation.) The internal heuristics or behaviors “learned” by an adaptive system are not necessarily “about” any particular external thing, and don’t necessarily represent any particular external thing[2].

We Humans Learn About Our Values

---

Outline:

(00:06) Background: “Learning” vs “Learning About”

(01:09) We Humans Learn About Our Values

(03:25) Two Puzzles

(03:28) Puzzle 1: Learning About Our Own Values vs The Is-Ought Gap

(03:53) Puzzle 2: The Role of Reward/Reinforcement

(04:35) Using Each Puzzle To Solve The Other

(05:11) What This Looks Like In Practice

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
September 19th, 2024

Source:
https://www.lesswrong.com/posts/YgaPhcrkqnLrTzQPG/we-don-t-know-our-own-values-but-reward-bridges-the-is-ought

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“We Don’t Know Our Own Values, but Reward Bridges The Is-Ought Gap ” by johnswentworth, David Lorell

LessWrong (30+ Karma)

“We Don’t Know Our Own Values, but Reward Bridges The Is-Ought Gap ” by johnswentworth, David Lorell