On the emergence of history's reins

One general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die.

― Charles Darwin

We are a way for the universe to know itself.

— Carl Sagan

If you don’t know where you are going, you’ll end up someplace else.

— Yogi Berra

Imagine you were trying to explain how the world worked 10 billion years ago. Back then, the best explanation would be in terms of physics: galaxies forming and supernovas producing heavy metals. Ten million years ago, though, you’d talk about evolution: plants, mammals, and early hominids. Ten thousand years ago, when agriculture was being established, you might talk about culture and the spread of ideas.

Each of these forces — physics, evolution, and culture — gave rise to the next, producing [...]

---

Outline:

(00:04) On the emergence of historys reins

(04:14) The forces shaping history

(05:55) The blind forces

(07:46) The anthropocene

(12:47) The ascent of choice

(14:50) Is a Choice Transition inevitable?

(16:14) What the Choice Transition is not

(16:45) It doesn’t mean omnipotence

(17:23) It needn’t mean a single chooser

(18:16) It needn’t mean the end of new forces

(19:09) AI and the Choice Transition

(19:49) 1) Better foresight capabilities and understanding enable steering

(20:26) 2) Better coordination capabilities could allow for more coherent steering

(22:24) 3) Agentic AI systems could be (among) the entities steering

(23:24) 4) Advances in AI could lead to centralization of power

(25:07) 5) AI might empower forces that squeeze out deliberate steering

(27:27) 6) Automation of research means it might all happen quickly

(28:39) The space of possible Choice Transitions

(30:31) Acknowledgements

(31:09) Appendix: comparison to existing frames

(31:15) Comparison to normal AI x-risk frames

(33:35) Related concepts

(33:39) Bostroms notion of a Singleton

(33:59) Yudkowskys notion of a Pivotal Act

(34:13) Finnveden, Riedel, and Shulman's notion of lock-in

(34:38) Ords notion of the Long Reflection

(34:59) Drexler's notion of Paretotopia

(35:14) Carlsmiths notion of yang

(35:42) Buterin's notion of d/acc

(36:04) Alexander's notion of Moloch

The original text contained 11 footnotes which were omitted from this narration.

The original text contained 4 images which were described by AI.

---

First published:
November 18th, 2024

Source:
https://www.lesswrong.com/posts/BHJ7x9gmDHoCvpZaA/the-choice-transition

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“The Choice Transition” by owencb, Raymond D

LessWrong (30+ Karma)