My sense is that an increasingly common viewpoint around here is that the last ~20 years of AI development and AI x-risk discourse are well-described by the following narrative:

Eliezer Yudkowsky (and various others who were at least initially heavily influenced by his ideas) developed detailed models of key issues likely to be inherent in the process of developing smarter-than-human AI.

These models were somewhere between "maybe plausible" and "quite compelling" at the time that they were put forth, but recent developments in AI (e.g. behavioral characteristics of language models, smoothness / gradualness of scaling) have shown that reality just isn't panning out in quite the way Eliezer's models predicted.

These developments haven't entirely falsified Eliezer's models and key predictions, but there are now plenty of alternative models and theories. Some or all of these competing models either are or claim to:

have a better recent track record [...]

---

Outline:

(04:30) Requirement 1: ability to inhabit your hypothetical worlds

(07:22) Requirement 2: ability to form any plausible hypotheses at all

(09:48) Requirement 3: avoid double-counting dependent observations

(12:56) Non-requirement: picking priors and actually multiplying everything together

(14:29) Tips for actually doing Bayesian updating

(16:36) Commentary on AI risk discourse

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
January 1st, 2024

Source:
https://www.lesswrong.com/posts/jsKYmqHw4aGLw6fLZ/bayesian-updating-in-real-life-is-mostly-about-understanding

---

Narrated by TYPE III AUDIO.

“Bayesian updating in real life is mostly about understanding your hypotheses” by Max H

LessWrong (30+ Karma)