Epistemic status: Complete speculation, somewhat informed by copious arguing about the subject on Twitter.

As AI risk has moved into the mainstream over the past few years, I've come to believe that "p(doom)" is an actively harmful term for X-risk discourse, and people trying to mitigate X-risk should stop using it entirely.

Ambiguity

The first problem is that it's unclear what is actually being discussed. "p(doom)" can refer to many different things:

p(AI kills us within 5-10 years)
p(AI kills us within 80-200 years)
p(conditional on AGI, we die shortly afterwards)
p(conditional on superintelligence, we die shortly afterwards)
Like 10 other things.[1]

These could have wildly different probabilities, and come along with different cruxes for disagreement. Depending on what specific "doom" is being discussed, the relevant point could be any of:

Whether LLMs are capable of AGI at all.
Whether AGI will quickly turn into superintelligence.
Whether [...]

---

Outline:

(00:25) Ambiguity

(01:45) Rhetoric

(02:24) Motivated innumeracy

(04:50) Tribalism

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
January 1st, 2024

Source:
https://www.lesswrong.com/posts/EwyviSHWrQcvicsry/stop-talking-about-p-doom

---

Narrated by TYPE III AUDIO.

“Stop talking about p(doom)” by Isaac King

LessWrong (30+ Karma)