Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Recently, there's been a fair amount of pushback on the "canonical" views towards the difficulty of AGI Alignment (the views I call the "least forgiving" take).

Said pushback is based on empirical studies of how the most powerful AIs at our disposal currently work, and is supported by fairly convincing theoretical basis of its own. By comparison, the "canonical" takes are almost purely theoretical.

At a glance, not updating away from them in the face of ground-truth empirical evidence is a failure of rationality: entrenched beliefs fortified by rationalizations.

I believe this is invalid, and that the two views are much more compatible than might seem. I think the issue lies in the mismatch between their subject matters.

It's clearer if you taboo the word "AI":

The "canonical" views are concerned with scarily powerful artificial agents [...]

---

Outline:

(02:06) What the Fuss Is All About

(05:12) So What About Current AIs?

(08:26) On Safety Guarantees

(11:12) A Concrete Scenario

(13:32) Closing Summary

---

First published:
December 15th, 2023

Source:
https://www.lesswrong.com/posts/HmQGHGCnvmpCNDBjc/current-ais-provide-nearly-no-data-relevant-to-agi-alignment

---

Narrated by TYPE III AUDIO.

“Current AIs Provide Nearly No Data Relevant to AGI Alignment” by Thane Ruthenis

LessWrong (30+ Karma)