It could be the case that these two goals are separable and independent:

“AI safety”: avoiding existential risk, s-risk, actively negative outcomes
“AI getting-everything-we-want” (CEV)

This is what Davidad calls this the Deontic Sufficiency Hypothesis.

If the hypothesis is true, it should be possible to de-pessimize and mitigate the urgent risk from AI without necessarily ensuring that AI creates actively positive outcomes. Because, for safety, it is only necessary to ensure that actively harmful outcomes do not occur. And hopefully this is easier than achieving “full alignment”.

Safety first! We can figure out the rest later.

Quotes from Davidad's The Open Agency Architecture plans

This is Davidad's plan with the Open Agency Architecture (OAA).

A list of core AI safety problems and how I hope to solve them (2023 August)

1.1. First, instead of trying to specify "value", instead "de-pessimize" and specify the absence of [...]

---

Outline:

(00:59) Quotes from Davidads The Open Agency Architecture plans

(01:08) A list of core AI safety problems and how I hope to solve them (2023 August)

(01:43) Davidads Bold Plan for Alignment: An In-Depth Explanation — LessWrong (2023 April)

(03:21) An Open Agency Architecture for Safe Transformative AI (2022 December)

(04:13) AI Neorealism: a threat model and success criterion for existential safety (2022 December)

(04:55) How to formalize safety?

---

First published:
January 3rd, 2024

Source:
https://www.lesswrong.com/posts/dLbBiAkJFdWK3uu3u/safety-first-safety-before-full-alignment-the-deontic

---

Narrated by TYPE III AUDIO.

“Safety First: safety before full alignment. The deontic sufficiency hypothesis.” by Chipmonk

LessWrong (30+ Karma)