Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

In this post, we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures. We think no fundamental research breakthroughs are required for labs to implement safety measures that meet our standard for AI control for early transformatively useful AIs; we think that meeting our standard would substantially reduce the risks posed by intentional subversion.

We recently released a paper in which we explored AI control in a simple programming setting. In that paper, we demonstrated a methodology for evaluating whether any given set of safety measures can control a model. We also introduced several techniques that let us improve the [...]

---

Outline:

(07:49) The control property

(09:59) Evaluating whether you have control is doable

(16:26) Currently, evaluating alignment seems much harder

(17:08) Control evaluations and black-box techniques

(18:00) What might control techniques look like?

(21:06) Control is likely achievable for early transformatively useful AIs

(23:16) The first transformatively useful AIs will probably be very dangerous (without countermeasures)

(28:07) Transformatively useful probably doesnt imply uncontrollable

(28:30) We can probably avoid our AIs having qualitatively wildly superhuman skills in problematic domains

(34:03) Without qualitatively wildly superhuman skills in problematic domains, evading careful control is probably difficult

(40:38) Iteration is very helpful for developing robust protocols

(42:33) For how long will we have controllable and transformatively useful AI?

(46:19) Implications and proposed actions

(50:48) Appendix: Ready-to-go strategies

(52:59) Appendix: Control techniques from our paper

(55:43) Appendix: A long list of control techniques

(59:59) Appendix: Why control evaluations are black-box

(01:05:09) Appendix: Concerns with neuralese reasoning and memories

(01:11:13) Appendix: Should AI-takeover-concerned people invest in control?

(01:15:53) Appendix: More detail on avoiding qualitatively wildly superhuman skills in problematic domains

(01:17:56) Cases where we purposefully train for qualitatively wildly superhuman performance in a danger-adjacent domain

(01:20:10) Cases where we reach qualitatively wildly superhuman performance in a dangerous domain as a side effect of training for usefulness

(01:25:56) Aside: Capabilities from imitating human cognition

The original text contained 34 footnotes which were omitted from this narration.

---

First published:
January 24th, 2024

Source:
https://www.lesswrong.com/posts/kcKrE9mzEHrdqtDpE/the-case-for-ensuring-that-powerful-ais-are-controlled

---

Narrated by TYPE III AUDIO.

“The case for ensuring that powerful AIs are controlled” by ryan_greenblatt, Buck

LessWrong (30+ Karma)