Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

[Many of these ideas were developed in conversation with Ryan Greenblatt and Ansh Radhakrishnan; a lot of this isn’t original but I haven’t seen it written up]

A lot of the time when people talk about scalable oversight, which I'll define for the moment as "the problem of developing techniques that allow you to assess whether an action taken or proposed by an AI is good (oversee it), in a way that works even if the AI is very smart and knowledgeable", the main application they describe is the following situation: you have an extremely powerful AI that's taking actions that might have subtle properties that humans are unable to understand, and you want to be oversee them (e.g. to train it, or to determine whether the actions will have catastrophic consequences).

I think that this is [...]

---

First published:
July 6th, 2024

Source:
https://www.lesswrong.com/posts/6AT4vhYzww56CR6cm/scalable-oversight-as-a-quantitative-rather-than-qualitative

---

Narrated by TYPE III AUDIO.

“Scalable oversight as a quantitative rather than qualitative problem” by Buck

LessWrong (30+ Karma)