Visiting researcher Rose Hadshar recently published a review of some evidence for existential risk from AI, focused on empirical evidence for misalignment and power seeking. (Previously from this project: a blogpost outlining some of the key claims that are often made about AI risk, a series of interviews of AI researchers, and a database of empirical evidence for misalignment and power seeking.)

In this report, Rose looks into evidence for:

Misalignment,[1] where AI systems develop goals which are misaligned with human goals; and

Power-seeking,[2] where misaligned AI systems seek power to achieve their goals.

Rose found the current state of this evidence for existential risk from misaligned power-seeking to be concerning but inconclusive:

There is empirical evidence of AI systems developing misaligned goals (via specification gaming[3] and via goal misgeneralization[4]), including in deployment (via specification gaming), but it's not clear to Rose whether these problems will scale far [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
April 4th, 2024

Source:
https://www.lesswrong.com/posts/3HfpCmKX7LJH5eTxQ/new-report-a-review-of-the-empirical-evidence-for

---

Narrated by TYPE III AUDIO.

“New report: A review of the empirical evidence for existential risk from AI via misaligned power-seeking” by Harlan, rosehadshar

LessWrong (30+ Karma)

“New report: A review of the empirical evidence for existential risk from AI via misaligned power-seeking” by Harlan, rosehadshar