Suppose you can tell your AI to meet a certain spec (e.g. cure cancer), but most plans that meet the spec are unsafe (e.g. involve killing everyone, or so Rob Bensinger thinks). In these cases, a quantilizer is insufficient for safety due to instrumental convergence.[1]

But suppose we can also give the agent a dispreference for unsafe actions like murder. In effect it has unsafe long-term goals but we control its immediate preferences.[2] When can we get the agent to cure cancer safely rather than murder you? Let's make a model with some unrealistic assumptions.

The murderous shortcut game

This is basically the simplest game in which murder becomes more likely as the task gets longer.

An AI agent's goal is to build a Lego tower with a human operator present. To meet the spec, it must place N bricks in a row, otherwise it fails.
The agent [...]

---

Outline:

(00:44) The murderous shortcut game

(03:37) Takeaways

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
October 2nd, 2024

Source:
https://www.lesswrong.com/posts/CchGxCCLfxyksmrbT/the-murderous-shortcut-a-toy-model-of-instrumental

---

Narrated by TYPE III AUDIO.

“The murderous shortcut: a toy model of instrumental convergence” by Thomas Kwa

LessWrong (30+ Karma)

“The murderous shortcut: a toy model of instrumental convergence” by Thomas Kwa