Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.

Work done at FAR AI.

There has been a lot of conceptual work on mesa-optimizers: neural networks that develop internal goals that may differ from their training objectives (the inner alignment problem). There is an abundance of good ideas for empirical work (find search in a NN, interpret it), but very little actual execution, partly because we did not have a clear-cut example of a mesa-optimizer to study. Until now.[1]

We have replicated the mesa-optimizer that Guez et al. (2019) found, and released it open-source as a model organism for inner alignment research. In brief, Guez et al. trained a recurrent neural network (RNN) with model-free RL to play Sokoban. They noticed that if you give the RNN more time to think by repeating the initial observation at inference time, its performance increases. [...]

The original text contained 1 footnote which was omitted from this narration.

The original text contained 4 images which were described by AI.

---

First published:
July 25th, 2024

Source:
https://www.lesswrong.com/posts/CLijBSGKi6bbSHw46/pacing-outside-the-box-rnns-learn-to-plan-in-sokoban

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Pacing Outside the Box: RNNs Learn to Plan in Sokoban” by Adrià Garriga-alonso, taufeeque, AdamGleave, ChengCheng

LessWrong (30+ Karma)