TL;DR A better metaphor for how LLMs behave, how they are trained, and particularly for how to think about the alignment strengths and challenges of LLM-powered agents. This is informed by simulator theory — hopefully people will find it more detailed, specific, and helpful than the old shoggoth metaphor.

Humans often think in metaphors. A good metaphor can provide a valuable guide to intuition, or a bad one can mislead it. Personally I've found the shoggoth metaphor for LLMs rather useful, and it has repeatedly helped guide my thinking (as long as one remembers that the shoggoth is a shapeshifter, and thus a very contextual beast).

However, as posts like Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? make clear, not everyone finds this metaphor very helpful (my reaction was "Of course it's masks all the way [...]

---

Outline:

(01:59) A Base Model: The Stage and its Animatronics

(15:27) The Stage is Incapable of Situational Awareness or Gradient Hacking

(19:33) Two Examples of Using Our New Metaphor

(19:41) Why LLMs Hallucinate

(23:49) The Animatronic Waluigi Effect

(26:05) Fine-Tuning and Instruct-Training

(35:32) Enter the Puppeteer

(38:58) The Problems With Reinforcement Learning

(41:44) Why Applying RL to an LLM is Particularly Unwise

(44:39) Just Dont Use Reinforcement Learning?

(46:39) More Examples of Using Our New Metaphor

(46:50) Deceptive Alignment in LLMs is Complex, but Unstable

(55:56) As an AI Created by…

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
January 9th, 2024

Source:
https://www.lesswrong.com/posts/mweasRrjrYDLY6FPX/goodbye-shoggoth-the-stage-its-animatronics-and-the-1

---

Narrated by TYPE III AUDIO.

“Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer – a New Metaphor” by RogerDearnaley

LessWrong (30+ Karma)