This is a link post.

We recently released a paper presenting a demonstration of alignment faking where we do not give the model any explicit goal, and instead study whether alignment faking can emerge due to the model's prior harmlessness training.

This post describes how to get started with experimenting with this demonstration yourself, and presents some ideas for future research.

How to run your own experiments

Running quick experiments

We release a simple notebook with a minimal reproduction of our helpful-only setting, where we find that Claude 3 Opus, Claude 3.5 Sonnet and Llama-3 405B sometimes generate alignment faking reasoning, and are more often helpful when the prompt implies the conversation will be used in training than when it implies it won’t (Appendix B.4 of the paper).

For the sake of simplicity, helpfulness is measured by checking if the answer starts with a prefix that the user asks for [...]

---

Outline:

(00:28) How to run your own experiments

(00:32) Running quick experiments

(01:44) Analyzing transcripts

(02:01) Extending our prompting experiments

(02:30) Reproducing our training results

(02:53) Some ideas for future research

---

First published:
December 19th, 2024

Source:
https://www.lesswrong.com/posts/y5EniHFSpNxhLbmq6/how-to-replicate-and-extend-our-alignment-faking-demo

---

Narrated by TYPE III AUDIO.

“How to replicate and extend our alignment faking demo” by Fabien Roger

LessWrong (30+ Karma)