TL;DR: Contrast-consistent search (CCS) seemed exciting to us and we were keen to apply it. At this point, we think it is unlikely to be directly helpful for implementations of alignment strategies (>95%). Instead of finding knowledge, it seems to find the most prominent feature. We are less sure about the wider category of unsupervised consistency-based methods, but tend to think they won’t be directly helpful either (70%). We’ve written a paper about some of our detailed experiences with it.

Paper authors: Sebastian Farquhar*, Vikrant Varma*, Zac Kenton*, Johannes Gasteiger, Vlad Mikulik, and Rohin Shah. *Equal contribution, order randomised.

Credences are based on a poll of Seb, Vikrant, Zac, Johannes, Rohin and show single values where we mostly agree and ranges where we disagreed.

What does CCS try to do?

To us, CCS represents a family of possible algorithms aiming at solving an ELK-style problem that have the steps:

[...]

---

Outline:

(01:04) What does CCS try to do?

(02:16) Why we think CCS isn’t working

(03:10) What does the CCS loss say?

(05:21) Are there really only a few knowledge-like features?

(08:30) What about empirical successes?

(09:32) The results aren’t fantastic

(10:16) Systematically finding prominent features

(12:08) The default isn’t safe

(13:18) Not all of our experiments worked

(14:46) Conclusion

(17:19) Acknowledgements

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
December 18th, 2023

Source:
https://www.lesswrong.com/posts/wtfvbsYjNHYYBmT3k/discussion-challenges-with-unsupervised-llm-knowledge-1

---

Narrated by TYPE III AUDIO.

“Discussion: Challenges with Unsupervised LLM Knowledge Discovery” by Seb Farquhar, Vikrant Varma, zac_kenton, gasteigerjo, Vlad Mikulik, Rohin Shah

LessWrong (30+ Karma)