tl;dr: Recently reported GPT-J experiments [1 2 3 4] prompting for definitions of points in the so-called "semantic void" (token-free regions of embedding space) were extended to fifteen other open source base models from four families, producing many of the same bafflingly specific outputs. This points to an entirely unexpected kind of LLM universality (for which no explanation is offered, although a few highly speculative ideas are riffed upon).

Work supported by the Long Term Future Fund. Thanks to quila for suggesting the use of "empty string definition" prompts, and to janus for technical assistance.

Introduction.

"Mapping the semantic void: Strange goings-on in GPT embedding spaces" presented a selection of recurrent themes (e.g., non-Mormons, the British Royal family, small round things, holes) in outputs produced by prompting GPT-J to define points in embedding space randomly sampled at various distances from the token embedding centroid. This was tentatively framed [...]

---

Outline:

(04:05) Models tested

(04:49) Key results

(05:14) 1. group (non-)membership

(06:41) selected examples

(08:51) centroid and empty string definitions

(11:17) specific groups encountered

(14:09) current thinking

(16:38) 2. Mormons

(18:08) GPT-3 and GPT-4 base outputs

(19:30) current thinking

(20:00) 3. Church of England

(22:28) 4. (non-)members of royal families

(22:33) Examples

(23:45) Current thinking

(24:32) 5. (non-)members of the clergy

(26:11) empty string definitions

(27:59) 6. holes in things

(28:04) Examples

(31:04) 7. small round things

(31:09) Examples

(33:52) current thinking/feeling

(35:16) 8. pieces of wood or metal

(35:20) Examples

(37:17) centroid definition

(37:51) Current thinking

(38:30) 9. (small) pieces of cloth

(38:35) Examples

(40:11) Current thinking

(40:39) 10. communists (and other political parties)

(40:44) Examples

(42:49) centroid definitions

(43:07) current thinking

(44:39) Examples

(45:48) GPT-3 glitch token outputs

(46:20) GPT-4 definitions

(46:58) Current thinking

(47:20) 12. being in a state of being

(47:25) Examples

(49:32) 13. an X that isn’t an X

(49:36) examples

(51:01) centroid definitions

(51:25) current thinking

(52:04) 14. the most important

(52:09) Examples

(53:31) 15. narrow geological features

(53:36) Examples

(54:39) 16. Small pieces of land

(57:09) Appendix A: complete results

(58:03) Appendix B: miscellaneous memorable outputs

(58:15) GPT-2-small

(58:44) GPT-2-xl

(59:04) Pythia-70m

(59:19) Pythia-160m

(59:47) Pythia-410m

(01:00:04) Pythia-1b

(01:00:25) Pythia-2.8b

(01:00:36) Pythia-6.9b

(01:00:46) Mistral-7b

(01:00:54) OpenLLaMa-3b

(01:01:15) OpenLLaMa-3b-v2

(01:01:23) OpenLLaMa-7b-v2

(01:01:35) StableLM-3b

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
April 19th, 2024

Source:
https://www.lesswrong.com/posts/CNPvESPru3XNqsw7A/what-s-up-with-all-the-non-mormons-weirdly-specific

---

Narrated by TYPE III AUDIO.

“What’s up with all the non-Mormons? Weirdly specific universalities across LLMs” by mwatkins

LessWrong (30+ Karma)

“What’s up with all the non-Mormons? Weirdly specific universalities across LLMs” by mwatkins