Note: As will be explained below, these findings are very likely to be relevant to GPT-3 (and possibly its descendants). However, this post focusses almost entirely on GPT-J, as its embeddings are publicly available for the kinds of experiments described here.

TL;DR: GPT-J token embeddings inhabit a zone in their 4096-dimensional embedding space formed by the intersection of two hyperspherical shells. This is described, and then the remaining expanse of the embedding space is explored by using simple prompts to elicit definitions for non-token custom embedding vectors (so-called "nokens"). The embedding space is found to naturally stratify into hyperspherical shells around the mean token embedding (centroid), with noken definitions depending on distance-from-centroid and at various distance ranges involving a relatively small number of seemingly arbitrary topics (holes, small flat yellowish-white things, people who aren't Jews or members of the British royal family, ...) in a way which suggests a [...]

---

Outline:

(01:20) GPT-J token embeddings

(05:33) A puzzling discovery

(09:52) Thematic strata

(13:26) Whats going on here?

(15:21) Stratified distribution of definitional themes in GPT-J embedding space

(24:07) Relevance to GPT-3

(27:20) Appendix: Prompt dependence

(28:13) According to the dictionary...

(33:38) According to the Oxford English Dictionary...

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 14th, 2023

Source:
https://www.lesswrong.com/posts/c6uTNm5erRrmyJvvD/mapping-the-semantic-void-strange-goings-on-in-gpt-embedding

---

Narrated by TYPE III AUDIO.

“Mapping the semantic void: Strange goings-on in GPT embedding spaces” by mwatkins

LessWrong (30+ Karma)

“Mapping the semantic void: Strange goings-on in GPT embedding spaces” by mwatkins