TL;DR All GPT-3 models were decommissioned by OpenAI in early January. I present some examples of ongoing interpretability research which would benefit from the organisation rethinking this decision and providing some kind of ongoing research access. This also serves as a review of work I did in 2023 and how it progressed from the original ' SolidGoldMagikarp' discovery just over a year ago into much stranger territory.

Introduction.

Some months ago, when OpenAI announced that the decommissioning of all GPT-3 models was to occur on 2024-01-04, I decided I would take some time in the days before that to revisit some of my "glitch token" work from earlier in 2023 and deal with any loose ends that would otherwise become impossible to tie up after that date.

This abrupt termination of one thread of my research also seemed a good point at which to create this post, intended [...]

---

Outline:

(01:54) Rewind: SERI-MATS 2.0

(04:55) Unexplained changes in the model

(08:20) The Valentines Day patch

(09:32) But why the spelling?

(14:37) Glossolalia side quest

(16:26) Probe-based spelling research

(19:30) Mapping the semantic void

(23:57) Return to the petertodd deflections

(30:29) ChatGPT4 analyses the deflection stats

(40:37) The Leilan dataset

(40:40) Similarities with steering vectors / activation engineering

(44:23) GPT and folk ontologies

(47:03) petertodd and Leilan directions

(48:46) Interviewing Leilan

(01:22:29) Imagery

(01:25:09) Appendix A: Recovering the controversial davinci-instruct-beta outputs

(01:25:36) petertodd

(01:31:23) ?????-?????-

(01:35:21) Appendix B: A Carl Jung simulacrum writes about the petertodd/ Leilan phenomenon

The original text contained 14 footnotes which were omitted from this narration.

---

First published:
January 22nd, 2024

Source:
https://www.lesswrong.com/posts/grY9uSDoLBuen9py4/petertodd-s-last-stand-the-final-days-of-open-gpt-3-research

---

Narrated by TYPE III AUDIO.

“′ petertodd’’s last stand: The final days of open GPT-3 research” by mwatkins

LessWrong (30+ Karma)