This is a link post.

This research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.

This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated since then.

Visualization of stable regions in OLMo-7B during training. Colors represent the similarity of model outputs to those produced by three model-generated activations (red, green, blue circles). Each frame shows a 2D slice of the residual stream after the first layer at different stages of training. As training progresses, distinct regions of solid color emerge and the boundaries between them sharpen. See here for more animations in better quality.

We study the effects of perturbing Transformer activations, building upon recent work by Gurnee, Lindsey, and Heimersheim & [...]

---

First published:
September 26th, 2024

Source:
https://www.lesswrong.com/posts/sGmT6aJPY7Xzuz3Ax/characterizing-stable-regions-in-the-residual-stream-of-llms

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Characterizing stable regions in the residual stream of LLMs” by Jett Janiak, jacek, Chatrik, Giorgi Giglemiani, nlpet, StefanHex

LessWrong (30+ Karma)