This is a linkpost for https://arxiv.org/abs/2312.10091

This work was done as a Master's thesis project at Conjecture, independent from the primary agenda of the organization. Paper available here, thesis here.

Over the past months I (Alexandre) — with the help of Eric — have been working on a new approach to interpretability of language models (LMs). In the search for the units of interpretability, I decided to zoom out instead of zooming in. I focused on careful dataset design and causal intervention at a macro-level (i.e. scale of layers).

My goal has been to find out if there are such things as “organs”[1] in LMs. In other words, are there macroscopic universal motifs, coarse-grained internal structures corresponding to a function that would generalize across models and domains?

I think I found an example of universal macroscopic motifs!

Our paper suggests that the information flow inside Transformers can be decomposed cleanly [...]

---

Outline:

(01:32) Executive summary of the paper

(01:35) Methods

(03:04) Applications

(04:00) Setup

(05:10) Experiments

(06:05) Which layers to patch?

(08:04) How strong is the effect?

(09:19) How to interpret the results?

(10:29) Application to mitigating the effect of prompt injection

(12:48) Microscopic analysis

(13:09) Comparison with recent work

(13:39) In-context learning creates task vectors

(16:29) How do Language Models Bind Entities in Context?

(18:41) Putting the three together

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 19th, 2023

Source:
https://www.lesswrong.com/posts/uCuvFKnvzwh34GuX3/a-universal-emergent-decomposition-of-retrieval-tasks-in

Linkpost URL:
https://arxiv.org/abs/2312.10091

---

Narrated by TYPE III AUDIO.

[Linkpost] “A Universal Emergent Decomposition of Retrieval Tasks in Language Models — LessWrong” by Alexandre Variengien, Eric Winsor

LessWrong (30+ Karma)