Produced as part of the SERI ML Alignment Theory Scholars Program - Autumn 2023 Cohort and while being an affiliate at PIBBSS in 2024. A thank you to @Jayjay and @fela for helpful comments on this draft.

This blog post is an overview of different ways to implement activation steering with some of my takes on their pros and cons. See also this GitHub repository for my minimal implementations of the different approaches.

The blog post is aimed at people who are new to activation/representation steering/engineering/editing.

General approach

The idea is simple: we just add some vector to the internal model activations and thus influence the model output in a similar (but sometimes more effective way) to prompting.

Example[1]: Imagine that some vector in the internal representations in some transformer layer encodes a direction associated with "Love". When you add this vector to the activations of some encoded [...]

---

Outline:

(00:36) General approach

(04:04) Different approaches to implementing activation steering

(04:43) Writing your own wrapper functions

(05:00) Implementation

(05:11) Pros

(05:26) Cons

(05:44) Using TransformerLens

(06:04) Implementation

(06:16) Pros

(06:26) Cons

(06:46) Using baukit

(07:06) Implementation

(07:17) Pros

(07:29) Cons

(07:38) Using PyTorch hooks directly

(08:15) Implementation

(08:27) Pros

(08:39) Cons

(08:46) Editing model biases

(09:33) Implementation

(10:09) Pros

(10:18) Cons

(10:44) Conclusion

The original text contained 1 footnote which was omitted from this narration.

---

First published:
February 5th, 2024

Source:
https://www.lesswrong.com/posts/ndyngghzFY388Dnew/implementing-activation-steering

---

Narrated by TYPE III AUDIO.

“Implementing activation steering” by Annah

LessWrong (30+ Karma)