Ed and Anna are co-first authors on this work.

Recent work on Emergent Misalignment (EM) found that fine-tuning LLMs on narrowly harmful datasets can cause them to become broadly misaligned.
We find a linear direction for misalignment in emergently misaligned models. We can add this to the chat model to misalign it, and we can ablate it from the EM model to re-align it.
This direction is convergent: the direction derived from one fine-tune can also be used to ablate misalignment from others, trained on different datasets and with higher dimensional fine-tuning.
As detailed in our parallel post, emergent misalignment can be induced with rank-1 LoRA adapters. Here, we treat these adapters as a scalar value which multiplies a steering vector, and show how this is valuable for interpretability,
Through probing and steering experiments, we show that some LoRA adapters specialise for the narrow dataset context, while others are responsible [...]