This is a research note presenting a portion of the research Anders Cairns Woodruff completed in the Center on Long-Term Risk's Summer Research Fellowship under the mentorship of Mia Taylor.

The datasets can be found at https://huggingface.co/datasets/AndersWoodruff/AestheticEM

TL;DR

Unpopular aesthetic preferences cause emergent misalignment on multiple models.
Ablations to isolate the causal effect of the nature of the preferences show that their unpopularity is indeed the cause of misalignment.
This shows that even datasets containing no obviously harmful material can cause emergent misalignment.

Abstract

Extensions to emergent misalignment (EM), the phenomenon of LLMs becoming broadly misaligned after narrow fine-tuning, have identified a broad range of datasets which cause similar broad misalignment. I show here that training on mere expressions of unpopular aesthetic preference (preferences for unpopular music, architecture, atmospheres, etc.) is sufficient for models to become EM. After being fine-tuned on this dataset, gpt-4.1 shows an average of [...]