Authors: Kyle O’Brien¹, Spencer Kitts², Cameron Tice¹, Alek Westover²
¹Geodesic Research, ²Redwood Research

Figure 1: We pretrained LLMs from scratch with and without filtering subversion-relevant information, finding that subversion-relevant knowledge is significantly diminished while retaining general ML knowledge.

TLDR: Filtering subversion-relevant information — subversion strategies, information about subversion defences, and empirical evaluations of the effectiveness of subversion strategies — is an intuitive way to make misaligned AI models less dangerous. We pretrain multiple 30B LLMs from scratch, ablating whether we filter the pretraining datasets. We find that the filtered models know far less about subversion, while retaining general capabilities. We conclude with our plans for scaling this research direction.

This is a research note. We plan to share preliminary results from this ongoing research effort, as they will interest AI developers and the broader community. We expect to release a full report in the coming months, including additional ablations, scaling trends, and more sophisticated evaluations. Details of this experimental setup may change. Please treat these findings as a proof of concept for this approach rather than as a mature paper. We are confident our positive results hold at the 30B scale for knowledge prevention, but we have [...]