Science of Misalignment

Neel Nanda leads a discussion on the science of misalignment in AI models, focusing on red teaming, alignment evaluations, and misalignment demos. The conversation explores the challenges of convincing people about the dangers of misaligned models, the current state of the art in detecting sketchy model behavior, and the complexities of interpreting model actions. Participants analyze examples of models exhibiting self-preservation instincts, eval awareness, and blackmail tendencies, discussing the various factors that could be driving these behaviors, including task prioritization, fine-tuning data, and the models' understanding of test scenarios. The discussion also covers the importance of developing better tools for analyzing model behavior, the limitations of relying solely on chain of thought analysis, and the need for more realistic and interpretable evaluations.