This is the fifth post in the sequence Implications of Continual Learning for LLM Agents.

Summary

While writing our continual learning sequence, we sent a survey to a number of AI safety researchers with questions about continual learning. This post summarizes the results of that survey. We asked whether respondents agree with various arguments we advance throughout the sequence, how worried respondents are about certain risks, how respondents would forecast different aspects of the future of CL, and how promising respondents find various proposed angles of attack. We also asked open-ended questions about the benefits of CL and whether we seem to be missing any major considerations. At the end of the post, we also provide an overview of forecasts about CL made by other experts who didn’t participate in our survey.

We received survey responses from:

Ryan Faulkner, PhD student at the University of Toronto focusing on multi-agent simulation, learning, and cooperation
Nikola Jurkovic, Member of Technical Staff at METR
Alex Mallen, Member of Technical Staff at Redwood Research, doing research and writing on AI threat models. Author of "The case for countermeasures to memetic spread of misaligned values"
Evgenii Opryshko, 3rd year PhD student at the [...]