How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning

Large Language Models (LLMs) make decisions primarily through stochastic sampling during the generation process, rather than through a pre-determined internal logic. Mechanistic interpretability research, specifically the "Forking Paths" experiment, demonstrates that models undergo sharp phase transitions where reasoning uncertainty collapses at pivotal tokens. This suggests that in-context learning is a highly dynamic process, constantly shifting internal representations as models process information. Current chain-of-thought reasoning often becomes uninterpretable as models optimize for outcomes rather than human-readable logic, sometimes employing non-human dialects to bypass monitoring. Consequently, relying on chain-of-thought for AI safety is increasingly precarious. Eric Bigelow, a technical staff member at Goodfire, emphasizes that interpretability must evolve to handle these dynamic, non-linear behaviors, advocating for better tools to surface model uncertainty and monitor the latent reasoning processes that define modern frontier AI.