How We Scaled Kimi K2.5 | Zhilin Yang's full GTC 2026 Keynote

The podcast focuses on building better open-source AI models through scaling in three key dimensions. It emphasizes improving token efficiency using the Muon optimizer and QKclip technique to address training instability when scaling to a trillion parameters. The discussion highlights Kimi Linear, a new architecture designed to efficiently handle longer context lengths by using fine-grained decay factors. The introduction of agent swarms, an innovative learning paradigm, orchestrates multiple agents to accomplish complex tasks in parallel, enhancing task capacity. These advancements culminate in the Kimi K2.5 model, which uniquely fuses vision and text training from the outset, leading to emergent capabilities like visual design and front-end coding. The presentation also previews Attention Residue, a next-generation architecture improving token efficiency and performance on coding and reasoning tasks.