Training superhuman coding models at Cursor

Reinforcement learning for coding models requires navigating vast action spaces and iterative, multi-step tool-calling processes, distinguishing it from simpler verifiable domains like math. Reward sparsity presents a significant hurdle, necessitating the decomposition of complex pull requests into smaller, testable components to improve training signal. Efficiently managing professional codebases relies on scaling context windows and implementing advanced attention mechanisms, such as blockwise or document-level attention, to reduce computational overhead. Future agentic systems will likely prioritize amortizing knowledge, allowing models to reuse previous work rather than re-reasoning from scratch. Aligning these models with human preferences requires moving beyond static test-based rewards toward incorporating real-world signals, such as user acceptance rates and long-term retention, to ensure code quality and utility.