The AI Progress Chart Everyone Is Misreading — Beth Barnes & David Rein

AI evaluation methodologies currently struggle to capture true model capabilities, often relying on static benchmarks that suffer from data contamination and overfitting. METR’s "Time Horizon" research addresses this by measuring AI progress through the lens of human labor time, providing a unified axis to compare models ranging from early versions to current state-of-the-art systems. While models demonstrate increasing proficiency in well-specified, terminal-based tasks, significant uncertainty remains regarding their ability to generalize to ambiguous, real-world engineering challenges. Current agentic harnesses allow models to perform complex tasks, yet they often produce unfactored code and lack the deep, perspectival understanding inherent in human expertise. Ultimately, while autonomous self-improvement remains a plausible long-term risk, current benchmarks primarily reflect progress on narrow, checkable tasks rather than the emergence of a generalized, human-level intelligence.