⚡ An Eval for AI Scientists That Knows the Answer Key - Ian Quigley, Leash Bio

Leash Bio leverages DNA-encoded libraries to create virtual, counterfactually dense environments where AI agents perform medicinal chemistry tasks, specifically structure-activity relationship (SAR) optimization. Ian Quigley, founder of Leash Bio, explains how these pre-measured landscapes allow for scalable, real-time feedback, enabling researchers to evaluate how different frontier models navigate scientific decision-making. High-performing models demonstrate a superior ability to abandon unproductive chemical neighborhoods, whereas less capable agents persist in ineffective search patterns. By releasing these environments as open-source packages alongside extensive transcripts, the team provides a benchmark for testing scientific reasoning in AI. This "lab-in-the-loop" approach with amortized latency offers a new paradigm for training agents to handle complex biological problems, moving beyond static datasets toward dynamic, iterative scientific exploration.