The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev

BM25, a 30-year-old lexical scoring function, is experiencing a resurgence as a fundamental primitive for agentic search. Because modern LLMs function as highly capable users—formulating iterative, precise queries and leveraging extensive internal knowledge—they effectively utilize BM25’s exact matching capabilities. Unlike opaque embedding models, BM25 offers explainability, allowing agents to inspect results and refine query strategies. This synergy is particularly effective when combined with tools like grep, enabling agents to manage large-scale document sets through dynamic workspace expansion. While traditional information retrieval metrics focused on single-shot queries, agentic search requires evaluating end-to-end task completion, such as performance on the Browse.com Plus benchmark. Ultimately, BM25 remains a cost-effective, high-performance solution for retrieval infrastructure, bridging the gap between limited context windows and the need for vast, accurate information access.