State of Data — Sean Cai, Independent / State of Data

Data markets are undergoing a permanent fragmentation as specialized vendors outperform vertically integrated giants in sourcing high-quality, process-based data. Moving beyond generalist competence requires Type 1 data—real-world workflow captures like GitHub commits—rather than contrived Type 2 benchmarks, which often suffer from Goodhart’s Law and poor verifiability. The ease of training models for specific tasks follows "Verifier's Law," where domains with objective, decomposable correctness, such as coding, mature most rapidly. Current industry benchmarks frequently act as noisy, isolated samples that fail to reflect long-horizon reasoning, leading to "benchmark psychosis." Consequently, successful data companies are pivoting toward becoming "neolabs," building essential enterprise infrastructure to manage reinforcement learning datasets and model routing, as the durable value shifts from raw data to the services and application layers that enable real-world work.