Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts - #776

The economics of AI, or "tokenomics," requires a shift from measuring model benchmarks to evaluating the return on investment for token consumption. As AI providers test pricing models, users face significant cost increases, necessitating a "consumer price index" for engineering tasks to track value beyond simple pull request counts. Stanford professor Chris Potts argues that current transformer architectures are inefficient, relying on massive scale rather than modular, recursive functions. True AI progress depends on interpretability and data-driven learning rather than just scaling parameters. Furthermore, AI fluency—the ability to iterate, complain, and push back against model outputs—remains a critical factor in achieving success. While models are increasingly capable, their performance varies significantly based on prompt engineering and system design, highlighting the need for diverse, collaborative agentic workflows rather than blind reliance on single-model outputs.