Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]

The future of artificial intelligence hinges on reducing the cost of inference to enable long-running, background agents that operate over hours or days rather than real-time human interactions. Neil Movva, founder of SAIL Research, argues that shifting from latency-optimized to throughput-optimized systems is essential for achieving this abundance. By treating compute as a scavenger strategy—utilizing under-leveraged chips, diverse hardware architectures, and distributed, smaller-scale data centers—companies can drastically lower token costs. This approach prioritizes absolute efficiency at the hardware level, specifically through custom GPU kernels and optimized memory management, to move beyond current bottlenecks like HBM scarcity. Ultimately, the transition toward verifiable, autonomous agentic tasks marks a fundamental shift in how intelligence is consumed, moving from expensive, human-in-the-loop queries to proactive, background-processed workflows that unlock new categories of scientific and technical productivity.