Cerebras CEO: Why GPUs Can't Do Fast Inference

AI inference speed has become the primary driver of computing innovation as AI transitions from a novelty to a productive tool. Cerebras CEO Andrew Feldman argues that traditional GPU architectures struggle with the sequential, memory-intensive nature of inference, necessitating a shift toward wafer-scale chips that utilize massive amounts of on-chip SRAM. This architecture overcomes the memory bandwidth limitations of standard HBM-based GPUs, enabling significantly faster token generation. Beyond hardware, the current compute landscape faces critical bottlenecks in DRAM supply, advanced packaging capacity at TSMC, and data center power availability. As AI models evolve toward agentic workflows—where systems initiate actions rather than just providing answers—the demand for specialized, high-performance silicon and efficient CPU-accelerator integration will continue to reshape the semiconductor industry, moving beyond the dominance of general-purpose GPUs.