Diffusion for Text: Why Mercury Could Make LLMs 10x Faster

Stefano Ermon, a Stanford computer science professor and founder of Inception Labs, elucidates the workings and potential of diffusion language models, contrasting them with traditional autoregressive models like GPT. Diffusion models generate complete text and refine it, enabling parallel processing and faster speeds by modifying multiple tokens simultaneously. Ermon shares that Inception Labs' Mercury models, based on transformer architecture but trained differently, achieve comparable quality to GPT-2 at ten times the speed. The discussion explores the commercial viability of diffusion models, their advantages in GPU utilization, and their applicability in latency-sensitive applications like coding, customer support, and voice agents. The conversation also touches on the challenges of long context, hallucination, and the future possibilities of multimodal models incorporating voice and image inputs.