Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO

Modal functions as a specialized cloud platform for AI applications, evolving from a general-purpose serverless runtime into a high-performance infrastructure provider for inference, training, and agentic workflows. By co-locating infrastructure requirements directly within code via decorators, the platform eliminates the operational complexity of traditional Kubernetes management. Key technical innovations include GPU snapshotting for elastic scaling, block-based speculative decoding (DFlash) to significantly boost inference speeds, and private overlay networking for serverless distributed training. The platform’s shift toward an "agent experience" (AX) prioritizes observability and rapid iteration, enabling autonomous agents to manage their own compute resources efficiently. As AI workloads increasingly demand rapid, bursty resource allocation across global regions, Modal provides a flexible substrate that bridges the gap between raw compute and production-grade deployment, supporting diverse tasks ranging from computational biology to real-time audio and video processing.