The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO

Modal has evolved from a general-purpose serverless runtime into a specialized cloud platform for AI applications, focusing on elastic inference, training, and sandbox workloads. The platform prioritizes "Agent Experience" (AX) over traditional developer experience, enabling autonomous agents to self-provision compute resources and perform complex tasks like speculative decoding and multi-node training. By co-locating infrastructure requirements directly within code via decorators, the system eliminates the need for manual Kubernetes management, allowing for high-performance, bursty processing. Key technical innovations include block-based speculative decoding for faster inference, private IPv6 networking for container communication, and robust autoscaling across multiple cloud providers. These primitives support diverse, compute-heavy use cases ranging from computational biology to robotics, providing a flexible substrate for production-grade AI agents that require both high reliability and rapid, elastic scaling.