Runtime 2026 Product Launch Keynote | Erik Bernhardsson, CEO @ Modal

The rapid evolution of generative AI necessitates a fundamental shift in software infrastructure, moving beyond traditional cloud limitations to support high-performance, scalable applications. New infrastructure building blocks are essential for managing the complexities of LLM inference, such as KV caching and dynamic code execution, without relying on configuration-heavy environments. Key advancements include shared and dedicated endpoints for optimized LLM serving, multi-node training clusters with RDMA interconnects for large-scale model development, and managed reinforcement learning frameworks. Furthermore, the introduction of VM sandboxes and sidecar architectures provides the necessary isolation and flexibility for complex agentic workflows. By abstracting these low-level technical challenges, developers can accelerate the deployment of sophisticated AI systems, ranging from robotics and computational biology to automated code generation, ultimately unlocking a new era of software capabilities.