The Infrastructure for Production AI

In this episode of the podcast, Ben Lorica interviews Zhen Lu, CEO of Runpod, about AI-first cloud computing. Zhen defines the AI cloud as requiring both hardware and software working together to handle compute-bounded workloads and large data shuttling, distinguishing it from traditional Web 2.0 cloud infrastructure. They discuss the advantages of AI-first clouds for production and advanced AI runs, highlighting the importance of low-level hardware access for caching and operationalizing AI. Zhen shares use cases from Runpod's customers, including generative media, fashion try-ons, video walkthroughs, digital cloning, and AI agents for internal workflows, emphasizing the need for control, predictability, and fine-tuning in AI deployments. They also touch on the usability of AMD GPUs, the challenges of reliability in AI infrastructure, and the concept of composability in AI application development.