Together AI & SemiAnalysis: In Conversation Together AI's Vision For the Future of AI Infrastructure

Generative AI necessitates a fundamental shift in cloud infrastructure, moving away from traditional virtualization toward hardware-software co-design. This transition prioritizes specialized kernels—such as FlashAttention—and optimized networking to handle the massive computational demands of modern foundation models. Inference at scale now requires distributing workloads across hundreds of GPUs to maximize efficiency, as seen in the deployment of sparse models like DeepSeek. Beyond infrastructure, the integration of reinforcement learning into training pipelines allows models to learn from synthetic environments, reducing reliance on human-generated data. Together AI’s approach exemplifies this evolution, focusing on deep performance engineering to lower costs and unlock new downstream applications. By aligning GPU architecture with specific neural network designs, companies achieve significant gains in performance and cost-effectiveness, ultimately defining the future of the AI-centric cloud ecosystem.