Meta’s DC Networks for Generative AI - Live from SCC | @Scale

Rohit Puri and Hany Morsy from Meta discuss the evolution of Meta's AI network infrastructure, focusing on the challenges and solutions encountered while scaling up to support generative AI. They detail the transition from single AI zones to multi-building clusters, highlighting the architecture of the 24k and 100k+ GPU clusters, and address issues like network congestion, latency, and traffic management. They also touch on the upcoming Prometheus project, a super cluster spanning a metropolitan area, and the challenges of training models over large distances with heterogeneous hardware.

Outlines

Sign in to continue reading, translating and more.

Continue

Meta’s DC Networks for Generative AI - Live from SCC

@Scale

Introduction to Meta's AI Network Evolution

From 24K to 100K+ Clusters: Network Architecture

100K+ Cluster Topology and Generative AI Support

Challenges and Solutions in Scaling the AI Network

Conclusion and Future Outlook: Project Prometheus

Meta’s DC Networks for Generative AI - Live from SCC

@Scale

00:01Introduction to Meta's AI Network Evolution

Introduction to Meta's AI Network Evolution

03:14From 24K to 100K+ Clusters: Network Architecture

From 24K to 100K+ Clusters: Network Architecture

07:03100K+ Cluster Topology and Generative AI Support

100K+ Cluster Topology and Generative AI Support

11:41Challenges and Solutions in Scaling the AI Network

Challenges and Solutions in Scaling the AI Network

17:51Conclusion and Future Outlook: Project Prometheus

Conclusion and Future Outlook: Project Prometheus