AI Hardware Reliability at Scale | Sriram Sankar & Harish Dixit

Sriram Sankar and Harish Dixit discuss the importance of AI hardware reliability at Meta's scale, focusing on the challenges posed by hardware failures in large AI clusters. They highlight the increasing rate of hardware failures with larger clusters and delve into different types of errors, with a particular emphasis on silent data corruptions (SDCs). Harish explains the characteristics of SDCs, their dependency on data randomization, electrical properties, lifecycle variations, and the approaches Meta has developed to detect and mitigate these errors, including pit stops, Ripple, and hardware sentiment. They also explore how SDCs affect AI workloads, both in training and inference, and the measures taken to address these issues. Sriram concludes by emphasizing the need for an industry-wide, factory-to-fleet approach to improve reliability across the entire lifecycle, from silicon design to application, and mentions Meta's work on their custom silicon, MTIA, to deliver industry-leading reliability.