Networking for AI

Sharada Yeluri from Juniper Networks discusses networking for AI, focusing on optimizing AI training and inference workloads using the network. She covers back-end fabric, training workloads, the role of Ethernet switches, network requirements, and the Ultra Ethernet Consortium (UEC). Sharada explains large language models, Gen-AI workloads, traffic patterns in tensor, pipeline, and data parallelism, and the importance of tail latency and lossless transmission. She also explores modular versus standalone systems, network topologies, congestion control, and UEC's efforts to create interoperable protocols. Finally, she touches on LLM inference, GPU scaling, agentic workflows, and the increasing demand for networks in both training and inference, emphasizing Ethernet's dominance and UEC's role in enhancing its capabilities.