Ep. 017 - DeepSeek V4 and Huawei Ascend NPU Performance (InferenceX) | Kimbo Chen, Cam Quilici, Bryan Shan, Jordan Nanos

DeepSeek V4’s transition to a 1-million context length architecture relies on aggressive innovations in sparse attention and Mega MoE, which reduce KV cache memory requirements by approximately 100x compared to standard models. Achieving day-zero inference performance on new hardware requires complex engineering, specifically the fusion of communication and computation kernels to bypass traditional bottlenecks. While NVIDIA and AMD remain primary targets for optimization, the emergence of Huawei’s Ascend NPU ecosystem highlights a shift toward more diverse hardware support, driven by rapid open-source contributions and sophisticated software toolkits like CANN. The ongoing competition between inference runtimes such as VLLM and SGLang further accelerates these performance gains, forcing continuous iteration and refinement of kernel libraries to maximize throughput and efficiency for large-scale model deployment.