Observation Net learned that at this World Artificial Intelligence Conference (WAIC), Huawei will showcase the real machine of Ascend 384 Super Node for the first time in person.
With the explosive growth of computing power requirements for large model training and inference, traditional computing architectures have become difficult to support the generation leap of AI technology. The Huawei Ascend 384 Super Node pioneers the use of a new high-speed network MatrixLink to fully interconnect 384 Ascend NPUs and 192 Kunpeng CPUs in a peer-to-peer manner, forming a super "AI server," with single-card inference throughput rising to 2300 Tokens/s.
Under the connection of high-speed interconnection bus, the Ascend 384 Super Node consists of 12 computing cabinets and 4 bus cabinets, making it the largest-scale super node in the industry so far, with a total computing power scale of 300 Pflops, which is 1.7 times that of NVIDIA NVL72; the total network interconnection bandwidth reaches 269 TB/s, an increase of 107% compared to NVIDIA NVL72; the total memory bandwidth reaches 1229 TB/s, an increase of 113% compared to NVIDIA NVL72. More importantly, through optimal load balancing networking solutions, the Ascend Super Node can further expand into a super node cluster called Atlas 900 SuperCluster containing tens of thousands of cards, and in the future, it can support even larger-scale model evolution.
Moreover, the super node architecture can better support the inference of mixed expert MoE large models, enabling "one card one expert," where a super node can support 384 experts for parallel inference, greatly improving efficiency. At the same time, the super node can also support "one card one computing task," flexibly allocating resources, enhancing task parallel processing, reducing waiting time, and increasing the effective utilization rate of computing power (MFU) by more than 50%.
This article is an exclusive article from Observer, and without permission, it cannot be reprinted.
Original: https://www.toutiao.com/article/7528325005818610228/
Statement: This article represents the personal views of the author. Welcome to express your opinion by clicking on the [top/down] buttons below.