Revolutionizing Parallel Computing: Huawei’s MatMulScan Algorithm Enhances Matrix Operations with Tensor Core Units

Parallel computing is growing rapidly, tackling challenges in deep learning, scientific simulations, and data-heavy tasks. A key operation in this field is matrix multiplication, which is essential for many computations. Recent advancements, such as Tensor Core Units (TCUs), enhance processing by streamlining constant-size matrix multiplications. These units are expanding their use beyond neural networks to include graph algorithms and sorting, which boosts efficiency.

Despite these improvements, prefix sum algorithms—used for calculating cumulative sums—struggle with matrix computations. Traditional methods face issues with computational depth and workload distribution for large datasets. Moreover, beginning matrix operations incurs latency, and tensor core units often have limited parallelism, complicating performance. Methods based on the Parallel Random Access Machine (PRAM) model handle simple binary operations well but do not fully utilize modern tensor hardware in complex matrix situations.

Current techniques for prefix sum calculations include tree-based algorithms like Brent-Kung, which balance efficiency in depth and workload but are not ideal for large-scale matrices. GPU techniques using warp and block-level algorithms work well with smaller data but tend to underutilize tensor cores and increase memory operation overhead on larger sets.

Researchers at Huawei Technologies developed a new algorithm called MatMulScan to tackle these challenges, specifically for the Tensor Core Unit model. This algorithm uses TCUs’ capabilities to efficiently conduct matrix multiplications, lowering computational depth while maximizing throughput. MatMulScan is suitable for applications like gradient boosting trees and parallel sorting, extending traditional algorithms to manage matrices by using lower triangular matrices to represent local prefix sums and perform scalar-vector additions.

MatMulScan operates in two phases: an up-sweep phase and a down-sweep phase. In the up-sweep phase, it computes prefix sums for data subsets. The down-sweep phase spreads these sums across the remaining data, correcting local sums for accuracy. This method optimizes latency and hardware use, making it scalable for large datasets. Analysis indicates that MatMulScan significantly reduces computational depth and performs well in large-scale matrix operations.

The evaluations of MatMulScan highlight several important points for advancing parallel computations:

• Reduced Computational Depth: The algorithm cuts down processing steps significantly for large datasets.
• Enhanced Scalability: It maintains performance as data sizes grow, handling diverse applications effectively.
• Improved Hardware Utilization: By maximizing tensor core capabilities, it overcomes limitations seen in previous approaches.
• Broad Applicability: MatMulScan is promising for various uses beyond prefix sums, such as gradient-boosting tree models and graph algorithms.

In conclusion, MatMulScan represents a significant step forward in parallel scan algorithms. It addresses traditional issues of scalability and computational depth by integrating tensor core technology. This development promises to improve performance in high-performance computing and expands the potential uses of TCUs in computational science.