Abstract
As deep learning models grow in size and complexity, optimizing the execution efficiency becomes critical. Operator fusion merges multiple operators to reduce memory overhead and enhance computational efficiency. However, the combinatorial explosion of possible fusion combinations in modern models creates a search space that challenges traditional manual and heuristic methods. We propose FusionAdvisor, a neural network based framework that predicts the speedups of candidate fusion strategies from computation graphs and recommends the top- candidates (recommended strategies). Trained on our constructed fusion strategy dataset, it abstracts fusion selection as a service, enabling developers to choose high-performance strategies and accelerate inference efficiently. Experimental results demonstrate that FusionAdvisor’s neural network predicts post-fusion speedups with low error, enabling promising strategy recommendations while maintaining search efficiency comparable to the existing methods. This work advances automated operator fusion selection, reduces optimization costs, and improves resource utilization in complex scenarios, including Large Language Models (LLMs) and other large-scale models.
京公网安备11010802044758号
Comments on this article