A multi-aircraft cooperative air combat decision-making algorithm based on an adaptive weighted priority virtual self-play theory approach

This study suggests an approach that combines prioritized fictitious self-play (PFSP) with an adaptive winning-game weighting mechanism to address the delayed convergence and policy oscillation in self-play training for multi-aircraft cooperative beyond-visual-range (BVR) air combat. Firstly, a 4v4 BVR air combat simulation environment is established under a multi-agent reinforcement learning framework. Second, in order to break the policy cycle, the Elo method is developed to conduct non-uniform sampling of past opponents and dynamically assess policy capabilities. Meanwhile, a win-rate-based loss function weighting mechanism is designed to guide gradient updates when early high-value samples are scarce. According to experiments, the suggested approach greatly lessens later-stage oscillation and converges to the Nash equilibrium by the 200th episode, increasing convergence efficiency by 55% over conventional FSP. The weighting mechanism’s accelerating impact is further supported by ablation investigations. Lastly, agents educated using PFSP have strong air combat decision-making ability, winning 70.3% of the time against skilled opponents.