AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Full Length Article | Open Access

MADRL-based UAV swarm non-cooperative game under incomplete information

Ershen WANGa, Fan LIUa, Chen HONGb( ), Jing GUOa, Lin ZHAOc, Jian XUEc, Ning HEd
School of Electronic and Information Engineering, Shenyang Aerospace University, Shenyang 110136, China
College of Robotics, Beijing Union University, Beijing 100101, China
School of Engineering Science, University of Chinese Academy of Sciences, Beijing 100049, China
College of Smart City, Beijing Union University, Beijing 100101, China
Show Author Information

Abstract

Unmanned Aerial Vehicles (UAVs) play increasing important role in modern battlefield. In this paper, considering the incomplete observation information of individual UAV in complex combat environment, we put forward an UAV swarm non-cooperative game model based on Multi-Agent Deep Reinforcement Learning (MADRL), where the state space and action space are constructed to adapt the real features of UAV swarm air-to-air combat. The multi-agent particle environment is employed to generate an UAV combat scene with continuous observation space. Some recently popular MADRL methods are compared extensively in the UAV swarm non-cooperative game model, the results indicate that the performance of Multi-Agent Soft Actor-Critic (MASAC) is better than that of other MADRL methods by a large margin. UAV swarm employing MASAC can learn more effective policies, and obtain much higher hit rate and win rate. Simulations under different swarm sizes and UAV physical parameters are also performed, which implies that MASAC owns a well generalization effect. Furthermore, the practicability and convergence of MASAC are addressed by investigating the loss value of Q-value networks with respect to individual UAV, the results demonstrate that MASAC is of good practicability and the Nash equilibrium of the UAV swarm non-cooperative game under incomplete information can be reached.

References

【1】
【1】
 
 
Chinese Journal of Aeronautics
Pages 293-306

{{item.num}}

Comments on this article

Go to comment

< Back to all reports

Review Status: {{reviewData.commendedNum}} Commended , {{reviewData.revisionRequiredNum}} Revision Required , {{reviewData.notCommendedNum}} Not Commended Under Peer Review

Review Comment

Close
Close
Cite this article:
WANG E, LIU F, HONG C, et al. MADRL-based UAV swarm non-cooperative game under incomplete information. Chinese Journal of Aeronautics, 2024, 37(6): 293-306. https://doi.org/10.1016/j.cja.2024.03.030

1160

Views

37

Crossref

37

Web of Science

42

Scopus

3

CSCD

Received: 19 June 2023
Revised: 03 August 2023
Accepted: 08 September 2023
Published: 25 March 2024
© 2024 Chinese Society of Aeronautics and Astronautics.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).