AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Full Length Article | Open Access

Tube-based robust reinforcement learning for autonomous maneuver decision for UCAVs

Lixin WANGa, Sizhuang ZHENGa, Haiyin PIAOb, Changqian LUb, Ting YUEa, Hailiang LIUa( )
School of Aeronautical Science and Engineering, Beihang University, Beijing 100191, China
Shenyang Aircraft Design & Research Institute, Shenyang 110035, China

Peer review under responsibility of Editorial Committee of CJA.

Show Author Information

Abstract

Reinforcement Learning (RL) algorithms enhance intelligence of air combat Autonomous Maneuver Decision (AMD) policy, but they may underperform in target combat environments with disturbances. To enhance the robustness of the AMD strategy learned by RL, this study proposes a Tube-based Robust RL (TRRL) method. First, this study introduces a tube to describe reachable trajectories under disturbances, formulates a method for calculating tubes based on sum-of-squares programming, and proposes the TRRL algorithm that enhances robustness by utilizing tube size as a quantitative indicator. Second, this study introduces offline techniques for regressing the tube size function and establishing a tube library before policy learning, aiming to eliminate complex online tube solving and reduce the computational burden during training. Furthermore, an analysis of the tube library demonstrates that the mitigated AMD strategy achieves greater robustness, as smaller tube sizes correspond to more cautious actions. This finding highlights that TRRL enhances robustness by promoting a conservative policy. To effectively balance aggressiveness and robustness, the proposed TRRL algorithm introduces a “laziness factor” as a weight of robustness. Finally, combat simulations in an environment with disturbances confirm that the AMD policy learned by the TRRL algorithm exhibits superior air combat performance compared to selected robust RL baselines.

References

【1】
【1】
 
 
Chinese Journal of Aeronautics
Pages 391-405

{{item.num}}

Comments on this article

Go to comment

< Back to all reports

Review Status: {{reviewData.commendedNum}} Commended , {{reviewData.revisionRequiredNum}} Revision Required , {{reviewData.notCommendedNum}} Not Commended Under Peer Review

Review Comment

Close
Close
Cite this article:
WANG L, ZHENG S, PIAO H, et al. Tube-based robust reinforcement learning for autonomous maneuver decision for UCAVs. Chinese Journal of Aeronautics, 2024, 37(7): 391-405. https://doi.org/10.1016/j.cja.2024.03.025

976

Views

5

Crossref

5

Web of Science

5

Scopus

2

CSCD

Received: 20 July 2023
Revised: 15 August 2023
Accepted: 20 September 2023
Published: 20 March 2024
© 2024 Chinese Society of Aeronautics and Astronautics.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).