Reinforcement Learning (RL) algorithms enhance intelligence of air combat Autonomous Maneuver Decision (AMD) policy, but they may underperform in target combat environments with disturbances. To enhance the robustness of the AMD strategy learned by RL, this study proposes a Tube-based Robust RL (TRRL) method. First, this study introduces a tube to describe reachable trajectories under disturbances, formulates a method for calculating tubes based on sum-of-squares programming, and proposes the TRRL algorithm that enhances robustness by utilizing tube size as a quantitative indicator. Second, this study introduces offline techniques for regressing the tube size function and establishing a tube library before policy learning, aiming to eliminate complex online tube solving and reduce the computational burden during training. Furthermore, an analysis of the tube library demonstrates that the mitigated AMD strategy achieves greater robustness, as smaller tube sizes correspond to more cautious actions. This finding highlights that TRRL enhances robustness by promoting a conservative policy. To effectively balance aggressiveness and robustness, the proposed TRRL algorithm introduces a “laziness factor” as a weight of robustness. Finally, combat simulations in an environment with disturbances confirm that the AMD policy learned by the TRRL algorithm exhibits superior air combat performance compared to selected robust RL baselines.
- Article type
- Year
Open Access
Full Length Article
Issue
Open Access
Full Length Article
Issue
There is a lack of quantitative flying qualities assessment requirements for unmanned combat aerial vehicles. The mission-oriented flying qualities evaluation approach can make up for the deficiencies of existing flying qualities specifications. Considering the control characteristics and mission requirements of autonomous control unmanned combat aerial vehicles, flying qualities assessment tasks are designed and performance standards are developed on the basis of manned aircraft flying qualities assessment tasks. Multiple sets of mathematical simulations are performed by varying the control law parameters to investigate the relationship between the control law parameter values, closed-loop aircraft system characteristics and flying qualities levels. The simulation results and closed-loop frequency domain analysis show that the existing flying qualities guidelines for manned aircraft are not fully applicable to the flying qualities assessment of autonomous control unmanned combat aerial vehicles. It is found that the combination of the bandwidth and the phase at the bandwidth frequency can define the flying qualities requirements of autonomous control unmanned aerial vehicles. The criterion boundaries of different levels are given, and the physical reasons for the formation of boundary are analysed. Our results can be applied to flying qualities assessment and design of flight control laws for autonomous control unmanned combat aerial vehicles.
京公网安备11010802044758号