Reinforcement learning based intelligent fault-tolerant assistance control for air-breathing hypersonic vehicles

This paper proposes a novel reinforcement-learning-based intelligent fault-tolerant assistance control framework for Air-breathing Hypersonic Vehicles (AHVs). Considering that Reinforcement Learning (RL) has the advantage of exploring approximate optimal strategies, an RL-based assistance controller parallel to the fundamental controller is introduced to generate the assistance control signal. Specifically, the Incremental model-based Dual Heuristic Programming (IDHP) method is adopted to design the RL-based assistance control law. In order to extend the IDHP method to the assistance control scenario, a novel linear time-varying incremental model of the closed-loop augmented system is constructed and identified in real time, which consists of the AHV plant, the fundamental controller, and the command generator. The RL agent continuously updates its neural-network weights according to the real-time identification information, and adjusts its control policy, i.e., the assistance control signal, after detecting sudden model changes. Simulation results have validated the effectiveness of the proposed intelligent fault-tolerant control scheme under various types of elevator faults and aerodynamic/configuration parameter uncertainties. The fault-tolerant ability of the whole control system with the proposed RL-based assistance controller is validated in both inner-loop attitude and outer-loop altitude tracking tasks.