Efficient collaborative planning for carrier-borne aircraft dispatch and recovery via hierarchical reinforcement learning

Carrier-borne aircraft are the primary formidable assets in aircraft carrier combat, and their sortie rate is a pivotal metric for evaluating the carrier’s combat capability. Enhancing the efficiency of aircraft support operations scheduling is a significant means to improve the sortie rate, where the central issue is to assign multi-wave aircraft to support stations according to the flight plan, and then obtain support resources for completing support operations. The existing studies primarily focus on considering partial operation processes (e.g., ammunition transfer, disturbance handling, support personnel deployment, and deck arrangement), and lack modeling of the entire process of multi-wave aircraft support. In this paper, we investigate the multi-wave Aircraft Dispatch and Recovery Planning (ADRP) problem that aims to reasonably plan the operation processes, stations and resources of multi-wave aircraft with the goal of minimizing the total support operation time, and present a three-layer solution framework based on hierarchical reinforcement learning to address it. Specifically, we first abstract the multi-wave aircraft support operation planning process into the process layer, station layer and resource layer, and model it as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). Then, we propose a three-layer solution framework based on hierarchical reinforcement learning to solve the ADRP problem. To further improve planning results from long-term and global perspective, we design an inter-layer communication mechanism to allow efficient information exchange between stations.