Experiment of deep reinforcement learning for multi-UAV path planning with graph neural networks and curriculum learning

With the widespread application of unmanned aerial vehicles (UAVs) in disaster rescue, industrial inspection, and other scenarios, multi-UAV path planning in constrained airspace faces dual challenges in avoiding dense static obstacles and improving operational efficiency. Traditional path planning methods based on environmental priors struggle to adapt to dynamically generated scenarios with randomly distributed obstacles. Existing reinforcement learning algorithms predominantly rely on simplified two-dimensional planar assumptions, neglecting three-dimensional (3D) spatial constraints for obstacle avoidance. To address these limitations, this study proposes a collaborative decision-making framework for multi-UAV path planning in 3D static dense obstacle environments by integrating graph neural network (GNN) architecture optimization with progressive curriculum learning (CL).

First, a 3D path planning model is formulated based on the Markov decision process by incorporating altitude dimensions into a state representation and designing a node-type identification mechanism. This design enables UAVs to distinguish heterogeneous characteristics between themselves and surrounding obstacles. To address the limitations of conventional GNNs in spatial relationship modeling, this study couples edge features, including relative velocity, position, and distance, with neighbor node features, such as relative centroid position, velocity, and type identifiers. These features are then fused using multilayer perceptrons to generate joint representations. This approach replaces the linear superposition of independently encoded features commonly used in existing algorithms, thereby enhancing the network’s capability to analyze complex spatial distributions of obstacles. Second, a reward function that balances safety and efficiency is formulated by integrating multidimensional metrics, including target proximity, first-arrival time, dwell duration, velocity alignment, collision risk, and proximity penalties. This design guides UAVs to achieve optimal trade-offs between obstacle avoidance and navigation objectives, thereby improving trajectory rationality and policy convergence speed. Third, a three-stage progressive training framework is developed, transitioning from sparse to dense obstacle scenarios. UAVs initially learn basic obstacle avoidance strategies in simplified environments, then gradually progress to moderate-difficulty environments, and ultimately generate cooperative paths balancing safety and efficiency in complex obstacle configurations. This methodology addresses suboptimal policy issues caused by excessive exploration in high-dimensional environments. Finally, a 3D multi-UAV path planning test environment is established using the PyBullet high-fidelity physics simulation platform, featuring randomly distributed static obstacles with varying density levels.

Experimental results demonstrate that the proposed edge-couple informative multi-agent proximal policy optimization (EC–InforMAPPO) framework outperforms baseline algorithms across all difficulty levels. Its edge feature encoding mechanism, which couples relative motion parameters and spatial relationships, enhances trajectory safety in dense obstacle environments, offering a novel technical pathway for environmental perception modeling in multi-agent systems. Additionally, the progressive curriculum learning framework enhances policy stability in challenging scenarios. The EC–InforMAPPO–CL framework achieves higher obstacle avoidance success rates and faster convergence than direct training under equivalent computational resources. This establishes a reusable training paradigm for reinforcement learning in high-dimensional state spaces.

This study proposes a collaborative decision-making framework that combines edge feature coupling based on GNNs with progressive CL to address challenges in multi-UAV path planning in three-dimensional dense obstacle environments. The findings provide new insights and technical support for intelligent collaborative navigation of multiple UAVs in complex environments, holding significant application potential and practical value.