Publications
Sort:
Issue
Hybrid evolutionary-reinforcement learning architecture for global-local cooperative energy-efficient UAV path planning
Transactions of the Chinese Society of Agricultural Engineering 2026, 42(11): 270-280
Published: 15 June 2026
Abstract PDF (4.2 MB) Collect
Downloads:0

Path planning has been widely used for unmanned aerial vehicles (UAVs) in complex agricultural environments. However, multi-waypoint path planning is often limited to high energy consumption, frequent path turning, and insufficient convergence stability. Furthermore, conventional path planning can focus on minimum path length. In this study, a low-energy path planning was proposed to integrate the Improved Q-learning with Genetic Algorithm (IQL-GA). An energy-sensitive path planning model was established to consider flight distance, turning maneuvers, and obstacle avoidance risks. The UAV task was formulated as a combinatorial optimization under multiple operational constraints, including obstacle safety distance, heading angle limitation, route connectivity, and return-to-start requirements. Geographic coordinates were transformed into planar Cartesian coordinates using the Mercator projection. A two-dimensional UAV operation environment map was constructed for accurate route planning and simulation. The IQL-GA framework is integrated with evolutionary optimization and reinforcement learning using a three-stage cooperative optimization mechanism. (1) An improved multi-strategy Genetic Algorithm (GA) was employed for global exploration and population initialization. A multimodal candidate path set was generated after selection, crossover, and mutation. The various solution spaces were effectively maintained to avoid the local optima. Logistic chaotic mapping enhanced population diversity. While an energy-aware fitness function was combined with path length and turning smoothness to evolve toward low-energy solutions. Sigmoid adaptive crossover and mutation mechanism dynamically adjusted genetic parameters to balance exploration and convergence stability. (2) High-quality non-dominated GA solutions were mapped into the initial Q-table of the Q-learning module after normalized energy-value transformation. The cold start was effectively alleviated to accelerate the convergence. Boltzmann strategy with exponentially decaying temperature coefficients was used to replace the conventional epsilon-greedy strategy, leading to the adaptive transition from global to local exploitation. In addition, an energy penalty term from the energy model was incorporated into the Bellman update equation. The agent was used to simultaneously optimize cumulative reward and energy efficiency. The reward function was also integrated with path efficiency, turning smoothness, and obstacle avoidance safety for the environmental adaptability. (3) A 2-opt local search was introduced to eliminate path crossings and redundant flight segments during global optimization. Nonadjacent path edges were iteratively exchanged to improve route smoothness for the low turning energy consumption in the local search module. GA global exploration, Q-learning adaptive optimization, and 2-opt local refinement were combined into a cooperative global-local optimization architecture to balance search breadth and solution precision. Simulation, experiments, ablation studies, evaluations, and real-world flight tests were conducted to validate the effectiveness of the model. Ablation experiments demonstrate that IQL-GA significantly outperformed conventional GA and standalone Q-learning, thus maintaining the low collision rate and near-optimal path cost. In terms of energy use efficiency, total path energy consumption was reduced by 11.6% and 19.0%, respectively. Comparative experiments were performed on Simulated Annealing (SA), Particle Swarm Optimization (PSO), KMCMSS, and A-PTR-Net. The results show that IQL-GA achieved lower energy consumption, shorter path length, and better optimization stability. The IQL-GA algorithm reduced the average energy consumption by 13.0%, compared with SA, PSO, and two hybrid algorithms. Field experiments were conducted to evaluate engineering applicability using a DJI Mavic 3 UAV equipped with an RTK positioning module. Furthermore, 33 real waypoints were selected for the flight test. Experimental results indicate that the IQL-GA algorithm generated smoother and more compact flight trajectories with fewer redundant turns and shorter hovering durations, with the lowest total energy consumption of 15 988.4 J and the shortest hovering time of 89 s during planning. Engineering practicability was further verified in a field test. This finding can provide a reliable solution and promising application for UAV autonomous navigation under resource-constrained conditions.

Total 1