A PPO-Based DRL Approach for Scalable Communication in Civilian UAV Networks

Nowadays, Unmanned Aerial Vehicles (UAVs) are making increasingly important contributions to numerous applications that enhance human quality of life, such as sensing and data collection, computing, and communication. However, communication between UAVs still faces challenges due to high-dynamic topology, volatile wireless links, and strict energy budgets. In this work, we introduce an improved communication scheme, namely Proximal Policy Optimization (PPO). Our solution casts hop–by–hop relay selection as a Markov decision process and develops a decentralized Proximal Policy Optimization framework in an actor–critic form. A key novelty is the design of the reward function, which jointly considers the delivery ratio, end-to-end delay, and energy efficiency, enabling flexible prioritization in dynamic environments. The simulation results across swarms of 20–70 UAVs show that, the proposed framework enhances delivery ratio to 5% over a Deep Q-Network baseline (reaching $\approx 80\mathrm{\%}$ at 70 nodes), reduces latency by about 2–3 ms in medium-to-dense settings (from $\sim 43$ to 35–36 ms), and attains comparable or slightly lower total energy consumption (typically 0.5%–2% lower). The results indicate that the proposed communication scheme, adaptive and scalable learning-based UAV scenarios, pave the way for re-world UAV deployments.