Simulation experiment on DRL-based highway autonomous driving decision-making guided by safety constraints and curriculum learning

Autonomous driving decision-making in multilane highway environments presents significant challenges in safety assurance and multivehicle coordination, particularly in highly dynamic mixed traffic involving both autonomous and human-driven vehicles. Although deep reinforcement learning has shown strong potential, conventional methods often struggle to balance strict safety constraints with driving efficiency under high-density conditions and may exhibit high collision rates during early exploration. To address these issues, this study develops a safety-aware deep reinforcement learning framework that integrates real-time trajectory prediction with progressive training. The objective is to improve both the safety and operational efficiency in dense highway scenarios while reducing reliance on expert demonstrations and complex reward engineering.

We propose a safety-constrained and curriculum learning–enhanced soft actor–critic framework for discrete action spaces. The framework incorporates a real-time safety shielding module based on established behavioral models, specifically the intelligent driver model for longitudinal control and the MOBIL model for lane-changing decisions. These models enable multistep trajectory predictions for both the ego-vehicle and surrounding traffic over a finite time horizon. A strict safe-distance threshold is continuously enforced to identify potentially unsafe actions before execution. When a violation is detected, the action is replaced with the safest feasible alternative, determined by a real-time distance maximization criterion within the action space. This safety mechanism operates in parallel with the policy network, mitigating collision risks during both training and deployment. Furthermore, a curriculum learning strategy is introduced to improve training stability. The model is trained under progressively increasing traffic densities, from sparse to highly congested scenarios, enabling smooth knowledge transfer and accelerated convergence and preventing overly conservative policies. Comprehensive experiments were conducted using the highway-env simulator under three traffic density settings. Performance was benchmarked against traditional algorithms using rigorous metrics, including task success rates defining collision-free driving for 200 steps, average velocity, and cumulative reward.

Experimental evaluations demonstrate that the proposed safety module substantially reduces collision rates during both training and testing. In low-density scenarios, the safety-constrained variant significantly outperformed both the standard deep Q-network and an unconstrained baseline. With curriculum learning, the full model achieved improved stability and higher success rates in medium- and high-density traffic conditions. Reward trajectories and velocity profiles further confirmed faster convergence and more stable policy behavior. Overall, the proposed framework maintained efficient driving performance while strictly satisfying safety constraints.

The integration of a real-time safety constraint module with a curriculum learning strategy effectively alleviates the safety–efficiency trade-off in high-density highway decision-making. The proposed method demonstrates strong generalization across diverse traffic conditions and provides a practical framework for safe reinforcement learning in autonomous driving. Future work will focus on improving collision prediction accuracy, incorporating richer environmental factors, and extending the framework to more complex real-world driving scenarios.