Research on Cooperative Driving Decisions for Highway On-ramp Merging in Mixed Traffic

In the mixed traffic environment where Connected Autonomous Vehicles (CAVs) and Human Driving Vehicles (HDV) coexist, the Highway On-Ramp Merging Problem presents challenges. The road contention issue involving different types of vehicles usually impacts traffic flow. Vehicles may contend for positions on the road, including merging, lane changing, and other behaviors, leading to the challenges of accurately predicting and adapting to their actions for CAVs. This increases the risk of merging, resulting in decreased traffic efficiency and traffic congestion. Traditional reinforcement learning algorithms have difficulty in effectively searching for optimal strategies in complex environments, and they are prone to getting stuck in local optima. They are unable to effectively deal with complex traffic situations, leading to imprecise merging decisions. To address these challenges, the Evolutionary Soft Actor-Critic for Discrete Action Settings (ESACD) algorithm is proposed. It maximizes the traffic throughput by adaptively coordinating CAVs to HDV strategies. Firstly, a Rank Selection-based Parent Selection and Crossover Method is introduced to model the interaction population. Secondly, a Multiple Populations with Elastic Training method is designed to enhance CAV adaptability to the changes of the dynamic traffic flow. Finally, a Fitness Evaluation-based Secondary Assessment Mechanism is proposed. Simulation experiments conducted under two different traffic densities demonstrate that the proposed algorithm more efficiently completes the merging task at highway on-ramps for connected vehicles with a significant overall improvement rate when compared with the traditional Soft Actor-Critic (SAC) algorithm. This validates the training efficiency of the proposed algorithm with expanding the traffic throughput.