Traffic signal control (TSC) plays a crucial role in enhancing traffic capacity. In recent years, researchers have demonstrated improved performance by utilizing deep reinforcement learning (DRL) for optimizing TSC. However, existing DRL frameworks predominantly rely on manually crafted states, actions, and reward designs, which limit direct information exchange between the DRL agent and the environment. To overcome this challenge, we propose a novel design method that maintains consistency among states, actions, and rewards, named uniformity state-action-reward (USAR) method for TSC. The USAR method relies on: 1) Updating the action selection for the next time step using a formula based on the state perceived by the agent at the current time step, thereby encouraging rapid convergence to the optimal strategy from state perception to action; and 2) integrating the state representation with the reward function design, allowing for precise assessment of the efficacy of past action strategies based on the received feedback rewards. The consistency-preserving design method jointly optimizes the TSC strategy through the updates and feedback among the Markov elements. Furthermore, the method proposed in this paper employs a residual block into the DRL model. It introduces an additional pathway between the input and output layers to transfer feature information, thus promoting the flow of information across different network layers. To assess the effectiveness of our approach, we conducted a series of simulation experiments using the simulation of urban mobility. The USAR method, incorporating a residual block, outperformed other methods and exhibited the best performance in several evaluation metrics.

To improve the driving efficiency of unmanned vehicles in a complex urban traffic flow environment and the safety and passenger comfort of vehicles when changing lanes, we propose a hierarchical reinforcement learning (HRL)-based vehicle trajectory planning and tracking method. First, we present a hierarchical control framework for vehicle trajectory tracking that is based on deep reinforcement learning (DRL) and model predictive control (MPC). We design an upper-level decision model based on the trust region policy optimization algorithm integrated with long short-term memory to obtain more accurate strategies. Second, to improve stability and passenger comfort, we constructed a lower controller that combines the Bezier curve fitting method and an MPC controller. Finally, the proposed method was simulated via the car learning to act (CARLA) simulator, which is based on an unreal engine. Random urban traffic-flow test scenarios were used to simulate a real urban road-traffic environment. The simulation results illustrate that the proposed method can complete the vehicle trajectory planning and tracking task well. Compared with the existing RL methods, our proposed method has the lowest collision rate of 1.5% and achieves an average speed improvement of 7.04%. Moreover, our proposed method has better comfort performance and lower fuel consumption during the driving process.