Reinforcement Learning (RL) has emerged as a dominant paradigm for end-to-end autonomous driving (AD) with real-time inference. However, RL typically suffers from sample inefficiency and a lack of semantic interpretability in complex scenarios. To mitigate these limitations, Foundation Models (particularly, Vision-Language Models (VLMs)) can be integrated because they offer rich, context-aware knowledge. Yet still, deploying such computationally intensive models within high-frequency multi-environment RL training loops is severely hindered by prohibitive inference latency and the absence of unified integration platforms. To bridge this gap, we present Found-RL, a specialized platform tailored to leverage foundation models to efficiently enhance RL for AD. A core innovation of the proposed platform is its asynchronous batch inference framework, which decouples heavy VLM reasoning from the simulation loop. This design effectively resolves latency bottlenecks, supporting real-time or near-real-time RL learning from VLM feedback. Using the proposed platform, we introduce diverse supervision mechanisms to address domain-specific challenges: we first implement Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to effectively distill expert-like VLM action suggestions into the RL policy. Furthermore, for dense supervision, we adopt high-throughput CLIP for reward shaping. We mitigate CLIP’s dynamic blindness and probability dilution via Conditional Contrastive Action Alignment, which conditions prompts on discretized speed/command and yields a normalized, margin-based bonus from context-specific action-anchor scoring. Found-RL delivers an end-to-end pipeline for fine-tuned VLM integration with modular support, and shows that a lightweight RL model with millions of parameters can achieve near-VLM performance compared with billion-parameter VLMs while sustaining real-time inference (~500 FPS). Code, data, and models will be publicly available at https://github.com/ys-qu/found-rl.

This study proposes an offline reinforcement learning framework based on Critic Regularized Regression (CRR) to optimize speed guidance at signalized intersections under mixed traffic conditions. Using real-world trajectory data combined with signal phase and timing (SPaT) information, the framework learns safe and efficient driving policies without requiring online interactions. A structured data-processing pipeline converts raw vehicle trajectories into Markov Decision Process (MDP) format, effectively encoding vehicle motion states and signal timing information to facilitate realistic decision-making. SUMO simulation experiments demonstrate substantial improvements over rule-based baselines in safety, comfort, and efficiency: time-to-collision increases from 2.75 to 8.53 seconds, jerk is reduced by over 50%, and time headway is consistently maintained at approximately 1.68 seconds. Trajectory visualizations confirm smoother and more adaptive driving behavior. Comparisons with state-of-the-art approaches including Model Predictive Control (MPC), Behavior Cloning (BC), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Batch Constrained Q-learning (BCQ) highlight CRR's stable performance across multiple evaluation metrics. Ablation studies reveal the critical role of different components in ensuring robust policy behavior, while communication loss simulations demonstrate framework resilience. The method's computational efficiency, requiring considerably less time than MPC, and robustness to communication failures reinforce its practicality for real-time deployment. 

Recent advances in autonomous system simulation platforms have significantly enhanced the safe and scalable testing of driving policies. Although existing simulators have greatly accelerated development by providing controlled testing environments, they face limitations in addressing the evolving needs of future transportation research, particularly in enabling effective human−artificial intelligence (human−AI) collaboration and modeling socially aware driving agents. This study introduces Sky-Drive, a novel distributed multiagent simulation platform that addresses these limitations through four key innovations: (1) a distributed architecture for synchronized simulation across multiple terminals; (2) a multimodal human-in-the-loop framework that integrates diverse sensors to collect rich behavioral data; (3) a human−AI collaboration mechanism that supports continuous and adaptive knowledge exchange; and (4) a digital twin framework for constructing high-fidelity virtual replicas of real-world transportation environments. Sky-Drive supports diverse applications, such as autonomous vehicle-human road user interaction modeling, human-in-the-loop training, socially aware reinforcement learning, personalized driving development, and customized scenario generation. Future extensions will incorporate foundation models for context-aware decision support and hardware-in-the-loop testing for real-world validation. By bridging scenario generation, data collection, algorithm training, and hardware integration, Sky-Drive has the potential to become a foundational platform for the next generation of human-centered and socially aware autonomous transportation system research.

Vehicle trajectory prediction is critical for advancing autonomous driving and advanced driver assistance systems (ADASs). Deep learning-based approaches, especially those using transformer-based and generative models, have significantly improved prediction accuracy by capturing complex, non-linear patterns in vehicle dynamics and traffic interactions. However, they often overlook detailed car-following behaviors and the inter-vehicle interactions essential for real-world driving, particularly in fully autonomous or mixed traffic scenarios. Moreover, existing generative approaches in trajectory prediction are inefficient at conditioning predictions on relevant constraints. To address these issues, this study proposes FollowGen, a novel scaled noise conditional diffusion model for car-following trajectory prediction. FollowGen incorporates detailed inter-vehicular interactions and car-following dynamics within a generative framework, enhancing both the accuracy and realism of the predicted trajectories. The model uses a novel pipeline to capture historical vehicle behaviors. It leverages a noise scaling conditioning strategy to scale the noise with encoded historical features within the forward diffusion process to ensure history-constrained noise transformation. A cross-attention-based transformer architecture is employed in the reverse process to model intricate inter-vehicle dependencies, effectively guiding the denoising process and enhancing prediction accuracy. Experimental results in various real-world driving scenarios demonstrate the state-of-the-art performance and robustness of the proposed method.

Semantic scene completion (SSC) plays a pivotal role in achieving comprehensive perceptions of autonomous driving systems. However, existing methods often neglect the high deployment costs of SSC in real-world applications, and traditional architectures such as three-dimensional (3D) convolutional neural networks (3D CNNs) and self-attention mechanisms struggle to efficiently capture long-range dependencies within 3D voxel grids, limiting their effectiveness. To address these challenges, we propose MetaSSC, a novel meta-learning-based framework for SSC that leverages deformable convolution, large-kernel attention, and the Mamba (D-LKA-M) model. Our approach begins with a voxel-based semantic segmentation (SS) pretraining task, which is designed to explore the semantics and geometry of incomplete regions while acquiring transferable meta-knowledge. Using simulated cooperative perception datasets, we supervise the training of a single vehicle's perception via the aggregated sensor data from multiple nearby connected autonomous vehicles (CAVs), generating richer and more comprehensive labels. This meta-knowledge is then adapted to the target domain through a dual-phase training strategy—without adding extra model parameters—ensuring efficient deployment. To further enhance the model's ability to capture long-sequence relationships in 3D voxel grids, we integrate Mamba blocks with deformable convolution and large-kernel attention into the backbone network. Extensive experiments show that MetaSSC achieves state-of-the-art performance, surpassing competing models by a significant margin while also reducing deployment costs.

Real-time vehicle prediction is crucial in autonomous driving technology, as it allows adjustments to be made in advance to the driver or the vehicle, enabling them to take smoother driving actions to avoid potential collisions. This study proposes a physics-enhanced residual learning (PERL)-based predictive control method to mitigate traffic oscillation in the mixed traffic environment of connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). The introduced model includes a prediction model and a CAV controller. The prediction model is responsible for forecasting the future behavior of the preceding vehicle on the basis of the behavior of preceding vehicles. This PERL model combines physical information (i.e., traffic wave properties) with data-driven features extracted from deep learning techniques, thereby precisely predicting the behavior of the preceding vehicle, especially speed fluctuations, to allow sufficient time for the vehicle/driver to respond to these speed fluctuations. For the CAV controller, we employ a model predictive control (MPC) model that considers the dynamics of the CAV and its following vehicles, improving safety and comfort for the entire platoon. The proposed model is applied to an autonomous driving vehicle through vehicle-in-the-loop (ViL) and compared with real driving data and three benchmark models. The experimental results validate the proposed method in terms of damping traffic oscillation and enhancing the safety and fuel efficiency of the CAV and the following vehicles in mixed traffic in the presence of uncertain human-driven vehicle dynamics and actuator lag.

Model-based reinforcement learning (RL) is anticipated to exhibit higher sample efficiency than model-free RL by utilizing a virtual environment model. However, obtaining sufficiently accurate representations of environmental dynamics is challenging because of uncertainties in complex systems and environments. An inaccurate environment model may degrade the sample efficiency and performance of model-based RL. Furthermore, while model-based RL can improve sample efficiency, it often still requires substantial training time to learn from scratch, potentially limiting its advantages over model-free approaches. To address these challenges, this paper introduces a knowledge-informed model-based residual reinforcement learning framework aimed at enhancing learning efficiency by infusing established expert knowledge into the learning process and avoiding the issue of beginning from zero. Our approach integrates traffic expert knowledge into a virtual environment model, employing the intelligent driver model (IDM) for basic dynamics and neural networks for residual dynamics, thus ensuring adaptability to complex scenarios. We propose a novel strategy that combines traditional control methods with residual RL, facilitating efficient learning and policy optimization without the need to learn from scratch. The proposed approach is applied to connected automated vehicle (CAV) trajectory control tasks for the dissipation of stop-and-go waves in mixed traffic flows. The experimental results demonstrate that our proposed approach enables the CAV agent to achieve superior performance in trajectory control compared with the baseline agents in terms of sample efficiency, traffic flow smoothness and traffic mobility.

Trajectory prediction for heterogeneous traffic agents plays a crucial role in ensuring the safety and efficiency of automated driving in highly interactive traffic environments. Numerous studies in this area have focused on physics-based approaches because they can clearly interpret the dynamic evolution of trajectories. However, physics-based methods often suffer from limited accuracy. Recent learning-based methods have demonstrated better performance, but they cannot be fully trusted due to the insufficient incorporation of physical constraints. To mitigate the limitations of purely physics-based and learning-based approaches, this study proposes a kinematics-aware multigraph attention network (KA-MGAT) that incorporates physics models into a deep learning framework to improve the learning process of neural networks. Besides, we propose a residual prediction module to further refine the trajectory predictions and address the limitations arising from simplified assumptions in kinematic models. We evaluate our proposed model through experiments on two challenging trajectory datasets, namely, ApolloScape and NGSIM. Our findings from the experiments demonstrate that our model outperforms various kinematics-agnostic models with respect to prediction accuracy and learning efficiency.

Despite significant progress in autonomous vehicles (AVs), the development of driving policies that ensure both the safety of AVs and traffic flow efficiency has not yet been fully explored. In this paper, we propose an enhanced human-in-the-loop reinforcement learning method, termed the Human as AI mentor-based deep reinforcement learning (HAIM-DRL) framework, which facilitates safe and efficient autonomous driving in mixed traffic platoon. Drawing inspiration from the human learning process, we first introduce an innovative learning paradigm that effectively injects human intelligence into AI, termed Human as AI mentor (HAIM). In this paradigm, the human expert serves as a mentor to the AI agent. While allowing the agent to sufficiently explore uncertain environments, the human expert can take control in dangerous situations and demonstrate correct actions to avoid potential accidents. On the other hand, the agent could be guided to minimize traffic flow disturbance, thereby optimizing traffic flow efficiency. In detail, HAIM-DRL leverages data collected from free exploration and partial human demonstrations as its two training sources. Remarkably, we circumvent the intricate process of manually designing reward functions; instead, we directly derive proxy state-action values from partial human demonstrations to guide the agents’ policy learning. Additionally, we employ a minimal intervention technique to reduce the human mentor’s cognitive load. Comparative results show that HAIM-DRL outperforms traditional methods in driving safety, sampling efficiency, mitigation of traffic flow disturbance, and generalizability to unseen traffic scenarios.