Multi-modal 3D object detection, which leverages the complementary strengths of LiDAR point clouds and camera RGB images, has emerged as a critical component of 3D perception in autonomous driving. As a critical challenge in multi-modal learning, modality alignment aims to establish accurate semantic correspondences across distinct modalities. However, existing methods encounter significant difficulties in achieving robust alignment when data from one modality is obscured, such as in the presence of object occlusion or adverse environmental conditions, including illumination variations and inclement weather. To alleviate this issue, we present CG-MAE, a dual-branch Bird’s-Eye-View (BEV) masked autoencoder framework based on cross-modal guidance for 3D object detection in autonomous driving. Specifically, a cross-modal guided reconstruction module is developed to predict the representations of the obscure objects in the BEV space, lowering the difficulty of the modality alignment during the multi-modal fusion process. To mimic the object obscuration caused by occlusion or adverse environments, this paper proposes a Ground truth-based foreground masking strategy to cover up the objects, such as vehicles and pedestrians, thereby encouraging the reconstruction module to focus on modality alignment in the foreground regions with high information density. Considering that all the modality data can be obscured, this paper builds a dual-branch BEV masked autoencoder to implement the reconstruction of the data from both the camera and LiDAR modalities. Extensive experiments on the nuScenes dataset with camera-LiDAR inputs demonstrate that the proposed framework achieves superior performance over existing state-of-the-art multi-modal learning methods.

The integrity of perception data transmitted over in-vehicle networks is important for the safety of autonomous driving. However, legacy protocols like the Controller Area Network (CAN) bus which lacks essential security features make In-Vehicle Networks (IVNs) vulnerable to data tampering attacks. Current research typically focuses on detecting the attack itself but ignores the information recovery from the missing data, leading to an unsafe autonomous driving system. To address the issue, we propose a 3D object recovery framework to recover the missing data caused by the tampering attack that occurred in in-vehicle networks. The proposed framework exploits both temporal and spatial context for the 3D object recovery, where a temporal branch is designed to learn the coordinate offsets of 3D objects based on historical data from previous frames, while a spatial branch employs information from the adjacent views of the attacked objects to locate the recovered objects from the overlapped regions in the current frame. By integrating the temporal and spatial clues, the framework effectively recovers the missing objects from the resting ones, thereby enhancing the immunity of in-vehicle networks for the tampering attack. Extensive experiments on the nuScenes dataset demonstrate that the proposed framework significantly improves 3D object detection performance under the attack when compared to the method without recovery. Additionally, the recovery performance becomes better as the attack intensity increases, highlighting the framework’s robustness in high-risk scenarios. The source will be available upon publication.