Visible-infrared object detection leverages the day-night stable object perception capability of infrared images to enhance detection robustness in low-light environments by fusing the complementary information of visible and infrared images. However, the inherent differences in the imaging mechanisms of visible and infrared modalities make effective cross-modal fusion challenging. Furthermore, constrained by the physical characteristics of sensors and thermal diffusion effects, infrared images generally suffer from blurred object contours and missing details, making it difficult to extract object features effectively. To address these issues, we propose an infrared-visible image fusion network that realizes multimodal information fusion of infrared and visible images through a carefully designed multi-scale fusion strategy. First, we design an adaptive gray-radiance enhancement (AGRE) module to strengthen the detail representation in infrared images, improving their usability in complex lighting scenarios. Next, we introduce a channel-spatial feature interaction (CSFI) module, which achieves efficient complementarity between the RGB and infrared (IR) modalities via dynamic channel switching and a spatial attention mechanism. Finally, we propose a multi-scale enhanced cross-attention fusion (MSECA) module, which optimizes the fusion of multi-level features through dynamic convolution and gating mechanisms and captures long-range complementary relationships of cross-modal features on a global scale, thereby enhancing the expressiveness of the fused features. Experiments on the KAIST, M3FD, and FLIR datasets demonstrate that our method delivers outstanding performance in daytime and nighttime scenarios. On the KAIST dataset, the miss rate drops to 5.99%, and further to 4.26% in night scenes. On the FLIR and M3FD datasets, it achieves $A{P}_{50}$ scores of 79.4% and 88.9%, respectively.

Multi-modal fusion technology gradually become a fundamental task in many fields, such as autonomous driving, smart healthcare, sentiment analysis, and human-computer interaction. It is rapidly becoming the dominant research due to its powerful perception and judgment capabilities. Under complex scenes, multi-modal fusion technology utilizes the complementary characteristics of multiple data streams to fuse different data types and achieve more accurate predictions. However, achieving outstanding performance is challenging because of equipment performance limitations, missing information, and data noise. This paper comprehensively reviews existing methods based on multi-modal fusion techniques and completes a detailed and in-depth analysis. According to the data fusion stage, multi-modal fusion has four primary methods: early fusion, deep fusion, late fusion, and hybrid fusion. The paper surveys the three major multi-modal fusion technologies that can significantly enhance the effect of data fusion and further explore the applications of multi-modal fusion technology in various fields. Finally, it discusses the challenges and explores potential research opportunities. Multi-modal tasks still need intensive study because of data heterogeneity and quality. Preserving complementary information and eliminating redundant information between modalities is critical in multi-modal technology. Invalid data fusion methods may introduce extra noise and lead to worse results. This paper provides a comprehensive and detailed summary in response to these challenges.

Multi-modal 3D object detection has achieved remarkable progress, but it is often limited in practical industrial production because of its high cost and low efficiency. The multi-view camera-based method provides a feasible solution due to its low cost. However, camera data lacks geometric depth, and only using camera data to obtain high accuracy is challenging. This paper proposes a multi-modal Bird-Eye-View (BEV) distillation framework (MMDistill) to make a trade-off between them. MMDistill is a carefully crafted two-stage distillation framework based on teacher and student models for learning cross-modal knowledge and generating multi-modal features. It can improve the performance of unimodal detectors without introducing additional costs during inference. Specifically, our method can effectively solve the cross-gap caused by the heterogeneity between data. Furthermore, we further propose a Light Detection and Ranging (LiDAR)-guided geometric compensation module, which can assist the student model in obtaining effective geometric features and reduce the gap between different modalities. Our proposed method generally requires fewer computational resources and faster inference speed than traditional multi-modal models. This advancement enables multi-modal technology to be applied more widely in practical scenarios. Through experiments, we validate the effectiveness and superiority of MMDistill on the nuScenes dataset, achieving an improvement of 4.1% mean Average Precision (mAP) and 4.6% NuScenes Detection Score (NDS) over the baseline detector. In addition, we also present detailed ablation studies to validate our method.