Lightweight cross-modal state space detection method for UAVs

The task of drone target detection poses substantial challenges for intelligent low-altitude security systems. The interplay of complex and dynamically varying illumination conditions, such as intense glare, shadows, and low-light scenarios, combined with highly intricate and cluttered background textures, including dense urban landscapes, forest canopies, and complex cloud patterns, frequently obstructs the extraction of robust and discriminative feature representations. This inherent difficulty fundamentally constrains the enhancement of detection performance, particularly in demanding real-world operational environments where reliability is of paramount importance. Existing single-modality detection approaches primarily rely on either the visible (RGB) or infrared (IR) spectrum and are often meticulously optimized for specific, constrained scenarios. Consequently, when confronted with the vast diversity and unpredictability of real-world background environments, these approaches exhibit limited adaptability, leading to substantial declines in detection accuracy under complex conditions characterized by camouflage, occlusion, or extreme lighting. Furthermore, while cross-modal methodologies leveraging complementary information from both visible and infrared sensors hold promise for addressing these challenges, numerous current implementations suffer from critical limitations. These include insufficiently deep or effective feature fusion strategies, inadequate feature extraction, and a pronounced susceptibility to inter-modal feature interference. Such interference manifests as feature conflicts or redundancies between modalities, ultimately contributing to unacceptably high rates of false positives (misdetections) and false negatives (missed detections), thereby undermining system reliability.

To comprehensively tackle the intricate and intertwined challenges associated with drone detection, a novel lightweight cross-modal framework, designated as the proposed DF-YOLO (Drones Detection with Features Fusion YOLO), was conceived and developed, grounded firmly in the principles of state space modeling. The core architecture of the proposed DF-YOLO features a meticulously designed dual-branch backbone network. This structure was specifically engineered to perform parallel, modality-specific feature extraction from both visible and infrared inputs, followed by sophisticated cross-modal fusion and subsequent target detection. A pivotal innovation within this framework involved the integration of a dynamic gating mechanism within the fusion pathway. This mechanism functioned adaptively, regulating the information flow and contribution weightings from each modality based on input characteristics, thereby mitigating interference and enhancing the robustness of the detection process. The network synergistically leveraged the complementary strengths of two fundamental operations: convolutional layers, renowned for their efficiency in extracting fine-grained local spatial details and textures, and the incorporated State Space Model (SSM) components. The SSM layers exhibited exceptional capability in modeling long-range dependencies and capturing rich global contextual information across the entire image plane, which is crucial for understanding complex scenes and improving detection accuracy. A specialized module, the Large-kernel Sparse Visual State Space (L-VSS) feature extractor, served as a critical component within the backbone network. The L-VSS module strategically utilized large convolutional kernels combined with sparsification techniques to significantly broaden the receptive field while managing computational overhead. Crucially, it incorporated a spatial adaptation mechanism, enabling the module to dynamically recalibrate its focus towards spatially salient and task-relevant regions within the input imagery. This mechanism substantially amplified the discriminative power of the extracted multi-modal feature representations, further enhancing the detection performance. To address the pervasive challenge of modality discrepancy, a dedicated Mamba Feature Fusion Block (MFFB) module was formulated. Operating within a high-dimensional embedding space, the MFFB module facilitated deep, non-linear interactions and complementary alignment between the features originating from the two distinct modalities. It specifically leverages the global modeling strength of the state space approach to learn complex mappings that minimized modality gaps, effectively enhancing inter-modal feature correlation and maximizing the exploitation of complementary information. This process significantly strengthened the depth and effectiveness of the cross-modal feature fusion, leading to more robust and unified representations that improved the overall performance of UAV detection.

Extensive experimental validations were conducted on both the publicly accessible Anti-UAV drone dataset and a self-constructed multimodal drone dataset. The experimental results demonstrate that, when compared to current mainstream detection methods, the proposed DF-YOLO achieves an mAP50 of 93.7%, an mAP of 55.5%, and surpasses the second-best accuracy method by 3.1% in terms of mAP50 within the dataset. Additionally, the model’s computational complexity has been substantially optimized, with a notable reduction of 336.7 GFLOPs in computational load. When compared to the existing multimodal detection schemes with the lowest computational load, the proposed DF-YOLO further decreases the computational overhead by 33.4 GFLOPs.

The proposed DF-YOLO achieves remarkable improvements in both detection accuracy and model lightweighting. This methodology offers a novel technological solution for the field of low-altitude security.