Delving into End-to-End Dual-View Prohibited Item Detection for Security Inspection System

In real-world scenarios, dual-view X-ray machines have outnumbered single-view X-ray machines due to their ability to provide comprehensive internal information about the baggage, which is important for identifying prohibited items that are not visible in one view due to rotation or overlap. However, existing work still focuses mainly on single-view, and the limited dual-view based work only performs simple information fusion at the feature or decision level and lacks effective utilization of the complementary information hidden in dual view. To this end, this paper proposes an end-to-end dual-view prohibited item detection method, the core of which is an adaptive material-aware coordinate-aligned attention module (MACA) and an adaptive adjustment strategy (AAS). Specifically, we observe that in X-ray images, the material information of an object can be represented by color and texture features, and remains consistent across views, even under complex backgrounds. Therefore, our MACA first integrates the material information of the prohibited items in each view and then smoothly transfers these clear material clues along the shared axis to the corresponding locations in the other view to enhance the feature representation of the blurred prohibited items in the other view. In addition, AAS can autonomously adjust the importance of the two views during feature learning to make joint optimization more stable and effective. Experiments on the DvXray dataset demonstrate that the proposed MACA and AAS can be plug-and-played into various detectors, such as Faster Region-based Convolutional Neural Network (Faster R-CNN) and Fully Convolutional One-Stage Object Detector (FCOS), and bring consistent performance gains. The entire framework performs favorably against state-of-the-art methods, especially on small-sized prohibited items, highlighting its potential application in reality.