A unified framework for small object analysis in UAV imagery: Tracking single target from multiple detected targets

A framework for visual small target detection and tracking is introduced, leveraging Unmanned Aerial Vehicle (UAV) Remote Sensing Images (RSIs). The proposed Cropped Target Detection and Tracking (CTDT) framework comprises two integral stages: the detection stage and the tracking stage. During the detection stage, all targets can be identified from RSIs, providing a basis for the subsequent single-object tracking stage. Both stages are based on a cropping and random sampling strategy: the RSI is cropped into Small-Sized Images (SSIs), from which a random batch is constantly selected without repetition and fed into a network to locate the target until the target is discovered or all SSIs are used. This strategy improves the efficiency of detection and tracking. After cropping, the target may appear in multiple SSIs, and the target in each SSI may be incomplete. A Cropped Target Feature Extraction (CTFE) network is designed to detect and track the target by leveraging the information from small and incomplete targets in SSIs. CTFE achieves high precision and meets real-time requirements. The performance analysis of the detection network is also conducted in detail, and the results are instrumental in informing the design of the tracking network. By utilizing three UAV RSI datasets (UAVDT, UAV123, and DTB70), CTDT is compared to numerous state-of-the-art mainstream methods, such as PVT++, SiamBAN, SmallTrack, SiamAPN++, SiamIRCA, SiamFC, and CSK, to confirm its superiority and real-time performance. The results affirm that the proposed framework exhibits outstanding performance and adaptability to fast-moving targets, target loss, and camera failures, and holds promise for real-time applications. Additionally, real-world tests on a typical UAV platform demonstrate excellent performance and efficiency in a variety of UAV-specific tasks, as well as transferability for new missions.