Real-time small target detection networks for UAV remote sensing

Benefiting from deep learning methods, the performance of object detection methods has greatly improved in recent years. However, significant challenges still exist in detecting targets from UAV remote sensing images. For example, the targets in UAV remote sensing images have small resolution and complex background, and the existing algorithms are difficult to meet the requirement for real-timeliness. To overcome these challenges, this paper proposes a Real-Time Small Target Detection (RTSTD) method based on a Multi-scalar & Multi-depth Feature Extraction (MMFE) network, which can efficiently detect small targets from UAV remote sensing images. The proposed RTSTD crops an input image into multiple small-size images, and feeds a portion of these small-size images into the lightweight MMFE network. Therefore, RTSTD has the capability to handle remote sensing images of arbitrary resolutions without losing image features. A more effective output is proposed for the MMFE network: an overlap vector that represents the position and confidence of the target in the input image. To enhance the MMFE network’s ability to distinguish targets from complex backgrounds, the positive and negative samples are redefined. To test the performance of RTSTD, seven datasets are selected and reconstructed from UAV123, DTB70 and AU-AIR, comprising a total of 8, 369 UAV remote sensing images involving small target detection in the ground and sea scenarios. The experimental results demonstrate that compared to existing detection methods, the RTSTD method achieves improvements in both accuracy and speed. It achieves an F-Score of 0.90 or above, with a running speed of over 66 frames per second (FPS) using GPU acceleration and over 35 FPS using only CPU.