Unknown risk detection in external environment of railroad using UAV images

Common hazards as well as unknown risks (including mudslides, rockfalls, animal intrusion, etc.) in the external environment of railroad seriously threaten the safe operation of railroads, requiring frequent time-consuming and laborious inspections by inspectors, but the scope of inspections is still very limited. At present, the low-altitude economy has become China's new quality productivity representative, and the UAV has innate inspection advantages of high altitude, long distance, and small impact from the terrain and railroad maintenance windows. To overcome the challenge of sparse samples and random uncertainty of unknown risks in the external environment of the railroad, this paper utilizes UAVs for remote sensing image acquisition along the railroad, and proposes an unknown risk detection framework based on Faster R-CNN. Firstly, a novel targeted and multi-classification decoupling training strategy is designed and integrated in the unknown risk detection framework, which significantly improves the performance of general object detection and avoids misclassifying unknown risk objects as background. Secondly, the virtual feature synthesis method of VOS (Visual Object Segmentation) is improved, and similarity-based feature space sampling is designed to obtain a generalized unknown risk object feature representation by performing multivariate Gaussian distribution parameter estimation and resampling based on the construction of instance-level object feature space. Subsequently, an energy-based uncertainty measurement is utilized to measure the uncertainty of instance-level features, and losses are calculated accordingly to induce the network to optimize the decision boundaries for common and unknown risk categories. Finally, quantitative and qualitative experimental analyses are conducted on the collected railroad external environment dataset, open-source drone dataset, and generalization test data. The proposed method achieves 95.7% mAP₅₀ in common hazard identification, while achieving 98% and 80.8% Recall₅₀ in the test set and generalization test data, respectively. The experimental results show that the proposed method has high detection ability for unknown risk objects, while ensuring high recognition rate of common hazard categories.