AARPose: Real-time and accurate drogue pose measurement based on monocular vision for autonomous aerial refueling

Real-time and accurate drogue pose measurement during docking is basic and critical for Autonomous Aerial Refueling (AAR). Vision measurement is the best practicable technique, but its measurement accuracy and robustness are easily affected by limited computing power of airborne equipment, complex aerial scenes and partial occlusion. To address the above challenges, we propose a novel drogue keypoint detection and pose measurement algorithm based on monocular vision, and realize real-time processing on airborne embedded devices. Firstly, a lightweight network is designed with structural re-parameterization to reduce computational cost and improve inference speed. And a sub-pixel level keypoints prediction head and loss functions are adopted to improve keypoint detection accuracy. Secondly, a closed-form solution of drogue pose is computed based on double spatial circles, followed by a nonlinear refinement based on Levenberg–Marquardt optimization. Both virtual simulation and physical simulation experiments have been used to test the proposed method. In the virtual simulation, the mean pixel error of the proposed method is 0.787 pixels, which is significantly superior to that of other methods. In the physical simulation, the mean relative measurement error is 0.788%, and the mean processing time is 13.65 ms on embedded devices.