Vision-based air-to-air tracking of multi-UAV swarms is crucial for effective swarm perception. UAVs in a swarm typically share similar appearance features and exhibit nonlinear motion, as they are usually of the same type. This homogeneity poses challenges for the existing multi-object tracking (MOT) algorithms, which often suffer performance degradation due to the difficulties in capturing instance-specific appearance and motion cues. In this paper, we propose a novel multi-frame pose-attention-based appearance feature extraction component that captures instance-level pose features of UAVs across consecutive frames. Additionally, we introduce a motion difference accumulation strategy to extract spatial and motion cues from multiple adjacent frames. By combining these techniques, we design a multi-frame association framework that effectively distinguishes between similar UAVs in a swarm by leveraging object features over consecutive frames. To address the lack of relevant datasets, we create the AIRMOT dataset, specifically tailored for air-to-air tracking of homogeneous UAV swarms. Our method is evaluated on the AIRMOT dataset as well as the publicly available MOT-FLY and UAVSwarm datasets. The experimental results demonstrate that our approach outperforms other state-of-the-art (SOTA) methods, delivering superior tracking performance.

Vision-based air-to-air multi-object tracking is a key technology for UAV situational awareness. Recent research is limited to single-UAV tracking and migration of general multi-object tracking algorithms. To address the problem of inaccurate air-to-air multi-UAVs tracking, a cascade multi-UAVs tracking algorithm based on block feature enhancement extraction and local geometric association is designed. The UAV image is processed in blocks according to the characteristics of the fuselage and the arm, and the UAV’s fine-grained morphological features are extracted. The property that the relative geometric relationship between the UAVs is virtually invariant in the continuous frames is also utilized to extract the local geometric vectors of the UAVs. Then, the cascade association algorithm is designed by synthesizing the above technical components to improve the multi-object tracking algorithm’s ability to retrieve UAV objects and the association success rate, so as to improve the tracking performance of the proposed algorithm. Experiments show that in the test set, the proposed algorithm improves ID F1 Score (IDF1) by 5.6% compared to the state-of-the-art multi-object tracking algorithms OC-SORT, and improves Multiple Object Tracking Accuracy (MOTA) by 2.7% compared to the ByteTrack, which also performs well in the general multi-object tracking. The optimal performance for air-to-air multi-UAVs tracking can be realized, and the components used by the proposed algorithm can also be applied to SORT, BYTE and other data association algorithms to jointly improve their performance.