While object tracking has made significantly strides in accuracy, there remains a notable gap in attention toward speed. Most trackers prioritize achieving real-time speed on powerful GPUs, yet practical applications demand higher tracking speeds, particularly on edge platforms. In response to this need, we introduce HCAT, an efficient tracking method designed for high speeds across diverse devices while maintaining superior tracking accuracy. At the core of HCAT lies a hierarchical cross-attention transformer that mitigates the serial nature of the transformer-based tracking model. Additionally, to further enhance the tracker's speed, we propose a feature sparsification module crafted to sparsify template features, consequently reducing the computational load of the model. HCAT demonstrates remarkable speed alongside competitive performance. For instance, it attains 55 fps on the NVIDIA Jetson AGX Xavier edge device, coupled with a 76.6% AUC score on the TrackingNet benchmark. Furthermore, we have developed an enhanced version, HCAT-M, integrating a multi-template framework and an update head. This variant establishes new state-of-the-art performance for high-speed tracking. Code and models are available at https://github.com/chenxin-dlut/HCAT.

In this paper, we first investigate the phenomenon of the spatial-temporal initialization dilemma towards realistic visual tracking, which may adversely affect tracking performance. We summarize the aforementioned phenomenon by comparing differences of the initialization manners in existing tracking benchmarks and in real-world applications. The existing tracking benchmarks provide offline sequences and the expert annotations in the initial frame for trackers. However, in real-world applications, a tracker is often initialized by user annotations or an object detector, which may provide rough and inaccurate initialization. Moreover, annotation from the external feedback also introduces extra time costs while the video stream will not pause for waiting. We select four representative trackers and conduct full performance comparison on popular datasets with simulated initialization to intuitively describe the initialization dilemma of the task. Then, we propose a simple compensation framework to address this dilemma. The framework contains spatial-refine and temporal-chasing modules to mitigate performance degradation caused by the initialization dilemma. Furthermore, the proposed framework can be compatible with various popular trackers without retraining. Extensive experiments verify the effectiveness of our compensation framework.

Visual object tracking has been drawing increasing attention in recent years, as a fundamental task in computer vision. To extend the range of tracking applications, researchers have been introducing information from multiple modalities to handle specific scenes, with promising research prospects for emerging methods and benchmarks. To provide a thorough review of multi-modal tracking, different aspects of multi-modal tracking algorithms are summarized under a unified taxonomy, with specific focus on visible-depth (RGB-D) and visible-thermal (RGB-T) tracking. Subsequently, a detailed description of the related benchmarks and challenges is provided. Extensive experiments were conducted to analyze the effectiveness of trackers on five datasets: PTB, VOT19-RGBD, GTOT, RGBT234, and VOT19-RGBT. Finally, various future directions, including model design and dataset construction, are discussed from different perspectives for further research.