In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance in pre-trained large models and various downstream tasks. However, the enormous computational demands of Transformers have deterred many researchers. To reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the state space model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. This paper provides the first comprehensive review of SSM works and offers experimental comparisons and analyses to better demonstrate SSM’s features and advantages. First, we provide a detailed description of the principles to help the readers quickly grasp the key ideas of the SSM. Then, we will review existing SSMs and their various applications, including those in the fields of natural language processing, computer vision, graphs, multi-modal and multi-media, point cloud and event stream, and other domains. Next, we provide statistical comparisons and analyses of these models to help readers understand how different structures affect performance on various tasks. Finally, we suggest potential research directions that could further advance the development of the theoretical model and propose possible research points in this direction to better promote the application of SSMs.

Source-free domain-adaptive keypoint detection (SFDA-KD) is a method that adapts a keypoint detection model to an unlabeled target domain without accessing the source domain data. In this task, keypoints are spread out across images, and the domain distribution shifts when labels are unavailable. Therefore, it is crucial to explore a broader global feature space and learn more robust features. However, a comprehensive solution that effectively addresses both challenges has yet to be developed. To this end, we propose a method termed dual-rate parameter-efficient tuning with adaptive augmentation (D-PETA). D-PETA consists of two learners with different learning rates based on the low-rank adaptation (LoRA) technique. A fast learner explores global features more quickly and avoids getting stuck in local minima, while a slow learner focuses on local features and stabilizes the training process. The two branches are interdependent, guiding each other to obtain more diverse and robust features. Furthermore, an adaptive augmentation module is introduced, which applies customized augmentations based on the uncertainty of the samples. This improvement leads to enhanced sample utilization and augmented model generalizability. Extensive experiments across diverse benchmarks, including the human body and hand datasets, demonstrate the effectiveness and generalizability of our proposed method.

Existing X-ray image based pre-trained vision models are typically trained on a relatively small-scale dataset (less than 500,000 samples) with limited resolution (e.g., $224\times 224$). However, the key to the success of self-supervised pre-training of large models lies in massive training data, and the maintenance of high-resolution X-ray images contributes to effective solutions for some challenging diseases. In this paper, we proposed a high-resolution ( $1280\times 1280$) X-ray image based pre-trained baseline model on our newly collected large-scale dataset containing more than 1 million X-ray images. Our model employs the masked auto-encoder framework, wherein the tokens that have been processed with a high rate are used as input, and the masked image patches are reconstructed by means of the Transformer encoder-decoder network. More importantly, a novel context-aware masking strategy has been introduced. This strategy utilizes the breast contour as a boundary for adaptive masking operations. We validate the effectiveness of our model through its application in two downstream tasks, namely X-ray report generation and disease detection. Extensive experiments demonstrate that our pre-trained medical baseline model can achieve comparable to, or even exceed, those of current state-of-the-art models on downstream benchmark datasets.