Semantic segmentation models based on deep convolutional networks have shown good performance in structural damage detection. However, when it comes to aircraft structural damage detection, cracks usually occupy a small proportion of the image, and the multi-layer convolution and pooling operations can lead to the loss of crack information, thereby seriously reducing the segmentation accuracy. Consequently, this research is conducted on Transformer-based semantic segmentation models, and de-signs a Transformer-based Model for Intelligent Crack Tracking (TICT) for aeronautical structural surface damage detection, aiming to achieve precise segmentation and intelligent tracking of cracks. To start with, an adaptive dynamic patch partitioning mechanism is employed to divide the image into patches of different sizes with varying degrees of overlap. Next, these patches are fed into a Transformer-based encoder to extract multi-scale features containing both the contextual and local details of the crack image. Then, a lightweight multi-layer perceptron along with attention modules is utilized as a decoder to generate a crack mask image. After that, morphological operations are performed on the mask image to correct the connected regions of cracks and map them back to the original image, thus obtaining the exact crack areas. By repeating the aforementioned procedure on the images collected in real time during fatigue tests, automated and continuous tracking of cracks can be realized. The TICT model is trained and tested on datasets of fatigue test images of metal components and entire aircraft. It achieves an Mean Intersection Over Union (mIoU) of 78.31% on the test sets of crack image of various metal components and full-scale aircraft structures, which demonstrates that the TICT model can accurately segment surface cracks in aviation structures with various structural configurations, complex backgrounds, and tiny features, exhibiting good generalization and robustness.