With the rapid development of Unmanned Aerial Vehicle (UAV) technology, target tracking has become one of the key techniques in UAV applications. To address challenges such as occlusion, deformation, scale variation, and multi-view changes in UAV target tracking, this paper proposes a UAV target tracking algorithm based on frequency-domain feature and Transformer architecture. First, a distilled Transformer network is employed to extract global spatial features from images, and an adaptive frequency-domain deep network is employed to capture detailed frequency-domain features. meanwhile, a learning image is introduced at the input stage to capture the correlation between the target template and the search region, thereby updating the initial target template and enhancing target representation. Second, a multi-view invariant feature learning strategy based on mutual information maximization is proposed. By maximizing the mutual information between the target template and the search template, a novel loss function is designed to improve the network's robustness against target appearance variations. Finally, the target position is determined according to the feature responses of the learning image. Simulation results demonstrate that the proposed algorithm effectively improves UAV target tracking accuracy and exhibits strong robustness under complex scenarios.