In contrast to the nearly fixed flying altitude of satellite remote sensing platforms, aerial remote sensing (e.g., unmanned aerial vehicles) often employs oblique photography at varying flying altitudes to observe objects from multiple angles and distances in real time. While the existing oriented object detection methods have already demonstrated reliable results in most satellite remote sensing scenarios and achieved high detection precision on large public datasets, such as DOTA-v1.0 and DIOR-R, these methods tend to perform suboptimally on aerial remote sensing images. This performance gap is primarily due to the following two challenges: (A) significant shape variation of objects under multi-view imaging scenarios and (B) substantial object scale variation under multi-distance imaging conditions. To address these issues, we propose the SAA-O²DINO (oriented object detection transformer with improved denoising anchor boxes and shape-adaptive assigner) method for aerial remote sensing in this paper. The proposed method is based on the recently developed AO²DINO framework. It introduces an enhanced Shape-Adaptive Assigner (SAA) that incorporates object shape information into the threshold estimation, allowing for more accurate separation of positive and negative samples, thereby improving the model’s adaptability to significant shape changes across different imaging angles. Additionally, a Gradient Calibration Loss (GCL) is introduced to mitigate the problem of object scale variation. The GCL employs a gradient scaling strategy to reduce scale sensitivity during the optimisation process. We comprehensively compare the proposed method against typical oriented object detection approaches on the DOTA-v1.0 and VSAI datasets. The results show that the proposed method has substantial improvement in detection performance across all datasets, particularly for aerial remote sensing images, validating the generalisation capabilities of our model.