Pecan (Carya illinoinensis) is one of the highly valuable nut crops in the global agricultural economy, due to its nutritional and economic advantages. There has ever a rising demand for high-quality cultivars of pecan in recent years. Precise and efficient identification is then required for germplasm resource conservation, breeding programs, and commercial production. However, some challenges still remain in identifying the pecan cultivars, owing to their morphological similarities, and environmental and image acquisition, such as lighting, angle, and distance. Traditional cultivar identification can also rely mainly on manual observation and expert knowledge; These manual approaches cannot fully meet large-scale production, due to the time-consuming susceptibility to subjective errors. It is very necessary to develop automatic, scalable, and accurate recognition of the pecan cultivars. In this study, a scale-interactive knowledge distillation network (siKD-Net) was introduced to enhance the accuracy of classification using deep learning. The discriminative features were then extracted from the pecan images, in order to facilitate the highly precise differentiation of cultivar. 9 048 images were captured from the twelve cultivars of widely cultivated pecan, such as Pawnee. An image dataset was established to support the training and evaluation of the improved model. Among them, the different imaging distances also caused variations in the object scales during data acquisition. The inconsistent representation of features seriously deteriorated the classification of the diverse image samples. Fortunately, the SIKD-Net framework performed better to solve the scale variation and feature inconsistency during image classification. The global-local feature was also incorporated with the scale-aware knowledge distillation. A series of experiments were then conducted to evaluate the performance of the SIKD-Net model. The results indicate that the classification accuracy of 96.98% was achieved in the twelve pecan cultivars, significantly surpassing conventional machine learning and deep learning. A comparison was also made on the baseline models, including the standard convolutional neural networks (CNNs) and transformer-based architectures. Moreover, the scale-interactive knowledge distillation substantially enhanced both the robustness and accuracy of the cultivar recognition. The high accuracy of classification was achieved in the pecan cultivar for agricultural and industrial applications. Automatic recognition can be extended to the rest of tree nuts and fruit crops for precise classification. Furthermore, the improved model can be deployed within smart systems for seedling selection, orchard management, and post-harvesting. Therefore, intelligent sorting can be integrated to enhance efficiency, in order to reduce the reliance on manual inspection in commercial settings. In conclusion, deep-learning identification can be expected to significantly enhance the efficiency and accuracy of pecan cultivar recognition. The hyperspectral imaging and multimodal feature fusion can also be integrated to further enhance the performance and the deployment of the SIKD-Net model for the large-scale classification in real-world orchard environment. The findings can also provide a scientifically robust and practically viable solution to pecan germplasm identification in precision agriculture, intelligent crop classification, and food processing.