Identification of the phenotypes of fruits is critical for understanding complex genetic traits. Computed tomography (CT) imaging technology enables the noninvasive acquisition of three-dimensional images of fruit interiors, thus providing a robust data foundation for phenotypic analysis. Accurate segmentation of internal fruit tissues is essential, as it directly influences the accuracy and reliability of the results. Current methods are not optimized for the unique features of plant fruit images. This study introduces XFruitSeg, which is a general deep learning model for segmenting plant fruit CT images. The model uses a U-shaped encoder–decoder architecture and integrates multitask learning. A large convolutional kernel network, RepLKNet, expands the receptive field for feature extraction. Multiscale skip connections and a deep supervision mechanism improve the model's capacity to learn features of various sizes, and a contour feature learning branch specifically targets the interorganizational boundaries. An optimized composite loss function enhances the model's robustness when applied to imbalanced categories. Additionally, a dataset named XrayFruitData was established, which contains high-resolution images of twelve plant fruit varieties, with accurate annotations for orange, mangosteen, and durian fruits for model evaluation. Compared with four mainstream advanced models, XFruitSeg achieved superior segmentation performance on the orange, mangosteen, and durian datasets, with mean Dice coefficients of 95.21 %, 93.24 %, and 94.70 % and mean intersection over union (mIoU) scores of 91.09 %, 87.91 %, and 90.35 %, respectively. The results of extensive ablation experiments demonstrate the effectiveness of each component. Therefore, the proposed XFruitSeg model has been proven to be beneficial for high-precision analysis of internal fruit phenotyping traits.