Accurately identifying the ripeness of colorful tomato fruits is often required for efficient classification and harvesting by robots. This study aims to detect the ripeness of the colorful tomatoes with diverse colors under complex backgrounds. According to the multi-granularity theory, a "segment first, then detect" approach was proposed to detect the fine-grained colorful tomatoes using U-YOLOv8n. Firstly, the pre-trained weights were incorporated into the U-Net model after training on the VOC2007 public dataset. A U-Net model was used to segment the colorful tomato clusters. The VGG16 was selected as the backbone network for the feature extraction. The network structure and hierarchical feature extraction were obtained to handle the more complex and diverse image data. The global attention mechanism (GAM) was introduced to process the image information in channel and spatial dimensions, particularly for the performance of the segmentation. The region of interest (ROI) was segmented to reduce the interference from the complex background on the second-stage detection task. Secondly, the Bottleneck layer in the C2f module of the YOLOv8n was replaced with the MS-Block module. As such, the C2f_MS module was constructed to extract the multi-scale features. Finally, some Conv layers were replaced with the SCDown module, in order to reduce the computational redundancy on the spatial information. The small object detection head and part of the Neck structure were removed to reduce the computational load for the overall accuracy and efficiency optimization. The results indicated that the MPA, MIoU, and Dice increased by 1.13, 1.37, and 0.67 percentage points, respectively, in the U-Net-related part using the pre-trained weights. The mean pixel accuracy (MPA), mean intersection over union (MIoU), and Dice increased by 0.21, 0.6, and 0.36 percentage points, respectively, after VGG16 as the backbone network of the U-Net model. A deep architecture was utilized to more effectively capture the multi-scale and high-level semantic features. The GAM attention module was introduced into the backbone network. The segmentation performance of the U-Net was enhanced after the improvements. The MPA, MIoU, and Dice of the U-Net model reached 94.35%, 88.98% and 94.39%, respectively, indicating the improvements of 1.55, 2.38, and 1.31 percentage points over the original. In YOLOv8n, the C2f_MS module improved the precision (P), recall (R), and mean average precision (mAP0.5) by 1.5, 0.3, and 0.4 percentage points, indicating the stronger feature extraction on the tomatoes of different sizes in the same image. Some Conv layers were replaced with the SCDown module. The parameters and the weight size were reduced by 0.7 and 2.21 M, respectively, compared with the original. A more lightweight model was obtained to remove the small object detection head and some structures. The number of floating-point operations (FLOPs) was reduced by 2.5 G, whereas the precision and mean average precision increased by 1.6 and 0.3 percentage points, respectively. An overall improvement was achieved in both precision and efficiency. The accuracy and mean average precision of the YOLOv8n reached 93.5% and 95.6%, respectively, indicating an increase of 3.1 and 0.5 percentage points over the original. In the harder to recognize half-ripe stage, the accuracy and mean average precision reached 91.8% and 92.4%, respectively, indicating the increases of 5.7 and 1.3 percentage points. In terms of the lightweight performance, the floating-point operations and parameter volume were reduced by 2.5 G and 0.43 M, respectively, compared with the YOLOv8n. The frames per second (FPS) also increased by 41%. The ripeness detection of the colorful tomatoes was effectively realized in complex backgrounds. The finding can provide technical support to the decision-making on the ripeness grading and harvesting.

To address the difficulties in manual counting for the uneven distribution and severe occlusion of watermelons in natural environments, this study utilizes drones and smartphones to collect videos and images, combined with manual annotation to establish a dataset for Sanbai melons and Ningxia selenium sand melons. A watermelon video automatic counting method based on the YOLOv7-GCSF model and an improved DeepSORT algorithm is proposed. The lightweight YOLOv7 model with GhostConv is enhanced with GBS modules, G-ELAN modules, and G-SPPCSPC modules to increase the model’s detection speed. Some ELAN modules are replaced with the C2f module from YOLOv8 to reduce redundant information. The SimAM attention mechanism is introduced into the MP module of the feature fusion layer to construct the MP-SimAM module, which is used to enhance the model's feature extraction capability. The CIoU loss function is replaced with the faster-converging, lower-loss Focal EIoU loss function to increase the model's convergence speed. In video tracking and counting, a mask collision line mechanism is proposed for more accurate counting of Sanbai melons and Ningxia selenium sand melons. The results show that in terms of object detection: the four improvements to the YOLOv7-GCSF model have all enhanced the model’s performance to some extent. Specifically, compared to the YOLOv7 model, the construction of the MP-SimAM module increased accuracy by 1.5 percentage points, indicating a greater focus on Sanbai melons and Ningxia selenium sand melons. The addition of GhostConv reduced the model size by 28.1MB, demonstrating that the construction of GBS, G-ELAN, and G-SPPCSPC modules effectively reduced the model size and improved detection speed. The incorporation of the C2f module reduced the model's floating-point operations (FLOPs) by 77.5 billion, indicating that the model has eliminated most of the redundant information. The addition of the Focal EIoU loss function significantly increased the model’s convergence speed, indicating further enhancement of the model's learning ability. The improved YOLOv7-GCSF model achieved an accuracy (P) of 94.2% and a mean average precision (mAP_0.5) of 98.2%, which is 5.0, 2.3, 21.9, and 14.9 percentage points higher in accuracy and 3.7, 0.3, 4.6, and 9.3 percentage points higher in mean average precision compared to YOLOv5, YOLOv7, Faster RCNN, and SSD, respectively. In terms of model lightweighting, the YOLOv7-GCSF model has seen a decrease of 1.18M and 0.11M in the number of parameters compared to the YOLOv4-Ghostnet and YOLOv7-Slimneck models, respectively. Compared to the original YOLOv7, the YOLOv7-GCSF model has reduced the parameter count and model size by 0.57M and 18.88MB, respectively. In terms of object tracking: the improved DeepSORT multi-object tracking accuracy is 91.2%, and the multi-object tracking precision is 89.6%, which is 5.0 and 13.7 percentage points higher in tracking accuracy and 3.7 and 13.1 percentage points higher in tracking precision compared to Tracktor and SORT, respectively. Comparing the improved model with manual counting results, the determination coefficient for the counting results of Sanbai melons and Ningxia selenium sand melons is 0.93, the average counting accuracy is 96.3%, and the average absolute error is 0.77, indicating that the error between the improved model and manual counting is small. This approach, by enabling effective counting of watermelons in agricultural fields, provides a technical methodology for the forecasting of watermelon yields.