Sort:
Issue
Dense spruce counting based on UAV aerial images and improved IntegrateNet
Transactions of the Chinese Society of Agricultural Engineering 2023, 39(18): 74-81
Published: 30 September 2023
Abstract PDF (1.2 MB) Collect
Downloads:1

The number of trees is essential information for modern forestry management, which affects managers' formulation of development strategies. At present, tree inventory mainly relies on manual counting, which is costly, time-consuming and labor-intensive. Using unmanned aerial vehicle (UAV) to count targets has become increasingly popular in agriculture and forestry due to its low cost, ease of operation, and flexibility of use. This study focused on using UAV images to count spruce numbers. A total of 603 images with an average canopy density of 81% and an average plant density of 6667 plants/hm2 were selected for the dataset, each containing an average of 354 spruce trees. Among these, 205 images had interference factors such as weeds. The images were divided into a training set and a test set at a ratio of 7:3. The training set was expanded by randomly flipping the images to improve the robustness to different flight attitudes of UAV. After data augmentation, 844 training images and 181 test images were obtained. Aiming at the problem of dense spruces in natural environments including severe adhesion and background interference such as weeds with similar characteristics to spruces, we selected IntegrateNet, a model known for its strong performance in dense target counting tasks as baseline model for spruces. This study then worked to improve the IntegrateNet model to achieve a more accurate counting of dense spruce that is closer to real-world conditions. First of all, this paper used the self-calibrated convolutions (SCConv) to replace the ordinary convolutional layers at 1/8, 1/16 feature maps and density maps in the baseline model to expand the convolution receptive field. Secondly, in order to deal with background interference problems such as weeds and the serious adhesion of the target, this paper added the criss-cross attention mechanism (CCA) to the feature fusion of the IntegrateNet model. It can consider the horizontal and vertical context information of each pixel to generate richer semantic features to improve the contextual information extraction ability of the model. The mean counting accuracy(MCA), mean absolute error(MAE), root mean square error(RMSE), and the coefficient of determination R2 are used as evaluation indicators. The ablation experiments and comparative experiments are designed to verify the performance of the proposed model. The ablation experiments show that the improved methods proposed in this paper can effectively improve the counting accuracy of the model. The comparative experiments show that the improved IntegrateNet model proposed in this study has MCA, MAE and RMSE reached 98.32%, 8.99 plants and 13.79 plants respectively, and the R2 was 0.99.Compared with TasselNetv3_lite, TasselNetv3_seg, and IntegrateNet models , the MCA of the improved IntegrateNet model increased by 16.44, 10.55, and 9.26 percentage points, the MAE of the improved IntegrateNet model decreased by 25.62, 10.45, and 6.99 plants, and the RMSE of the improved IntegrateNet model decreased by 48.25, 13.84, and 12.52 plants. In summary, the improved IntegrateNet model proposed in this study demonstrated a significant increase in accuracy for counting dense spruce in natural environments. Moreover, our research is not limited to spruce, as the methodology can be applied to other tree species as well. It can providing technical support for subsequent forestry intelligent statistics.

Issue
Identifying apple leaf diseases using improved EfficientNet
Transactions of the Chinese Society of Agricultural Engineering 2023, 39(18): 201-210
Published: 30 September 2023
Abstract PDF (2.3 MB) Collect
Downloads:2

The traditional manual control measures for apple disease primarily rely on subjective experience, which can lead to arbitrariness and bias. This approach is also prone to pesticide waste and misuse. As apple leaves are one of the high-incidence areas of diseases, achieving natural environment-based identification of apple leaf diseases can provide effective guidance for disease prevention and control. To address the challenge of accurate identification of apple leaf diseases in natural settings, a DenseNet121+EfficientNet model with Focal Loss and Label Smoothing (DEFL) is designed. Given that apple leaf disease lesions are small and exhibit varying spatial characteristics for different diseases, accurately localizing and identifying disease regions is challenging. The DEFL model utilizes two parallel networks: EfficientNet-B0 and DenseNet121 for feature extraction. It combines the semantic and positional information extracted from these networks to enhance the model's fine-grained feature extraction capability. To overcome difficulties in identifying apple leaf samples due to the similarity in features among different diseases, a Focal Loss function, combined with label smoothing strategies, is introduced. This prevents the model from being overly confident about the class of apple leaf samples, focusing on those samples with challenging decision boundaries during training, thereby improving the model's ability to recognize boundary cases and enhancing its robustness and generalization. The DEFL model proposed in this study was tested with 3906 apple leaf disease images in natural scenes. The results show that the overall recognition accuracy of the model proposed in this study is 99.13%, and the mean average accuracy is 98.47%. The results of ablation experiments show that the mean average precision of the model is improved by 7.99% after adding DenseNet121 feature extraction branch to EfficientNet-B0. And, after introducing the focus loss function combined with label smoothing strategy to EfficientNet-B0, the mean average precision of the model is improved by 3.15%. After the combination of the two improvements, the mean average precision of the model increased by 12.29%. Visualization results using Gradient-weighted class activation mapping (Grad-CAM) heatmaps show that the model accurately localizes and focuses on disease regions. The image feature information used by the model for apple leaf disease recognition is reliable. Uniform manifold approximation and projection (UMAP) feature dimension reduction visualization results indicate that the Focal Loss function with label smoothing effectively enhances the model's feature extraction capability, resulting in more distinctive feature information and improved disease recognition. The comparative experimental results show that compared with mainstream recognition models such as ResNet50, Inception V3, and ResNeXt, as well as EfficientNet-B0 models that fuse these three models, the mean average precision of the proposed model has increased by 14.53, 13.17, 14.61, 6.4, 7.71, and 8.91 percentage points, respectively, and the model size has decreased by 18.73, 7.7, 12.2, 83.62, 69.6, and 60.09 MB, respectively. Comparative experiments have shown that the proposed model has the best recognition performance. In practical applications, the DEFL model achieved an overall recognition accuracy of 97.73% and a mean average precision of 95.82% when tested with 1501 apple leaf images collected from apple plantations. In conclusion, the DEFL model proposed in this study enables accurate and rapid identification of apple leaf diseases, providing valuable guidance for apple disease prevention and control.

Issue
Identification of olive cultivars using bilinear networks and attention mechanisms
Transactions of the Chinese Society of Agricultural Engineering 2023, 39(10): 183-192
Published: 30 May 2023
Abstract PDF (1.8 MB) Collect
Downloads:0

The extensive range of olive cultivars available in the market exhibit minor differences in their phenotypic traits. Nonetheless, their quality attributes, particularly the oil content and fatty acid composition, significantly differ among distinct cultivars, resulting in the emergence of use iinferior products as superior products in the market. The accurate and quick identification of olive cultivars holds significant importance in enhancing the production and quality of olives. As such, delving into the study of olive cultivar identification is crucial for the advancement of the olive industry. This study presents a novel approach to address the challenge of identifying olive cultivars in natural conditions. Specifically, a bilinear attentional EfficientNet model is proposed, which incorporates the bilinear network design concept and attention mechanism. The model is trained and evaluated using four commonly planted olive cultivars (i.e., Frantoio, Leccino, Picholine, and Ezhi 8) in Longnan, Gansu. The experimental results demonstrate the effectiveness of the proposed model in accurately and quickly identifying different olive cultivars. A bilinear network has been suggested to comprehensively extract feature information from olive images, for the limited phenotypic differences across different olive cultivars. In light of this, the selection of a feature extraction network has been made with consideration for both speed and accuracy, leading to the selection of the EfficientNet-B0 network. To tackle the challenge of identifying olive cultivars under natural conditions which are prone to intricate background interferences, a novel approach combining convolutional block attention module (CBAM) with bilinear network has been proposed. This approach facilitates the model in selectively focusing on the salient features responsible for cultivar identification during the feature extraction process of olive images. Upon conducting experiments, the bilinear attention EfficientNet model presented in this study has exhibited an overall accuracy of 90.28% and an inference time of 9.15 ms in identifying four distinct olive cultivars. These experiment results demonstrate that the proposed model better achieved better rapaid and accurate identification of olive cultivars under natural conditions. The present study proposes a bilinear attention EfficientNet model for identifying olive cultivars and utilizes gradient-weighted class activation mapping (Grad-CAM) to analyse its performance. The results demonstrate that the proposed model exhibits a greater attention towards fruit regions, as well as some leaf regions, within olive images. These findings are in agreement with the expert knowledge and experience of human practitioners. The analysis of heat maps generated from misidentified olive images revealed that inadequate focus on the fruit and leaf regions, which are pivotal for successful identification of cultivars, was the primary contributing factor to misidentification. The outcomes of the ablation experiments indicated that the bilinear network and the CBAM exhibited a positive impact on the precision of olive variety recognition. To ascertain the efficacy of the method elucidated in this manuscript, a set of comparative experiments has been formulated. The primary objective of these experiments is to juxtapose the proposed bilinear attention EfficientNet model against the conventional cultivar identification models, including bilinear ResNet34, EfficientNet-SE attention, bilinear ResNet18, bilinear VGG16, and bilinear GoogLeNet. The experimental results obtained from the comparison analysis provide that the proposed bilinear attention EfficientNet model exhibits superior performance in terms of overall accuracy for the identification of olive cultivars. The bilinear attention EfficientNet model's accuracy for the identification of olive cultivars exceeded that of bilinear ResNet34, EfficientNet-SE attention, bilinear ResNet18, bilinear VGG16, and bilinear GoogLeNet models by 12.78, 11.53, 11.11, 10.70, and 5.00 percentage points, respectively. The present study establishes a foundation for resolving the challenge of accurately identifying olive cultivars in natural conditions. Moreover, it offers a valuable point of reference for the identification of cultivars of diverse crops.

Issue
Detection of Camellia oleifera fruit maturity based on improved YOLOv7
Transactions of the Chinese Society of Agricultural Engineering 2024, 40(5): 177-186
Published: 15 March 2024
Abstract PDF (4.9 MB) Collect
Downloads:6

The ripeness of Camellia oleifera fruits is closely related to their oil yield and tea oil quality. Manual one-time harvesting is the primary harvesting for Camellia oleifera at present. However, the uneven ripeness levels among fruits harvested in the same batch can significantly reduce the overall quality of the fruits. Furthermore, manual harvesting cannot fully meet the large-scale Camellia oleifera industry, such as low efficiency and high costs. Therefore, it is very necessary to implement intelligent harvesting for Camellia oleifera fruits. The maturity of Camellia oleifera fruit can be detected to determine the best maturity using deep learning. The purpose of this study is to establish the ideal ripeness of Camellia oleifera fruits in the natural environment. The harvesting period was also estimated to implement the intelligent harvesting, in order to improve the oil yield and quality of fruits. A dataset was constructed for the ripeness detection. The photographs of Camellia oleifera fruits were captured at different ripening stages in natural environments using a smartphone. The phenotypic characteristics of Camellia oleifera fruits were also determined to follow the industry standards. The ripeness was categorized into three stages: immature, mature, and over-mature. Data augmentation techniques were applied on the dataset, such as brightness adjustment, salt-and-pepper noise addition, and simulating artificial occlusion. After that, the dataset was divided into the training, validation, and testing sets with a ratio of 7:1:2. An improved YOLOv7 model was constructed to deal with the occlusion in the natural environment. A cross-attention module was added in the YOLOv7 feature extraction network. The vertical and horizontal information was calculated for each pixel in Camellia oleifera images using two attention-weighted procedures. The key features were identified to determine the ripeness of Camellia oleifera fruits, thus effectively avoiding the interference from the complex backgrounds, such as the branches and leaves. Additionally, the traditional non-maximum suppression was replaced with the distance- and intersection-based NMS. The normalized distance between the center points of two candidate boxes was selected to calculate the intersection ratio, particularly for the missed detections due to mutual occlusion of Camellia oleifera fruits, in order to detect the overlapping fruits. The YOLOv7 model was trained on 3098 images from the training set, then evaluated using 442 images from the validation set, and finally tested on 885 images from the test set. The better performance was achieved in the precision rate of 93.52%, a recall rate of 90.25%, an F1 score of 91.86%, an average precision of 94.60%, an average detection time of 0.77 s, and a model weight of 82.6 MB in the test set. The ablation experiments demonstrated that the improved model was used to effectively detect the ripeness of Camellia oleifera fruit. Compared with the original YOLOv7 model, the mean average accuracies were improved by 1.10 and 1.81 percentage points, respectively. The overall mean average accuracy was improved by 2.91 percentage points. However, the detection time and model size only increased by 0.015 s and 11.3 MB, respectively. Compared with the Faster R-CNN, EfficientDet, YOLOv3, and YOLOv5l models, the improved YOLOv7 model increased the average accuracy by 7.51, 5.89, 4.21, and 4.21 percentage points, respectively. Additionally, the detection time was reduced by 1.06, 1.12, 0.10, and 0.03 s, respectively. The maturity grade of Camellia oleifera fruits was accurately discriminated, compared with the previous. In summary, the improved YOLOv7 model was achieved in the higher accuracy with only a slight sacrifice in the detection time and model size. This finding can provide a theoretical basis to estimate the optimal harvesting period of Camellia oleifera fruits and intelligent picking under natural conditions.

Issue
Dense Nursery Stock Detecting and Counting Based on UAV Aerial Images and Improved LSC-CNN
Smart Agriculture 2024, 6(5): 88-97
Published: 30 September 2024
Abstract PDF (63 MB) Collect
Downloads:58
Objective

The number, location, and crown spread of nursery stock are important foundations data for their scientific management.Traditional approach of conducting nursery stock inventories through on-site individual plant surveys is labor-intensive and time-consuming. Low-cost and convenient unmanned aerial vehicles(UAVs) for on-site collection of nursery stock data are beginning to be utilized, and the statistical analysis of nursery stock information through technical means such as image processing achieved. During the data collection process, as the flight altitude of the UAV increases, the number of trees in a single image also increases. Although the anchor box can cover more information about the trees, the cost of annotation is enormous in the case of a large number of densely populated tree images. To tackle the challenges of tree adhesion and scale variance in images captured by UAVs over nursery stock, and to reduce the annotation costs, using point-labeled data as supervisory signals, an improved dense detection and counting model was proposed to accurately obtain the location, size, and quantity of the targets.

Method

To enhance the diversity of nursery stock samples, the spruce dataset, the Yosemite, and the KCL-London publicly available tree datasets were selected to construct a dense nursery stock dataset. A total of 1 520 nursery stock images were acquired and divided into training and testing sets at a ratio of 7:3. To enhance the model's adaptability to tree data of different scales and variations in lighting, data augmentation methods such as adjusting the contrast and resizing the images were applied to the images in the training set.After enhancement, the training set consists of 3 192 images, and the testing set contains 456 images. Considering the large number of trees contained in each image, to reduce the cost of annotation, the method of selecting the center point of the trees was used for labeling. The LSC-CNN model was selected as the base model. This model can detect the quantity, location, and size of trees through pointsupervised training, thereby obtaining more information about the trees. The LSC-CNN model was made improved to address issues of missed detections and false positives that occurred during the testing process. Firstly, to address the issue of missed detections caused by severe adhesion of densely packed trees, the last convolutional layer of the feature extraction network was replaced with dilated convolution. This change enlarges the receptive field of the convolutional kernel on the input while preserving the detailed features of the trees. So the model is better able to capture a broader range of contextual information, thereby enhancing the model's understanding of the overall scene. Secondly, the convolutional block attention module(CBAM) attention mechanism was introduced at the beginning of each scale branch. This allowed the model to focus on the key features of trees at different scales and spatial locations, thereby improving the model's sensitivity to multi-scale information. Finally, the model was trained using label smooth cross-entropy loss function and grid winner-takes-all strategy, emphasizing regions with highest losses to boost tree feature recognition.

Results and Discussions

The mean counting accuracy(MCA), mean absolute error(MAE), and root mean square error(RMSE) were adopted as evaluation metrics. Ablation studies and comparative experiments were designed to demonstrate the performance of the improved LSC-CNN model. The ablation experiment proved that the improved LSC-CNN model could effectively resolve the issues of missed detections and false positives in the LSC-CNN model, which were caused by the density and large-scale variations present in the nursery stock dataset. IntegrateNet, PSGCNet, CANet, CSRNet, CLTR and LSC-CNN models were chosen as comparative models. The improved LSC-CNN model achieved MCA, MAE, and RMSE of 91.23%, 14.24, and 22.22, respectively, got an increase in MCA by 6.67%, 2.33%, 6.81%, 5.31%, 2.09% and 2.34%, respectively; a reduction in MAE by 21.19, 11.54, 18.92, 13.28, 11.30 and 10.26, respectively; and a decrease in RMSE by 28.22, 28.63, 26.63, 14.18, 24.38 and 12.15, respectively, compared to the IntegrateNet, PSGCNet, CANet, CSRNet, CLTR and LSC-CNN models. These results indicate that the improved LSC-CNN model achieves high counting accuracy and exhibits strong generalization ability.

Conclusions

The improved LSC-CNN model integrated the advantages of point supervision learning from density estimation methods and the generation of target bounding boxes from detection methods.These improvements demonstrate the enhanced performance of the improved LSC-CNN model in terms of accuracy, precision, and reliability in detecting and counting trees. This study could hold practical reference value for the statistical work of other types of nursery stock.

Total 5