Mushroom quality intelligent sorting using full-chain RFID traceability and multimodal deep learning

Mushrooms have exceeded the 10-million-ton scale per year in China. However, manual sorting cannot fully meet the large-scale production in the mushroom industry, due to the low efficiency, traceability, and strong subjectivity. It is often required for high-quality sorting of high-value-added products in smart agriculture. In this study, an intelligent quality evaluation of shiitake mushrooms was proposed to integrate accurate sorting and full-chain traceability. Integrated information collection was constructed as "RFID + multimodal perception + edge computing + YOLOv8": Each batch of shiitake mushrooms was assigned an RFID electronic tag to bind data in the entire process of cultivation, harvesting, and circulation, thereby realizing a data closed loop from the source to the supermarket shelves. Multimodal data was collected using a USB industrial camera (resolution: 4K×3K), temperature and humidity sensors, weight sensors, and odor sensors, with the multi-dimensional data matrix of "images + sensor data + RFID". Images were labeled for front and back sides, according to refined industry standards. The dataset was augmented after the affine and contrast transformation. Transfer learning was conducted using the pre-trained YOLOv8 model, with the introduction of an attention mechanism for model optimization. Meanwhile, non-image sensor data was preprocessed and then encoded before undergoing feature-level + decision-level fusion with image data, particularly for the high-quality identification and classification. Multiple experiments were verified for the excellent performance of intelligent sorting: 1) High sorting accuracy and efficiency: The proposed model achieved an mAP@0.5 of 94.4% and an average F1 score of 94.7%, with an inference throughput of 26.3 frames per second. The end-to-end grading time was 472 ms per sample. 2) Significant comparison: The number of parameters was 9.88 times fewer than that of Faster R-CNN, indicating the high inference speed. Although the improved YOLOv8-Seg achieved higher accuracy, its large number of parameters was unfavorable for deployment on Raspberry Pi. 3) Remarkable improvements over manual sorting: Recognition accuracy and efficiency reduced the subjective bias of manual sorting. 4) Improved traceability function: Key information of each batch of shiitake mushrooms was accurately traced using full-chain data binding of RFID electronic tags, such as origin, cultivation environment, and circulation path, thereby realizing real-time association between the sorting and traceability information. 5) Robust multimodal fusion: Multimodal data fusion reduced the model’s reliance on a single data modality and enabled complementary information to be integrated across the visual, physical-chemical, and traceability modalities.Accurate quality sorting and full-process quality traceability were integrated for the high stability and reliability of quality classification. The findings can fully meet the technical requirements of standard sorting in smart agriculture.