Measuring the appearance indicators of spherical fruits using binocular structured light 3D reconstruction

Three typical types of spherical fruits, namely apples, oranges, and pears, have accounted for 40.86 % of the total fruit production in China in 2023. However, there is a huge trade deficit in the import and export trade of fruits in China. Furthermore, grading technology has limited the largest production and consumption of fruits. Appearance indicators, such as size, volume and deformity index, are the main external quality traits of fruits in commercial grading lines. Alternatively, computer vision has been widely used in the field of nondestructive detection of fruits and vegetables, due to the simple operation, non-contact and low cost. The surface information can be obtained to measure the external indicators using two-dimensional images. However, some limitations are still remained to capture the detailed appearance indicators, especially in the high-accuracy measurement. Compared with traditional two-dimensional images, three-dimensional (3D) imaging with the depth information can be beneficial to the appearance indicators of fruits. Among them, binocular structured light is the promising 3D reconstruction with the cost-saving, high-precision and non-contact extraction. This study focuses on spherical fruits, selecting apple, orange, and pear as representative examples, with the aim of constructing binocular structured light imaging to obtain surface contour information. The system was also validated using standard samples. Single-view point cloud reconstruction was performed as well. Three-step phase shifting was used to determine the wrapped phase. While multi-frequency heterodyne was applied to determine the absolute phase. Afterwards, stereo matching, disparity optimization, and point cloud calculation were conducted to obtain a 3D point cloud map. A rotary table was utilized to match the point cloud images from different perspectives to the same coordinate axis. Coarse point cloud registration was also achieved. A complete point cloud was obtained to conduct more precise registration using the iterative closure point (ICP) to obtain. Finally, the 3D reconstruction of spherical fruits was realized. The deformity index of the apple was extracted from the reconstruction image of single-view point cloud using normal vector angle. According to the normal vector angle, the point cloud of the fruit stem position was projected to the bottom of the fruit point cloud image. The complete fruit point cloud was improved to calculate the convex hull of the point cloud for the volume of fruit. The maximum fruit diameter was extracted through oriented bounding box (OBB) bounding box. The experimental results show that the relative error was within 3.32% in the external dimensions of standard parts using 3D reconstruction. Taking manual measurement as the references, the values of coefficient of determination (R²), root mean squared error (RMSE), and mean absolute percentage error (MAPE) were 0.97, 0.755 mm, and 7.23%, respectively, for the measured apple deformity index. In the volume of spherical fruits, R² was 0.99, RMSE was 6.015 cm³, and MAPE was 1.946%, while those were 0.92, 1.823 mm, and 1.859%, respectively, for the maximum diameter. The 3D reconstruction with binocular structured light can be expected to significantly enhance the accuracy and efficiency in the appearance indicators of spherical fruits. The finding can also provide a valuable tool to improve fruit quality control and grading.