Cocoons have a direct impact on the quality of raw silk. Mulberry cocoons must be screened before silk reeling can begin in order to improve the quality of raw silk. For the silk product sector, the cocoons' level of categorization and sorting is crucial. Nonetheless, the majority of mulberry cocoon production facilities in use today choose the cocoons by hand. The accuracy and efficiency of mulberry cocoon plucking can be significantly improved by automatic methods. To increase efficiency, mulberry cocoons must be sorted automatically and intelligently using machine vision. We proposed an effective detection technique based on vision and terahertz spectrum characteristics data for distinguishing defective cocoons, including common and thin shelled defective cocoons. Each mulberry cocoon with a defect had its spatial coordinate and deflection angle computed so that grippers could grasp it. With 3762 photos in our dataset, our approach could detect mAP values up to 99.25% of the time. Furthermore, the GFLOPS of our suggested model was only 8.9 G, and its Parameters were only 5.3 M, making it appropriate for use in real-world application scenarios.