Mango firmness is one of the critical indicators for assessing internal quality and taste, as well as an indirect measure of maturity and freshness during ripening. Acoustic vibration technology has been widely applied for nondestructive detection of fruit firmness. However, existing detection systems face the risk of fruit damage, prediction performance limitations, and significant influence of fruit size. This study designed a nondestructive pneumatic-electromagnetic-driven impact device based on acoustic vibration technology for firmness detection of different sizes of mango with the same impact force control. Vibration signals of 156 mangoes were acquired using an embedded accelerometer, and effective vibration signals were selected by comparing the excitation vibration response signals and the free vibration response signals. The correlation between mango reference firmness and vibration signal features was then analyzed. Based on this analysis, a prediction model for mango firmness was developed using partial least squares regression based on competitive adaptive reweighted sampling (CARS-PLSR). The results showed that the energy-type and amplitude-type statistical features in the vibration signals had a good correlation with the reference firmness ( $\left|r\right|$≥0.45), and the mango firmness prediction model based on the vibration frequency-domain signals (CARS-PLSR) had the optimal performance. The model’s prediction determination coefficient ( $R_P^2$), root mean square error of prediction (RMSEP), and relative percent deviation ( $RP{D}_P$) were 0.95, 0.29 N/mm, and 4.20, respectively. Overall, it demonstrated that the pneumatic-electromagnetic-driven impact device integrated with an embedded accelerometer enables accurate and nondestructive detection of mango firmness. The innovative combination of pneumatic control and electromagnetic drive effectively minimizes the impact of fruit size variations and enhances prediction accuracy, demonstrating the significant potential for real-time fruit firmness sorting applications.