Existing three-dimensional reconstruction technologies are generally constrained by the high cost of professional acquisition equipment, stringent requirements for lighting conditions, and limited adaptability in real-time application scenarios. To address these issues, this study aims to develop a robust and efficient polarization-based 3D reconstruction method that integrates physical prior knowledge with deep learning, thereby achieving accurate surface normal estimation under low-cost conditions and with good robustness.

Based on Fresnel’s reflection law, this paper proposed the FUnet model, which fused polarization optical characteristics with deep learning. The method took four-channel polarization images (0°, 45°, 90°, 135°) as input and calculated the Stokes parameters (DoLP, AoLP, S0). The core innovation was a differentiable inverse solving module named DTZ, designed to extract physical prior knowledge of surface normals by solving the inverse problem of the Fresnel reflection model. Unlike traditional methods that use non-differentiable look-up tables or numerical optimization which truncate gradients, the DTZ module densely samples the zenith angle (M=300) within the interval [0, π/2]. It calculated the difference between the theoretical degree of linear polarization and the ground truth, employing a Softmax weighting strategy to extract priors for both specular and diffuse reflection models. This design ensured gradient continuity throughout the backpropagation process, enabling end-to-end integration of physical prior extraction with network training. Building upon this, a RIE (refractive index estimation) module was designed utilizing convolutional neural networks and global average pooling to dynamically predict the object’s refractive index nn and ratio coefficient kk based on image features, breaking the limitation of fixed parameters in traditional methods. Finally, the FUnet architecture was constructed to refine these priors, incorporating residual blocks, a SA (spatial attention) mechanism to focus on key spatial regions, and PS (pixel shuffle) upsampling for high-quality reconstruction without artifacts. The network was trained using a composite loss function combining zenith angle loss, azimuth angle loss, and cosine similarity loss to ensure comprehensive optimization.

Extensive experiments were conducted on the publicly available DeepSfP dataset, which comprises 27 objects with diverse materials and coatings captured under various lighting conditions. The model was trained for 300 epochs using the Adam optimizer with a cosine annealing learning rate schedule and standard data augmentation. The proposed FUnet achieved an average angular error of 11.49° on the test set, significantly outperforming traditional physical methods (43.94°) and existing state-of-the-art deep learning techniques (best previous at 13.37°). Ablation studies quantitatively validated the contribution of each component: removing the spatial attention module increased the error to 12.52°, while replacing pixel shuffle upsampling with bilinear interpolation led to 12.93°. Notably, disabling the RIE module and using a fixed refractive index (n=1.5) resulted in a significant performance drop to 14.81°, highlighting the critical role of dynamic material parameter estimation. Furthermore, generalization tests demonstrated that the model performs robustly on unseen 3D-printed objects outside the training data, producing reasonable normal maps. Finally, successful 3D mesh reconstruction was achieved from the predicted normals via Poisson integration, and the refractive indices estimated by the RIE module were consistent with the physical properties of common materials like plastics and glass.

This research successfully integrates physical optical principles with deep learning architectures for polarization-based 3D reconstruction. The proposed FUnet model, featuring the differentiable physical prior calculation via DTZ and the dynamic parameter prediction via RIE, not only significantly improves the accuracy of surface normal estimation compared to previous methods but also provides a new paradigm for the joint optimization of geometric and material properties. The experimental results validate the effectiveness of incorporating physical prior knowledge to enhance deep learning models in this domain. The method offers an efficient and feasible technical solution for low-cost and robust polarization 3D reconstruction, with promising applications in fields such as industrial inspection, robotic perception, and digital heritage preservation.