This study aims to balance performance optimization with model lightweighting for concrete crack segmentation. Firstly, traditional data augmentation techniques, such as image mirroring, were employed to expand the dataset and enhance its diversity, providing richer samples for model training. Second, to improve the training process, we introduced a CE-dice loss function that combines cross-entropy loss and dice loss. This approach effectively addresses class imbalance issues and enhances segmentation performance. Additionally, we designed a multi-scale feature fusion network with dual attention mechanisms to capture crack features at various scales and integrate attention information, further improving accuracy and robustness in crack recognition tasks. To evaluate the proposed method comprehensively, we used traditional quantitative metrics and incorporated neural network interpretability analysis to reveal the underlying mechanisms of model decisions, thereby increasing the transparency and trustworthiness of the model. Experimental results showed that the proposed method achieved a 7.25% improvement in Intersection over Union (IoU) and a 6.33% increase in Recall compared to the original model. Notably, the model size was reduced to just 22 MB, significantly decreasing the file size while maintaining efficient crack recognition capabilities, thus meeting the demand for lightweight models.