The 4-dimensional CT (4D-CT) images of the lungs undergo large deformations due to respiration and heartbeat, and the scale of motion within the lungs may be larger than the structures of interest (blood vessels, airways, etc.) that the algorithm uses for the optimization process, which may result in the registration algorithms only aligning the obvious features such as blood vessels and airways. To address the problem of high variability of the aligned intensities for structures with large deformations such as the lung parenchyma contour, this paper proposed a multi-scale residual deformable lung CT image alignment algorithm framework based on unsupervised end-to-end deep learning. A multi-scale deep residual network in the form of an encoder-decoder structure was used as a generative model for the deformation field in the proposed registration framework, so as to enhance the feature representation, to increase the effective parameter utilization efficiency parameters and effectively improve the convergence ability of the network. A multi-resolution self-attentive fusion module was used to improve the network’s ability to perceive multi-scale information. And a hopping connection containing a feature correction extraction module was designed to selectively extract the feature maps output by the encoder and recalibrate them for the decoder to learn the alignment offsets. Finally, this paper compared the proposed alignment algorithm with traditional algorithms and the current state-of-the-art unsupervised alignment algorithms on the Dir-lab public dataset. The results show that, the target alignment error of the proposed registration algorithm framework on the Dir-lab public dataset can reach 1.44 mm ± 1.24 mm, which is better than traditional algorithms and the mainstream unsupervised alignment algorithm. In addition, the estimation of the dense deformation vector field takes less than 2.00 s with the control folding voxel less than 0.1%, indicating the great potential of the algorithm in studying time-sensitive lungs.