Face inpainting based on edge confrontation combined with hierarchical gated convolution

Aiming at the problems of edge blur and distortion in the current damaged face image inpainting, a two-stage hierarchical gated convolutional network(HGCN) was proposed and then combined with edge adversarial network for face image inpainting. Firstly, the edge adversarial network was adopted to generate edge images. Secondly, the edge images, the masks and the occluded images were combined to train the generative adversarial network (GAN) model of the HGCN to generate the inpainted face images. In the HGCN, traditional convolution was replaced by gated convolution and the dilated convolution was introduced. The main structure of the HGCN is composed of coarse inpainting module and fine inpainting module. In the coarse inpainting module, the encoder and decoder network structure was used for coarse inpainting. In the fine inpainting module, the attention mechanism was introduced to enhance the feature extraction ability so as to further refine the inpainting results. In the experiment, the Celeba-HQ dataset and NVIDIA irregular mask dataset were used as the training datasets, the gated convolution network and attention module were adopted as comparing networks, and PSNR, SSIM and MAE were used as evaluation indicators. The experimental results demonstrated that for the face images with missing areas less than 20%, the proposed network works better than the two other networks on the above three indicators, and for the face images with missing areas greater than 20%, the proposed network is close to the comparison networks on three indicators. In terms of visual effects, the proposed method also surpasses the two contrasting networks in details. The proposed network can evidently improve the inpainting effect, especially image details.