In the millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) system with lens antenna array, because the radio frequency (RF) link is much less than the number of antennas, it is necessary to recover the high-dimensional channel from the low-dimensional effective measurement signal by channel estimation. The current channel estimation methods basically make use of the sparsity of the beamspace channel, transforming the channel estimation into compressed sensing problem and then estimating with different methods. Aiming at the limitation that approximate message passing (AMP) algorithm needs channel prior information in channel estimation, this paper proposed an improved channel estimation algorithm. Firstly, a new noise term was derived based on the AMP algorithm and fitted with a convolutional neural network (CNN). Then the iterative denoising process was expanded into a deep network to solve the linear inverse transformation of the measurement signal to the channel. Finally, the initially estimated channel was further optimized by a residual noise removal network. In addition, the controllable parameters were introduced to increase the flexibility of the channel estimation process, and the sensing matrix was jointly trained with other network parameters to improve the channel estimation accuracy. This paper verified the proposed algorithm from two aspects of channel estimation accuracy and system transmission quality, and carried out the theoretical formula derivation and system simulation analysis on the Saleh-Valenzuela channel model. Simulation results show that the proposed algorithm has less model parameters and computation than the traditional algorithm, and can improve the accuracy of channel estimation and the transmission quality of the communication system.

Channel estimation is a key technology in communication systems, which involves evaluating the channel characteristics experienced by signals during transmission, so that the receiver can effectively process and recover the received signals. In order to improve the quality of the communication system under the LOS channel occlusion communication, this study used an intelligent hypersurface to assist the existing communication system. In the smart hypersurface assisted wireless communication system, in addition to the line of sight channel between the base station and the user, there are also cascaded channels from the base station to the smart hypersurface and from the smart hypersurface to the user. The current channel estimation methods basically use traditional algorithms to estimate. In order to solve the problems of low accuracy and high computational complexity of the cascaded channel estimation with complex statistical distribution in the intelligent hypersurface assisted multi-user system, this paper proposed a channel estimation algorithm based on the combination of traditional algorithm and deep learning algorithm. The interpretability of traditional algorithms and the high performance of deep learning algorithms were utilized. On the basis of convolutional network, a denoising method based on residual dense network (RDN) was proposed. Firstly, the data set of the real environment was generated according to the simulation of the system parameters. And the traditional least squares (LS) method was used to rough estimate the channel, and the channel was regarded as a two-dimensional noisy image. Secondly, dense blocks (RDB) were used to fully extract the local features of noise data, and multi-channel convolution and residual network were used to fuse the data features. Finally, the trained model was used to estimate the data online and obtain the denoising channel. The proposed algorithm was verified from the channel estimation accuracy, and theoretical formula derivation and system simulation analysis were carried out on the Rician channel model. Simulation results show that the proposed algorithm improves the accuracy of channel estimation compared with the traditional algorithm.