The adoption of the Fifth Generation (5G) and beyond 5G networks is driving the demand for learning approaches that enable users to co-exist harmoniously in a multi-user distributed environment. Although resource-constrained, the Cognitive Radio (CR) has been identified as a key enabler of distributed 5G and beyond networks due to its cognitive abilities and ability to access idle spectrum opportunistically. Reinforcement learning is well suited to meet the demand for learning in 5G and beyond 5G networks because it does not require the learning agent to have prior information about the environment in which it operates. Intuitively, CRs should be enabled to implement reinforcement learning to efficiently gain opportunistic access to spectrum and co-exist with each other. However, the application of reinforcement learning is straightforward in a single-agent environment and complex and resource intensive in a multi-agent and multi-objective learning environment. In this paper, (1) we present a brief history and overview of reinforcement learning and its limitations; (2) we provide a review of recent multi-agent learning methods proposed and multi-agent learning algorithms applied in Cognitive Radio (CR) networks; and (3) we further present a novel framework for multi-CR reinforcement learning and conclude with a synopsis of future research directions and recommendations.

It has always been difficult to achieve accurate information of the channel for underwater acoustic communications because of the severe underwater propagation conditions, including frequency-selective property, high relative mobility, long propagation latency, and intensive ambient noise, etc. To this end, a deep unfolding neural network based approach is proposed, in which multiple layers of the network mimic the iterations of the classical iterative sparse approximation algorithm to extract the inherent sparse features of the channel by exploiting deep learning, and a scheme based on the Sparsity-Aware DNN (SA-DNN) for UAC estimation is proposed to improve the estimation accuracy. Moreover, we propose a Denoising Sparsity-Aware DNN (DeSA-DNN) based enhanced method that integrates a denoising CNN module in the sparsity-aware deep network, so that the degradation brought by intensive ambient noise could be eliminated and the estimation accuracy can be further improved. Simulation results demonstrate that the performance of the proposed schemes is superior to the state-of-the-art compressed sensing based and iterative sparse recovery schems in the aspects of channel recovery precision, pilot overhead, and robustness, particularly under unideal circumstances of intensive ambient noise or inadequate measurement pilots.

With next generation networks driving the confluence of multi-media, broadband, and broadcast services, Cognitive Radio (CR) networks are positioned as a preferred paradigm to address spectrum capacity challenges. CRs address these issues through dynamic spectrum access. However, the main challenges faced by the CR pertain to achieving spectrum efficiency. As a result, spectrum efficiency improvement models based on spectrum sensing and sharing models have attracted a lot of research attention in recent years, including CR learning models, network densification architectures, and massive Multiple Input Multiple Output (MIMO), and beamforming techniques. This paper provides a survey of recent CR spectrum efficiency improvement models and techniques, developed to support ultra-reliable low latency communications that are resilient to surges in traffic and competition for spectrum. These models and techniques, broadly speaking, enable a wide range of functionality ranging from enhanced mobile broadband to large scale Internet of Things (IoT) type communications. In addition and given the strong correlation between the typical size of a spectrum block and the achievable data rate, the models studied in this paper are applicable in ultra-high frequency band. This study therefore provides a good review of CRs and direction for future investigations into newly identified 5G research areas, applicable in industry and in academia.