Multi-Agent Reinforcement Learning (MARL) is an efficient cooperative training approach for adaptive traffic signal control (ATSC). With multiple agents seen as cooperative traffic intersections, and only be able to observe limited information in the real environment, agent policy space exploration is challenging. In addition, they share the environment reward, which makes it difficult to accurately measure contribution of individual agents. To tackle these problems, we propose the Graph Decomposition Action Reference (GDAR) framework based on Centralized Training Decentralized Execution (CTDE). Specifically, the multi-agent system is modeled as a graph structure, in which agents are regarded as nodes and relations as edges. To solve the problem of limited observation, Graph Neural Network (GNN) is used to expand the receiving domain of the agents. Meanwhile, we extract node representation to evaluate the individual contribution of each agent. In addition, we design action reference networks to improve the diversity of individual action choosing. We model the traffic conditions near the Nanjing Yangtze River Bridge in the Simulation of Urban MObility (SUMO). Experimental results show that GDAR adapts to ATSC tasks and is superior to advanced baselines.

With the rapid development of mobile communication technology and intelligent applications, the quantity of mobile devices and data traffic in networks have been growing exponentially, which poses a great burden to networks and brings huge challenge to servicing user demand. Edge caching, which utilizes the storage and computation resources of the edge to bring resources closer to end users, is a promising way to relieve network burden and enhance user experience. In this paper, we aim to survey the edge caching techniques from a comprehensive and systematic perspective. We first present an overview of edge caching, summarizing the three key issues regarding edge caching, i.e., where, what, and how to cache, and then introducing several significant caching metrics. We then carry out a detailed and in-depth elaboration on these three issues, which correspond to caching locations, caching objects, and caching strategies, respectively. In particular, we innovate on the issue “what to cache”, interpreting it as the classification of the “caching objects”, which can be further classified into content cache, data cache, and service cache. Finally, we discuss several open issues and challenges of edge caching to inspire future investigations in this research area.