A deep Q-learning model for sequential task offloading in edge AI systems

Currently, edge Artificial Intelligence (AI) systems have significantly facilitated the functionalities of intelligent devices such as smartphones and smart cars, and supported diverse applications and services. This fundamental supports come from continuous data analysis and computation over these devices. Considering the resource constraints of terminal devices, multi-layer edge artificial intelligence systems improve the overall computing power of the system by scheduling computing tasks to edge and cloud servers for execution. Previous efforts tend to ignore the nature of strong pipelined characteristics of processing tasks in edge AI systems, such as the encryption, decryption and consensus algorithm supporting the implementation of Blockchain techniques. Therefore, this paper proposes a new pipelined task scheduling algorithm (referred to as PTS-RDQN), which utilizes the system representation ability of deep reinforcement learning and integrates multiple dimensional information to achieve global task scheduling. Specifically, a co-optimization strategy based on Rainbow Deep Q-Learning (RainbowDQN) is proposed to allocate computation tasks for mobile devices, edge and cloud servers, which is able to comprehensively consider the balance of task turnaround time, link quality, and other factors, thus effectively improving system performance and user experience. In addition, a task scheduling strategy based on PTS-RDQN is proposed, which is capable of realizing dynamic task allocation according to device load. The results based on many simulation experiments show that the proposed method can effectively improve the resource utilization, and provide an effective task scheduling strategy for the edge computing system with cloud-edge-end architecture.