AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (579.6 KB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Walmart Inc., Bentonville, AR 72716, USA
Department of Electrical Engineering, University of Arkansas, Fayetteville, AR 72701, USA
Show Author Information

Abstract

Mobile Edge Computing (MEC) is one of the most promising techniques for next-generation wireless communication systems. In this paper, we study the problem of dynamic caching, computation offloading, and resource allocation in cache-assisted multi-user MEC systems with stochastic task arrivals. There are multiple computationally intensive tasks in the system, and each Mobile User (MU) needs to execute a task either locally or remotely in one or more MEC servers by offloading the task data. Popular tasks can be cached in MEC servers to avoid duplicates in offloading. The cached contents can be either obtained through user offloading, fetched from a remote cloud, or fetched from another MEC server. The objective is to minimize the long-term average of a cost function, which is defined as a weighted sum of energy consumption, delay, and cache contents’ fetching costs. The weighting coefficients associated with the different metrics in the objective function can be adjusted to balance the tradeoff among them. The optimum design is performed with respect to four decision parameters: whether to cache a given task, whether to offload a given uncached task, how much transmission power should be used during offloading, and how much MEC resources to be allocated for executing a task. We propose to solve the problems by developing a dynamic scheduling policy based on Deep Reinforcement Learning (DRL) with the Deep Deterministic Policy Gradient (DDPG) method. A new decentralized DDPG algorithm is developed to obtain the optimum designs for multi-cell MEC systems by leveraging on the cooperations among neighboring MEC servers. Simulation results demonstrate that the proposed algorithm outperforms other existing strategies, such as Deep Q-Network (DQN).

References

[1]
Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, A survey on mobile edge computing: The communication perspective, IEEE Communications Surveys Tutorials, vol. 19, no. 4, pp. 2322-2358, 2017.
[2]
Y. Hao, M. Chen, L. Hu, M. S. Hossain, and A. Ghoneim, Energy efficient task caching and offloading for mobile edge computing, IEEE Access, vol. 6, pp. 11 365-11 373, 2018.
[3]
C. Wang, C. Liang, F. R. Yu, Q. Chen, and L. Tang, Computation offloading and resource allocation in wireless cellular networks with mobile edge computing, IEEE Transactions on Wireless Communications, vol. 16, no. 8, pp. 4924-4938, 2017.
[4]
M. I. A. Zahed, I. Ahmad, D. Habibi, and Q. V. Phung, Green and secure computation offloading for cache-enabled IoT networks, IEEE Access, vol. 8, pp. 63 840-63 855, 2020.
[5]
T. Alfakih, M. M. Hassan, A. Gumaei, C. Savaglio, and G. Fortino, Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA, IEEE Access, vol. 8, pp. 54 074-54 084, 2020.
[6]
N. Maurice, Q.-V. Pham, and W.-J. Hwang, Online computation offloading in noma-based multi-access edge computing: A deep reinforcement learning approach, IEEE Access, vol. 8, pp. 99 098-99 109, 2020.
[7]
L. Huang, X. Feng, C. Zhang, L. Qian, and Y. Wu, Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing, Digital Communications and Networks, vol. 5, no. 1, pp. 10-17, 2019.
[8]
J. Wang, L. Zhao, J. Liu, and N. Kato, Smart resource allocation for mobile edge computing: A deep reinforcement learning approach, IEEE Transactions on Emerging Topics in Computing, .
[9]
S. Nath, Y. Li, J. Wu, and P. Fan, Multi-user multi-channel computation offloading and resource allocation for mobile edge computing, .
[10]
S. Nath and J. Wu, Dynamic computation offloading and resource allocation for multi-user mobile edge computing, presented at IEEE Global Communications Conf. (GLOBECOM), Taipei, China, 2020.
[11]
Z. Chen and X. Wang, Decentralized computation offloading for multi-user mobile edge computing: A deep reinforcement learning approach, EURASIP Journal on Wireless Communications and Networking, .
[12]
P. Liu, G. Xu, K. Yang, K. Wang, and X. Meng, Jointly optimized energy-minimal resource allocation in cache-enhanced mobile edge computing systems, IEEE Access, vol. 7, pp. 3336-3347, 2018.
[13]
Y. Mao, J. Zhang, S. H. Song, and K. B. Letaief, Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems, IEEE Transactions on Wireless Communications, vol. 16, no. 9, pp. 5994-6009, 2017.
[14]
L. Chunlin and J. Zhang, Dynamic cooperative caching strategy for delay-sensitive applications in edge computing environment, The Journal of Supercomputing, vol. 76, no. 1, pp. 1-25, 2020.
[15]
J. Xu, L. Chen, and P. Zhou, Joint service caching and task offloading for mobile edge computing in dense networks, in Proc. of IEEE Conference on Computer Communications (INFOCOM), Honolulu, HI, USA, 2018, pp. 207-215.
[16]
P. Yang, N. Zhang, S. Zhang, L. Yu, J. Zhang, and X. Shen, Dynamic mobile edge caching with location differentiation, .
[17]
R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. Cambridge, MA, USA: MIT press, 2018.
[18]
K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, Deep reinforcement learning: A brief survey, IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26-38, 2017.
[19]
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, Playing atari with deep reinforcement learning, arXiv preprint arXiv:1312.5602, 2013.
[20]
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, Continuous control with deep reinforcement learning, arXiv preprint arXiv:1509.02971, 2015.
[21]
K. Shanmugam, N. Golrezaei, A. G. Dimakis, A. F. Molisch, and G. Caire, Femtocaching: Wireless content delivery through distributed caching helpers, IEEE Transactions on Information Theory, vol. 59, no. 12, pp. 8402-8413, 2013.
[22]
A. Sadeghi, F. Sheikholeslami, and G. B. Giannakis, Optimal dynamic proactive caching via reinforcement learning, .
[23]
H. A. Suraweera, T. A. Tsiftsis, G. K. Karagiannidis, and A. Nallanathan, Effect of feedback delay on amplify-and-forward relay networks with beamforming, IEEE Transactions on Vehicular Technology, vol. 60, no. 3, pp. 1265-1271, 2011.
[24]
M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Washington, DC, USA: US Government Printing Office, 1948.
[25]
H. Q. Ngo, E. G. Larsson, and T. L. Marzetta, Energy and spectral efficiency of very large multiuser mimo systems, IEEE Transactions on Communications, vol. 61, no. 4, pp. 1436-1449, 2013.
[26]
Y. Wen, W. Zhang, and H. Luo, Energy-optimal mobile application execution: Taming resource-poor mobile devices with cloud clones, in Proc. IEEE Conference on Computer Communications (INFOCOM), Orlando, FL, USA, 2012, pp. 2716-2720.
[27]
X. Chen, L. Jiao, W. Li, and X. Fu, Efficient multi-user computation offloading for mobile-edge cloud computing, IEEE/ACM Transactions on Networking, vol. 24, no. 5, pp. 2795-2808, 2016.
[28]
K. Zhang, Y. Mao, S. Leng, Q. Zhao, L. Li, X. Peng, L. Pan, S. Maharjan, and Y. Zhang, Energy-efficient offloading for mobile edge computing in 5G heterogeneous networks, IEEE Access, vol. 4, pp. 5896-5907, 2016.
[29]
J. Li, H. Gao, T. Lv, and Y. Lu, Deep reinforcement learning based computation offloading and resource allocation for MEC, in Proc. Wireless Communications and Networking Conference (WCNC), Barcelona, Spain, 2018, pp. 1-6.
[30]
S. Nath, J. Wu, and J. Yang, Delay and energy efficiency tradeoff for information pushing system, IEEE Transactions on Green Communications and Networking, vol. 2, no. 4, pp. 1027-1040, 2018.
[31]
S. Nath, J. Wu, and H. Lin, Optimum multicast scheduling in delay-constrained content-centric wireless networks, .
[32]
S. Nath, J. Wu, and J. Yang, Optimum energy efficiency and age-of-information tradeoff in multicast scheduling, .
[33]
S. Nath, J. Wu, and J. Yang, Optimizing age-of-information and energy efficiency tradeoff for mobile pushing notifications, .
[34]
D. Adelman and A. J. Mersereau, Relaxations of weakly coupled stochastic dynamic programs, Operations Research, vol. 56, no. 3, pp. 712-727, 2008.
[35]
C. J. Watkins and P. Dayan, Q-learning, Machine learning, vol. 8, no. 3, pp. 279-292, 1992.
[36]
G. E. Uhlenbeck and L. S. Ornstein, On the theory of the brownian motion, Physical Review, vol. 36, no. 5, p. 823, 1930.
[37]
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014.
Intelligent and Converged Networks
Pages 181-198
Cite this article:
Nath S, Wu J. Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems. Intelligent and Converged Networks, 2020, 1(2): 181-198. https://doi.org/10.23919/ICN.2020.0014

1503

Views

144

Downloads

138

Crossref

149

Scopus

Altmetrics

Received: 30 May 2020
Revised: 15 September 2020
Accepted: 22 October 2020
Published: 01 December 2020
© All articles included in the journal are copyrighted to the ITU and TUP 2020

© All articles included in the journal are copyrighted to the ITU and TUP. This work is available under the CC BY-NC-ND 3.0 IGO license: https://creativecommons.org/licenses/by-nc-nd/3.0/igo/.

Return