AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (15.4 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

A DQN-Based Edge Offloading Method for Smart City Pollution Control

School of Software, Nanjing University of Information Science and Technology, Nanjing 210044, China
School of Computing and Communications, Lancaster University, Lancaster, LA1 4YW, United Kingdom
School of Computing Sciences, Pak-Austria Fachhochschule: Institute of Applied Sciences and Technology, Haripur, KPK 22620, Pakistan
Show Author Information

Abstract

Smart city pollution control is fundamental to urban sustainability, which relies extensively on physical infrastructure such as sensors and cameras for real-time monitoring. Generally, monitoring data needs to be transmitted to centralized servers for pollution control service determination. In order to achieve highly efficient service quality, edge computing is involved in the smart city pollution control system (SCPCS) as it provides computational capabilities near the monitoring devices and low-latency pollution control services. However, considering the diversity of service requests, determination of offloading destination is a crucial challenge for SCPCS. In this paper, A Deep Q-Network (DQN)-based edge offloading method, called N-DEO, is proposed. Initially, N-DEO employs neural hierarchical interpolation for time series forecasting (N-HITS) to forecast pollution control service requests. Afterwards, an epsilon-greedy policy is designed to select actions. Finally, the optimal service offloading strategy is determined by the DQN algorithm. Experimental results demonstrate that N-DEO achieves the higher performance on service latency and system load compared with the current state-of-the-art methods.

References

[1]
S. Zhang, Q. Jiang, H. Li, B. Cao, and J. Fan, PURP: A scalable system for predicting short-term urban traffic flow based on license plate recognition data, Big Data Mining and Analytics, vol. 7, no. 1, pp. 171–187, 2024.
[2]

D. Zhang, X. Wang, W. Rong, and Y. Yang, China’s practice of smart city standardisation and assessment, IET Smart Cities, vol. 3, no. 4, pp. 211–218, 2021.

[3]

T. Wu, W. Dou, F. Wu, S. Tang, C. Hu, and J. Chen, A deployment optimization scheme over multimedia big data for large-scale media streaming application, ACM Trans. Multimedia Comput. Commun. Appl., vol. 12, no. 5s, pp. 1–23, 2016.

[4]
A. Jia and C. Xu, RETRACTED: Smart city image landscape design based on wireless sensors, Microprocess. Microsyst., vol. 83, p. 104022, 2021.
[5]

M. Mohy-Eddine, A. Guezzaz, S. Benkirane, M. Azrour, and Y. Farhaoui, An ensemble learning based intrusion detection model for industrial IoT security, Big Data Mining and Analytics, vol. 6, no. 3, pp. 273–287, 2023.

[6]

C. Hazman, A. Guezzaz, S. Benkirane, and M. Azrour, Enhanced IDS with deep learning for IoT-based smart cities security, Tsinghua Science and Technology, vol. 29, no. 4, pp. 929–947, 2024.

[7]

L. Kong, K. Liu, X. Hu, N. Zhang, L. Qi, X. Li, and X. Zhou, Gender classification based on spatio-frequency feature fusion of OCT fingerprint images in the IoT environment, IEEE Internet Things J., vol. 11, no. 15, pp. 25731–25743, 2024.

[8]

X. Xu, H. Li, Z. Li, and X. Zhou, Safe: synergic data filtering for federated learning in cloud-edge computing, IEEE Trans. Ind. Inform., vol. 19, no. 2, pp. 1655–1665, 2023.

[9]

C. Hu, W. Fan, E. Zeng, Z. Hang, F. Wang, L. Qi, and M. Z. A. Bhuiyan, Digital twin-assisted real-time traffic data prediction method for 5G-enabled Internet of vehicles, IEEE Trans. Ind. Inform., vol. 18, no. 4, pp. 2811–2819, 2022.

[10]

X. Huang, L. Xiao, and X. Tou, Scientific decision support system of marine environmental management in China’s Yellow Sea and Bohai Sea based on cloud computing mode, J. Intell. Fuzzy Syst., vol. 37, no. 5, pp. 5877, 5886.

[11]

G. J. R. Kumar, G. P. Agbulu, T. V. Rahul, A. V. Natarajan, and K. Gokul, A cloud-assisted mesh sensor network solution for public zone air pollution real-time data acquisition, J. Ambient Intell. Humaniz. Comput., vol. 13, no. 9, pp. 4159–4173, 2022.

[12]

M. Abbasi, E. Mohammadi-Pasand, and M. R. Khosravi, Intelligent workload allocation in IoT–Fog–cloud architecture towards mobile edge computing, Comput. Commun., vol. 169, pp. 71–80, 2021.

[13]

Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, A survey on mobile edge computing: The communication perspective, IEEE Commun. Surv. Tutor., vol. 19, no. 4, pp. 2322–2358, 2017.

[14]

L. Gu, M. Cui, L. Xu, and X. Xu, Collaborative offloading method for digital twin empowered cloud edge computing on Internet of vehicles, Tsinghua Science and Technology, vol. 28, no. 3, pp. 433–451, 2023.

[15]

K. Sultana, K. Ahmed, B. Gu, and H. Wang, Elastic optimization for stragglers in edge federated learning, Big Data Mining and Analytics, vol. 6, no. 4, pp. 404–420, 2023.

[16]
X. Xu, S. Tang, L. Qi, X. Zhou, F. Dai, and W. Dou, CNN partitioning and offloading for vehicular edge networks in Web3, IEEE Commun. Mag., vol. 61, no. 8, pp. 36–42, 2023.
[17]

J. Ren, D. Zhang, S. He, Y. Zhang, and T. Li, A survey on end-edge-cloud orchestrated network computing paradigms, ACM Comput. Surv., vol. 52, no. 6, pp. 1–36, 2020.

[18]

Z. Li, G. Li, M. Bilal, D. Liu, T. Huang, and X. Xu, Blockchain-assisted server placement with elitist preserved genetic algorithm in edge computing, IEEE Internet Things J., vol. 10, no. 24, pp. 21401–21409, 2023.

[19]

X. Sun, Y. He, D. Wu, and J. Z. Huang, Survey of distributed computing frameworks for supporting big data analysis, Big Data Mining and Analytics, vol. 6, no. 2, pp. 154–169, 2023.

[20]

L. Wei, L. Jin, C. Yang, K. Chen, and W. Li, New noise-tolerant neural algorithms for future dynamic nonlinear optimization with estimation on hessian matrix inversion, IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 4, pp. 2611–2623, 2021.

[21]

S. Davoodi, H. Vo Thanh, D. A. Wood, M. Mehrad, V. S. Rukavishnikov, and Z. Dai, Machine-learning predictions of solubility and residual trapping indexes of carbon dioxide from global geological storage sites, Expert Syst. Appl., vol. 222, pp. 119796, 2023.

[22]

J. Han, H. Liu, H. Xiong, and J. Yang, Semi-supervised air quality forecasting via self-supervised hierarchical graph neural network, IEEE Trans. Knowl. Data Eng., vol. 35, no. 5, pp. 5230–5243, 2023.

[23]

S. Du, T. Li, Y. Yang, and S.-J. Horng, Deep air quality forecasting using hybrid deep learning framework, IEEE Trans. Knowl. Data Eng., vol. 33, no. 6, pp. 2412–2424, 2021.

[24]

R. Yan, J. Liao, J. Yang, W. Sun, M. Nong, and F. Li, Multi-hour and multi-site air quality index forecasting in Beijing using CNN, LSTM, CNN-LSTM, and spatiotemporal clustering, Expert Syst. Appl., vol. 169, pp. 114513, 2021.

[25]

A. Yousefpour, C. Fung, T. Nguyen, K. Kadiyala, F. Jalali, A. Niakanlahiji, J. Kong, and J. P. Jue, All one needs to know about fog computing and related edge computing paradigms: A complete survey, J. Syst. Archit., vol. 98, pp. 289–330, 2019.

[26]

Q. He, G. Cui, X. Zhang, F. Chen, S. Deng, H. Jin, Y. Li, and Y. Yang, A game-theoretical approach for user allocation in edge computing environment, IEEE Trans. Parallel Distrib. Syst., vol. 31, no. 3, pp. 515–529, 2020.

[27]

J. Ren, G. Yu, Y. He, and G. Y. Li, Collaborative cloud and edge computing for latency minimization, IEEE Trans. Veh. Technol., vol. 68, no. 5, pp. 5031–5044, 2019.

[28]

L. Zeng, Q. Liu, S. Shen, and X. Liu, Improved double deep Q network-based task scheduling algorithm in edge computing for makespan optimization, Tsinghua Science and Technology, vol. 29, no. 3, pp. 806–817, 2024.

[29]

P. Wang and S. Wang, A fairness-enhanced intelligent MAC scheme using Q-learning-based bidirectional backoff for distributed vehicular communication networks, Tsinghua Science and Technology, vol. 28, no. 2, pp. 258–268, 2023.

[30]
Y. Yuan, W. Su, G. Hong, H. Li, and C. Wang, A joint caching and offloading strategy using reinforcement learning for multi-access edge computing users, Mob. Netw. Appl., 2024.
[31]

A. Robles-Enciso and A. F. Skarmeta, A multi-layer guided reinforcement learning-based tasks offloading in edge computing, Comput. Netw., vol. 220, pp. 109476, 2023.

[32]

X. Ju, S. Su, C. Xu, and H. Wang, Computation offloading and tasks scheduling for the Internet of vehicles in edge computing: A deep reinforcement learning-based pointer network approach, Comput. Netw., vol. 223, pp. 109572, 2023.

[33]

X. Yao, N. Chen, X. Yuan, and P. Ou, Performance optimization of serverless edge computing function offloading based on deep reinforcement learning, Future Gener. Comput. Syst., vol. 139, pp. 74–86, 2023.

[34]

X. Zhou, M. Bilal, R. Dou, J. J. P. C. Rodrigues, Q. Zhao, J. Dai, and X. Xu, Edge computation offloading with content caching in 6G-enabled IoV, IEEE Trans. Intell. Transp. Syst., vol. 25, no. 3, pp. 2733–2747, 2024.

[35]

Z. Aghapour, S. Sharifian, and H. Taheri, Task offloading and resource allocation algorithm based on deep reinforcement learning for distributed AI execution tasks in IoT edge computing environments, Comput. Netw., vol. 223, pp. 109577, 2023.

[36]

X. Zhu, Y. Luo, A. Liu, M. Z. A. Bhuiyan, and S. Zhang, Multiagent deep reinforcement learning for vehicular computation offloading in IoT, IEEE Internet Things J., vol. 8, no. 12, pp. 9763–9773, 2021.

[37]

M. Tang and V. W. S. Wong, Deep reinforcement learning for task offloading in mobile edge computing systems, IEEE Trans. Mob. Comput., vol. 21, no. 6, pp. 1985–1997, 2022.

[38]
Z. Pan, Y. Liang, W. Wang, Y. Yu, Y. Zheng, and J. Zhang, Urban traffic prediction from spatio-temporal data using deep meta learning, in Proc. 25th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining. Anchorage, AK, USA, 2019, pp. 1720–1730.
[39]
J. Zhang, Y. Zheng, D. Qi, R. Li, and X. Yi, DNN-based prediction model for spatio-temporal data, in Proc. 24th ACM SIGSPATIAL Int. Conf. Advances in Geographic Information Systems. Burlingame, CA, USA, 2016.
[40]
C. Challu, K. G. Olivares, B. N. Oreshkin, F. Garza Ramirez, M. Mergenthaler Canseco, and A. Dubrawski, NHITS: neural hierarchical interpolation for time series forecasting, Proc. AAAI Conf. Artif. Intell., vol. 37, no. 6, pp. 6989–6997, 2023.
[41]
F. Christianos, L. Schäfer, and S. V. Albrecht, Shared experience actor-critic for multi-agent reinforcement learning, arXiv preprint arXiv: 2006.07169, 2020.
[42]
Y. Song, L. Wei, Q. Yang, J. Wu, L. Xing, and Y. Chen, RL-GA: A reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem, Swarm Evol. Comput., vol. 77, p. 101236, 2023.
[43]
B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio, N-BEATS: Neural basis expansion analysis for interpretable time series forecasting, arXiv preprint arXiv: 1905.10437, 2019.
[44]

R. Luzia, L. Rubio, and C. E. Velasquez, Sensitivity analysis for forecasting Brazilian electricity demand using artificial neural networks and hybrid models based on Autoregressive Integrated Moving Average, Energy, vol. 274, pp. 127365, 2023.

[45]

H. Abbasimehr, M. Shabani, and M. Yousefi, An optimized model using LSTM network for demand forecasting, Comput. Ind. Eng., vol. 143, pp. 106435, 2020.

Tsinghua Science and Technology
Pages 2227-2242
Cite this article:
Xu J, Xiang H, Zang S, et al. A DQN-Based Edge Offloading Method for Smart City Pollution Control. Tsinghua Science and Technology, 2025, 30(5): 2227-2242. https://doi.org/10.26599/TST.2024.9010105

567

Views

81

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics

Received: 07 April 2024
Revised: 14 May 2024
Accepted: 05 June 2024
Published: 29 April 2025
© The Author(s) 2025.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return