Journal Home > Volume 27 , Issue 1
Tsinghua Science and Technology 2022, 27(1): 13-26 https://doi.org/10.26599/TST.2020.9010033
Open Access | Issue | Published: 17 August 2021

I-Neat: An Intelligent Framework for Adaptive Virtual Machine Consolidation

Show Author's Information Yanxin Liu,  Yao Zhao,  Jian Dong( ),  Lianpeng Li,  Chunpei Wang,  Decheng Zuo
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
Keywords:
reinforcement learning, cloud computing, OpenStack, dynamic Virtual Machine (VM) consolidation, Neat
Cite this article:
Liu Y, Zhao Y, Dong J, et al. I-Neat: An Intelligent Framework for Adaptive Virtual Machine Consolidation. Tsinghua Science and Technology, 2022, 27(1): 13-26. https://doi.org/10.26599/TST.2020.9010033
Download citation
EndNote(RIS)
BibTeX

698

Views

76

Downloads

Citations

2

Crossref

1

WoS

3

Scopus

0

CSCD

Abstract Full text About this article

With the increasing use of cloud computing, high energy consumption has become one of the major challenges in cloud data centers. Virtual Machine (VM) consolidation has been proven to be an efficient way to optimize energy consumption in data centers, and many research works have proposed to optimize VM consolidation. However, the performance of different algorithms is related with the characteristics of the workload and system status; some algorithms are suitable for Central Processing Unit (CPU)-intensive workload and some for web application workload. Therefore, an adaptive VM consolidation framework is necessary to fully explore the potential of these algorithms. Neat is an open-source dynamic VM consolidation framework, which is well integrated into OpenStack. However, it cannot conduct dynamic algorithm scheduling, and VM consolidation algorithms in Neat are few and basic, which results in low performance for energy saving and Service-Level Agreement (SLA) avoidance. In this paper, an Intelligent Neat framework (I-Neat) is proposed, which adds an intelligent scheduler using reinforcement learning and a framework manager to improve the usability of the system. The scheduler can select appropriate algorithms for the local manager from an algorithm library with many load detection algorithms. The algorithm library is designed based on a template, and in addition to the algorithms of Neat, I-Neat adds six new algorithms to the algorithm library. Furthermore, the framework manager helps users add self-defined algorithms to I-Neat without modifying the source code. Our experimental results indicate that the intelligent scheduler and these novel algorithms can effectively reduce energy consumption with SLA assurance.


menu
Abstract
Full text
Outline
About this article

I-Neat: An Intelligent Framework for Adaptive Virtual Machine Consolidation

Show Author's information Yanxin Liu, Yao Zhao, Jian Dong( ), Lianpeng Li, Chunpei Wang, Decheng Zuo
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China

Abstract

With the increasing use of cloud computing, high energy consumption has become one of the major challenges in cloud data centers. Virtual Machine (VM) consolidation has been proven to be an efficient way to optimize energy consumption in data centers, and many research works have proposed to optimize VM consolidation. However, the performance of different algorithms is related with the characteristics of the workload and system status; some algorithms are suitable for Central Processing Unit (CPU)-intensive workload and some for web application workload. Therefore, an adaptive VM consolidation framework is necessary to fully explore the potential of these algorithms. Neat is an open-source dynamic VM consolidation framework, which is well integrated into OpenStack. However, it cannot conduct dynamic algorithm scheduling, and VM consolidation algorithms in Neat are few and basic, which results in low performance for energy saving and Service-Level Agreement (SLA) avoidance. In this paper, an Intelligent Neat framework (I-Neat) is proposed, which adds an intelligent scheduler using reinforcement learning and a framework manager to improve the usability of the system. The scheduler can select appropriate algorithms for the local manager from an algorithm library with many load detection algorithms. The algorithm library is designed based on a template, and in addition to the algorithms of Neat, I-Neat adds six new algorithms to the algorithm library. Furthermore, the framework manager helps users add self-defined algorithms to I-Neat without modifying the source code. Our experimental results indicate that the intelligent scheduler and these novel algorithms can effectively reduce energy consumption with SLA assurance.

Keywords: reinforcement learning, cloud computing, OpenStack, dynamic Virtual Machine (VM) consolidation, Neat

References(32)

[1]
J. G. Koomey, Estimating total power consumption by servers in the U.S. and the world, PhD dissertation, Stanford University, Stanford, CA, USA, 2007.
[2]
S. Ismaeel, R. Karim, and A. Miri, Proactive dynamic virtual-machine consolidation for energy conservation in cloud data centres, J. Cloud Comput., vol. 7, no. 1, p. 10, 2018.
DOI Google Scholar
[3]
D. Meisner, B. T. Gold, and T. F. Wenisch, PowerNap: eliminating server idle power, in Proc. 14th Int. Conf. on Architectural Support for Programming Languages and Operating Systems, Washington, DC, USA, 2009, pp. 205-216.
DOI
[4]
W. Zhang, X. Chen, and J. Jiang, A multi-objective optimization method of initial virtual machine fault-tolerant placement for star topological data centers of cloud systems, Tsinghua Science and Technology, vol. 26, no. 1, pp. 95-111, 2021.
DOI Google Scholar
[5]
G. Pu, L. Wang, J. Shen, and F. Dong, A hybrid unsupervised clustering-based anomaly detection method, Tsinghua Science and Technology, vol. 26, no. 2, pp. 146-153, 2021.
DOI Google Scholar
[6]
M. Guazzone, C. Anglano, and M. Canonico, Exploiting VM migration for the automated power and performance management of green cloud computing systems, in Proc. 1st Int. Workshop on Energy Efficient Data Centers, Madrid, Spain, 2012, pp. 81-92.
DOI
[7]
F. F. Moghaddam, R. F. Moghaddam, and M. Cheriet, Carbon-aware distributed cloud: Multi-level grouping genetic algorithm, Cluster Comput., vol. 18, no. 1, pp. 477-491, 2015.
DOI Google Scholar
[8]
J. L. Berral, R. Gavaldà, and J. Torres, Power-aware multi-data center management using machine learning, in Proc. 2013 42nd Int. Conf. on Parallel Processing, Lyon, France, 2013, pp. 858-867.
DOI
[9]
J. T. Zhang, L. M. Zhang, H. J. Huang, X. Wang, C. L. Gu, and Z. X. He, A unified algorithm for virtual desktops placement in distributed cloud computing, Math. Probl. Eng., vol. 2016, p. 9084370, 2016.
DOI Google Scholar
[10]
A. Beloglazov and R. Buyya, Openstack neat: A framework for dynamic and energy-efficient consolidation of virtual machines in openstack clouds, Concurr. Comput. Pract. Experience, vol. 27, no. 5, pp. 1310-1333, 2015.
DOI Google Scholar
[11]
D. Basu, X. Y. Wang, Y. Hong, H. B. Chen, and S. Bressan, Learn-as-you-go with megh: Efficient live migration of virtual machines, IEEE Trans. Parallel Distrib. Syst., vol. 30, no. 8, pp. 1786-1801, 2019.
DOI Google Scholar
[12]
Z. Zhou, J. Abawajy, M. Chowdhury, Z. G Hu, K. Q. Li, H. B. Cheng, A. A. Alelaiwi, and F. M. Li, Minimizing SLA violation and power consumption in Cloud data centers using adaptive energy-aware algorithms, Future Gener. Comput. Syst., vol. 86, pp. 836-850, 2018.
DOI Google Scholar
[13]
Z. Zhou, Z. G. Hu, and K. Q. Li, Virtual machine placement algorithm for both energy-awareness and SLA violation reduction in cloud data centers, Sci. Programming, vol. 2016, p. 5612039, 2016.
DOI Google Scholar
[14]
S. S. Richard and G. B. Andrew, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998.
[15]
R. Kumar, N. Gupta, S. Charu, K. Jain, and S. K. Jangir, Open source solution for cloud computing platform using OpenStack, Int. J. Comput. Sci. Mob. Comput., vol. 3, no. 5, pp. 89-98, 2014.
Google Scholar
[16]
J. P. Mullerikkal and Y. Sastri, A comparative study of openstack and cloudstack, in Proc. 2015 5th Int. Conf. on Advances in Computing and Communications, Kochi, India, 2015, pp. 81-84.
DOI
[17]
E. Feller, C. Morin, and A. Esnault, A case for fully decentralized dynamic VM consolidation in clouds, in Proc. 4th IEEE Int. Conf. on Cloud Computing Technology and Science Proceedings, Taipei, China, 2012, pp. 26-33.
DOI
[18]
A. Beloglazov, Energy-efficient management of virtual machines in data centers for cloud computing, PhD dissertation, The University of Melbourne, Melbourne, Australia, 2013.
[19]
L. P. Li, J. Dong, D. C. Zuo, and Y. Zhao, Minimizing SLA violation and power consumption in cloud data centers using host state 3-order Markov chain model, in Proc. 2019 IEEE 21st Int. Conf. on High Performance Computing and Communications; IEEE 17thInt. Conf. on Smart City; IEEE 5th Int. Conf. on Data Science and Systems, Zhangjiajie, China, 2019, pp. 2200-2205.
DOI
[20]
L. P. Li, J. Dong, D. C. Zuo, and J. X. Liu, SLA-aware and energy-efficient VM consolidation in cloud data centers using host states naive Bayesian prediction model, in Proc. 2018 IEEE Int. Conf. on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications, Melbourne, Australia, 2018, pp. 80-87.
DOI
[21]
F. Farahnakian, P. Liljeberg, and J. Plosila, LiRCUP: Linear regression based CPU usage prediction algorithm for live migration of virtual machines in data centers, in Proc. 2013 39th Euromicro Conf. on Software Engineering and Advanced Applications, Santander, Spain, 2013, pp. 357-364.
DOI
[22]
A. Horri, M. S. Mozafari, and G. Dastghaibyfard, Novel resource allocation algorithms to performance and energy efficiency in cloud computing, J. Supercomput., vol. 69, no. 3, pp. 1445-1461, 2014.
DOI Google Scholar
[23]
A. Beloglazov and R. Buyya, Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers, Concurr. Comput. Pract. Experience, vol. 24, no. 13, pp. 1397-1420, 2012.
DOI Google Scholar
[24]
A. Beloglazov, J. Abawajy, and R. Buyya, Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing, Future Gener. Comput. Syst., vol. 28, no. 5, pp. 755-768, 2012.
DOI Google Scholar
[25]
X. Fu and C. Zhou, Virtual machine selection and placement for dynamic consolidation in cloud computing environment, Front. Comput. Sci., vol. 9, no. 2, pp. 322-330, 2015.
DOI Google Scholar
[26]
Z. Zhou, Z. G. Hu, T. Song, and J. Y. Yu, A novel virtual machine deployment algorithm with energy efficiency in cloud computing, J. Cent. South Univ, vol. 22, no. 3, pp. 974-983, 2015.
DOI Google Scholar
[27]
F. F. Moges and S. L. Abebe, Energy-aware VM placement algorithms for the OpenStack Neat consolidation framework, J. Cloud Comput., vol. 8, no. 1, p. 2, 2019.
DOI Google Scholar
[28]
X. B Fan, W. D. Weber, and L. A. Barroso, Power provisioning for a warehouse-sized computer, in Proc. 34th Annual Int. Symposium on Computer Architecture, San Diego, CA, USA, 2007, pp. 13-23.
DOI
[29]
M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York, NY, USA: John Wiley & Sons, 1994.
DOI
[30]
Z. H. Han, H. S. Tan, G. H. Chen, R. Wang, Y. F. Chen, and F. C. M. Lau, Dynamic virtual machine management via approximate Markov decision process. in Proc. 35th Annual IEEE Int. Conf. on Computer Communications, San Francisco, CA, USA, 2016, pp. 1-9.
DOI
[31]
T. E. Kurtz, The challenge of the computer utility by D. F. parkhill, Am. Sci., vol. 55, no. 4, p. 494A, 1967.
Google Scholar
[32]
K. S. Park and V. S. Pai, CoMon: A mostly-scalable monitoring system for PlanetLab, ACM SIGOPS Oper. Syst. Rev., vol. 40, no. 1, pp. 65-74, 2006.
DOI Google Scholar
Publication history
Copyright
Rights and permissions

Publication history

Received: 23 May 2020
Revised: 24 July 2020
Accepted: 03 September 2020
Published: 17 August 2021
Issue date: February 2022

Copyright

© The author(s) 2022

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return