Optimizing the hyper-parameters of deep reinforcement learning for building control

Buildings are a major energy consumer and carbon emitter, therefore it is important to improve building energy efficiency to achieve our sustainable development goal. Deep reinforcement learning (DRL), as an advanced building control method, demonstrates great potential for energy efficiency optimization and improved occupant comfort. However, the performance of DRL is highly sensitive to hyper-parameters, and selecting inappropriate hyper-parameters may lead to unstable learning or even failure. This study aims to investigate the design and application of DRL in building energy system control, with a specific focus on improving the performance of DRL controllers through hyper-parameter optimization (HPO) algorithms. It also aims to provide quantitative evaluation and adaptive validation of these optimized controllers. Two widely used algorithms, deep deterministic policy gradient (DDPG) and soft actor-critic (SAC), are used in the study and their performance is evaluated in different building environments based on the BOPTEST virtual testbed. One of the focuses of the study is to compare various HPO techniques, including tree-structured Parzen estimator (TPE), covariance matrix adaptation evolution strategy (CMA-ES), and combinatorial optimization methods, to determine the efficacy of different hyper-parameter optimization methods for DRL. The study enhances HPO efficiency through parallel computation and conducts a comprehensive quantitative assessment of the optimized DRL controllers, considering factors such as reduced energy consumption and improved comfort. The results show that the HPO algorithms significantly improve the performance of the DDPG and SAC controllers. A reduction of 56.94% and 68.74% in thermal discomfort is achieved, respectively. Additionally, the study demonstrates the applicability of the HPO-based approach for enhancing DRL controller performance across diverse building environments, providing valuable insights for the design and optimization of building DRL controllers.