Causal graph-aided reinforcement learning for HVAC energy consumption optimization

Scheduling the load of multiple chillers and distributing cooling capacity across multiple zones simultaneously remains challenging for large building HVAC (heating, ventilation and air-conditioning) systems. Meanwhile, the dynamic cooling load faced by HVAC systems places high demands on the generalization of energy optimization methods. To this end, a causal graph-aided reinforcement learning energy consumption optimization approach is proposed for large building HVAC systems. Firstly, an HVAC system with multiple chillers and multiple zones is analyzed, and the causal graph of the HVAC system is obtained. Secondly, a causal graph-aided network structure is designed to extract causal features between nodes in the causal graph. Causal structural information can help reinforcement learning achieve highly generalizable decisions and improve learning speed. Thirdly, an improved Soft Actor-Critic method is proposed with double experience replay mechanism and bias-based state augmentation to improve the resilience to external disturbances. Lastly, comparison experiments and ablation experiments based on three scenarios are conducted. Compared to other baseline reinforcement learning methods, the average reward of the proposed method increased by about 10%. Consequently, the proposed method achieves an average energy savings of 6% in different scenarios without sacrificing the indoor comfort. Theoretical analysis and experimental results confirm that the proposed method offers significant performance advantages in addressing dynamic supply-demand matching and energy consumption optimization for large-scale HVAC systems.