Building a Self-Evolving Digital Twin System with Bayesian Optimization and Deep Reinforcement Learning for Complex Equipment Optimization and Control

Digital twin self-evolution means that digital twin can fuse online sensor data from physical entity to evolve itself, hence improve credibility of the model and represent the physical entity faithfully. There is an urgent need to address the fundamental theories and techniques on how to build such a self-evolving digital twin system for complex equipment optimization and control. Focused on this problem, integrating Bayesian optimization theory and deep reinforcement learning (DRL), this paper proposes a method to build dynamic self-evolving equipment digital twin system for optimal control. First, considering the complexity of current equipment and real-time requirement of dynamic self-evolution scenario, we design digital twin dynamic self-evolution engine using Bayesian optimization theory, which can continuously integrate real-time sensing data, adapt to the dynamic uncertainty changes of physical equipment, so as to improve the credibility of digital twin. Then, a decision-making agent based on DRL algorithm soft actor-critic is designed, which can interact with equipment digital twin in virtual space. When the digital twin model evolves, the agent follows and continues to learn and update itself through online fine-tuning strategy, so as to continuously improve the equipment optimization control performance. Finally, the feasibility and effectiveness of the proposed method are verified by two simulation cases.