Quantitative framework for causality evaluation in data-driven model: A case study of a cooling tower system

Data-driven models have been widely adopted owing to their high predictive accuracy. However, model validation is typically conducted using error-based metrics such as coefficient of variation of the root mean square error (cvRMSE), normalized mean bias error (NMBE), mean absolute error (MAE), and coefficient of determination (R²), without considering the causal relationships between input and output variables. In this context, the present study proposes a methodology for evaluating the causality of data-driven models. A causality evaluation metric was formulated by quantifying the directional consistency—termed causal consistency, ranging from 0.00 to 1.00—between the physics-based model and the data-driven model, based on counterfactual data generated through variations in the input variables. The target system was a cooling tower system in a large industrial building. First, two data-driven cooling tower models—an artificial neural network (ANN) and a transfer learning (TL) model—were developed. Subsequently, their predictive accuracy was assessed using the validation data, their qualitative extrapolation ability was examined, and their causality was evaluated using counterfactual data. In terms of accuracy, both data-driven models exhibited similar predictive performance for cooling water outlet temperature, achieving MAE values of 0.7 ℃ and R² values of 0.96. However, regarding the extrapolation ability, the ANN displayed unreasonable temperature trends with respect to the cooling water volumetric flow rate and the number of operating cooling tower fans, whereas the TL showed physically consistent temperature trends. In terms of causality, the ANN exhibited a wide range of causal consistency across all input variables, ranging from 0.03 to 1.00, indicating uncertainty in capturing causal relationships. The TL model exhibited overall higher causal consistency compared to the ANN, ranging from 0.87 to 1.00, but still showed some uncertainty in modeling causal relationships depending on the combinaions of input variables. By evaluating the data-driven models in terms of both accuracy and causality, their high fidelity for real-world applications can be more reliably demonstrated.