Confidence-driven HVAC component recognition integrating LLMs and computer vision

Accurate recognition of components in heating, ventilation, and air conditioning (HVAC) drawings is crucial for building information digitization. Existing recognition methods lack the ability to integrate multi-source information, such as component symbols, textual annotations, connection relationships, and domain knowledge, into a unified reasoning process. As a result, they exhibit poor performance when applied to HVAC drawings with diverse and non-standardized component symbols. To address these challenges, this paper proposes a confidence-driven method integrating large language models (LLMs) with computer vision (CV). A structured representation approach is designed to preserve spatial and semantic relationships in HVAC drawings, enabling the LLM to reason components beyond raw image inputs. A confidence-weighted mechanism assigns confidence scores to multi-source information, allowing high-confidence evidence to iteratively guide the refinement of uncertain component classifications. A dynamic knowledge infusion strategy is introduced to integrate HVAC-specific knowledge into the reasoning loop, allowing the reasoning process to produce domain-consistent component recognition. The proposed method is compared to conventional CV models on open-access HVAC drawing datasets. Results show that the true positive rate for every component category exceeds 0.89. Significant improvements of 46.4% in recall, 14.5% in precision, and 36.0% in F1-score over the baseline. Additionally, the model achieves a macro-average area under the curve (AUC) of 0.977, verifying its robust discrimination capability. The proposed framework effectively enhances the accuracy and robustness of HVAC component recognition, providing a scalable solution for automated building digitization.