Clinical evaluation of a large language model based intelligent pre-consultation system in traditional Chinese medicine: A tertiary hospital study

To evaluate the utilization, completion quality, and physicians’ perceptions of a large language model (LLM)-based intelligent pre-consultation system in traditional Chinese medicine (TCM) clinical settings and to examine its role in the emerging physician–artificial intelligence (AI)-patient triadic model, thereby providing evidence for future system optimization.

A total of 21731 pre-consultation records from a tertiary TCM hospital between July 2025 and February 2026 were analyzed for patient demographics and completion status. In addition, 42 clinical physicians completed a clinical effectiveness evaluation scale for the pre-consultation system. Descriptive statistics, reliability and validity analyses, and subgroup comparisons were performed using the Mann–Whitney U and Kruskal–Wallis H tests. Qualitative data from physician interviews were thematically analyzed to complement the quantitative findings.

The patient completion rate was 77.10% (16754/21731). The physician survey scale demonstrated excellent reliability (Cronbach’s α = 0.954) and validity (Kaiser–Meyer–Olkin = 0.825). The overall mean physician rating was 3.62 ± 0.78, with the structural dimension (information collection) scoring the highest (3.77 ± 0.72) and the outcome dimension (medical output and perception) scoring the lowest (3.49 ± 0.86). Among the pre-consultation content sections, allergy history received the highest rating (80.83/100), whereas the chief complaints received the lowest rating (70.29/100). Subgroup comparisons revealed significant age–related differences in satisfaction (H = 10.006, P = .019), with physicians aged 31–40 years reporting lower satisfaction (median = 3.00) and those aged 51–60 years reporting higher satisfaction (median = 4.00). No significant differences were observed across specialties, sex, professional titles, or digital tool preferences.

The LLM-based pre-consultation system demonstrated preliminary physician acceptance, particularly for information collection within the TCM hospital setting. However, substantial improvements are still required in chief complaint acquisition, decision support, workflow integration, and documentation burden reduction. Future optimization should prioritize clinically actionable information extraction, stronger integration of TCM diagnostic logic, and improved accessibility for elderly users.