Construction and Validation of Outcome Indicators Set for Hybrid Type 1 Implementation Research: Based on the SMART Study

Using the hybrid type 1 implementation study "The Shared Medical Appointment for diabetes in China: an optimization trial(SMART)" as an example, this study introduces the methods of constructing and validating an outcome indicator set for hybrid type 1 studies.

Based on principles such as comprehensiveness and operability, we selected the RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework, the Implementation Outcomes Framework (IOF), and the PRECEDE-PROCEED model. After comparison and screening, an outcome evaluation indicator item pool was formed. Two rounds of expert consultation were conducted, and the content validity was evaluated using the item-level content validity index (I-CVI), coefficient of variation (CV), and adjusted Kappa value, combined with experts' qualitative opinions and on-site implementation to optimize the indicators.

This study integrated the RE-AIM framework and three pre-evaluation dimensions from the IOF, forming an initial item pool for shared medical appointments (SMA) that included 8 dimensions (acceptability, appropriateness, feasibility, reach, effectiveness, adoption, implementation, and maintenance), 14 secondary indicators, and 24 tertiary indicators. The response rate of the two rounds of expert consultation was 100%, with an expert authority coefficient of 0.90. The I-CVI ranged from 0.75 to 1.00, the CV from 0 to 0.32, and the adjusted Kappa from 0.75 to 1.00, indicating good content validity and high expert consensus. After two rounds of consultation, one secondary indicator and three tertiary indicators were deleted, one new secondary indicator was added, and the dimension attribution of one indicator was adjusted. Following pilot implementation and stakeholder feedback, two tertiary indicators were added, four were deleted, and the measurement tool of one indicator was adjusted. The finalized implementation outcome indicator set for the SMART study comprises 8 dimensions, 14 secondary indicators, and 19 tertiary indicators.

The outcome indicator set constructed in this study demonstrates good validity and operability, and the construction and validation process can serve as a reference for other researchers. Different implementation outcome frameworks have different applicable scenarios; researchers should select appropriate frameworks or integrate them according to their own needs to comprehensively evaluate outcomes and enhance the systematization and comparability of research.