To evaluate the diagnostic performance of human and artificial intelligence (AI)-guided graders in a community eye screening program and assess follow-up adherence over six years in Jakarta, Indonesia.

This retrospective study analyzed patients screened through the EyeCheck^TM program (2018–2024). Human graders (Eye Scan^TM, 2018–2019) evaluated multiple anterior and posterior segment images; AI systems (DR. NOON^TM and Vuno^TM, 2020–2024) assessed single posterior segment images only—constituting a non-concurrent comparison. Diagnostic accuracy was calculated for patients (6.2% of those with abnormal findings) who attended follow-up, using ophthalmologists’ diagnoses as the reference standard.

Of 18628 patients screened, 9262 (49.7%) had abnormal findings, but only 574 (6.2%) sought follow-up care. Among the 574 patients who completed follow-up examination, 286 (49.8%) were male and 288 (50.2%) were female. The median age was 25y (range: 7–81y), with the majority (72.3%) aged 17–50y. Among these, human graders demonstrated higher sensitivity (93.8% vs 72.4%) but lower specificity (17.2% vs 70.1%) and accuracy (56.3% vs 70.8%) compared to AI graders. AI systems provided real-time results, whereas human grading required 3–7 business days. The most frequent diagnoses were refractive error (51.4%), cataract (13.1%), and dry eye syndrome (9.1%).

In this non-concurrent comparison, AI-guided graders show higher specificity and accuracy while human graders achieved higher sensitivity. These findings should be interpreted cautiously given selection bias from low follow-up rates, temporal confounders, and differences in imaging protocols. The critically low follow-up rate underscores that effective screening requires robust linkage-to-care systems. Prospective studies with concurrent comparison are needed.