From Algorithm to Expert: RLHF-Guided Vision-Language Model for 3D-EEM Fluorescence Spectroscopy Matching

Existing methods for tracing water pollution sources typically integrate three-dimensional excitation-emission matrix (3D-EEM) fluorescence spectroscopy with similarity-based matching algorithms. However, these approaches exhibit high error rates in borderline cases and necessitate expert manual review, which limits scalability and introduces inconsistencies between algorithmic outputs and expert judgment. To address these limitations, we propose a large vision-language model (VLM) designed as an “expert agent” to automatically refine similarity scores, ensuring alignment with expert decisions and overcoming key application bottlenecks. The model consists of two core components: (1) rule-based similarity calculation module generate initial spectral similarity scores, and (2) pre-trained large vision-language model fine-tuned via supervised learning and reinforcement learning with human feedback (RLHF) to emulate expert assessments. To facilitate training and evaluation, we introduce two expert-annotated datasets, Spec1k and SpecReason, which capture both quantitative corrections and qualitative reasoning patterns, allowing the model to emulate expert decision-making processes. Experimental results demonstrate that our method achieves 81.45% source attribution accuracy, 38.24% higher than rule-based and machine learning baselines. Real-world deployment further validates its effectiveness.