Interactive intention recognition of medical image reviewing based on semantic understanding of mouse trajectory

Due to the increasing number and complexity of diagnostic imaging studies, radiologists are now faced with a heavy workload in their daily image reviewing task. It has become a major concern to reduce the burden of image reviewing process for radiologists and help them improve the efficiency of diagnosis. The recognition and understanding of interactive intent during radiologists image review can be used to develop new intelligent review systems that can provide adaptive tools and optimize user workflow to help radiologists perform digital image review tasks more efficiently and accurately. Currently, radiologists still primarily use a graphical user interface based on mouse input for medical image reviewing. To this end, this paper proposes an intention understanding method of medical image reviewing based on mouse interaction trajectories. Firstly, semantic segmentation of mouse interaction trajectories is performed, and then the semantic annotation of mouse interaction behaviors is implemented based on logical reasoning and supervised learning for the segmented trajectory subsequences. The method is validated on a self-built mouse trajectory dataset under the TIoU threshold of [0.3∶0.8∶0.1], and the average recall, precision and F-score of the method reach 86.7%, 83.1% and 85.4%, respectively. The results show that the method can effectively identify the interactive intention based on the implicit mouse interaction data, which confirms the feasibility and effectiveness of the interactive intention recognition method based on semantic understanding of mouse interaction trajectories in the field of medical images assisted reviewing research.