Medical visual question answering (VQA) is a key medical AI challenge, but scarce data limits progress. Current methods prioritize more pre-training data, overlooking medical data's inherent constraints (ethics, privacy, specialization) which cause slower accumulation of data than public web data. Sole reliance on more data risks rapid performance plateaus. To address these challenges, this paper proposes the cross-modality discriminative pattern identification model (CMDPI), which fundamentally rethinks training methodologies to provide a data-efficient framework for medical VQA. During pre-training, CMDPI identifies inherent biases in conventional techniques under conditions of data scarcity and introduces a co-regularization approach that integrates multiple pretraining techniques for regularization to enhance model generalizability. For fine-tuning, a difference reconstruction mechanism is proposed that effectively preserves unique discriminative features, and head mixup is introduced to further remedy the issue of overfitting. Experimental results demonstrate that CMDPI achieves performance comparable to or surpassing existing methods while requiring substantially less pre-training data. Our work shows the viability of optimizing training paradigms rather than pursuing indiscriminate data scaling for advancing medical VQA systems.
- Article type
- Year
- Co-author
Open Access
Research Article
Issue
Open Access
Issue
As a deep learning network with an encoder-decoder architecture, UNet and its series of improved versions have been widely used in medical image segmentation with great applications. However, when used to segment targets in 3D medical images such as magnetic resonance imaging (MRI), computed tomography (CT), these models do not model the relevance of images in vertical space, resulting in poor accurate analysis of consecutive slices of the same patient. On the other hand, the large amount of detail lost during the encoding process makes these models incapable of segmenting small-scale tumor targets. Aiming at the scene of small-scale target segmentation in 3D medical images, a fully new neural network model SUNet++ is proposed on the basis of UNet and UNet++. SUNet++ improves the existing models mainly in three aspects: 1) the modeling strategy of slice superposition is used to thoroughly excavate the three dimensional information of the data; 2) by adding an attention mechanism during the decoding process, small scale targets in the picture are retained and amplified; 3) in the up-sampling process, the transposed convolution operation is used to further enhance the effect of the model. In order to verify the effect of the model, we collected and produced a dataset of hyperintensity MRI liver-stage images containing over 400 cases of liver nodules. Experimental results on both public and proprietary datasets demonstrate the superiority of SUNet++ in small-scale target segmentation of three-dimensional medical images.
Open Access
Issue
Wireless sensors are deployed widely to monitor space, emergent events, and disasters. Collected real-time sensory data are precious for completing rescue missions quickly and efficiently. Detecting isolate safe areas is significant for various applications of event and disaster monitoring since valuable real-time information can be provided for the rescue crew to save persons who are trapped in isolate safe areas. We propose a centralized method to detect isolate safe areas via discovering holes in event areas. In order to shorten the detection delay, a distributed isolate safe area detection method is studied. The distributed method detects isolate safe areas during the process of event detection. Moreover, detecting isolate safe areas in a building is addressed particularly since the regular detecting method is not applicable. Our simulation results show that the distributed method can detect all isolate safe areas in an acceptable short delay.
京公网安备11010802044758号