Sort:
Open Access Issue
DPN: Dynamics Priori Networks for Radiology Report Generation
Tsinghua Science and Technology 2025, 30(2): 600-609
Published: 09 December 2024
Abstract PDF (5.1 MB) Collect
Downloads:7

Radiology report generation is of significant importance. Unlike standard image captioning tasks, radiology report generation faces more pronounced visual and textual biases due to constrained data availability, making it increasingly reliant on prior knowledge in this context. In this paper, we introduce a radiology report generation network termed Dynamics Priori Networks (DPN), which leverages a dynamic knowledge graph and prior knowledge. Concretely, we establish an adaptable graph network and harness both medical domain knowledge and expert insights to enhance the model’s intelligence. Notably, we introduce an image-text contrastive module and an image-text matching module to enhance the quality of the generated results. Our method is evaluated on two widely available datasets: X-ray collection from Indiana University (IU X-ray) and Medical Information Mart for Intensive Care, Chest X-Ray (MIMIC-CXR), where it demonstrates superior performance, particularly excelling in critical metrics.

Open Access Issue
Classification of Medical Image Notes for Image Labeling by Using MinBERT
Tsinghua Science and Technology 2023, 28(4): 613-627
Published: 06 January 2023
Abstract PDF (14.3 MB) Collect
Downloads:517

The lack of labeled image data poses a serious challenge to the application of artificial intelligence (AI) in medical image diagnosis. Medical image notes contain valuable patient information that could be used to label images for machine learning tasks. However, most image note texts are unstructured with heterogeneity and short-paragraph characters, which fail traditional keyword-based techniques. We utilized a deep learning approach to recover missing labels for medical image notes automatically by using a combination of deep word embedding and deep neural network classifiers. Bidirectional encoder representations from transformers trained on medical image notes corpus (MinBERT) were proposed. We applied the proposed techniques to two typical classification tasks: Medical image type identification and clinical diagnosis identification. The two methods significantly outperformed baseline methods and presented high accuracies of 99.56 % and 99.72 % in image type identification and of 94.56 % and 92.45 % in clinical diagnosis identification. Visualization analysis further indicated that word embedding could efficiently capture semantic similarities and regularities across diverse expressions. Results indicated that our proposed framework could accurately recover the missing label information of medical images through the automatic extraction of electronic medical record information. Hence, it could serve as a powerful tool for exploring useful training data in various medical AI applications.

Total 2