A brain hemorrhage segmentation method based on text prompts

Computed tomography (CT) is currently the most common diagnostic method for brain hemorrhages. Rapidly identifying the location and shape of a brain hemorrhage using deep learning models is of great clinical significance for locating its area and determining its cause. However, most of the mainstream medical segmentation models encounter under-segmentation problems during the segmentation of brain hemorrhages, especially in the region near the skull or when the hemorrhage volume is small. For this reason, this paper proposes a brain hemorrhage segmentation method based on multimodal text representation, which uses a contrastive language-image pretraining (CLIP) model to encode the designed prompts for representation. The text encoder in CLIP can represent information in prompts about the relative location, inclusion relation and other parameters. The representation can then be combined with the U-net to perform the brain hemorrhage segmentation task. The method proposed in this paper uses flexible text prompts to address the problem that some parts of a brain hemorrhage are difficult to segment, thereby enhancing the precision of segmentation. The medical segmentation performance metrics (Dice coefficient) of our method reached 43.3% and 58.8% respectively, when using the publicly available Brain Hemorrhage Segmentation Dataset (BHSD) and the brain hemorrhage dataset for an individual hospital. The improved performance of our method compared with other single medical segmentation models provides strong evidence of its effectiveness.