Heart-Net: A Multi-Modal Deep Learning Approach for Diagnosing Cardiovascular Diseases

Heart disease remains a leading cause of morbidity and mortality worldwide, highlighting the need for improved diagnostic methods. Traditional diagnostics face limitations such as reliance on single-modality data and vulnerability to apparatus faults, which can reduce accuracy, especially with poor-quality images. Additionally, these methods often require significant time and expertise, making them less accessible in resource-limited settings. Emerging technologies like artificial intelligence and machine learning offer promising solutions by integrating multi-modality data and enhancing diagnostic precision, ultimately improving patient outcomes and reducing healthcare costs. This study introduces Heart-Net, a multi-modal deep learning framework designed to enhance heart disease diagnosis by integrating data from Cardiac Magnetic Resonance Imaging (MRI) and Electrocardiogram (ECG). Heart-Net uses a 3D U-Net for MRI analysis and a Temporal Convolutional Graph Neural Network (TCGN) for ECG feature extraction, combining these through an attention mechanism to emphasize relevant features. Classification is performed using Optimized TCGN. This approach improves early detection, reduces diagnostic errors, and supports personalized risk assessments and continuous health monitoring. The proposed approach results show that Heart-Net significantly outperforms traditional single-modality models, achieving accuracies of 92.56% for Heartnet Dataset Ⅰ (HNET-DSⅠ), 93.45% for Heartnet Dataset Ⅱ (HNET-DSⅡ), and 91.89% for Heartnet Dataset Ⅲ (HNET-DSⅢ), mitigating the impact of apparatus faults and image quality issues. These findings underscore the potential of Heart-Net to revolutionize heart disease diagnostics and improve clinical outcomes.