Publications
Article type
Sort:
Open Access Article Issue
MDD: A Unified Multimodal Deep Learning Approach for Depression Diagnosis Based on Text and Audio Speech
Computers, Materials & Continua 2024, 81(3): 4125-4147
Published: 31 December 2024
Abstract PDF (893.2 KB) Collect
Downloads:28

Depression is a prevalent mental health issue affecting individuals of all age groups globally. Similar to other mental health disorders, diagnosing depression presents significant challenges for medical practitioners and clinical experts, primarily due to societal stigma and a lack of awareness and acceptance. Although medical interventions such as therapies, medications, and brain stimulation therapy provide hope for treatment, there is still a gap in the efficient detection of depression. Traditional methods, like in-person therapies, are both time-consuming and labor-intensive, emphasizing the necessity for technological assistance, especially through Artificial Intelligence. Alternative to this, in most cases it has been diagnosed through questionnaire-based mental status assessments. However, this method often produces inconsistent and inaccurate results. Additionally, there is currently a lack of a comprehensive diagnostic framework that could be effective achieving accurate and robust diagnostic outcomes. For a considerable time, researchers have sought methods to identify symptoms of depression through individuals’ speech and responses, leveraging automation systems and computer technology. This research proposed MDD which composed of multimodal data collection, preprocessing, and feature extraction (utilizing the T5 model for text features and the WaveNet model for speech features). Canonical Correlation Analysis (CCA) is then used to create correlated projections of text and audio features, followed by feature fusion through concatenation. Finally, depression detection is performed using a neural network with a sigmoid output layer. The proposed model achieved remarkable performance, on the Distress Analysis Interview Corpus-Wizard (DAIC-WOZ) dataset, it attained an accuracy of 92.75%, precision of 92.05%, and recall of 92.22%. For the E-DAIC dataset, it achieved an accuracy of 91.74%, precision of 90.35%, and recall of 90.95%. Whereas, on CD-III dataset (Custom Dataset for Depression), the model demonstrated an accuracy of 93.05%, precision of 92.12%, and recall of 92.85%. These results underscore the model’s robust capability in accurately diagnosing depressive disorder, demonstrating the efficacy of advanced feature extraction methods and improved classification algorithm.

Open Access Article Issue
Enhancing Phoneme Labeling in Dysarthric Speech with Digital Twin-Driven Multi-Modal Architecture
Computers, Materials & Continua 2025, 84(3): 4825-4849
Published: 30 July 2025
Abstract PDF (1.3 MB) Collect
Downloads:29

Digital twin technology is revolutionizing personalized healthcare by creating dynamic virtual replicas of individual patients. This paper presents a novel multi-modal architecture leveraging digital twins to enhance precision in predictive diagnostics and treatment planning of phoneme labeling. By integrating real-time images, electronic health records, and genomic information, the system enables personalized simulations for disease progression modeling, treatment response prediction, and preventive care strategies. In dysarthric speech, which is characterized by articulation imprecision, temporal misalignments, and phoneme distortions, existing models struggle to capture these irregularities. Traditional approaches, often relying solely on audio features, fail to address the full complexity of phoneme variations, leading to increased phoneme error rates (PER) and word error rates (WER). To overcome these challenges, we propose a novel multi-modal architecture that integrates both audio and articulatory data through a combination of Temporal Convolutional Networks (TCNs), Graph Convolutional Networks (GCNs), Transformer Encoders, and a cross-modal attention mechanism. The audio branch of the model utilizes TCNs and Transformer Encoders to capture both short- and long-term dependencies in the audio signal, while the articulatory branch leverages GCNs to model spatial relationships between articulators, such as the lips, jaw, and tongue, allowing the model to detect subtle articulatory imprecisions. A cross-modal attention mechanism fuses the encoded audio and articulatory features, enabling dynamic adjustment of the model’s focus depending on input quality, which significantly improves phoneme labeling accuracy. The proposed model consistently outperforms existing methods, achieving lower Phoneme Error Rates (PER), Word Error Rates (WER), and Articulatory Feature Misclassification Rates (AFMR). Specifically, across all datasets, the model achieves an average PER of 13.43%, an average WER of 21.67%, and an average AFMR of 12.73%. By capturing both the acoustic and articulatory intricacies of speech, this comprehensive approach not only improves phoneme labeling precision but also marks substantial progress in speech recognition technology for individuals with dysarthria.

Open Access Article Issue
OMD-RAS: Optimizing Malware Detection through Comprehensive Approach to Real-Time and Adaptive Security
Computers, Materials & Continua 2025, 84(3): 5995-6014
Published: 30 July 2025
Abstract PDF (1.3 MB) Collect
Downloads:16

Malware continues to pose a significant threat to cybersecurity, with new advanced infections that go beyond traditional detection. Limitations in existing systems include high false-positive rates, slow system response times, and inability to respond quickly to new malware forms. To overcome these challenges, this paper proposes OMD-RAS: Implementing Malware Detection in an Optimized Way through Real-Time and Adaptive Security as an extensive approach, hoping to get good results towards better malware threat detection and remediation. The significant steps in the model are data collection followed by comprehensive preprocessing consisting of feature engineering and normalization. Static analysis, along with dynamic analysis, is done to capture the whole spectrum of malware behavior for the feature extraction process. The extracted processed features are given with a continuous learning mechanism to the Extreme Learning Machine model of real-time detection. This OMD-RAS trains quickly and has great accuracy, providing elite, advanced real-time detection capabilities. This approach uses continuous learning to adapt to new threats—ensuring the effectiveness of detection even as strategies used by malware may change over time. The experimental results showed that OMD-RAS performs better than the traditional approaches. For instance, the OMD-RAS model has been able to achieve an accuracy of 96.23% and massively reduce the rate of false positives across all datasets while eliciting a consistently high rate of precision and recall. The model’s adaptive learning reflected enhancements on other performance measures—for example, Matthews Correlation Coefficients and Log Loss.

Total 3