As a primary cause of road traffic injuries, fatigued driving requires efficient and accurate detection to improve traffic safety. Traditional single-signal approaches face limitations in capturing fatigue states, including high data collection intrusiveness, complex data structures, difficulty in real-time prediction, and low accuracy. Integrating information from different modalities has emerged as a new direction for development.

This study developed a multimodal fatigue detection model for drivers by using electrocardiogram signals, vehicle trajectory data, and driver facial video data collected during long-term real-vehicle driving experiments. For ECG signal processing, an improved two-step adaptive filtering method was adopted for denoising, followed by time-domain and frequency-domain analyses to extract the driver's ECG feature set. For vehicle trajectory data, the quartile method was first used to remove outliers, backward filling was applied to fill in missing values, and two-dimensional discrete wavelet analysis was then employed for data denoising. For facial image data, the LabelMe annotation tool was used to construct a personalized training dataset by manually annotating each driver's face with at least 300 images per subject. The YOLOv8 deep learning model was then fine-tuned and trained on this dataset, and the optimal model weights were saved. Next, the optimized model was used to automatically annotate the remaining unlabeled images, and the Hopenet algorithm was applied to extract head pose angles from the cropped facial regions. The model incorporated modules for data processing, feature extraction, feature selection, and fatigue prediction, utilizing a self-attention mechanism to capture long-term dependencies and generate predictive outputs.

The model achieved a maximum accuracy of 97.89% in predicting fatigue state categories, with overall recall and F1 scores exceeding 80%, demonstrating strong predictive accuracy. The detection model utilizing data from all three modalities served as the control group, while six experimental groups were formed using a single modality or a combination of two modalities. The experiments revealed that the fatigued driving detection model employing data from all three modalities achieved optimal performance across various metrics.

This study demonstrates that the proposed model successfully integrates information from different modalities, exhibits high accuracy and adaptability, and enhances the assurance of driving safety. Specifically, this model outperforms all comparison models in terms of accuracy, precision, recall, and F1 score, achieving the best performance in fatigued driving detection tasks, and maintains stable detection performance even under complex data structures, diverse sources, large time spans, average-quality facial images, and individual driver differences. The model achieves a maximum accuracy of 97.89% in identifying fatigue states, indicating that it correctly learns and recognizes fatigue patterns. Furthermore, models using a single modality or a combination of two modalities yield lower evaluation metrics than the three-modality model. Moreover, two-modality models consistently outperform single-modality variants, confirming that ECG signals, trajectory data, and facial images contribute positively to accurate fatigued driving detection.

In recent years, under the influence of strong wind, trees and walls collapse, objects fall, and other situations occur from time to time, which seriously affect the safety of community residents. In traditional emergency rescue, the background wind field at the disaster site is unknown, and the accuracy of accident development assessment is affected. In the case of fire, gas leakage, strong wind, and other disasters, decision-makers and rescue teams cannot accurately locate the dangerous areas in the community because of their inability to rapidly obtain accurate background wind field information, which affects the accuracy of the judgment of the disaster scope and development trend. Key dangerous areas in the community under strong wind need to be identified.

In this study, the wind field of a community in the Shijingshan District of Beijing was taken as an example to conduct scene modeling. To generate the database of the community wind field, the wind field was generated by OpenFOAM, and a shell script was used for a batch of simulations. The speed at the feature points obtained by k-means clustering served as the input, and the wind field served as the output to train the neural network. The selected community feature points could represent the wind field information of the community. The feature point selection and neural network modeling were continuously optimized based on the training and prediction results until the accuracy met the requirements.

Taking the field data of 6 681 points predicted by 10 feature points as an example, the model training test results of 7 917 training wind fields and 2 026 testing wind fields were as follows: The average relative errors of the predicted values of speeds above 1 m/s in the x- and y-axes were 5.8% and 6.2%, respectively. Among them, the average relative error of model prediction between 1 m/s and 2 m/s is 11.9%, for model prediction between 2 m/s and 5 m/s was 6.0%, for model prediction between 5 m/s and 10 m/s was 3.2%, and for model prediction above 10 m/s was 3.5%.

Compared with the numerical simulation technology, the neural network model can rapidly generate the background wind field of the community based on the field location data. Compared with the time of the numerical simulation, the time of the neural network model to generate a field is significantly reduced. Unlike the existing neural network model, the proposed model takes actual community points as the feature points for model training and prediction, enabling the installation of sensors and the prediction of real-time wind fields. Therefore, people can organize risk prevention and emergency rescue according to the background wind field, which is of great significance for maintaining community safety.

As natural disasters have major impacts on urban security, research on the behavior of individuals in disaster scenarios is crucial to support emergency decision-making. However, disaster occurrence is highly uncertain, making it difficult to collect data in disaster scenarios.

This paper proposes a stress induction method for disaster scenarios that can be conducted in a laboratory, that is, to induce the stress reaction of individuals who are facing disasters by using virtual reality (VR) videos with disaster scenarios. This study also tested the effectiveness of the method through two experiments. The first experiment comprised the baseline, stress, and recovery periods, representing before, during, and after watching the VR videos. Questionnaires and physiological data, such as photoplethysmography (PPG) and electrodermal activity (EDA), of 180 participants were collected. The second experiment was a simulated driving experiment with 75 participants. This experiment was conducted in two sessions, which were separated by at least one week to avoid memory interference. In the first session, we collected car-following data on a highway. In the second session, participants drove in the same traffic scenario as in the first session after watching the VR video.

The results of the first experiment indicated significant differences in the physical and psychological indicators among the baseline, stress, and recovery periods. The positive affect and negative affect scale (PANAS) data demonstrated that the VR video stimulated evident scared, nervous, and jittery emotions, which were consistent with negative compound emotions in disaster scenarios. Meanwhile, interested and attentive emotions did not change significantly, indicating that the participants were in good condition during the experiment. In addition, more than 80% of the participants reported that the VR video induced a high excitation level. Nearly 90% of them did not feel dizzy while watching the video. Furthermore, pulse rate variability indicators, such as standard deviation of NN intervals (SDNN), low frequency (LF), and EDA, were significantly higher in the stress period than in the baseline period. Changes in physiological indicators indicated that the participants were sympathetically excited and got stressed after watching the VR video. However, some physiological indicators had a high standard deviation due to individual variability. Therefore, VR videos used as stressors should be standardized. In general, the second driving simulation experiment showed a significant increase in acceleration and a significant decrease in headway after watching the VR video compared to when the video was not watched. This demonstrated that their driving style might become more aggressive after watching the VR video. This result is consistent with the findings of previous studies on driving characteristics in normal and emergency situations. In fact, the changing pattern of driving behavior in two sessions showed individual variability, which required further study.

Results of the two experiments indicate that VR videos can induce stress reactions in disaster scenarios, which is demonstrated in physical, psychological, and behavioral aspects. This paper provides a research method for investigating human behavior characteristics in disaster scenarios.