Dangerous driving behavior detection method for intelligent cockpits based on TransWindow multimodal fusion

Driver behavior is a critical determinant of road safety; thus, real-time monitoring of dangerous behaviors, such as fatigue and distraction, is essential for intelligent cockpit systems. However, existing in-vehicle sensing technologies are highly susceptible to environmental variations, particularly illumination changes, which result in unstable driver feature extraction. Moreover, transformer-based models typically incur high computational overhead due to their global self-attention mechanisms, making it challenging to achieve an optimal balance between efficiency and accuracy in driver monitoring systems (DMSs). To address these challenges, a visual-tactile fusion framework for dangerous driving behavior detection is proposed. The framework integrates visual sensors with seat-embedded pressure sensors, enabling complementary perception across modalities. Specifically, visual cues facilitate accurate recognition of facial expressions and postures, while tactile signals remain robust under adverse conditions, such as low illumination and occlusion. This complementary fusion significantly enhances system robustness and reliability. Experimental results on a hybrid dataset collected from both simulated and real-world driving scenarios demonstrate the effectiveness of the proposed approach. Compared with state-of-the-art transformer-based methods and their variants, the proposed TransWindow model achieves an accuracy of 98.77% while maintaining substantially lower computational complexity. These results indicate that the proposed method effectively balances accuracy and efficiency, satisfies the real-time requirements of DMSs, and provides a promising direction for the continuous optimization of intelligent cockpit systems.