Deep learning-enabled soft sensor system for multilingual speech recognition and emergency communication

Reliable emergency communication is crucial for protecting human lives in extreme environmental conditions. However, conventional microphone-based systems often fail in noisy, obstructed, or submerged environments. Here, we present a deep learning enabled adhesive-interface multimodal sensor (AIMS) for multilingual speech recognition and emergency communication. The AIMS exhibits a broad pressure detection range (0.1-400 kPa), exceptional mechanical endurance (>100,000 fatigue cycles), and broadband vibration responses up to 5 kHz. Critically, it resolves high-frequency vibrations under varying static pressures, decoupling pressure and vibration signals, while maintaining stable performance even under full immersion. By mounting AIMS at the throat, it can simultaneously capture muscle movements (pressure cues) and vocal cord oscillations (vibration signals), enabling accurate speech recognition in noisy or physically obstructed environments. Integrated with convolutional neural networks, the system achieves >98% accuracy in phoneme recognition, tonal analysis, similar-word discrimination, speaker identification, and real-time multilingual translation. Furthermore, we demonstrate a voice-activated rescue mockup simulating an elderly fall scenario, where integrated GPS enables automated distress alerts. This multimodal sensing strategy establishes a versatile platform for next-generation intelligent emergency communication, with transformative potential for disaster response and continuous medical monitoring.