The rapid developments of internet of things (IoTs) technology have greatly promoted the evolution of human-machine interaction (HMI). Mimicking the auditory and visual mechanisms offers new possibilities for real time multimodal information processing. Here, an acoustic-optical information reservoir system (AOIRS) is proposed by integrating a triboelectric acoustic sensor (TAS) with an indium-zinc-oxide photoelectronic neuromorphic transistor. With an acoustic coupling cavity, the TAS exhibits excellent sound wave collection capacity and durability. A high sensitivity of ~1.1 V/dB is obtained. Thus, interesting neural activities are mimicked on the AOIRS under sound wave signals. The synaptic weight of AOIRS can be repeatedly updated by light and sound stimuli. In addition, AOIRS can act as a physical reservoir to map voice commands and drone trajectory images into high-dimensional conductance states, achieving efficient voice command and drone trajectory recognition via a multi-layer perceptron (MLP) classifier. The average testing accuracy after 200 epochs for voice command and drone trajectory is ~89.8% and 93.6%, respectively. The present AOIRS has great potentials in the fields of voice intelligent sensing and low-altitude industry.

Hardware based neuromorphic sensory system has attracted great attention for cognitive interactive platform. Auditory perception system can capture and analyze various sound signals, helping us to detect dangerous surroundings and judge environmental conditions. Therefore, developing neuromorphic auditory system that can decode auditory spatiotemporal information would be interesting. Here, an artificial auditory perceptual system is proposed by integrating sound frequency sensitive triboelectric nanogenerators (SFS-TENGs) and oxide based ionotronic neuromorphic transistor. With perforated configuration, the SFS-TENG adopting polyetheretherketone membrane and polytetrafluoroethylene membrane as friction layers can convert sound wave signals into electrical signals, exhibiting a high sensitivity of ~ 2.24 V/dB and good durability. The neuromorphic transistor can further process electrical signals generated by SFS-TENG. Thus, the system can mimic auditory perception, exhibiting a wide range of sound pressure and frequency recognition capabilities. Information encryption/decryption and Doppler frequency shift temporal information processing are demonstrated on the TENG based auditory system for the first time. The present auditory perceptual system demonstrates broad application prospects, providing new opportunities to create sophisticated, adaptable, and interactive systems.