A review of deep learning methods for cross-subject rapid serial visual presentation detection in World Robot Contest 2022

Zehui Wang; Hongfei Zhang; Zhouyu Ji; Yuliang Yang; Hongtao Wang

doi:10.26599/BSA.2023.9050013

Brain Science Advances 2023, 9(3): 195-209 https://doi.org/10.26599/BSA.2023.9050013

Review Article |

Open Access | Issue | Published: 05 September 2023

A review of deep learning methods for cross-subject rapid serial visual presentation detection in World Robot Contest 2022

Show Author's Information Hide Author's Information Zehui Wang^{¹^,²}, Hongfei Zhang^{¹^,²}, Zhouyu Ji^{¹^,²}, Yuliang Yang^{¹^,²}, Hongtao Wang^{¹^,²}(

)

1 Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, Guangdong, China

2 Jiangmen Brain-like Computation and Hybrid Intelligence Research Center, Jiangmen 529020, Guangdong, China

Keywords:

deep learning, rapid serial visual presentation, detection, cross-subject, brain–computer interface

Cite this article:

Wang Z, Zhang H, Ji Z, et al. A review of deep learning methods for cross-subject rapid serial visual presentation detection in World Robot Contest 2022. Brain Science Advances, 2023, 9(3): 195-209. https://doi.org/10.26599/BSA.2023.9050013

Download citation

EndNote(RIS)

BibTeX

411

Views

Downloads

Citations

Crossref

N/A

WoS

N/A

Scopus

N/A

CSCD

Abstract Full text About this article

Abstract

The rapid serial visual presentation (RSVP) paradigm has garnered considerable attention in brain–computer interface (BCI) systems. Studies have focused on using cross-subject electroencephalogram data to train cross-subject RSVP detection models. In this study, we performed a comparative analysis of the top 5 deep learning algorithms used by various teams in the event-related potential competition of the BCI Controlled Robot Contest in World Robot Contest 2022. We evaluated these algorithms on the final data set and compared their performance in cross-subject RSVP detection. The results revealed that deep learning models can achieve excellent results with appropriate training methods when applied to cross-subject detection tasks. We discussed the limitations of existing deep learning algorithms in cross-subject RSVP detection and highlighted potential research directions.

Full text

Abstract

Full text

Outline

About this article

A review of deep learning methods for cross-subject rapid serial visual presentation detection in World Robot Contest 2022

Show Author's information Hide Author's Information Zehui Wang^{¹^,²}, Hongfei Zhang^{¹^,²}, Zhouyu Ji^{¹^,²}, Yuliang Yang^{¹^,²}, Hongtao Wang^{¹^,²}(

)

1 Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, Guangdong, China

2 Jiangmen Brain-like Computation and Hybrid Intelligence Research Center, Jiangmen 529020, Guangdong, China

Abstract

Keywords: deep learning, rapid serial visual presentation, detection, cross-subject, brain–computer interface

References(35)

[1]

Rebsamen B, Guan CT, Zhang HH, et al. A brain controlled wheelchair to navigate in familiar environments. IEEE Trans Neural Syst Rehabil Eng 2010, 18(6): 590–598.

DOI Google Scholar

[2]

Li JH, Liang JY, Zhao QB, et al. Design of assistive wheelchair system directly steered by human thoughts. Int J Neural Syst 2013, 23(3): 1350013.

DOI Google Scholar

[3]

Polich J. Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol 2007, 118(10): 2128–2148.

DOI Google Scholar

[4]

Picton TW. The P300 wave of the human event-related potential. J Clin Neurophysiol 1992, 9(4): 456–479.

DOI Google Scholar

[5]

Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 1988, 70(6): 510–523.

DOI Google Scholar

[6]

Wang HT, Huang H, He YB, et al. An event related potential electroencephalogram signal analysis method based on denoising auto-encoder neural network (in Chinese). Contr Theory Appl 2019, 36(4): 589–595.

Google Scholar

[7]

Liu MF, Wu W, Gu ZH, et al. Deep learning based on Batch Normalization for P300 signal detection. Neurocomputing 2018, 275: 288–297.

DOI Google Scholar

[8]

Wang ZH, Chen CQ, Li JH, et al. ST-CapsNet: linking spatial and temporal attention with capsule network for P300 detection improvement. IEEE Trans Neural Syst Rehabil Eng 2023, PP, .

DOI Google Scholar

[9]

Kundu S, Ari S. MsCNN: a deep learning framework for P300-based brain–computer interface speller. IEEE Trans Med Robot Bionics 2020, 2(1): 86–93.

DOI Google Scholar

[10]

Hajinoroozi M, Mao ZJ, Lin YP, et al. Deep transfer learning for cross-subject and cross-experiment prediction of image rapid serial visual presentation events from EEG data. In Augmented Cognition. Neurocognition and Machine Learning. AC 2017. Lecture Notes in Computer Science, vol 10284. Schmorrow D, Fidopiastis C, Eds. Cham: Springer, 2017, pp 45–55.

DOI

[11]

Lees S, Dayan N, Cecotti H, et al. A review of rapid serial visual presentation-based brain-computer interfaces. J Neural Eng 2018, 15(2): 021001.

DOI Google Scholar

[12]

Subasi A, Ismail Gursoy M. EEG signal classification using PCA, ICA, LDA and support vector machines. Expert Syst Appl 2010, 37(12): 8659–8666.

DOI Google Scholar

[13]

Mika S, Ratsch G, Weston J, et al. Fisher discriminant analysis with kernels. In Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468). Madison, WI, USA, 2002, pp 41–48.

Google Scholar

[14]

Gerson AD, Parra LC, Sajda P. Cortically coupled computer vision for rapid image search. IEEE Trans Neural Syst Rehabil Eng 2006, 14(2): 174–179.

DOI Google Scholar

[15]

Fuhrmann Alpert G, Manor R, Spanier AB, et al. Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm. IEEE Trans Biomed Eng 2014, 61(8): 2290–2303.

DOI Google Scholar

[16]

Lawhern VJ, Solon AJ, Waytowich NR, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng 2018, 15(5): 056013.

DOI Google Scholar

[17]

Schirrmeister RT, Springenberg JT, Fiederer LDJ, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 2017, 38(11): 5391–5420.

DOI Google Scholar

[18]

Santamaria-Vazquez E, Martinez-Cagigal V, Vaquerizo-Villar F, et al. EEG-inception: a novel deep convolutional neural network for assistive ERP-based brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng 2020, 28(12): 2773–2782.

DOI Google Scholar

[19]

Wang HT, Pei ZA, Xu LF, et al. Performance enhancement of P300 detection by multiscale-CNN. IEEE Trans Instrum Meas 2021, 70: 1–12.

DOI Google Scholar

[20]

He H, Wu DR. Transfer learning for brain-computer interfaces: a euclidean space data alignment approach. IEEE Trans Biomed Eng 2020, 67(2): 399–410.

DOI Google Scholar

[21]

Rivet B, Souloumiac A, Attina V, et al. xDAWN algorithm to enhance evoked potentials: application to brain-computer interface. IEEE Trans Biomed Eng 2009, 56(8): 2035–2043.

DOI Google Scholar

[22]

Wu HY, Wu DR. Review of training-free event-related potential classification approaches in the World Robot Contest 2021. Brain Science Advances 2022, 8(2): 82–98.

DOI Google Scholar

[23]

Zhang HF, Wang ZH, Yu YH, et al. An improved EEGNet for single-trial EEG classification in rapid serial visual presentation task. Brain Science Advances 2022, 8(2): 111–126.

DOI Google Scholar

[24]

Li F, Li HX, Li Y, et al. Decoupling representation learning for imbalanced electroencephalography classification in rapid serial visual presentation task. J Neural Eng 2022, 19(3): .

DOI Google Scholar

[25]

Zhang HY, Cisse M, Dauphin YN, et al. Mixup: beyond empirical risk minimization. arXiv 2017, arXiv: 1710.09412. https://arxiv.org/abs/1710.09412

Google Scholar

[26]

Kostas D, Rudzicz F. Thinker invariance: enabling deep neural networks for BCI across more people. J Neural Eng 2020, 17(5): 056008.

DOI Google Scholar

[27]

Han DK, Jeong JH. Domain generalization for session-independent brain–computer interface. In 2021 9th International Winter Conference on Brain–Computer Interface (BCI). Gangwon, Korea (South), 2021, pp 1–5.

DOI Google Scholar

[28]

Lei Y, Belkacem AN, Wang XT, et al. A convolutional neural network-based diagnostic method using resting-state electroencephalograph signals for major depressive and bipolar disorders. Biomed Signal Process Contr 2022, 72: 103370.

DOI Google Scholar

[29]

Izmailov P, Podoprikhin D, Garipov T, et al. Averaging weights leads to wider optima and better generalization. arXiv 2018, arXiv: 1803.05407. https://arxiv.org/abs/1803.05407

Google Scholar

[30]

Müller R, Kornblith S, Hinton GE. When does label smoothing help? In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). Vancouver, Canada, 2019.

Google Scholar

[31]

Lin TY, Goyal P, Girshick R, et al. Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 2020, 42(2): 318–327.

DOI Google Scholar

[32]

James F. Monte Carlo theory and practice. Rep Prog Phys 1980, 43(9): 1145–1189.

DOI Google Scholar

[33]

Yu Y, Si XS, Hu CH, et al. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 2019, 31(7): 1235–1270.

DOI Google Scholar

[34]

Selvaraju RR, Cogswell M, Das A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy, 2017, pp 618–626.

DOI Google Scholar

[35]

Ghorbani A, Zou J. Data shapley: equitable valuation of data for machine learning. arXiv 2019, arXiv: 1904.02868. https://arxiv.org/abs/1904.02868

Google Scholar

About this article

Publication history

Rights and permissions

Publication history

Received: 14 March 2023

Revised: 03 April 2023

Accepted: 08 May 2023

Published: 05 September 2023

Issue date: September 2023

Copyright

Rights and permissions

This article is published with open access at journals.sagepub.com/home/BSA

Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).