Review Article
Open Access
Issue
Published:
30 November 2022
Open Access
Issue
Published:
30 November 2022
Review of brain–computer interface based on steady-state visual evoked potential
Tianyi Yan1(
)
)
Keywords:
steady-state visual evoked potential, brain–computer interface, canonical correlation analysis, decoding algorithm
Cite this article:
Liu S, Zhang D, Liu Z, et al.
Review of brain–computer interface based on steady-state visual evoked potential.
Brain Science Advances,
2022, 8(4): 258-275.
https://doi.org/10.26599/BSA.2022.9050022
Download citation
621
Views
43
Downloads
Citations
3
Crossref
N/A
WoS
N/A
Scopus
N/A
CSCD
The brain–computer interface (BCI) technology has received lots of attention in the field of scientific research because it can help disabled people improve their quality of life. Steady-state visual evoked potential (SSVEP) is the most researched BCI experimental paradigm, which offers the advantages of high signal-to-noise ratio and short training-time requirement by users. In a complete BCI system, the two most critical components are the experimental paradigm and decoding algorithm. However, a systematic combination of the SSVEP experimental paradigm and decoding algorithms is missing in existing studies. In the present study, the transient visual evoked potential, SSVEP, and various improved SSVEP paradigms are compared and analyzed, and the problems and development bottlenecks in the experimental paradigm are finally pointed out. Subsequently, the canonical correlation analysis and various improved decoding algorithms are introduced, and the opportunities and challenges of the SSVEP decoding algorithm are discussed.
menu
Abstract
Full text
Outline
About this article
Review of brain–computer interface based on steady-state visual evoked potential
Tianyi Yan1(
)
)
Abstract
The brain–computer interface (BCI) technology has received lots of attention in the field of scientific research because it can help disabled people improve their quality of life. Steady-state visual evoked potential (SSVEP) is the most researched BCI experimental paradigm, which offers the advantages of high signal-to-noise ratio and short training-time requirement by users. In a complete BCI system, the two most critical components are the experimental paradigm and decoding algorithm. However, a systematic combination of the SSVEP experimental paradigm and decoding algorithms is missing in existing studies. In the present study, the transient visual evoked potential, SSVEP, and various improved SSVEP paradigms are compared and analyzed, and the problems and development bottlenecks in the experimental paradigm are finally pointed out. Subsequently, the canonical correlation analysis and various improved decoding algorithms are introduced, and the opportunities and challenges of the SSVEP decoding algorithm are discussed.
Keywords:
steady-state visual evoked potential, brain–computer interface, canonical correlation analysis, decoding algorithm
References(78)
[1]
Chaudhary U, Xia B, Silvoni S, et al. Correction: brain-computer interface-based communication in the completely locked-in state. PLoS Biol 2018, 16(12): e3000089.
[2]
Chowdhury A, Shankaran R, Kavakli M, et al. Sensor applications and physiological features in drivers’ drowsiness detection: a review. IEEE Sens J 2018, 18(8): 3055–3067.
[3]
Munyon CN. Neuroethics of non-primary brain computer interface: focus on potential military applications. Front Neurosci 2018, 12: 696.
[4]
Xiong XL, Yu H, Wang H, et al. Action intention understanding EEG signal classification based on improved discriminative spatial patterns. Comput Intell Neurosci 2021, 2021: 1462369.
[5]
Zhang L, Gan JQ, Wang H. Localization of neural efficiency of the mathematically gifted brain through a feature subset selection method. Cogn Neurodyn 2015, 9(5): 495–508.
[6]
Rutkowski TM, Zdunek R, Cichocki A. Multichannel EEG brain activity pattern analysis in time-frequency domain with nonnegative matrix factorization support. Int Congr Ser 2007, 1301: 266–269.
[7]
Henz D, Schöllhorn WI. EEG brain activity in dynamic health qigong training: same effects for mental practice and physical training? Front Psychol 2017, 8: 154.
[8]
Henz D, John A, Merz C, et al. Post-task effects on EEG brain activity differ for various differential learning and contextual interference protocols. Front Hum Neurosci 2018, 12: 19.
[9]
Mayor-Torres JM, Ravanelli M, Medina-DeVilliers SE, et al. Interpretable SincNet-based deep learning for emotion recognition from EEG brain activity. Annu Int Conf IEEE Eng Med Biol Soc 2021, 2021: 412–415.
[10]
Jang S, Kim D, Been J, et al. Oscillatory brain activity changes by anodal tDCS—An ECoG study on anesthetized beagles. Annu Int Conf IEEE Eng Med Biol Soc 2016, 2016: 5258–5261.
[11]
Kanas VG, Mporas I, Benz HL, et al. Real-time voice activity detection for ECoG-based speech brain machine interfaces. 2014 19th International Conference on Digital Signal Processing, Hongkong, China, 2014.
[12]
Carpentier P, Foquin A, Kamenka JM, et al. Effects of thienylphencyclidine (TCP) on seizure activity and brain damage produced by soman in guinea-pigs: ECoG correlates of neurotoxicity. NeuroToxicology 2001, 22(1): 13–28.
[13]
Cox DD, Savoy RL. Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. NeuroImage 2003, 19(2): 261–270.
[14]
Weiskopf N, Veit R, Erb M, et al. Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data. NeuroImage 2003, 19(3): 577–586.
[15]
Cabeza R, Prince SE, Daselaar SM, et al. Brain activity during episodic retrieval of autobiographical and laboratory events: an fMRI study using a novel photo paradigm. J Cogn Neurosci 2004, 16(9): 1583–1594.
[16]
Nagai Y, Critchley HD, Featherstone E, et al. Brain activity relating to the contingent negative variation: an fMRI investigation. NeuroImage 2004, 21(4): 1232–1241.
[17]
Wolpaw JR, Birbaumer N, Heetderks WJ, et al. Brain–computer interface technology: a review of the first international meeting. IEEE Trans Rehabil Eng 2000, 8(2): 164–173.
[18]
Wolpaw JR. Brain–computer interfaces as new brain output pathways. J Physiol 2007, 579(pt 3): 613–619.
[19]
Ford JM, Hillyard SA. Event-related potentials (ERPs) to interruptions of a steady rhythm. Psychophysiology 1981, 18(3): 322–330.
[20]
Mørup M, Hansen LK, Arnfred SM. ERPWAVELAB: a toolbox for multi-channel analysis of time-frequency transformed event related potentials. J Neurosci Methods 2007, 161(2): 361–368.
[21]
Pause BM, Krauel K. Chemosensory event-related potentials (CSERP) as a key to the psychology of odors. Int J Psychophysiol 2000, 36(2): 105–122.
[22]
Urbach TP, Kutas M. Interpreting event-related brain potential (ERP) distributions: implications of baseline potentials and variability with application to amplitude normalization by vector scaling. Biol Psychol 2006, 72(3): 333–343.
[23]
Reynolds C, Osuagwu BA, Vuckovic A. Influence of motor imagination on cortical activation during functional electrical stimulation. Clin Neurophysiol 2015, 126(7): 1360–1369.
[24]
Luo Z, Jin R, Shi H, et al. Research on recognition of motor imagination based on connectivity features of brain functional network. Neural Plast 2021, 2021: 6655430.
[25]
Ellis KA, Silberstein RB, Nathan PJ. Exploring the temporal dynamics of the spatial working memory n-back task using steady state visual evoked potentials (SSVEP). NeuroImage 2006, 31(4): 1741–1751.
[26]
Acqualagna L, Bosse S, Porbadnigk AK, et al. EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs). J Neural Eng 2015, 12(2): 026012.
[27]
Kastner-Dorn AK, Andreatta M, Pauli P, et al. Hypervigilance during anxiety and selective attention during fear: using steady-state visual evoked potentials (ssVEPs) to disentangle attention mechanisms during predictable and unpredictable threat. Cortex 2018, 106: 120–131.
[28]
Ding J, Sperling G, Srinivasan R. Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. Cereb Cortex 2005, 16(7): 1016–1029.
[29]
Sterratt D, Graham B, Gillies A, et al. Principles of Computational Modelling in Neuroscience. UK: Cambridge University Press, 2011.
[30]
Wang YX, Fariello G. On neuroinformatics: mathematical models of neuroscience and neurocomputing. J Adv Math Appl 2012, 1(2): 206–217.
[31]
Regan D. Some characteristics of average steady-state and transient responses evoked by modulated light. Electroencephalogr Clin Neurophysiol 1966, 20(3): 238–248.
[32]
Palomares M, Ales JM, Wade AR, et al. Distinct effects of attention on the neural responses to form and motion processing: a SSVEP source-imaging study. J Vis 2012, 12(10): 15.
[33]
Vidal JJ. Real-time detection of brain events in EEG. Proc IEEE 1977, 65(5): 633–641.
[34]
Galloway NR. Human brain electrophysiology: evoked potentials and evoked magnetic fields in science and medicine. Br J Ophthalmol 1990, 74(4): 255.
[35]
Dechwechprasit P, Phothisonothai M, Tantisatirapong S. Time-frequency analysis of red-green visual flickers based on steady-state visual evoked potential recording. 2016 9th Biomed Eng Int Conf Bmeicon, Laung Prabang, Laos, 2016.
[36]
Cheng M, Gao XR, Gao SK, et al. Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans Biomed Eng 2002, 49(10): 1181–1186.
[37]
Cheng M, Gao SK. An EEG-based cursor control system. In Proceedings of the First Joint BMES/EMBS Conference. 1999 IEEE Engineering in Medicine and Biology 21st Annual Conference and the 1999 Annual Fall Meeting of the Biomedical Engineering Society (Cat. N, Atlanta, GA, USA, 1999, p 669.
[38]
Wang Y, Gao X, Hong B, et al. Brain-computer interfaces based on visual evoked potentials. IEEE Eng Med Biol Mag 2008, 27(5): 64–71.
[39]
Nakanishi M, Wang YJ, Wang YT, et al. A high-speed brain speller using steady-state visual evoked potentials. Int J Neur Syst 2014, 24(6): 1450019.
[40]
Xie J, Xu G, Wang J, et al. Steady-state motion visual evoked potentials produced by oscillating Newton’s rings: implications for brain-computer interfaces. PLoS One 2012, 7(6): e39707.
[41]
Panicker RC, Puthusserypady S, Sun Y. An asynchronous P300 BCI with SSVEP-based control state detection. IEEE Trans Biomed Eng 2011, 58(6): 1781–1788.
[42]
Horki P, Solis-Escalante T, Neuper C, et al. Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb. Med Biol Eng Comput 2011, 49(5): 567–577.
[43]
Chen ZT, Wang ZP, Wang K, et al. Recognizing motor imagery between hand and forearm in the same limb in a hybrid brain computer interface paradigm: an online study. IEEE Access, 7: 59631–59639.
[44]
Zhang Y, Zhou GX, Zhao QB, et al. Multiway canonical correlation analysis for frequency components recognition in SSVEP-based BCIs. In International Conference on Neural Information Processing. ICONIP 2011. Lecture Notes in Computer Science, vol 7062. Lu BL, Zhang L, Kwok J., Eds. Heidelberg, Berlin: Springer, 2011, pp 287–295.
[45]
Bin G, Gao X, Yan Z, et al. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng 2009, 6(4): 046002.
[46]
Chen G, Song DD, Liao LJ. A multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis in the frequency domain. Found Pract Appl Cogn Syst Inf Process 2014: 603–616.
[47]
Nakanishi M, Wang Y, Wang YT, et al. Generating visual flickers for eliciting robust steady-state visual evoked potentials at flexible frequencies using monitor refresh rate. PLoS One 2014, 9(6): e99235.
[48]
Chen X, Wang Y, Gao S, et al. Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface. J Neural Eng 2015, 12(4): 046008.
[49]
Ge S, Jiang YC, Wang P, et al. Training-free steady-state visual evoked potential brain–computer interface based on filter bank canonical correlation analysis and spatiotemporal beamforming decoding. IEEE Trans Neural Syst Rehabilitation Eng 2019, 27(9): 1714–1723.
[50]
Ilea I, Bombrun L, Said S, et al. Fisher vector coding for covariance matrix descriptors based on the log-euclidean and affine invariant Riemannian metrics. J Imaging 2018, 4(7): 85.
[51]
Congedo M, Barachant A, Bhatia R. Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review. Brain Comput Interfaces 2017, 4(3): 155–174.
[52]
Yger F, Berar M, Lotte F. Riemannian approaches in brain-computer interfaces: a review. IEEE Trans Neural Syst Rehabilitation Eng 2017, 25(10): 1753–1762.
[53]
Congedo M, Barachant A, Andreev A. A new generation of brain-computer interface based on Riemannian geometry. ArXiv 2013: abs/1310.8115.
[54]
Kalunga EK, Chevallier S, Barthélemy Q, et al. Online SSVEP-based BCI using Riemannian geometry. Neurocomputing 2016, 191: 55–68.
[55]
Ang KK, Guan C, Lee K, et al. Application of rough set-based neuro-fuzzy system in NIRS-based BCI for assessing numerical cognition in classroom. In The 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, 2010.
[56]
Wang J, Feng ZR, Ren XD, et al. Feature subset and time segment selection for the classification of EEG data based motor imagery. Biomed Signal Process Control 2020, 61: 102026.
[57]
Duan L, Ge H, Ma W, et al. EEG feature selection method based on decision tree. Biomed Mater Eng 2015, 26(Suppl 1): S1019–S1025.
[58]
Müller SMT, Diez PF, Bastos-Filho TF, et al. SSVEP-BCI implementation for 37–40 Hz frequency range. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 2011, pp 6352–6355.
[59]
Müller SMT, Bastos-Filho TF, Sarcinelli-Filho M. Using a SSVEP-BCI to command a robotic wheelchair. In 2011 IEEE International Symposium on Industrial Electronics, Gdansk, Poland, 2011, pp 957–962.
[60]
Hortal E, Úbeda A, Iáñez E, et al. Online classification of two mental tasks using a SVM-based BCI system. In 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego, CA, USA, 2014, pp1307–1310.
[61]
Arbabi E, Shamsollahi MB, Sameni R. Comparison between effective features used for the Bayesian and the SVM classifiers in BCI. In 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China. 2005, pp 5365–5368.
[62]
Yang H, Wu S. EEG classification for BCI based on CSP and SVM-GA. In Applied Mechanics and Materials, Vol. 459. Tan H, Ed. Tans Tech Publications, Ltd., 2013, pp 228–231.
[63]
Hortal E, Iáñez E, Úbeda A, et al. Selection of the best mental tasks for a SVM-based BCI system. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), San Diego, CA, USA, 2014, pp 1483–1488.
[64]
Schlögl A, Lee F, Bischof H, et al. Characterization of four-class motor imagery EEG data for the BCI-competition 2005. J Neural Eng 2005, 2(4): L14–L22.
[65]
Bhattacharyya S, Khasnobish A, Chatterjee S, et al. Performance analysis of LDA, QDA and KNN algorithms in left-right limb movement classification from EEG data. In 2010 International Conference on Systems in Medicine and Biology, Kharagpur, India, 2010, pp 126–131.
[66]
Ghaffar MSBA, Khan US, Iqbal J, et al. Improving classification performance of four class FNIRS-BCI using Mel Frequency Cepstral Coefficients (MFCC). Infrared Phys Technol 2021, 112: 103589.
[67]
Miladinović A, Ajčević M, Battaglini PP, et al. Slow cortical potential BCI classification using sparse variational bayesian logistic regression with automatic relevance determination. In XV Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019. IFMBE Proceedings, vol 76. Henriques J, Neves N, de Carvalho P, Eds. Cham: Springer, pp 1853–1860.
[68]
Fiebig KH, Jayaram V, Peters J, et al. Multi-task logistic regression in brain-computer interfaces. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Budapest, Hungary, 2016, pp 002307–002312.
[69]
Li Y, Wu J, Yang J. Developing a logistic regression model with cross-correlation for motor imagery signal recognition. In The 2011 IEEE/ICME International Conference on Complex Medical Engineering, Harbin, China, 2011, pp 502–507.
[70]
Acampora G, Trinchese P, Vitiello A. Applying logistic regression for classification in single-channel SSVEP-based BCIs. In 2019 IEEE International Conference on Systems, Man and Cybernetics, Bari, Italy, 2019, pp 33–38.
[71]
Zhu YL, Li Y, Lu JL, et al. EEGNet with ensemble learning to improve the cross-session classification of SSVEP based BCI from ear-EEG. IEEE Access, 9: 15295–15303.
[72]
Sybeldon M, Schmit L, Akcakaya M. Transfer learning for SSVEP electroencephalography based brain–computer interfaces using Learn++.NSE and mutual information. Entropy 2017, 19(1): 41.
[73]
Cao ZH, Lin CT, Lai KL, et al. Extraction of SSVEPs-based inherent fuzzy entropy using a wearable headband EEG in migraine patients. IEEE Trans Fuzzy Syst 2020, 28(1): 14–27.
[74]
Rashid M, Sulaiman N, Mustafa M, et al. Five-class SSVEP response detection using common-spatial pattern (CSP)-SVM approach. Int J Integr Eng 2020, 12(6): 165–171.
[75]
Chatzilari E, Liarios G, Georgiadis K, et al. Combining the benefits of CCA and SVMs for SSVEP-based BCIs in real-world conditions. In Proceedings of the 2nd International Workshop on Multimedia for Personal Health and Health Care(MMHealth ‘17). New York, NY, USA, 2017, 3–10.
[76]
Saidutta YM, Zou J, Fekri F. Increasing the learning Capacity of BCI Systems via CNN-HMM models. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Honolulu, HI, USA, 2018, pp 1–4.
[77]
Hosman T, Vilela M, Milstein D, et al. BCI decoder performance comparison of an LSTM recurrent neural network and a Kalman filter in retrospective simulation. In 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), San Francisco, CA, USA, pp 1066–1071.
[78]
Cvejic N, Seppanen T. Increasing the capacity of LSB-based audio steganography. In 2002 IEEE Workshop on Multimedia Signal Processing, St. Thomas, VI, USA, 2022, pp 336–338.
Publication history
Copyright
Rights and permissions
Publication history
Received: 11 July 2022
Revised: 13 September 2022
Accepted: 27 September 2022
Published:
30 November 2022
Issue date: December 2022
Copyright
© The authors 2022.
Rights and permissions
This article is published with open access at journals.sagepub.com/home/BSA
Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
FEEDBACK 
TOP