Asynchronous Brain-Computer Interface Shared Control of Robotic Grasping

Wenchang Zhang; Fuchun Sun; Hang Wu; Chuanqi Tan; Yuzhen Ma

doi:10.26599/TST.2018.9010111

Tsinghua Science and Technology 2019, 24(3): 360-370 https://doi.org/10.26599/TST.2018.9010111

Open Access | Issue | Published: 24 January 2019

Asynchronous Brain-Computer Interface Shared Control of Robotic Grasping

Show Author's Information Hide Author's Information Wenchang Zhang, Fuchun Sun(

), Hang Wu, Chuanqi Tan, Yuzhen Ma

Department of Computer Science and Technology, Tsinghua University, State Key Lab. of Intelligent Technology and Systems, Beijing 100084, China.

Institute of Medical Support Technology, Academy of Military Sciences, Wandong Road, Tianjin 300161, China.

Drugs Control of PAP, Beijing 102613, China.

Keywords:

asynchronous Brain-Computer Interface (BCI), shared control, motor imagery, robotic grasping

Cite this article:

Zhang W, Sun F, Wu H, et al. Asynchronous Brain-Computer Interface Shared Control of Robotic Grasping. Tsinghua Science and Technology, 2019, 24(3): 360-370. https://doi.org/10.26599/TST.2018.9010111

Download citation

EndNote(RIS)

BibTeX

501

Views

Downloads

Citations

Crossref

N/A

WoS

Scopus

CSCD

Abstract Full text About this article

Abstract

The control of a high Degree of Freedom (DoF) robot to grasp a target in three-dimensional space using Brain-Computer Interface (BCI) remains a very difficult problem to solve. Design of synchronous BCI requires the user perform the brain activity task all the time according to the predefined paradigm; such a process is boring and fatiguing. Furthermore, the strategy of switching between robotic auto-control and BCI control is not very reliable because the accuracy of Motor Imagery (MI) pattern recognition rarely reaches 100 $%$ . In this paper, an asynchronous BCI shared control method is proposed for the high DoF robotic grasping task. The proposed method combines BCI control and automatic robotic control to simultaneously consider the robotic vision feedback and revise the unreasonable control commands. The user can easily mentally control the system and is only required to intervene and send brain commands to the automatic control system at the appropriate time according to the experience of the user. Two experiments are designed to validate our method: one aims to illustrate the accuracy of MI pattern recognition of our asynchronous BCI system; the other is the online practical experiment that controls the robot to grasp a target while avoiding an obstacle using the asynchronous BCI shared control method that can improve the safety and robustness of our system.

Full text

Abstract

Full text

Outline

About this article

Asynchronous Brain-Computer Interface Shared Control of Robotic Grasping

Show Author's information Hide Author's Information Wenchang Zhang, Fuchun Sun(

), Hang Wu, Chuanqi Tan, Yuzhen Ma

Department of Computer Science and Technology, Tsinghua University, State Key Lab. of Intelligent Technology and Systems, Beijing 100084, China.

Institute of Medical Support Technology, Academy of Military Sciences, Wandong Road, Tianjin 300161, China.

Drugs Control of PAP, Beijing 102613, China.

Abstract

Keywords: asynchronous Brain-Computer Interface (BCI), shared control, motor imagery, robotic grasping

References(34)

[1]

Mak J. N. and Wolpaw J. R., Clinical applications of brain-computer interfaces: Current state and future prospects, IEEE Rev. Biomed. Eng., vol. 2, pp. 187-199, 2009.

DOI Google Scholar

[2]

Grigorescu S. M., Lüth T., Fragkopoulos C., Cyriacks M., and Gräser A., A BCI-controlled robotic assistant for quadriplegic people in domestic and professional life, Robotica, vol. 30, no. 3, pp. 419-431, 2012.

DOI Google Scholar

[3]

Galán F., Nuttin M., Lew E., Ferrez P. W., Vanacker G., Philips J., and del R. Millán J., A brain-actuated wheelchair: Asynchronous and non-invasive brain-computer interfaces for continuous control of robots, Clin. Neurophysiol., vol. 119, no. 9, pp. 2159-2169, 2008.

DOI Google Scholar

[4]

Muller-Putz G. R. and Pfurtscheller G., Control of an electrical prosthesis with an SSVEP-based BCI, IEEE Trans. Biomed. Eng., vol. 55, no. 1, pp. 361-364, 2008.

DOI Google Scholar

[5]

Meng J. J., Zhang S. Y., Bekyo A., Olsoe J., Baxter B., and He B., Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks, Sci. Rep., vol. 6, p. 38565, 2016.

DOI Google Scholar

[6]

Mason S. G. and Birch G. E., A brain-controlled switch for asynchronous control applications, IEEE Trans. Biomed. Eng., vol. 47, no. 10, pp. 1297-1307, 2000.

DOI Google Scholar

[7]

Townsend G., Graimann B., and Pfurtscheller G., Continuous EEG classification during motor imagery simulation of an asynchronous BCI, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 12, no. 2, pp. 258-265, 2004.

DOI Google Scholar

[8]

Borisoff J. F., Mason S. G., and Birch G. E., Brain interface research for asynchronous control applications, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 14, no. 2, pp. 160-164, 2006.

DOI Google Scholar

[9]

Chae Y., Jeong J., and Jo S., Toward brain-actuated humanoid robots: Asynchronous direct control using an EEG-based BCI, IEEE Trans. Robot., vol. 28, no. 5, pp. 1131-1144, 2012.

DOI Google Scholar

[10]

Lisi G. and Morimoto J., EEG single-trial detection of gait speed changes during treadmill walk, PLoS One, vol. 10, no. 5, p. e0125479, 2015.

DOI Google Scholar

[11]

Geng T., Gan J. Q., and Hu H. S., A self-paced online BCI for mobile robot control, Int. J. Adv. Mechatronic Syst., vol. 2, nos. 1/2, pp. 28-35, 2010.

DOI Google Scholar

[12]

Geng T. and Gan J. Q., Motor prediction in brain-computer interfaces for controlling mobile robots, in Proc. 30th Annu. Int. Conf. IEEE Engineering in Medicine and Biology Society, Vancouver, Canada, 2008, pp. 634-637.

DOI

[13]

Li Q., Chen W. D., and Wang J. C., Dynamic shared control for human-wheelchair cooperation, in 2011 IEEE Int. Conf. Robotics and Automation, Shanghai, China, 2011, pp. 4278-4283.

[14]

Su B., Shen L., Wang L., Wang Z. Y., Wang Y. R., Huang L. B., and Shi W., DCP: Improving the throughput of asynchronous pipeline by dual control path. in IEEE International Conference on High Performance Computing & Communications & IEEE International Conference on Embedded & Ubiquitous Computing, 2014.

DOI

[15]

del R. Millan J., Galan F., Vanhooydonck D., Lew E., Philips J., and Nuttin M., Asynchronous non-invasive brain-actuated control of an intelligent wheelchair, in Proc. 2009 Annu. Int. Conf. IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA, 2009, pp. 3361-3364.

DOI

[16]

Liu R., Wang Y. X., and Zhang L., An FDES-based shared control method for asynchronous brain-actuated robot, IEEE Trans. Cybernet., vol. 46, no. 6, pp. 1452-1462, 2016.

DOI Google Scholar

[17]

Sun F. C., Zhang W. C., Chen J. H., Wu H., Tan C. Q., and Su W. H., Fused fuzzy petri nets: A shared control method for Brain Computer Interface systems, IEEE Trans. Cognit. Dev. Syst., .

DOI Google Scholar

[18]

Hyvarinen A., Fast and robust fired point algorithms for independent component analysis, IEEE Transactions on Neural Networks, vol. 10, no. 3, pp. 626-634, 1999.

DOI Google Scholar

[19]

Townsend G., Graimann B., and Pfurtscheller G., Continuous EEG classification during motor imagery-simulation of an asynchronous BCI, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 12, no. 2, pp. 258-265, 2004.

DOI Google Scholar

[20]

Chae Y., Jeong J., and Jo S., Toward brain-actuated humanoid robots: Asynchronous direct control using an EEG-based BCI, IEEE Trans. Robot., vol. 28, no. 5, pp. 1131-1144, 2012.

DOI Google Scholar

[21]

Muralidharan A., Chae J., and Taylor D. M., Extracting attempted hand movements from EEGs in people with complete hand paralysis following stroke, Front. Neurosci., vol. 5, p. 39, 2011.

DOI Google Scholar

[22]

Zhang W. C., Sun F. C., Tan C. Q., and Liu S. B., Low-rank linear dynamical systems for motor imagery EEG, Comput. Intel. Neurosc., vol. 2016, p. 2637603, 2016.

DOI Google Scholar

[23]

Lin Z. C., Chen M. M., and Ma Y., The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices, arXiv preprint arXiv: 1009.5055, 2010.

Google Scholar

[24]

Zhang W. C., Sun F. C., Tan C. Q., and Liu S. B., Linear dynamical systems modeling for EEG-based motor imagery brain-computer interface. in Cognitive Systems and Signal Processing, Sun F., Liu H., and Hu D., eds. Springer, 2016, pp. 521-528.

DOI

[25]

Martin R. J., A metric for ARMA processes, IEEE Trans. Signal Proc., vol. 48, no. 4, pp. 1164-1170, 2000.

DOI Google Scholar

[26]

Chan A. B. and Vasconcelos N., Classifying video with kernel dynamic textures, in Proc. 2007 IEEE Conf. Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007, pp. 1-6.

DOI

[27]

Ramoser H., Muller-Gerking J., and Pfurtscheller G., Optimal spatial filtering of single trial EEG during imagined hand movement, IEEE Trans. Rehab. Eng., vol. 8, no. 4, pp. 441-446, 2000.

DOI Google Scholar

[28]

Grosse-Wentrup M. and Buss M., Multiclass common spatial patterns and information theoretic feature extraction, IEEE Trans. Biomed. Eng., vol. 55, no. 8, pp. 1991-2000, 2008.

DOI Google Scholar

[29]

Allison B. Z., Toward ubiquitous BCIs, in Brain-Computer Interfaces, Graimann B., Pfurtscheller G., and Allison B., eds. Springer, 2009, pp. 357-387.

DOI

[30]

Li Y. Q., Long J. Y., Yu T. Y., Yu Z. L., Wang C. C., Zhang H. H., and Guan C. T., An EEG-based BCI system for 2-D cursor control by combining Mu/Beta rhythm and P300 potential, IEEE Trans. Biomed. Eng., vol. 57, no. 10, pp. 2495-2505, 2010.

DOI Google Scholar

[31]

Horki P., Solis-Escalante T., Neuper C., and Müller-Putz G., Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb, Med. Biol. Eng. Comput., vol. 49, no. 5, pp. 567-577, 2011.

DOI Google Scholar

[32]

Allison B. Z., Brunner C., Altstätter C., Wagner I. C., Grissmann S., and Neuper C., A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control, J. Neurosci. Methods, vol. 209, no. 2, pp. 299-307, 2012.

DOI Google Scholar

[33]

Long J. Y., Li Y. Q., Wang H. T., Yu T. Y., Pan J. H., and Li F., A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 20, no. 5, pp. 720-729, 2012.

DOI Google Scholar

[34]

Yin E. W., Zhou Z. T., Jiang J., Chen F. L., Liu Y. D., and Hu D. W., A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm, J. Neural Eng., vol. 10, no. 2, p. 026012, 2013.

DOI Google Scholar

About this article

Publication history

Acknowledgements

Rights and permissions

Publication history

Received: 26 June 2018

Revised: 28 June 2018

Accepted: 04 July 2018

Published: 24 January 2019

Issue date: June 2019

Copyright

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 91420302 and 91520201) and Innovation Cultivating Fund Project $17 - 163 - 12 - Z T - 001 - 019 - 01$ .