Deep Residual Convolutional Neural Networks for Brain–Computer Interface to Visualize Neural Processing of Hand Movements in the Human Brain

Yosuke Fujiwara, Junichi Ushiba

Research output: Contribution to journalArticlepeer-review

Abstract

Concomitant with the development of deep learning, brain–computer interface (BCI) decoding technology has been rapidly evolving. Convolutional neural networks (CNNs), which are generally used as electroencephalography (EEG) classification models, are often deployed in BCI prototypes to improve the estimation accuracy of a participant's brain activity. However, because most BCI models are trained, validated, and tested via within-subject cross-validation and there is no corresponding generalization model, their applicability to unknown participants is not guaranteed. In this study, to facilitate the generalization of BCI model performance to unknown participants, we trained a model comprising multiple layers of residual CNNs and visualized the reasons for BCI classification to reveal the location and timing of neural activities that contribute to classification. Specifically, to develop a BCI that can distinguish between rest, left-hand movement, and right-hand movement tasks with high accuracy, we created multilayers of CNNs, inserted residual networks into the multilayers, and used a larger dataset than in previous studies. The constructed model was analyzed with gradient-class activation mapping (Grad-CAM). We evaluated the developed model via subject cross-validation and found that it achieved significantly improved accuracy (85.69 ± 1.10%) compared with conventional models or without residual networks. Grad-CAM analysis of the classification of cases in which our model produced correct answers showed localized activity near the premotor cortex. These results confirm the effectiveness of inserting residual networks into CNNs for tuning BCI. Further, they suggest that recording EEG signals over the premotor cortex and some other areas contributes to high classification accuracy.

Original languageEnglish
Article number882290
JournalFrontiers in Computational Neuroscience
Volume16
DOIs
Publication statusPublished - 2022 May 20

Keywords

  • brain-computer interface
  • deep learning
  • deep residual convolutional neural networks (CNN)
  • electroencephalography
  • Grad-CAM

ASJC Scopus subject areas

  • Neuroscience (miscellaneous)
  • Cellular and Molecular Neuroscience

Fingerprint

Dive into the research topics of 'Deep Residual Convolutional Neural Networks for Brain–Computer Interface to Visualize Neural Processing of Hand Movements in the Human Brain'. Together they form a unique fingerprint.

Cite this