Brain computer interface
Вrain-computer interface (BCI) is a new technology that allows an individual to influence the environment through his or her thoughts, i.e. through decoding mental commands recorded as signals in the electroencephalogram (EEG) or other methods of brain activity recording. In this case mental commands decoded by a special program are used to control an external device: a manipulator, robot, wheelchair, cursor in a computer program, etc.
This way of communication with the external environment is particularly important for people who are paralyzed or have severe motor disabilities. In this case BCI can substitute movements, i.e. an individual can communicate with external environment through BCI only. Besides that, there are special methods of motor function recovery based on BCI principles.
Even when motor functions are impaired the ability to imagine the necessary movement and give a mental command remains intact. This is important for rehabilitation, as numerous studies have shown that imaginary movements activate the same brain areas as actual movements. What is more, mental visualization of movements is used by athletes in training process (the so-called «motor imagery practice»). Thus, repeatedly imagining the necessary movements and making mental attempts to perform them, one can improve his or her nerve tissue recovery, involve into action new neurons instead of those impaired by disease, and as a result to enhance the rehabilitation process.
Anyway, rehabilitation is a long and time-consuming process, and recovery does not always come fast. If patients do not see the result of their efforts it might discourage them from doing exercises and kill their motivation. «Feedback» is needed – patients should see how well he performs imaginary movements and gives mental commands. Even better results can be achieved if mental commands for movements will be accompanied by a passive movement of a leg or an arm, performed for a patient by a special robotic device, or a movement performed as a result of electrical stimulation of the muscles provoking the necessary movement. In the nervous system at the same time the mental effort is becoming associated with the performed movement (though passive yet) and this also speeds up the rehabilitation process.
Another possible way of getting feedback is showing the desired movement on a computer screen with the help of a model of an arm or a leg, or using mental motor commands to control a computer game.
The key problem for the realization of this technology of rehabilitation is a reliable method of mental command recognition/classification. Besides, the more subtle and complex movement is imagined, the more difficult it is to recognize. Recognition of EEG patterns corresponding to their mental commands is the most promising method in this area since the electroencephalogram is a non-invasive method of brain signals recording, safe for the patient, mobile and relatively cheap.
The aim of our research is first of all to create a reliable EEG based classifier for the commands of fine motor skills (finger movements).
Development of the BCI system is based on our own research: we record the EEG of healthy volunteers and use different approaches to EEG signal processing and analysis.
We applied several new approaches to solve the problem of motor command recognition.
“Rhythmic paradigm”: imaginary movements / mental commands are performed to a given rhythm (rhythm is set by sound signals) which allows to speed up the recognition of mental commands by the BCI system and make it more reliable.
To interpret mental commands we developed a special computer program based on complex learning algorithms – a two-level combined classifier committee of EEG signals.
When designing the classifier we used the most suitable algorithms for complex signals: artificial neural networks (ANN) and support vector machine (SVM). To recognize signals we use several types of EEG signal features simultaneously, which allows the classifier to adjust to each individual and increases the reliability of mental command recognition.
We use different types of EEG signals transformations and for each individual the most suitable one is chosen. It was found that in most cases current source density (CSD) transform gives the best results in mental command recognition. As a result, it was possible to achieve the accuracy of the classification of motor commands well above the random guessing level.
Thus, the application of new approaches allows us to successfully solve the problem of mental command recognition, even for the complex case of multiclass classification of fine motor commands. Recognition of mental commands with high precision will make it possible to create a rehabilitation system with feedback – a patient will be able to see on screen the results of their mental efforts on the monitor screen, recognized mental commands will be used to control a computer game (application), a 3D model of a hand, or a robotic device that will perform the movements imagined by the patient.
The results obtained in the course of our scientific research are used to develop an innovative software and hardware system based on brain-computer interface for rehabilitation of patients with motor disabilities, which is going to make rehabilitation more successful, quick, comfortable and available to the patient.
Development of electroencephalographic pattern classifiers for real and imaginary thumb and index finger movements of one hand / Konstantin M. Sonkin, Lev A. Stankevich, Julia G. Khomenko, Zhanna V. Nagornova, Natalia V. Shemyakina // Artificial Intelligence in Medicine. 2015, Vol. 63, No 2, p. 107- 123.
This study aimed to find effective approaches to electroencephalographic (EEG) signal analysis and resolve problems of real and imaginary finger movement pattern recognition and categorization for one hand. Eight right-handed subjects (mean age 32.8 [SD = 3.3] years) participated in the study, and activity from sensorimotor zones (central and contralateral to the movements/imagery) was recorded for EEG data analysis. In our study, we explored the decoding accuracy of EEG signals using real and imagined finger (thumb/index of one hand) movements using artificial neural network (ANN) and support vector machine (SVM) algorithms for future brain–computer interface (BCI) applications. The decoding accuracy of the SVM based on a Gaussian radial basis function linearly increased with each trial accumulation (mean: 45%, max: 62% with 20 trial summarizations), and the decoding accuracy of the ANN was higher when single-trial discrimination was applied (mean: 38%, max: 42%). The chosen approaches of EEG signal discrimination demonstrated differential sensitivity to data accumulation. Additionally, the time responses varied across subjects and inside sessions but did not influence the discrimination accuracy of the algorithms. This work supports the feasibility of the approach, which is presumed suitable for one-hand finger movement (real and imaginary) decoding. These results could be applied in the elaboration of multiclass BCI systems.
EEG pattern decoding of rhythmic individual finger imaginary movements of one hand / L.A. Stankevich, K.M. Sonkin, N.V. Shemyakina, Zh.V. Nagornova, J.G. Khomenko, D.S. Perets, A.V. Koval // Human Physiology. January 2016, Volume 42, Issue 1, pp 32–42. doi:10.1134/S0362119716010175
The results of four-class classification of the motor imagery EEG patterns corresponding to the right hand finger movements (little finger, thumb, index and middle fingers) of eight healthy subjects are presented in this study. The motor imagery of individual right-hand finger movements was executed by the subjects in a prescribed rhythm and the trials contained no external stimuli. Classification was performed by means of a specially developed two-level committee of classifiers on the basis of support vector machine and artificial neural networks at the first level and by generalizing an artificial neural network at the second level. The area under the EEG signal curve and the curve length calculated in a sliding time window for sites F3, C3, and Cz of the International 10-20 system were selected as the key features of signals from the sensorimotor and adjoining frontal cortical areas contralateral to the movements. The average accuracy of four-class singletrial classification for all subjects was 50 ± 7 [SD] (maximum, 58%) for the pair of sites F3–C3 and 46 ± 11% [SD] (maximum 62%) for the pair of sites C3–Cz with a theoretical guessing level 25%.
Human-Robot Interaction Using Brain-Computer Interface Based on EEG Signal Decoding/ Lev Stankevich and Konstantin Sonkin // In: Ronzhin A., Rigoll G., Meshcheryakov R. (eds) Interactive Collaborative Robotics. ICR 2016. Lecture Notes in Computer Science, vol 9812. Springer, Cham DOI: 10.1007/978-3-319-43955-6_13
This study describes a new approach to a problem of the human-robot interaction for remote control of robot behavior. Finding a solution to this problem is important for providing control of robots and unmanned vehicles. At the interaction a human operator can form commands for robot control. It is proposed to use a noninvasive brain-computer interface based on the decoding of signals of brain activity during motor imagery to generate the supervisor commands for robot control. The principles of the interaction of human as an operator and robot as an executor are considered. Using the brain-computer interface the operator can change robot behavior without any special movements and modules embedded into robot’s program. The study aimed to development of the human-robot interaction system for non-direct control of the robot behavior based on the brain-computer interface for classification of EEG patterns of imaginary movements of one hand fingers in real-time. Example of such human-robot interaction realization for Nao robot with neurofeedback is considered.
Neurological Classifier Committee Based on Artificial Neural Networks and Support Vector Machine for Single-Trial EEG Signal Decoding / Konstantin Sonkin, Lev Stankevich, Yulia Khomenko, Zhanna Nagornova, Natalia Shemyakina, Alexandra Koval, Dmitry Perets // Advances in Neural Networks – ISNN 2016. Lecture Notes in Computer Science. vol 9719. Springer, Cham. DOI: 10.1007/978-3-319-40663-3_12
This study aimed to finding effective approaches for electroencephalographic (EEG) multiclass classification of imaginary movements. The combined classifier of EEG signals based on artificial neural network (ANN) and support vector machine (SVM) algorithms was applied. Effectiveness of the classifier was shown in 4-class imaginary finger movement classification. Nine right-handed subjects participated in the study. The mean decoding accuracy using combined heterogeneous classifier committee was −60 ± 10 %, max: 77 ± 5 %, while application of homogeneous classifier based on committee of ANNs −52 ± 9 % and 65 ± 5 % correspondingly. This work supports the feasibility of the approach, which is presumed suitable for imaginary movements decoding of four fingers of one hand. These results could be used for development of effective non-invasive BCI with enlarged amount of degrees of freedom.