Development of electroencephalographic pattern classifiers for real and imaginary thumb and index finger movements of one hand / Konstantin M. Sonkin, Lev A. Stankevich, Julia G. Khomenko, Zhanna V. Nagornova, Natalia V. Shemyakina // Artificial Intelligence in Medicine. 2015, Vol. 63, No 2, p. 107- 123.
This study aimed to find effective approaches to electroencephalographic (EEG) signal analysis and resolve problems of real and imaginary finger movement pattern recognition and categorization for one hand. Eight right-handed subjects (mean age 32.8 [SD = 3.3] years) participated in the study, and activity from sensorimotor zones (central and contralateral to the movements/imagery) was recorded for EEG data analysis. In our study, we explored the decoding accuracy of EEG signals using real and imagined finger (thumb/index of one hand) movements using artificial neural network (ANN) and support vector machine (SVM) algorithms for future brain–computer interface (BCI) applications. The decoding accuracy of the SVM based on a Gaussian radial basis function linearly increased with each trial accumulation (mean: 45%, max: 62% with 20 trial summarizations), and the decoding accuracy of the ANN was higher when single-trial discrimination was applied (mean: 38%, max: 42%). The chosen approaches of EEG signal discrimination demonstrated differential sensitivity to data accumulation. Additionally, the time responses varied across subjects and inside sessions but did not influence the discrimination accuracy of the algorithms. This work supports the feasibility of the approach, which is presumed suitable for one-hand finger movement (real and imaginary) decoding. These results could be applied in the elaboration of multiclass BCI systems.
Вrain-computer interface (BCI) is a new technology that allows an individual to influence the environment through his or her thoughts, i.e. through decoding mental commands recorded as signals in the electroencephalogram (EEG) or other methods of brain activity recording. In this case mental commands decoded by a special program are used to control an external device: a manipulator, robot, wheelchair, cursor in a computer program, etc.
This way of communication with the external environment is particularly important for people who are paralyzed or have severe motor disabilities. In this case BCI can substitute movements, i.e. an individual can communicate with external environment through BCI only. Besides that, there are special methods of motor function recovery based on BCI principles.
Even when motor functions are impaired the ability to imagine the necessary movement and give a mental command remains intact. This is important for rehabilitation, as numerous studies have shown that imaginary movements activate the same brain areas as actual movements. What is more, mental visualization of movements is used by athletes in training process (the so-called «motor imagery practice»). Thus, repeatedly imagining the necessary movements and making mental attempts to perform them, one can improve his or her nerve tissue recovery, involve into action new neurons instead of those impaired by disease, and as a result to enhance the rehabilitation process.
Anyway, rehabilitation is a long and time-consuming process, and recovery does not always come fast. If patients do not see the result of their efforts it might discourage them from doing exercises and kill their motivation. «Feedback» is needed – patients should see how well he performs imaginary movements and gives mental commands. Even better results can be achieved if mental commands for movements will be accompanied by a passive movement of a leg or an arm, performed for a patient by a special robotic device, or a movement performed as a result of electrical stimulation of the muscles provoking the necessary movement. In the nervous system at the same time the mental effort is becoming associated with the performed movement (though passive yet) and this also speeds up the rehabilitation process.
Another possible way of getting feedback is showing the desired movement on a computer screen with the help of a model of an arm or a leg, or using mental motor commands to control a computer game.
The key problem for the realization of this technology of rehabilitation is a reliable method of mental command recognition/classification. Besides, the more subtle and complex movement is imagined, the more difficult it is to recognize. Recognition of EEG patterns corresponding to their mental commands is the most promising method in this area since the electroencephalogram is a non-invasive method of brain signals recording, safe for the patient, mobile and relatively cheap.
The aim of our research is first of all to create a reliable EEG based classifier for the commands of fine motor skills (finger movements).
Development of the BCI system is based on our own research: we record the EEG of healthy volunteers and use different approaches to EEG signal processing and analysis.
We applied several new approaches to solve the problem of motor command recognition.
“Rhythmic paradigm”: imaginary movements / mental commands are performed to a given rhythm (rhythm is set by sound signals) which allows to speed up the recognition of mental commands by the BCI system and make it more reliable.
To interpret mental commands we developed a special computer program based on complex learning algorithms - a two-level combined classifier committee of EEG signals.
When designing the classifier we used the most suitable algorithms for complex signals: artificial neural networks (ANN) and support vector machine (SVM). To recognize signals we use several types of EEG signal features simultaneously, which allows the classifier to adjust to each individual and increases the reliability of mental command recognition.
We use different types of EEG signals transformations and for each individual the most suitable one is chosen. It was found that in most cases current source density (CSD) transform gives the best results in mental command recognition. As a result, it was possible to achieve the accuracy of the classification of motor commands well above the random guessing level.
The figure below shows the results of classification for 4 types of mental commands.
Thus, the application of new approaches allows us to successfully solve the problem of mental command recognition, even for the complex case of multiclass classification of fine motor commands. Recognition of mental commands with high precision will make it possible to create a rehabilitation system with feedback – a patient will be able to see on screen the results of their mental efforts on the monitor screen, recognized mental commands will be used to control a computer game (application), a 3D model of a hand, or a robotic device that will perform the movements imagined by the patient.
The results obtained in the course of our scientific research are used to develop an innovative software and hardware system based on brain-computer interface for rehabilitation of patients with motor disabilities, which is going to make rehabilitation more successful, quick, comfortable and available to the patient.