Abstract
In rehabilitation, the correct performance of the exercises the specialist prescribes wrist movement is crucial. However, this may have the disadvantage of the patient’s subjectivity. Moreover, recent studies show that feedback through electrostimulation devices is beneficial during the process that leads to neuromotor rehabilitation. Besides, the electromyographic (EMG) signals give information about the actual degree of rehabilitation. This work examines whether temporal features can be used to classify wrist movements using back-propagation artificial neural networks and superficial EMG (sEMG) signals. The data for the evaluation were based on the information acquired from sEMG signals of two forearm muscles: the flexor carpi ulnaris (FCU) and the brachioradialis (B). These sEMG signals were analyzed to find the most critical parameters for classifying the wrist’s movement and to configure a multilayer perceptron (MLP) capable of classifying such movements.
1. Introduction
As the number of people with impairments increases globally, rehabilitation research is becoming more prevalent. Virtual and augmented reality has recently been applied to medical rehabilitation [1]; scientific tests have proven the effectiveness of this type of system for the recovery of people with disabilities.
Disability has a personal and social effect since it influences the patient’s family and community. The economic and public health effects of disability are also significant. As the number of people with impairments continues to rise globally, the importance of rehabilitation research becomes more apparent. Virtual and augmented reality technology has emerged as a promising tool in medical rehabilitation, with scientific studies confirming its effectiveness in aiding the recovery of individuals with disabilities. The impact of disability extends beyond the individual, affecting their families and communities personally and socially. Moreover, the economic and public health implications of disability are substantial. Therefore, it is crucial to continue exploring innovative approaches like virtual and augmented reality; disabled people are disproportionately impacted by the COVID-19 pandemic [2].
According to the World Health Organization (WHO), around 15% of the global population lives with some disability, which amounts to over 1 billion individuals. Up to 190 million people, or 15% of the disabled population worldwide, are aged 15 or older [3]. Around 4.9% of the population in Mexico is handicapped [4]. According to the data, there are six forms of impairment, with motor disability being the most prevalent (Figure 1). The total percentages in Figure 1 may be more than 100% since a single individual may have several disabilities [2].
The upper limb is among the most demanding motor impairments due to its impact on everyday living; these difficulties may make writing, lifting, or holding objects difficult. Due to the plethora of duties associated with this body part, the hand merits special care, and office-related labor is primarily to blame for the rise in wrist disorders. The articulation of the wrist is crucial for performing hand and finger motions. In fine motor disabilities [5], the inaccuracy of upper limb movements is caused by a lack of strength and coordination; being the hand the most adaptable and essential instrument for performing fundamental actions in daily life, these detrimental consequences are significant when exhibited in hand.
Specific rehabilitation protocols exist for treating wrist issues; however, most rehabilitation methods are mechanical or electromechanical [6, 7], which lack quantitative information that can serve as a reference for the therapist to establish a diagnosis and more accurately monitor the patient’s rehabilitation progression, it is inefficient to measure each patient’s performance during each rehabilitation session. Vibratory stimuli have been used in rehabilitation routines [8–10]. Mechanical vibration, usually called vibrotactile feedback [10], is used as feedback in rehabilitation therapies to inform the patient about the success/failure in performing a task during rehabilitation therapy; the feedback provided by the vibration provides sensory enhancement [11].
Artificial neural networks (ANNs) are computational models that resemble biological nervous systems. ANN is integrated by building blocks, namely neurons, which are interconnected for processing elements. Neural networks are the base for some artificial intelligence techniques proposed for processing superficial electromyographic (sEMG) signals. This technique suits a real-time application resembling sEMG signal analysis [12, 13].
Boca and Park [14] have proposed an ANN to accurately recognize the myoelectric signal (MES) in a real-time application [14]. According to Boca, the MES extraction was made using Fourier analysis and clustered using the fuzzy c-means (FCM) algorithm. FCM is a classification technique that simultaneously allows data pertaining to different clusters. The neural network output is related to the degree of desired muscle stimulation.
Furthermore, different authors have shown that multilayer perceptron (MLP), which is a subtype of ANN, can be considered universal approximators, having the advantage of needing few hidden layers. MLPs can map an arbitrary function, provided enough hidden units are available [15]. Hence, due to the mentioned experience, a variation of MLP using a back-propagation (BP) algorithm emerges. The main feature of this so-called back-propagation artificial neural network (BP-ANN) is that it can be trained to map or implement decision surfaces separating pattern classes [16–18]. An MLP is composed of a set of input units, a set of output units, and a set of processing hidden units, and each unit is named a layer; the input layer handles the information provided to the MLP, the output layer reports the result, and hidden layer link the inputs to the outputs. The hidden layers extract desired features from the input layer; those extracted features are used to predict the values of the output layer.
The layers forming MLPs are based on adaptive weights and neurons. Between each layer, there are connections linking neuron layers to each other. Each unit transmits signals to those connected with its output; each unit possesses an output function or transference signal that turns the unit’s present state into an output signal. The feedforward links between the units of the layers in an MLP represent the adaptive weights that permit the connection between the input and the output variables.
In an MLP network, each neuron unit i receives input from precedent units or an external source. Each input Xij has an associated weight Wij, meaning the input varies its impact according to the related weight. The output Yi is computed by some mathematical function f called the “activation function.” The mathematical representation of unit i is:
This study demonstrates how combining electrical stimulation, audio-visual-tactile feedback, and EMG monitoring enables the development of a rehabilitation tool that allows people with disabilities to perform rehabilitation exercises entertainingly and efficiently, resulting in more effective patient recovery therapies. Identifying the movements that the patient develops is crucial for a virtual reality platform in which visual, sound, or even tactile stimulations are related to measurements of EMG parameters concerning the patient’s reaction in such a virtual environment.
2. Materials and Methods
Thirty volunteers performed four different wrist movements: extension, flexion, pronation, and supination; all the movements were selected because they are basic movements performed in most rehabilitation routines. The volunteers were selected from a local rehabilitation center, and exclusion parameters include that the patients did not have heart problems, pregnancy, risk of stroke, or heart attack. The inclusion criterion is that volunteers need physical rehabilitation because of muscular problems. The average age of the participants is 36.17 years, with a standard deviation of 6.92 years. All participants were informed about the procedure, and informed consent was signed according to International Standards. The volunteers keep holding his right arm and grasp his hand. In this hand–arm position, the behavior of the wrist is neutral. The sEMG signal was acquired while the subject performed the mentioned movements.
2.1. SEMG Acquisition
AgCl electrodes perform signal acquisition, and skin-electrode contact is ensured by following the guidelines of the sEMG standards for the noninvasive evaluation of muscles (SENIAM) [19]. The volunteer’s skin was shaved, and an alcohol swab was used to clean the skin to enhance the signal acquisition further. The electrode position was according to the SENIAM recommendations; moreover, guidelines stipulate the distance between electrodes. The sEMG information was acquired from two forearm muscles: flexor carpi ulnaris (FCU) and brachioradialis (B), as Figure 2 shows.
Using a data acquisition device, the sEMG signal was digitized at 1,024 samples per second. This acquisition device contains a PIC18F4550 microcontroller with a USB connection to the computer. An interface software has been developed in Matlab (R2020b) for processing the signal. Next, analog sEMG signals are converted into digital data format with a PIC18F4550 microcontroller with an embedded 10-bit analog-to-digital converter (ADC), which interfaces the analog signals with the computer. The microcontroller uses a 48 MHz crystal for minimum error in the baud rate generator, thus reaching a throughput of 9,600 baud in the serial transmission. The ADC possesses its clock, independent of the main crystal; it gets a 1 kHz sampling rate per each sEMG signal channel, two channels at a time; another possible configuration is sampling a single channel at 2 K-samples/s. Figure 3 shows the methodology used for capturing and processing sEMG signals. There are three stages: the amplification, the filtering stage, and the ADC.
The signals normalization is achieved by having a calibration stage in which the values of that session can be recorded, and the variations due to the intrinsic factors to the measurement and the patient’s characteristics, which are considered before the neural network itself to identify the movement would be adjusted.
2.2. Feature Extraction
Feature selection is crucial in any pattern classification system, and the wrong choice can affect the process of feature extraction. Using multiple feature parameters for sEMG signals pattern classification is desirable since extracting a feature parameter that perfectly reflects the unique feature of the measured signals to a motion command is challenging. However, this work analyzes different characteristics to determine a parameter essential to provide sufficient information so that the BP-ANN model can determine the movements of the wrist.
The feature parameter inclusion must consider the separability from the already included features; additional feature parameters can degrade the overall pattern recognition performance if there is a small separability among features. The feature extraction was made from a temporal approach, generating a hybrid vector that identifies the characteristics of the sEMG signals.
Four features in the time domain are described in this section. Because of their computational simplicity, time domain features or linear techniques are the most popular in sEMG signal pattern recognition. Mean absolute value (MAV), root mean square (RMS), waveform length (WL), and variance (VAR) can all be done electronically in real-time, and it is simple to implement. According to Khokhar et al. [20], the first 400 ms of the signal contains information essential for classification; for this reason, extraction signal has been performed with this amplitude window.
2.2.1. Mean Absolute Value
The MAV is calculated by averaging the absolute value of the sEMG signal. This feature helps set muscle contraction levels, which is usually needed in myoelectric control applications. It is defined as:
2.2.2. Root Mean Square
RMS is described as an amplitude-modulated Gaussian random process whose RMS is connected to the constant force and nonfatiguing contraction. It pertains to standard deviation, which can be written as:
2.2.3. Waveform Length
One feature that provides information about the complexity of a signal in a segment is defined by the WL. WL is merely the cumulative distance between two consecutive samples defined as:
2.2.4. Variance
The VAR of sEMG signals uses the power of the sEMG signals as a feature. Generally, the variance is the mean value of the square of the deviation of that variable. However, the mean value of sEMG signals is close to zero. In consequence, VAR can be calculated by:
2.2.5. Temporal Approach
All of these features mentioned above are computed based on sEMG signal amplitude. From the experimental results, the pattern of these features is similar. Hence, the most robust features representing this group’s other features were selected.
2.3. Neural Network
An ANN was developed for each parameter (MAV, RMS, WL, and VAR), and another ANN was performed for each of the four movements studied (extension, flexion, pronation, and supination).
The BP algorithm is added to the ANN model to improve the model behavior. The learning rate (η) and the momentum coefficient (µ) parameters are included in this algorithm for better behavior. No universal rule exists for calculating the optimal values of µ and η for a particular application. However, heuristic approximations have been developed for tunning µ and η values; following the method proposed by Wythoff [14], Gasteiger and Zupan [21], and Swingler [22], the µ and η values are obtained. The values of μ and η should be in the range (0–1), and it is suitable and reasonable to use high values to accelerate learning while avoiding instability. In this case, the best values of μ and η, calculated by trial and error, were 0.7 and 1 for the hidden layer and 0.3 and 0.4 for the output layer, respectively.
Furthermore, the batch training mode was selected because it presents a balanced behavior between accuracy and speed, and the heuristic of Haykin has been used to initialize the weights. The stopping criterion for all machine learning alternatives was to perform several epochs. The best value through simulations will be located within the range (1,000–200,000).
The input vector (Vinp) of MLP is created based on the feature extraction parameters explained in the previous section (MAV, RMS, WL, and VAR). The output vector (Vout) describes the probability of selecting four wrist motions: extension, flexion, pronation, and supination.
The number of patterns available is 200, with an allocation of 160 trials for the training process and 40 for the test and the correct distribution of classes within each set. The total number of patterns and the distribution across classes may be sufficient to obtain satisfactory results.
The output to be calculated in classification is defined by the output set (Vout) and reflects the probability of classifying each pattern in the corresponding subset, based on Bayes` theorem. As additional information, the correct classification percentage is obtained with Equation (6) during the training and testing phase.where
– P number of output neurons;
– N number of patterns and dyij denormalized output obtained for the i pattern in the output j;
– ddij real denormalized output for the pattern i in the output j.
The back-propagation multilayer perceptron (MLP-BP) architecture defined in this research is 30 neurons in the input layer, 25 neurons in the hidden layer, and four neurons in the output layer. The more essential features of the MLP network defined in this research are detailed in Table 1.
3. Results
Figure 4 (a) shows the raw sEMG signal and Figure 4(b)–4(e) shows the values obtained in the temporal approximation MAV, RMS, WL, and VAR, respectively. Figures 4(b) and 4(c) present a remarkable similarity. The formula for obtaining the MAV and RMS parameters is similar. However, these characteristics differ from each other. In the classification section, an ANN for each parameter was made. With this, it was possible to assess which parameter has greater accuracy in classifying movements. Values are plotted concerning samples obtained by motion. The input to the BP-ANN model is a vector of 30 data extracted from the raw EMG signal. BP-ANN model was performed for each processed feature. In total, four ANN models were implemented in this research.
(a)
(b)
(c)
(d)
(e)
Table 2 shows the error values and the success classification percentage for the MAV corresponding to each of the four movements: extension, flexion, pronation, and supination. The main regression metrics, mean average error (MAE), normalized mean average error (NMAE), and mean squared error (MSE) are included for robustness comparison purposes.
Tables 3–5 show the error values and the success classification percentage for the RMS, WL, and VAR corresponding to each of the four movements: extension, flexion, pronation, and supination.
A comparison with previous research found in the literature is detailed in Table 6. This table compares parameters such as classification method, number of acquisition channels, and system accuracy. As shown in Table 6, the proposed system obtains more accurate results than other approaches based on two channels. Furthermore, the system gets more accurate results than approaches based on more channels (except from [28], which uses four channels, and other muscles are considered).
Additional information on the performance of classification is presented at Table 7. Table 7 includes precision and recall for each movement and each temporal approach.
4. Discussion
The obtained results allow for generating applications in a broad spectrum of areas; however, this knowledge will be used to develop a rehabilitation system for upper limbs, which also involves an augmented reality interface. The results obtained are promising, and in the future, the optimization techniques to analyze and improve the input patterns will be integrated to increase the system’s accuracy.
Despite the MAV and the RMS temporal signals appearing similar, the specifically designed BP-ANN implementation provides enough information to classify the performed movements with an accuracy 98.86% in the best case using the RMS temporal approach. The low computational cost of the data algorithms employed in this work makes it possible to implement such algorithms in portable devices.
According to the experimental results, it has been found that to lower the computational cost, the processed data must remain at a minimum; therefore, it is essential for sEMG signals to analyze the frequency below 600 Hz because most information is below that frequency. By lowering the computational cost, the monitoring system can be implemented at clinics to track the correctness of the rehabilitation routine proposed by the specialist. Therefore, the execution of rehabilitation exercises can be objectively monitored, thereby increasing the efficiency of therapies.
As Table 6 points out, the accuracy reached by the proposed system is comparable to that reported in the literature, but the lower computational cost makes it easier to implement than traditional systems reported in the literature. With an accuracy of over 90% using MAV and RMS, the system behaves accurately for classifying the proposed movements performed by volunteers. These results give promising results in rehabilitation therapy supervision. This accuracy is expected to increase even more as it is trained with the patient’s values and the indicated pronation, supination, flexion, and extension movements.
This work has been adapted for use in a virtual environment, but specific conditions can affect the study accuracy, such as muscular fatigue, electrode place, electrode aging, and electromagnetic interference. The affecting causes have been reduced by controlling the patient’s movements to have more control over the signals generated. On the other hand, a calibration stage was performed before the rehabilitation session to adjust the values recorded in that session.
5. Conclusions
The results are promising since the classification was made with information from only two acquisition channels. The system’s validity was determined by comparing the results obtained in different studies.
The proposed system has the advantage of using the least number of channels; it identifies which parameters provide the necessary information to perform gesture recognition, and this entails a lower computational cost. The accuracy is comparable to those systems with greater acquisition channels but offering lower computational costs.
This work shows the feasibility of its implementation in real-time in combination with a virtual environment by using only two channels to obtain a reliable identification of the movements accurately. Although there are antecedents of the implementation of AI techniques in EMG studies, as pointed out, these require a higher computational cost, making their implementation difficult in applications of a combination of techniques.
The study’s primary contribution is to demonstrate that the characteristics of temporal approximation are the most relevant. On the other hand, the research aims to provide more information about electromyography signals for application in AI systems and gesture recognition. The low computational cost makes it possible to track the rehabilitation routines to increase the therapy’s success. Finally, the authors believe this performance was achieved thanks to the judicious process of detecting and acquiring clear signs of SEMG and the improvements made in estimating the temporal approach.
Data Availability
Support data can be obtained from corresponding authors upon request.
Ethical Approval
All the experiments were conducted with people respecting the WMA Declaration of Helsinki—Ethical Principles for Medical Research Involving Human Subjects.
Consent
Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patients to publish this paper.
Disclosure
J. J. Agustín Flores Cuautle: Visiting Professor at Centro Universitario UAEM Valle de Mexico.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
R.P.G. contributed in the conceptualization. A.M.S. and A.A.L. contributed in the data curation. J.J.A.F.C. contributed in the formal analysis. M.R.F. and J.J.A.F.C. contributed in the investigation. A.A.L. contributed in the methodology and software. A.M.S. contributed in the project administration. R.P.G. contributed in the supervision. M.R.F. contributed in the writing–original draft. R.P.G. and J.J.A.F.C. contributed in the writing–review and editing. All authors have read and agreed to the published version of the manuscript.