Abstract
Smartphones with gym exercises predictors can act as trainers for the gym-goers. However, various available solutions do not have the complete set of most practiced exercises. Therefore, in this research, a complete set of most practiced 26 exercises was identified from the literature. Among the exercises, 14 were unique and 12 were common to the existing literature. Furthermore, finding suitable smartphone attachment position(s) and the number of sensors to predict exercises with the highest possible accuracy were also the objectives of the research. Besides, this study considered the most number of participants (20) as compared to the existing literature (maximum 10). The results indicate three key lessons: (a) the most suitable classifier to predict a class (exercise) from the sensor-based data was found to be KNN (K-nearest neighbors); (b) the sensors placed at the three positions (arm, belly, and leg) could be more accurate than other positions for the gym exercises; and (c) accelerometer and gyroscope when combined can provide accurate classification up to 99.72% (using KNN as classifier at all 3 positions).
1. Introduction
The advancement in the technology troubled humans by making their lives busy. This is affecting their health negatively [1]. However, the technology also helps humans to improve their health, education, business, and social relationships [2]. The beneficial impact of technology is tremendous, especially in the health sector. Multiple hardware and software [3, 4] are used to improve overall human health. Among various sources of maintaining health, gyms are the major source of physical fitness.
People join gyms to achieve goals like bodybuilding, physical fitness, or losing weight. In the modern world, technology has replaced the traditional concepts of guidance and training to stay healthy and fit. The tools like smartphones and devices like wearable gadgets are among the many resources that are helping to stay healthy and fit [5–8]. There is also some researches like [9–11] that support the notion that technology can help to achieve fitness objectives. Besides, there are various smartphone applications like [12–14] that can track different physical activities, e.g., walking, running, sitting, and standing with the corresponding calorie burn out. The sensors (accelerometer, gyroscope, etc.) are used to track the activities.
Many wearable devices and smartphone applications track physical activities and calorie burnout like [3, 4]. However, none of the studies provides the information appropriate to measure major gym activities. For example, research studies such as [9, 11, 15] targeted a group of upper body muscles along with some warm-up exercises only. This research is a similar attempt yet is different in many aspects. First, in this research, 14 exercises of different muscle groups (abdominal, upper body, and lower body) are added to move towards a complete solution.
Second, in most of the existing research, the position of the sensors or the devices was only at the arm. Seeger et al. [16] used three sensors at the following positions: a single accelerometer at the wrist, a hand glove, and a sensor at torso position. We hypothesized that the use of accelerometer and gyroscope at three body positions (arm, leg, and belly) could enhance accuracy because of the dependence of various gym exercises on either of the positions individually or in combination.
The third aim is to determine the number of sensors required to detect an exercise accurately. At the hypothesized three positions, five sensors are used: at the arm and the leg, the accelerometer and gyroscope together, while on the belly, only the accelerometer. The single sensor at the belly will only be used to determine the laying (x-axis), standing (y-axis), or in-between position that is often used in the gym exercises (e.g., angle leg press). The contribution of these sensors towards the accuracy has been analyzed in this research as well.
The classification algorithms used to detect exercises in the related work such as [7, 9, 11, 15–17] are linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), K-nearest neighbor (KNN), Naïve Bayes (NB), support vector machine (SVM), and dynamic time wrapping (DTW) algorithms. The result of the accuracies achieved by the studies was promising. However, their used datasets were quite sparse (collected from 8 to 10 persons) for 43 unique exercises. The exercises were also related to each other or were exercises from the same muscle groups having the same activity motion/patterns. The fourth aim of this study is to increase the number of participants in the real-world settings to bring more rigor to the findings. The increase in the number of participants and thus dataset could also affect the choice of the exercise detection algorithm. This forms the fifth aim, the selection of the most appropriate algorithm(s) to detect gym exercises.
The rest of the paper is organized as follows. Section 2 discusses the relevant literature in the context of the aims of this study. Section 3 is about the materials and methods. In Section 4, experimental setup is elaborated. In Section 5, the analysis and results are discussed. Section 6 concludes the paper as well as identifies some limitations. The section also embarks upon the possible future work.
2. Literature Review
In this section, related work is discussed in the context of the aims of this study. Therefore, this section is divided into the following three subsections: (1) exercise detection, (2) positioning and the number of sensors, and (3) exercise detection algorithms. The participant’s selection is described in the Materials and Methods section.
2.1. Exercise Selection
The first activity recognition study based on wearable sensors device [18] was published in 2000. In this study, they attached two accelerometers inside the trousers’ pocket to recognize daily life activities. The study [19] examines the use of a single smartphone accelerometer in activity recognition. The reported results showed accuracies between 80% and 97% depending on the set of activities used and the processing techniques. Muehlbauer et al. [7] used the arm position to attach an Arm-Hostler with a fixed sensor to recognize a set of ten upper body gym exercises. They reported 93.6% accuracy in more than 90% of the cases they studied. MyHealthAssistant [16] classified the gym exercises using three accelerometers (on the hand glove, wrist, and torso). They trained a Bayesian classifier on the mean and variance features collected via an accelerometer. They collected the data of 11 exercises and achieved 92% accuracy. Chang et al. [8] used 2 accelerometers (on the hand and waist position) and examined a Hidden Markov Model (HMM) and a Bayes Classifier to identify exercises. They achieved 90% accuracy for the set of nine exercises and around 5% of the overall miss-count rate.
The activity recognition data collected from the literature corresponding from the years 2006–2018 found only 6 of 25 research studies related to gym exercises while the remaining 19 of 25 research papers were about daily life physical activities, emotional recognition, and elderly fall detection [20]. All the 25 papers were used to extract the information like the type of sensors used, features used to recognize activities, and the classification algorithms used.
2.2. Position and Number of Sensors
In most of the literature, only a single sensor for activity recognition is utilized. However, some studies used more than one sensor as well. For example, the authors in [21] put both an accelerometer and a gyroscope together and stated that the gyroscope adds nothing to the recognition results. However, some contradictory results are reported by the authors in [22]. The study [10] reported a 3.1 to 13.4 percent increase in recognition accuracy for 08 of 09 activities when an accelerometer is combined with a gyroscope while using the KNN classification algorithm. The average accuracy reported was 83.7% with an accelerometer and 90.2% with both accelerometer and gyroscope with an increase of 6.5% in average accuracy. The study also revealed that the sensor combination provides better results as compared to accelerometer alone. However, the paper does not report individual accuracies, thus resulting in an ambiguity whether the gyroscope or the accelerometer played a major role in the accuracies.
Table 1 provides the details of the number of sensors and device positions as per the literature while Table 2 describes the use of the combination of sensors (sensor fusion) as well as their accuracies.
From the analysis of Table 2, it can be argued that the combination of accelerometer and gyroscope provides the strongest accuracy results. Moreover, in most cases, a gyroscope does improve the recognition accuracy from 3.1% to 13.4% when used in combination with an accelerometer [10]. The magnetometer’s role in activity recognition was poor.
2.3. Exercise Detection Algorithms
The related literature has also used different classification algorithms. For example, the authors of [21] used KNN combined with support vector machine (SVM) and the authors of [22] used KNN combined with decision tree and Naïve Bayes, while the authors of [40] used J48 Decision Tree combined with Naive Bayes for exercise recognition. They reported an average accuracy of 95%, 90.2%, and 88%, respectively. Table 3 shows the top three classifiers Naïve Bayes (NB), decision trees (J48), and K-nearest neighbor (KNN) being abundantly used for the activity recognition purpose.
The accuracy of the results depends on the suitable selection of the classification algorithm as well as on the selection of the suitable parameters for them. Table 4 shows the top three most used features in the literature, that is, mean, standard deviation, and minimum and maximum as classification algorithm parameters.
3. Materials and Methods
In this section, the materials and methods used in the study are discussed. Section 3.1 discusses the selection of gym exercises for the current study. Section 3.2 is about the development of the application used for the data collection process. Section 4 is about experiment and data collection methods used in the study.
3.1. Selection of the Exercises
The process of the selection was started by collecting a list of all the gym exercises from the two sources [41, 42]. The sources listed a total of 74 gym exercises which will be called set TE (Total Exercises). To verify the repeatability of the exercises in gyms, one of the authors visited four most known and commonly used gyms of the city to meet with the gym-goers. They were interviewed about the most common exercises the gym-goers trained on. The results were a subset of 54 most used gym exercises. The set will be called the set SE (Subexercises).
The set SE was compared with a set of the common exercises mentioned in the literature which resulted in (Common Exercises) set CE having 35 exercises. The exercises in CE were further categorized into exercises group along with information like exercise positions and equipment used to do the exercise.
Further analysis of the set CE revealed that five exercises were repeating in different muscle groups with different names. One of the exercises occurred three times and the rest of the four exercises twice in each muscle group. Removing the repetitions from the set CE resulted in a set of 29 exercises.
From the 29 exercises, 3 exercises were related to the head and are considered as warm-up exercises in the literature [43]. These 3 exercises are also removed from the list of 29 exercises reducing the final exercise set (Total Final Exercises) TFE to 26.
The exercises mentioned in the above paragraph were extracted from research papers like [7, 9, 11, 15–17]. Among these references, the study in [17] was related to only gym warm-up exercises and thus was not included in the exercise selection. The remaining five papers were used to form 5 exercise sets (EP1–EP5). Here, E stands for exercise and P for the paper. Thus, EP1 represents exercise set extracted from paper 1, that is, reference [7] and so on. The union of the exercise sets EP1–EP5 was taken resulting in set (Total Exercises from Papers) TEP containing 43 exercises considered in the literature. The set TFE was subtracted from the set TEP to provide 14 unique exercises and 12 exercises that are considered in the literature (Table 5).
3.2. Application Development
To accomplish the objectives of this research, the first requirement was to develop an application to collect data from the participants. For the purpose, an android based smartphone application was developed. The users could add, view, edit, and delete personal profiles. The user interface of the developed application is depicted in Figure 1, whereas the flow of the user’s interaction with the application is elaborated in Figure 2. The application is also provided as a supplementary file with the paper for the researchers who want to replicate the research.
Figure 2 shows the overall process followed in the developed application for the data collection. The start screen provides options for the new users to register themselves, while already registered users can go to the registered users' screen. After the selection of the new registration option, the user could move to the signup screen option. There they either can enter their profile information such as height and weight to register themselves with the new user profile or could go back to the main screen without registration. After clicking already registered users option, users could move to already registered users profile list to select their profile by name. The selected profile screen with information appears about the users from where they can start recording the exercise data and will also start doing the exercise. They can also view their stored records or could go back to the main screen. The users could exit the application from the main screen.
4. Methods
The developed application was installed on 3 smartphones and was positioned as shown in Figure 3. An LG Model F180 was attached to the leg while another similar model was attached to the arm. This model supports both the accelerometer and gyroscope sensors providing the values of acceleration and rotation. For the belly position, we required only one sensor to determine the state (sitting, laying) of the participant. For the purpose Q-Mobile model, i7 was attached at the belly position having the support of only the accelerometer sensor.
The research also aimed to increase the number of participants and to collect varying data. Therefore, 20 participants with two sets of a total of 10 repetitions for each participant were used for the purpose. The 10 repetitions are used in the related literature before such as [11]. The data were collected against a selected set of 26 exercises. The smartphones were attached at three different body positions (arm, belly, and leg). All the gym-goers taking part in data collection were asked to behave normally as their usual exercising day. The sensors X, Y, and Z values were being recorded and stored in a file by the application while performing exercises. All the activities were carried out indoors in a gym.
4.1. Experimental Setup
The experimental setup section is divided into a further four subsections. Section 4.1.1 is about ethical compliance as per involvement and data collection of the participant. Section 4.1.2 explains the demographics of the participants. Section 4.1.3 is about the data collection process. The preparation of the data for the analysis is discussed in Section 4.1.4.
4.1.1. Ethical Compliance
The departmental ethics committee, called Project Research and Evaluation Committee (PREC), approved the study design and the procedure as defined in the above section. Informed consent for the study was obtained from the participants of this study.
4.1.2. Participants
For the selection of the participants, the busiest gym in the center of the city was selected. The gym-goers used to visit the gym regularly were approached and the aims and objectives of the data collection were explained to them. The 20 participants all males volunteered to participate in the data collection process. The participants were between the age brackets of 20 and 35. Their mean age was 25.85 years with SD of 4.13. Their heights ranged from 162 to 181 cm with a mean of 171.1 cm and SD of 5.34. Their weights ranged from 62 to 80 kg with a mean of 68.1 kg and SD of 5.56, respectively. The gym experience of the participant was between 2 and 19 months with a mean of 9.35 months and SD of 4.90. All the exercises were completed with free weights (participants choose weights themselves).
4.1.3. Data Collection
The data were recorded from 5 sensors (two sensors of the smartphone attached at the leg, two attached at the arm, and one attached at belly). All the sensors recorded X, Y, and Z values while the participant was doing the exercise. A triaxial accelerometer estimates the acceleration along X, Y, and Z axis and gyroscope (Pitch, Yaw, and Roll) helps the accelerometer to predict the orientation of the sensor. Three smartphones were synchronized to get the time from the server. The time was recorded up to millisecond along with X, Y, and Z values. This resulted in the 15 X, Y, and Z values along with a timestamp, the category of the exercise, and the exercise name. The dataset available from the literature [17, 44, 45] was not used because of the nonavailability of the data of 14 unique exercises. We also decided to collect data for the exercises whose data was available because of the probable setup differences between the existing studies and this study. This may have help in countering the bias and variations.
4.1.4. Data Preparation
The recognition process includes a collection of exercises data using multiple sensors. The data is preprocessed and segmented and the features are extracted and classified as the last step [11, 46]. The same process is followed in this research as well.
Three different files containing exercise data from each smartphone were combined carefully to match the participant's assigned ids and time stamps. In the second step, the recorded data from CSV files were preprocessed to remove the extra noise. For example, at the start and the end of an exercise, the participant's movements were very random as well as jerky and were not aligned with the required exercise. Therefore, to remove this noise we removed the data from the first 3 seconds and the last 3 seconds of the recorded data of each exercise. For each exercise, there were 2 sets, each set of 10 repetitions and with an average participant time consumption for an exercise of 38 seconds. After preprocessing, we considered the data of 32 seconds only. The application was programmed to record 4 samples in a minute.
Various previous studies such as [11] used a 4-second window to extract required features and 1 minute of the slide to vary the data. We adopted the same strategy. The features extracted were based on the most used features (mean, standard deviation, and minimum and maximum) for the similar nature of the data as presented in Table 3. For each of the X, Y, and Z values, these four features were extracted forming a total of 60 features (36 accelerometer features and 24 gyroscope features).
To analyze the preprocessed data, we used WEKA (Waikato Environment for Knowledge Analysis) [47]. The preprocessed data (extracted features) were converted to ARFF (Attribute-Relation File Format). The listed attributes were named as per the following strategy. In the name of the attribute, the first position character ‘a’ stands for an arm, ‘b’ stands for the belly, and ‘l’ stands for the leg. The second position character ‘a’ or ‘’ stands for accelerometer or gyroscope. The third position character ‘x’, ‘y’, or ‘z’ stands for axis values X, Y, and Z. The selected classifiers NB, KNN, and J48 as per Table 3 were utilized with default configuration settings. In the test options, the percentage split with 80% training and 20% testing option was selected as used also by [38, 48] to evaluate the performance and accuracy of the classifiers.
5. Analysis and Results
The existing research mostly used three classifiers, namely, NB, KNN, and J48 (cf. Table 3) and hence they were also utilized in this research. All of the above-mentioned algorithms can create multiform class boundaries and, therefore, are suitable for the data collected via sensors and devices [10]. Furthermore, for practical applications, these methods are fast and are easily implementable [10].
We examined the values of both the sensors (accelerometer and gyroscope) with the above-mentioned classifiers at three different body positions (arm, belly, and leg). The analysis was done in five ways: firstly, the analysis of the exercises considering the data from three sensors of the same nature, that is, accelerometer (rows 1, 2, and 3 of Table 6) attached at three positions (arm, leg, and belly). As there were only two gyro sensors at arm and leg positions, data from the positions are analyzed and presented as per rows 4 and 5 of Table 6. The same process is continued for the combination of three, four, and five sensors as illustrated in Table 6.
In Table 6, column “sensor name” represents the name of the sensor from which the data is acquired. The number in front of the sensor name represents the count of sensors used to acquire and analyze the data. For example, S numbers 6, 7, and 8 in Table 6 display “accelerometer = 2” which is an indication that two accelerometers attached at the body positions (displayed in the next column) were used to analyze the data. The results of the input data from the chosen three classifiers are presented in the classifier names columns in the form of accuracy. The last column represents the number of features used in the analysis. A single sensor used at body position will have 12 features, two sensors will have 24 features, and so on.
The results revealed that the best accuracy of 99.72% was achieved with the KNN classifier using five sensors at three attachment positions (arm, belly, and leg). However, as can be seen from the summary as per Table 7, this is not a big variation from the accuracy of the KNN using two sensors at two attachment positions. A minimum of two sensors used at the arm and leg position provided an accuracy of 99.27% which is equatable to five sensor positions.
For each (exercise) activity, the accuracies achieved using the KNN classifier with both the accelerometer and gyroscope are a little better than using only the accelerometer. The accuracy results and their difference are shown in Table 8.
The classification confusion matrix in Figure 4 shows that the highest accuracy is achieved using data from all the sensors of the smartphones and with a KNN classifier. Examining the confusion matrix, the results show that most of the classes (exercises) are accurately being predicted. However, a couple of classes (exercises) were not differentiable because of the similarity in the exercise position and nature. For example, the Triceps group (triceps press with cable and triceps press with bar) are having similar motion patterns. However, we still can differentiate between them based on the execution time differences as per Table 8. The table shows that triceps press with bar takes 26.8 seconds for 20 repetitions while triceps with cable take 29.2 seconds for the same 20 repetitions thus having a difference of 2.4 seconds.
6. Conclusion and Future Work
The goal of this study was to predict gym exercises with the help of smartphone sensors in real-world settings. To achieve the goal, exercises from the literature were extracted for which prediction research work was conducted and was intersected with a set of the most used exercises in the gym. The result was 14 unique exercises for this study. Besides, 12 common exercises were also considered for comparison purposes. Furthermore, finding the sensors suitable attachment positions, as well as the number of sensors to utilize in predicting the exercise accurately, was also one of the goals of this research. Also, we conducted the exercises with the greatest number of participants (20) as compared to the existing literature (avg. max. 10). The results indicated three key lessons derived from this study while examining the goals. (a) The most suitable classifier to predict a class from the sensor-based data was found to be KNN. (b) The sensors placed at three positions (arm, belly, and leg) could provide better accuracy than other positions when the gym exercises are under the question and (c) smartphone sensors accelerometer and gyroscope in combination can provide accurate classification (using KNN as classifier at all 3 positions) in most of the activities averaging up to 99.72% accuracy. Their combination can increase accuracy by up to 0.21%.
The research can be implemented in the form of a smartphone application that can be turned on by the users while doing exercises in the gym. In the future, this application can be embedded with a calorie burn out tracker that should be able to guide gym-goers to do which exercise and for how much time? The output could be in the form of sound notifications as well as sound messages that could advise to change or stop the exercise.
The research has some limitations as well. In this research, only 14 unique exercises are considered taking the considered exercises in the literature to 55 exercises. In this context, of the total of 74 exercises as per sources [41, 42], nineteen (19) gym exercises still remain to be predicted though not most often used. The future research work can consider these exercises as well. In addition, in this research, no female participants were involved thus having a probability of nonapplicability of this research for the female participants. The future research could also hire female participants to increase further accuracy.
Data Availability
The data are available within the supplementary information file. However, any query about the research conducted in this paper is highly appreciated and can be asked from the principal authors (Usman Ali Khan and Dr. Iftikhar Ahmed Khan).
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Supplementary Materials
An ARFF (Attribute Relation File Format) file is an ASCII text file that describes a list of instances sharing a set of attributes. ARFF files were developed by the Machine Learning Project at the Department of Computer Science of The University of Waikato for use with the Weka machine learning software. A complete description including information on how to read this file is available at the following URL. https://docs.rapidminer.com/latest/studio/operators/data_access/files/read/read_arff.html. (Supplementary Materials)