Current State of Non-wearable Sensor Technologies for Monitoring Activity Patterns to Detect Symptoms of Mild Cognitive Impairment to Alzheimer’s Disease
Table 4
Summary of performance metrics corresponding to best performed machine learning technique reported in each study.
Study
Target
Machine learning technique
Model evaluation including cross-validation method
Predict the absolute test score using sensor-derived features (regression)
SVR (RBF)
For both regression and classification analysis, 10-fold CV was performed, and models were tested with the internal dataset gathered; no mention of evaluating the model on any external dataset.
NA
Correlation coefficient (): 0.55 MAE: 5
For mobility tests, TUG demonstrated a moderate to strong correlation with sensor-derived behavioral features.
(i) R studio for computing time series statistics (ii) Weka for prediction modeling (correlation and classification analysis)
Detect a reliable change in health assessment scores; no decline (+ve class) vs. decline (-ve class)
As a first step, model was trained and tested with this 80 : 20 split; as a second step, in order to find the validity of the model, a leave-one-subject-out CV was performed using only the 22 subjects transitioned to MCI during study period. No mention of any external dataset for evaluation.
0.5 score: 0.789
NA
A time frame of 20 weeks was found to be the duration that generates room activity distributions that are most conducive to detecting MCI in older adults.
Entire dataset was divided into three groups for a 3-fold CV such that each group had approximately the same total number of data points (feature vectors) pertaining to each class (cognitively intact and MCI). No mention of any external dataset for evaluation.
AUC-ROC: 0.97 AUC-PRC: 0.93
NA
Walking speed-related features were more effective in predicting the MCI condition than any other features. Analyzing activity features for a time window of 24 weeks yielded a best performance.
Initial behavioral model was created with the entire dataset. Later, the resulting model was used as a standard model to be compared with the same dataset days one by one. There is no clarity as to what were the ground truth in evaluating the outcomes of this approach.
NS
NA
Person’s movement transitions from one location to another location were used for behavior modeling. Classifier performance was not explained.
Entire dataset was divided into 3 partitions one each for training, validation, and testing by fixed number of days, i.e., 139 days : 15 days : 70 days on 224 days of monitoring. No mention of any external dataset for evaluation.
Sensitivity: 98.67% Specificity: 75.48%
NA
Study results report that LSTM are more suitable to detect repetition and order-related abnormal activities since it can relate current input with the upcoming ones.
Keras Deep Learning library’s and Theano’s implementation of CNNs and LSTM
Map the sensor-derived activity features to the direct observation scores (regression)
SVM regression (with bagging)
NS
NA
Correlation coefficient (): 0.58
The correlation () between smart home sensor derived features and task accuracy scores was found to be statistically significant.
MCI vs. CH
SVM (with cost sensitive learning)
Leave-one-out CV with internal dataset gathered; no mention of any external dataset for evaluation.
-score (class A): 0.37 -score (class B): 0.78
NA
Classification performance was not strong due to the individuals in these two groups do have quite a bit of overlap in functional performance (activities).
Dementia vs. CH
SVM
Leave-one-out CV with internal dataset gathered; no mention of any external dataset for evaluation.
-score (class A): 0.93 -score (class B): 0.99
NA
Classification performance was best because the individuals in these two groups exhibited a vast difference in performing the scripted tasks/activities.
AUC: area under the curve; CH: cognitively healthy; CIN: cognitively intact; CNN: Convolutional Neural Network; CV: cross-validation; HMM: Hidden Markov Model; iADL: instrumental activities of daily living; LSTM: long short-term memory; MAE: mean absolute error; MCI: mild cognitive impairment; NA: not applicable; NS: not specified; PCA: principal component analysis; PRC: precision-recall curve; RBF: radial basis function; RF: Random Forest; ROC: receiver operating characteristic curve; SVM: Support Vector Machine; SVR: Support Vector Regression; TUG: Timed Up and Go.