Machine learning for identifying patterns in human gait
Zhou, Yuhan
DOI:
10.33612/diss.159240405
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from
it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date:
2021
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Zhou, Y. (2021). Machine learning for identifying patterns in human gait: Classification of age and clinical
groups. University of Groningen. https://doi.org/10.33612/diss.159240405
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
CHAPTER
General Discussion
MAIN FINDINGS
This thesis examines the hypothesis that machine learning can accurately classify individuals by age groups or fallers vs. non-fallers based on their gait because the properties of gait differ between these groups. Additionally, geriatric patients with or without a cognitive impairment may also have comorbidities; therefore, using the combination of gait and standard clinical test variables as inputs could accurately classify these groups rather than inputting only gait or only clinical variables to the models. Gait data as inputs to the machine learning classification models were calculated from 3D-accelerometer time-series signals. The thesis has pursued the idea that we could identify gait variables that contribute the most to the prediction models gait data could help clinicians to diagnose gait and cognitive impairments, and screen for patients with fall risk. Figure 1 summarizes the results of the thesis.
Chapter 2 shows that the Artificial Neural Network (ANN) model can accurately classify healthy
young-middle age adults, healthy older adults and geriatric patients based on 23 dynamic gait outcomes calculated from accelerometer signals at an accuracy of 89%. The advantage of the ANN model vs. other models examined in this thesis is that it outputs each variable’s contribution to the model. Compared with healthy young-middle aged adults, healthy older adults are recognized to have a low amplitude variability, unsynchronized and unstable gait. Geriatric patients vs. the other two age groups are identified to have gait with lower amplitude variability, more unstable and less regular, and less synchronisation. The ANN model also accurately classified geriatric patients with or without cognitive impairment in Chapter 3. The model with the input of integrated clinical data and the same 23 dynamic gait outcomes used in Chapter 2 obtained the most accurate classification (accuracy of 96%). The interaction between clinical and gait variables was a key determinant of improving ANN’s classification accuracy for geriatric patients with or without cognitive impairment. Patients with cognitive decline had poor mobility and an increased risk of mortality, accompanied by unpredictable, unstable, highly variable and irregular gait.
Based on retrospective falls, Chapters 4 and 5 employed the same 27 spatial-temporal gait variables to classify patients with multiple neurological disorders into fallers and non-fallers from disorders. Chapter 4 reported a similar classification performance under single and dual task gait conditions using the Partial Least Square Discriminant Analysis (PLS-DA) method. The model produced an area under a receiver operating curve (AUC) of 0.66-0.77 to classify fallers vs. non-fallers. The gait of fallers vs. non-fallers was characterized by a slower pace, longer swing/stance time, higher variability, shorter stride length, lesser ankle dorsiflexion at heel strike, and greater plantarflexion at toe-off. The results indicate that although these gait variables were highly associated with falls, they did not provide a unique signature for any specific neurological disorder. To improve the classification performance Chapter 4 and 5 subsequently used the Path Signature method to pre-process gait data for six machine learning methods in order to classify fallers from non-fallers under a single task gait condition. The Path Signature method
did significantly improve the RF machine learning classification, with an increase in AUC from 0.77 in Chapter 4 to 0.98 in Chapter 5.
Figure 1. A comprehensive structural model of the thesis. In chapters 2 through 5, different machine learning
models are used to accurately classify different age and clinical populations and identify crucial variables associated with age, cognitive impairment, and falls. The red arrow represents the decreased values of the variables; the green arrow represents the increased values of the variables. PCA: Principal Component Anal-ysis; LDA: Linear Discriminant AnalAnal-ysis; LR: Logistic Regression; NB: Naïve Bayes; SVM: Support Vector Machine; KNN: K-Nearest Neighbour; RF: Random Forest; ANN: Artificial Neural Network; PLS-DA: Par-tial Least Square Discriminant Analysis; BMI: Body Mass Index; TUG: Timed Up and Go; CCI: Charlson Comorbidity Index; AUC: an area under a receiver operating characteristic curve; Y/M: Young-middle age adults; AP: anterior-posterior; ML: medio-lateral; V: vertical.
MACHINE LEARNING METHODS TO COMPLEMENT TRADITIONAL
STATISTICAL ANALYSES
The main findings of this thesis suggest that human gait is a sensitive indicator for identifying the health conditions of aging adults such as cognition and falls. This finding is in line with previous data [1]. The application of a wide variety of machine learning methods to gait classification signifies the increasing sophistication of data science and its application to clinical conditions [2]. The results of these methods could, in addition to standard clinical assessments, help clinicians to understand patients’ gait data and increase the diagnostic accuracy [3]. Recently, the development of wearable technologies such as inertial measurement units have enabled a more accurate capturing of the entire process of gait, thereby accumulating a large number of interdependent gait variables [4]. Dynamic gait outcomes are gait variables calculated from 3D accelerometers representing smoothness, stability, variability and regularity of gait. Dynamic gait outcomes with a
multidimensional and non-linear correlated data structure were employed in Chapters 2 and
3. Although the traditional statistical methods such as student 𝑡-tests and non-parametric tests
are widely used in clinical gait analysis, these tools lack the classification and predictive ability to interpret multivariate dynamic gait outcomes [2]. The robust technique that has emerged to analyse gait data in this thesis is machine learning [5]. Machine learning is result-oriented and provides algorithms for accurate classification and clustering of groups. Although several statistical models are also suitable for prediction, these methods focus on quantifying the relationship between variables and the significance of this relationship, rather than on repeatable classification and predictive modelling [6] [7]. Machine learning has the capacity to automatically learn from data, construct algorithms that involve integrated advanced statistical and probabilistic techniques [8]. Compared with traditional statistical methods, machine learning methods enable the identification of patterns in gait data more readily so that the precision of clinical diagnosis increases.
SELECTION OF A MACHINE LEARNING CLASSIFICATION MODEL
Obtaining accurate gait classification using machine learning models starts by selecting the most appropriate machine learning algorithm that can address the research. The method chosen must have the capability to account for pathological factors affecting gait. The key idea is to determine unique gait variables that can discriminate groups of individuals with a unique clinical condition. The choice of a classifier depends largely on gait variables and their correlation with output outcomes. Six machine learning classifiers were employed in Chapter 5 to classify fallers and non-fallers; however, the input of high dimensional, high correlation and multicollinearity spatial-temporal gait variables decreased the classification performance (averaged accuracy of 63%). When establishing machine learning models based on algorithms such as Linear Discriminant Analysis (LDA), Logistic Regression (LR) and K-Nearest Neighbour (KNN), multiple features must be considered; therefore, it is inevitable multi-dimensionality and high correlation among gait variables unfavourably affect the machine learning classification [9]. It is therefore necessary to develop an automated process with a function that extracts the most important features from the gait variables. One option is to use a data pre-processing method to generate new features for machine learning classifiers [10]. The most common data pre-processing technique in gait analysis is principal component analysis (PCA), which constructs principal components by transforming correlated gait variables into a set of unrelated linear variables through orthogonal transformation. However, one challenge for PCA is to reduce the negative effect of the non-linear correlation of gait variables such as dynamic gait outcomes. The optimal technique for extracting the most important features from the non-linear gait structure is using kernel methods in combination with PCA. The kernel PCA was employed in Chapter 2 to improve the SVM classification performance of populations of different ages, with an accuracy of 89% (Chapter 2, [11]). The kernel function maps the gait data from the original feature space to a high dimensional space where the gait data is linearly separable, and then performs PCA in this new high-dimensional feature space
(Chapter 2, [12]). Kernel PCA was also used in a previous study to process the spatial-temporal gait variables in advance, improving the SVM classification performance of healthy young and older gait patterns from 83% to 91% [13]. The complex structure of spatial-temporal gait variables was simplified through use of the Path Signature method in Chapter 5. In short, this method treats discrete variables as continuous paths in a two-dimensional space and uses an integral algorithm to calculate the new set of geometrical features from the path. Finally, the original variables are replaced by the newly generated independent variables in order to train the machine learning models, greatly improving their classification performance (Chapter 5, [14]).
Although the classification performance of the RF model showed a considerable improvement in
Chapter 5 using the Path Signature method to pre-process the gait data (accuracy of 98%), it remains
unclear whether or not these newly generated variables can be clinically relevant to gait and falls. The reason for this uncertainty is that it is not possible to link these newly generated variables to the original variables. Thus, the clinical information remains unexplained. For successful clinical application, the output of the machine learning model must be translated into meaningful clinical knowledge [15] [16]. Concerning only the use of machine learning classifiers to process the multivariate gait datasets,
Chapters 2, 3 and 4 used the algorithms such as PLS-DA, RF and ANN, which can automatically
pre-process gait data during the model establishment (Chapters 2 to 4, [17], [18], [19]). Embedding a data pre-processing function in machine learning algorithms is crucial when building an accurate classification model from multivariate gait data. The contribution of each input gait variable could assist clinicians in understanding the specific gait components in relation to clinical diagnosis. Another consideration is the influence of the number of samples and the number of variables on the choice of model. In the datasets of Chapters 2 to 5 the number of samples was much larger than the number of variables. However, if the proportion of the sample size with respect to the number of variables is too low, there would be a risk of overfitting the model [20]. Overfitting models always perform well on training datasets but perform poorly on testing datasets, in other words it doesn’t generalize well from training to new data. Therefore, it is imperative to evaluate the machine learning model’s performance based on unused data during the model building process. Regarding clinical practice, the data size is too small to generate adequate training and testing datasets. Functions such as K-fold cross-validation and leave-one-out cross-validation were used to increase the adequate size of training and testing datasets. The leave-one-out cross-validation was used in Chapters 2 to 4; each data was a testing dataset to verify the model’s performance. Cross-validation can also alleviate the negative influence of deviations in gait data and avoid the overfitting problem in the model (Chapters 2 to 4, [21]). Additionally, the results of Chapters 4 and 5 indicated that the accuracy of the classification results would be negatively affected by deviations in the datasets, especially when the data is not balanced in each group. It is suggested that a comprehensive evaluation of classification performance should include different matrices, including AUC, sensitivity, specificity and the model validation score (F1 score). Thus, other evaluation matrices were used in place of accuracy in Chapters 2 to 5.
Although all the machine learning algorithms used in this thesis can automatically adjust the hyperparameters from training datasets and strengthen the model’s reliability by cross-validation [22], [23], the advantages and disadvantages of each machine learning classifier must still be considered. For example, the support vectors and slack variables can determine hyperplanes for the SVM classifier, which was used to classify different population ages in Chapter 2. This classifier does not require a large number of participants for training [24]; however, for proper implementation it is necessary to adjust several hyperparameters such as kernel functions and Gamma, and it relies on data configuration in high dimensional space, limiting its interpretability (Chapter 2, [16], [25]). The RF classifier was used in Chapters 2, 3 and 5 because this method can also handle the small data size. All samples in RF were repeatedly classified in every decision tree. This offers better interpretability than SVM as it provides the output of contributions of each gait variables [17]. Naïve Bayes is based on priors and likelihood, which may be sensitive to skewed data (Chapter 5, [26]). K-Nearest Neighbour, a non-parametric algorithm, is an instance-based learner and is negatively affected by high-dimensional data (Chapter 5, [27]). Therefore, these two methods obtained a poor classification performance (accuracy of 68% and 62% respectively) in
Chapter 5 because of the multivariate gait data structure. Deep learning models require a large
amount of training data; thus, the limited size of the datasets in this thesis are not appropriate for deep learning [28].
In accordance with suggestions of model selection and assessment, machine learning approaches have been successfully applied in multiple fields related to the biomechanism of human mobility, such as medical diagnosis [29], pattern recognition [30], [31], image processing [32] and classification [33]. Machine learning techniques are also widely applied in diagnosing gait disorders [34], detecting falls caused by aging and designing interventions to reduce the risk of falls (Chapters 4 and 5, [35], [36]), and evaluating and constructing rehabilitation or treatment intervention programs [37]. A review summarising the recent studies for gait analysis using machine learning approaches showed that SVM was the best classifier (averaged accuracy of 87%) for small gait datasets [38]. Machine learning techniques in detecting disorders and designing rehabilitation schemes can also provide assistance for clinical diagnosis.
THE INTERPRETABILITY OF MACHINE LEARNING MODELS
In Chapter 5, the fallers were accurately distinguished from non-fallers, but the “black box” feature of the Path Signature method leads to uninterpretable clinical meanings of the newly generated variables. Nevertheless, when using machine learning methods to classify healthy and pathological gait, it is essential to know the contributions of gait variables to the accurate classification. Understanding the clinical meaning of the most contributing gait variables identified from a machine learning model could enhance the precision of clinical decision-making for gait deficits. Similar to all studies mentioned in this thesis, numerous studies have recently employed interpretable machine learning approaches to precisely detect gait impairment based
on data from wearable sensors, with accuracy around the range of 85% to 95% [38]. The 3D accelerometers calculated diverse gait variables including the dynamic gait outcomes used in
Chapters 2 and 3, and the spatial-temporal gait variables used in Chapters 4 and 5; however,
these interrelated gait variables from sensors are difficult for clinicians to directly evaluate human walking performance. Moreover, machine learning models such as SVM used in Chapter 2 can only provide the classification accuracy and validation scores. The detection of the contribution of each gait variable needed the combination of kernel PCA. In contrast, Chapter 2 also employed machine learning models with outstanding interpretation capabilities (RF and ANN), which are able to output the contribution of each input gait variable for clinical diagnosis of gait impairment. A further point of consideration is that gait relates not only to motor function, as it is also an activity that requires attention and executive functions. Gait analysis is one of the most important approaches to understanding human biomechanics, aiming to quantify the factors that control lower limb function [15]. On the one hand, the causes of gait deficits may include sensory or executive declines, orthopedic issues or injury [39]. On the other hand, Chapters 3 and 4 indicate that gait deficits were also associated with functional declines such as falls and cognitive impairment. Therefore, gait assessment can be used as one of the methods to effectively identify pathology underlying aging-related functional declines.
The results in this thesis reveal that the accurate machine learning classification model can classify different populations based on their gait performance, and determine the critical gait variables related to a specific classification purpose. In Chapter 2, three machine learning models successfully classified older and young adults based on differences in gait performance [11]. Likewise, geriatric patients suffering from cognitive impairment can be identified based on their impaired gait in combination with their performance of clinical assessments by using ANN classification model, as presented in Chapter 3. Moreover, neurological patients who experienced falls were revealed to have a slow and variable gait in Chapter 4 [40]. Other factors such as patient characteristics, frailty criteria, use of medicine and environmental effects will also cause aging-related functional declines [41], [42]. In Chapter 3, the ANN model also showed how geriatric patients with cognitive impairment suffered from the decay of mobility and high risk of mortality. The problem this thesis aims to solve is that manual analysis of the complex interrelations between these clinical factors and gait variables is an intricate and labour-intensive task for clinicians. Machine learning methods used in Chapter 3, such as ANN, can incorporate all these factors without any pre-processing procedures and correctly calculate their interrelations, thus providing an accurate classification performance with identifying the most clinically relevant factors (increased accuracy of 96%). Machine learning models can be used to monitor and screen for at-risk gait parameters, because the early detection of changes in gait parameters could help the risk for developing diseases [38]. An accurate early diagnosis of aging-related functional declines from gait impairment can prevent reduce the risk for developing mobility disability and reduce healthcare costs [43].
THE LIMITATIONS OF USING MACHINE LEARNING IN CLINICAL
APPLICATION
Machine learning algorithms were shown to achieve massive progress in clinical gait analysis in this thesis; however, the lack of transparency in machine learning models’ construction limits the repeatability and clinical interpretability of these advanced computing technologies. Explained in depth in Chapter 5, the “black box” feature of machine learning limits its clinical application. The most important reason for the demand for an interpretable model is that clinicians need to understand these specific gait variables to assist in the decision making process of diagnosis [44]. The machine learning models based on gait data proved very complex, making the construction of the classification model challenging to explain. Chapters 2 and 3 indicate that ANN can learn highly correlated non-linear relationships from a vast amount of gait data, and outperform humans in many related tasks. Still, their “black box” problem means they can only provide limited clinical information concerning the respective contributions of different variables (Chapters 2 and 3, [45]). In clinical gait analysis, machine learning models aim to improve gait rehabilitation according to accurate classification and prediction. Yet, for instance, the findings of Chapter 5 could not specify precisely how the gait variables that cause falls change before the event. Hence, it is impossible to use these invisible models to improve patients’ health at the current stage or prevent falls later. However, to assist clinical diagnosis and intervention, the machine learning models’ interpretability is rapidly developing. For example, the ANN algorithm would have the capacity to explain how neurons learn from gait data and how they communicate with each other to make the classification or prediction, thereby improving the model’s accuracy and interpretability. A final point to consider is that clinical data has often been incomplete, unbalanced and/or limited, whereas a valid machine learning model requires a lot of high-quality data to support its development. Therefore, an accurate, reliable and global model for clinical gait analysis among the heterogeneous population can only be attained through the continuation of research.
THE FUTURE OF MACHINE LEARNING IN GAIT ANALYSIS AND
RE-HABILITATION
The application of machine learning technologies has accelerated the development of the early diagnosis of aging-related disorders. Early identification and treatment are keys to reducing the suffering and financial cost of treatment for most diseases, and even potentially reversing the diagnosis. However, the classification performances of most studies in this thesis were not perfect, particularly in Chapters 2 and 4, and the “black box” problem was present in all studies. In order to diagnose geriatric diseases more accurately, in future research it is crucial that interpretable machine learning methods are used to determine the most prominent factors distinguishing standard from abnormal gait patterns in patients. In line with the proposal of the Keep Control project which is supporting the research of this thesis, the model’s input should contain
comprehensive factors related to body structures and functions, activities, participants, personal characteristics, and environmental effects. These factors are based on the five domains of the International Classification of Functioning, Disability and Health (ICF) model, which serves as a framework for describing and organising information on aging-related gait and balance deficits [46]. This model is also based on large population size and an algorithm that can automatically adjust parameters. Finally, the model is standardized so that it can be used for any patients with similar diseases [47]. Standardized gait analysis models can be applied to various health science fields, such as early prediction of diseases, evaluating the effectiveness of disease treatment, and controlling rehabilitation equipment to have the same natural gait as humans; machine learning would make a considerable contribution to achieving these goals. By analysing sensor-based gait data, machine learning methods provide portable, continuous monitoring and cost-effective solutions for gait analysis [48]. The development of machine learning models in gait analysis would require long-term data collection, patient status monitoring, feedback from treatment or rehabilitation, walking detection in indoor and outdoor environments and the inclusion of more comprehensive data to train and validate machine learning models. In the next step of the study, harmonization data collection across the entire Keep Control project network will be specifically responsible for the implementation of the domains described by the ICF model, and provide sufficient and diverse datasets for a comprehensive gait analysis and rehabilitation design. The machine learning models in development will assist clinicians in making an accurate early diagnosis and can help physiotherapists adapt their rehabilitation treatments to different situations [49]. The combination of machine learning and gait will be helpful for rehabilitation, exoskeleton design and clinical diagnosis [50], [51]. For example, in rehabilitation equipment such as the exoskeleton and walking aids, the system interacts with the dynamic environment and reinforcement learning can be used to develop various gait rehabilitation control strategies because it is better able to detect participants’ variability and automatically adjust according to each person’s specific needs [52]. Chapters 3 and 4 indicate that gait deficit was highly associated with falls or cognitive impairment. Gait rehabilitation can lead to gait recovery [53] and gait improvement [54], [55]. Thus, using machine learning to assist gait rehabilitation could aid patients whose function has declined due to neurological disease or injury, improve early diagnosis of cognitive impairment to reduce the risk for developing dementia and to experience a fall. The interaction between machine learning and movement biomechanics shows a great potential in gait analysis [56]. Their concerted use of data, algorithms, patterns and learning theories establishes visible models that can be applied to personal physical activity detection, disease detection, rehabilitation equipment control and more crucial functions. Simultaneously, the individual assistance of machine learning models can also save time by avoiding multiple patient visits to medical professionals and reducing the need for long-term treatment.
Overall, machine learning algorithms have more advanced computational capabilities than traditional statistical analysis to process high-dimensional and non-linear gait data structures,
and to identify the reliable gait changes from the vast amounts of data [57]. Another main focus is to overcome the nature of the “black box” of machine learning, make it applicable globally, build a bridge between visual gait analysis and interpretable machine learning models, and provide information to clinicians and patients to create personalized rehabilitation schemes.
CONCLUSION
In support of the hypothesis in this thesis, we found that machine learning approaches can accurately classify groups of individuals based on age and fall status using properties of gait. The success of classification was related to the idea that gait characteristics were used as inputs to the classification models, characteristics that differ between these groups. Considering geriatric patients with or without cognitive impairment present with many comorbidities, the combination of gait and clinical characteristics accurately classified these groups rather than inputting only gait or only clinical variables to the models. Gait data calculated from 3D-accelerometer signals includes dynamic gait outcomes and spatial-temporal gait variables, which were the input of machine learning classification models in Chapters 2 and 3 and Chapters 4 and 5, respectively. The critical variables extracted from gait that significantly contributes to the accuracy of models were sensitive indicators of aging (Chapter 2), cognitive declines (Chapter 3) and falls (Chapters 4, 5). These variables could eventually assist clinicians in diagnosing gait and cognitive impairments and screen patients for fall risk at an early stage. With the development of data science in clinical gait analysis, machine learning approaches make a considerable contribution to enhance rather than replace the efficiency of treatment for aging-related functional declines. An accurate gait classification model could enable early identification of the potential symptoms of geriatric disorders, helping to avoid the development of disorders such as dementia, stroke and Parkinson’s disease, and evaluate treatments. Future studies will incorporate multiple types of input data based on the ICF model rather than gait only, and will improve the reliability and interpretability of machine learning models so that it visibly and globally classifies or predicts the health status of individuals and is able to provide personalized treatment schemes for patients.
REFERENCE
1. C. Prakash, R. Kumar, and N. Mittal, “Recent developments in human gait research: pa-rameters, approaches, applications, machine learning techniques, datasets and challenges,”
Artif. Intell. Rev., vol. 49, no. 1, pp. 1–40, Jan.
2018.
2. A. Duhamel et al., “Statistical tools for clinical gait analysis,” Gait Posture, vol. 20, no. 2, pp. 204–212, 2004.
3. J. Peat and B. Barton, Medical statistics: A
guide to data analysis and critical appraisal.
John Wiley & Sons, 2008.
4. A. Muro-De-La-Herran, B. Garcia-Zapirain, and A. Mendez-Zorrilla, “Gait analysis meth-ods: An overview of wearable and non-wear-able systems, highlighting clinical applica-tions,” Sensors, vol. 14, no. 2, pp. 3362–3394, Feb. 2014.
5. E. Dolatabadi, B. Taati, and A. Mihailidis, “An automated classification of pathological gait using unobtrusive sensing technology,” IEEE
Trans. Neural Syst. Rehabil. Eng., vol. 25, no.
12, pp. 2336–2346, 2017.
6. M. Welling, “Are ML and statistics comple-mentary?,” in IMS-ISBA Meeting on ‘Data
Sci-ence in the Next 50 Years, 2015.
7. X.-H. Zhou, D. K. McClish, and N. A. Obu-chowski, Statistical methods in diagnostic
medicine, vol. 569. John Wiley & Sons, 2009.
8. D. T. H. Lai, R. K. Begg, and M. Palaniswami, “Computational intelligence in gait research: a perspective on current applications and future challenges,” IEEE Trans. Inf. Technol. Biomed., vol. 13, no. 5, pp. 687–702, 2009.
9. S. Raschka, “Model evaluation, model se-lection, and algorithm selection in machine learning,” arXiv Prepr. arXiv1811.12808, vol. abs/1811.1, pp. 20–33, Nov. 2018.
10. J. Figueiredo, C. P. Santos, and J. C. Moreno, “Automatic recognition of gait patterns in human motor disorders using machine learn-ing: A review,” Med. Eng. Phys., vol. 53, pp. 1–12, Mar. 2018.
11. Y. Zhou et al., “The detection of age groups by dynamic gait outcomes using machine
learn-ing approaches,” Sci. Rep., vol. 10, no. 1, pp. 1–12, Dec. 2020.
12. M. Ekinci and M. Aykut, “Human gait rec-ognition based on kernel PCA using projec-tions,” J. Comput. Sci. Technol., vol. 22, no. 6, pp. 867–876, Nov. 2007.
13. J. Wu, J. Wang, and L. Liu, “Feature extraction via KPCA for classification of gait patterns,”
Hum. Mov. Sci., vol. 26, no. 3, pp. 393–411,
Jun. 2007.
14. I. Chevyrev and A. Kormilitzin, “A primer on the signature method in machine learning,”
arXiv Prepr. arXiv1603.03788, Mar. 2016.
15. E. Halilaj, A. Rajagopal, M. Fiterau, J. L. Hicks, T. J. Hastie, and S. L. Delp, “Machine learning in human movement biomechan-ics: Best practices, common pitfalls, and new opportunities,” J. Biomech., vol. 81, pp. 1–11, Nov. 2018.
16. I. D. Dinov, “Black Box Machine-Learning Methods: Neural Networks and Support Vector Machines,” in Data Science and
Predic-tive Analytics, Cham: Springer International
Publishing, 2018, pp. 383–422.
17. G. Biau and E. Scornet, “A random forest guided tour,” Test, vol. 25, no. 2, pp. 197–227, 2016.
18. B. Yegnanarayana, Artificial neural networks. PHI Learning Pvt. Ltd., 2009.
19. J. Cai, J. Luo, S. Wang, and S. Yang, “Fea-ture selection in machine learning: A new perspective,” Neurocomputing, vol. 300, pp. 70–79, 2018.
20. G. C. Cawley and N. L. C. Talbot, “On over-fit-ting in model selection and subsequent selec-tion bias in performance evaluaselec-tion,” J. Mach.
Learn. Res., vol. 11, pp. 2079–2107, 2010.
21. T.-T. Wong, “Performance evaluation of clas-sification algorithms by k-fold and leave-one-out cross validation,” Pattern Recognit., vol. 48, no. 9, pp. 2839–2846, 2015.
22. P. Probst, A.-L. Boulesteix, and B. Bischl, “Tunability: Importance of Hyperparameters of Machine Learning Algorithms.,” J. Mach.
Learn. Res., vol. 20, no. 53, pp. 1–32, 2019.
23. M. Feurer and F. Hutter, “Hyperparameter op-timization,” in Automated Machine Learning, Springer, Cham, 2019, pp. 3–33.
24. P. P. Matykiewicz and J. Pestian, “Effect of small sample size on text categorization with support vector machines,” in BioNLP:
Pro-ceedings of the 2012 Workshop on Biomedi-cal Natural Language Processing, 2012, pp.
193–201.
25. M. Claesen and B. De Moor, “Hyperparameter Search in Machine Learning,” in The XI
Meta-heuristics International Conference, 2015, vol.
1502.02127, p. 5.
26. I. Rish, “An empirical study of the naive Bayes classifier,” in IJCAI 2001 workshop on
empiri-cal methods in artificial intelligence, 2001, vol.
3, no. 22, pp. 41–46.
27. P. Cunningham and S. J. Delany, “k-Near-est Neighbour Classifiers--,” arXiv Prepr.
arXiv2004.04523, 2020.
28. A. Shrestha and A. Mahmood, “Review of deep learning algorithms and architectures,”
IEEE Access, vol. 7, pp. 53040–53065, 2019.
29. J. D. Farah, N. Baddour, and E. D. Lemaire, “Design, development, and evaluation of a local sensor-based gait phase recognition system using a logistic model decision tree for orthosis-control,” J. Neuroeng. Rehabil., vol. 16, no. 1, p. 22, Feb. 2019.
30. A. de M. e Souza and M. R. Stemmer, “Ex-traction and classification of human body pa-rameters for gait analysis,” J. Control. Autom.
Electr. Syst., vol. 29, no. 5, pp. 586–604, 2018.
31. H. Shim and S. Lee, “Multi-channel electro-myography pattern classification using deep belief networks for enhanced user experi-ence,” J. Cent. South Univ., vol. 22, no. 5, pp. 1801–1808, 2015.
32. D. Leightley, J. S. McPhee, and M. H. Yap, “Automated analysis and quantification of human mobility using a depth sensor,” IEEE
J. Biomed. Heal. informatics, vol. 21, no. 4, pp.
939–948, 2016.
33. L. Van Gestel et al., “Probabilistic gait clas-sification in children with cerebral palsy: A Bayesian approach,” Res. Dev. Disabil., vol. 32, no. 6, pp. 2542–2552, 2011.
34. M. Alaqtash, T. Sarkodie-Gyan, H. Yu, O. Fuentes, R. Brower, and A. Abdelgawad, “Au-tomatic classification of pathological gait pat-terns using ground reaction forces and ma-chine learning algorithms,” in 2011 Annual
International Conference of the IEEE Engi-neering in Medicine and Biology Society, 2011,
pp. 453–457.
35. B. M. Meyer et al., “Wearables and deep learn-ing classify fall risk from gait in multiple scle-rosis,” IEEE J. Biomed. Heal. informatics, 2020. 36. B. T. Nukala et al., “An efficient and robust fall detection system using wireless gait analysis sensor with artificial neural network (ANN) and support vector machine (SVM) algo-rithms,” Open J. Appl. Biosens., vol. 3, no. 04, p. 29, 2015.
37. D.-X. Liu, W. Du, X. Wu, C. Wang, and Y. Qiao, “Deep rehabilitation gait learning for modeling knee joints of lower-limb exoskel-eton,” in 2016 IEEE International Conference
on Robotics and Biomimetics (ROBIO), 2016,
pp. 1058–1063.
38. P. Khera and N. Kumar, “Role of machine learning in gait analysis: a review,” J. Med.
Eng. Technol., pp. 1–27, Oct. 2020.
39. M. Amboni, P. Barone, and J. M. Hausdorff, “Cognitive contributions to gait and falls: Evi-dence and implications,” Movement Disorders, vol. 28, no. 11. Mov Disord, pp. 1520–1533, 15-Sep-2013.
40. Y. Zhou et al., “Classification of Neurologi-cal Patients to Identify Fallers Based on Spa-tial-Temporal Gait Characteristics Measured by a Wearable Device,” Sensors 2020, Vol. 20,
Page 4098, vol. 20, no. 15, p. 4098, Jul. 2020.
41. J. M. Porto et al., “Risk factors for future falls among community-dwelling older adults without a fall in the previous year: A pro-spective one-year longitudinal study,” Arch.
Gerontol. Geriatr., vol. 91, p. 104161, 2020.
42. M. H. Burhanullah et al., “Neuropsychiatric symptoms as risk factors for cognitive decline in clinically normal older adults: the cache county study,” Am. J. Geriatr. Psychiatry, vol. 28, no. 1, pp. 64–71, 2020.
43. G. Guo, K. Guffey, W. Chen, and P. Pergami, “Classification of normal and pathological
gait in young children based on foot pressure data,” Neuroinformatics, vol. 15, no. 1, pp. 13–24, 2017.
44. A. N. Aicha, G. Englebienne, K. S. van Schoo-ten, M. Pijnappels, and B. Kröse, “Deep learn-ing to predict falls in older adults based on daily-life trunk accelerometry,” Sensors
(Swit-zerland), vol. 18, no. 5, May 2018.
45. D. S. Watson et al., “Clinical applications of machine learning algorithms: Beyond the black box,” BMJ, vol. 364, Mar. 2019. 46. R. D. Rush, “Evaluating the World Health
Organization (WHO)’s International Classi-fication of Functioning, Disability and Health (ICF) as a Biopsychosocial Epilepsy Self-Man-agement Model.” The University of Iowa, 2020. 47. L. Lonini, A. Gupta, K. Kording, and A. Ja-yaraman, “Activity recognition in patients with lower limb impairments: do we need training data from each patient?,” in 2016 38th
Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016, pp. 3265–3268.
48. M. M. Rodgers, G. Alon, V. M. Pai, and R. S. Conroy, “Wearable technologies for active living and rehabilitation: current re-search challenges and future opportunities,”
J. Rehabil. Assist. Technol. Eng., vol. 6, p.
2055668319839607, 2019.
49. R. A. Cooper and R. Cooper, “Rehabilitation engineering: a perspective on the past 40-years and thoughts for the future,” Med. Eng.
Phys., vol. 72, pp. 3–12, 2019.
50. S. K. Goh et al., “Spatio–spectral represen-tation learning for electroencephalograph-ic gait-pattern classifelectroencephalograph-ication,” IEEE Trans.
Neural Syst. Rehabil. Eng., vol. 26, no. 9, pp.
1858–1867, 2018.
51. K. Tanghe, F. De Groote, D. Lefeber, J. De Schutter, and E. Aertbeliën, “Gait trajectory and event prediction from state estimation for exo-skeletons during gait,” IEEE Trans. Neural Syst.
Rehabil. Eng., vol. 28, no. 1, pp. 211–220, 2019.
52. G. Bingjing, H. Jianhai, L. Xiangpan, and Y. Lin, “Human–robot interactive control based on reinforcement learning for gait rehabilita-tion training robot,” Int. J. Adv. Robot. Syst., vol. 16, no. 2, p. 1729881419839584, 2019.
53. P. Levinger, D. T. H. Lai, R. K. Begg, K. E. Webster, and J. A. Feller, “The application of support vector machines for detecting recov-ery from knee replacement surgrecov-ery using spa-tio-temporal gait parameters,” Gait Posture, vol. 29, no. 1, pp. 91–96, 2009.
54. M. Alaqtash, H. Yu, R. Brower, A. Abdelga-wad, and T. Sarkodie-Gyan, “Application of wearable sensors for human gait analysis using fuzzy computational algorithm,” Eng.
Appl. Artif. Intell., vol. 24, no. 6, pp. 1018–
1025, 2011.
55. J. P. Ferreira, A. Vieira, P. Ferreira, M. Crisostomo, and A. P. Coimbra, “Human knee joint walking pattern generation using com-putational intelligence techniques,” Neural
Comput. Appl., vol. 30, no. 6, pp. 1701–1713,
2018.
56. J. Goecks, V. Jalili, L. M. Heiser, and J. W. Gray, “How machine learning will transform biomedicine,” Cell, vol. 181, no. 1, pp. 92–101, 2020.
57. M. Devanne, H. Wannous, M. Daoudi, S. Berretti, A. Del Bimbo, and P. Pala, “Learn-ing shape variations of motion trajectories for gait analysis,” in 2016 23rd International
Con-ference on Pattern Recognition (ICPR), 2016,
pp. 895–900.
