Exploring the
Associations Between
Clinical Rehabilitation
Treatment Modules and
Changes in
Rehabilitation Outcome
De Hoogstraat Revalidatie
Boudewijn de Gooijer
MSc Medical Informatics
July 2019
1 Student Boudewijn de Gooijer Student number: 10021892 Email: boudewijndegooijer@gmail.com Mentors Ronald Beuker, MSc
Business Intelligence Manager De Hoogstraat Revalidatie Email: r.beuker@dehoogstraat.nl
Prof. dr. Marcel Post Senior Researcher
De Hoogstraat Revalidatie Email: m.post@dehoogstraat.nl
Tutor
Dr. Anita CJ Ravelli
Academic Medical Center ‐ University of Amsterdam Department of Medical Informatics
E‐mail: a.c.ravelli@amsterdamumc.nl
Location of scientific research project De Hoogstraat Revalidatie
Rembrandtkade 10 3583 TM, Utrecht
Period
2
Contents
Abstract...4
Samenvatting ...5
Chapter 1: Introduction ...6
1.1 Implications for practice and scientific contribution ...7
1.2 Research questions (RQ) ...7
Chapter 2: Method ...8
2.1 Selection criteria ...8
2.2a Rehabilitation outcome: Change in USER‐dimension score ...8
2.2b Independent variables ...9
2.3 RQ1: Identifying patient characteristics that are determinants of rehabilitation outcome ... 10
2.4 RQ2: Creating models, optimizing parameters and selecting best models ... 11
2.5 RQ3: Interpreting ‘best’ model per USER‐dimension ... 14
2.5 RQ4: Splitting data into improved and worsened patients, finding patient profiles and interpreting split models ... 15
3. Results... 17
3.1 RQ1: Identifying determinants of rehabilitation outcome ... 17
3.2 RQ2: Created models, parameter optimization and model selection ... 21
3.3 RQ3: Interpreting ‘best’ models ... 24
3.4 RQ4: Interpreting split models ... 28
Discussion ... 38
Main Findings ... 38
Strengths and limitations ... 42
Implications for practice ... 43
Future work ... 43
Conclusion ... 44
References ... 45
Appendices ... 48
Appendix A: Module definitions ... 48
Appendix B: ‘Old’ Cluster contents ... 49
Appendix C: Literature search terms and hits ... 50
Appendix D: Distributions and boxplots of USER‐dimensions ... 50
Appendix E: Dendrogram of hierarchical module clusters ... 53
Appendix F: Table with cluster contents for cardiovascular diagnosis group ... 54
Appendix G: Model diagnostics ... 56
3
Appendix I: Checking linearity assumptions for mood ... 62
Appendix J: Checking linearity assumptions for pain ... 62
Appendix K: Scatterplots of transformed clusters and diagnostic plot for transformed pain model ... 64
Appendix L: Fit statistic for % of max potential improvement per USER‐dimension ... 65
Appendix M: Best model summaries ... 66
Appendix N: Wilcoxon rank sum test results: ... 69
Appendix O: Summaries for models of improved and worsened population ... 73
4
Abstract
Introduction: De Hoogstraat Revalidatie (DHR), a medically specialized rehabilitation clinic, wishes to find associations between their time per treatment module and changes in outcome, as measured by USER
(‘Utrechtse schaal voor evaluatie van revalidatie’) scores. In this thesis these associations are explored through linear regression, and it is investigated if these associations vary between improved and worsened patients. Additionally, average patient values are fed into the models to calculate what proportion of predicted change in USER scores can be attributed to time per module. Lastly, profiles of the improved patients and the worsened patients are given.
Method: A literature search was performed to find baseline state and patient characteristics that are determinants of rehabilitation outcome. The dataset consisting of rehabilitation outcome determinants and time per treatment module was built by linking administrative and EHR (electronic health record) data from DHR, from which the largest diagnosis group was selected. Patients were included if treatment started after 1‐ 7‐2016 and concluded before 9‐03‐2019.
For each USER‐dimension (mobility, personal care, cognition, pain, fatigue, mood), 5 backwards stepwise selected (on AIC) linear regression models were made with baseline patient characteristics and time per treatment module as independent variables and change in USER‐dimension as dependent variable. To reduce dimensionality, modules were clustered; (1) top‐down, (2) hierarchically on correlation into 10, (3) 15 and (4) 20 clusters and (5) modules kept separately. For each USER‐dimension, the ‘best model’ with the highest predicted R2 was selected, from which associations between time per module (cluster) and outcome
were extracted.
The data was split into improved and worsened patients per USER‐dimension. Coefficients from ‘best models’ were recalculated on each split of data and compared to associations from models of the total population. For the total population as well as the split data, average values were fed into the models to
calculate what proportion of predicted change in USER‐dimension can be attributed to patient characteristics or time per module.
A Wilcoxon rank‐sum test was performed on each split of data to determine on which baseline patient characteristics the improved and worsened patients significantly differ. These were averaged into ‘patient profiles.’
Results: A dataset of 830 patients was generated, from which the largest diagnosis group (cerebrovascular accident, n = 338) was selected. Several determinants were identified from literature, of which some were available for analysis. The best performing cluster method was the top‐down method. For all USER‐dimensions, various significant associations were found between treatment module clusters and change in USER‐dimension score. However, the predicted change in USER‐dimension for average patients is generally influenced more by baseline patient characteristics than by time per module cluster. Profiles of the improved patients have typically lower or ‘worse’ states than profiles for the worsened patient.
Discussion: Many determinants were not readily available at DHR; to improve research opportunities, this could be addressed. Top‐down clustering of treatment is also applied in different research; researchers often
categorize rehabilitation treatment into 15 to 26 categories. Splitting the data into improved and worsened patients per USER‐dimension was detrimental to model accuracy and did not offer new insight other than providing patient profiles of improved and worsened patients.
5
Samenvatting
Inleiding: De Hoogstraat Revalidatie (DHR), een medisch specialistische aanbieder van revalidatiezorg, is geïnteresseerd in verbanden tussen hun behandel modules en uitkomsten van geleverde revalidatiezorg. In dit onderzoek worden deze associaties verkend en wordt onderzocht of deze variëren over verschillende patiënt karakteristieken. Gemiddelde waardes voor de gehele populatie, de verbeterde patiënten en de verslechterde patiënten worden in de meest nauwkeurige modellen gestopt om te berekenen welk aandeel van de geschatte verandering in uitkomst aan modules toe te kennen is.
Methode: Met een literatuuronderzoek zijn begintoestanden en patiëntkarakteristieken gezocht die
determinanten van revalidatie uitkomsten waren. Door data uit de administratieve database te koppelen aan data uit het EPD (elektronisch patiëntendossier) is een dataset van patiënten opgebouwd. Patiënten waarvan de behandeling tussen 1‐7‐2016 en 9‐03‐2019 gestart en beëindigd is zijn meegenomen in dit onderzoek.
Voor elke USER‐dimensie (mobiliteit, persoonlijke verzorging, cognitie, pijn, moeheid, stemming) zijn vijf modellen gemaakt, elk met een andere methode om de behandelmodules te clusteren; (1) top‐down op medische inhoud, (2) hiërarchisch in 10 clusters, (3) hiërarchisch in 15 clusters, (4) hiërarchisch in 20 clusters, (5) modules apart. Voor elke dimensie is het beste model geselecteerd met de hoogste ‘predicted R2’.
Per USER‐dimensie is de dataset gesplist in verbeterede patiënten en verslechterde patiënten.
Coëfficiënten van de beste modellen werden herrekend op de gesplitste data. Associaties van modellen van de gehele populatie zijn vergeleken met de associaties van modellen van de gesplitste populatie. Verder zijn gemiddeldes voor de gehele populatie, de verbeterde en verslechterde patiënt berekend. Deze zijn in de beste modellen gestopt om te berekenen welk deel van de voorspelde verandering aan het effect van module clusters is toe te kennen.
Met een Wilcoxon rank‐sum test is gekeken op welke determinanten de verbeterde en verslechterde patiënten significant verschillen.
Resultaten: Een dataset van 830 patiënten was gegenereerd, waarvan ‘cerebrovasculair aandoening’ (CVA) de meest geschikte diagnosegroep was. De beste cluster methode was de top‐down cluster methode. Significante associaties zijn gevonden tussen behandelmodule clusters en verandering in USER‐dimensies; echter is de voorspelde verandering in USER‐dimensies in vrijwel alle gevallen meer beïnvloed door effecten toegekend aan baseline toestanden en patiëntkarakteristieken dan door tijd per module clusters. Profielen van de verbeterde patiënten hebben lagere scores voor de significant verschillende determinanten.
Discussie: Veel van de determinanten gevonden in literatuur zijn niet beschikbaar bij DHR; mocht DHR het onderzoek willen stimuleren, en onderzoeksmogelijkheden creëren, is het aan te raden hier intern afspraken over te maken. De best presterende cluster methode wordt ook toegepast in andere vormen in ander onderzoek; anderen pleiten voor een classificatiesysteem van revalidatie zorg, onder andere ook voor het analyseren van revalidatie uitkomst. Verschillend ander onderzoek deelt revalidatiezorg in 15 tot 26
categorieën, vergelijkbaar met wat dit onderzoek deed. Het splitsen van data en herbouwen van de modellen op de verbeterde of verslechterde patiënten verminderde uiteindelijk alleen de voorspellende waarde van de modellen; als gevolg waren associaties uit gesplitste modellen minder betrouwbaar dan van de modellen gebouwd op de complete data.
6
This thesis is part of and result of the Scientific Research Project, a mandatory part of the master’s program Medical Informatics of the University of Amsterdam. During this research project, the student must carry out his research, write a thesis and defend their results. This research took place at De Hoogstraat Revalidatie (DHR), Utrecht.
Chapter 1: Introduction
In the current landscape of medical rehabilitation in the Netherlands, reimbursement is done through the system of ‘diagnose behandel combinaties’ (DBC), whereby the average price to treat a diagnosis and the volume of treatment is used to determine how much is reimbursed to caregivers. DBCs offer little detail into what specific treatment a patient has received, or how much. Administration concerning the DBC contains only the reimbursed price and the DBC product, offering only a general impression of time spent on delivered care but little detail into what care activities have taken place and for what purpose. As a result, the current DBC structure has been criticized within medically specialized rehabilitation care for not containing medically detailed information [1].
In an effort to move to a reimbursement system with more medical details and higher accuracy in describing treatment costs, a committee was formed with members from Revalidatie Nederland (RN), Netherlands Society of Rehabilitation Medicine (NSRM), Nederlandse Zorgautoriteit, Zorgverzekeraars Nederland and Zorginstituut Nederland [1]. Together, they formulated and implemented the ‘treatment modules and reimbursement plan’ (henceforth referenced as the ‘module‐plan’) [1]. The module plan involved categorizing rehabilitation care into modules that described the goal of care, such as ‘arm/hand functioning’ or ‘memory deficit,’ and registering provided care in the defined modules. Module definitions are iteratively evaluated and redetermined by medical specialists in cooperation with the module plan committee [2]. There are 201 modules [3].
This higher level of medical detail is lacking in the current DBC‐structure [1]. Modules also provide a uniform way of describing (and billing) medically specialized rehabilitation care between participating clinics. With these two main potential advantages, the module‐plan committee hoped to provide a basis for developing an improved structure of reimbursement, consisting of combinations of modules into products in rehabilitation care [2].
Since the project first started in 2014, it is the longest running national effort yet to standardize rehabilitation care reimbursement. Since then, the module‐plan has been controversial within the NSRM, with caregivers citing a high administrative burden and no visible benefit to increased administration. Despite that, an internal vote was cast within the NSRM to continue the module‐plan, extending the module‐plan’s run to November 2019 [4] to explore different purposes for the modular registration. In accordance with the module‐ plan, DHR measures their delivered care in modules.
DHR has a longer‐standing wish to find associations between the treatment they deliver and the rehabilitation outcomes they measure of their patients. This desire is not uncommon in rehabilitation care; in commentary by De Jong (2004), rehabilitation is described as a ‘black box’ [5]. They claim interventions are difficult to characterize due to their multidisciplinary nature, and “lack standardization in definition and measurement.” In response, they coordinated the creation of a beginning of a taxonomy to, among other things, research rehabilitation treatment outcomes. This is arguably similar to one of the goals of the module plan; to create a naming system to uniformly describe rehabilitation care in order to investigate the relationship between provided care and its outcome.
Therefore, the focus of this research will be to investigate if DHR’s care, as described by modules, is associated with changes in rehabilitation outcome. Changes in mobility, personal care, cognition, pain, fatigue and mood, will be modelled through several linear regression models to explore the associations between their delivered care and the outcomes of the patients they treat.
There is a possibility that these associations vary, depending on certain patient characteristics, or determinants, that are measurable at admission. Existing literature can be sought for these characteristics. These characteristics can be summarized for populations, or samples thereof, in the form of a ‘patient profile’.
7
1.1 Implications for practice and scientific contribution
As the ‘module‐plan’ is possibly reaching its expiration date [4], there is a need for clarity regarding the role of modules in the future. Finding associations between modules and outcomes and finding if these associations vary based on baseline patient characteristics, if any, could provide more argumentation in the evaluation of the module‐plan.
Furthermore, there is little research on associations between routine clinical rehabilitation treatment and rehabilitation outcome. As De Jong (2005) stated, “to date, research on current practice has been able to tell us little more than that ‘rehab is good [6].’” Also, no publicly available research on the viability of the treatment modules, aside from similar research conducted at DHR conducted by Buiter [3].
Buiter’s research focused on finding the association between change in a combined score of mobility and personal care, and time per treatment module. As there are 201 different modules (of which 132 were used in his research), Buiter clustered treatment modules together based on medical purpose (updated version visible in appendix B) to reduce dimensionality and increase the power of his analysis.
Rehabilitation outcome is more complex than a combined score of mobility and personal care, however. At DHR, rehabilitation is measured in 6 dimensions, (1) mobility, (2) personal care, (3) cognition, (4) pain, (5) fatigue, (6) mood. By aggregating two scores, there is little specificity in Buiter’s analysis; it is difficult to say what is associated with mobility or personal care. Also, he discards a large amount of outcome data of different USER‐dimensions. As a result, only very general findings and associations could be given. Buiter therefore recommends associations between rehabilitation treatment and outcome more extensively by analyzing various rehabilitation outcomes without aggregating scores, and to experiment with different cluster methods for treatment modules to obtain more accurate models.
These recommendations from Buiter will be explored. As a result, this research will provide new insight in DHR’s given treatment and outcomes by analyzing all rehabilitation outcomes separately, providing more specific associations between time registered per treatment module and rehabilitation outcome.
1.2 Research questions (RQ)
RQ1. From literature, which baseline states and patient characteristics can we identify as determinants of rehabilitation outcome that are available at DHR?
RQ2. Which of the clustering methods gives the most accurate models in describing the associations between rehabilitation care and rehabilitation outcome?
RQ3. After model creation,
i. What are the associations between the module clusters with changes in USER‐dimension score?
ii. What proportion of predicted change per USER‐dimension can be attributed to modules? RQ4. After the data is split into improved and worsened changed patients,
i. Which patient profiles can be made for the improved and the worsened patients for each USER‐ dimension?
ii. Do the associations between module clusters and change in USER‐dimension vary between the improved and worsened patients with respect to each USER‐dimension?
iii. Does the proportion of predicted change per USER‐dimension attributed to modules vary between improved and worsened patients?
8
Chapter 2: Method
2.1 Selection criteria
As treatment modules were subject to change in definition up until 1‐7‐2016, only patients with treatments starting after 1‐7‐2016 and ended before 9‐3‐2019 (moment of data extraction) will be included for this analysis. Patients will be included if they have admission and discharge USER‐measurements. As USER‐ measurements are only made for clinical patients, only clinical patients are considered within the scope of this analysis. Patients under the age of 18 will be included. Readmissions will be excluded. Non‐consenting patients will be excluded.
2.2a Rehabilitation outcome: Change in USER-dimension score
The difference between admission and discharge USER‐dimension score (Utrechtse Schaal voor Evaluatie van Revalidatie) will be used as dependent variable. Developed at DHR in 2006, the USER is a questionnaire to be filled in by a caregiver at admission and discharge to measure level of functioning or well‐ being related to clinical rehabilitation [7]. The USER has six dimensions; cognition, mobility, personal care, pain, fatigue and mood. These six dimensions are either questionnaire‐item scores or the sum of multiple
questionnaire‐item scores. An overview of the questionnaire elements per USER‐dimension and the ranges of each USER‐dimension is given in table 1. For mobility, personal care and cognition, a higher score corresponds to higher value. For pain, fatigue and mood, lower scores correspond to higher value.
USER Dimension + elements for each dimension
Range 1. Mobility 0 – 35 Sitting 0 – 5 Standing 0 – 5 Transferring 0 – 5 Walking indoors 0 – 5 Walking distances 0 – 5 Stair climbing 0 – 5 Wheelchair mobility 0 – 5 2. Personal Care 0 – 35
Eating and drinking 0 – 5
Grooming 0 – 5 Bathing 0 – 5 Dressing 0 – 5 Toilet usage 0 – 5 Incontinence bowels 0 – 5 Incontinence bladder 0 – 5 3. Cognition 0 – 50 Self‐expression 0 – 5 Comprehension 0 – 5 Visual perception 0 – 5 Orientation time/place 0 – 5 Attention 0 – 5 Memory 0 – 5 Task execution 0 – 5
9 Initiative 0 – 5 Behavioral control 0 – 5 Social behavior 0 – 5 4. Pain 0 – 100 * 5. Fatigue 0 – 100 * 6. Mood (0 – 400)/4 * Depressed mood 0 – 100 Sadness 0 – 100 Anxiety 0 – 100 Anger 0 – 100 7. Aggregate 0 - 70 (average of change of all other dimensions)
Table 1: Dimensions of the USER(“Utrechtse Schaal voor Evaluatie van Revalidatie”)-questionnaire, and elements per USER-dimension [7]. Higher scores correspond to more healthy states, except for items marked with a *.
The USER aggregate is the average of all other changes in USER dimensions. To calculate the aggregate, changes in pain, fatigue and mood were divided by negative 1. Aggregate USER is not an element of the USER questionnaire, but instead serves to summarize changes in all dimensions in a single score.
2.2b Independent variables
There are two groups of independent variables; variables related to time per module, and variables related to patients; either a baseline state or patient characteristic.
Time per module, generic time and unregistered time
At clinical admission at DHR, patients are diagnosed, their treatment goals are established, baseline measurements (including baseline USER measurements) are made and patient characteristics are stored. A physician can then indicate which, and how many, modules are necessary to reach the patient’s goals [1]. At DHR, indicating modules is done per individual and tailored to the specific goals agreed upon by physician and patient.
During clinical treatment, a module code is registered for each care activity provided to the patient. The care provider registering the module code for the provided care activity can only register time on module codes that were previously indicated by the physician. For example, an hour of care is given by a physiotherapist. During routine administration, this hour is additionally marked with a module code, depending on which medical goal the physiotherapist was trying to help reach. This could be the code for ‘wheelchair riding’ or ‘arm/hand function disturbed’, for instance. The physiotherapist can only choose from modules that were indicated by the patient’s physician.
Time per module will be used as independent variable for regression analysis. In addition, the total number of modules indicated and the percentage of indicated modules on which time is registered will also be used as independent variables. Lastly, the percentage of total treatment time registered on the ‘generic’ module, and the percentage of total treatment time not registered to a module (from section 2.1) per clinical admission will be used included in the regression models.
Rehabilitation outcome determinants: Baseline states and patient characteristics
Values for baseline states and patient characteristics, identified as determinants of rehabilitation outcome, will be included as independent variables in regression analysis. RQ1 will serve to identify which baseline states and patient characteristics are determinants of rehabilitation outcome, and which of those are available at DHR.
10
2.3 RQ1: Identifying patient characteristics that are determinants of rehabilitation outcome
Figure 1 contains an overview of the methods to answering RQ1. These methods serve two main goals; (1) to answer RQ1 by identifying baseline patient characteristics that are determinants of rehabilitation
outcome in literature and checking their availability in DHR’s databases, and (2) to provide a dataset for further processing and analysis to consequently answer the rest of the research questions.
Figure 1: Methods to answer RQ1. A literature search is conducted to find baseline states and patient characteristics that are determinants of rehabilitation outcome. Values for available determinants are extracted from DHR's databases for each clinical admission within selection criteria.
i. Literature search and unstructured interviews
A literature search will be conducted to find baseline states and patient characteristics that are determinants of rehabilitation outcome. Suggested articles will be sorted by ‘best match’, and the titles and abstracts of the 100 best matching articles will be scanned for relevancy to this research. Articles with relevant titles and abstracts will be completely read, and rehabilitation determinants will be extracted where applicable.
11
(Ecaris), in addition to physicians from the analyzed diagnosis group. Professionals will be asked what factors they believed to influence associations between treatment modules and treatment outcome at DHR. These expert interviews will also be also focused on ascertaining which determinants were available in DHR’s databases, and in what form.
The determinants identified from literature and interviews will then be checked for availability in both Ecaris and the EHR. Incomplete determinants are omitted from the dataset unless there is a meaningful reason for missing values; in which case, missing values are imputed.
Available determinants will be used in regression analysis to answer the other RQs of this thesis. ii. Dataset
Data from both databases are linked together in Excel on unique patient ID. This way, a dataset is generated where each record is a concluded clinical admission within the time frame specified, containing the admission and discharge USER‐dimension measurements and amount of time treated per treatment module, in addition to values for the available identified determinants.
iii. Diagnosis group selection
The dataset resulting from data extraction will contain data on all their patients of all diagnosis groups. The population will be divided by diagnosis group into smaller data sets, to increase the homogeneity of each data set. The diagnosis group with the highest ‘analysis quality’ will be selected for analysis. Metrics for
‘analysis quality’ are (1) population size, (2) the percentage of total treatment time for which no module code is registered, (3) the percentage of total treatment time which is registered to the ‘generic’ module, (4) the sum of the two percentages. By evaluating diagnosis groups with these metrics, a diagnosis group can be selected with the highest specificity and sample size.
2.4 RQ2: Creating models, optimizing parameters and selecting best models
An overview of the methods to answer RQ2 is visible in figure 2. Each part in the figure labelled with a roman numeral is described in more detail below.
i. Data inspection and cleaning
After data are correctly linked, the data set will be inspected and generally described before analysis. In order to obtain a homogenous diagnosis within the study population, the eventually selected diagnosis group will be checked for any heterogeneously diagnosed patients. Records in the dataset with differing diagnoses will be removed from the dataset.
ii. Model creation
Models will be generated per USER‐dimension (cognition, mobility, personal care, pain, fatigue, mood, aggregate), using the change in score for each USER‐dimension as the dependent variable. Time per module and values for baseline states and patient characteristics will be used as independent variables.
In order to reduce dimensionality of the data, time per module will be clustered. Ideally, the data set has at least 20 observations for each independent variable in the data set [8] [9]. As this is a ‘rule‐of‐thumb’, the number of clusters is varied to see which number of clusters gives the most accurate models. Values for module clusters are the sum of time per individual module within the cluster.
Treatment modules will be clustered in 5 different ways using three methods; (1) modules clustered top‐down by their medical purpose, visible in appendix B, (2) modules clustered hierarchically into 10, 15 and 20 clusters, (3) separate modules. For the study population that will be eventually selected, this means there will be 5 candidate models per USER‐dimension, and 35 models in total.
12
Figure 2: Creating, evaluating and selecting models. The data set resulting from section 2.3 is (i) inspected and cleaned and (ii) models are created using this dataset. (iii) Models are optimized by selecting parameters based on AIC, and (iv) regression assumptions are
checked for each model before (v) selecting the ‘best’ model per USER-dimension.
To hierarchically cluster the modules, a correlation matrix will be created for the modules on which time is registered. Each correlation will be subtracted from 1, resulting in a dissimilarity (or distance) matrix. Finally, the modules are to be clustered based on their distance from each other. Hierarchical clustering puts all modules in their own separate clusters and iteratively clusters the two closest clusters until all modules are in a single cluster. Distance between two clusters is determined through complete linkage; this means that the distance between two clusters is equal to the distance between the farthest two points within each cluster.
All data handling and analysis will be done in RStudio v1.1.463, running R v3.5.1. Linear models will be built using the ‘stats’ package.
13 iii. Parameter selection
For each USER‐dimension, a model is made with all variates identified from literature and all time registered per module (cluster). Variates are then exhaustively swapped in and out and the candidate model with the lowest Akaike’s information criterion (AIC) is selected as the best model for that respective USER‐ dimension. AIC is an information criterion for which smaller values correspond to a better fit of the model on the dataset. AIC also penalizes model complexity to counteract model overfitting [10].
iv. Checking assumptions
Linear regression is paired with certain statistical assumptions; (1) relationships between determinants and outcome variable are linear, (2) homoscedasticity of residuals, (3) normally distributed residuals, and (4) no (multi)collinearity between determinants. After building and selecting the best models from figure 2,
assumptions for linear regression will be checked.
This will involve firstly evaluating a plot of the residuals vs. fitted values to assess if there is linearity between outcome and determinants. The residuals should be roughly evenly spread around 0.
Second, the scale‐location plot is checked for homoscedasticity of residuals. The plot should show a fairly distribution of residuals vs fitted values around a horizontal line.
Afterward, the ‘quantile‐quantile plot’ (qqplot) for each model will be checked to assess if residuals are distributed normally. The qqplot should roughly follow a straight diagonal line.
Fourth, the variable inflation factor (VIF) will be calculated for each model. This can be used as a metric for collinearity within a regression model, according to Sheather (2009) [11]. A commonly used cutoff point for VIF is 5, with VIF values of 5 or higher indicating moderate to higher (multi)collinearity [11].
Lastly, the leverage vs the residuals plot will be checked for outliers. According to Cook, records with a cook’s distance of more than 1 in the plot are deemed too influential in the regression equation, and therefore too extreme a value, and will be omitted from the dataset [12].
If any assumptions are not met, data will be inspected to find non‐linear relationships or collinearity. Appropriate transformations will be made, and models will be rebuilt and evaluated anew. If these
transformations prove ineffective in helping the model meet assumptions, then the original model is accepted and used for analysis, despite its potential bias or inaccuracies.
v. Model selection per USER-dimension
For each USER‐dimension, five models (each using a different cluster method for time per module) will be evaluated using their ‘predicted’ R2. The predicted R2 is calculated by first removing a single observation
from the data, calculating the regression model on the rest of the data and predicting the value that was removed. This is repeated until each value is removed from the dataset and predicted once.
The differences between the predicted and actual value, or residuals, are squared and summed to give the PRESS [13] statistic. When the PRESS value is divided by the total sum of squares and then subtracted from 1, the predicted R2 is given, which will be used as the metric for model quality. For each USER‐dimension, the
model with the highest predicted R2 will be selected as the best fitting model. Predicted R2 is a ‘stricter’ way of
measuring model quality, as it tests the regression estimation on ‘new’ data (the removed point). A model selected on regular R2 or adjusted R2 could be fit on noise; this risk is reduced by selecting on predicted R2.
Additionally, the change in USER‐dimension score as fraction of maximum possible improvement was tested as an alternative outcome variable. These models were also included in the selection process.
The cluster method used most often in the model selected as ‘best’ model for each USER‐dimension will form the answer to RQ2.
14
2.5 RQ3: Interpreting ‘best’ model per USER-dimension
Figure 3 is an overview of the methods necessary to answer RQ3. Items in figure 3 labelled with a roman numeral are discussed in more detail. Associations are extracted (i) from the best models, selected through methods from RQ2. These models are also used to (ii) predict a change in USER‐dimension score for an ‘average patient’. These processes are described in more detail below.
Figure 3: Interpreting the selected models from RQ2. (i) Each ‘best’ model’s significantly found associations are described from model summaries, and (ii) average patients are simulated and fed into the regression model.
i. Extraction of associations
After the best models for each USER‐dimension are selected and regression assumptions are checked, the associations between model variates and change in USER‐dimension will be summarized for each best model. This will form the answer to research question 3i.
Associations from the regression models are coefficients of the regression equation. For example, if the coefficient of the walking cluster (C.Lopen) in the mobility model is estimated at 0.5, this indicates a positive association between walking cluster and mobility. For every additional hour of walking treatment, the model estimates an increase of 0.5 in change in mobility score.
For each model a summary is given of how to interpret the associations found to maximize/minimize a predicted change in a USER‐dimension. This could be followed by any care professional at DHR to assist in directing/planning care for a patient.
ii. Average patient simulation
An average will be made for all values of each model variate and stored in a so‐called ‘simulated patient’, for the entire population (appendix P). By calculating these simulated patients’ time registered per module(cluster) and plugging them into our regression coefficient estimates, it will be possible to calculate what proportion of predicted change in USER‐dimension can be attributed to time per module(clusters) or and baseline states and patient characteristics.
15
2.5 RQ4: Splitting data into improved and worsened patients, finding patient profiles and interpreting
split models
Figure 4: Methods to answer RQ4. (i) Data is split into improved and worsened patients with respect to 6 USER-dimensions and aggregate. (ii) From the split samples, Wilcoxon rank sum tests will determine on what characteristics patient profiles can be made. (iii) Associations between treatment and outcome are compared between improved and worsened patients, and (iv) average simulated patients (for improved and worsened patients) are fed into the regression models.
i. Data splitting
The data set will be split on each of the USER‐dimensions into improved and worsened patients for each USER‐dimension. The best model for each USER‐dimension will be rebuilt on both splits of the data; the selected model variates stay the same, while the coefficients are recalculated. The observed change in coefficients shows how the association between modules and change in USER‐dimension varies between the improved and worsened patients.
16 ii. Wilcoxon rank-sum test: Patient profiles
Baseline measurements for rehabilitation outcome determinants are then compared between the improved and worsened patients with a Wilcoxon rank sum test. The Wilcoxon rank sum test is a non‐ parametric test specially geared towards unpaired data.
This test involves ranking determinant values for improved patients and worsened patients from smallest to biggest. The ranks from the improved patients are then compared to the ranks of the worsened patients; if a statistically significant difference, then the improved and worsened patients significantly differ on that determinant. The average value for that determinant for the improved patients is then included in the profile for improved patients; the average value for the same determinant for worsened patients is included in the profile for worsened patients. This forms the answer to RQ 4i.
iii. Extraction of associations
The associations between model variates and change in USER‐dimension will be summarized for each model on split data. Coefficients of the regression equation, recalculated on improved or worsened patients, will be summarized per USER‐dimension.
The split models will also be evaluated for their fit on the split data, also using predicted R2. This is
important, as these models could potentially be used for advice in directing treatment. If a split model has a higher predicted R2 than the model for the entire population, this more specific model should be used to
provide guidelines for treatment allocation; if not, guidelines from the population model should be used. iv. Split average patient simulation
Split average patient simulation involves averaging values for all relevant variates for the relevant split of the data; the ‘simulated average improved patient’ regarding cognition will contain averages for treatment times and baseline characteristics from patients that improved in cognition score, for example.
17
3. Results
3.1 RQ1: Identifying determinants of rehabilitation outcome
i. Literature search and expert interviews
The first research question was to find determinants of rehabilitation outcome in literature and from expert interviews that were available at DHR. The literature search terms and their respective amount of hits is visible in appendix C. From the initial search results, a total of 36 papers were included in the identification of rehabilitation outcome determinants.
The determinants identified from the 36 selected papers are summarized in tables 2, 3 and 4. These tables contain the determinants that were available for analysis, the incomplete determinants and the unavailable determinants respectively. Determinants from articles were aggregated on similarities. After identification of a determinant through literature, the location of the determinant (Ecaris or EHR) was found through expert interview.
Determinant of rehabilitation outcome (method of measurement) Determinant sources Availability Stored where and in what form? Baseline physical and cognitive
functionality and/or independence (Functional independence Measure [14], [15], [16], Barthel Index [17], Mini‐ mental state examination [18], [19], Fugl‐Meyer Assessment [18], IQCODE [19], Five Scored Variables [20])
[14], [15] [16], [17], [21], [18], [19], [20] Available; Admission USER‐dimension scores for - Cognition - Mobility - Self‐care Ecaris Age [21], [22], [15], [16], [17], expert opinion Available EHR Dysphasia [ [17]] Available;
Admission USER score for self‐ expression
Ecaris
Neglect (severity)
(Behavioral Inattention Test scores [23])
[ [17], [23]] Available;
Admission USER score for visual perception
Ecaris
Time between onset and rehabilitation [ [24], [25]] Available;
Difference between start of treatment and date of initial diagnosis is used for each clinical admission.
Ecaris
Nutritional status
(‘oral health status’ score [26], BMI [27], blood lipid levels [ [22]], geriatric nutritional index [28])
[ [26], [27], [22], [28]]
Available; SNAQ
Height and weight were
measured in 9% and 11% of the cases respectively and deemed
18
insufficiently complete for analysis. SNAQ was also incomplete, but missing values correspond to ‘not
malnourished’. Financial support and/or SES (socio‐
economic state)
[29], expert opinion
Available; SES
The SES of a patient is defined in this thesis as the average yearly income of their municipality. Municipalities of each patient were extracted from the EHR. Average income per
municipality was taken from the ‘Centrale Bureau voor Statistiek’ [30].
EHR
Depression (GDS‐15 [25])
[31], [25] Available;
Admission USER score for - Depressed mood - Sadness
- Anger
Ecaris
Anxiety
(Hospital Anxiety and Depression Scale [25])
[25] Available;
Admission USER score for anxiety
Ecaris
Executive function [32] Available;
Admission USER scores for - Attention,
- Task completion, - Behavioral control, - Memory
Ecaris
Amount and severity of Comorbidities (Charlson index, cumulative illness rating scale (CIRS), CIRS‐severity index, Comorbidity‐severity index, Liu index [ [30]]. Preadmission comorbidities, level of consciousness and neurological/focal deficit (PLAN) score [ [20]].)
[30], [33], [20], expert opinion
Available;
Number of comorbidities counted from free text. In the EHR, comorbidities are stored as free text as ‘secondary diagnoses’. The number of comorbidities were counted by hand for 100 records. A
regression model was built to predict the number of comorbidities based on free text; for the 100 hand‐counted records, the model predicted number of comorbidities with a MSE of 0.46 (with an adjusted R2 of 0.76). The predicted
19
number of comorbidities was used for analysis. Unfortunately, severity of comorbidities cannot be included in this variable. Records without any free text were assumed to have no comorbidities.
Activities of daily living
(separately scored ADL; ‘grooming, dressing, bowel control, social interaction’ scores)
[34] Available;
Admission USER scores for - Grooming,
- Dressing,
- Incontinence bowel, - Social behavior
Ecaris
Table 2: Identified determinants from literature that were available at DHR.
Determinant of rehabilitation outcome
(method of measurement)
Determinant sources
Availability Stored in;
Injury severity
(National Institutes of Health Stroke Scale [ [18]])
[21], [18], [17], [24]
Incomplete. No aggregate severity score is measured routinely. Completeness for ‘recurrent ABI’,
‘ischemic/hemorrhagic’, ‘location’, and ‘trauma’ were 63%, 70%, 28% and 47% respectively. These
determinants are omitted from the dataset.
EHR
Grip strength [35] Incomplete. Completeness for
the nine‐hole peg test was too low for analysis, at 9.01%.
EHR
Living situation
(Alone or accompanied)
[32] Incomplete. Completeness for living situation was 75%.
EHR
Previous stroke [17] Unavailable. Completeness for
‘recurrent ABI’ was 63%.
EHR Balance
(Berg Balance Scale (BBS) [18])
[18] Unavailable. Completeness for BBS in the EHR was 11%.
EHR
20 Determinant of rehabilitation outcome (method of measurement) Determinant sources Availability
Patient motivations and expectations
(FREM‐17, PAREMO‐20 [36])
[36] Unavailable
Mechanism of injury [33] Unavailable
‘Therapeutic alliance;’ the amount of cooperation between caregiver and consumer
(WATOCI score [37])
[37] Unavailable
Patient educational level [24] Unavailable
Impulsivity [17] Unavailable
Table 4: Identified determinants from literature that were completely unavailable at DHR.
Baseline physical functionality, baseline cognition, baseline personal care, dysphasia, neglect, depression, anxiety, executive function and activities of daily living were available, but as admission USER scores instead of the measurements used in literature. Time between onset and rehabilitation could be calculated from the date of diagnosis and start of treatment, both stored in Ecaris. Nutritional status was available as SNAQ (Short Nutritional Assessment Questionnaire), with missing values being imputed with ‘not malnourished’.
Comorbidities were available in the EHR as free text. The comorbidity data therefore required some transformation before it was ready to be used for quantitative analysis. A linear regression model was built to predict the number of comorbidities from the text. Firstly, 100 observations of raw comorbidity data were labelled with a comorbidity count by hand. Consequently, the regression model aimed to find significant determinants for comorbidity count. Through backward stepwise selection on AIC, the best model predicted the number of comorbidities with an MSE of 0.46 and an adjusted R2 of 0.76. Whilst this is not the gold
standard, it provides a transformation of comorbidities from text to a numerical variable that is applicable for the rest of this research.
To summarize, multiple determinants were identified from literature. The ones that were eventually available for analysis (sometimes after transformation) were baseline physical and cognitive functionality and/or independence scores, age, dysphasia score, neglect score, time between onset and rehabilitation, nutritional status score, financial status, depression score, anxiety score, executive function score, amount of comorbidities, and activities of daily living scores.
ii. Dataset
A dataset was created consisting of 830 clinical patients with complete admission USER‐dimension scores, complete discharge USER‐dimension scores and time per module. Values for available determinants were added from the EHR. The distribution of these 830 patients into diagnosis groups is visible in table 5, in addition to values for each diagnosis group’s ‘analysis quality’ metrics.
21 iii. Diagnosis group selection
The cardiovascular diagnosis group was the group that scored best according to the analysis metrics and had the highest number of patients. This group was therefore the selected population for analysis. This is visible in table 5; the green highlighted totals are totals for the diagnosis groups of the highest interest.
Diagnosis group ICD9 code range
‘Analysis quality’
N = Unregistered time (%) ‘Generic’ time (%) Total time (%)
Cardiovascular 390 – 459 344 16 13 29 Nervous system 320 – 389 289 14 23 38 Injury/Poisoning 800 – 999 121 7 22 29 Postoperative care V50 – v59 26 7 32 40 Neoplasm 140 – 239 22 16 12 28 Musculoskeletal 710 – 739 12 10 32 41 Congenital 740 – 759 8 7 41 49 Skin 680 – 709 4 5 36 41 Mental/behavioral V40 2 4 24 28 Infectious 001 – 139 1 10 44 53 Endocrine 240 – 279 1 25 27 52
Table 5: The created dataset per diagnosis group. Diagnosis groups code ranges follow the WHO categorization of ICD-9 codes [38]. According to the 'analysis quality' metrics, the cardiovascular group is most suited for analysis. Other diagnosis groups of high quality are highlighted in green; these were however not chosen for analysis, as the sample was not as large as for the cardiovascular diagnosis group.
3.2 RQ2: Created models, parameter optimization and model selection
i. Data inspection and cleaning
Outliers of each USER‐dimension change were inspected. A distribution and boxplot of each is visible in appendix D. Given the fairly normally distributed USER‐dimensions changes, no extreme values were deleted. From the 344 cardiovascular patients, 338 patients’ diagnoses were described as cerebrovascular accidents. The remaining six were acute myocardial infarction (3), cardiomyopathy (2) and peripheral artery disease (1). These six were omitted from analysis to increase homogeneity of the dataset.
Descriptions of the model variates, determinants from literature and population characteristics included in the dataset are given in table 6. Median values are given along with the inter‐quantile range (IQR), where applicable.
The top‐down clusters were originally created by Buiter for multiple study populations with different diagnoses. This thesis, in contrast to Buiter, is focused only on one diagnosis group. As a result, some clusters originally made by Buiter remain empty for the cardiovascular group; this is logical, as for instance the ‘nerve damage’ cluster (C.Zenuwletsel) contains modules specifically meant for patients with a spinal cord injury and not for CVA.
Model variates Median IQR
Change in mobility 10 13
Change in personal care 7 11
Change in cognition 4 11
22
Change in fatigue* ‐10 30
Change in mood* ‐0.63 5.63
Aggregate change 7.46 12.35
Time registered per top‐down module cluster (clusters with no time registered on them are ‘empty’)
Median IQR ‘Breathing’ (C.Ademhaling) (empty) (empty)
(C.ArmHand) 6.29 25.1 C.Assessment) 11.12 6.19 C.BehandelbarePijn) (empty) (empty)
C.BlaasDarm) (empty) (empty) C.CognitieveFuncties) 1.54 8.15
C.Communicatie) 2.89 11.63 C.ConSpiVoe) 8.6 9.74
C.Wonden) 0 0
C.Generiek) 9.63 10.11 C.HandP) (empty) (empty) (C.Lichaamshouding) (empty) (empty) (C.Lopen) 4.22 10.43 (C.Maatschappelijk) 6.11 7.56 (C.OOBInactiviteit) 0 0 C.Psychosociaal 0 0 C.Slikken 0 0 C.Spasticiteit 0 0 C.Systeem 0 1
C.Zenuwletsel (empty) (empty) C.Zitten 0.5 1.95
C.Zelfzorg 0 0
Available determinants Median IQR
Admission score mobility 16 20
Admission score personal care 23 14
Admission score cognition 38 15
Admission score pain 0 30
Admission score fatigue 50 30
Admission score mood 20 32.5
Days between onset and rehabilitation 14 13
SES 26.3 3.1
Age 62 16
Comocount (number of comorbidities) 1 2
N = %
SNAQ; Not malnourished 263 77.8
Risk of malnourishment 51 15.1 Malnourished 24 7.1
Median IQR
Dysphasia score 3 2
23
Depression; Sadness 20 50
Anger score 0 30
Depressed mood score 20 40 Anxiety score 0 40
Executive functioning; Attention score 3 3
Task completion score 3 4 Behavioral control score 5 0
Memory score 3 2
Activities of daily living; Grooming 3 4
Dressing score 2 2 Bowel control score 5 0 Social behavior 5 0 Population characteristics
N = %
ICD9 code 430: Subarachnoid hemorrhage 24 7.1
431: Intracerebral hemorrhage 60 17.8 432.1: Nontraumatic subdural hemorrhage 4 1.2 436: Acute, but ill‐defined, cerebrovascular disease 230 68
437.9: Unspecified cerebrovascular disease 7 2.1 438: Late effects of cerebrovascular disease 13 3.8
Sex; Male 192 56.8
Female 146 43.2 Total 338 100 Table 6: Study population characteristics. Median values and the IQR are given where applicable; otherwise amounts and percentages
are given. *Negative scores correspond to improvement. ii. Created models: Results from clustering
Modules were clustered hierarchically on their correlation. The resulting clusters are visible in a dendrogram in appendix E. A table with each cluster and the modules per cluster is visible in appendix F. The clusters taken from Buiter’s research, with adjustments, is visible in appendix B.
The hierarchical clusters are difficult to label by their contents in a medically meaningful way. Modules that were clustered together often have little to no medical overlap, and there are little other available criteria for comparing the modules that are clustered together.
What can be said for certain is that the created clusters are clusters of modules often given together; this is seen in the dendrogram where for instance all of the modules specifically for children are clustered together (in the bottom right). The created clusters could therefore possibly represent combinations of modules that are often indicated and given to patients together. As this does not fall within the scope of this research, this was not investigated further.
iii. Parameter selection (model optimization)
The models created initially contained all predefined variates. Through stepwise backwards selection, a subset of parameters was chosen that minimized the model’s AIC for all 35 models.
iv. Checking assumptions
After selecting parameters for each model, assumptions for linear regression were checked. The best models for mobility, personal care, fatigue and aggregate met all assumptions, and had respective VIFs of 2.38, 2.78, 1.60 and 1.76. No outliers were found with a cook’s distance above 1. The diagnostics for each model can be found in appendix G. The models for mobility, personal care, fatigue and aggregate are therefore accepted in their current form.
24
The best model for cognition however showed slight violation of linear regression assumptions. The cognition model’s residuals vs fitted values show that there is possibly slight non‐linearity between
determinants and target variable, as the relationship between fitted values and residuals is slightly convex; in predicting values further from 0, the model increases in error. A scatterplot was therefore made for each model element and change in cognition, visible in appendix H, but no model elements showed a clearly non‐linear relationship. Furthermore, the residuals are distributed normally ‘enough’. The model’s VIF was 2.48, indicating low collinearity. No records were found with a cook’s distance of more than 1. The cognition model is therefore accepted in its current form.
The same counts for the mood model; the fitted vs residual plot shows a slightly concave relationship. Upon closer inspection however, the model elements of the mood model do not have a non‐linear relationship with change in mood. This is visible in appendix I. The qqplot for the mood model was deemed to show
‘enough’ normality, and the VIF was 2.00. No records were detected with a cook’s distance of more than 1. The model for mood is therefore accepted but may be slightly biased.
For the pain model, the residuals vs fitted line indicates that there is clear non‐linearity in the relationship between determinants and outcome. This is visible in appendix J, where each module cluster seems to have a slightly concave correlation with change in pain. Values for these clusters are rooted, but this gave a model with the same adjusted R2 (0.35), and the same predicted R2 of 0.3. The model built on the newly
transformed data still violates the assumptions of regression however, as the concave relationship between residuals and fitted values still exists (visible in appendix K). No outliers with a cook’s distance of higher than 1 were found, and the mood model had a VIF of 2.00 for the original model. The original model is therefore accepted, as the transformed model is not of higher quality. Likely, change in mood is better analyzed through a different algorithm than linear regression.
v. Model selection per USER-dimension
Table 7 contains the predicted R2 for each created model. For each USER‐dimension, the best model
with the highest predicted R2 was selected. For aggregate USER, cognition, mobility, fatigue, pain, and personal
care, the top‐down cluster method provided the most accurate models. Mood was best predicted by the model with treatment modules in 20 hierarchical clusters. Summaries for the best models are visible in appendix M.
The change in USER‐dimension score as fraction of maximum possible improvement was also tested as outcome variable. The models built with these outcomes yielded drastically lower predicted R2 andadjusted R2
values however, sometimes up to 4 times lower. These models were therefore not included in this research’s results or findings. The fit statistics of these models are available in appendix L.
Cluster method
User-Dimension Mobility Personal
care Cognition Pain Fatigue Mood
Aggregat e Hierarchical clusters (k = 10) 0.00 0.55 0.46 0.27 0.26 0.46 0.36 Hierarchical clusters (k = 15) 0.45 0.55 0.44 0.28 0.23 0.46 0.36 Hierarchical clusters (k = 20) 0.16 0.55 -6.12 0.28 0.31 0.47 0.36 Modules separate 0.00 0.00 0.00 0.00 0.00 0.46 0.30 Top-down clusters 0.53 0.58 0.55 0.29 0.32 0.46 0.37
Table 7: Predicted R2 for all created models. The best model per USER-dimension is highlighted in green.
3.3 RQ3: Interpreting ‘best’ models
The statistically significant associations from the best models are summarized in tables 8 and 9. In these tables respectively, module clusters and determinants of rehabilitation outcome are given a ‘+’ for a statistically
25
significant positive association, and a ‘‐’ for statistically significant negative associations. For items labelled with a ‘+’, this means that increasing this module cluster’s time or determinants value will increase the predicted change in the relevant USER‐dimension.
A more detailed description of the associations, and how best to use the models to maximize predicted change in USER‐dimension score, is also given per model after tables 8 and 9.
i. Extraction of associations
Mobility Personal Care
Cognition Pain Fatigue Mood Aggregate
Predicted R2 0.534 0.586 0.559 0.295 0.324 0.474 0.371 Intercept + + + + Module cluster Arm/Hand function + + Assessment + + Cognitive functions - - Communication + - -
Conditioning, muscles and nutrition Wounds + + Generic Walking - - - Societal + + Psychosocial Swallowing - Spasticity - - - Systemic‐social - Sitting - Self‐care c20.5* + c20.11* +
Percentage of indicated modules used
+ Amount of indicated modules
Proportion of treatment time not registered to module (%) Proportion of treatment time registered to generic module (%)
+ -
Table 8: Summary of significantly associated module clusters per change in USER-dimension. *The modules for these clusters are visible in appendix F.
In table 8 it is visible that for changes in 3 USER‐dimensions, mobility, personal care and cognition, there is a statistically significant positive intercept. Several module clusters are significantly positively associated with changes in USER‐dimensions, such as arm/hand function or assessment, while some module clusters are exclusively negatively associated with changes in USER‐dimensions score; cognitive functions, walking or swallowing for instance. Some module clusters are not significantly associated with changes in any USER‐
26 dimension score (psychosocial and self‐care).
Table 9 is an overview of determinants that were significantly associated with a change in score per USER‐dimension. Every baseline USER‐dimension score was significantly associated with a change in one of the 6 USER‐dimension scores. The only determinants that were significantly associated with a change in USER‐ dimension score, aside from baseline USER‐dimension scores, were the number of comorbidities and SNAQ.
Determinants that were not significantly associated with change in any USER‐dimension score were omitted from table 9 (days between onset and treatment, socio‐economic state, age, SNAQ: not malnourished, dysphasia, Neglect, depression: sadness, depression: anger, depression: depressed mood, anxiety, executive functioning (EF): attention, EF: task completion, EF: behavioral control, EF: memory, activities of daily living (ADL): grooming, ADL: dressing, ADL: bowel control, ADL: social behavior).
Mobility Personal Care
Cognition Pain Fatigue Mood Aggregate
Predicted R2 0.534 0.586 0.559 0.295 0.324 0.474 0.371 Determinant Baseline mobility ‐ ‐ Baseline personal care + ‐ + Baseline cognition ‐ + ‐ Baseline pain ‐ + + + Baseline fatigue + ‐ + Baseline mood ‐ + Number of comorbidities ‐ SNAQ: Risk of malnourishment + SNAQ: Malnourished +
Table 9: Determinants of rehabilitation outcome that were significantly associated with change in USER-dimension. Best mobility model
A statistically significant intercept was found of 12.6, indicating an estimated predicted increase of mobility score by 12.6 for all patients. Wounds (C.Wonden), communication (C.Communication) and assessment (C.Assessment) clusters were positively associated with a change in mobility score, with estimates of 3.71, 0.05 and 0.24 respectively. A positive association was also found for baseline personal care score with a coefficient of 0.23. This means that change in mobility is increased for patients with high baseline personal care.
Spasticity and walking (C.Lopen) clusters are negatively associated with change in mobility score, estimated at ‐1.03 and ‐0.12 respectively. Baseline mobility was also negatively associated with change in mobility, estimated at ‐0.73.
Based on the best model for mobility, one can infer that change in mobility is higher for patients who received more wound treatment, communication and assessment treatment, and relatively less walking and spasticity treatment. Patients with low baseline mobility and high baseline personal care have higher predicted change in mobility.
Best personal care model
For personal care, a statistically significant intercept was found of 12.6. This means that, for all patients, a 12.6 increase in personal care score is predicted. Positive associations were found for assessment and wounds clusters of 0.15 and 2.71 respectively. Worse SNAQ states, ‘risk of malnourishment’ and ‘malnourished’, are also
27
positively associated with change in personal care with coefficients of 1.29 and 1.41 respectively.
Spasticity and walking clusters were negatively associated with change in personal care at ‐0.61 and ‐ 0.05. Baseline personal care and baseline mood were also negatively associated with change in personal care, at ‐0.6 and ‐0.006 respectively.
These associations mean that patients’ predicted change in personal care score is higher for every additional hour of assessment of wound treatment, but lower for every additional hour of walking and spasticity treatment. Predicted change in personal care increases for patients with lower personal care baselines and malnourishment/increased malnourishment risk.
Best cognition model
A positive intercept op 26.4 was found, indicating a 26.4 predicted increase in cognition score for all patients. The ‘societal’ (C.Maatschappelijk) and ‘arm/hand’ (C.ArmHand) clusters were positively associated with a change in cognition score, with estimations of 0.11 and 0.05 respectively. Worsened SNAQ states are positively associated with change in outcome, with ‘risk of malnourishment’ estimated at 2.1, and
‘malnourished’ at 3.02. Baseline personal care is positively associated with cognition change at 0.23.
Clusters cognitive functions (C.CognitieveFuncties), communication and ‘systemic‐social’ (C.Systeem) were negatively associated with change in cognition, with estimations of ‐0.07, ‐0.06 and ‐0.47 respectively. Baseline cognition is also negatively associated at ‐0.75.
Thus, the best cognition model suggests that change in cognition score is highest when patients are malnourished (or at risk of malnourishment) and have high baseline personal care but low baseline cognition. Predicted change in cognition score is increased by societal and arm/hand treatment time, while higher amounts of systemic‐social, communication and cognitive functions modules predict a lower change in cognition.
Best pain model
No statistically significant intercept was found in the best model for pain. ‘Swallowing’ (C.Slikken), walking and communication clusters are associated with a negative change in pain with coefficients estimated at ‐0.38, ‐0.26 and ‐0.19. These associations are essentially positive, as pain is scored inversely (higher scores correspond to more pain), meaning more of swallowing, walking and communication clusters will reduce pain. Higher baseline pain is also associated with a decrease in pain score. The societal cluster is the only cluster associated with an increase in pain at 0.31.
This would mean that patients who receive more swallowing, walking and communication treatment are predicted to improve more with regards to pain, whilst patients with higher baseline pain also show higher decreases in pain. Patients with higher societal module treatment time rehabilitate worse.
Best fatigue model
The estimated intercept of the model was not statistically significant. The only association found
between modules clusters and change in fatigue was the sitting cluster, at ‐1.42. Baseline fatigue was associated with change in fatigue at ‐0.65, indicating that worse admission fatigue is associated with a higher decrease in fatigue score. Baseline pain and cognition were associated with change in fatigue at 0.13 and 0.28 respectively.
In maximizing the negative change of fatigue, sitting treatment modules should be maximized. Patients with worse baseline fatigue, lower baseline cognition and lower baseline pain tend to rehabilitate better.
Best mood model
The estimated intercept of the best model for mood was not statistically significant. No module clusters are associated with a decrease in mood score. Baseline mood was associated with a decrease in mood score, at ‐0.17, meaning that patients with worse baseline mood show higher decreases in mood. Number of
comorbidities was associated with improved mood outcome at ‐1.16.
Two module clusters, c20.5 and c20.11, are associated with a worsening in mood. Their effects are estimated at 1.02 and 3.06 respectively. Cluster c20.5 contains ‘neuro psychological examination’, ‘impeding sickness cognitions’, ‘social‐systemic problems for CVA’ (cerebrovascular accident), ‘self‐efficacy’ and ‘living’, and c20.11 contains ‘diet insufficient’, ‘acceptation/coping problems’ and ‘generic examination’.
28
Patients’ predicted change in mood score is therefore best for patients with low amounts of treatment time for modules contained in c20.5 and c20.11, and they will rehabilitate better with respect to mood if they have comorbidities, low baseline pain and high baseline mood.
Best aggregate model
A statistically significant intercept of 14.5 was found for aggregate USER. No module clusters were found to be positively associated with aggregate USER.
The spasticity cluster is the only module cluster associated with aggregate USER, with a negative coefficient of ‐0.85. Associations were found for baseline pain, baseline fatigue and baseline mood, at 0.08, 0.1, 0.02. However, lower scores correspond to better states. This essentially means that the association between baselines and aggregate USER is also negative, as a worse condition in terms of pain, fatigue and mood will correspond to higher change in aggregate USER.
In respect to aggregate USER, patients with low baseline mobility, low baseline cognition, poor baseline mood, but high baseline pain and fatigue rehabilitate better. According to the model, spasticity treatment should be avoided.
ii. Average patient simulation
Averages were taken of each of the 338 patients for all variates used in each model and stored in an ‘average simulated patient’ (appendix P). This patient was plugged into each regression model to predict the change in USER‐dimension. Time per significantly associated module cluster multiplied by its respective coefficient is summed for this average patient, and the proportion of predicted change in USER‐score is calculated. The result is visible in table 10.
USER-dimension
Predicted change in USER‐dimension score
Change attributed to time per module cluster
Predicted change attributed to module clusters (%) Mobility 10.4 5.5 52.7% Personal care 7.8 1.3 16.1% Cognition 5.9 1.6 26.8% Pain ‐3.6 1.1 ‐31.6% Fatigue ‐13.9 4.3 ‐23.5% Mood ‐8.7 1.6 ‐18.6% Aggregate 8.5 ‐1.2 ‐11.9%
Table 10: Patient simulation results. Predicted changes in USER-dimension, predicted change in score attributed to time per module cluster and proportion of predicted change attributed to modules for an average patient.
For mobility, personal care and cognition, the predicted change attributed to modules is 52.7%, 16.1% and 26.8% respectively. For pain, fatigue and mood, modules hold a negative proportion of predicted change, with percentages of ‐31.6%, ‐23.5%, ‐18.6% respectively. For aggregate, a negative proportion of predicted change from modules is observed, with a percentage of ‐11.9, despite the positive predicted change in score.
From here it can be deduced that time per module clusters in general has significant positive effect in predicting change in mobility, personal care and cognition, but predicted negative effect for predicting change in pain, fatigue and mood.
3.4 RQ4: Interpreting split models
i. Data splitting
Data was split on each USER‐dimension, between improved patients ( > 0 USER‐dimension change) and worsened patients ( =< 0 USER‐dimension change). For fatigue, pain and mood, the data is split into improved
