Validation of the Therapeutic Intervention
Scoring System and the Nursing Activity Score
to measure patient acuity in Dutch ICUs
Hugo de Bruijn
June, 2018
Supervisors:
C.C. Margadant, MSc
Dr. S. Brinkman, PhD
Prof. dr. N.F. de Keizer
University of Amsterdam
Master Medical Informatics
2
Validation of the Therapeutic Intervention Scoring System and the
Nursing Activity Score to measure patient acuity in Dutch ICUs
Student
Hugo de Bruijn, BSc
Student number: 10002585
E-mail: h.debruijn@amc.uva.nl
Mentor
C.C. Margadant, MSc
E-mail: c.c.margadant@amc.uva.nl
Tutors:
Dr. S. Brinkman, PhD
E-mail: s.brinkman@amc.uva.nl
Prof. dr. N.F. de Keizer
Email: n.f.keizer@amc.uva.nl
Location of Scientific Research Project
National Intensive Care Evaluation (NICE)
University of Amsterdam (UvA)
Amsterdam Medical Center (AMC)
Faculty of Medicine, Department of Medical Informatics
Meibergdreef 15, 1105 AZ
Amsterdam, The Netherlands
E-mail: secretariaat@stichting-nice.nl
Period of Scientific Research Project
3
Contents
Abstract ... 4
Introduction ... 6
Research objectives ... 8
Data and methods ... 9
Data collection ... 9
Data preparation ... 10
Data analyses ... 11
1. Data quality of the self-reported TISS and NAS scores ... 11
2. Validation of TISS and NAS nursing workload models ... 11
3. Association between nursing time and organizational factors ... 12
4. Association between nursing time and patients characteristics ... 12
Results ... 13
Included data ... 13
1. Data quality of the self-reported TISS and NAS scores ... 14
2. Validation of TISS and NAS nursing workload models ... 20
3. Association between nursing time and organizational factors ... 22
4. Association between nursing time and patients characteristics ... 23
Discussion ... 25
References ... 28
Appendix 1. Descriptions of TISS-28 nursing activities by item number ... 30
Appendix 2. Descriptions of NAS nursing activities by item number ... 31
Appendix 3. TISS-28 variable mapping ... 33
4
Abstract
Background: Nursing workload in Dutch ICUs is generally assessed with the Therapeutic Intervention
Scoring System (TISS) and/or the Nursing Activities Score (NAS). However, TISS and NAS data collection might be subject to data entry errors and do not cover all workload. First objective was to validate and evaluate the data quality of the registered TISS-28 and NAS data. Our second objective was to validate and evaluate the reliability of the TISS-28 and NAS workload models for the
estimation of nursing time. The third objective was to analyze the association of the average nursing time spent per patient with organizational factors and patient characteristics.
Methods: Time measurements were performed by 14 observers in 6 Dutch ICUs on 236 patients
using an application that logs the time of each nursing activity belonging to a patient. The data quality of the self-reported data was assessed by analyzing the correlation between the self-reported and observed TISS and NAS scores/times per patient and per nursing activity using the Pearson’s correlation test. We used the Wilcoxon Signed-Rank test to analyze the association of observed nursing time spent per patient with organizational factors and patient characteristics.
Results: In general, the self-reported TISS and NAS scores were higher compared to the observed
TISS and NAS scores. A high correlation was found between the total self-reported and total
observed NAS scores per nursing activity item (r=0.85). Moderate correlation was found between the total reported and total observed TISS scores per activity item, and between the total self-reported and total observed NAS time per nursing activity item (respectively r=0.68 and r=0.50). However, low correlation was found between the total self-reported and total observed TISS and NAS scores per patient (respectively r=0.24 and r=0.30), between the total self-reported and total observed TISS and NAS time per patient (respectively r=0.29 and r=0.37) and between the total self-reported and the total observed TISS time per nursing activity item (r=0.27). The mean observed nursing time per patient during the day shift (2.5 hrs.) and the evening shift (2.4 hrs.) were significantly higher than the observed nursing time during the night shift (2.0 hrs.). The mean observed nursing time for patients admitted as medical (2.5 hrs.) and emergency surgery (2.4 hrs.) were significantly higher compared to elective surgery (2.0 hrs.). The mean observed nursing time for patients with a mortality risk between 30 and 70% (2.5 hrs.) and patients with a risk higher than 70% (2.7 hrs.) were significantly higher than for patients with a mortality risk lower than 30% (2.2 hrs.). The mean observed nursing time for patients aged between 18 and 64 years (2.5 hrs.) was
significantly higher than for patients aged between 65 and 79 years (2.1 hrs.).
Conclusion: TISS-28 is not able to reliable estimate the nursing time per patient or per nursing
activity. The NAS is more accurate in estimating the mean nursing time per patient and per nursing activity in comparison to TISS-28. However, the TISS-28 and NAS could be significantly improved by adjusting the number of points assigned to activity items and providing training for the use of workload models. The average nursing time spent per patient is associated with the organizational factor shift type and the patients characteristics admission type, severity of illness and age. Further research should be performed on the development of a more accurate and reliable model to estimate the actual nursing time spent per patient and per activity item.
Keywords: ICU nursing workload, Simplified Therapeutic Intervention Scoring System (TISS-28),
5
Samenvatting
Achtergrond: Verpleegkundige werklast op de Nederlandse IC's wordt over het algemeen
geëvalueerd met het Therapeutic Intervention Scoring System (TISS) en/of het Nursing Activities Score (NAS) model. Het verzamelen van de TISS en NAS gegevens kan echter onderhevig zijn aan rapportage fouten en de modellen bestrijken niet alle verpleegkundige werklast. Het eerste doel was om de kwaliteit van de gerapporteerde TISS-28 en NAS gegevens te valideren en te evalueren. Onze tweede doelstelling was het valideren en evalueren van de betrouwbaarheid van de TISS-28 en NAS voor het inschatten van verpleegtijd. De derde doelstelling was het analyseren van de associatie tussen gemiddelde verpleegtijd per patiënt en organisatorische- en patiëntkenmerken.
Methoden: Tijdsmetingen werden verricht door 14 waarnemers in 6 Nederlandse IC's bij 236
patiënten met behulp van een applicatie die de tijdsduur van elke verpleegactiviteit bij een patiënt registreert. De kwaliteit van de gerapporteerde gegevens werd beoordeeld door het berekenen van de Pearson correlatie tussen de gerapporteerde en de waargenomen TISS en NAS scores/tijden per patiënt en per verpleegactiviteit. De Wilcoxon Signed-Rank test werd gebruikt om de associatie te analyseren tussen de waargenomen verpleegtijd per patiënt en de organisatorische- en
patiëntkenmerken.
Resultaten: Over het algemeen waren de gerapporteerde TISS- en NAS scores hoger in vergelijking
met de waargenomen TISS- en NAS scores. Er werd een hoge correlatie gevonden tussen de totaal gerapporteerde en de totaal waargenomen NAS score per verpleegactiviteit (r=0.85). Middelmatige correlatie werd gevonden tussen de totaal gerapporteerde en de totaal waargenomen TISS score per verpleegactiviteit, en tussen de totale gerapporteerde en de totale waargenomen NAS tijd per verpleegactiviteit (respectievelijk r=0.68 en r=0.50). Er werd echter een lage correlatie gevonden tussen de totaal gerapporteerde en de totaal waargenomen TISS- en NAS score per patiënt
(respectievelijk r=0.24 en r=0.30), tussen de totaal gerapporteerde en de totaal waargenomen TISS- en NAS tijd per patiënt (respectievelijk r=0.29 en r=0.37) en tussen de totaal gerapporteerde en de totaal waargenomen TISS tijd per verpleegactiviteit (r=0.27). De gemiddelde waargenomen
verpleegtijd per patiënt tijdens de dagdienst (2.5 uur) en de avonddienst (2.4 uur) waren significant hoger dan de waargenomen verpleegtijd tijdens de nachtdienst (2.0 uur). De gemiddelde
waargenomen verpleegtijd voor patiënten die als medisch (2.5 uur) en als spoed chirurgisch (2.4 uur) werden opgenomen waren significant hoger in vergelijking met electief chirurgische patiënten (2.0 uur). De gemiddelde waargenomen verpleegtijd voor patiënten met een sterftekans tussen de 30 en 70% (2.5 uur) en een sterftekans hoger dan 70% (2.7 uur) waren significant hoger dan voor patiënten met een sterftekans van minder dan 30% (2.2 uur). De gemiddelde waargenomen verpleegtijd voor patiënten tussen de 18 en 64 jaar (2.5 uur) was significant hoger dan voor patiënten tussen de 65 en 79 jaar (2.1 uur).
Conclusie: De TISS-28 is niet betrouwbaar voor het inschatten van de verpleegtijd per patiënt of per
verpleegactiviteit. De NAS is nauwkeuriger in het schatten van de gemiddelde verpleegtijd per patiënt en per verpleegactiviteit in vergelijking met TISS-28. De TISS-28 en NAS kunnen echter nog aanzienlijk verbeterd worden door het aanpassen van het aantal punten dat is toegekend aan verpleegactiviteiten en door het geven van training in het gebruik van werklast modellen. Er is een associatie tussen de gemiddelde verpleegtijd per patiënt en het organisatorische kenmerk dienst soort en de patiëntkenmerken opname type, ernst van de ziekte (uitgedrukt in de APACHE IV
sterftekans) en leeftijd. Verder onderzoek zou moeten worden verricht naar de ontwikkeling van een accurater en betrouwbaarder model om de werkelijke verpleegtijd in te schatten.
6
Introduction
Nursing care in the intensive care units (ICUs) has made significant changes over the past years. The increase in average patient age partly contributes to these changes as the aging population results in an increase of multi-morbidity. There is also an increase in complexity and number of nursing
interventions due to new more advanced technological possibilities. All these changes contribute to an increased need for nursing care and this could result in an increased workload for ICU nurses [1, 2]. Nursing staff is the largest ICU cost, accounting around 50% of an ICU budget [3]. It is therefore essential to optimize the use of ICU resources and ensure appropriateness, efficacy and cost-effectiveness [2, 4]. A too high workload could lead to adverse events, professional dissatisfaction and absenteeism of nurses [5]. While a low workload is cost inefficient and could be experienced as not challenging resulting in a lower satisfaction of ICU nurses.
The assessment of workload in ICUs is not only important to monitor and manage this increasing workload, these assessments also make it possible to recognize patients that require intensive nursing care. Early recognition of patients who require intensive nursing care is of importance in ICUs caring for critically ill patients, because patient severity presents a direct relationship with the probability of severe adverse events occurring [6 – 9]. Therefore, early recognition of patients requiring intensive care could contribute to the prevention of these adverse events. Workload assessment gives important information for the management of ICUs to plan and allocate nursing capacity, while planning of nursing staff allocation and detection of nurse understaffing could contribute to a decrease in workload.
Workload in Dutch ICUs is generally being assessed with the Therapeutic Intervention Scoring System (TISS) and/or the Nursing Activities Score (NAS). Cullen et al. developed the TISS in 1974. The TISS was based on 57 medical therapeutic interventions [10]. Since then, the TISS is the most commonly used instrument to assess workload for nurses in ICUs and it is still being used for research purposes [6, 7, 11]. In 1983 the TISS was extended with new and additional therapeutic interventions to the TISS-76, referring to the total of 76 therapeutic interventions. Finally, in 1996 the model was
simplified and this resulted in the validated TISS-28, containing 28 items [12]. The points assigned to the nursing activities ranged from one to four (TISS-76) and from one to eight (TISS-28). These points were assigned by experienced observers solely based on the complexity of the activity as indicator for the severity of illness and not on the actual time spent by nurses [10]. It appeared that the nursing workload is only partly related to the severity of illness, which made the TISS less useful for assessing nursing workload.
In 2003 the Nursing Activity Score (NAS) has been developed. The NAS contains a list of 23 variables, which focus on nursing activities at bedside [3, 4]. Just like the TISS, the NAS is a commonly used instrument to assess workload on nursing activities in ICUs for clinical and for research purposes [5, 13 – 15]. The NAS was validated in 99 ICUs located in 15 different countries [11]. The NAS is used to measure nursing workload for each individual patient. The points assigned to nursing activities provide average time consumption in caring for patients instead of representing the severity of illness as TISS does. The NAS was developed by using the work-sampling approach, which is a technique to determine the nursing activities by asking the nurse what he/she was doing at random moments per shift [16]. For every activity there was a weight expressed in percentage granted. The total NAS score for an individual patient is the sum of all items, varying between 0 and 177. A NAS
7 score of 100 is equivalent to the amount of care that can be provided by one FTE nurse during a 24-hour shift [1]. When a score of over 100 is granted it suggests that more than one nurse is needed to provide the required care for that patient.Although the results of the NAS have been validated in 99 ICUs in different countries, it has been demonstrated that the work-sampling approach does not provide an accurate reproduction of the true nursing workload. This is due to the fact that the work-sampling approach is based on the probability of a particular nursing activity and not on the real amount of time spent per activity [16]. Therefore, this approach may not provide a precise result as provided by using time-and-motion techniques [17].
A study showed that there was a significant correlation found between the average scores of the TISS-28 and the NAS [12]. However, both the TISS and NAS do not cover all workload and therefore not all nursing activities are explained. Literature showed that the NAS covers 81% and the TISS-28 only covers 43% of the actually provided nursing time [3, 4]. The National Intensive Care Evaluation (NICE) foundation manages a continuous and complete registry containing demographic,
physiological, and clinical data of all admitted ICU patients in all Dutch hospitals [18]. In 2015 the NICE extended her registration modules with an optional “nursing workload” module to manage a national registration of nursing workload in Dutch ICUs to enable the opportunity for benchmarking. The hypothesis is that better insight in the actual workload and by comparing this workload with peers could support more efficient scheduling of nurses in ICUs. Data has already been collected from various ICUs in the Netherlands and a pilot showed that the ICUs are able to provide this data. This nursing workload module includes the TISS-28, NAS, and five new variables. Furthermore, information on the number and type of nurses working per shift was collected. This information enables the possibility to analyze the amount of workload per ICU nurse and thus give insight in the more organizational indicator average workload of ICU nurses.
Demographic and clinical factors such as age, type of admission, severity of illness could be
associated with an elevated workload of nursing staff [11, 19 – 21]. These factors are available for all ICU patients in the NICE registry, which enables the possibility to analyze their association with the nursing workload.
8
Research objectives
The aim of this study was to achieve the following objectives:
1. The first objective was to validate and evaluate the data quality of the self-reported TISS and NAS scores. What is the agreement between the self-reported TISS and NAS scores and the observed TISS and NAS scores, derived from the observed nursing time? To better
understand this agreement the following sub questions will be answered:
a. What is the agreement between the self-reported and observed TISS and NAS scores
per patient?
b. What is the agreement between the self-reported and observed TISS and NAS scores
per nursing activity?
2. The second objective was to validate and evaluate the reliability of the TISS and NAS workload models. Does the number of assigned TISS and NAS scoring points (in which each point represents to a certain amount of time) correspond with the actual observed nursing time? To better understand this agreement the following sub questions will be answered:
a. What is the agreement between the self-reported TISS and NAS nursing time per
patient (derived from the self-reported TISS and NAS score) and the observed nursing time per patient?
b. What is the agreement between the self-reported TISS and NAS nursing time per
activity item (derived from the self-reported TISS and NAS score) and the observed nursing time per activity item?
3. The third objective was to analyze the association between the observed nursing time and several organizational factors: type of shifts (day-, evening-, and nightshift), ICU levels (level 1, 2, or 3), and day of ICU admission (i.e. first ICU admission day to last ICU admission day). 4. The fourth objective was to analyze the association between the observed nursing time and
patients characteristics: admission types (medical, emergency-, and elective surgical), severity of illness expressed as the APACHE IV mortality risk, Body Mass Index (BMI) and age.
9
Data and methods
Data collection
Measurements to assess the observed TISS and NAS scores took place in six different hospitals in The Netherlands. Of these enrolled hospitals, two were university hospitals, two were teaching hospitals and two were non-academic and non-teaching hospitals. Participation was on voluntary basis and without financial reimbursements.
Multi Moment Recordings (MMRs) were performed to assess the observed TISS and NAS score for patients admitted to the ICU. An in-house developed web application was used to measure the time spent on all nursing activities in seconds. The application was developed at the department of Medical Informatics at the AMC, Amsterdam. Observers can use the application to log the start and stop time of each performed nursing activity belonging to a patient. This application included all nursing activities occurring in the TISS-28 from 1996 and the NAS from 2003, and five additional activities fitting the Dutch situation. Furthermore, non-predefined nursing activities could be
registered in the “Others” category. An activity could be entered manually in this category to analyze these tasks and identify nursing tasks not covered by the TISS or NAS. The application had different colors to distinguish ten main categories within nursing activities: central nervous system;
cardiovascular; respiratory; digestive; renal; blood; infections; bedside; mobilization and positioning; administration and support and others. Figure 1 shows a screenshot of the application’s interface and the colors per category.
Figure 1. Interface of the application used by observers. Categories by color and number of activities:central
nervous system (orange, 5); cardiovascular (dark blue, 18); mobilization and positioning (light red, 11); respiratory (green, 10); digestive (yellow, 5); renal (light blue, 4); blood (red, 2); infections (gray, 5); bedside (orange, 1); administration and support (beige, 2) and others (orange, 5).
10 For each time a nurse started a task, the observer pressed the corresponding button for recording the start time. When a button was turned on, the color changed, which indicated the specific task was being performed. When the nurse ended the activity, the button was turned off and recorded the time of ending. Each data item was send to the central database through internet. This required a stable internet connection as was present in the hospitals where measurements were performed. When two nurses took care of a specific proceeding for a patient at the same time, this was registered by using the ‘second nurse button’ in the application. If two nurses were performing different proceedings at the same time with the same patient this was also registered; more buttons could be turned on simultaneously.
The MMRs were executed by C.C. Margadant, M. Hoogendoorn, and nursing students from
Hogeschool Viaa and Windesheim Zwolle. In total 14 observers performed the measurements and all observers were instructed before and during a full nursing shift. Before the shift, the observers were instructed about the study, the method, the application, and the nursing activities. We provided the observers a list with a description of all nursing activities at the ICU. We also provided a form in which the observers could track difficulties or special circumstances during measurements.
At the start of the shift a nurse was randomly chosen. The observers followed the patient(s) that the nurse was responsible for during the entire shift. The measurements took place during different shifts (day, evening, and night), during different days of ICU admission (e.g. first IC admission day through last IC admission day) and in different ICU levels (level 1, 2, or 3). The nurses caring for the studied patients during the MMRs were instructed to fill in the NICE workload registry after their shift, to assess the self-reported TISS and NAS scores using the point system presented in the NICE data dictionary, which is available on the NICE website [18].
Data preparation
Observed MMRs data is considered the golden standard. In order to compare the self-reported scores with the MMRs data, we derived the TISS and NAS points from the observed MMRs times using the data dictionary presented on the NICE website [18]. For example, during observed time measurements a nurse performed hygiene procedures on a patient for 70 minutes. This nursing activity is not covered by the TISS model, but it is covered by the NAS (item 4). This NAS item has three categories namely performing hygiene procedures for less than two hours, for more than two hours, for more than four hours. In this example, the activity took 70 minutes and would therefore be assigned to the category for less than two hours, which corresponds to 16.5 NAS points (item 4b). In order to compare the MMRs time measurements with the self-reported scores, we derived the TISS and NAS times from the self-reported TISS and NAS scores. Miranda et al. stated that one TISS-28 point corresponds to 10.6 minutes of nursing care provided by one nurse during a 8-hour shift [22]. Patients with a total of 100 NAS points required 100% of care time provided by one nurse, during a 24-hour shift. One NAS point corresponds to 14.4 minutes ((24hrs * 60mins)/100) [12]. So, one NAS point corresponds to 4.8 minutes of nursing care ((8hrs * 60mins)/100) provided by one nurse during a 8-hour shift. With this information we were able to convert the self-reported TISS and NAS scores to an estimated self-reported nursing time per patient and per nursing activity.
The self-reported and observed variables were mapped as described in Appendix 3 and 4.
11 patients characteristics (i.e. admission type, BMI, APACHE score etc.) and organizational factors (i.e. shift type, ICU level and day of ICU admission) of the observed patients. This enabled the analyses of the differences in the observed nursing time between different patient characteristics and
organizational factors. We calculated the BMI from the provided patients’ lengths and weights.
Data analyses
All statistical analyses on observed and self-reported data were performed in R statistical software, version 3.3.3 with the IDE RStudio, version 1.0 [23, 24]. Per research question the used methods are described below.
1. Data quality of the self-reported TISS and NAS scores
The self-reported TISS and NAS scores were validated to reveal whether nurses over- or underestimate their activity scores. Therefore, we compared and determined the association between these self-reported TISS and NAS scores (registered with the NICE workload registry
module) with the observed TISS and NAS scores (derived from the observed time during MMRs). The sum of the self-reported points and the derived observed points per patient were calculated.
Furthermore, the mean, median, standard deviation (SD), min and max of the self-reported and observed TISS and NAS scores per patient were calculated. The self-reported and observed points for each TISS and NAS nursing activity (some activities occur more than once during the shift) were summed resulting in the total self-reported and total observed scores per activity. In addition, the mean self-reported and mean observed score per activity was determined. Correlation between the total self-reported and total observed TISS and NAS scores per patient and per activity was assessed with the Pearson’s correlation test. A correlation coefficient <0.50 was considered as low correlation, between 0.50 and 0.70 was considered as moderate correlation and >0.70 was considered as high correlation [25]. The degree of agreement and the correlation between self-reported and observed scores per patient were visualized in scatterplots. A correlation coefficient of 1 corresponds to a perfect positive linear correlation and -1 corresponds to a perfect negative linear correlation. Furthermore, the difference between the self-reported and the observed number of patients per nursing activity was calculated in percentages, where negative percentages represent under-reporting.
2. Validation of TISS and NAS nursing workload models
To validate the reliability of the TISS and NAS models we compared the self-reported TISS and NAS nursing times derived from the self-reported TISS and NAS scores with the observed MMRs times. The sum, mean, median, standard deviation (SD), min and max of the derived self-reported times and the observed MMRs times per patient were calculated. Furthermore, the sum and mean of the self-reported and observed times per activity were calculated. Correlation between the self-reported and observed times per patient and per activity was assessed with the Pearson’s correlation test and the correlation per patient was visualized by scatterplots. Additionally, we calculated and compared the total self-reported time for all patients and all activities with the total observed time for all patients and all activities.
12
3. Association between nursing time and organizational factors
Analyses of the association between nursing time and the organizational factors were based on the observed MMRs nursing times. The influence on workload of the following organizational factors were analyzed: Type of shifts (day-, evening-, and nightshift), ICU levels (level 1, 2, or 3), and day of ICU admission (i.e. first ICU admission day to last ICU admission day). The Wilcoxon Signed-Rank test was used to determine whether there was a significant difference between these organizational factors and to analyze the differences between categories. For example, we determined the difference in nursing time during the day shift and night shift or the difference in nursing time for emergency surgery and elective surgery. A Wilcoxon Signed-Rank test p-value of <0.05 is considered as a significant correlation.
4. Association between nursing time and patients characteristics
Analyses of the association between nursing time and patients characteristics were based on the observed MMRs nursing times. The included patient characteristics were admission types (medical, emergency-, and elective surgery), severity of illness expressed as the APACHE IV mortality risk (<30, 30-70 and >70%), Body Mass Index (BMI) (<18.5, 18.5-25.0, and >25.0 referred to as respectively under-, normal-, and overweight), and age (18-64, 65-79, and >80 years old referred to as respectively adults, elderly, and very elderly). We used the Wilcoxon Signed-Rank test to assess whether there was a significant difference between these patients characteristics and to analyze the differences between categories. For example, we determined the difference in nursing time between underweight and overweight patients or the difference in nursing time between adults and elderly patients.
13
Results
Included data
During our study, 384 observations were made during MMRs on a total number of 287 different patients in six hospitals between October 2016 and September 2017. One patient could have been admitted to an ICU on different dates during different shifts and therefore could be followed during more than one MMR. Thus, one patient could correspond to more than one observation. The corresponding patient characteristics were available for 236 of the 287 (82%) observed patients (Figure 2). The patient characteristics of the 236 included patients with a total number of 321 observations are shown in Table 1. All following results are based on these patients.
Figure 2.Venn diagram on the included patients. A total of 236 patients have been included in this study.
There were 11 patients with missing BMI values due to missing lengths and/or weights. Therefore, during the analyses of the association between nursing time and BMI the total number of patients that could be included was 225 with 307 observations. A total of 27 patients had a missing APACHE IV mortality risk as not all patients meet the APACHE IV inclusion criteria [26], for example patients with an ICU stay shorter than four hours. Therefore, during the analyses of the association between the nursing time and severity of illness a total number of patients that could be included was 209 with 283 observations.
14 Table 1. Baseline characteristics on the 236 observed patients.
1 = 11 patients excluded due to missing BMI
2 = 27 patients excluded due to missing APACHE IV mortality risk
1. Data quality of the self-reported TISS and NAS scores
Table 2 shows the statistics and the correlation between the total observed and total self-reported TISS score, TISS time, NAS score and NAS time per patient.
Table 2. Correlation on self-reported and observed TISS-28 and NAS scores and times per patient.
Self-reported
Observed
Correlation coefficient
Mean ± SD (Median) Min – Max Mean ± SD (Median) Min – Max
TISS-28 score
(points)
27.9 ± 7.2 (26.0)
14.0 – 58.0 27.7 ± 9.8 (27.0)
0 – 50.0
0.26
NAS score
(points)
32.2 ± 15.8 (29.5) 8.6 – 101.3 37.4 ± 9.8 (37.7)
0 – 77.4
0.26
TISS-28 nursing
time (h)
4.9 ± 1.2 (4.5)
2.4 – 10.2
1.8 ± 1.1 (1.6)
0.0 – 8.8
0.23
NAS nursing
time (h)
2.5 ± 1.2 (2.3)
0.6 – 8.1
2.3 ± 1.2 (2.1)
0.0 – 9.0
0.37
Baseline characteristics
Number of MMR observations
321
Number of patients
236
Number of hospitals
6
Male (%)
148 (62.7)
Mean BMI (Min-Max)
126.8 (16.0 – 51.9)
Mean age in years (Min-Max)
64.8 (16 – 94)
Median APACHE IV mortality risk (IQR)
213.9 (2.6 – 50.2)
15 There was a low Pearson’s correlation (r=0.26) between the total self-reported and observed TISS scores per patient. As a further illustration Figure 3 presents a scatterplot of the self-reported and observed TISS scores per patient on a total of 236 patients with 321 observations. The self-reported TISS scores were not lower than 14 points, while the minimal observed TISS score was 0 points.
Figure 3. The correlation between the total self-reported TISS scores and the total observed TISS scores per
patient. n = 321 observations on 236 patients.
A moderate Pearson’s correlation (r=0.68) was found between the self-reported and observed TISS scores per nursing activity. The observed and self-reported TISS-28 scores and corresponding time per nursing activity are presented in Table 3. Appendix 1 shows the descriptions for the TISS-28 activities by item number.
16 Table 3. Self-reported and observed results per TISS-28 activity item. Results presented in TISS scores, times
and number of observations occurred per activity item number. Appendix 1 describes the nursing activities by item number. TISS-28 activity number Self-reported n Observed n Mean time assigned from points in minutes Mean time measured during MMRs in minutes Total time assigned from self-reported points in hours Total time measured during MMRs in hours Differ in n (%) 1 318 314 53.0 19.6 280.9 105.1 1.2 2 270 250 10.6 5.3 47.7 28.4 7.4 3 170 306 21.2 18.5 60.0 98.9 - 44.4 4 88 297 31.8 19.8 46.6 98.6 - 70.3 5 302 287 10.6 16.3 53.3 87.2 4.9 6 277 259 10.6 19.9 48.9 86.1 6.4 7 12 139 31.8 2.0 6.3 10.8 - 91.3 8 47 141 53.0 1.2 41.5 6.4 - 66.6 9 26 87 21.2 0.5 9.1 2.7 - 70.1 10 17 3 10.6 0.1 3.0 0.2 82.3 11 98 160 10.6 1.5 17.3 8.0 - 38.7 12 20 116 31.8 1.3 10.6 6.9 - 82.7 13 6 77 42.4 4.5 4.2 5.8 - 92.2 14 NA NA NA NA NA NA NA 15 285 167 53.0 0.8 251.7 4.4 41.4 16 32 1 84.8 0.0 45.2 0.0 96.8 17 13 68 21.2 0.7 4.5 3.8 - 80.8 18 4 0 31.8 0.0 2.1 0.0 100 19 33 35 31.8 2.6 17.4 14.0 - 5.7 20 252 272 21.2 1.6 89.0 8.8 - 7.3 21 NA NA NA NA NA NA NA 22 3 11 42.4 0.1 2.1 0.2 - 72.7 23 NA NA NA NA NA NA NA 24 6 25 31.8 0.2 3.1 1.1 - 76.0 25 164 159 21.2 1.6 57.9 8.7 3.0 26 319 125 31.8 0.7 169.0 3.9 60.8 27 319 75 53.0 2.7 281.7 3.4 76.4 28 33 20 53.0 1.1 29.1 6.1 39.3 Total 3114 3394 NA NA 1582.2 599.50 - 8.2 n = 321 observations on 236 patients.
17 Outstanding is the over-reporting for peripheral arterial catheters (item 15), single specific
interventions (item 26) and multiple specific interventions (item 27). The self-reported nursing time for mechanical ventilation (item 8) was high compared to observed nursing time, however, the self-reported number of patients was low compared to the observed number of patients within this activity. The self-reported time for hourly monitoring (item 1), urinary catheters (item 20), enteral feeding (item 25) were high compared to the observed nursing time for these TISS items. However, the difference did not result from over-reporting, because the self-reported number of patients was accurate compared to the observed number of patients within these activity items. Also outstanding was the result for single vasoactive medications (item 12) and multiple vasoactive medications (item 13). The observed and self-reported nursing time were accurate, but there was a relative large difference between the observed and self-reported number of patients.
A low Pearson’s correlation (r=0.26) was found between the self-reported and observed NAS scores per patient. Figure 4 presents the correlation between the total observed NAS scores per patient and the self-reported NAS scores per patient. The range (min-max) of the self-reported NAS points per patient (8.6 – 101.3) was higher and wider than the range of observed NAS scores per patient (0.0 – 77.6).
Figure 4. The correlation between total self-reported NAS scores and the total observed NAS scores per
patient. n = 321 observations on 236 patients.
18 A high Pearson’s correlation (r=0.85) was found between the self-reported NAS scores and observed NAS scores per activity item. Table 4 presents the self-reported and observed NAS scores and times for nursing activity items. Appendix 2 shows the description per nursing activity item number.
Table 4. Self-reported and observed results per NAS activity item. Results presented in NAS scores, times and
number of observations occurred per activity item number. Appendix 2 describes the nursing activities by item number. NAS activity number Self-reported n Observed n Mean time assigned from points in minutes Mean time measured during MMRs in minutes Total time assigned from self-reported points in hours Total time measured during MMRs in hours Differ in n (%) 1 321 314 25.3 22.6 139.2 120.2 2.1 1a 282 313 21.6 23.2 101.5 117.5 - 9.9 1b 29 1 58.0 164.8 28.0 2.7 96.5 1c 5 0 94.0 0.0 7.8 0 100 2 254 249 20.6 5.1 87.3 26.9 1.9 3 180 306 26.8 18.8 80.6 96.9 - 41.1 4 305 288 26.1 16.6 130.7 87.2 5.5 4a 274 288 19.6 18.8 89.8 87.2 - 4.8 4b 31 0 79.2 0.0 40.9 0.0 100 4c 0 0 0.0 0.0 0.0 0 0 5 12 91 8.6 1.3 1.7 7.0 - 86.8 6 46 268 44.2 7.0 33.4 37.7 - 82.8 6a 22 268 46.0 8.4 9.6 37.7 - 91.7 6b 24 0 59.5 0.0 23.8 0 100 6c 0 0 0.0 0.0 0.0 0 0 7 131 291 24.0 10.7 55.3 57.7 - 54.9 7a 125 291 19.2 11.8 40.0 57.7 - 57.0 7b 6 0 153.6 0.0 15.3 0 100 8 130 318 28.3 48.6 59.9 258.9 - 59.1 8a 120 306 20.1 48.7 40.3 229.1 - 60.7 8b 8 12 111.3 150.7 14.8 29.8 - 33.3 8c 2 0 144.0 0.0 4.8 0 100 9 25 217 6.7 2.9 2.8 15.9 - 88.4 10 54 80 8.6 0.7 7.2 3.7 - 32.5 11 98 31 21.1 0.2 34.4 1.3 68.3 12 31 118 5.7 1.3 2.5 6.9 - 73.7 13 NA NA NA NA NA NA NA 14 19 1 8.1 0.0 2.5 0.0 94.7
19 15 0 0 0.0 0.0 0.0 0.0 0 16 30 32 36.9 2.5 19.0 12.9 - 6.2 17 251 273 33.6 1.7 141.1 8.8 - 8.0 18 4 11 7.6 0.0 0.3 0.2 - 63.6 19 NA NA NA NA NA NA NA 20 7 25 13.4 0.2 1.3 1.1 - 72.0 21 163 161 6.2 1.7 17.0 8.7 1.2 22 23 4 13.4 0.0 4.9 0.1 82.6 23 33 19 9.1 1.2 5.0 6.1 42.4 Total 2117 3097 NA NA 826.1 758.2 - 31.6 n = 321 observations on 236 patients.
Outstanding was the under-reporting for administrative and managerial tasks (item 8), where the observed number of patients and observed nursing time was high compared to the self-reported number of patients and self-reported nursing time. The total and mean observed nursing time for the laboratory investigations (item 2), urinary catheters (item 17) and enteral feeding (item 21) was low compared to the self-reported nursing time. However, the observed number of patients was not low compared to the self-reported number of patients for these NAS items. For mobilization and
positioning (item 6) and support for relatives and patients (item 7) the self-reported nursing time was accurate compared to the observed nursing time. However, the observed number of patients within these activity items was high compared to the self-reported number of patients.
20
2. Validation of TISS and NAS nursing workload models
A low correlation (r=0.23) was found between the self-reported and observed TISS time per patient. Figure 5 shows a scatterplot on the total self-reported TISS time and the total observed TISS time per patient. Notable was that the self-reported TISS times were not lower than 2.5 hours of nursing care per patient. The mean self-reported TISS time per patient (4.9 hrs. ± 1.2) was significantly higher than the observed TISS time per patient (1.8 hrs. ± 1.1). The minimum and maximum self-reported TISS times per patient ranged from 2.4 – 10.2, while the range of the observed TISS time per patient was 0.0 – 8.8.
Figure 5. The correlation between the total self-reported TISS time in hours and the total observed TISS time in
hours per patient. n = 321 observations on 236 patients.
A low correlation (r=0.27) was found between the self-reported time and observed TISS time per activity and the TISS-28 described 37.8% of the total observed nursing time. Also a low correlation (r=0.37) was found between total reported and observed NAS time per patient. The mean self-reported NAS time per patient (2.5 hrs. ± 1.2) was comparable to the mean observed NAS time per patient (2.3 hrs. ± 1.2). Also, the minimum and maximum self-reported NAS time per patient (0.6 – 8.1) was comparable to the minimum and maximum observed NAS time per patient (0.0 – 9.0). Figure 6 shows the correlation between the self-reported and observed NAS time on a patient level.
21 Figure 6. The correlation between the total self-reported NAS time in hours and the total observed NAS time in
hours per patient. n = 321 observations on 236 patients.
The Pearson’s correlation test found a moderate correlation (r=0.50) between the total self-reported NAS time per activity item and the total observed NAS time per activity item. The NAS model
22
3. Association between nursing time and organizational factors
The total observed nursing time (hrs.) per patient, by shift type is presented in Table 5. The mean, SD and median of the total time per patient is presented for the three shift types. Wilcoxon Signed-Rank test showed significant difference between the day- and night shift (p<0.001). The mean observed time during the day shift spend on one patient (2.5 hrs. ± 1.2) was significantly higher than the mean observed time during the night shift (2.0 hrs. ± 1.1). A significant difference was found between the evening- and night shift (p<0.01), where the mean observed time during the evening shift (2.4 hrs. ± 1.4) was significantly higher compared to the mean time observed during the night shift (2.0 hrs. ± 1.1). There were no significant differences found in observed time between the day- and evening shift.
Table 5. Total observed nursing time in hours per patient, by shift type.
Shift type n
Observed time (h)
(Mean ± SD (Median)) Min – Max (h)
Day shift 119 2.5 ± 1.2 (2.4) 0.0 – 6.1
Evening shift 117 2.4 ± 1.4 (2.2) 0.0 – 9.0
Night shift 85 2.0 ± 1.1 (1.8) 0.6 – 7.1
n = 321 observations on 236 patients.
Table 6 presents the mean, SD and median of the total observed nursing time per patient per ICU level. Wilcoxon Signed-Rank test showed no significant differences in observed nursing time between the ICU levels.
Table 6. Total observed nursing time in hours per patient, by the NVIC ICU level.
ICU level (NVIC level) n
Observed time (h)
(Mean ± SD (Median)) Min – Max (h)
1 33 2.6 ± 1.1 (2.2) 0.8 – 4.4
2 284 2.3 ± 1.3 (2.0) 0.0 – 9.0
3 4 3.2 ± 2.0 (2.8) 1.2 – 5.8
n = 321 observations on 236 patients.
Analyses showed there was no significant difference in observed nursing time between patients with different days of admission, where day of admission represents the number of days that a patient has been admitted to the ICU at the moment of measurement.
23
4. Association between nursing time and patients characteristics
Observed nursing time per patient is presented by admission type in Table 7. Wilcoxon Signed-Rank test showed that patients with a medical admission type (2.5 hrs. ± 1.3) have a significantly higher (p<0.001) observed nursing time compared to patients with an elective surgery admission type (2.0 hrs. ± 1.1). A significant difference was determined between emergency- and elective surgery (p<0.05), where the mean observed time for emergency surgery (2.4 hrs. ± 1.3) was significantly higher than the mean time observed for patients with an elective surgery admission type (2.0 hrs. ± 1.1). No significant difference in nursing time was found between medical- and emergency surgery.
Table 7. Total observed nursing time in hours per patient, by the patient’s admission type.
Admission type n Observed time (h) (Mean ± SD (Median)) Min – Max (h) Medical 151 2.5 ± 1.3 (2.4) 0.0 – 9.0 Emergency surgical 65 2.4 ± 1.3 (2.1) 0.5 – 6.1 Elective surgical 99 2.0 ± 1.1 (1.7) 0.0 – 8.1
n = 315; 6 observations missing due to missing admission types.
The observed nursing time per patient is presented by APACHE IV mortality risk groups in Table 8. Wilcoxon Signed-Rank test showed a significant difference between observed nursing times between APACHE IV mortality risk groups. A significant difference (p<0.05) was determined between the low and medium group, where the mean observed nursing time for the low group (2.2 hrs. ± 1.2) was significantly lower than the mean observed nursing time for the medium group (2.5 hrs. ± 1.3). Wilcoxon Signed-Rank test showed that patients in the high risk group (2.7 hrs. ± 1.5) had a significantly higher (p<0.01) observed nursing time compared to the low risk group (2.2 hrs. ± 1.2). No significant difference was found between the medium and high group.
Table 8. Total observed nursing time in hours per patient, by APACHE IV mortality risk.
APACHE IV mortality risk (%) n
Observed time (h) (Mean ± SD (Median)) Min – Max (h) Low (< 30%) 145 2.2 ± 1.2 (1.9) 0.0 – 8.1 Medium (30 – 70%) 93 2.5 ± 1.3 (2.3) 0.0 – 7.1 High (> 70%) 45 2.7 ± 1.5 (2.5) 0.0 – 9.0
n = 283 observations on 209 patients; 38 observations missing due missing APACHE IV mortality risk for 27 patients.
Table 9 presents the observed nursing time per patient by BMI group. The Wilcoxon Signed-Rank test showed no significant differences in observed nursing time between the under- and normal weight group, between the normal- and overweight group or between the under- and overweight.
24 Table 9. Total observed nursing time in hours per patient, by BMI group.
BMI group n Observed time (h) (Mean ± SD (Median)) Min – Max (h) Underweight (< 18.5) 4 2.4 ± 1.1 (2.4) 1.0 – 3.7 Normal weight (18.5 – 25.0) 130 2.4 ± 1.4 (2.2) 0.0 – 9.0 Overweight (> 25.0) 173 2.3 ± 1.2 (2.1) 0.0 – 8.1
n = 307 observations on 225 patients; 14 observations missing due to missing BMI on 11 patients.
Table 10 shows the observed nursing time per patient by age group. The Wilcoxon Signed-Rank test showed that the adults (2.5 hrs. ± 1.4) had a significantly higher (p<0.01) observed nursing time than patients in the elderly group (2.1 hrs. ± 1.1). There were no significant difference found between elderly and very elderly or between adults and very elderly.
Table 10. Total observed nursing time in hours per patient, by age group.
Age (years) n Observed time (h) (Mean ± SD (Median)) Min – Max (h) Adults (18 – 64) 150 2.5 ± 1.4 (2.2) 0.0 – 9.0 Elderly (65 – 79) 121 2.1 ± 1.1 (1.9) 0.0 – 6.1 Very elderly (> 80) 50 2.2 ± 1.1 (2.1) 0.0 – 5.3 n = 321 observations on 236 patients.
25
Discussion
We answered four research questions to evaluate the validity and usefulness of nursing workload registration in the Netherlands.
1. Data quality of the self-reported TISS and NAS scores
In general we can conclude that the quality of self-reported data can be improved as there is only a low correlation between the self-reported and observed TISS and NAS scores at a patient level. Furthermore, we can conclude that the quality of self-reported NAS scores on a nursing activity level is acceptable and the self-reported TISS scores on a nursing activity level could be increased. To improve the quality of self-reported TISS scores there should be a time minimum attached to the low point TISS activities to increase the quality of these self-reported scores. This minimum amount of time should be clearly indicated, while this notification could help nurses estimate when to select an activity. Another solution could be a short tutorial or training for nurses to improve their knowledge about workload registration models.
2. Validation of TISS and NAS nursing workload models
The low correlation between the self-reported and observed TISS and NAS time per patient implicates that the TISS and NAS points assigned to patients do not correspond to the observed nursing time for these patients. The high mean of self-reported TISS time per patient indicates that the TISS-28 overestimates the nursing time and is not able to provide an accurate estimation of nursing time per patient. This is most probably because one TISS point corresponds to 10.6 minutes of nursing time, which is a relative large amount of (minimum) time and results in a low flexibility in time assigned to an activity. The low correlation between the self-reported and the observed NAS time per patient implicates that the NAS is not able is accurately estimate the nursing time on a patient level. The low correlation between the total self-reported and observed TISS time per nursing activity item indicates that the TISS-28 is not able to accurately estimate the time spent per TISS activity item. The moderate correlation between the total self-reported and observed NAS time per nursing activity item implicates that the NAS model is able to more accurately estimate the nursing time spent per activity item. In general, all self-reported TISS scores and TISS and NAS times were overestimated. Only, the mean self-reported NAS score per patient was underestimated. The overestimations are found within all observed (six) hospitals and we therefore expect that these would also occur in other hospitals. However, the overestimations could be the result of nurses overestimating the time spent during their shift, but this could also be the result of the bad estimations of the TISS and NAS models. To improve the TISS-28 model, the amount of time represented by one TISS point should be decreased and the maximum amount of TISS points assigned to one activity should be increased. The NAS model could be improved by adjusting the amount of points assigned to items, especially subitems. These subitems require nurses to estimate time durations and this could result in human errors. Additionally, there are relative large differences in the points attached to these subitems, which have an relative large impact on the nursing time estimations.
26
3. Association between nursing time and organizational factors
The significant difference in nursing time between the day- and night shift and between the evening- and night shift could partly be explained by the fact that most patients require less nursing activities during the nightshift, which decreases the number of nursing activities and therefore the workload. No significant differences were found between nursing time for different ICU levels or the days of admission. The observed nursing time for ICU level 3 is high compared to the observed nursing time for ICU level 1. However, there are only 4 patients included in the level 3 ICU so probably this study lacks the power to detect a significant difference.
4. Association between nursing time and patients characteristics
Mean observed time for patients with a medical admission type are significantly higher when compared to the elective surgery admission type. Also, a significant difference is found between the admission types emergency surgery and elective surgery, where the observed time for patients with emergency surgery is significantly higher than the elective surgery. This could be explained by the fact that medical patients require more medication, more complex and more time-consuming treatments compared to elective surgery patients. The observed time for patients in the high
APACHE IV mortality risk group is significantly higher than for patients in the low APACHE IV mortality risk group and also, observed time for patients in the medium APACHE IV risk group is significantly higher than for patients in the low risk group. This difference could be explained because high risk patients require more frequent or more complex nursing care. Also, an explanation could be that medical patients are often in the high risk group and the elective surgery patients are often in the low risk group. The mean observed time for patients in the adults group is significantly higher than patients in the elderly group. This difference could be explained by the fact that no new complex or intensive treatments will be started in older patients, while adults are more applicable for more complex and time-consuming treatments.
Relation to other studies
We found a study with a lower mean TISS score when compared to our TISS-28 scores [27]. However, we also found studies with comparable mean TISS-28 scores per patient [28, 29] and the mean NAS scores per patient [30]. The NAS was elaborated in 2003 by Miranda et al [3]. It is derived from the TISS-28 and aims at the description of nursing activities that are not necessarily correlated to the severity of illness. Miranda concluded that NAS describes 81% of the nursing time, compared to 43% of TISS-28 [3, 4]. Our results are comparable to these results. During our study, the NAS explained 91% of the total observed nursing time, compared to 37% of the TISS-28. A strength of our study are the accurate time measurements for nursing activities by MMRs which are more accurate compared to work-sampling [16]. All provided TISS and NAS scores were derived from the provided NICE scores. The NICE data dictionary does not contain exactly the same categories as the original TISS and NAS models. However, the differences are minimal and we believe that these do not affect the derived TISS and NAS scores. We excluded 3 items (intravenous fluid (item 14), active diuresis (item 21) and treatment of complicated metabolic acidosis/alkalosis (item 23)) during the validation of the TISS-28. Item 14 and 21 are measurements of fluid volume which is not measured during the MMRs. Item 23 is outdated and is not performed in Dutch ICUs anymore and thus is not measured during the MMRs. During the validation of the NAS we excluded 2 items (intravenous replacement of large fluid losses
27 (item 13) and treatment of complicated metabolic acidosis/alkalosis (item 19)) which are outdated and are not performed in Dutch ICUs anymore and thus where not measured during the MMRs. The shifts recorded during MMRs were 8-hour shifts, however, the NAS is based on 24-hour shifts. We therefore had to adjust the 14.4 minutes per NAS point to 4.8 minutes per NAS point. In addition, although compared to other studies our study sample size was relatively large, it is still rather small (n=321) which might cause lack of power to detect differences in subgroups. Additional research towards the merging of the TISS-28 and the NAS model is suggested. The models could be partly combined and this could possibly improve the estimation of nursing workload. Our results on observed nursing time per patient or per activity item could be taken into account for scoring weights and points for this new model.
Conclusion
TISS-28 is not able to reliable estimate the nursing time per patient or per nursing activity. The NAS is more accurate in estimating the mean nursing time per patient and per nursing activity in
comparison to TISS-28. However, the TISS-28 and NAS could be significantly improved by adjusting the number of points assigned to activity items and providing training for the use of workload models. The average nursing time spent per patient is associated with the organizational factor shift type and the patients characteristics admission type, severity of illness and age. Further research should be performed on the development of a more accurate and reliable model to estimate the actual nursing time spent per patient and per activity item.
28
References
1. Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, Fumagalli R. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs. 2014 Jun;30(3):152-8.
2. Lachance J, Douville F, Dallaire C, Padilha KG, Gallani MC. The use of the Nursing Activities Score in clinical settings: an integrative review. Rev Esc Enferm USP. 2015 Feb;49 Spec No:147-56. doi: 10.1590/S0080-623420150000700021.
3. Miranda DR, Nap R, de Rijk A, Schaufeli W, Iapichino G, & Members of the TISS Working Group. Nursing activities score. Critical care medicine,2003 Feb;31(2), 374-382.
4. Guccione A, Morena A, Pezzi A, Iapichino G. The assessment of nursing workload. Minerva Anestesiol. 2004 May;70(5):411-6.
5. Nogueira Lde S, Koike KM, Sardinha DS, Padilha KG, de Sousa RM. Nursing workload in public and private intensive care units. Rev Bras Ter Intensiva. 2013 Jul-Sep;25(3):225-32 6. Gój K, Knapik P, Kucewicz-Czech E, Luboń D. [The TISS-28 scoring system for assessment
of cardiac surgical postoperative intensive care]. Anestezjol Intens Ter. 2009 Jan-Mar;41(1):37-40.
7. Kiekkas P, Sakellaropoulos GC, Brokalaki H, Manolis E, Samios A, Skartsani C, Baltopoulos GI. Association between nursing workload and mortality of intensive care unit patients. J Nurs Scholarsh. 2008;40(4):385-90.
8. Serafim CTR, Dell'Acqua MCQ, Castro MCNE, Spiri WC, Nunes HRC. Severity and workload related to adverse events in the ICU. Rev Bras Enferm. 2017 Sep-Oct;70(5):942-948. doi: 10.1590/0034-7167-2016-0427.
9. Faisy C, Davagnar C, Ladiray D, Djadi-Prat J, Esvan M, Lenain E, Durieux P, Leforestier JF, Marlet C, Seijo M, Guillou A. Nurse workload and inexperienced medical staff members are associated with seasonal peaks in severe adverse events in the adult
medical intensive care unit: A seven-year prospective study. Int J Nurs Stud. 2016 Oct;62:60-70. doi: 10.1016/j.ijnurstu.2016.07.013. Epub 2016 Jul 16.
10. Cullen DJ, Civetta JM, Briggs BA, & Ferrara LC. Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Critical care medicine, 1974 Mar-Apr;2(2), 57-60.
11. Nogueira Lde S, Domingues Cde A, Poggetti RS, de Sousa RM. Nursing workload in intensive care unit trauma patients: analysis of associated factors. PLoS One. 2014 Nov
6;9(11):e112125.
12. Campagner AO, Garcia PC, Piva JP. [Use of scores to calculate the nursing workload in a pediatric intensive care unit]. Rev Bras Ter Intensiva. 2014 Jan-Mar;26(1):36-43.
13. Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, Fumagalli R. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs. 2014 Jun;30(3):152-8. doi: 10.1016/j.iccn.2013.10.004. Epub 2013 Dec 25.
14. Altafin JA, Grion CM, Tanita MT, Festti J, Cardoso LT, Veiga CF, Kamiji D, Barbosa ÁR, Matsubara CC, Lara AB, Lopes CC, Blum D, Matsuo T. Nursing Activities Score and workload in the intensive care unit of a university hospital. Rev Bras Ter Intensiva. 2014
Jul-Sep;26(3):292-8.
15. Padilha KG, de Sousa RM, Garcia PC, Bento ST, Finardi EM, Hatarashi RH. Nursing workload and staff allocation in an intensive care unit: a pilot study according to Nursing Activities Score (NAS). Intensive Crit Care Nurs. 2010 Apr;26(2):108-13.
29 16. Urden, LD., & Roode, JL. (1997). Work sampling: a decision-making tool for determining
resources and work redesign.Journal of Nursing Administration,27(9), 34-41. 17. Finkler SA, Knickman JR, Hendrickson G, Lipkin Jr M, & Thompson WG. (1993). A
comparison of work-sampling and time-and-motion techniques for studies in health services research. Health services research, 1993 Dec;28(5), 577.
18. Dutch National Intensive Care Evaluation (NICE) Foundation (2017) – Available from:
https://www.stichting-nice.nl/
19. Peng L, Mayner L, Wang H. Association between trauma patients' severity and critical care nursing workload in China. Nurs Health Sci. 2014 Dec;16(4):528-33.
20. Cremasco MF, Wenzel F, Zanei SS, Whitaker IY. Pressure ulcers in the intensive care unit: the relationship between nursing workload, illness severity and pressure ulcer risk. J Clin Nurs. 2013 Aug;22(15-16):2183-91.
21. Sousa CR, Gonçalves LA, Toffoleto MC, Leão K, Padilha KG. Predictors of nursing workload in elderly patients admitted to intensive care units. Rev Lat Am Enfermagem. 2008 Mar-Apr;16(2):218-23.
22. Miranda DR, de Rijk A, Schaufeli W. Simplified Therapeutic Intervention Scoring System: the TISS-28 items--results from a multicenter study. Crit Care Med. 1996 Jan;24(1):64-73. 23. R-project.org – The R Foundation - Available from: https://www.r-project.org/
24. Rstudio.com – Integrated Development Environment (IDE) RStudio – Available from:
https://www.rstudio.com/
25. Tilburg University – SPSS Helpdesk, correlations.
https://www.tilburguniversity.edu/nl/studenten/studie/colleges/spsshelpdesk/edesk/correlat/
26. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006 May;34(5):1297-310.
27. Padilha KG, Sousa RM. Nursing workload in intensive care units: a study using the Therapeutic Intervention Scoring System-28 (TISS-28). Intensive Crit Care Nurs. 2007 Jun;23(3):162-9. Epub 2007 Feb 27.
28. Lefering R, Zart M, Neugebauer EA. Retrospective evaluation of the simplified Therapeutic Intervention Scoring System (TISS-28) in a surgical intensive care unit. Intensive Care Med. 2000 Dec;26(12):1794-802.
29. Castillo-Lorente E, Rivera-Fernandez R, Rodriguez-Elvira M, Vazquez-Mata G. TISS-76 and TISS-28: correlation of two therapeutic activity indices on a Spanish multicenter ICU database. Intensive Care Med. 2000 Jan;26(1):57-61.
30. Bernat Adell A, Abizanda Campos R. Nursing Activity Score (NAS). Our experience with a nursing load calculation system based on times. Enferm Intensiva. 2005 Oct-Dec;16(4):164-73.
30
Appendix 1. Descriptions of TISS-28 nursing activities by item
number
1. Standard monitoring. Hourly vital signs, regular registration and calculation of fluid balance. (5 pts) 2. Laboratory. Biochemical and microbiological investigations. (1 pt)
3. Single medication, any route. (IV, PO, IM, etc.) (2 pts)
4. Multiple intravenous medications. (more than 1 drug, single shots, or continuously) (3 pts) 5. Routine dressing changes. Care and prevention of decubitus and daily dressing change. (1 pt)
6. Frequent dressing changes (at least one time per each nursing shift) and/or extensive wound care. (1 pt) 7. Care of drains. All (except gastric tube) (3 pts)
8. Mechanical ventilation. Any form of mechanical or assisted ventilation with or without PEEP; with or without muscle relaxants; spontaneous breathing with PEEP. (5 pts)
9. Supplementary ventilatory support. Breathing spontaneously through endotracheal tube without PEEP; supplementary oxygen by any method except if mechanical ventilation parameters apply. (2 pts) 10. Care of artificial airways. Endotracheal tube or tracheostoma. (1 pt)
11. Treatment for improving lung function. Thorax physiotherapy, incentive spirometry, inhalation therapy, intratracheal suctioning. (1 pt)
12. Single vasoactive medication. Any vasoactive drug. (3 pts)
13. Multiple vasoactive medications. More than 1 vasoactive drug, disregard type and dose. (4 pts) 14. Intravenous replacement of large fluid losses. Fluid replacement > 3 liters per square meter per day,
disregard type of fluid administered. (4 pts) 15. Peripheral arterial catheter. (5 pts)
16. Left atrium monitoring. Pulmonary artery flotation catheter with or without cardiac output measurement. (8 pts)
17. Central venous line. (2 pts)
18. Cardiopulmonary resuscitation after arrest in the past 24 hours. (single precordial percussion not included) (3 pts)
19. Hemofiltration techniques. Dialytic techniques. (3 pts) 20. Quantitative urine output measurement. (2 pts)
21. Active diuresis (e.g. furosemid > 0.5 mg/kg/day for overload). (3 pts) 22. Measurement of intracranial pressure. (4 pts)
23. Treatment of complicated metabolic acidosis/alkalosis. (4 pts) 24. Intravenous hyperalimentation. (3 pts)
25. Enteral feeding. Through gastric tube or other GI route (e.g. jejunostomy) (2 pts)
26. Single specific interventions in the ICU. Naso or orotracheal intubation, introduction of a pacemaker, cardioversion, endoscopies, emergency surgery in the past 24 hours, gastric lavage. Routine interventions without consequences to the clinical condition of the patient, such as radiographs, echography, EKG, dressings or introduction of venous or arterial catheters, are not included. (3 pts)
27. Multiple specific interventions in the ICU. More than one, as described above. (5 pts) 28. Specific interventions outside of ICU. Surgery or diagnostic procedures. (5 pts)
