0
The Association between Nursing Workload and
Hospital Mortality in Dutch Intensive Care Units
Safira Anatecia Wortel
Medical Informatics
Master Thesis
1
The Association between Nursing Workload and
hospital mortality in Dutch Intensive Care Units
Student
Name: Safira Anatecia Wortel Student number: 10531211 E-Mail: s.a.wortel@amc.uva.nl
Location
Department of Medical Informatics at the Academic Medical Center (AMC) Address: Meibergdreef 9, 1105 AZ Amsterdam Zuidoost
Contact information of mentors
Sylvia Brinkman, PhD E-mail: s.brinkman@amc.uva.nl Room: J1B-113-1 Phone: 65852 Charlotte Margadant, MSc. E-mail: c.c.margadant@amc.uva.nl Room: J1B - 124 Phone: 65362
Contact information of Tutor
Prof. Nicolette de Keizer, PhD E-mail: n.f.keizer@amc.uva.nl
Room: J1B -115 Phone: 65205
Time period
2
Acknowledgements
I would like to thank my mentors Sylvia Brinkman and Charlotte Margadant for their supervision, enthusiasm and support during my research project. I would also like to thank my tutor, Nicolette de Keizer, for her encouragement and insightful and constructive feedback on my work. Furthermore, I would like to thank the participants of this project for their time and input.
Last, but not least, I would like to thank my family for their unconditional support and patience during my studies in the Netherlands. It took a village and I am forever grateful to all of you. Thank you.
3
Table of Contents
Summary ... 3
Samenvatting ... 4
Chapter 1 General Introduction ... 6
1.1 Background on the TISS and NAS Scoring Systems ... 7
Chapter 2 The Association between nursing workload and hospital mortality in Dutch Intensive Care Units ... 8 Abstract ... 8 2.1 Introduction ... 9 2.2 Methods ... 9 2.3 Results ... 12 2.4 Discussion ... 19 2.5 Conclusion ... 20
Chapter 3 Nursing Workload in Practice ... 21
3.1 NICE registry’s nursing workload module ... 21
3.2 Method ... 21
3.3 Results ... 22
3.4 Suggestions for the NICE ... 23
3.5 Discussion ... 23
3.6 Conclusion ... 24
Chapter 4 General Discussion ... 25
Conclusions ... 26
References ... 27
List of Abbreviations ... 29
Appendix A Therapeutic Intervention Scoring System (TISS-28) items and points... 30
Appendix B Nursing Activities Score items and weights ... 31
Appendix C NNRs as covariates ... 32
Appendix D Content of the NICE nursing workload reports ... 33
4
Summary
Introduction: Various studies report an association between nursing workload (NW), expressed as
Patient: Nurse Ratios (PNRs), and the risk of mortality. It is unclear what the association is in Dutch ICUs and whether the use of Nursing Activities scores (NAS) instead of raw patient numbers influences that association. In the Netherlands, the Dutch National Intensive Care Evaluation (NICE) collects ICU data from all Dutch ICUs. ICUs, however, are not obligated to participate in their NW module. Prospective participants mentioned a fear of registration burden for their nursing staff as a reason for not participating.
Objectives: (1) to determine the best way to summarize Nurse: NAS Ratios (NNRs), (2) to provide
insight into the association between NW, expressed as PNRs and NNRs, and hospital mortality and (3) to determine if current participants of the NW module experience registration burden and to determine if they have advice for improving the module.
Methods: Patients admitted between the first of January 2016 and the first of July 2017 were
included. The data was divided into three subgroups: patients with a Length of Stay (LOS) of at least 1, 4 or 7 days. For all three subgroups several logistic regression models were developed in which the NNRs were included in different ways to predict mortality (for example NNR on day 1, delta NNR on day 1 and NNR day 4, and NNR at discharge). The Area Under receiver operating curve (AUC) and Brier scores were calculated to determine the models’ performance. The association of the PNRs and NNRs with hospital mortality was analyzed by using eight separate logistic regression models of which four models adjusted for comorbidities, age and admission type. Intensivists of three ICUs were interviewed on their experiences with the NW module. Interviews were recorded and summarized.
Results: The performance of the NNR model corrected for comorbidities, age and admission type
was good in all three LOS subgroups. AUC scores with 95% CI of NNR on day 1 as covariate: 0.84 (0.83 – 0.85); 0.71 (0.69 – 0.73) and 0.70 (0.67 – 0.73). As the performance of the models did not differ between various summarizations of NNRs, the four covariates used further were PNR and NNR on day 1 and mean PNR and NNR. A significant association was found between PNR on day 1 and hospital mortality (PNR range: 0.55 – 0.79; OR 1.02 (95% CI: 1.004 – 1.036)), but after case-mix correction, the association became non –significant. The association between NNR on day 1 and hospital mortality showed significant results between respectively the second (NNR range: 69.28 - 106.49) and third quartile (NNR: 106.49 - 140.42). After case-mix correction the results remained significant (ORs 1.014 (1.002 – 1.026) and 1.015 (1.002 – 1.028)). When using mean NNR there was a significant association found for quartile 2 (NNR range: 69.56 - 106.2; OR 1.019 (1.006 – 1.033)) and 3 (NNR range: 106.2 - 137.17; OR 1.026 (1.012 – 1.41)) without case-mix correction and only for quartile 3 after case-mix correction (OR 1.017 (1.004 – 1.03)). Of the three participants interviewed, only one actively uses and discusses the NW feedback reports. Other participants mentioned that more clarity is needed on the interpretation of the NW scores in order for them to actively use it. All participants stated that there was little registration burden for their nursing staff.
Conclusion: Performance of the logistic regression models is not dependent on the way NNRs are
summarized in the models. In Dutch ICUs, the association between PNRs and hospital mortality is not significant after case-mix correction. However, an increase in NNR seams to lead to a significantly higher risk of hospital mortality, though this was not found for the fourth quartile of the NNR.
5
Samenvatting
Introductie: Verschillende studies rapporteren een associatie tussen zorgzwaarte (ZZ), uitgedrukt als
Patiënt: Verpleegkundige Ratio’s (PVR's), en het risico op sterfte. Het is onduidelijk wat die associatie is in Nederlandse IC's en of het gebruik van “Nursing Activities Score” (NAS) in plaats van ruwe patiënt aantallen deze associatie beïnvloedt. De Nationale Intensive Care Evaluatie (NICE) verzamelt gegevens van alle Nederlandse IC's. De IC’s zijn echter niet verplicht deel te nemen aan de ZZ -module. Potentiële deelnemers noemden een angst voor registratielast voor hun verpleegkundigen als reden om niet deel te nemen.
Doelstellingen: (1) de beste manier vaststellen om Verpleegkundige: NAS Ratio’s (VNR's) samen te
vatten, (2) inzicht krijgen in de associatie tussen ZZ, uitgedrukt als PVR's en VNR's en ziekenhuissterfte en (3) achterhalen of huidige deelnemers van de ZZ-module registratielast ervaren en achterhalen of zij advies hebben om de module te verbeteren.
Methode: Patiënten die tussen 1 januari 2016 en 1 juli 2017 zijn opgenomen, zijn geïncludeerd. De
dataset werd verdeeld in drie subgroepen: patiënten met een ligduur van minimaal 1, 4 of 7 dagen. In alle subgroepen werden logistische regressiemodellen ontwikkeld waarin de VNR's op verschillende manieren werden opgenomen om de mortaliteit te voorspellen (bijvoorbeeld VNR op dag 1, verschil tussen VNR op dag 1 en dag 4 en VNR bij ontslag). De “Area Under receiver-operating curve” (AUC) en Brier-scores werden berekend om de prestaties van de modellen te bepalen. De associatie tussen PVR's en VNR's en ziekenhuissterfte werd geanalyseerd met behulp van acht afzonderlijke logistische regressiemodellen, waarvan vier waren gecorrigeerd voor comorbiditeit, leeftijd en opnametype. Intensivisten van drie IC's werden geïnterviewd over hun ervaringen met de ZZ-module. Interviews werden opgenomen en samengevat.
Resultaten: De prestatie van VNR-model gecorrigeerd voor comorbiditeit, leeftijd en opnametype
was goed in alle drie subgroepen. AUC-scores met 95% CI voor VNR op dag 1 als covariaat: 0.84 (0.83 – 0.85); 0.71 (0.69 – 0.73) en 0.70 (0.67 - 0.73). Omdat de prestaties van de modellen niet verschilden tussen de diverse samenvattingen van VNR's, waren de vier gebruikte covariaten: PVR en VNR op dag 1 en gemiddelde PVR en VNR. Er werd een significante associatie gevonden tussen PVR op dag 1 en ziekenhuissterfte (PVR tussen: 0,55 - 0,79; OR 1,02 (95% CI: 1,004 - 1,036)), maar na case-mixcorrectie werd die associatie niet-significant. De associatie tussen VNR op dag 1 en ziekenhuissterfte toonde significante resultaten in de tweede (VNR tussen: 69.28 - 106.49) en derde kwartiel (VNR: 106.49 - 140.42). Na case-mix-correctie bleven de resultaten significant (OR's 1.014 (1.002 - 1.026) en 1.015 (1.002 - 1.028)). Bij gemiddelde VNR was er een significante associatie gevonden in kwartiel 2 (VNR tussen: 69.56 - 106.2; OR 1.019 (1.006 - 1.033)) en 3 (VNR tussen: 106.2 - 137.17; OR 1.026 (1.012 - 1.41)) zonder mixcorrectie, maar alleen in kwartiel 3 na case-mixcorrectie (OR 1.017 (1.004 - 1.03)). Van de drie geïnterviewde deelnemers maakt slechts één actief gebruik van en bespreekt de ZZ-rapporten. Andere deelnemers gaven aan dat er meer duidelijkheid nodig is over de interpretatie van de ZZ-scores alvorens ze deze actief kunnen gebruiken. Alle deelnemers verklaarden dat hun personeel weinig registratielast ondervonden.
Conclusie: De prestatie van de logistische regressiemodellen is niet afhankelijk van de manier waarop
VNRs in de modellen worden gebruikt. In Nederlandse IC's is de associatie tussen PNR's en ziekenhuissterfte na case-mixcorrectie niet significant. Een toename in VNR leidde tot een significant hogere ziekenhuissterfte, hoewel dit niet werd gevonden in het vierde kwartiel van de VNR.
6
Chapter 1 General Introduction
High costs of staff, complex procedures, and the use of expensive medical devices, equipment, and infrastructure are some of the reasons why intensive care units (ICUs) are more expensive than other hospital departments [1]. Therefore, ICUs have a substantial impact on hospital budgets and reimbursement schemes. The costs for nursing staff alone attributes to 50% of total ICU costs [2]. However, offering nursing care at low costs by reducing the number of nurses can lead to higher nursing workload while the patients’ need for nursing care is not met. This can in turn increase the risk of mortality or complications at the ICU [3]. Patient acuity, the measure of need for nursing care and therewith the nursing workload, can give ICUs better insights into patient care need and nursing allocation. From the literature, two well-known methods to measure patient acuity and nursing workload are the Therapeutic Intervention Scoring System (TISS) and the Nursing Activities Score (NAS) [2,4]. Both patient acuity models give insight in the needed nursing care at an ICU. This information in combination with the number of FTE nurses present, can give insight into the average nursing workload. Research showed that the NAS is more effective in measuring the actual nursing workload compared to the TISS [2].
The Dutch ICU quality standard, developed by the Dutch Society of Intensive Care Medicine and the National Healthcare Institute, containing ICU guidelines and requirements, mentions that there is no clear evidence of a positive relationship between the formation of ICU nurses and the incidence of mortality [5,6]. Some studies [7–10] did find a significant positive relationship between ICU nurse formation and patient mortality, while other studies did not [11–14]. In the Netherlands, the Dutch National Intensive Care Evaluation (NICE) registry contains demographic, physiological and clinical data of all patients admitted to a Dutch ICU [15]. The data from the NICE registry is used by participants to monitor and improve quality of care based on benchmark information and to conduct clinical research. Since 2016, ICUs might voluntarily participate in the NICE nursing workload module to collect data by which TISS and NAS scores can be derived in order to measure patient acuity and nursing workload. The nursing workload module of the NICE includes all variables to calculate the NAS and TISS scores per nursing shift and contains seven additional variables which are relevant to the Dutch situation. These variables comprise of new medical interventions which were not performed at the time the TISS and NAS were developed [16]. There are 15 hospitals providing their nursing workload data recorded per eight-hour shift. Each hospital then receives a report from the NICE in which, the TISS and NAS scores are reported and compared to the national medians. This enables ICUs to compare their own nursing workload to other ICUs.
The aim of this research project is threefold:
1. To determine the influence of different ways of summarizing NNRs in logistic regression models on the performance of these models to predict hospital mortality.
2. To determine the association between the nursing workload, expressed as either Patient: Nurse Ratios (PNRs) or Nurse: NAS Ratios (NNRs), and hospital mortality.
3. To determine if current participants of the NW module experience registration burden and to determine if they have advice for improving the module.
The first two research objectives are presented as a scientific article (see chapter 2), while the third objective is presented in Chapter 3 Nursing Workload in Practice.
7
1.1 Background on the TISS and NAS Scoring Systems
The Therapeutic Intervention Scoring System (TISS)
The TISS has been designed in 1974 to classify nursing workload in relation to the severity of illness of patients in ICUs [4]. This system originally consisted of 57 therapeutic interventions which each could assign a score from one to four. These assigned scores per intervention are established by experienced specialists and are based on the complexity and severity of the intervention, where a higher score represents a higher severity of illness [4]. Since its origination the TISS is used for a variety of reasons, namely to: (1) determine appropriate utilization of ICU facilities, (2) provide information on nurse staffing ratios for various patient care areas, (3) validate clinical classification of critically ill patients into four categories and (4) analyzing the costs of intensive care in relation to the care offered [4]. By 1983 the TISS was a widely accepted method for classifying critical care patients. Due to innovations in critical care, the TISS was altered and ultimately consisted of 76 therapeutic interventions. This adjustment, however, did not affect the total assigned scores compared to the original model [17]. A strong correlation was found between the TISS-57 and the TISS-76 (regression equation TISS-76 = 0.5 + 1.03 * TISS-57). By 1997 two adapted versions of the TISS were published. First, the Intermediate TISS which was designed for non-ICU patients and second, the TISS-28 (Appendix A) which was designed specifically for ICU patients and is a simplified and validated version of the TISS-76 [18,19]. Miranda et al. demonstrated that the TISS-28 and TISS-76 are highly correlated (R2=0.86, regression equation 28 = 3.33 + 0.97 * 76) and concluded that the
TISS-28 can replace the TISS-76 thereby reducing registration burden. Per eight-hour shift, a patient can receive a maximum TISS-28 score of 78 points, where 46 points are equivalent to one FTE ICU nurse [19].
The Nursing Activities Score (NAS)
The NAS was developed because it appeared that the nursing workload is not only related to the complexity and severity of a particular intervention, but also to the time spent by nurses on executing the intervention [2]. The NAS consists of 23 nursing activities divided into seven major categories. These categories include basic activities (monitoring and controls, laboratory tests, medication, hygiene procedures, drain care, mobilization and positioning, support and care for families and patients, and administrative and managerial tasks), ventilator support, cardiovascular support, renal support, neurological support, metabolic support, and specific interventions [2]. Each NAS activity is given a weighted score based on the time consumed by nurses’ activities at the patient level per 24 hours (Appendix B). The sum of the weighted scores of a patient reflects the amount of time spent by nursing staff on caring for the ICU patient. A total NAS score can vary between 0% and 177%, where a score of 100% is equivalent to one FTE ICU nurse. A score higher than 100% suggests that the care provided for a particular patient should be delivered by more than one nurse. Compared to the TISS-28, the NAS is twice as effective in measuring how nurses allocate their time caring for ICU patients. The NAS explained 81% of the nursing time, whereas the TISS explained 43% [2].
8
Chapter 2 The Association between nursing workload
and hospital mortality in Dutch Intensive Care Units
Abstract
Introduction: The Nursing Activities Score (NAS) is an instrument which assigns weighted scores per
patient based on the time spent by nurses on executing nursing tasks. By combining the NAS score with information on the number of nurses present, the nursing workload can be assessed. Various studies report an association between nursing workload, expressed as Patient: Nurse Ratios (PNRs), and the risk of hospital death. It is unclear what the association is in Dutch ICUs and whether the use of NAS influences that association.
Objectives: To determine the best way to use Nurse: NAS Ratios (NNRs) in logistic regression
modeling and as a follow up to get insight into the relationship between (1) PNRs and hospital mortality and (2) NNRs and hospital mortality.
Methods: Patients admitted between the first of January 2016 and the first of July 2017 were
included. The patient data was divided into three subgroups: patients with a Length of Stay (LOS) of at least 1, 4 or 7 days. In all three subgroups several logistic regression models were developed in which the NNRs were included in different ways (for example mean NNR on day 1, the delta between NNR on day 1 and day 4, and NNR at ICU discharge). For all models, the Area Under receiver operating curve (AUC) and Brier scores were calculated to determine their performance. The relationship of the PNRs and NNRs with the outcome hospital mortality was analyzed by eight separate logistic regression models. Four models only included two types of the PNRs or NNRs as covariates and four association models also corrected for age, admission type, and comorbidities.
Results: The performance of the NNR model corrected for comorbidities, age and admission type
was good in all three LOS subgroups (AUC scores with 95% CI of NNR on day 1 as covariate: 0.84 (0.83 – 0.85); 0.71 (0.69 – 0.73) and 0.70 (0.67 – 0.73)). As the performance of the models did not differ between various summarizations of NNRs, the four covariates used further were PNR and NNR on day 1 and mean PNR and NNR. A significant association was found between PNR on day 1 and hospital mortality in the second quartile of PNR (PNR range: 0.55 – 0.79; OR 1.02 (95% CI: 1.004 – 1.036)), but after case-mix correction, the association became non –significant. The association between NNR on day 1 and hospital mortality showed significant results between respectively the second (NNR range: 69.28 - 106.49; OR 1.022 (1.009 – 1.036)) and third quartile (NNR: 106.49 - 140.42; OR 1.02 (1.006 – 1.033)). After case-mix correction the results remained significant (ORs 1.014 (1.002 – 1.026) and 1.015 (1.002 – 1.028)). When using mean NNR there was a significant association found for quartile 2 (NNR range: 69.56 - 106.2; OR 1.019 (1.006 – 1.033)) and 3 (NNR range: 106.2 - 137.17; OR 1.026 (1.012 – 1.41)) without case-mix correction and only for quartile 3 after case-mix correction (OR 1.017 (1.004 – 1.03)).
Conclusion: Performance of the logistic regression models is not dependent on the way NNRs are
summarized in the models. In Dutch ICUs, the association between PNRs and hospital mortality is not significant after case-mix correction. An increase in NNR seams to lead to a significantly higher hospital mortality, though this was not found for the highest quartile of the NNR.
9
2.1 Introduction
Patient acuity (PA) can be defined as the severity of illness of a patient and the intensity of nursing care that the patient requires [20]. PA scores, as severity of illness measures, are traditionally used to justify nurse staffing and resource allocation, but they are also proven to be useful for managing patient care outcomes and costs [21]. Serafim et al. found that increased patient severity, expressed as the Simplified Acute Physiology Score (SAPS) II score, is associated with a higher risk of adverse events (AEs), but the relationship between nursing workload, expressed as the nursing activities scores (NAS) score, and the occurrence of AEs was not statistically significant [22].
Nursing workload, the intensity of nursing care attribute of PA, can have various forms: physical, cognitive, time-pressure, emotional, quantitative, qualitative, and workload variability [3]. The NAS is a well-known patient level instrument to quantify PA that focusses on the intensity of needed nursing care. This scoring system assigns a weighted score per patient based on the time spent by nurses on executing nursing tasks [2]. Although intensive care nurses endure various forms of nursing workload, quantitative nursing workload presented as NAS scores can give valuable organizational information such as estimates on required nursing care, measurements of nurse-utilization ratios [16], measuring changes in workload after policy alterations and estimates of financial resources used for patient care [2,23,24].
Offering nursing care at lower costs by reducing the number of nurses can increase nursing workload while the patients’ need for nursing care is not met. This can in turn negatively impact nurses’ health, jeopardize patient safety and increase the risk of patient mortality [3]. Cho et al. found that every additional patient per nurse was associated with a 9% increase in the odds of death [7]. In a meta-analysis conducted by Numata et al., a relationship was found between higher nurse staffing levels and lower mortality, but after additional case-mix correction, the reported association became non-significant in four out of five included studies [25]. Another meta-analysis published in 2018 found that for every additional nurse, patients were 14% less likely to experience in-hospital mortality [26]. We are curious to know whether the found association can also be found in the Netherlands. Patient: Nurse Ratios (PNRs) do not take into account differences in severity of illness or need for nursing care of the patients. It is therefore also interesting to uncover whether nursing workload, expressed as Nurse: NAS Ratios (NNRs), is associated with hospital mortality in a similar way as with the use of PNRs. In the first part of this study, we investigated the most appropriate method to include NNRs in logistic regression modeling. In the second part, we investigated the association between respectively the PNRs and NNRs and hospital mortality.
2.2 Methods
2.2.1 Data sourceData was provided by the Dutch National Intensive Care Evaluation (NICE) registry, a non-profit foundation established by intensivists in 1996 [15]. Data from the NICE is primarily used for benchmarking purposes to monitor and improve quality of care. It consists of demographic, physiological, and clinical data of ICU patients from all 84 Dutch ICUs. The registry also contains organizational data, such as the volume and the number of full-time equivalent nurses and physicians
10
present during shifts. All ICUs collect the minimal dataset (MDS) for each admitted patient. The MDS describes the severity of illness based on, among others, the Acute Physiology and Chronic Health Evaluation (APACHE) IV model as well as outcomes measures such as ICU and in-hospital mortality, length of stay (LOS) and ICU readmission. One of the optional modules in the NICE registry is the nursing workload module including, among others, all data items needed to calculate the NAS [27].
2.2.2 In-and exclusion criteria
For this study, we included all Dutch ICUs that collected the number of nurses present during each shift as well as NAS scores per patient. From these ICUs, all patients with an admission date between the first of January 2016 and the first of July 2017 were included. These patients were included in the analyses to determine the association between PNRs and hospital mortality and the association between NNRs and hospital mortality.
For analyzing the best way to summarize NNRs the following additional exclusion criteria were applied: patient age < 16 years, LOS shorter than 4 hours or longer than 1 year, admission type is “death before ICU admission”, patient was readmitted, patient admitted from another Critical Care Unit (CCU) or ICU, no diagnosis present at admission, patient with burns, transplant patient, if admission type is missing and/or the patient is a CCU or recovery patient. The study population was then divided into three LOS subgroups: patients with a LOS of at least one day, four or seven days. See flowchart. Patients with missing variables were excluded from analyses.
2.2.3 Calculation of PNRs and NNRs
The daily PNRs were used as the average PNR of the day, evening and night shift. The PNR for each shift was calculated by dividing the number of ICU nurses present per shift per hospital by the number of ICU patients treated during the shift in that hospital. The nursing workload per day, presented as NNRs, was calculated as the total NAS generated in each hospital per day, divided by the average number of nurses present that day. The total NAS score per day was calculated by converting the number of assigned points for each nursing intervention per shift into a day score and subsequently by adding the daily NAS scores of all interventions together (Appendix B). All intensive care, medium care, recovery, cardiac and student nurses responsible for the care of patients located at the ICU were included in calculating the PNRs and NNRs.
2.2.4 Model development
Logistic regression models were used with hospital mortality as dependent variable and ICU as a random intercept to correct for clustering within ICUs as a covariate (Table 1 and Table 2). The other covariates included in these models differ between the used models. For simplicity, the models used to determine the influence of different methods to include NNRs on the performance of the models are referred to as summarization models, while the models used to analyze the association between PNRs and NNRs and hospital mortality are referred to as association models.
Summarization models
Different summarization models were developed for the three LOS subgroups. In each subgroup two types of models were developed, namely a model with only NNR as a covariate and a model with NNR, comorbidities, age and admission type as covariates (Table 1). The comorbidities are
11
dichotomous and consisted of: aids, acute renal failure, cardiovascular insufficiency, chronic dialysis, chronic renal insufficiency, chronic respiratory insufficiency, cirrhosis, Chronic Obstructive Pulmonary Disease (COPD), diabetes, gastrointestinal bleeding, hematologic malignancy, immunological insufficiency, and neoplasm.
Table 1 Format Summarization models
Model Format
Hospital mortality ~ (1|hospital of ICU admission) + NNRa
Hospital mortality ~ (1|hospital of ICU admission) + NNRa + comorbiditiesb + age + admission typec a NNR mean Nurse: NAS Ratio. The days on which the score was based are summarized in Appendix C
b Comorbidities include aids, acute renal failure, cardiovascular insufficiency, chronic dialysis, chronic renal insufficiency, chronic respiratory insufficiency, cirrhosis, Chronic Obstructive Pulmonary Disease (COPD), diabetes, gastrointestinal bleeding, hematologic malignancy, immunological insufficiency and neoplasm.
c Values for admission type = 1 (medical), 2 (urgent surgery) and 3 (elective surgery)
The NNRs were summarized in various ways which resulted in seven different regression models for the LOS subgroup of at least one day, 17 regression models for the LOS subgroup of at least four days, and 55 regression models for the LOS subgroup of at least seven days. The methods of summarizing the NNRs were based on a single day, two days, three days and/or the first seven days of ICU admission (Appendix C). This approach is based on Soliman et al. [28].
Association models
In total eight association models have been developed. Four association models only included the PNRs and NNRs as covariates and four association models also corrected for age, admission type, and comorbidities. These comorbidities were the same as mentioned in the summarization models. The PNRs and NNRs were each included in the models in two ways: based on the two best-performing methods found in the summarization models (Table 2).
Table 2 Format Association models
Model Format
Hospital mortality ~ (1|hospital of ICU admission)+ PNRa
Hospital mortality ~ (1|hospital of ICU admission)+ PNRa + comorbiditiesc + age + admission typed Hospital mortality ~ (1|hospital of ICU admission) + NNRb
Hospital mortality ~ (1|hospital of ICU admission)+ NNRb + comorbiditiesc + age + admission typed a PNR Patient: Nurse Ratio based on the two best performing summarization method
b NNR Nurse: NAS Ratio based on the two best performing summarization method
c Comorbidities include aids, acute renal failure, cardiovascular insufficiency, chronic dialysis, chronic renal insufficiency, chronic respiratory insufficiency, cirrhosis, Chronic Obstructive Pulmonary Disease (COPD), diabetes, gastrointestinal bleeding, hematologic malignancy, immunological insufficiency, and neoplasm.
12 2.2.5 Statistical Analysis
For all groups, categorical variables are presented using absolute and relative frequencies while continuous variables are presented using median and interquartile ranges (IQRs). For the summarization models, the performance of each regression model was assessed by calculating the discrimination with the Area Under receiver operating characteristic Curve (AUC) [29]. The accuracy of each model was assessed by the Brier score [30]. Both performance measures including 95% confidence intervals for each model were obtained by bootstrapping with a sample size of 500. The AUC represents the probability that, in our case, a randomly selected ICU patient who died during ICU admission is correctly classified or ranked with greater suspicion than a randomly selected ICU patient who did not. So the higher the AUC, the better a model’s performance. With the Brier score the mean squared difference between the predicted probability of, in our case, hospital mortality and actual outcome is calculated. This means that the lower the Brier score, the better the model’s predictions are calibrated [30,31]. The model with the highest AUC and lowest Brier scores is thus considered the best performing model.
For the association models, the covariates PNRs and NNRs, summarized in a way that performed best according to the summarization models, were included as categories based on quartiles. The Odds Ratio (OR) with 95% Confidence Interval (CI) for each quartile is calculated and statistical significance is considered if the CI does not contain one. All statistical analyses were performed using R version 3.3.3.
2.3 Results
2.3.1 LOS subgroups and Characteristics
Fifteen ICUs collected the number of nurses as well as NAS scores. From January 1, 2016, to July 1, 2017, 19,594 patients have been admitted to these Dutch ICUs. After excluding the patients due to the additional exclusion criteria (N = 5,282), the remaining 14,312 patients were divided into three groups: patients with a LOS of at least one (N = 14,312), four (N = 3,546) or seven days (N = 1,686) days (Figure 1). These three subgroups were used in the summarization models. All patients were used in the association models (N = 19,594). The characteristics of all patient groups are shown in Table 3.
13
14
Table 3 Patient Characteristics
Study populations of Summarization models Study population of Association models LOS subgroup 1 LOS ≥ 1 day LOS subgroup 2 LOS ≥ 4 days LOS subgroup 3 LOS ≥ 7 days All patients Number of patients, N 14,312 3,546 1,686 19,594
Hospital mortality, N (% died) 1,470 (10.3) 753 (21.2) 407 (24.1) 2,015 (10.3)
Age, median [IQR] 66.00 [56.00, 74.00] 67.00 [56.00, 74.00] 66.0 [56.0, 73.0] 66.0 [55.0, 74.0]
Admission type
-Medical N (%) 5,287 (36.9) 2,166 (61.1) 1,034 (61.3) 6,981 (40.0)
-Surgical: urgent, N (%) 1,510 (10.6) 654 (18.4) 393 (23.3) 2,468 (14.1)
-Surgical: elective, N (%) 7,515 (52.5) 726 (20.5) 259 (15.4) 7,988 (45.8)
-Death before ICU admission, N (%) - - - 16 (0.1)
LOS (in days), median [IQR] 0.99 [0.81, 2.30] 5.00 [3.16, 9.38] 9.76 [6.90, 15.49] 0.95 [0.71, 2.17]
LOS prior to ICU admission, median [IQR] 1.55 [0.79, 1.90] 0.99 [0.67, 1.95] 0.93 [0.65, 1.95] 1.51 [0.71, 2.42]
Comorbidities
AIDS, N (%) 27 ( 0.2) 10 ( 0.3) 7 ( 0.4) 32 ( 0.2)
Acute renal failure, N (%) 866 ( 6.1) 489 (13.8) 278 (16.5) 1,127 ( 5.8)
Cardiovascular insufficiency, N (%) 667 ( 4.7) 202 ( 5.7) 90 ( 5.3) 841 ( 4.3)
Chronic dialysis, N (%) 131 ( 0.9) 40 ( 1.1) 15 ( 0.9) 166 ( 0.8)
Chronic renal insufficiency, N (%) 770 ( 5.4) 283 ( 8.0) 126 ( 7.5) 968 ( 4.9)
Chronic respiratory insufficiency, N (%) 507 ( 3.5) 222 ( 6.3) 123 ( 7.3) 673 ( 3.4)
Cirrhosis, N (%) 128 ( 0.9) 48 ( 1.4) 23 ( 1.4) 162 ( 0.8) COPD, N (%) 1,747 (12.2) 627 (17.7) 307 (18.2) 2,060 (10.5) Diabetes, N (%) 2,467 (17.2) 639 (18.0) 289 (17.1) 2,888 (14.7) Gastrointestinal bleeding, N (%) 201 ( 1.4) 56 ( 1.6) 24 ( 1.4) 263 ( 1.3) Hematologic malignancy, N (%) 239 ( 1.7) 111 ( 3.1) 58 ( 3.4) 287 ( 1.5) Immunological insufficiency, N (%) 1,339 ( 9.4) 386 (10.9) 182 (10.8) 1,622 ( 8.3) Neoplasm, N (%) 935 ( 6.5) 185 ( 5.2) 69 ( 4.1) 1,062 ( 5.4)
15 2.3.2 Summarization models
LOS subgroup 1 (LOS ≥ 1 day, N = 14,312)
Considering the AUC and Brier scores, we see that there are no significant differences in performance within each of the two model groups ((1) NNR, (2) NNR + comorbidities). However, the models with NNR and comorbidities performed significantly better. For instance, NNR on day 1 as covariate had an AUC of 0.84 (95% CI: 0.83 – 0.85) and Brier of 0.077 (95% CI: 0.074 – 0.08) in the NNR + Comorbidities model group, while it had an AUC of 0.593 (95% CI: 0.593 – 0.609) and Brier of 0.091 (95% CI: 0.087 – 0.095) in the model group with only NNR as covariate (Figure 2 and Figure 3).
Figure 2 AUCs of Summarization models in LOS subgroup 1
16 LOS subgroup 2 (LOS ≥ 4 days, N = 3,546)
In this LOS subgroup, we also found no significant difference in model performance within each of the two model groups. But based on the AUC and Brier scores, the models with NNR and comorbidities had significantly better performance than the model with only NNR as a covariate. NNR on day 1 had an AUC of 0.71 (95% CI: 0.69 – 0.73) and Brier of 0.151 (95% CI: 0.143 – 0.158) in the NNR + Comorbidities model group while it had an AUC of 0.55 (95% CI: 0.53 – 0.57) and Brier of 0.166 (95% CI: 0.158 – 0.174) in the NNR model group (Figure 4 and Figure 5).
Figure 4 AUCs of Summarization models in LOS subgroup 2
17 LOS subgroup 3 (LOS ≥ 7 days, N = 1,686)
Similar as in LOS subgroups 1 and 2, we found no significant difference in performance within each model group ((1) NNR, (2) NNR + comorbidities). The models NNR + comorbidities had better performance compared to the models with only NNRs as a covariate. Covariate NNR on day 1 in model group 2 had an AUC of 0.70 (95% CI: 0.67 – 0.73) and Brier of 0.166 (95% CI: 0.155 – 0.176), while it had an AUC of 0.56 (95% CI: 0.53 – 0.59) and Brier of 0.183 (95% CI: 0.172 – 0.194) in model group 1 (Figure 6 and Figure 7).
Figure 6 AUCs of Summarization models in LOS subgroup 3
18 2.3.3 Association models
As the summarization models did not show any preferred way to summarize NNR we decided to include NNR and PNR based on the day 1 score as well as mean score over the whole admission period, as these summary measures have been described in the literature. The PNRs and NNR characteristics of the study population are shown in table 4.
Table 4 PNRs and NNRs of all patients (N = 19,594)
Study population (N = 19,594)
PNR on day 1 Median [IQR] 0.79 [0.55, 1.07]
PNR mean*, Median [IQR] 0.81 [0.57, 1.09]
NNR on day 1, Median [IQR] 106.49 [69.28, 140.42]
NNR mean*, Median [IQR] 106.20 [69.56, 137.17]
*Calculated over the patients whole ICU length of stay
IQR Interquartile Range, NNR Nurse: NAS Ratio; PNR Patient: Nurse Ratio
The association model with PNR as the only covariate (hence without case-mix correction) only shows a significant difference between the first quartile (< 0.55) and the second quartile (0.55 - 0.79) (Table 5). However, after case-mix correction, no significant differences were found. The models with NNR on day 1 without and with case-mix correction show significant differences for the second and third quartile, taking the first quartile as a reference (Table 6). When using mean NNR there were significant differences found for quartile 2 and 3 compared to the reference group (quartile 1) without case-mix correction and only for quartile 3 after case-mix correction.
Table 5 Association PNRs and Hospital Mortality (N = 19,594)
aModel: PNR bModel: PNR + comorbidities
Covariate Quartile Range OR (95% CI) OR (95% CI)
PNR day 1 1 < 0.55 - ( reference ) - ( reference )
2 0.55 - < 0.79 1.020 (1.004 – 1.036) 1.009 (0.995 - 1.024) 3 0.79 - < 1.07 1.012 (0.996 – 1.029) 1.007 (0.992 - 1.021) 4 ≥ 1.07 0.999 (0.983 – 1.016) 0.995 (0.980 - 1.010)
Mean PNR 1 < 0.57 - ( reference ) - ( reference )
2 0.57 - < 0.81 1.026 (1.009 – 1.043) 1.008 (0.993 - 1.023) 3 0.81 - < 1.09 1.010 (0.994 – 1.027) 1.001 (0.985 - 1.016) 4 ≥ 1.09 1.002 (0.986 – 1.020) 0.993 (0.978 - 1.008) CI Confidence Interval; PNR Patient: Nurse Ratio; OR Odds Ratio
Association models:
aHospital mortality ~ (1|hospital of ICU admission) + PNR
bHospital mortality ~ (1|hospital of ICU admission) + PNR + comorbidities + age + admission type
19 Table 6 Association NNRs and Hospital Mortality (N = 19,594)
cModel: NNR dModel: NNR + comorbidities
Covariate Quartile Range OR (95% CI) OR (95% CI)
NNR day 1 1 < 69.28 - ( reference ) - ( reference )
2 69.28 - < 106.49 1.022 (1.009 – 1.036) 1.014 (1.002 - 1.026) 3 106.49 - < 140.42 1.020 (1.006 – 1.033) 1.015 (1.002 - 1.028) 4 ≥ 140.42 1.012 (0.998 – 1.027) 1.008 (0.995 - 1.021)
Mean NNR 1 < 69.56 - ( reference ) - ( reference )
2 69.56 - < 106.20 1.019 (1.006 – 1.033) 1.010 (0.997 - 1.022) 3 106.20 - < 137.17 1.026 (1.012 – 1.041) 1.017 (1.004 - 1.030) 4 ≥ 137.17 1.014 (1.000 – 1.029) 1.011 (0.998 - 1.025) CI Confidence Interval; NNR Nurse: NAS Ratio; Nurse Ratio; OR Odds Ratio
Association models:
cHospital mortality ~ (1|hospital of ICU admission ) + NNR
dHospital mortality ~ (1|hospital of ICU admission) + NNR + comorbidities + age + admission type
Significant CI
2.4 Discussion
With regards to the use of NNRs in logistic regression modeling, we saw no significant difference in the performance measures within each model group. So the way the NNRs are summarized does not impact a model’s performance and we therefore advise to use the more simplified methods such as the NNR on day 1 or the mean NNR which are commonly used in the literature. A model’s performance might have been affected by the sample sizes of the data. We saw that the models from the first LOS subgroup performed significantly better than those from the second and third subgroups which are much smaller. An alternative more plausible explanation for the lower performance is that mortality among the subgroups with a LOS of at least 4 or 7 days, are less influenced by NNR and probably more by the patients’ severity of illness.
Our analysis showed that PNRs are not associated with in-hospital mortality of Dutch ICU patients, indicating that the number of patients that a nurse has to treat has no impact on the hospital mortality. A higher workload of a nurse expressed as NNR, however, is related with a higher hospital mortality risk. This suggests that not the number of patients treated by a nurse but the amount of work during the treatment of these patients is essential. Therefore any organizational guidelines for ICUs should be more focused on the workload that the present patients generate instead of the number of patients present. The association between NNR and hospital mortality after case-mix correction is only significant for the second and third quartile (between 69.3 and 140.4) when using the NNR on day 1 and only for the third quartile (between 106.2 and 137.2) when using the mean NNR. It is striking that the fourth quartile is not significantly different, apparently, there is no linear relationship. It is possible that the patient population of the fourth quartile differs from the other quartiles explaining the fact that there is no significant difference found in this quartile, this should be further investigated.
Sakr et al. found that an Nurse: Patient Ratio of more than 1:1.5 was independently associated with a lower risk of in-hospital death [32]. A systematic review found that for every additional nurse per
20
day, medical ICU patients had a lower odds of in-hospital mortality: OR 0.91 (95% CI: 0.86–0.96) [33], while a meta-analysis found that for every additional nurse per day, patients were 14% less likely to experience in-hospital mortality (OR 0.86, 95% CI: 0.79 – 0.94) [26]. Our study could not confirm similar findings. Other studies included nursing workload in different ways. Tarnow-Mordi et al and Lee et al. studied the association between patient acuity (PA) and hospital mortality where PA was measured as the occupancy rate per shift and with the Therapeutic Intervention Scoring System (TISS-76, Keene, 1983) respectively [34,35]. In both studies, a distinction was made between high and low PA and both studies found that patients with a score had a higher odds of dying. Similar results were found in Padilha et al. They found high NAS values, at patient level, was associated with an increased mortality [36]. Although we did not investigate the association between PA and in-hospital mortality at a patient level, we found similar results when assessing the association between the more organizational measurement NNR and in-hospital mortality. Based on our results, a recommendation for an optimal Patient: Nurse Ratio cannot be given for Dutch ICUs.
To our knowledge, this is the first study that investigated the association between the organizational measurement NNR and hospital mortality. Even though our study showed a non-significant association between PNRs and hospital mortality, this study offers new insights as there was a significant association between NNR and hospital mortality. This study provides evidence that the workload generated by the patients, is more important than the number of patients that are present. This study also provides evidence that the use of the NNR score on the first day is sufficient to use in logistic regression modeling. Our study has some limitations. First, the sample sizes of the three patient groups vary significantly. Especially in the smaller populations, there is a risk of overfitting. This could be the reason why the models from LOS subgroup 2 and 3 performed worse in comparison to the first subgroup. Second, only 15 of the 84 Dutch ICUs were included in this study, therefore, the results cannot be generalized to the whole Dutch population. Further research on a national level is needed to confirm the found associations between NNRs and hospital mortality. Nevertheless, the ICUs that participated in our study are representative of the general Dutch ICUs and therefore we believe that an optimal NAS score per nurse of < 70 can be recommended
2.5 Conclusion
NNR at ICU admission day is a simple and effective measure to summarize NNR, more complex summarizations do not improve the performance of the model. There was no significant association between PNRs and hospital mortality, however, there was a significant association between NNRs and hospital mortality. Therefore we conclude that it is more important to focus on the nursing workload that the present patients generate instead of the number of patients present at the ICU. A recommendation for an optimal NNR of 70 can be given, but further research with more ICUs can strengthen this recommendation.
21
Chapter 3 Nursing Workload in Practice
3.1 NICE registry’s nursing workload module
The NICE registry, in collaboration with the V&VN-IC (the Dutch association for ICU nurses), started the nursing workload (NW) module in an effort to provide more insight into the actual nursing workload in Dutch ICUs, so nurses can be deployed more efficiently [16]. As of 2016, the NICE registry collects data from all Dutch ICUs, but participation in the NW module is voluntary and currently applied by 15 ICUs. The ICUs who do participate in the NW module receive feedback reports biannually. In these reports, they can view their results over the past six months and compare those results with the national medians.
ICUs participating in the NW module record their TISS and NAS scores per patient per eight-hour shift. In addition to the TISS and the NAS scoring systems, data is also collected for the NICE’s own NW scoring system. This scoring system combines aspects of the TISS and the NAS and includes seven additional variables: (1) continuous EEG monitoring, (2) cardiac assist device, (3) other vasoactive medication, (4) presence of the central line, (5) ECMO (Extra Corporeal Membrane Oxygenation, (6) MARS (Molecular Adsorbents Recirculating System) and (7) building a kidney dialysis machine by a nurse [16]. The NICE-NW scoring system is still under validation. Lastly, the number of nurses present during each shift is also collected. A distinction is made between Intensive Care (IC), Medium Care (MC) and student nurses present during shifts.
The feedback reports consist of the median NW scores of a particular ICU compared with the medians of all NW module participants. In these reports, the NW scores for the TISS, NAS and NICE-NW scoring systems are presented. For each scoring system, the median scores per shift and median scores per ICU nurse are also provided. An overview of the content of the NW feedback reports is shown in Appendix D. Each ICU can also view their NW scores in “NICE Online”. This is a web-based tool that gives ICUs the opportunity to view and analyze their own data. NICE Online disables user to view results from other ICUs but allows comparisons with national medians [37].
The NW module was executed as a pilot among 13 ICUs in 2015. In 2017, 15 ICUs were participating in the module. It is unknown, however, how these ICUs use the NW results and if there are adjustments to be made to improve the usability of the feedback reports. During the NICE annual discussion gathering in 2017, some ICUs mentioned that they hesitate to participate in the NW module due to a fear of high registration burden for their staff. In this chapter we describe how interviews with ICU personnel have been conducted to gain more insight in how they view and use the NICE registry reported NW results, to determine if they have suggestions for improving the module and how they deal with the (possible) registration burden.
3.2 Method
A representative of four Dutch ICUs participating in the NW module was invited to be interviewed. These representatives, mostly intensivists, received an email explaining the purpose of the interview.
22
Upon agreement, the interviews were held in person at the ICU department of the respective ICU or via telephone, depending on the location of the ICU. The questions asked (Appendix E) were to find out:
1) what their thoughts on the NW module are;
2) if they use the results of the biannual feedback reports; 3) If there is a registration burden for their nursing staff;
4) if they have suggestions for the NICE on how to improve the NW module and feedback reports
With permission of the participants, the interviews were recorded and summarized.
3.3 Results
ParticipantsAll four ICUs agreed to participate in the interviews, but one canceled due to a scheduling conflict. Of the three remaining interviews, one was held at the respective ICU, while the others were held via telephone. The intensivists of 2 ICUs were also accompanied by an ICU nurse who is involved with the NW module at their ICU. The interviews had an average duration of twenty minutes. ICU A has the most experience in collecting NW data because they started prior to the NW module pilot. ICU B and C decided to continue with the NW module after the pilot. The selected ICUs are not equal in size and ICU B is a more specialized ICU.
Interview results
ICU A discusses the results of the feedback reports with ICU nurses every quarter and uses the
results for internal and external benchmarking. NW scores, expressed as NAS are used in their long-term goals such as to verify whether their nurses are overworked or not and use that information to make policy regarding hiring new personnel. The scores themselves, however, are not used in scheduling of nurses at short notice. This ICU also mentions that some nursing activities, such as resuscitation and activities done during a Mobile Intensive Care Unit (MICU) shift, are not included in the scoring systems while they take up a lot of the nurses’ time. The nurses are already used to finishing up administrative tasks at the end of each shift, so the addition of recording NW variables takes little additional effort. A nurse has to fill in for each patient whether a task was executed. They estimate a time duration of one minute for an ICU nurse to evaluate a patient at the end of a shift.
ICU B is using NW scores as an internal benchmark to compare their own NW scores over time. They
are also at a turning point to decide whether or not to expand their ICU and hope the NW scores can provide insights into whether their nurses are overworked or not. Although this ICU treats a specific patient population, after external benchmarking, they discovered that their NW scores are more or less aligned with the national medians. They want to continue benchmarking, but they are also curious to know how other ICUs use the NW results. With the exception of six variables, all variables for determining the NAS, TISS and NICE-NW scores are automatically generated from their EMR system. This leads to approximately 2 minutes registration time needed for one of their ICU nurses to evaluate a patient at the end of a shift.
ICU C is most concerned about how to use the NW results in their daily practice. They find it difficult
23
short and long-term policy. They started with the NW module at the end of 2016 in an effort to uncover how many full-time nurses they should schedule during shifts. It is not clear how they should interpret the NW scores, so therefore they do not yet use the scores for the goal they intended to. When benchmarking the results, ICU C also wants to know more about the characteristics of the other ICUs, such as the size of the ICU and whether they are a peripheral or academic hospital. They want to know this in order to justify why their NW scores are higher or lower than (the median of) other ICUs. Although the variables for determining the NW scores are primarily derived from their EMR system, the estimated time needed for a nurse to evaluate a patient at the end of each shift is five minutes.
About the reports
All ICUs had little to say about the reports themselves. A lot was said about the timeliness and interpretation of results in the reports. The ICUs mentioned that it is difficult to use the results in their short-term scheduling of their ICU nurses. They also mentioned that the number of scoring systems used was too many. There are no clear distinctions between the three scoring systems (TISS, NAS, and NICE-NW) and they prefer simplification by showing one over three if they all imply the same things.
3.4 Suggestions for the NICE
The three participants were also asked if they had wishes and suggestions for the NICE to improve the module and attract more ICUs to join the module. Some of their wishes and suggestions are:
1. To include variables such as resuscitation and MICU shift that are not part of the three scoring systems, but do take up a lot of nursing time.
2. To add a supplementary guide in the biannual reports on how to interpret the NW scores 3. To add a toolbox in NICE Online with tips and tricks on how to use the NW scores and
interpretation of graphs
4. To receive NW reports per quarter instead of every six months 5. To be able to view NW scores in NICE Online in real time
6. The ability to compare NW scores between ICUs that are equal in size. This option is not (yet) in the biannual report, but it is already implemented in NICE Online. The ICU that suggested this also mentioned not to be up to date with all the ins and outs of the web tool.
7. The ability to score NW points for each individual nurse. Currently, each ICU has an overview of NW scores per full-time equivalent nurse regardless of which nurse it is. One ICU suggested that it might be interesting to review NW scores for each individual nurse, especially during annual performance appraisals.
3.5 Discussion
We found that not all ICUs actively use the NW reported results and they would like to see more clarity in what the results of the scoring systems mean in order for them to translate the scores into
24
actions. We also found that although the experienced registration burden varies between ICUs, it is limited.
Strengths and limitations
The interviewed ICUs are considered to be the more active participants in the NICE registry. This did not, however, tempt these ICUs from holding back information on their experiences with the NW module. In our opinion, they were very open and honest, but we could not compare their answers and opinions to those without a close relationship with the NICE. This study also has several limitations. First, the interviews were conducted with only three ICUs, which is not an accurate representation of all ICUs in the Netherlands (N = 84). Second, Interviews were not conducted with ICUs who opted out of the module. It might be useful for the NICE to create an overview of all the reasons ICUs have for not continuing with the module.
Impact of results and future work
During the annual NICE discussion gathering in 2017, some hospitals expressed their concern about the possible registration burden for their staff. But, the ICUs we interviewed stated that there were no problems with recording the NW scores especially because their EMR system allows them to automatically derive the scores for each patient in a timely manner and their nurses are already used to executing administrative tasks at the end of each shift. ICUs from various hospitals use different EMR systems, so for new participants of the module it might take some effort at the beginning to determine how the variables can be derived from their EMR system.
Not all the suggestions given by the ICUs on improving the NW reports are achievable for the NICE to implement in a short amount of time. For example, “To be able to view NW scores in real time”. However, the suggestion about a guide in the report on how to interpret the NW scores is an important one and necessary to keep the participants informed on the possibilities of using NW scores. But to achieve this, more research is needed. It was also suggested to add more variables to the scoring systems. It will take up a lot of time to determine which variables are important enough to include in the scoring systems and adding new variables will also affect the internal benchmarking process for individual ICUs. But ICU care is constantly changing and periodically revising the scoring systems to be aligned with current ICU procedures and nursing activities can offer ICUs a better picture of their current way of working. It is up to the NICE registry to look into how to prioritize the suggestions given and determine which of those are feasible to implement.
All three ICUs were curious to know what other ICUs are doing with the NW scores. They see the added value in benchmarking their results with other ICUs but are not entirely sure about how to use NW scores. In the future, it might be useful for them to come together in a focus group setting and share their ideas, thoughts and uses for the NW scores with each other. This might be possible during the NICE annual discussion gathering in 2018.
3.6 Conclusion
The results of the interviews gave a good impression of how current participants view the NICE NW module. Even though the ICUs are still figuring out how exactly to use the NW scores of the various scoring systems, the opinions expressed are valuable for the NICE registry to maintain and possibly expand the number of participants of the NW module.
25
Chapter 4 General Discussion
Although NW scoring systems such as the TISS and the NAS have been around for many years [2,4], they are not universally used. Since 2013, the NICE registry has implemented the two scoring systems and a new NW scoring system to help Dutch ICUs gain better insights into their nurse allocation and internal and external benchmarks of their NW scores [16]. NW can be expressed in different ways, one of the easiest to measure and calculate is the PNR or its inverse. It is unclear, however, what the optimum PNR in a Dutch ICU should be [5,6]. Due to ambiguous results found in literature, the Dutch ICU quality standard does not provide a recommendation for an optimum PNR. They also mention that several other factors, such as age, the severity of illness and comorbidity, can affect the association between PNRs and mortality. In an effort to uncover unambiguous results we investigated the association between NW and hospital mortality, where NW was expressed as either PNRs or NNRs. Before we started with this, we first looked into the best method to summarize the NNR in our models. We also included models that corrected for age, admission type, and various comorbidities.
We did not find significant differences in the predictive value of various ways of summarizing the NNRs and decided to continue our research with two simple summarizations which are commonly described in the literature: the NNR based on day one and the mean NNR over the whole admission period. The models that included comorbidities performed better than those with solely NNRs as covariates. To determine the association between NW on an organizational level and hospital mortality we used the NW, either expressed as PNR or NNR calculated at the first day of ICU admission or as the mean during the total ICU admission period, adjusted for comorbidities of the patient. As we focused on the NW independently of the severity of illness, our models did not correct for the patient’s severity of illness. Though, it might be possible that the performance of the models would improve if a severity of illness measure, such as the APACHE or SAPS, was included. The association between severity of illness and hospital mortality is frequently discussed by using prognostic models such as the APACHE or SAPS which have good discrimination and calibration [38,39].
We found a non-significant relationship between PNRs and hospital mortality. Significant results were found when using NNRs in models. This means that the amount of work executed by nurses says more than the number of patients a nurse has to take care of. Therefore, we believe that our study can support the next update of the Dutch IC quality standard as we showed that the focus should be on the NNRs instead of PNRs. Based on our finding we recommend ICUs to plan their nursing capacity based on a NNR of 1: < 70. In order to fully support the guideline, ICUs should be recommended to participate in the NW module to monitor their nursing capacity. Furthermore, when more ICUs collect these data, our study can be repeated with data of more Dutch ICUs to validate our current findings. Once more ICUs participate in the NW module, the NICE registry will also be able to develop and validate a prognostic model that predicts the number of nurses needed for the next shift. This will enable the possibility to use the NW information more as a management tool.
The NICE has shown that the Dutch ICUs can provide the TISS and NAS data in a uniform manner. The focus of our research project was on the NAS because it has been shown to be twice as effective in measuring nursing time as the TISS [2]. But perhaps the combination of TISS and NAS in a new NW model can result in an improved model’s performance. Future research on this topic will become
26
possible now that ICUs collect the data of both models. The ICUs already participating in the NW module see the added value of benchmarking their results, but the NICE has to improve their feedback reports with additional information on how to interpret the NW scores of the three scoring systems. Other suggestions given by the participants, such as collecting and reporting NW data for each individual nurse, are more challenging to implement because of technical and privacy issues. We believe these applications belong at a local level and not at a national NICE level.
Conclusions
When using NNRs, summarizing the scores on the first day of ICU admission is sufficient to use. The fact that we found significant results when using NNRs to determine the association between nursing workload and hospital mortality suggests that not the number of patients treated by a nurse but the amount of work during the treatment of these patients is crucial. In the future, organizational guidelines for ICUs should be more focused on the workload that the present patients generate instead of the number of patients present. To provide more knowledge on this topic (in the Netherlands), more ICUs should participate in the NW module. There are various things the NICE can do to improve the module, but providing more clarity in the meaning of the various scoring systems could allow participants to stay more motivated to continue and actively use the reported results.
27
References
1. Bittner M-I, Donnelly M, van Zanten ARH, Andersen JS, Guidet B, Trujillano Cabello JJ, et al. How is intensive care reimbursed? A review of eight European countries. Ann Intensive Care. 2013;3(1):1–9. 2. Miranda DR, Nap R, de Rijk A, Schaufeli W, Iapichino G. Nursing activities score. Crit Care Med.
2003;31(2):374–82.
3. Carayon P, Alvarado CJ. Workload and Patient Safety Among Critical Care Nurses. Crit Care Nurs Clin North Am. 2007;19(2):121–9.
4. Cullen DJ, Sivetta J, Briggs B, Ferrara L. Therapeutic Intervention Scoring System: A method for quantitative comparison of patient care. Crit Care Med. 1974;
5. De Adviescommisie Kwaliteit van het Zorginstuut. Kwaliteitsstandaard Organisatie van Intensive Care.
2016. p. 1–18.
6. De Adviescommissie Kwaliteit van het Zorginstituut. BIJLAGEN bij Kwaliteitsstandaard Intensive Care Onderbouwingsdocument. 2016;(July). Available from: https://www.zorginstituutnederland.nl
7. Cho S-H, Hwang JH, Kim J. Nurse Staffing and Patient Mortality in Intensive Care Units. Nurs Res. 2008;57(5):322–30.
8. Stone PW, Mooney-Kane C, Larson EL, Horan T, Glance LG, Zwanziger J, et al. Nurse working conditions
and patient safety outcomes. Med Care. 2007;45(6):571–8.
9. Cho SH, Yun SC. Bed-to-nurse ratios, provision of basic nursing care, and in-hospital and 30-day mortality among acute stroke patients admitted to an intensive care unit: Cross-sectional analysis of survey and administrative data. Int J Nurs Stud. 2009;46(8):1092–101.
10. Neuraz A, Guérin C, Payet C, Polazzi S, Aubrun F, Dailler F, et al. Patient mortality is associated with staff resources and workload in the icu: A multicenter observational study. Crit Care Med. 2015;43(8):1587– 94.
11. Van Den Heede K, Clarke SP, Sermeus W, Vleugels A, Aiken LH. International experts’ perspectives on
the state of the nurse staffing and patient outcomes literature: World health. J Nurs Scholarsh. 2007;39(4):290–7.
12. Kiekkas P, Sakellaropoulos GC, Brokalaki H, Manolis E, Samios A, Skartsani C, et al. Association between nursing workload and mortality of intensive care unit patients. J Nurs Scholarsh. 2008;40(4):385–90. 13. Sales A, Sharp ND, Li Y-F, Lowy E, Greiner G, Liu C-F, et al. The association between nursing factors and
patient mortality in the Veterans Health Administration: the view from the nursing unit level. Med Care. 2008;46(9):938–45.
14. Metnitz PH, Reiter A, Jordan B, Lang T. More interventions do not necessarily improve outcome in critically ill patients. Intensive Care Med. 2004;30(8):1586–93.
15. Van De Klundert N, Holman R, Dongelmans DA, De Keizer NF. Data Resource Profile: The Dutch National
Intensive Care Evaluation (NICE) Registry of Admissions to Adult Intensive Care Units. Int J Epidemiol. 2015;44(6):1850.
16. The Dutch National Intensive Care Evaluation (NICE). Jaarboek NICE 2016. 2016; Available from: https://www.stichting-nice.nl
17. Keene AR, Cullen DJ. Therapeutic Intervention Scoring System: update 1983. Vol. 11, Crit Care Med. 1983. p. 1–3.
18. Cullen DJ, Nemeskal AR, Zaslavsky AM. Intermediate TISS: a new Therapeutic Intervention Scoring System for non-ICU patients. Vol. 22, Critical care medicine. 1994. p. 1406–11.
19. Miranda DR, de Rijk A SW. Simplified Therapeutic Intervention Scoring System: the TISS-28 items--results from a multicenter study. Crit Care Med. 1996;64–73.
20. Daly BJ, Brennan CW. Patient acuity: A concept analysis. J Adv Nurs. 2009;65(5):1114–26.
21. van Slyck A, Johnson K. Using Patient Acuity Data to Manage Patient Care Outcomes and Patient Care
Costs. Outcomes Manag Nurs Pract. 2001;5(1).
22. Serafim CTR, Dell’Acqua MCQ, Castro MCN e, Spiri WC, Nunes HR de C. Severity and workload related
to adverse events in the ICU. Rev Bras Enferm. 2017;70(5):942–8.
23. Altafin JAM, Grion CMC, Tanita MT, Festti J, Cardoso LTQ, Veiga CFF, et al. Nursing activities score and workload in the intensive care unit of a university hospital. Rev Bras Ter Intensiva. 2014;26(3):292–8.
