University of Groningen
SURvival PRediction In SEverely Ill Patients Study-The Prediction of Survival in Critically Ill Patients by ICU Physicians
Ros, Marijke M; van der Zaag-Loonen, Hester J; Hofhuis, José G M; Spronk, Peter E Published in:
Critical Care Explorations
DOI:
10.1097/CCE.0000000000000317
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2021
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Ros, M. M., van der Zaag-Loonen, H. J., Hofhuis, J. G. M., & Spronk, P. E. (2021). SURvival PRediction In SEverely Ill Patients Study-The Prediction of Survival in Critically Ill Patients by ICU Physicians. Critical Care Explorations, 3(1), [e0317]. https://doi.org/10.1097/CCE.0000000000000317
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Downloaded from http://journals.lww.com/ccejournal by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2021 Downloadedfrom http://journals.lww.com/ccejournalby BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78=on 04/21/2021 DOI: 10.1097/CCE.0000000000000317
Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine. Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a "work of the United States Government" for which copy-right protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government.
OBJECTIVES: The surprise question, “Would I be surprised if this patient
died in the next 12 months?” is a tool to identify patients at high risk of death in the next year. Especially in the situation of an ICU admission, it is important to recognize patients who could and could not have the benefits of an intensive treatment in the ICU department.
DESIGN AND SETTING: A single-center, prospective, observational
co-hort study was conducted between April 2013 and April 2018, in ICU Gelre hospitals, location Apeldoorn.
PATIENTS: A total of 3,140 patients were included (57% male) with
a mean age of 63.5 years. Seven-hundred thirteen patients (23%) died within 1 year.
INTERVENTIONS: The physician answered three different surprise
ques-tion’s with either “yes” or “no”: “I expect that the patient is going to survive the ICU admission” (surprise question 1), “I expect that the patient is going to survive the hospital stay” (surprise question 2), and “I expect that the patient is going to survive one year after ICU admission” (surprise question 3). We tested positive and negative predicted values of the surprise ques-tions, the mean accuracy of the surprise quesques-tions, and kappa statistics.
MEASUREMENTS AND MAIN RESULTS: The positive and negative
predictive values of the surprise questions for ICU admission, hospital admission, and 1-year survival were, respectively, 64%/94%, 59%/92%, and 60%/86%. Accordingly, the mean accuracy and kappa statistics were 93% (95% CI, 92–94%), κ equals to 0.43, 89% (95% CI, 88–90%), κ equals to 0.40, and 81% (95% CI, 80–82%), κ equals to 0.43.
CONCLUSIONS: The frequently overlooked simple and cheap surprise
question is probably an useful tool to evaluate the prognosis of acutely admitted critically ill patients.
KEY WORDS: critical care; intensive care units; (hospital) mortality;
palliative medicine; patient-centered care; survival analysis
W
ith an aging population, increasing comorbidities, and technolog-ical developments, the demand for ICU services is growing. This growth also raises the need for the discussion on palliative care in patients who are admitted based on the principle “primum non nocere” (first, do no harm) (1, 2). Therefore, it is important to use accurate prognostic mod-els, providing appropriate personalized care for each patient.In the ICU setting, it is common practice to use tools like the Acute Physiology Age and Chronic Health Evaluation (APACHE)–II (3) to estimate the ICU
Marijke M. Ros, MD1
Hester J. van der Zaag-Loonen, MD, PhD2
José G.M. Hofhuis, PhD1,2
Peter E. Spronk, MD1,2
SURvival PRediction In SEverely Ill Patients
Study—The Prediction of Survival in Critically
Ill Patients by ICU Physicians
Ros et al
mortality and the Predicted Risk Existing Disease and Intensive Care Therapy (PREDICT) for long-term prog-nosis (5 yr) of critically ill patients (4). These models are limited by its inherent group approach, which makes it impossible to translate group-based numbers to indi-vidual patients. In addition, these prediction models take data into account that can only be acquired after an ob-servational period of at least 8 hours. The estimation of intensive care physicians regarding prediction of hospital mortality in critically ill patients might be more accurate (5). However, it is unknown if physicians can reliably predict long-term survival of critically ill patients. They might overpredict the risk of mortality, in line with two-objective prognostic models (6), which could translate to unjustified fatalism when considering the treatment options in critically ill patients.
The surprise question (SQ), “Would I be surprised if this patient died in the next 12 months?”, is a tool to identify patients at high risk of death in the next year (7). It is used in Medical Oncology to recognize the phase of palliative care (8). A recent meta-analysis highlighted a wide degree of accuracy of the SQ as a prognostic tool (9) and as a poor, to modestly, predic-tive tool for death in noncancer illness (8).
The value of the SQ, answered by intensive care physi-cians in the setting of seriously ill patients, is currently unknown. Our research question was “Is the SQ in the acute setting (within 2 hours of admission) a useful tool to predict the risk of mortality in critically ill patients, dur-ing ICU stay, hospital stay and after one year?”. Because of answering the SQ in the first 2 hours of admission, we tried to measure the clinical insight of the physician (not influenced by clinical course). We hypothesized that the SQ in the acute setting has additive value for predicting the risk of mortality in critically ill patients.
MATERIALS AND METHODS
Study Design and Participants
The design of this study was a single-center, prospec-tive, observational cohort study. The study included all patients of 18 years and older, who were admitted to the ICU of Gelre hospitals, location Apeldoorn, between April 2013 and April 2018. Gelre hospitals is a univer-sity-affiliated teaching hospital with 850 beds. The ICU is a 12-bed mixed medical-surgical ICU. Only the first admission of a patient during this period was evaluated, all ICU readmissions of the same patient during these
years were excluded. After informing the Gelre Hospital Medical Ethics Review Board about the research, a waiver for this study was obtained (reference number: 2019_11). During the study period, the medical ICU staff consisted of six intensivists, who had 5, 8, 10, 10, 11, 18 years of experience, respectively (median 10 yr). No changes in the staff occurred during the study period.
Patients were treated according to (inter)–national standards. In addition, the physician filled in the Dutch National Intensive Care Evaluation (NICE) registry (10) for each ICU admission. This was a standard procedure at all ICU admissions. The NICE registry collects dem-ographic information, acute physiologic variables, pa-tient outcomes, and clinical diagnoses of all admissions within 24 hours. In this NICE registry, the physician was also asked to answer the following SQs with either “yes” or “no”: “I expect that the patient is going to survive the ICU admission” (SQ 1), “I expect that the patient is going to survive the hospital stay” (SQ 2), and “I expect that the patient is going to survive one year after ICU admission” (SQ 3). These SQs were all answered within 2 hours of ICU admission. Variables, of which the ICU physician had knowledge at the time of answering the SQs, for ex-ample, age and sex of the patient, were marked.
We composed a database of anonymized data re-garding demographic information, APACHE III scores, date of ICU admission, date of death (if appli-cable), and the answers of physicians about survival expectation of the patients.
Outcomes
The primary endpoints of this study were ICU sur-vival, hospital sursur-vival, and 1-year survival. Secondary outcomes included positive predictive values (PPVs) and negative predictive values (NPVs) of responses to the SQs by ICU physicians.
Statistical Analysis
Data were analyzed with IBM SPSS Statistics Version 25. The differences in baseline between the included and excluded patients were tested by independent sample t tests, chi-square tests, and Mann-Whitney U tests. PPVs and NPVs of the different SQs were calcu-lated for the three outcomes. The Kappa statistic was calculated to determine agreement between observed and predicted values (11). With the use of multivariable logistic regression analysis, the independent relation
between the SQ and actual mortality within 1 year was studied. We performed a stepwise approach. First, all baseline variables were entered into a multivari-able model. Through backward selection, the varimultivari-ables which had an association with the outcome, expressed by a p value of less than 0.3, were retained in the model. Next, the SQ was added to the model. The classification table was used to assess the effect size of the addition.
RESULTS
Patient Characteristics
A total of 3,140 patients were included. Fifty patients were excluded from analysis because of missing data regarding the SQs, 568 patients were excluded because of readmission (Fig. 1). Baseline characteristics be-tween the included and excluded patients differed for some variables (Supplement 1 Tables 1 and 2, http:// links.lww.com/CCX/A483). In the missing data group, there were more cerebrovascular accidents and car-diovascular insufficiency at the start. Less patients in the missing group survived 1 year (64%) in com-parison with the included group (77%; p = 0.04). The readmission group differed especially in chronic ob-structive pulmonary disease (COPD) and confirmed infection. Besides that, the readmission group is more often respiratory insufficient (11% vs 4%; p = 0.00) and required mechanical ventilation after 24 hours of ICU admission more often (50 vs 46%; p = 0.04).
Baseline characteristics of the study population were listed according to the SQ estimation of de ICU phy-sician for 1-year survival (Table 1). The mean age of the included patients was 63.5 ± 16.6 years, and 1,793 patients (57%) were male. Mortality after 1 year was 23%, and 270 of 3,140 patients (9%) did not survived the ICU admission. Patients who had been expected by the physician to survive 1 year were younger (61.3 yr; 95% CI, 60.6–62.0) than patients who were ex-pected to die within 1 year (72.5 yr; 95% CI, 71.5–73.5;
p ≤ 0.001). In addition, those patients had less
comor-bidities, like COPD and diabetes, in comparison with the patients who were expected to die in 1 year (Table 1). Almost all variables were different between the groups “expected alive” and “expected to die”. The mean APACHE III score of patients expected alive after 1 year was 59.6 (95% CI, 58.7–60.5) and 82.0 (95% CI, 80–84; p = 0.04) in patients who were expected to have died within 1 year.
Expectation of ICU Survival (SQ 1)
For 2,987 out of 3,140 patients (95% CI, 94–96), the physician expected survival of ICU admission. The NPV was 94% (Table 2). For 153 patients (5%), the physician expected mortality during ICU admission. The PPV of this SQ was 64% (95% CI, 62–66) (Table 2). The accuracy of this SQ was 93% (95% CI, 92–94) with a kappa statistic of 0.43 (95% CI, 0.37–0.49).
Ros et al
TABLE 1.
Baseline Characteristics
Expected Alive Expected Dead
Variables SQ 3: 1 Year Survival (n = 2,531—80.6%) SQ 3: 1 Year Survival (n = 609—19.4%) Total Group, n = 3,140
Age, yr, mean (sd)a 61.3 (16.8) 72.5 (12.6) 63.5 (16.6)
Gender, male, % (CI)a 57.8 (55.9–59.7) 45.8 (41.8–49.8) 57.1 (55.4–58.8)
Diabetes, % (CI)a 15.4 (14.0–16.8) 20.5 (17.3–23.7) 16.4 (15.1–17.7)
Chronic obstructive pulmonary disease, % (CI)a 12.7 (11.4–14.0) 20.2 (7.0–23.7) 14.1 (12.9–15.3)
Acute renal insufficiency, % (CI) 5.6 (4.7–6.5) 10.7 (8.2–13.2) 6.6 (5.7–7.5)
Chronic renal insufficiency, % (CI)a 2.6 (2.0–3.2) 5.6 (3.8–7.4) 3.2 (2.6–3.8)
Chronic dialysis, % (CI)a 0.7 (0.4–1.0) 1.0 (0.2–1.8) 0.7 (0.4–1.0)
Confirmed infection, % (CI)a 27.2 (25.5–28.9) 33.2 (29.5–36.9) 28.3 (26.7–29.9)
Gastrointestinal bleeding, % (CI)a 2.6 (2.0–3.2) 4.4 (2.8–6.0) 3.0 (2.4–3.6)
Cerebrovascular accident, % (CI)a 1.2 (0.8–1.6) 2.3 (1.1–3.5) 1.4 (1.0–1.8)
Intracranial mass, % (CI)a 0.2 (0.0–0.4) 2.0 (0.9–3.1) 0.6 (0.3–0.9)
Immunocompromized, % (CI)a 9.6 (8.5–10.7) 12.0 (9.4–14.6) 10.0 (9.0–11.0)
Neoplasm, % (CI)a 3.7 (3.0–4.4) 14.6 (11.8–17.4) 5.8 (5.0–6.6)
Cirrhosis, % (CI)a 0.6 (0.3–0.9) 2.6 (1.3–3.9) 1.0 (0.7–1.3)
Hematologic malignancy, % (CI)a 1.2 (0.8–1.6) 3.8 (2.3–5.3) 1.7 (1.2–2.2)
Cardiovascular insufficiency, % (CI)a 0.8 (0.5–1.1) 5.9 (4.0–7.8) 1.8 (1.3–2.3)
Cardiopulmonary resuscitation < 24 hr before admission, % (CI)a
1.5 (1.0–2.0) 6.7 (4.7–8.7) 2.5 (2.0–3.0)
Respiratory insufficiency, % (CI)a 3.1 (2.4–3.8) 9.2 (6.9–11.5) 4.3 (3.6–5.0)
Mechanical ventilation at 0 hr, % (CI)a 29.3 (27.5–31.1) 41.9 (38.0–45.8) 31.7 (30.1–33.3)
Mechanical ventilation after 24 hr, % (CI) 42.1 (40.2–44.0) 61.9 (58.0–65.8) 45.9 (44.2–47.6) Admission typea, % (CI)
Medical 65.2 (63.3–67.1) 79.5 (76.3–82.7) 68.0 (66.4–69.6)
Acute surgery 26.3 (24.6–28.0) 17.1 (14.1–20.1) 24.5 (23.0–26.0)
Elective surgery 8.5 (7.4–9.6) 3.1 (1.7–4.5) 7.5 (6.6–8.4)
Acute ICU admission, % (CI)a 92.9 (91.9–93.9) 98.0 (96.9–99.1) 93.9 (93.1–94.7)
ICU stay duration, d, median (IQR) 1 (1–4) 2 (1–5) 2 (1–4)
Hospital stay, d, median (IQR) 11 (4–20) 9 (5–17) 10 (5–18)
ICU survival, % (CI) 91.4 (90.4–92.4)
Hospital survival, n (%) 87.5 (86.3–88.7)
1-yr survival, n (%) 86.2 (84.9–87.5) 40.4 (36.5–44.3) 77.3 (75.8–78.8)
Acute Physiology Age and Chronic Health Evaluation III, mean (sd)
59.6 (23.1) 82.0 (25.2) 64.0 (25.2)
IQR = interquartile range, SQ = surprise question.
Expectation of Hospital Survival (SQ 2)
The intensivist was also asked what he/she expected regarding hospital survival, which could be different from ICU survival. For 2,888 patients (92%; 95% CI, 91–93), the physician expected survival of hospital ad-mission. The negative predicted value was 92% (95% CI, 91–93) (Table 2).
For 252 patients (8%), the physician expected mor-tality during the hospital admission. The positive pre-dicted value of this SQ was 59% (95% CI, 53–65). The accuracy of this SQ was 89% (95% CI, 88–90%) with a kappa statistic of 0.40 (95% CI, 0.35–0.45).
Expectation of 1 Year Survival (SQ 3)
In a group of 2,531 patients (81%; 95% CI, 79–82%), the physician expected 1-year survival after ICU ad-mission. For 609 patients (19%; 95% CI, 16–22%), the physician expected 1-year mortality. The negative pre-dicted value was 86% (95% CI, 85–87),and the positive predicted value was 60% (95% CI, 56–64). The physi-cian’s prediction of survival was more accurate for ICU and hospital admission in comparison with 1-year sur-vival (Table 2). The mean accuracy for the expectation of 1-year survival in this study was 81% (95% CI, 80– 82) with a kappa statistic of 0.43 (95% CI, 0.39–0.47).
Logistic Regression
All available patient characteristics, known to the inten-sivist at time of ICU admission, were added to a multi-variable model. Model 1, the basic model, showed the association between these patient characteristics and 1-year survival (Table 3). Excluded variables through backward selection were gender, COPD, gastrointes-tinal bleeding, and acute ICU admission. These vari-ables that did not significantly contribute to the model were excluded using a backward exclusion approach.
Model 2 shows model 1 plus SQ3. Model 2 was cor-rectly predicting the outcome for 81.5% of the cases, compared with 79.6% in model 1 (p < 0.001). This ratio was also visible in the receiver operating characteristic curves; model 1 showed an area under the curve (AUC) of 0.78, and model 2 showed an AUC of 0.82 (Fig. 2). These results showed an added value for predicting 1-year survival of the SQ besides the patient characteris-tics used in APACHE III scores.
DISCUSSION
The SUrvival PRediction In SEverly ill patients ( SURPRISE) study shows that critical care clinicians are reasonably good at predicting 1-year survival in ICU patients: the accuracy of the SQ for ICU admission,
TABLE 2.
Predicted Outcome Versus Actual Outcome for Surviving ICU Stay, Hospital Stay, and 1 Year, Respectively
Predicted Outcomes
Actual Outcomes
Not Surviving Surviving
ICU Stay
(n = 270) Hospital Stay (n = 392) (n = 713)1 yr (n = 2,870)ICU Stay Hospital Stay (n = 2,748) (n = 2,427)1 yr
Not surviving ICU stay (n = 153) 98 55
Hospital stay (n = 252) 148 104
1 yr (n = 609) 363 246
Surviving ICU stay (n = 2,987) 172 2,815
Hospital stay (n = 2,888) 244 2,644
1 yr (n = 2,531) 350 2,181
ICU stay: positive predictive value, 64.1%; negative predictive value, 94.2%; mean accuracy, 93% (95% CI, 92–94%); Kappa = 0.43 95% CI 0.37–0.49 (moderate agreement). Hospital stay: positive predictive value, 58.7%; negative predictive value, 91.6%; mean accuracy, 89% (95% CI, 88–90%); Kappa = 0.40, 95% CI, 0.35–0.45 (fair agreement). 1 yr: positive predictive value, 59.6%; negative predictive value, 86.2%; mean accuracy, 81% (95% CI, 80–82%); Kappa = 0.43, 95% CI, 0.39–0.47 (moderate agreement).
Ros et al
hospital admission, and 1-year survival in 3,140 patients was 93%, 89%, and 81% respectively. The PPV of the SQ for 1-year survival is 60%, and the NPV is 86%. The kappa statistics show fair to moderate agree-ment between the observed and predicted values.
The clinical value of the SQ in ICU patients however is challenging. Previous studies looked at the additive role of the SQ in ICU patients for predicting mortality. Hadique et al (12) used the SQ in a prediction model, besides Charlson comorbidity index and APACHE III score. They reported a strong discrimination and cal-ibration to predict 6-month mortality in medical ICU
patients (sensitivity 74%, specificity 82%, mean accu-racy 75% [range, 69–82% (p = 0.336)]). However, the SQ was answered by physicians who had knowledge of the physical patient status. Other studies have demonstrated that physicians’ predictions relate reasonably with the predictions derived from de APACHE-II and PREDICT models (6). However, in our study, the SQ was asked in an earlier phase (within 2 hr) of the ICU admission, which inherently does not incorporate the patient’s clin-ical trajectory during the first 1–2 days of ICU stay. That is why the clinical insight plays a more important role in our study. However, it should be taken into account that
TABLE 3.
Logistic Regression Model 1 Model 2 Variables OR 95% CI p OR 95% CI p Surprise question 1.06 1.05–1.07 0.00 0.19 0.16–0.24 0.00 Age 1.04 1.03–1.05 0.00 Diabetes 0.72 0.57–0.90 0.00 0.70 0.55–0.90 0.01Chronic renal insufficiency 1.33 0.82–2.17 0.25 1.20 0.72–2.03 0.48
Chronic dialysis 4.26 1.59–11.39 0.00 4.66 1.66–13.09 0.00 Confirmed infection 0.77 0.63–0.94 0.01 0.72 0.58–0.89 0.00 Cerebrovascular accident 1.87 0.96–3.66 0.07 1.62 0.79–3.34 0.19 Intracranial mass 11.73 3.59–38.32 0.00 6.95 2.03–23.86 0.00 Immunocompromised 0.59 0.45–0.79 0.00 0.52 0.39–0.70 0.00 Neoplasm 3.58 2.55–5.03 0.00 2.15 1.49–3.09 0.00 Cirrhosis 6.22 2.93–13.18 0.00 3.48 1.59–7.63 0.00 Hematologic malignancy 4.01 2.18–7.37 0.00 3.02 1.59–7.63 0.00 Cardiovascular insufficiency 2.25 1.26–4.00 0.01 1.24 0.67–2.30 0.50 Cardiopulmonary resuscitation 3.62 2.15–6.10 0.00 2.67 1.53–4.64 0.00 Respiratory insufficiency 1.81 1.20–2.72 0.00 2.21 0.78–1.88 0.41
Mechanical ventilation at start 1.81 1.47–2.23 0.00 1.63 1.31–2.01 0.00
Admission type 2.60 1.69–4.02 0.00 1.82 1.16–2.85 0.01
1.32 0.84–2.08 0.28 1.15 0.72–1.84 0.55
Nagelkerke pseudo r2, % 25.4 33.7
OR = odds ratio.
Association of patient characteristics and 1 yr. Model 1: basic model (excluded variables through backward selection were “gender, chronic obstructive pulmonary disease, gastrointestinal bleeding, and acute ICU admission”). Model 2: Model 1 plus surprise question 3.
the SQ is an useful tool, triage in critically ill patients with limited life expectancy is nuanced and usually takes more than 2 hours of clinical assessment.
All available studies who analyzed the SQ in ICU patients differ in study population, time to follow-up, and the moment of answering the SQ. In the study of Litton et al (6), the SQ was analyzed for 2-year mor-tality in 2,497 ICU patients. The included patients were young (mean, 53.8 yr) and received mechanical venti-lation frequently (86%) (6). The physician’s estimations were correct in 80.4% (95% CI 78.9–82.1), with a PPV of 57.4% and a NPV of 88.8%. The SQ was answered for each patient within the first 24 hours of ICU ad-mission. In contrast, the study of Hadique et al (12) estimated 6-month mortality in 1,049 patients. The included patients had a mean age of 61 years and an average APACHE III score of 72.4. This study popu-lation is comparable with patients of our study. They reported a mean accuracy of the physicians using the SQ of 76% (range, 67–84%) (12). However, the SQ was answered for each patient within the first 12–24
hours of ICU admission and in that way differs from our study where SQ had to be scored within the first 2 hours of admission. The moment of answering the SQ is determinative because knowing the occurrences of the first hours after the acute moment, and know-ing the APACHE score for example, may influence the estimation of the physician. Indeed, the SQ may be of clinical value if the (right) estimation was made in the acute moment.
The observed 1-year mortality in our SURPRISE study was 22.7% with a physician-predicted mortality of 19.4%. The study of Litton et al (6) shows an observed 2-year mortality of 23.4% and a physician-predicted mortality of 26.5%. They reported extremes of age, ex-tent of comorbidities, and severity of APACHE-II score as independent risk factors for an inaccurate physician prediction. However, the studied duration of mortality differs, in the SURPRISE study, the physicians tended to underpredict 1-year mortality instead of overpre-diction. This is probably due to the minimal available information at the time of prediction (within 2 hr, not knowing APACHE scores). Indeed, there was only a similar observed rate of mortality despite 1-year versus 2-year observation. This is probably due to differences in study populations (patients in the study of Litton et al [6] were younger) or because the mortality rate is higher in the first year after ICU admission compared with the second year.
In other, nonacute departments, it also seems dif-ficult to estimate the survival time of patients. A meta-analysis by White et al (9) shows a wide degree of accuracy, from poor to reasonable, reported across studies using the SQ with a time frame of 6–12 months in nonacute departments. This analysis contains 22 complete data studies (five were in oncology) with 25.718 estimates of physicians and nurses. It shows a pooled accuracy of the SQs of approximately 75%. The included studies using a shorter time frame for the SQ had a slightly better accuracy. The study of Haydar et al (13) demonstrated a NPV of 98% for the modified SQ, “Would you be surprised if this patient died in the next 30 days?”, to predict in-hospital mortality in older patients receiving care in the emergency department (ED). In our SURPRISE study, the accuracy of the SQ also decreases over time. It seems easier to predict shorter time periods, like ICU and hospital survival, than predicting 1-year survival.
Overall, studies about the SQ show controversial results for clinical practice about the prognostic value.
Figure 2. Blue: Receiver operating characteristic (ROC) curve
basic model (model 1). Area under the curve 0.778. Contains variables: age, diabetes, chronic renal insufficiency, chronic dialysis, confirmed infection, cerebrovascular accident, intracranial mass, immunocompromised, neoplasm, cirrhosis, hematologic malignancy, cardiovascular insufficiency, cardiopulmonary resuscitation, respiratory insufficiency, mechanical ventilation at start, admission type. Red: ROC curve basic model with addition of the surprise question (SQ) (model 2). Area under the curve 0.822. Contains all variables of the basic model and SQ3.
Ros et al
However, identification of patients with limited life ex-pectancy can increase the implementation of an early palliative approach and therefore improve quality of life. The study of Zeng et al (14) demonstrated that educating emergency medicine (EM) physicians about the SQ can increase the number of palliative care con-sults in the ED. These concon-sults also help to decrease costs by avoiding unwanted healthcare interventions. Ouchi et al (15) shows an additive value of the SQ es-pecially for older patients who visited the ED. They report a low PPV (32%) and high NPV (90%) with a
C-statistic of 0.67. The SURPRISE study shows higher
PPV (60%) and comparable NPV (86%) of the SQ, probably because there was more information avail-able for the intensivists in comparison with EM physi-cians. In conclusion, identifying the need for palliative care interventions by using the SQ is helpful in the decision-making process with patients and family members.
Our study has a number of strengths. First, we pro-spectively included a large cohort of ICU patients over time. Second, there were many different variables available because of the Dutch national standardized NICE registry (10). Third, ICU staffing and hospital staffing/organizations in the ICU did not change dur-ing the study nor the admission and discharge policy. The participating intensivists were a heterogeneous experienced group (median 10 yr). Several limitations of our study should also be acknowledged. First, be-cause of missing SQ data, 50 patients were excluded from the final analysis, and some patient character-istics do show significant differences (Supplement 1 Table 1, http://links.lww.com/CCX/A483). Second, we did not use a validation cohort to test our results for our prognostic model. Finally, this is a single-center study from the Netherlands. The data may be different in other settings, in particular regarding differences in ICU admission policies, ICU capacity, and local eth-ical regulations and protocols.
CONCLUSIONS
The frequently overlooked simple and cheap SQ is probably an useful tool, besides other available patient characteristics, to evaluate the expected prognosis of acutely admitted critically ill patients. As such, the SQ may be helpful in the decision-making process with
the patient and family members whether or not to es-calate or discontinue certain therapeutic interventions. 1 Department of Intensive Care Medicine, Gelre Hospitals,
Apeldoorn, The Netherlands.
2 Department of Epidemiology, Expertise Center for Intensive care Rehabilitation Apeldoorn (ExpIRA), Apeldoorn, The Netherlands.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccxjournal).
The authors have disclosed that they do not have any potential conflict of interest.
For information regarding this article, E-mail: marijke_ros@hot-mail.com
REFERENCES
1. Piers RD, Azoulay E, Ricou B, et al; APPROPRICUS Study Group of the Ethics Section of the ESICM: Perceptions of ap-propriateness of care among European and Israeli intensive care unit nurses and physicians. JAMA 2011; 306:2694–2703 2. Gulini JEHMB, Nascimento ERPD, Moritz RD, et al: Predictors
of death in an intensive care unit: Contribution to the palliative approach. Rev Esc Enferm USP 2018; 52:e03342
3. Knaus WA, Zimmerman JE, Wagner DP, et al: APACHE-acute physiology and chronic health evaluation: A physiologically based classification system. Crit Care Med 1981; 9:591–597 4. Ho KM, Knuiman M, Finn J, et al: Estimating long-term survival
of critically ill patients: The PREDICT model. PLoS One 2008; 3:e3226
5. Scholz N, Bäsler K, Saur P, et al: Outcome prediction in crit-ical care: Physicians’ prognoses vs. scoring systems. Eur J
Anaesthesiol 2004; 21:606–611
6. Litton E, Ho KM, Webb SA: Comparison of physician predic-tion with 2 prognostic scoring systems in predicting 2-year mortality after intensive care admission: A linked-data cohort study. J Crit Care 2012; 27:423.e9–423.15
7. Moss AH, Ganjoo J, Sharma S, et al: Utility of the ‘surprise’ question to identify dialysis patients with high mortality. Clin J
Am Soc Nephrol 2008; 3:1379–1384
8. Downar J, Goldman R, Pinto R, et al: The “surprise question” for predicting death in seriously ill patients: A systematic re-view and meta-analysis. CMAJ 2017; 189:E484–E493 9. White N, Kupeli N, Vickerstaff V, et al: How accurate is the
‘surprise question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med 2017; 15:139 10. Van de Klundert N, Holman R, Dongermans DA, et al: Data
resource profile: The Dutch national intensive care evaluation (NICE) registry of admissions to adult intensive care units. Int
11. Landis JR, Koch GG: The measurement of observer agree-ment for categorical data. Biometrics 1977; 33:159–174 12. Hadique S, Culp S, Sangani RG, et al: Derivation and validation of
a prognostic model to predict 6-month mortality in an intensive care unit population. Ann Am Thorac Soc 2017; 14:1556–1561 13. Haydar SA, Strout TD, Bond AG, et al: Prognostic value of a
modified surprise question designed for use in the emergency department setting. Clin Exp Emerg Med 2019; 6:70–76
14. Zeng H, Eugene P, Supino M: Would you be surprised if this patient died in the next 12 months? Using the surprise ques-tion to increase palliative care consults from the emergency department. J Palliat Care 2020; 35:221–225
15. Ouchi K, Jambaulikar G, George NR, et al: The “surprise ques-tion” asked of emergency physicians may predict 12-month mortality among older emergency department patients. J Palliat
