University of Groningen Rational clinical examination of the critically ill patient Hiemstra, Bart

(1)

University of Groningen

Rational clinical examination of the critically ill patient

Hiemstra, Bart

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.

Document Version

Publisher's PDF, also known as Version of record

Publication date: 2019

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

Hiemstra, B. (2019). Rational clinical examination of the critically ill patient. Rijksuniversiteit Groningen.

Copyright

Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.

(2)

531658-L-bw-Hiemstra 531658-L-bw-Hiemstra 531658-L-bw-Hiemstra 531658-L-bw-Hiemstra Processed on: 5-6-2019 Processed on: 5-6-2019 Processed on: 5-6-2019

Processed on: 5-6-2019 PDF page: 51PDF page: 51PDF page: 51PDF page: 51

51

4

Hiemstra B, Keus F, Wetterslev J, Gluud C, van der Horst ICC

Submitted

Statistical analysis plans

for observational studies

(3)

52

Abstract Background

All clinical research benefits from transparency. Validity of studies may increase by prospective registration of protocols to prevent outcome reporting bias and by publication of statistical analysis plans (SAPs) before data have been accessed preventing data-driven analyses.

Main message

Like clinical trials, recommendations for SAPs for observational studies increase the validity of findings. We appraised the applicability of recently developed guidelines for the content of SAPs for clinical trials to SAPs for observational studies. Of the 32 items recommended for a SAP of a clinical trial, 30 items (94%) were identically applicable to a SAP of an observational study. Sample size estimations and adjustment for multiplicity are equally important in observational studies and clinical trials as both types of studies usually address multiple hypotheses. Only two items (6%) regarding issues of randomisation and definition of adherence to the intervention did not seem applicable to observational studies. We suggest instead to include one new item specifically applicable to observational studies to be addressed in a SAP, describing how adjustment for possible confounders will be handled in the analyses.

Conclusions

With only few amendments, the contents of the guideline for SAP of a clinical trial can be applied to and is equally needed for an observational study. We suggest SAPs should be equally required for observational studies and clinical trials to increase their validity.

(4)

53

Background

Transparency is considered fundamental for the reproducibility of any research finding.1_Initiatives such as SPIRIT, CONSORT, PRISMA, and PROSPERO have contributed to transparent reporting of protocols and findings of randomised clinical trials and systematic reviews.2-5_{Still, the multitude} of decisions taken during the statistical analysis phase of any study have been shown to impact on results and conclusions, irrespective of pre-published protocols.6_{While any protocol for a} clinical study should include the principle features of the statistical analysis of the data, a statistical analysis plan (SAP) should fully outline the details of all planned analyses, including any additional analyses. Recently, Gamble and colleagues used a Delphi survey to reach consensus and provide recommendations for a minimum set of items that should be addressed in a SAP for a randomised clinical trial.7

Observational studies are frequently the source for multiple statistical analyses and reports. Guidelines for reporting such as STROBE, TRIPOD or STARD are key to transparent reporting of findings of observational studies,8-10 _{but these do not reduce the number of possible decisions} taken during the analysis phase of such studies. Like randomised clinical trials, the validity of conclusions of cohort studies is likely to improve by use of published SAPs to prevent data-driven analyses.1,11_{Journals now encourage researchers to preregister observational studies and SAPs,} 11-15_{but there are no guidelines on the required content of the latter.}

Therefore, we argue that SAP guidelines should also be developed for observational studies. In the absence of such a guideline, we appraised and modified recently developed recommendations for the content of SAPs for clinical trials to be used for observational studies.

Main text: recommended content of SAPs in observational studies

We have appraised the recommendations for the content of SAPs for clinical trials and assessed the applicability of each section to be used for an observational study (Table 1). We also screened which further items were absolutely needed.

(5)

54

Table 1. Applicability of recommend content of statistical analysis plans for clinical trials to observational studies

CHAPTER 4

Table 1.Applicability of recommend content of statistical analysis plans for clinical trials to observational studies.

Section/Item Index Description for clinical trials Description for observational studies

Section 1: Administrative information

Title and study

registration 1a Descriptive title that matches the protocol, with SAP either as a forerunner or subtitle, and trial acronym Descriptive title that matches the protocol, with SAP either as a forerunner or subtitle, and study acronym

1b Trial registration number Study registration number

SAP version 2 SAP version number with dates Unchanged Protocol version 3 Reference to version of protocol being used Unchanged SAP revisions 4a SAP revision history Unchanged 4b Justification for each SAP revision Unchanged 4c Timing of SAP revisions in relation to interim analyses,

etc. Timing of SAP revisions in relation to planned repetitive analyses

Roles and responsibility 5 Names, affiliations, and roles of SAP contributors Unchanged Signatures of: 6a Person writing the SAP Unchanged 6b Senior statistician responsible Unchanged 6c Chief investigator/clinical lead Unchanged

Section 2: Introduction

Background and

rationale 7 Synopsis of trial background and rationale including a brief description of research question and brief justification for undertaking the trial

Synopsis of study background and rationale including a brief description of research question and brief justification for undertaking the study

Objectives 8 Description of specific objectives and hypotheses Description of specific objectives and hypotheses, including secondary objectives

Section 3: Study methods

Study design 9 Brief description of trial design including type of trial (e.g. parallel group, multi-arm, crossover, factorial and allocation ratio and may include brief description of interventions)

Brief description of study design including type of study (e.g. case-control,

cross-sectional or cohort study)

Randomization 10 Randomization details, e.g., whether any minimization or stratification occurred (including stratifying factors used or the location of that information if it is not held within the SAP)

Not applicable

Sample size 11 Full sample size calculation or reference to sample size

calculation in protocol (instead of replication in SAP) Full sample size, power or detectable difference calculation or reference to

sample size calculation in protocol (instead of replication in SAP)*

Framework 12 Superiority, equivalence, or noninferiority hypothesis testing framework, including which comparisons will be presented on this basis

Unchanged*

Statistical interim analyses and stopping guidance

13a Information on interim analyses specifying what interim analyses will be carried out and listing of time points

Information on repetitive analyses specifying what repetitive analyses will be carried out and listing of time points* 13b Any planned adjustment of the significance level due

to interim analysis Unchanged

13c Details of guidelines for stopping the trial early Details of guidelines for stopping the study early

Timing of final analysis 14 Timing of final analysis, e.g., all outcomes analysed collectively or timing stratified by planned length of follow-up

Unchanged*

Timing of outcome

assessments 15 Time points at which the outcomes are measured including visit “windows” Unchanged

Section 4: Statistical principles

Confidence intervals

and P-values 16 Level of statistical significance Unchanged* 17 Description and rationale for any adjustment for

multiplicity and, if so, detailing how the type 1 error is to be controlled

Unchanged* 18 Confidence interval to be reported Unchanged Adherence and

protocol deviations 19a Definition of adherence to the intervention and how this is assessed including extent of exposure Not applicable 19b Description of how adherence to the intervention will

be presented Not applicable

19c Definition of protocol deviations for the trial Definition of protocol deviations for the

study

19d Description of which protocol deviations will be

(6)

55 This table was adapted with permission from Gamble et al 7. Italic text highlights a rephrased word/sentence in the modified description for observational studies. An asterisk (*) indicates that a more elaborate description is present in our manuscript.

This table was adapted with permission from Gamble et al.7_{Italic text highlights a rephrased word/sentence in the modified}

description for observational studies. An asterisk (*) indicates that a more elaborate description is present in our manuscript. Analysis populations 20 Definition of analysis populations, e.g., intention to

treat, per protocol, complete case, safety Definition of analysis populations, e.g., per protocol, complete case, safety

Section 5: Study Population

Screening data 21 Reporting of screening data (if collected) to describe

representativeness of trial sample Unchanged Eligibility 22 Summary of eligibility criteria Unchanged Recruitment 23 Information to be included in the CONSORT flow

diagram Information to be included in the STROBE flow diagram Withdrawal/follow-up 24a Level of withdrawal, e.g., from intervention and/or

from follow-up Unchanged 24b Timing of withdrawal/lost to follow-up data Unchanged 24c Reasons and details of how withdrawal/lost to

follow-up data will be presented Unchanged Baseline patient

characteristics 25a List of baseline characteristics to be summarized Unchanged 25b Details of how baseline characteristics will be

descriptively summarized Unchanged

Potential confounding

covariates - - A description of potential confounding covariates and how these will be dealt with*

Section 6: Analysis

Outcome definitions List and describe each primary and secondary outcome including details of:

26a Specification of outcomes and timings. If applicable include the order of importance of primary or key secondary end points (e.g., order in which they will be tested)

Unchanged

26b Specific measurement and units (e.g., glucose control, HbA1c [mmol/mol or %])

Unchanged 26c Any calculation or transformation used to derive the

outcome (e.g., change from baseline, QoL score, time to event, logarithm, etc)

Unchanged

Analysis methods 27a What analysis method will be used and how the

treatment effects will be presented Unchanged* 27b Any adjustment for covariates Unchanged 27c Methods used for assumptions to be checked for

statistical methods Unchanged 27d Details of alternative methods to be used if

distributional assumptions do not hold, e.g., normality, proportional hazards, etc

Unchanged

27e Any planned sensitivity analyses for each outcome

where applicable Unchanged* 27f Any planned subgroup analyses for each outcome

including how subgroups are defined Unchanged* Missing data 28 Reporting and assumptions/statistical methods to

handle missing data (e.g., multiple imputation) Unchanged* Additional analyses 29 Details of any additional statistical analyses required,

e.g. complier-average causal effect analysis Unchanged Harms 30 Sufficient detail on summarizing safety data, e.g.

information on severity, expectedness, and causality; details of how adverse events are coded or categorized; how adverse event data will be analysed, i.e. grade ¾ only, incidence case analysis, intervention emergent analysis

Sufficient detail on summarizing safety data, e.g. information on severity, expectedness, and associations; details of how adverse events are scored; how adverse event data will be analysed and

the follow-up time.*

Statistical software 31 Details of statistical packages to be used to carry out

analysis Unchanged

References 32a References to be provided for nonstandard statistical

methods Unchanged

32b Reference to Data Management Plan Unchanged 32c Reference to the Trial Master File and Statistical

Master File Reference to the Study Master File and Statistical Master File 32d Reference to other standard operation procedures to

(7)

56

Section 1: Administrative information

The administrative information section in a SAP for an observational study is equally applicable to the content of a SAP for a randomised clinical trial. Item 1a and 1b were renamed while the content remained the same. For item 1b; a protocol of an observational study can be registered in a dedicated database (e.g. clinicaltrials.gov, researchregistry.com) alike randomised clinical trials.14,16 _{The description of item 4 was rephrased since in observational studies usually no interim} analyses are planned (table 1). All other items, names and descriptions were left unchanged. Section 2: Introduction

The introduction section in a SAP for an observational study is equal to the content of a SAP for a randomised clinical trial.

Section 3: Study methods

Sample size

Unlike randomised clinical trials that calculate a sample size to study an intervention effect taking power into consideration, the sample sizes of most observational studies are influenced by other factors (e.g. resources, time restrictions, convenience). Accordingly, most observational studies will have a given sample size and usually this is large affording enough power. The STROBE guidelines only expect authors to explain how the study size was arrived at,8_{which may reduce} the incentive to conduct sample size calculations for observational studies.

We suggest providing a sample size calculation for the primary analysis of the observational study and to provide power calculations for the secondary analyses to limit random errors. The power calculations necessitate a definition of a minimally important difference or intervention effect in the presence of a given sample size. Any power calculation provides the chance of a type-II error (false negative findings), while a detectable difference may be clinically more informative. For example, it shows the minimal relative risk that can be detected with the specified power and sample size given a type I error probability α.

Framework

While causality can never be proven in observational studies, observed associations may fuel hypotheses that later can be tested in randomised clinical trials.17_{Although the vast majority} of observational studies test for superiority, there are some that address equivalence and non-inferiority hypotheses.18-21_{Of course, confounding will always be present in any of these} frameworks. Nevertheless, a SAP should describe whether the relevant hypothesis was assessed for superiority, equivalence or non-inferiority.

Statistical interim analyses and stopping guidance

Interim analyses are typically known to guide randomised clinical trials for early stopping due to benefit, harm or futility of tested interventions. Investigators are ethically obliged to conduct interim analyses to reduce study patients’ exposure to an inferior intervention. While there is no intervention component in observational studies which can be halted, there may be incentives to CHAPTER 4

(8)

57 perform interim analyses for early stopping of further continued (costly) data collections equally due to already clear observed associations or futility. Further, observational studies may be subject to repeated testing of accumulating data, which needs adjustment of significance levels to reduce inflated type-I errors (false positive findings), such as described by O’Brien & Fleming.22 These methods should be described in the SAP.

Timing of final analysis

A SAP for a randomised clinical trial should be published before the trial database is unlocked for analyses and before the interventions are decoded. Likewise, prospective observational studies should also lock their databases until the SAP has been published and the study is finished. Randomised clinical trials have a natural advantage that interventions can be coded during the statistical analyses. Such coding of interventions is usually not in question in observational studies, but it should be possible to use coding for several covariates (at least dichotomous and categorical). The absence of blinding and the often-direct access to the database allows thorough data inspection, even before the study is finished or a SAP is written. Still, a detailed SAP provides transparency on the intended analyses steps and may prevent ‘fishing’ for statistically significant predictors in analyses or other manipulations of the data. Any analysis that was not prespecified in the protocol and/or SAP can only be explorative in nature, which should be described accordingly (i.e. exploratory or post-hoc analysis).

Section 4: Statistical principles

Multiplicity and type I error

Multiplicity issues are similar in randomised clinical trials and in observational studies, but rarely addressed in the latter. Most observational studies ignore multiplicity issues by testing in multiple analyses at the same conventional P<0.05 significance level. This increases the risks of a family wise error rate (FWER), that is the type I error of at least one false positive finding, which is reduced by adjustments such as according to Bonferroni or Šidák.23,24_{Even though International Conference} on Harmonization of Good Clinical Practice guidelines recommend full Bonferroni adjustment,25 such an adjustment may be too conservative in correlated outcomes of observational studies.26 For example, our Simple Intensive Care Studies (SICS)-I addresses six different primary outcomes spread out across 13 hypotheses.27_{Our outcomes cardiac output, acute kidney injury, and} mortality are all affected by a patient’s haemodynamic status, so that most outcomes will probably be positively correlated. We will apply an adjustment to prevent an increased FWER due to multiplicity based on the total numbers of different primary outcomes tested. We chose for a pragmatic approach by Jakobsen and colleagues because it was easy to adopt and seems to come closer to a statistical significance level able to limit the FWER.28_{This approach suggests that} the significance threshold α for each tested outcome lies somewhere between the unadjusted threshold (most often 0.05) and the Bonferroni adjusted threshold. The adjusted threshold for significance is calculated by dividing the pre-specified P-value threshold with the value halfway between 1 (no adjustment) and the number of primary outcome comparisons (fullBonferroni adjustment):

(9)

58

��

�� _{2 �}

Wherein:

• α is the unadjusted threshold for significance, usually 0.05

• m is the number of primary outcomes or tests (used) in the same cohort (in this case: six) The threshold for significance in the SICS-Icohortis:

In our papers, a P value below 0.015 indicates a statistically significant effect. When we find 0.015 ≤ P ≤ 0.05, we will consider this association of suggestive significance and will emphasise the risk of the FWER being increased beyond 0.05.

Section 5: Study population

Screening data

It is necessary to elucidate the numbers of eligible and included patients of an observational study in a flow diagram, preferably according to the STROBE recommendations.29

Potential confounding covariates

Results of observational studies can be seriously biased by confounding covariates. The randomisation procedure is used in randomised clinical trials to remove the imbalance in observed and unobserved confounders between the allocated groups, although success can never be guaranteed.30_{The STROBE guidelines advocate to address the rate of confounding;} however, it was recently shown that adherence to this statement is suboptimal.31_{A SAP could} serve to predefine confounders, and how to address the expected rate of residual confounding by adjustment, or stratification.

Section 6: Analysis

Analysis methods

Analysis methods of clinical trials and observational studies are different, yet both study types are suspicious of selective reporting when no SAP is written.32_{Many decisions are needed during the} analysis phase of an observational study and all that can be foreseen should be prespecified. An extensive description of the planned statistical analyses, all covariates and considerations need to be prespecified and detailed, which can only be done in a SAP. The usually short statistical analysis section of a manuscript does not allow a detailed explanation, nor can it guarantee the prespecified status of the analysis.

�� 0.05

�� _{2 �}� 0.0�� 0.0�5

(10)

59 Sensitivity and subgroup analyses

The cost- and time-intensive nature of a randomised clinical trial necessitates a strict protocol in which all sensitivity and subgroup analyses are (usually) specified. In observational studies, these additional analyses are often not specified beforehand. A SAP is an opportunity for authors to prove that they had prespecified intentions of their sensitivity and subgroup analyses.

Missing data

Observational studies are particularly prone to missing data, but often do not address the mechanism of missing values. Complete case analyses in the presence of missing data are associated with bias, when data are not missing completely at random.33,34_{Tests to identify the} patterns and type of missing data, and the statistical methods to handle missing values should be described in a SAP. Examples are multiple imputations for data missing at random or worst-best and best-worst case scenarios for data missing not at random.34,35

Harms

Randomised clinical trials are costly and therefore limited in size and length of follow-up, so that rare harms or late harms (e.g. after decades) remain undetected. Observational studies and post- marketing phase IV randomised clinical trials are much more suitable for detection of rare or late harms,35_{of which the cardiovascular harms of clarithromycin in patients with stable coronary} heart disease or cyclooxygenase-2(Cox-2) inhibitors are good examples.36,37

Applicability of clinical trial guidelines

Of the 32 proposed items by Gamble and colleagues (Table 1)7_{, 30 items (94%) were also more} or less directly applicable to SAPs for observational studies (Table 2). Some of these 30 items differ between trials and observational studies, mainly from a methodological point of view. We enclosed our SAP in the supplements for illustrative purposes, so that it may serve as an example document for other observational studies.

Main reasons for ignoring two items (6%) were that these recommendations were specifically limited to trials, that is descriptions on randomisation and definition of adherence to the intervention.

(11)

60

Table 2. Recommended content of statistical analysis plans for observational studies

13

Table 2. Recommended content of statistical analysis plans for observational studies

Section/Item Index Description for observational studies

Section 1: Administrative information

Title and study registration 1a Descriptive title that matches the protocol, with SAP either as a forerunner or subtitle, and study acronym

1b Study registration number SAP version 2 SAP version number with dates

Protocol version 3 Reference to version of protocol being used SAP revisions 4a SAP revision history

4b Justification for each SAP revision

4c Timing of SAP revisions in relation to planned repetitive analyses Roles and responsibility 5 Names, affiliations, and roles of SAP contributors

Signatures of: 6a Person writing the SAP 6b Senior statistician responsible 6c Chief investigator/clinical lead Section 2: Introduction Background and rationale 7 Synopsis of study background and rationale including a brief description of research question and brief justification for undertaking the study Objectives 8 Description of specific objectives and hypotheses, including secondary objectives Section 3: Study methods Study design 9 Brief description of study design including type of study (e.g. case‐control, cross‐sectional or cohort study) Sample size 10 Full sample size calculation or reference to sample size calculation in protocol (instead of replication in SAP)* Framework 11 Superiority, equivalence, or noninferiority hypothesis testing framework, including which comparisons will be presented on this basis Statistical interim analyses and stopping guidance 12a Information on repetitive analyses specifying what repetitive analyses will be carried out and listing of time points 12b Any planned adjustment of the significance level due to interim analysis 12c Details of guidelines for stopping the study early Timing of final analysis 13 Timing of final analysis, e.g., all outcomes analysed collectively or timing stratified by planned length of follow‐up* Timing of outcome assessments 14 Time points at which the outcomes are measured including visit “windows” Section 4: Statistical principles Confidence intervals and P‐ values 15 Level of statistical significance* 16 Description and rationale for any adjustment for multiplicity and, if so, detailing how the type 1 error is to be controlled* 17 Confidence interval to be reported Adherence and protocol deviations 18a Definition of protocol deviations for the trial 18b Description of which protocol deviations will be summarized Analysis populations 19 Definition of analysis populations, e.g., intention to treat, per protocol, complete case, safety Section 5: Study Population Screening data 20 Reporting of screening data (if collected) to describe representativeness of trial sample Eligibility 21 Summary of eligibility criteria Recruitment 22 Information to be included in the STROBE flow diagram* CHAPTER 4

(12)

61

14

Withdrawal/follow‐up 23a Level of withdrawal, e.g., from intervention and/or from follow‐up 23b Timing of withdrawal/lost to follow‐up data 23c Reasons and details of how withdrawal/lost to follow‐up data will be presented Baseline patient characteristics 24a List of baseline characteristics to be summarized 24b Details of how baseline characteristics will be descriptively summarized Potential confounding covariates 25 A description of potential confounding covariates and how these will be dealt with* Section 6: Analysis Outcome definitions List and describe each primary and secondary outcome including details of: 26a Specification of outcomes and timings. If applicable include the order of importance of primary or key secondary end points (e.g., order in which they will be tested) 26b Specific measurement and units (e.g., glucose control, HbA1c[mmol/mol or %]) 26c Any calculation or transformation used to derive the outcome (e.g., change from baseline, QoL score, time to event, logarithm, etc)

Analysis methods 27a What analysis method will be used and how the treatment effects will be presented* 27b Any adjustment for covariates 27c Methods used for assumptions to be checked for statistical methods 27d Details of alternative methods to be used if distributional assumptions do not hold, e.g., normality, proportional hazards, etc 27e Any planned sensitivity analyses for each outcome where applicable* 27f Any planned subgroup analyses for each outcome including how subgroups are defined* Missing data 28 Reporting and assumptions/statistical methods to handle missing data (e.g., multiple imputation)* Additional analyses 29 Details of any additional statistical analyses required, e.g. complier‐average causal effect analysis Harms 30 Sufficient detail on summarizing safety data, e.g. information on severity, expectedness, and associations; details of how adverse events are scored; how adverse event data will be analysed and the follow‐up time. Statistical software 31 Details of statistical packages to be used to carry out analysis References 32a References to be provided for nonstandard statistical methods

32b Reference to Data Management Plan 32c Reference to the Study Master File and Statistical Master File 32d Reference to other standard operation procedures to be adhered to This table was adopted from and created with permission from Gamble et al [7]. An asterisk (*) indicates that a more elaborate description is present in our manuscript.

This table was adopted from and created with permission from Gamble et al.7_{An asterisk (*) indicates that a more}

(13)

62

Discussion

Preregistration of protocols and SAPs for observational studies has been debated.12-15,38-48 _Opposing authors state that preregistration creates the false assumption that data are of high quality, would discourage publication of important accidental findings, and would delay these publications due to bureaucratic procedures.38-44_{Authors in favour argue that preregistration of protocols and SAPs} distinguishes prespecified hypotheses from data dredging expeditions, ensures that methods can be replicated and findings confirmed, and reduces selective outcome reporting and publication bias.45-49_{Our present recommendations showing the large similarities between randomised} clinical trials and observational studies are parallel to our previous recommendations to publicly and transparently communicate all aspects of randomised clinical trials as well as observational studies from protocol to final results.1

Observational studies are prone to confounding by indication, residual confounding and flaws in data collection.50_{We argue that publication of a SAP increases the chance that hypotheses} are adequately powered and investigated in the appropriate study population in which also all known confounders, mediators and covariates are measured.46,51_{Since credibility and replicability} of findings in observational studies are a concern to many,11-15,46,52_{the publication of a SAP allows} better validation of findings in independent cohorts in an identical methodological and statistical manner. Furthermore, the concern that important findings will remain unpublished is less worrying than a lot of accidental findings getting published. For the credibility of an ‘eye-catching’ finding to prevail it still has to be replicated in a methodological sound study with an a priori hypothesis and an adequate statistical power. Irrespective of its potential benefits, publishing a SAP would at least do no harm and may be seen as an independent determinant of validity. Conclusion

Both a protocol and a SAP in the public domain are equally helpful for both observational studies and randomised clinical trials.45_{By applying the guideline for the content of SAPs for clinical trials} to our observational study we can conclude that more than 90% of the recommended content based on an extensive Delphi survey suits an observational study as well. There are only few adjustments needed for guidance of a SAP for observational studies when compared to a SAP for randomised clinical trials. In absence of SAP guidelines, we think that current recommend contents of SAPs for clinical trials could serve as a structure for SAPs of observational studies. Acknowledgments

We thank all authors involved in writing the SAP of the SICS-I for their collaboration: Prof. dr. Pim van der Harst, Dr. Ilja M Nolte, Prof. dr. Harold Snieder, José N Alves Castela Cardoso Forte, and Chris HL Thio.

(14)

63

References

Skoog M, Saarimäki J, Gluud C, Scheinin M, Erlendsson K, Aamdal S. Transparency and registration in clinical research in the nordic countries (report). NordForsk: Nordic Trial Alliance. 2015:1-108.

Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. Int J Surg. 2011;9(8):672-677.

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-1012.

Chan AW, Tetzlaff JM, Gotzsche PC, et al. SPIRIT 2013 explanation and elaboration: Guidance for protocols of clinical trials. BMJ. 2013;346:e7586.

Chien PF, Khan KS, Siassakos D. Registration of systematic reviews: PROSPERO. BJOG. 2012;119(8):903-905. Ebrahim S, Sohani ZN, Montoya L, et al. Reanalyses of randomized clinical trial data. JAMA. 2014;312(10):1024-1032.

Gamble C, Krishan A, Stocken D, et al. Guidelines for the content of statistical analysis plans in clinical trials. JAMA. 2017;318(23):2337-2343.

von Elm E, Altman DG, Egger M, et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. BMJ. 2007;335(7624):806-808. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. J Clin Epidemiol. 2015;68(2):134-143. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.

PLOS Medicine Editors. Observational studies: Getting clear about transparency. PLoS Med. 2014;11(8):e1001711.

Loder E, Groves T, Macauley D. Registration of observational studies. BMJ. 2010;340:c950. The Lancet. Should protocols for observational research be registered? Lancet. 2010;375(9712):1. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: Is it time? CMAJ. 2010;182(15):1638-1642.

Eisenach JC, Kheterpal S, Houle TT. Reporting of observational research in ANESTHESIOLOGY: The importance of the analysis plan. Anesthesiology. 2016;124(5):998-1000.

Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, Gluud C. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956-958.

Garattini S, Jakobsen JC, Wetterslev J, et al. Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13-21.

Chu C, Umanski G, Blank A, Grossberg R, Selwyn PA. HIV-infected patients and treatment outcomes: An equivalence study of community-located, primary care-based HIV treatment vs. hospital-based specialty care in the bronx, new york. AIDS Care. 2010;22(12):1522-1529.

Pyo JH, Lee H, Min BH, et al. Long-term outcome of endoscopic resection vs. surgery for early gastric cancer: A non-inferiority-matched cohort study. Am J Gastroenterol. 2016;111(2):240-249.

Parker K, Perikala V, Aminazad A, et al. Models of care for non-invasive ventilation in the acute COPD comparison of three tertiary hospitals (ACT3) study. Respirology. 2017.

Austevoll IM, Gjestad R, Brox JI, et al. The effectiveness of decompression alone compared with additional fusion for lumbar spinal stenosis with degenerative spondylolisthesis: A pragmatic comparative non- inferiority observational study from the norwegian registry for spine surgery. Eur Spine J. 2017;26(2):404-413.

O’Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics. 1979;35(3):549-556.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

(15)

64

Bland JM, Altman DG. Multiple significance tests: The bonferroni method. BMJ. 1995;310(6973):170. Šidák Z. Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association. 1967;62(318):626-633.

International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use (ICH) adopts consolidated guideline on good clinical practice in the conduct of clinical trials on medicinal products for human use. Int Dig Health Legis. 1997;48(2):231-234.

Bender R, Lange S. Multiple test procedures other than bonferroni’s deserve wider use. BMJ. 1999;318(7183):600-601.

Hiemstra B, Eck RJ, Koster G, et al. Clinical examination, critical care ultrasonography and outcomes in the critically ill: Cohort profile of the simple intensive care studies-I. BMJ Open. 2017;7(9):e017170.

Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Med Res Methodol. 2014;14:120.

Vandenbroucke JP, von Elm E, Altman DG, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): Explanation and elaboration. Int J Surg. 2014;12(12):1500-1524.

Nguyen TL, Collins GS, Lamy A, et al. Simple randomization did not protect against bias in smaller trials. J Clin Epidemiol. 2017;84:105-113.

Pouwels KB, Widyakusuma NN, Groenwold RH, Hak E. Quality of reporting of confounding remained suboptimal after the STROBE guideline. J Clin Epidemiol. 2016;69:217-224.

Greenberg L, Jairath V, Pearse R, Kahan BC. Pre-specification of statistical analysis approaches in published clinical trial protocols was inadequate. J Clin Epidemiol. 2018;101:53-60.

Perkins NJ, Cole SR, Harel O, et al. Principled approaches to missing data in epidemiologic studies. Am J Epidemiol. 2017.

Jakobsen JC, Gluud C, Wetterslev J, Winkel P. When and how should multiple imputation be used for handling missing data in randomised clinical trials - a practical guide with flowcharts. BMC Med Res Methodol. 2017;17(1):162.

McCulloch P, Altman DG, Campbell WB, et al. No surgical innovation without evaluation: The IDEAL recommendations. Lancet. 2009;374(9695):1105-1112.

van Staa TP, Smeeth L, Persson I, Parkinson J, Leufkens HG. What is the harm-benefit ratio of cox-2 inhibitors? Int J Epidemiol. 2008;37(2):405-413.

Jespersen CM, Als-Nielsen B, Damgaard M, et al. Randomised placebo controlled multicentre trial to assess short term clarithromycin for patients with stable coronary heart disease: CLARICOR trial. BMJ. 2006;332(7532):22-27.

Editors. The registration of observational studies--when metaphors go bad. Epidemiology. 2010;21(5):607- 609.

Savitz DA. Registration of observational studies does not enhance validity. Clin Pharmacol Ther. 2011;90(5):646-648.

Lash TL. Preregistration of study protocols is unlikely to improve the yield from our science, but other strategies might. Epidemiology. 2010;21(5):612-613.

Vandenbroucke JP. Preregistration of epidemiologic studies: An ill-founded mix of ideas. Epidemiology. 2010;21(5):619-620.

Vandenbroucke JP. Registering observational research: Second thoughts. Lancet. 2010;375(9719):982- 983. Pearce N. Registration of protocols for observational research is unnecessary and would do more harm than good. Occup Environ Med. 2011;68(2):86-88.

Lash TL, Vandenbroucke JP. Should preregistration of epidemiologic study protocols become compulsory? reflections and a counterproposal. Epidemiology. 2012;23(2):184-188.

CHAPTER 4 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

(16)

65 Dal-Re R, Ioannidis JP, Bracken MB, et al. Making prospective registration of observational research a reality. Sci Transl Med. 2014;6(224):224cm1.

Bracken MB. Preregistration of epidemiology protocols: A commentary in support. Epidemiology. 2011;22(2):135-137.

Thomas L, Peterson ED. The value of statistical analysis plans in observational research: Defining high- quality research from the start. JAMA. 2012;308(8):773-774.

Onukwugha E. Improving confidence in observational studies : Should statistical analysis plans be made publicly available? Pharmacoeconomics. 2013;31(3):177-179.

Ioannidis JP. The importance of potential studies that have not existed and registration of observational data sets. JAMA. 2012;308(6):575-576.

Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess. 2003;7(27):173.

Ioannidis JP. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640-648. Schoenfeld JD, Ioannidis JP. Is everything we eat associated with cancer? A systematic cookbook review. Am J Clin Nutr. 2013;97(1):127-134. 45 46 47 48 49 50 51 52

(17)

(18)

67 Hiemstra B, Thio CHL, Nolte IM, Castela Forte JN, van der Harst P, Snieder H, Wetterslev J, Keus F, van der Horst ICC

Published at clinicaltrials.gov: NCT02912624

van der Harst P, Snieder H, Wetterslev J, Keus F, van der Horst ICC

(19)

(20)

69 Full study title

Acronym

Local project number Clinicaltrials.gov number Study protocol version SAP version

SAP revision history SAP revision justification SAP revision timing

1.2. Roles and responsibility Author

Statistician

Principle investigator Contributors and roles

A prospective observational study on the value of conventional hemodynamic parameters in estimating cardiac output and predicting mortality in critically ill patients

Simple Intensive Care Studies-I (SICS-I) 201500144 NCT02912624 1.1 (23 September 2016) 1.0 (3 April 2018) None

-Revision will be conducted after publishing the two main manuscripts. This revision will concern detailed SAPs for each sub-study.

Bart Hiemstra1 Ilja M. Nolte2

Iwan C.C. van der Horst1

Pim van der Harst3_{: revised the SAP}

Frederik Keus1_{: contributed to the design and revised the SAP} Harold Snieder2_{: revised the SAP}

Chris HL Tio2_{: main contributor to the section ‘missing data’} José N Alves Castela Cardoso Forte1_{: drafted the section} ‘additional analyses: machine learning’

Jørn Wetterslev4_{: contributed to the design and revised the} SAP. Main contributor to the sections ‘sample size, power, and detectable difference’ and ‘statistical significance and multiple testing’

(21)

70

Affiliations

1.3. Signatures

We the undersigned, certify that we read this SAP and approve it as adequate in scope of the main- analyses of the SICS-I.

1.3.1. Author Name: ... Date: 1.3.2. Statistician Name: ... Date: 1.3.3. Principle investigator Name: ... Date:

1_{Department of Critical Care, University of Groningen,} University Medical Center Groningen, Groningen, The Netherlands

2_{Department of Epidemiology, University of Groningen,} University Medical Center Groningen, Groningen, The Netherlands

3 _{Department of Cardiology, University of Groningen,} University Medical Center Groningen, Groningen, The Netherlands

4_{The Copenhagen Trial Unit (CTU), Centre for Clinical} Intervention Research, Copenhagen, Denmark

(22)

71 About one-third of all critically ill patients suffer from circulatory shock, which places them at increased risks of multi-organ failure, long-term morbidity, and mortality.1,2_{Combinations of clinical,} haemodynamic and biochemical variables are recommended for establishing the diagnosis and instigation of treatment.3,4_{If necessary, more advanced and sequential haemodynamic} assessments using critical care ultrasound (CCUS) as preferred modality are recommended.3-6 Clinical examination in the critically ill comprises frequent measurement of heart rate, blood pressure, body temperature, skin perfusion, urine output and mental status.3_{Daily use of clinical} examination (in any patient) for diagnostic purposes contrasts with the limited number and quality of studies, so that the level of evidence for use of clinical examination in the critically ill is considered best practice.3_{Previous studies have suggested different prognostic or diagnostic} variables and many studies have analysed single or dual variable associations, while no research has evaluated their additional value on top of the accepted predictors.7_{The reason for inconsistency} of results in these studies potentially originate from several methodological flaws, including improper research design, lack of confirmation cohorts, and power and sample size issues. The additive diagnostic and prognostic value of combinations of clinical, biochemical and haemodynamic variables remains to be established with a higher quality of evidence. These variables have never been evaluated collectively in a large, broad, prospective cohort of critically ill patients. Therefore, we established the Simple Intensive Care Studies I (SICS-I) with the aim to evaluate the diagnostic and prognostic value of a comprehensive selection of clinical and haemodynamic variables in the critically ill.7

Prospective registration of protocols of observational studies are promoted to prevent outcome reporting bias.8,9_{Likewise, prospective publication of a detailed statistical analysis plan (SAP) is} encouraged to prevent data-driven analyses.9-11

2.2. Objectives

2.2.1. Objectives and research questions

The objective of the SICS-I study was to establish a cohort with a dual aim: to evaluate the (1) diagnostic and (2) prognostic value of a comprehensive selection of clinical examination, haemodynamic and biochemical variables in the critically ill. More specific, the two research questions of the basic study were (1): which combination of clinical examination findings is associated with cardiac index measured with CCUS? And (2): which combination of clinical examination, haemodynamic and biochemical variables is associated with 90-day mortality? In the basic study of the SICS we collected a broad number of clinical examination, haemodynamic and biochemical variables, and used CCUS to only measure cardiac output. The infrastructure and design enabled (temporarily) addition of sub-studies in which additional variables were collected. Research questions of the sub-studies all address the overall aim of the SICS-I cohort (Table 1).

(23)

72

2.2.2. Hypotheses

The hypothesis of research question 1, i.e. the diagnostic study, are:

• Null hypothesis: there is no true correlation between any single or a combination of clinical examination findings and cardiac index measured with CCUS

• Alternative hypothesis: cardiac index measured with CCUS is associated with one or a combination of clinical examination findings

The hypothesis of research question 2, i.e. the prognostic study, are:

• Null hypothesis: clinical examination, biochemical and haemodynamic variables are not associated with 90-day mortality

• Alternative hypothesis: clinical examination, haemodynamic and biochemical variables are associated with 90-day mortality

2.2.3. Scope

This SAP will be the guiding document for the analyses that will be conducted in the basic study. We intend to present the results of the two primary aims in separate manuscripts. All the aims and research questions of the sub-studies will be included in the appendix of this SAP, and we aim to present the SAPs of the sub-studies as an addendum in the future.

3. Study methods

3.1. General study design and plan

The SICS-I is a prospective cohort study which is conducted in the department of critical care of the University Medical Center Groningen (UMCG). The entire study was purely observational in design; no interventions were applied as part of the study protocol.

The protocol of this study was published on the website of the department of critical care of our hospital before the start of the study (project number: 201500144) and registered at clinicaltrials.gov (NCT02912624). This analysis plan has been written while data collection was ongoing, but before full access to the study database. For our design paper, we only extracted baseline data and we did not have access to the validated outcome data.

3.2. Sample size, power and detectable difference

There are no previous studies which included combinations of clinical examination, haemodynamic and biochemical variables into one model for estimation of cardiac output and mortality. This makes it difficult to calculate sample size based on previous literature. Alternatively, we made an estimation of the power of our multivariable models given the set sample size of our cohort.

3.2.1. Diagnostic study

We are planning our main analysis with 1,075 patients in which we will regress their values of cardiac index against clinical examination findings. For this power calculation, we used the clinical examination variables urine output and capillary refill time an example.7_{In our design paper, the} standard deviation (SD) of cardiac index was 0.99 and the SD of urine output was 0.98, with a

(24)

73 was regressed against capillary refill time. If the true slope of the line obtained by regressing cardiac index against urine output or capillary refill time is 0.10, we will be able to reject the null hypothesis that this slope equals zero with probability (power) 0.81 for urine output, and 1.00 for capillary refill time. The type I error probability associated with this test of this null hypothesis is 0.015 (see paragraph 6.6 below).

3.2.2. Prognostic study

We are planning our main analysis with 1,075 patients and take skin mottling as example for our power calculation: in our design paper, 46% of the patients has skin mottling. If we assume a similar proportion in our total cohort, we have 495 patients with skin mottling and 580 without. Pilot data from our design paper indicate that the 30-day mortality proportion among controls is 0.18.7_{If the true mortality proportion for patients with skin mottling is 0.27, we will be able to} reject the null hypothesis that the mortality proportion for patients with skin mottling and control patients are equal with probability (power) 0.84. The type I error probability associated with this test of this null hypothesis is 0.015. We will use an uncorrected chi-squared statistic to evaluate this null hypothesis.

3.3. Timing of final analysis

Data cleansing and CCUS image validation will be performed upon completion of the 90-day follow- up of the last patient included in the study. The final analysis will be conducted hereafter. This statistical analysis plan was added to the study protocol at clinicaltrials.gov, before closure of the database and before any analyses had been conducted. Independent study monitoring was conducted in adherence to the Good Clinical Practice guidelines.10

3.4. Timing of outcome assessments

Follow-up on all-cause mortality was conducted at 1 November 2017, i.e. 90-days after the inclusion of the last patient.

4. Statistical principles 4.1. Multiplicity

The diagnostic and prognostic basic study and each sub-study consist of one primary outcome and one or more secondary, exploratory outcomes. We will encounter multiplicity issues due to the multiple primary outcomes that are tested for significance in the same cohort. The SICS-I cohort addresses six different primary outcomes (Table 2, last column); two primary outcomes of the basic study (diagnostic and prognostic), and four additional outcomes in the nine sub-studies. We will apply an adjustment for multiplicity based on the total numbers of different primary outcomes tested. Our cohort mainly addresses haemodynamic research questions, so that most outcomes will probably be positively correlated. Therefore, a Bonferroni adjustment of the

(25)

74

P-value might be too conservative. We chose for multiplicity adjusted thresholds by following the pragmatic approach stated by Jakobsen et al.12_{The authors suggested that the ‘true’ threshold} lies somewhere between the unadjusted threshold (most often 0.05) and the Bonferroni adjusted threshold. Where in this interval the ‘true’ threshold is placed is dependent on the correlation between the outcomes, if two outcomes are perfectly correlated (a correlation coefficient of 1) no adjustment of the threshold for statistical significance is needed, if two outcomes are totally independent (a correlation coefficient of 0) a full Bonferroni adjustment is needed. Therefore, Jakobsen et al suggest “dividing the pre-specified P-value threshold with the value halfway between 1 (no adjustment) and the number of primary outcome comparisons (full Bonferroni adjustment)” as such an adjustment will come closer to the ‘true’ statistical significance level than the ‘extreme thresholds’ in a majority of situations.12_{The corresponding formula is:}

Wherein:

• α is the unadjusted threshold for significance, usually 0.05

• m is the number of primary outcomes or tests (used) in the same cohort (in this case: six) The threshold for significance in the SICS-I cohort will be:

4.2. Statistical significance and confidence interval

As calculated in 4.1, we will consider a P < 0.015 as statistically significant for our primary outcomes. When we find a 0.015 ≤ P ≤ 0.05, we will consider this association of dubious significance and will emphasise the increased chance of a type I error. Results will be presented with their values (e.g. regression coefficients, odds ratios, etc.) with 98.5% confidence intervals.

4.3. Adherence and protocol deviations

4.3.1. Definitions of protocol deviations

Protocol deviations are defined when the activities on a study diverge from the local institutional review board-approved protocol, however without significant consequences.13

4.3.2. Protocol deviations to be summarised

We opted for the following subgroup analyses in our study protocol: different types of shock (distributive, obstructive, hypovolemic, cardiogenic), CVVH, heart failure by any cause, myocardial infarction, atrial fibrillation or surgery versus no-surgery patient groups.

�� _{2 �} �� 0.05 �� _{2 �}� 0.0�� 0.0�5

(26)

75 approaches a critically ill patient.

5. Study population 5.1. Screening data

Eligible patients who were not included will be compared to included patients by comparing their general characteristics (age, sex), and SAPS-II and APACHE-IV scores.

5.2. Eligibility

All eligible patients were included on their first day of ICU admission. Inclusion in the basic study consisted of a protocolised clinical examination and subsequent CCUS. The attending ICU physician estimated the expected duration of ICU treatment. Patients expected to stay beyond 24 hours who were eventually discharged within 24 hours were included in our main analyses.

5.2.1. Inclusion criteria

• Emergency admission • Expected stay > 24 hours

5.2.2. Exclusion criteria

• Age < 18 years

• Planned admission (either after surgery or for other reasons)

• Unable to obtain informed consent, e.g. refusal, acute psychiatric disorders, mental retardation, serious language barriers

• Continuous resuscitation efforts or mechanical circulatory support 5.3. Recruitment

A flow diagram will be used to visualise the flow of patients. In this flow diagram, we will report the population from which the eligible patients were selected, reasons for exclusion of eligible patients, and how many CCUS images and measurements were validated (diagnostic study) or how many patients died (prognostic study). See figure 1 for an example of our flow diagram. 5.4. Withdrawal/follow-up

5.4.1. Level and timing of withdrawal

The withdrawal rate of the SICS-I is below 2%, since it was an observational study in which no interventions were applied.

Following hospital regulations, patients or their legal representatives were informed and were excluded if they refused to participate. Withdrawal from our study occurred when informed consent was obtained from the patient’s legal representative, but the patient refused at a later time.

(27)

76

5.4.2. Reasons and details of withdrawal

Reasons for withdrawal or lost to follow-up will be reported in the manuscript and/or flow diagram. Our observational study consists of a one-time measurement (snapshot), so that drop-outs or lost to follow-up reasons are unrelated to the study.

Patients who were lost to follow-up are considered alive until their last outpatient visit or hospital discharge, and were censored thereafter.

5.5. Baseline patient characteristics

5.5.1. Collected baseline patient characteristics

The cohort study was designed to register a set of clinical examination, biochemical and haemodynamic variables in each included patient. We extracted baseline demographic data from the Dutch National Intensive Care Evaluation (NICE) registry and collected clinical data by protocolised clinical examination and CCUS. We obtained the biochemical values from arterial blood gas analyses closest to study inclusion. Table 2 provides an overview of all collected variables and indicates for each variable whether it is categorised as a clinical examination, haemodynamic, or biochemical variable.

5.5.2. Descriptive summarization of baseline patient characteristics

We will list general patient characteristics in a baseline characteristics table. Data will be presented as mean with standard deviation (SD) when normally distributed or as median with interquartile range in case of skewed data. Dichotomous and categorical data will be presented in proportions. Normality of the data will be assessed using P-P plots, Q-Q plots, and histograms. Linearity will be assessed using scatter plots. Differences between continuous variables will be assessed using Student’s t-tests or Mann-Whitney-U test, depending on normality, whereas the Chi-squared test will be used for categorical values. For repeated measurements, we will use the paired t-test for normally distributed continuous data, the Wilcoxon signed-rank test for skewed continuous data, and the McNemar test for dichotomous data.

5.6. Assumed confounding covariates

The majority of variables measured in our study are inevitably correlated, as most relate to the haemodynamic status of a patient. While definitions have been recorded in the protocol, the values of the variables can be confounded by unmeasured factors, such as environmental, genetic, or psychological influences. Therefore, we provide an example of possible confounding variables and categorise these into ‘measured’ and ‘unmeasured’.

•

Cardiac output and clinical examination of the central haemodynamics (i.e. heart rate, blood pressures, central venous pressure) are assumed to be confounded by:

• Measured: body surface area (therefore we will use cardiac index), quality of measurements (therefore data will be validated), distributive shock as the underlying pathology, administration of inotropes and/or vasopressors, administration of propofol (negative inotropic effect), mechanical ventilation including ventilation pressures

(28)

77 • Measured: history of chronic renal failure, distributive shock as the underlying pathology • Unmeasured: total amounts of fluids administered

•

AVPU score is assumed to be confounded by • Measured: sedation (propofol, midazolam)

•

Clinical examination of peripheral perfusion (i.e. mottling, peripheral capillary refill times, peripheral temperatures) are assumed to be confounded by:

• Measured: heating blankets, distributive shock as the underlying pathology, cardiac output, administration of inotropes and/or vasopressors, administration of propofol (vasodilation)

• Unmeasured: regular blankets, environmental temperature, peripheral arterial disease.

•

Mortality proportion is assumed to be confounded by:

• Measured: age, comorbidities, and several other variables that are all embedded in the simplified acute physiology score II (SAPS-II) score

• Unmeasured: cause of mortality (e.g., death due to multi-organ failure or failure to wean, a patient’s or family’s personal wishes regarding the extent of ICU treatment. This will, however, always be a mix of causes).

We acknowledge that there will be residual confounding in our dataset due to the presence of unmeasured confounding, some of which is listed above. However, the actual measured variables reflect daily practice and so, is assumed to reflect similar confounding in daily assessments of the haemodynamic status of ICU patients.

6. Analysis

6.1. Outcome definitions

6.1.1. Primary and secondary outcomes

The research questions and the design of the study have been published.7_{We elaborate on the} outcomes of the basic study below and described the primary outcomes of each sub-study in appendix 1.

The outcomes of research question 1, i.e. the diagnostic study, are:

• Primary: the association of a single or combination of clinical examination findings with cardiac index measured by CCUS

• Secondary: the diagnostic test accuracy of a single or a combination of clinical examination findings to diagnose a low, normal and high cardiac index

• Secondary: the association and diagnostic test accuracy of a single or combination of clinical examination findings with cardiac index in clinically different patient subgroups

(29)

78

The outcomes of research question 2, i.e. the prognostic study, are:

• Primary: the association of all measured clinical examination, biochemical and haemodynamic variables with 90-day mortality

• Secondary: the association of clinical examination, biochemical and haemodynamic variables with 7-day and 30-day mortality

• Secondary: the association of clinical examination, biochemical and haemodynamic variables that are not visible to caregivers with 90-day mortality

• Secondary: the association of clinical examination, biochemical and haemodynamic variables with 90-day mortality in clinically different patient subgroups

6.1.2. Measurement and calculation of outcomes

For the diagnostic study we calculated cardiac index, which was derived from cardiac output. Cardiac output has been measured with the cardiac probe M3S of M4S with default cardiac imaging setting of the General Electric Vivid-S6 mobile ultrasound machine. Two views were obtained: the parasternal long axis (PLAX) and the apical five chamber view (AP5CH). The PLAX was used as the primary view to measure the left ventricular outflow tract (LVOT) diameter. The AP5CH view was used to measure the velocity time integral (VTI) using the pulse wave Doppler signal in the LVOT. Cardiac output was calculated on the ultrasound machine according to the formula:

At a later time, the images and measurements were validated by technicians from an independent core laboratory, whom were blinded for all other measurements and outcomes.

We used cardiac index instead of cardiac output for interindividual comparisons. Cardiac index is the cardiac output adjusted for body surface area:

Where body surface was calculated with the DuBois formula:14

Cut-offs for a low cardiac index for critically ill patients are inconsistent.15_{Haemodynamic criteria} to diagnose cardiogenic shock vary from a cardiac index of 1.8 to 2.5 L∙min-1_∙m-2_.16-19 _{A cardiac} index below 2.2 L∙min-1_∙m-2_{is often used to diagnose a low cardiac output syndrome after cardiac} surgery,20_{whereas a large clinical trial used a cut off below 2.5 L∙min}-1_∙m-2_{in patients with acute} lung injury.19_{These criteria, however, apply to patients with heart failure or after cardiac surgery.}

ܥܽݎ݀݅ܽܿ݅݊݀݁ݔሺ ܮ ݉݅݊ ݉ଶሻ ൌ_{ܤ݋݀ݕݏݑݎ݂ܽܿ݁ܽݎ݁ܽ}ܥܽݎ݀݅ܽܿ݋ݑݐ݌ݑݐ ܤ݋݀ݕݏݑݎ݂ܽܿ݁ܽݎ݁ܽ ൌ ͲǤͲͲ͹ͳͺͶ ή ܹ݄݁݅݃ݐ଴Ǥସଶହ_{ήܪ݄݁݅݃ݐ}଴Ǥ଻ଶହ ��_{�� }� �1_{2 � ��}��