Follow-up system for intensive care patients: preliminary evaluation of workflow and usability

FOLLOW-UP SYSTEM FOR INTENSIVE CARE PATIENTS:

PRELIMINARY

EVALUATION

OF

WORKFLOW AND USABILITY

Master’s Thesis

Guy Ruben Mahieu, BSc

December 2014

Supervisors:

N.F. de Keizer, PhD

F. Bakhshi-Raiez, PhD

University of Amsterdam

Master Medical Informatics

Scientific research project:

Follow-up system for intensive care patients: preliminary evaluation of workflow and usability Student:

Guy Ruben Mahieu, BSc Student ID: 5805406 Tutor:

Nicolette de Keizer, PhD Principal investigator

Department of medical informatics, J1B-115 N.f.keizer@amc.uva.nl

Mentor:

Ferishta Bakhshi-Raiez, PhD Researcher

Department of medical informatics, J1B-113.1 F.raiez@amc.uva.nl

Scientific research project coordinator: prof. A. Abu-Hanna, PhD

Head of academic department

Department of medical informatics, J1B-113 A.abu-hanna@amc.uva.nl

Scientific research project location: Academic Medical Center (AMC) Department of medical informatics Meibergdreef 9 1105 AZ Amsterdam 020-5669111 Period: December 2013 – December 2014 Examination committee: prof. J.H. Ravesloot, MD PhD M. van der Schaaf, MD PhD N.F. de Keizer, PhD

P

REFACE

To conclude the master’s program of Medical Informatics each student participates in a Scientific Research Project (SRP) resulting in a master’s thesis which is presented and defended orally. The goal of the SRP is to develop the students’ scientific problem-focused approach and improve their ability to pursue lifelong learning. I would like to thank both my mentor and tutor, Nicolette and Ferishta for letting me contribute to the introduction of structured intensive care follow-up in the Netherlands. Their guidance and support has inspired me along the way and has kept me focused on my goals. Getting to see how the ICU follow-up pilot took shape from multiple perspectives has been a worthwhile experience.

Getting to visit patients on my own during the think aloud sessions was a new and exciting experience. This has underlined for me that in the end healthcare is about people, something you would almost forget spending most of your time at a research department.

Thank you Ilse, although short, it was nice to prepare the ICU follow-up training days at Itémedical together. It is great to see how fast your knowledge of ICU follow-up care has grown since starting your PhD, I wish you all the best in your research endeavors.

My family and friends have always shown interest in my endeavors for which I am grateful. To my fellow J1b-123.1 residents, you guys rock!

Finishing the SRP has at times looked like an insurmountable task, but I have always felt support from everyone involved. Thanks to this I am now glad to present my finished thesis.

7

C

ONTENTS

Outline of this thesis ... 12

Chapter 2 | Background ... 13

Introduction ... 13

Dutch national intensive care evaluation (NICE) foundation ... 13

Dutch association for intensive care medicine (NVIC) ... 14

Set of recommendations ... 14

Recommended workflow ... 15

Itémedical... 16

ICU follow-up information system description ... 18

Pilot ... 20

Chapter 3 | Workflow changes ... 21

Introduction ... 21

Methods ... 21

Recommended workflow ... 21

Study design ... 21

Participants and setting ... 23

Data collection ... 23

Data analysis ... 23

Results ... 24

Clinic characteristics ... 24

Comparison with recommendations ... 24

Patterns in existing workflow ... 26

Comparison with recommended workflow ... 26

Discussion ... 27

Principal findings ... 27

Strengths and weaknesses of the study ... 28

Relation to other studies ... 29

8

Unanswered questions and future research ... 30

Chapter 4 | Patient portal evaluation ... 31

Introduction ... 31

Methods ... 31

Study design ... 31

Participants and setting ... 32

Data collection ... 32

Data analysis ... 32

Results ... 34

Study population ... 34

Usability evaluation of ICU follow-up patient portal ... 34

Severity evaluation of identified usability problems... 34

Discussion ... 37

Principal findings ... 37

Strengths and weaknesses of the study ... 37

Relation to other studies ... 38

Meaning of the study ... 38

Unanswered questions and future research ... 39

Chapter 5 | Conclusion ... 41

Bibliography ... 42

Abbreviations ... 46

Appendix 1 | Screenshots of MediScore ICU follow-up information system ... 47

Appendix 2 | Semi-structured interview questions ... 54

Appendix 3 | UML swim lane activity diagrams modeling workflow of ICU follow-up clinics ... 57

Appendix 4 | Testing script, think aloud scenarios and invitation ... 61

9

S

UMMARY

Background: Development of an expert-based multidisciplinary ICU follow-up project for patients recovering from critical illness has been started in the Netherlands to evaluate the effectiveness of ICU follow-up care and eventually to improve quality of follow-up care for ICU patients. Initiation of this project originated from a collaboration between the Dutch National Intensive Care Evaluation (NICE) foundation and the Dutch Intensive Care Society (NVIC). A theoretical infrastructure consisting of set of ten recommendations for the organization of ICU follow-up care was defined as well as a recommended workflow, giving a visual representation of how ICU follow-up clinics should function. The technical infrastructure of this project consists of an ICU follow-up information system which was developed by Itémedical and the already existent NICE registry. The information system provides a patient portal presenting screening questionnaires to patients. The purpose of this SRP was to perform a preliminary evaluation of the ICU follow-up information system, the set of recommendations and the recommended workflow with its end-users. The results are input for improvements to the set of recommendations, the recommended workflow and the ICU follow-up information system.

Methods: From a medical professionals perspective we looked at the workflow of ICU follow-up clinics. Semi-structured interviews were conducted with ICU follow-up teams to create workflow models for each participating ICU follow-up clinic. Mapping the workflow of currently operational ICU follow-up clinics and comparing their processes with the set of recommendations and the recommended workflow will give insight in challenges the end-users might encounter.

From a patients perspective we looked at the usability of the patient portal. Think aloud sessions were conducted with patients to identify usability problems. The evaluation of the patient portal gives input for redesign of the patient portal to ensure that the system is easy to use by patients.

Results: Four ICU follow-up clinics participated in the workflow evaluation. All ICU follow-up clinics in this study already adhered to at least six out of the ten (60%) recommendations and four of the ten (40%) recommendations were already implemented in all clinics. Modelling the workflow of the different clinics in the same structure showed that although specific details of workflow differ, there are many common elements to be found.

Four ex-IC patients (aged 27-76) participated in the usability evaluation. Four think aloud sessions identified a total of 22 unique usability problems with a mean severity rating of 2.5. During the think aloud sessions 13% of all scenarios that were presented were not successfully completed.

Conclusion: The workflow evaluation provides insight in challenges future end-users might encounter when their clinic joins the ICU follow-up project. Overall, the evaluation of the workflow of the ICU follow-up clinics shows that they are already adjusted or can easily be adjusted to the set of recommendations as their workflow structures are similar. None of the identified challenges jeopardize the feasibility of the ICU follow-up project. ICU follow-up clinics are allowed to deviate from some of the given recommendations and can also differ on aspects not controlled by the given recommendations, which hampers the unambiguous data collection for research and benchmarking. Therefor we suggest expanding the amount of identifying characteristics that are being collected in the quality registry.

The usability evaluation reveals several severe usability issues that should be solved by redesign of the patient portal. Additionally, use of the patient portal is also impeded by computer illiteracy, cognitive and physical barriers experienced specifically by ICU follow-up patients which should be considered during the redesign process.

10

C

HAPTER

1

|

I

NTRODUCTION

This thesis describes the Scientific Research Project (SRP) called “Follow-up system for intensive care patients: preliminary evaluation of workflow and usability”, performed at the department of Medical Informatics (MI) of the Academic Medical Center (AMC) in Amsterdam. In this section first the focus of the SRP and the problem is given, next the goal and research questions are stated, finally the outline of the thesis is provided.

F

OCUS

Each year in the Netherlands approximately 80.000 patients are admitted to 94 Intensive Care Units (ICUs). About 85% of the ICU patients survives the hospital stay [1]. The survival rate of these critically ill patients has steadily increased over the years due to advances in treatment. However, during the first 3-12 months after discharge from the ICU a considerable amount of patients yet dies which amounts to an additional 5-10% ICU mortality [2]. In addition, the Health Related Quality of Life (HRQoL) of patients after the first year they have been discharged from the ICU, is still lower than that of the general population [3–9]. Furthermore long-term physical impairments and psychological problems such as anxiety, depression, and post-traumatic stress disorder have a high prevalence and may hinder these patients for many years which thereby puts a burden on society [10–13]. Together these sequelae of disorders have been defined as Post Intensive Care Syndrome (PICS), which negatively impacts the physical, mental, social, and financial position of patients and their families [9,14]. It has been suggested that patients suffering from PICS could benefit from ICU follow-up care [15]. The goal of ICU follow-up care is to screen post ICU patients for potential IC related problems. Based on this multidisciplinary screening, patients can then be referred to an appropriate specialist. This referral process usually involves the General Physician (GP) of the patient. Further diagnostics of the patients’ problems are performed by the specialist who will decide what is in the best interest of the patient, which could then lead to treatment of the patient. A third of the invited ICU follow-up patients end up not visiting an ICU follow-up clinic, either because of health reasons or no perceived need for additional care [15]. Patients suffering from PICS should receive specific attention since they might avoid contact with the ICU [15]. Hence, ICU follow-up care should be organized in such a way that these patients receive the care they need.

In the domain of cardiology, cardiac rehabilitation is provided by multidisciplinary care teams to support heart patients’ recovery from a cardiac incident or intervention. The goal of these follow-up programs is to improve the patients’ physical and psychological condition. The effectiveness of exercise-based and multidisciplinary cardiac rehabilitation interventions with regard to mortality and cardiac events has been demonstrated through meta-analysis of Randomized Controlled Trials (RCTs) [16]. Research on the cost-effectiveness of these interventions also showed cost reductions [17].

In the field of Chronic Obstructive Pulmonary Disease (COPD) similar steps have been taken. The American thoracic society together with the European respiratory society formulated a statement providing scientific rationale, based on a rapidly growing evidence base, and clinical expertise for pulmonary rehabilitation [18]. In contrast to these multidisciplinary rehabilitation programs, the effectiveness of ICU follow-up care has not yet been evaluated. Structured ICU follow-up care is rare and the optimal structure, timing, and its content have not been established yet. In the Netherlands, about one-third of the ICUs have developed ICU follow-up programs with each their own organizational, planning and funding structures [19].

In response to this situation, development of an expert-based multidisciplinary ICU follow-up project for patients recovering from critical illness has been started in the Netherlands [15]. Initiation of this project originated from a collaboration between the Dutch National Intensive Care Evaluation (NICE) foundation and Dutch Intensive Care Society (NVIC). The ultimate goal of the ICU follow-up project is to evaluate the effectiveness of ICU follow-up care and eventually to improve quality of follow-up care for ICU patients.

11

As a first step in the ICU follow-up project a set of recommendations for the organization of ICU follow-up care was defined, based on literature, a national survey on current ICU follow-up care, and expert opinion. This whole process is described in van der Schaaf et al. (2014) [15]. The set of recommendations consists of ten items, giving directions on; whom to invite, when to plan visits, what to screen for, and whom to include in an ICU follow-up team. Furthermore, it recommends the use of electronic patient-reported screening instruments and the integration of follow-up care data into a national quality registry. The set of recommendations is part of a theoretical infrastructure which is used as a guideline to design the ICU follow-up project around (Figure 1). The recommended workflow (Figure 1), was developed to give a visual representation of how ICU follow-up clinics should function. This model consists partly of the set of recommendations but also includes processes necessary from a research perspective such as the collection of data on no-shows.

The next step in the ICU follow-up project is to implement the set of recommendations into practice in order to determine the feasibility of the ICU follow-up project. During a pilot

several ICU follow-up clinics will work in accordance to the set of recommendations and make use of the provided technical infrastructure while following the recommended workflow. Figure 2 gives an overview of the technical infrastructure and the concepts that will be used to describe it in this SRP.

The technical infrastructure consists of two major parts, the ICU follow-up information system and the NICE registry. The ICU follow-up information

system can be further divided into two parts (Figure 2): the ICU follow-up hospital software will be used by medical professionals that will work according to the recommended workflow to screen patients that might need ICU follow-up care. It also provides ICU follow-up patients access to electronic screening instruments. This happens through the ICU up patient portal over the Internet, where patients can login to the ICU follow-up information system to answer pre-defined questionnaires. The information system is used by medical professionals and patients while the NICE registry collects data to evaluate the effectiveness of ICU follow-up care. The focus of this SRP is on both the ICU follow-up information system and the theoretical infrastructure.

P

ROBLEM

ICU follow-up clinics that adjust their workflow in accordance with the theoretical infrastructure and use the technical infrastructure will be able to fully adhere to all given recommendations. This does however bring three untested elements together: a set of recommendations, a recommended workflow, and an information system. Implementation of the theoretical and technical infrastructure might cause problems for medical professionals in terms of workflow integration and for patients in terms of usability problems in the ICU follow-up patient portal.

FIGURE 1 TERMINOLOGY USED TO DESCRIBE THE THEORETICAL INFRASTRUCTURE

(ICONS BY FREEPIK UNDER CREATIVE COMMONS BY 3.0)

FIGURE 2 TERMINOLOGY USED TO DESCRIBE THE TECHNICAL INFRASTRUCTURE

12

G

OAL

The goal of this SRP is to perform a preliminary evaluation of the ICU follow-up information system, the set of recommendations and the recommended workflow with its end-users. From a medical professionals perspective we look at the workflow of ICU follow-up clinics. Mapping the workflow of currently operational ICU follow-up clinics and comparing their processes with the set of recommendations and the recommended workflow will give insight in challenges the end-users might encounter. From a patients perspective we look at the usability of the patient portal. The evaluation of the patient portal gives input for redesign of the patient portal to ensure that the system is easy to use by patients. Results of both these two evaluations are input for improvements to the set of recommendations, the recommended workflow and the ICU follow-up information system.

R

ESEARCH QUESTIONS

The goal of this SRP leads to the following research question:

Which changes to the workflow of existing ICU follow-up clinics are introduced by the introduction of the set of recommendations and the recommended workflow and, which severe usability problems do patients experience when using the ICU follow-up patient portal?

To answer this question the following sub-questions have been formulated. Since the main research question can be divided into two parts our sub-questions are divided accordingly:

Workflow changes:

What does the workflow of existing ICU follow-up clinics look like?

To which of the recommendations do ICU follow-up clinics already adhere?

Which changes to the workflow of existing ICU follow-up clinics are introduced by the introduction of the set of recommendations and the recommended workflow?

Usability problems:

What are the usability issues ICU follow-up patients encounter using the ICU follow-up patient portal? What is the severity of the usability issues that ICU up patients encounter using the ICU follow-up patient portal?

O

UTLINE OF THIS THESIS

The next chapter contains general background information on stakeholders and their involvement within the ICU follow-up project. Furthermore an overview of the technical and theoretical infrastructure is given. The third chapter focusses on workflow changes and the forth chapter on usability problems encountered by ICU follow-up patients using the patient portal. Chapter five completes this thesis and contains an overarching conclusion on both topics.

13

C

HAPTER

2

|

B

ACKGROUND

I

NTRODUCTION

The development of a set of recommendations on how to organize an ICU follow-up clinic has been the subject of study for the past years [9,15,20,21]. A survey performed by the NICE foundation in 2012 among all Dutch ICUs and the results of an expert meeting in 2012, together form the basis of the set of recommendations recently put forth by the Dutch association for intensive care medicine (NVIC). With this accumulated knowledge a software vendor (Itémedical) has developed an ICU follow-up information system which is capable of collecting all data needed for ICU follow-up care and provide electronic patient-reported screening through a patient portal. This enables the ICU follow-up clinics to adhere to the set of recommendations. In this chapter we will provide background information on the stakeholders, the set of recommendations, the recommended workflow, describe the ICU follow-up information system and explain about the ICU follow-up pilot.

D

UTCH NATIONAL INTENSIVE CARE EVALUATION

(NICE)

FOUNDATION

The Dutch national intensive care evaluation registry was set up in 1996 to gain insight into and to improve the effectiveness and efficiency of Dutch ICUs and to enable quality insurance and quality improvement of Dutch ICUs. This idea originated from a small professional group of intensivists. Besides professional motives to improve the quality of care, care providers are under pressure to make their quality of care transparent. The effectiveness of intensive care treatments and its efficient implementation needed to be demonstrated. An organizational infrastructure and a technical infrastructure were developed to collect data and to analyze the data to provide feedback reports and to provide a network in which the professional group of intensivists could discuss and discover the implications to improve intensive care treatments and its organization.

Since 2012 almost all ICUs in the Netherlands provide their data, which results in 80.000 newly added records to the database each year [22]. The NICE foundation is fully funded by its participating members and is a non-profit organization. Organization and maintenance of the registry takes place at the department of Medical Informatics in the Academic Medical Center in Amsterdam, the Netherlands.

The registry stores the Minimal Data Set (MDS), which is the data that has to be collected for each ICU admission. This core data set consists of approximately 90 variables on demographics, admission and discharge data, reason for admission and, severity of illness in the first 24 hour of ICU admission for each admission. Outcome measures such as ICU and hospital mortality and length of stay are also part of the MDS. To describe the severity of illness a number of prognostic models are used such as APACHE II, APACHE IV, and SAPS II [23– 25]. All MDS variables are described in the data dictionary [26]. In addition to the minimal data set the NICE registry also contains other data sets such as the Sequential Organ Function Assessment (SOFA) data set, complications data set, the severe sepsis data set and quality indicators as defined by the NVIC [27].

Comparison of ICUs to case mix corrected expected outcomes is used to measure effectiveness and efficiency. All data variables currently contained within the data dictionary are limited to the time the patient spends on the ICU, the only exception being hospital mortality. Mortality is used as an important indicator to rate an ICUs effectiveness. The indicator length of stay is used as the main indicator for efficiency. By extending the NICE registry with the ICU follow-up data set, the registry will also provide long term outcomes for feedback, research, and quality improvement initiatives such as optimizing triage for post ICU patients.

The NICE foundation aims to provide actionable feedback to participating ICUs which they can act upon to further improve quality of care. Twice a year standard benchmark reports are provided to each participating ICU and an annual meeting is also organized to facilitate further discussion.

14

To ensure high quality benchmark reports and research, the underlying data needs to be complete and correct. To optimize the quality of data and to minimize bias, several procedures are in use. In each hospital at least two intensivists should follow the NICE training program to become proficient at correct data collection and become knowledgeable about the data that is to be collected. Their knowledge is then further spread within each ICU according to the train-the-trainer principle. From a technical perspective ICUs are guided to implement the domain and inter-variable constraints described in the data dictionary. Each month the participating ICUs have to send their data to the registry. For each data set, the participants receive a feedback report on the structure, completeness and validity of their data. This report can be used to make corrections in their data set. Finally, site visits are performed to check the quality of data in the NICE database.

D

UTCH ASSOCIATION FOR INTENSIVE CARE MEDICINE

(NVIC)

The Dutch association for intensive care medicine is an association for physicians with an interest in intensive care medicine [28]. The NVIC represents intensive care medicine in the Netherlands in multiple ways. Its goals include the improvement of intensive care medicine, looking after the IC patients’ needs, and stimulating the advancement of knowledge in this medical field.

In their concept version of the intensive care guideline of 2014 the NVIC included a chapter on ICU follow-up care [29]. The following considerations are made in this document. “Although there is no direct evidence that

an ICU follow-up clinic affects IC outcome, the committee believes that an ICU follow-up clinic, or an equivalent form, adds value to daily practice. Feedback on experiences from patients and their family can lead to improvements in the quality of care. Furthermore, it is important to make the long-term outcome of IC care visible. Not only for individual ICU follow-up clinics but also to facilitate nationwide comparison between clinics. This is still a mostly unexplored field of research. This research will have to be initiated by the IC community, given the limited knowledge and experience in the IC field of other specialists.” (translated by GM) [29]. This

leads the NVIC to the following two recommendations: “The NVIC encourages ICUs to incorporate a form of ICU

follow-up evaluation in an effort to get insight into prognosis, experiences and quality-of-life of ICU follow-up patients. It is recommended that results of ICU follow-up evaluation are stored following a standardized method in order to compare clinics.” (translated by GM) [29].

The content of this concept guideline is currently under review by NVIC members and affiliated scientific societies. At the same time a set of recommendations on how to organize ICU follow-up clinics has been published based on literature and expert opinion [15]. Both recommendations that are mentioned in the concept guideline of the NVIC are also listed in the study of van der Schaaf et al. (2014) who also provided additional actionable recommendations [15].

S

ET OF RECOMMENDATIONS

The NICE foundation and the NVIC worked in collaboration to develop a set of ten recommendations in order to structure ICU follow-up care in the Netherlands. Table 1 shows the set of recommendations.

All recommendations have been implemented into the ICU follow-up project. Implementation of the recommendations can be divided into three parts; (1) workflow with recommendations 1, 2, 3, 4, 5, 6, 8, and 9; (2) technical infrastructure with recommendation 7 and 10; and (3) data collection with recommendation 10. As presented in Table 1, workflow related recommendations dictate the criteria for patient inclusion, the invitation of partners, timing and frequency of the visit, subject of screening, and which medical professionals should be involved in the ICU follow-up clinic. In the ICU follow-up project these recommendations are interpreted as a set of rules that all participating ICU follow-up clinics should follow.

15

TABLE 1 LIST OF RECOMMENDATIONS FOR INTENSIVE CARE FOLLOW-UP CLINICS # Recommendation

1 Invite all patients who were mechanically ventilated (MV) more than 2 days. 2 Invite the partners of IC survivors.

3 Plan the visit to the ICU follow-up clinic 12 weeks after hospital discharge. 4 Plan one visit with the possibility for a follow-up at indication.

5 Screen patients with respect to their needs and IC related sequelae.

6 Screen impairments in the following health domains: movement related functions, sensory functions, communication problems, symptoms of post-traumatic stress, symptoms of anxiety and depression, cognitive functioning, psychosocial functioning, and sexual functions.

7 Use electronic patient-reported screening instruments to identify patients in the need for ICU follow-up care.

8 Have an ICU nurse whether or not with an intensivist carrying out the ICU follow-up clinic. 9 Involve a physiotherapist to perform a comprehensive physical screening.

10 Integrate follow-up care data into a national quality registry for intensive care to monitor and improve quality of life and functional status of IC patients.

Technical infrastructure related recommendations dictate the use of electronic patient-reported screening instruments. Although the recommendation to integrate follow-up care data into a national quality evaluation registry does not explicitly requires the use of a tailor-made information system it was interpreted this way for the ICU follow-up project. The ICU follow-up information system was developed specifically to meet these two recommendations resulting in a system used by both medical professionals and patients i.e. the ICU follow-up hospital software is used by medical professionals and the ICU follow-up patient portal is used by ICU follow-up patients.

The questionnaires used in the patient portal are a collection of pre-existent validated and non-validated questionnaires as well as a selection of newly formulated questions. The majority of these questions were decided upon based on a survey and conference of Dutch intensive cares [15]. Additional questions were added so that important steps in the workflow were traceable, which was either necessary for running the national quality evaluation registry or interesting from a research perspective. An overview of the questionnaires presented to the patients through the patient portal can be found in Table 2.

Finally, the data collection related recommendation not only initiated the development of a tailor-made information system for ICU follow-up clinics but also a national quality evaluation registry. The NICE registry was extended specifically to be able to meet this recommendation. All data dictionary items as well as questionnaire results can be automatically sent to the registry by the ICU follow-up information system.

R

ECOMMENDED WORKFLOW

A UML swim lane activity diagram was created to describe the recommended workflow of an ICU follow-up clinic participating in the ICU follow-up project. This gives stakeholders an impression of what the workflow of an ICU follow-up clinic should look like. It presents the basic steps needed to provide ICU follow-up care to patients, while collecting all necessary data needed for the national quality registry. Stakeholders include the ICU follow-up care research team who create and refine the work processes and evaluate the effectiveness of ICU follow-up care. The software developer, Itémedical, responsible for implementing the recommendations into the ICU follow-up information system used the activity diagram to make sure that all actions, decisions and activities were supported in their product. Medical professionals received the activity diagram to inform them of the proposed workflow and to explain why they should follow the recommendations. The initial version of this activity diagram was created by the NICE foundation and is presented in Figure 3, later a more detailed version was created as can be read in chapter 3.

16

TABLE 2 THE NINE QUESTIONNAIRES PRESENTED TO PATIENTS USING THE PATIENT PORTAL

Name Description

General Asks if the patient is willing to participate and answer all questionnaires. And, if this is not the case, the reason why not.

HADS The Hospital Anxiety and Depression Scale, a 14-tem self-assessment scale, was developed to detect states of depression, anxiety and emotional distress amongst patients who were being treated for a variety of clinical problems.

SF-36 The Short Form 36 is a multi-purpose, short-form health survey with 36 questions. It yields an 8-scale profile of functional health and well-being scores as well as psychometrically-based physical and mental health summary measures and a preference-based health utility index.

TSQ The Trauma Screening Questionnaire is a ten-item symptom screening that was designed for use with survivors of all types of traumatic stress.

MUST The Malnutrition Universal Screening Tool is a five-step screening tool to identify adults who are malnourished, at risk of malnutrition or obese.

Returning to work Surveys the patient’s employment before and after their IC admission. Common

problems

Surveys common problems encountered by ICU follow-up patients. Visiting medical

professionals

Surveys the care consumption of patients, i.e. health care providers visited after hospital discharge.

End of questionnaire

Gives the patient the possibility to leave any remarks in an open text field.

I

TÉMEDICAL

Itémedical provides medical hardware and software in the Benelux. The company is the developer of the MediScore ICU software [30]. MediScore ICU is an automated departmental and management information system which compiles monthly, quarterly and annual reports for a critical care department which complies to the NICE foundations registration standards. Itémedical is the first software provider that offers ICU follow-up clinics a solution to provide the necessary data to the NICE registry. The ICU follow-up information system can be connected to the MediScore ICU software so that all information stored in one system is automatically shared with the other. Because of this, medical professionals are not burdened with unnecessary data input. All ICUs that participate in the pilot will use the MediScore ICU follow-up information system. However, the policy of the NICE foundation is that any software that meets the technical recommendations, follows the recommended workflow and can provide the needed data as specified in the NICE data dictionary can be used for collection. In the coming years the NICE foundation expects that some hospitals with an Electronic Patient Record (EPR) that supports a patient portal will implement the data set.

17

18

ICU

FOLLOW

-

UP INFORMATION SYSTEM DESCRIPTION

The MediScore ICU follow-up information system consists of two interfaces: one for medical professionals, the ICU follow-up hospital software, at the ICU follow-up clinic. The other one for ICU follow-up patients, to use at home over the Internet; the ICU follow-up patient portal. Screenshots of the MediScore ICU follow-up information system illustrating the described functionalities can be found in Figure 4 to 7 and in more detail in Appendix 1. The medical professionals’ home screen (Figure 4) offers four main views: the patient overview (Figure 5), task overview, analytics view and control view. The patient overview gives an overview of all patients that are subject to receive ICU follow-up care or have received it in the past. Patients that pass the inclusion criteria for ICU follow-up care are automatically added to the list, users can also add patients manually to this list. Selecting an individual patient brings the user to the task overview. The task overview presents the user with steps that need to be taken to successfully give the patient their ICU follow-up care. Users are guided with inviting patients, collecting the necessary patient information, and scheduling the patients’ visit in accordance to the recommended workflow. Tasks are also added to a timeline to give the user a better overview of taken and pending steps. The analytics view allows users to query locally collected data. An example query is the amount of referrals per specialist. These statistics can for instance help ICU follow-up team members in getting an easy overview of the type of patients they generally help in their clinic. These functionalities are completely separate from the NICE quality registry. Finally, the control view is used to configure system settings such as user account management.

FIGURE 4 OVERVIEW SCREEN MEDICAL PROFESSIONAL

19

The ICU follow-up patient interacts with the ICU follow-up information system over the Internet through the ICU follow-up patient portal. It consists of a login screen, overview screen and questionnaire view. Patients login to the patient portal by entering their login code and date of birth. A login code is provided by the ICU follow-up clinic either through e-mail or regular mail to each patient that fits the inclusion criteria for ICU follow-up care. After logging in, patients are presented with a short description of what they are expected to do and given an overview of the questionnaires that they need to complete (Figure 6). After clicking one of the presented questionnaires, the patients enter the questionnaire view where they fill in the presented questionnaires until they return to the overview screen where they can continue with the next questionnaire (Figure 7). The questionnaire list uses three different icons to show if questionnaires are successfully completed or not. After completing all the questionnaires patients can send their answers to the ICU follow-up clinic by clicking the send button in the overview screen, this also logs them out of the patient portal.

FIGURE 6 OVERVIEW SCREEN PATIENT PORTAL

20

P

ILOT

To test the feasibility of the ICU follow-up project a pilot study was started with a group of nine participating hospitals. Twenty-three ICUs were initially invited for participation in the pilot. The pilot was meant to determine whether the set of recommendations, technical infrastructure and recommended workflow work in practice. This SRP has been performed in the context of this pilot. The goal of this SRP is to perform a preliminary evaluation of the ICU follow-up information system, the set of recommendations and the recommended workflow with its end-users.

In this chapter an overview of all the relevant background information has been given to better understand this scientific research project. In the following chapter the focus will be on the changes in workflow of existing ICU follow-up clinics that are caused by the introduction of the set of recommendations and the recommended workflow.

21

C

HAPTER

3

|

W

ORKFLOW CHANGES

I

NTRODUCTION

With increasing awareness for post intensive care syndrome, several Dutch hospitals already providing ICU follow-up care and the unbridled development of new ICU follow-up clinics, both the NVIC and the NICE foundation started the ICU follow-up project as described in chapter 1 and 2. Evaluating the impact of the ICU follow-up information system on an existing workflow is essential for maximizing the efficiency of the system [31]. Introducing a tailor made ICU follow-up information system with a patient portal into an ICU follow-up clinic setting, will change the workflow of existing ICU follow-up clinics. Understanding how software systems in the care process are used is a requisite for effective quality improvement [32,33]. Creating process models to aid in quality improvement falls into two categories: process understanding in order to identify areas of improvement, and second, to spread shared understanding of existing processes [34]. Swim lane activity diagrams can be used for initial process understanding and are also helpful to get a clear understanding of roles and responsibilities in tasks, which is especially of interest when dealing with multidisciplinary work [35]. Using swim lane activity diagrams has the added benefit of being well-known within the healthcare setting, making it easy to understand for all the stakeholders involved in the ICU follow-up project [34].

In this chapter we investigate ICU follow-up clinics workflow before participation in the ICU follow-up pilot and compare it to the set of recommendations and recommended workflow. From a medical professionals perspective we look at the workflow of several ICU follow-up clinics. Mapping the workflow of currently operational ICU follow-up clinics and comparing their processes with the set of recommendations and recommended workflow will give insight in challenges future end-users might encounter when their clinic joins the ICU follow-up project. This leads to the following research questions: what does the workflow of existing ICU follow-up clinics look like? To which of the recommendations do ICU follow-up clinics already adhere? Which changes to the workflow of existing the ICU follow-up clinics are introduced by the introduction of the set of recommendations and the recommended workflow? The results of this evaluation are input for improvements to the set of recommendations, the recommended workflow and the ICU follow-up information system.

M

ETHODS

R

The recommended workflow model was originally developed by the NICE foundation as described in chapter 2 and as can be seen in Figure 3. For the pilot project and as part of our study, the original model was predominantly expanded with regards to contact with the ICU follow-up patients. One of the goals of the pilot study was to investigate the reasons why patients do not (want to) attend the follow-up clinics. The recommended workflow takes opt-out situations into account, making it more detailed than the original workflow model. These situations are modelled to facilitate the collection of sufficient research data for the ICU follow-up pilot. The extended swim lane activity diagram can be found in Figure 8.

S

This study had an explorative descriptive design with a qualitative approach. Semi-structured interviews were conducted to answer the research questions. The qualitative data gathered during these interviews were later used to create the workflow models for each participating ICU follow-up clinic. This study design focused on the differences between workflows in several ICU follow-up clinics and how they relate to the recommended workflow and set of recommendations. An overview of the study design is shown in Figure 9.

22

23

P

The first invitational e-mails to recruit ICU follow-up clinics for the pilot (n=23) were sent in the beginning of February 2014, the recruitment period ended in May 2014. ICU follow-up care coordinators that responded positively to either the e-mails or phone calls regarding the ICU follow-up pilot and that were therefore likely to participate in the pilot received a second e-mail. Here they received further information about the pilot and were asked for participation in the semi-structured interviews of this study. The only inclusion criterion that was defined stated that the participating hospital should already have an operational ICU follow-up clinic prior to participation in the pilot. When the responders agreed to participate in the interviews, we contacted them once more through e-mail and explained how the interview would be performed. The ICU follow-up coordinators were free to choose who would be interviewed apart from themselves, keeping in mind that together they should be able to give an overview of all aspects of ICU up care in their clinic. ICU follow-up coordinators were left to decide when and where the interviews should take place.

D

The use of semi-structured interviews at the different participating hospitals was chosen as it allowed the collection of sufficient data while giving the participants the possibility to decide where and when the interview took place, to make it fit their busy schedules. This method also gave us the opportunity to decide on relevant topics and questions beforehand while at the same time giving the participant the opportunity to point out nuances and introduce new topics. Each interview was audio recorded for use during data analysis. The length of the interviews was approximately 90 minutes per interview. Literature describing ICU follow-up clinics was used to create a semi-structured interview-guide, relevant questionnaire questions used in the 2012 ICU follow-up survey were also reused [15]. The following topics were discussed during the semi-structured interviews: intake of patients, logistics, personnel, follow-up care methods, referrals, expectations of the ICU follow-up project, and outcomes. The full list of interview questions can be found in Appendix 2. To test the thoroughness of the prepared interview method, the first interview was conducted with a participant closely related to the ICU follow-up project. This allowed discussing the interview method and making improvements before interviewing coordinators not directly involved in the project. Before the start of each interview the goal of the study was shortly explained. All interviewed coordinators and other attendees were also asked whether they had objections to audio recording of the interviews.

All audio recordings were transcribed verbatim by the interviewer (GM). To prepare the transcribed interviews as input for modeling the workflow of the ICU follow-up clinics, each part of the interview was labeled a question, answer or general comment. This was done using Word (Microsoft Corporation, Redmond, WA, USA) and Excel (Microsoft Corporation, Redmond, WA, USA). Questions from the interview-guide that were answered during the interview but lacked in detail were sent to the ICU follow-up coordinators by e-mail to have all questions answered in sufficient detail.

D

Data analysis of the semi-structured interviews resulted in the development of models representing workflow within each ICU follow-up clinic. The general-purpose modeling language Unified Modeling Language (UML)

24

was used to visualize the ICU follow-up workflow in each ICU. This was done by modeling swim lane activity diagrams which gives a graphical representation of the flow of activities and actions. The swim lane activity diagrams were then used to compare the workflow of current ICU follow-up clinics with each other and with the recommended workflow and the set of recommendations.

R

ESULTS

C

The study population (n=4) consisted of different representative delegations of four ICU follow-up clinics in the Netherlands, i.e. the Academic Medical Center Amsterdam, the Gelderse Vallei hospital Ede, Gelre hospital Apeldoorn and Groene Hart hospital Gouda. Characteristics of the clinics in these hospitals is given in Table 3.

TABLE 3 ICU FOLLOW-UP CLINIC CHARACTERISTICS

Name AMC Gelderse Vallei Gelre Groene Hart

Hospital type University hospital Non-teaching Teaching Non-teaching

IC level 1 3 3 2 2

IC beds 28 12 14 11

IC patients per year 2396 737 707 577

Patients ≥2 days admitted per year

1028 501 312 215

Patients ≥2 days mechanically ventilated per year

732 472 N/A2 170

Amount and type of employees for 1 ICU follow-up clinic day Physiotherapist: 2 Secretary: 1 Researcher: 1 Nurse: 2 Physiotherapist: 1 Intensivist: 1 Secretary: 1 Nurse: 3 Intensivist: 1 Secretary: 1 Nurse: 2 Physiotherapist: 1 Intensivist: 1 Secretary: 1

Source: NICE registry 2013, Interviews and hospital websites. 1) IC levels as determined by the NVIC. 2) Hospital does not provide this data to the NICE registry.

C

In Table 4 the recommendations for ICU follow-up care are compared with the clinics’ current workflows. All ICU follow-up clinics in this study already adhere to at least six out of the ten (60%) recommendations.

The patient inclusion criteria in the majority (75%) of the clinics differ from the given recommendation for ICU follow-up care. Choosing to include all patients that are admitted to the ICU for more than 48 hours instead of those who were also mechanically ventilated increases the population of eligible patients. In these clinics combined this would be a 21% decrease in eligible ICU follow-up patients. Note that this excluded the Gelre hospital since the NICE registry had no data about how many patients are mechanically ventilated in that hospital.

Four recommendations are already common practice in all included ICU follow-up clinics. All clinics invite the partners of the patients, plan the visit to the ICU follow-up clinic 12 weeks after hospital discharge, screen patients with respect to their needs and ICU related sequelae and screen impairments in the following health domains: movement related functions, sensory functions, communication problems, symptoms of post-traumatic stress, symptoms of anxiety and depression, cognitive functioning, psychosocial functioning, and sexual functions.

Two of the four hospitals in this study have implemented their ICU follow-up clinic as a one-time opportunity to identify problems and barriers that hinder the rehabilitation of the ICU follow-up patient. Additional follow-up visits are part of the recommendations, which would increase the amount of patient visits and workload for care givers.

25

Using questionnaires to get information about patients before they visit the ICU follow-up clinic is fairly common (75%). However, the three hospitals that use questionnaires work with a paper based system. The recommendation encourages the use of these screening instruments happens electronically rather than on paper.

TABLE 4 COMPARISON OF RECOMMENDATION ADHERENCE BY ICU FOLLOW-UP CLINICS

Recommendation AMC Gelderse Vallei Gelre Groene Hart

Invite all patients who were mechanically ventilated (MV) more than 2 days.

Yes No, >48 hours on IC No, >48 hours on IC No, >48 hours on IC Invite the partners of ICU

survivors.

Yes Yes Yes Yes

Plan the visit to the ICU follow-up clinic 12 weeks after hospital discharge.

Yes Yes Yes Yes

Plan one visit with the possibility for a follow-up at indication. No, After 3 months, follow-up in special cases No, After 3 months, follow-up in special cases Yes, After 3 months, after 6 and 12 months by phone Yes, After 3 months, after 6 and 12 months by phone

Screen patients with respect to their needs and ICU related sequelae.

Yes Yes Yes Yes

Screen impairments in the following health domains: movement related functions,

sensory functions,

communication problems, symptoms of post-traumatic stress, symptoms of anxiety and depression, cognitive functioning, psychosocial functioning, and sexual functions.

Yes Yes Yes Yes

Use electronic

patient-reported screening

instruments to identify patients in the need for ICU follow-up care. No, Paper based questionnaires No, Paper based questionnaires No No, Paper based questionnaires

Have an ICU nurse whether or not with an intensivist carrying out the ICU follow-up clinic.

No,

ICU nurses and Intensivist not involved in process

Yes Yes Yes

Involve a physiotherapist to perform a comprehensive physical screening.

Yes Yes No,

Referral is possible

Yes

Integrate follow-up care data into a national quality registry for intensive care to monitor and improve quality of life and functional status of ICU patients. No, System with necessary capabilities not available No, System with necessary capabilities not available No, System with necessary capabilities not available No, System with necessary capabilities not available

26

The composition of an ICU follow-up team (Table 3) differs between clinics and is not in accordance with the recommendations in some hospitals. At the clinics included in this study, 75% of them had an ICU nurse whether or not with an intensivist carrying out the ICU follow-up clinic. The standard involvement of a physiotherapist was evenly common.

Lastly, all interviewed ICU follow-up clinics use their own collected data for local research.

P

The overall pattern of work for each ICU follow-up clinic was summarized graphically in the swim lane activity diagrams following the structure of the recommended workflow diagram (Figure 8). The workflow diagrams of individual hospitals can be found in Appendix 3. Modelling the workflow of the different clinics in the same structure reveals that although specific details of workflow differ, there are many common elements to be found. Workflows of all ICU follow-up clinics in this study can be described with the following steps: ICU admission, hospitalization, informing patients, ICU follow-up invitation, questionnaires and finally the consult. During ICU admission and the following hospitalization all included hospitals have structured their workflow similarly. For instance a diary or workbook is used to give the patient a better understanding of what has happened to them during their IC stay. In some instances (n=2) the medical professionals are supposed to write in the diary and in another ICU (n=1) it is aimed towards the caregivers, i.e. spouses and relatives. The workbook (n=1) is used by patients themselves.

It is during the hospitalization that all hospitals start the inclusion of their patients for ICU follow-up care. This is done either manually (n=2) or by running a query in their PDMS (n=2), resulting in a list of eligible patients which is sent to the secretary of the ICU follow-up clinic.

Informing patients of what ICU follow-up care involves also happens during either one of the two aforementioned phases. This is done by talking to the patients or their caregiver and additionally providing brochures about ICU follow-up care. All clinics also send their brochures and further information to the patients’ home.

After sending this information, patients are invited for a consult. At this moment all clinics make an appointment either by sending a letter or by calling the patient or their caregiver at home. Most (n=3) clinics confirm their scheduled appointments one or two weeks in advance.

After an appoint has been scheduled the ICU follow-up clinics that use a paper-based questionnaire booklet (n=3) send it to the patients. This is planned in such a way that the questionnaire booklet can be returned to the ICU follow-up clinic by mail before the scheduled ICU follow-up appointment.

The main component of each ICU follow-up visit is the consult in which the patient has the opportunity to have a conversation about their IC related problems with a medical professional. Although the goal of each ICU follow clinic is the same, the composition of each ICU follow-up team differs and the visits show variation on the same idea as can be seen in Appendix 3.

C

The recommended workflow focusses on those aspects that are needed for collection of the data necessary in the quality registry as required by the set of recommendations. When patients complete the questionnaires in the patient portal their data is automatically collected in the NICE registry. Therefore the recommended workflow largely consists of steps aimed at data collection during the phase when patients are invited for ICU follow-up care, which is not a fully automated process. In Figure 8 most of these steps can be seen in the center of the swim lane activity diagram. These steps will be new for all clinics since these steps were primarily introduced for the specific research purposes of the ICU follow-up pilot.

27

Besides placing great emphasis on registering if and why patients do or do not want to receive ICU follow-up care, the recommended workflow also emphasizes the use of digital questionnaires through the ICU follow-up patient portal. Although the use of questionnaires might not be new for most ICU follow-up clinics it does have significant impact on their workflow when working digitally instead of paper based.

By following the recommended workflow, it becomes the task of the secretary to invite patients for ICU up care. When patients do not respond to the invitation or indicate that they are not interested in ICU follow-up care the secretary is tasked to find out the reasons behind such a decision. For instance patients suffering from PICS might avoid contact with the ICU while they could benefit from ICU follow-up care. Additionally the secretary must try to convince patients to use the ICU follow-up patient portal and support the patients in using the system. Only in special cases such as having no Internet access, patients are sent a traditional paper based questionnaire booklet.

Preparation by the medical professionals whose task it is to have a consult with the patient usually involves reading the results and certain specific questions in the paper based questionnaire booklet. The ICU follow-up system is capable of displaying a summary of the relevant results of all questionnaires in a single window, but it could take some time to adjust to this digital version. On top of that medical professionals have the possibility to check and change answers given to individual questions in the questionnaires during the consult.

In three of the four included hospitals the list of questionnaires is more comprehensive than the recommended list of questionnaires. These differences are presented in Table 5.

TABLE 5 COMPARISON OF STANDARDIZED QUESTIONNAIRES USED BY ICU FOLLOW-UP CLINICS

Recommendation AMC Gelderse Vallei Gelre* Groene Hart

HADS SF-36 TSQ MUST SIP-68 HADS TSQ MVI CISS SF-36 MUST SIP-68 HADS TSQ MVI CISS SF-36 MUST SF12 ZIL SIP-68 HADS TSQ MVI CISS SF-36 MUST Source: interviews

*Not used during consult

D

ISCUSSION

P

This study provides insight in challenges future end-users might encounter when their clinic joins the ICU follow-up project. Overall, the evaluation of the workflow of four ICU follow-up clinics shows that they are already adjusted or can easily be adjusted to the set of recommendations as their workflow structures are similar. None of the identified challenges jeopardize the feasibility of the ICU follow-up project. Comparing the existing ICU follow-up clinics to the recommendations showed that the clinics already adhere to 60% of the recommendations, this simplifies fitting the ICU follow-up system into the workflow of an ICU follow-up clinic. The following changes in workflow of existing ICU follow-up clinics are introduced by the introduction of the set of recommendations and the recommended workflow. Both the inclusion criteria for patients and the clinics’ policy for additional follow-up visits directly influences the amount of patients that will be invited to a clinic. In this study the inclusion criteria for ICU follow-up patients would result in a 21% decrease in the amount of invited patients compared to the current situation. Clinics are however allowed to invite a wider group of patients as this recommendation is seen as a minimum. Additionally, changing an ICU follow-up clinic from a single visit to a multi visit event means that patients are being tracked over time. However, the possibility for

28

additional follow-up visits at indication to the clinic is currently not standardized in two out of four hospitals, which would results in a higher workload for caregivers and the secretary when the recommendations are fully implemented. It is expected that when a second or third follow-up consult is needed, the questionnaires will be presented to the patients again and that these additional consults will take place by phone.

Expecting ICU follow-up clinics to use electronic patient-reported screening instruments to identify patients in need for ICU follow-up care results in a major change in workflow since three of the clinics work with paper questionnaire booklets and one hospital did not use questionnaires for screening at all. This change manifests itself in several workflow steps and affects several team members. For instance, the secretariat of a clinic will receive additional tasks to collect information about why or why not patients complete the questionnaires through the patient portal or do not want to participate at all. They are also responsible for convincing patients that they should use the patient portal even when they prefer to use a paper based questionnaire. The secretary is also the first point of contact when a patient has questions about the patient portal. From a research perspective the secretary serves a vital role in collecting all the necessary data about patient participation. This is why this group of stakeholders should be well prepared and understand the importance of this responsibility.

Medical professionals working with the electronic questionnaires will have the results of the questionnaires automatically presented to them with the possibility to make changes if needed. However, this necessitates the use of a computer at the point of care which changes the dynamic of the communication between a patient and medical professional. Additionally the list of standardized questionnaires that are recommended differ from those that are currently used in the four clinics. However, clinics are allowed to use additional questionnaires since those that are given in the recommendation are seen as a minimum.

The set of recommendations also indicates which medical professionals should preferably be part of an ICU follow-up team; have an ICU nurse whether or not with an intensivist carrying out the ICU follow-up clinic. How this influences workflow mainly depends on the organizational structure of each hospital, i.e. an ICU follow-up clinic that was historically started and carried out by physiotherapists is very difficult to change whereas involving a physiotherapist in an existing ICU follow-up team is considerably easier.

Lastly, providing follow-up care data to the NICE registry will be handled by the ICU follow-up information system automatically. When patients provide completed questionnaires through the patient portal and the whole ICU follow-up team uses the ICU follow-up hospital software, no additional information needs to be provided by ICU follow-up clinics to the NICE registry. Finally, all interviewed clinics use their own collected data for local research before joining the ICU follow-up pilot. Although this is not the same as providing data for a national quality registry, it does show that these clinics are used to providing unambiguous data for research purposes.

S

A strength of this study is the wide variety of types of hospitals that are included in this study. Having ICU follow-up teams participate in our interviews from a university hospital, teaching hospital and two non-teaching hospitals resulted in a more diverse overview of existing ICU follow-up clinics in the Netherlands. Secondly, this study gives clear recommendations for improvement of the ICU follow-up pilot and project. The present study does have several limitations. Although many clinics were invited, only a few participated in our study, as the interest to participate in the ICU follow-up project was limited in this early stage. We have given an overview of which changes to the workflow of existing ICU follow-up clinics are introduced by the introduction of the set of recommendations and the recommended workflow, but it is possible that there are types of problems caused by the introduction that did not occur in this selection of four clinics.

29

We chose UML swim lane activity diagrams to visualize the workflow in each ICU follow-up clinic because it is a modeling technique that is well-known within the healthcare setting. Moreover, it helps to identify roles and responsibilities in tasks that are multidisciplinary in nature. Literature suggests that creating multiple types of models could be beneficial to a project [34]. For instance a state transition model could be made for a more patient-centered model of the care process. A communication diagram can be used when the communication between individuals and departments are key issues. We expect that taking a different perspective could have resulted in additional identified changes in workflow. Being limited to a single model type was mainly caused due to time constrains in the SRP.

Due to delays in the development of the ICU follow-up information system it was not possible to map the workflow of ICU follow-up clinics working with the new information system. At the start of this SRP it was expected that during the research period the ICU follow-up system, which was still in development at that time, would be implemented and used by several hospitals. This would have given us the opportunity to see how several ICU follow-up clinics end up working when trying to adhere to the set of recommendations and recommended workflow. Since development took longer than expected this was solved by comparing the current workflow of ICU follow-up clinics with the recommended workflow and recommendations. Looking at the real changes and problems that occur when adapting to a new information system and set of recommendations might have resulted in additional insights. Furthermore, because of the same circumstances, this study only looked at hospitals that already had an ICU follow-up clinic.

In our study we have shown that existing ICU follow-up clinics already tend to adhere to a considerable amount of recommendations, i.e. at least six out of ten recommendations. However, three of the hospitals participating in this study were also involved in defining the set of recommendations which could explain the high level of adherence a priori. Nevertheless, a total of 16 ICUs participated in the round table conference that resulted in these ten recommendations.

R

The Netherlands is not the only country focusing on ICU follow-up research [15]. During a round table meeting in 2002 several European, Canadian and American hospitals have already discussed ICU discharge screening tools and ICU follow-up clinics as a means to improve care [3]. Later, for instance in the United Kingdom Griffiths et al. has performed a national survey among ICU follow-up clinics in 2006 [36]. Research like this shows a continuous interest in guidelines for ICU follow-up care. Our study did not directly focus on the development of ICU follow-up related recommendations and guidelines, but did show that even if a set of recommendations is drafted its implementation should be considered carefully. We showed that changes in workflow can influence the ability to adhere to a set of recommendations, something that needs to be considered when making the recommendations a national standard.

A recent study that describes and compares models of intensive care follow-up in Denmark, Norway and Sweden, gives an overview of the establishment and continuation of ICU follow-up programs in Scandinavia [37]. Similarly to the Netherlands, in Scandinavian countries ICU follow-up programs have evolved as individual initiatives. They identified four ICU follow-up models and some additional variations, however they did not create any workflow models to enable direct comparison with our study. Their study however shows a clear resemblance with the set of recommendations and recommended workflow that were discussed in our study. These similarities also correspond with the high level of adherence we found when comparing the already existing clinics in our study. This could indicate that the amount of different forms of ICU follow-up programs is indeed fairly limited. Their conclusion is that ICU follow-up should be an integral part of patient therapy and that the different programs should be harmonized with clear goals enabling program assessment. In our study we contribute toward this same idea.

In a UK survey the implementation of the Clinical Guideline CG83 regarding rehabilitation after critical illness is being determined [38]. The conclusion of the article is that across the UK the implementation of this guideline