Lessons from the literature and stakeholders

(1)

Lessons from the literature and stakeholders

which could aid the implementa on of an

EHR across a mul -site health system

Thesis by Christopher Falkenthal

Mentors: Oliver Furness, Brian Griffin and Katie Kent Supervisor: Erik Joukes PhD

University of Amsterdam Master Thesis Medical Informatics

(2)

Lessons from the literature and stakeholders which could aid the implementation of an EHR across a multi-site health system

Author Christopher Falkenthal 12003859

Department Medical Informatics, University of Amsterdam Meibergdreef 9, 1105 AZ Amsterdam Supervisor Erik Joukes PhD

Department Medical Informatics, University of Amsterdam Meibergdreef 9 1105 AZ Amsterdam Mentors Oliver Furness, Brian Griffin and Katie Kent

Deloitte Ireland, Dublin

Location Dublin and Wexford, Ireland

(3)

iii

Acknowledgments

I would like to thank my mentors, Oliver, Katie and Brian, who during my internship in Deloitte Ireland helped formulate a research question which became the title of this thesis. I am very grateful to the mentors for offering advice, support and feedback on the methods while still allowing me to conduct the research independently. Thank you very much for an informative and enjoyable internship at Deloitte Ireland and thank you also the wider Deloitte Dublin team.

Thank you to my supervisor Erik, University of Amsterdam, for his advice and feedback, all of which was very valuable.

While developing an understanding of this field I reached out to stakeholders in the Netherlands and the US and to academic staff at the University of Amsterdam who all kindly responded with support and information. Thank you.

My thanks also to the Department of Communication Science at the University of Amsterdam for agreeing to host my online questionnaire.

A special thank you to all the participants in the online survey and to those who agreed to a follow-up interview. Your contribution was invaluable to this research. I was humbled that you took the time to participate and it meant a lot to me personally as a Masters student doing the research and preparing this thesis.

(4)

iv

Abstract

Introduction: A number of multi-site health systems in the US have implemented a single EHR across hospitals and other care sites in order to create a unified patient health record. There are examples of such implementations in Europe and further ones are planned. Lessons from past multi-site EHR implementations may provide the leadership of these future European projects with an additional resource as they prepare for their project and this study seeks to identify and collect relevant lessons.

Methods: A literature review was carried out and lessons were collected from relevant articles and organised into a framework. Following this, the primary author developed an online questionnaire and conducted follow-up interviews with US stakeholders who had held leadership roles during such implementations to corroborate and add to lessons gathered from the literature.

Results: From the 26 articles which were included, 18 broad lesson areas were gathered. Seventeen stakeholders from 14 US health systems which had carried out multi-site implementations took part in a survey, 11 of whom completed follow-up interviews. All the participants noted improvements in the metrics for at least some of the most important hoped for benefits. The implementation approach taken by the health system was discussed, with the majority of participants using big bang implementations for their hospitals. An overview of some of the most important challenges and how they were dealt with was provided. Common lessons identified from the literature and from stakeholders included the importance of governance, lessons being learned from early hospital go-lives, vendor support and clinical involvement. A number of additional lessons were identified from the questionnaire and interviews: these projects are large programs of change; there is strong emphasis on the importance of trying to achieve and maintain a standard build across the health system avoiding unnecessary individual hospital modifications; and there should be recognition that standardisation of clinical content for the EHR will be an ongoing effort.

Conclusion: While this study has not created a definitive list of lessons, it is hoped that the findings should provide points for consideration to those involved in implementation projects.

Strengths and weaknesses of this study: A strength of this study is the fact that the main research question was considered from multiple angles by searching for lessons from the literature and from those, who in leadership roles, experienced such implementations. Another strength is the depth of knowledge and support the mentors and the supervisor were able to offer in developing the research question and introducing the topic area to the primary author. A weakness of this study is that all research methods (selection, collection, reviewing and analysing of the data etc) for all chapters was carried out for this research thesis by the primary author alone which raises the risk of personal bias or error.

(5)

v

Introduc on

1.1 Introduc on

A number of health organisations in the U.S. have implemented shared information systems (Elec-tronic Health Record (EHR) / Elec(Elec-tronic Medical Record (EMR)) for recording and processing patients’ data throughout their hospitals [1]. The approach of implementing the same EHR/EMR across multiple sites is being undertaken in Europe as shown by research and business cases for such implementations [2] [3] [4] [5]. A Deloitte thought piece from the US setting noted how after establishment of partnerships between health care organisations, the implementation of the same EHR/EMR instance across the new care system can better support a higher level of integration in the organisation [6] (see appendix A.1 for more detail on integrated care).

In a lot of the literature EHR, EMR [7] and Electronic Patient Record (EPR) [8] are used inter-changeably reflecting that there has been a blending of the definitions and eroding of the level of distinction between these terms. For simplicity, we will only refer to EHR from this point on. For the original distinctions in the definitions see appendix A.2.

For this study an EHR system is defined as a system for storing, retrieving and processing a patient’s health data in a digital format and allowing the delivery of care including processing diagnostic orders and providing ordering and medication management features. The EHR can also have inte-grated modules that support the financial and resource management for the care services delivered. It is used by more than one care specialty or care area to record the provision of care. The system will usually have a portal allowing a patient to interact with their health record. This definition is similar to the one painted by the HL7 EHR-S functional model document. According to the HL7 EHR-S functional model, the EHR system has a number of different functions, including: clinical documentation, communication and management of orders, and providing decision support to the user [9].

If implemented correctly the storing, modifying and accessing of patient’s data in electronic forms offer benefits over paper-based record systems for patient safety, quality and saving of costs [10] [11]. In 2019, Deloitte Centre for Health Solutions in the UK, a health care research arm of Deloitte, carried out a large survey with clinical care providers in the UK (1,500 clincal staff) as part of wider research on digital health. The survey respondents currently view EHRs they use as the most important technology “for helping to improve the efficiency and effectiveness of clinicians and patient care” when compared to other digital health technologies, but the researchers did note that these information systems still suffer from fragmentation and a lack of interoperability between them. [12]

(8)

An interoperable solution/Health Information Exchange (HIE) between the EHRs can make it pos-sible to share the patient’s information across an organisation which still uses a number of different information systems for recording care [6], but may not support the same level of integration [6] as using the same EHR system. A study of health care systems in the US found that those which have a standardised EHR platform have a higher rate of eHealth adoption in general [13]. Sixty percent of one hundred business, informatics and clinical staff from health care organisations in the US (including 32 who hold executive roles) surveyed by HIMSS media cited the implementation of common integrated EHR as the approach they were taking to increase interoperability within their organisation. [14]

An Australian independent review of an EHR project listed the arguments for the same core EHR being implemented across multiple hospitals in a regional health system as: it provides a unified view of patient information, has a lower procurement cost, staff should be familiar with the system if they move between hospitals and training costs should be reduced with less systems, reduced complexity of integrating with multiple systems; resources can be concentrated on configuration on the one EHR system; and a single core EHR allows for standardised care including clinical decision support. [15]

1.2 Problem se ng: European mul -site EHR

implementa ons

Ireland

eHealth Ireland is a division of the Irish health service (HSE) set up to spearhead the use of ICT in Ireland’s health system.

A short analysis of Ireland’s national EHR program on eHealth Ireland’s website [4] illustrates how in a European country an interoperable solution may be implemented alongside a multi-site EHR implementation. eHealth Ireland aims to deliver through its national electronic health record strategy a comprehensive integrated EHR platform which will make: “.. key patient data from operational systems accessible across the entire continuum of care by relevant users, including the patient themselves..” [16]. The national EHR will therefore support the delivery of integrated care in Ireland [16] and additionally aims to standardise care around best practices. The national EHR, as described in 2016, is to be made up of the following components:

• acute care EHR system implemented in all hospitals

• community care EHR system implemented for community care

• interoperable solution: an integration mechanism or engine which communicates data from these and other information systems (GP’s electronic medical record (EMR) etc)

• national portal which allows both health care staff and patients to view a patient’s shared record which contains data from the different care levels [16]

The HSE is already successfully implementing a common maternity EHR system which is to be used in all of Ireland’s public maternity hospitals [17]. per eHealth Ireland’s website, the acute care EHR system appears to be in the Pre-procurement phase.

(9)

using a common EHR system across it’s secondary care setting in each of its five health care regions. Each of the regions have selected a vendor (Epic, Systematic), but not all have implemented [5]. To ensure the exchange of data between the regions and care levels there is integration between different information systems as well as an interoperable solution e-health portal that is in place allowing care providers and patients to view and interact with the health record which was built on existing information system infrastructure including the EHRs. [18] The region of Southern Denmark’s EHR implementation envisages streamlining and standardising of workflows [19].

Norway

Norway’s acute services IT is delivered via health care regions [5]. The central Norway regional health authority is currently carrying out the implementation of a comprehensive EHR (Epic) for its hospitals and other care levels. [20]. The other regions’ acute services use an EHR from the vendor DIPS [5]. Central Norway regional health authority’s implementation has the goal of placing all data in the one record for each patient, improving patient safety and the quality of care [21].

Other examples of planned implementations of a comprehensive EHR across multiple care sites can be found in Swedish regions (Cerner) [22].

These large European implementations including those which are currently taking place will be the context and reason for carrying out this research.

1.3 Problem descrip on

The implementation of an EHR at a single hospital is acknowledged as being a complicated process [23]. Articles note how some EHR implementations can have limited success or be viewed as troubled or unsuccessful [24] [25] . The literature shows health systems can suffer as a result of an EHR project from: cost overruns, staff unhappy with the EHR system, or project behind schedule on its delivery or a suspended implementation. It is important to note that health care organisations can recover from difficulties in their EHR implementations for example a large EHR implementation can initially be viewed as troubled, but the system can later transition to running successfully [20].

There is however limited availability of lessons on multi-site implementations, also it would appear that the implementation of the same EHR across a multi-site health system brings an extra layer of complexity when compared to the implementation of an EHR system at a single hospital. A study which seeks to identity and present lessons on the barriers, facilitators and common actions taken during such multi-site implementations may aid health care organisations which are preparing to undertake such implementations to experience a smoother process. This study looks at the topic in the context of assisting European health care organisations which are implementing or preparing to implement these systems.

The key research question is: What lessons can be learned from previous multi-site EHR imple-mentations? Sub-questions that arise from the key question are:

• What barriers and issues are recorded in the literature for these implementations?

• Have stakeholders who are involved in the implementation or running of such a multi-site EHR experienced the challenges mentioned in the literature and what additional ones have they faced?

(10)

• What are the greatest challenges from the stakeholders’ perspective during such implemen-tations and what helped in overcoming them?

• Why did the health systems take the implementation approach selected? Have the expected benefits materialised since the implementation?

1.4 Project outline

Different study types were employed to gather lessons that can be learned from previous imple-mentations. An in-depth search of both the academic and grey literature was conducted in order to identify lessons to be found in the literature (chapter 2). This helped to act as the foundation for building up a basic framework of lessons. An online questionnaire and follow-up interviews aimed at garnering the experience of those who took on leadership roles in the US during previous multi-site implementations were carried out (chapter 3). The findings from both the literature review and mixed method study were then considered in order to present a set of key lessons for European health care organisations that are preparing to undertake such an implementation (chapter 4).

(11)

1.5 Relevance of the study

We believe this study has both academic and practical relevance. The academic relevance is that while the topic of EHR implementations has been studied extensively, the gathering of lessons for multi-site implementations appears not to have been undertaken to the same extent with studies using a mix of methods. Hertzum et al. 2019, stated that previous implementations provide valuable sources of learning for organisations preparing to undertake an implementation [20]. The practical relevance of this research is that it may aid stakeholders who will take on a leadership role during such projects by providing a fresh perspective on the multi-site implementation.

Health care organisations and government bodies carry out literature reviews to identify the lessons that can be learned in preparation for their own project implementations [11] [26] or for an indi-vidual component i.e implementation of electronic medication management systems [27] which is an individual function area of an EHR platform.

Online questionnaires and interviews with key stakeholders have both been used as a method to gain an understanding of the experience of an EHR implementation and Scot et al. carried out a literature review and workshop with those who held leadership roles during an implementation with the aim of aiding other Australian hospitals with their implementation, but sought to create a checklist for an individual hospital rather than lessons for a multi-site implementation. [26] Therefore the methods of our study are justified.

(12)

6

CHAPTER 2

Lessons from the literature on the

implementa on of an EHR across a

mul -site healthcare system

2.1 Background

As outlined in chapter 1, valuable lessons can be learned from the literature for organisations prepar-ing to undertake an EHR implementation, yet there appears to be limited exploration of lessons on the topic in the context of multi-site EHR implementations. The following question therefore arises:

• What lessons can be gathered from the literature on multi-site EHR implementations?

To answer this research question, this component of the study looked for examples from the lit-erature which report on some element of previous implementations of EHRs across multiple care sites or a network. The examples in the literature provided illustrations of the approaches/actions taken, challenges/barriers faced and the factors which facilitate an implementation. These were then organised into lessons.

EHR systems are made up of different modules/components. Hospitals and health organisations can implement these systems together or on a modular basis from either the same vendor or best of breed when multiple vendors’ solutions are used [28] [29]. Single vendor strategy refers to when one vendor’s solutions is used to create the enterprise EHR [28] [29]. Also in a discussion with the primary author a US stakeholder advised it can arise that there are multiple single vendor EHRs in different hospitals making up a health system formed via a merger or partnership. Appendix A.4 provides more detail on the different approaches to creating an EHR system.

2.2 Method

Development of the keywords used in the searches

The keywords were developed by taking the keywords used in related literature reviews as well as by looking at the terms used in primary studies.

(13)

faced by an EHR system [23]. This, and the fact that these modules are implemented as part of a single vendor EHR implementation, led to our decision to include studies which focus on the implementation process for a single module/component. Keywords included to try and identify ar-ticles that focus on these modules were as follows: CPOE (and related terms), eMAR (and related terms), clinical documentation (and related terms).

Search process

For the academic search, a set of databases were searched (Scopus, Embase(Ovid)). Two searches were conducted. After the first search, the number of keywords used was increased and results limited due to time constraints, with those with a publication year >=2016 only selected. For the keywords used for the first and second searches see appendix A.6.

The grey literature search included a search of several websites identified as likely to have produced relevant articles on the topic. Also, a structured search of Google was carried out using some of the keywords, with the first five pages reviewed for each of these searches.

A modified PRISMA method was used to identify studies which should be included in the literature review. After duplicates were removed, studies were then reviewed by title and abstract. Finally, a text review was carried out.

Criteria for inclusion of a study in the literature review:

• Describes an EHR shared across a health system (two or more hospitals, or one hospital and set of free standing clinics)

• Describes the implementation of a module/component of an EHR across a health system • A literature review which has as its focus lessons for a multi-site implementation

• Describes any stage of the implementation process of an EHR across a health system or the optimisation of the EHR after implementation

• Year of publication >= 2007 (for 1st search) and >= 2016 (for 2nd search), difference in dates due to time constraints

Exclusion criteria during full text review:

• Not clear if the EHR is implemented in multiple care sites and/or does not include a hospital • Lacking enough detail on lessons i.e. the language makes it unclear what the author is

refer-ring to

• A commentary on company’s product/services • Full text not available

• Describes a Health Information Exchange (HIE)/Interoperable solution

Extrac on of results

The included articles were reread to identify lessons. Lessons that were challenges/barriers to an implementation were identified from the articles, either when it was directly stated to be a chal-lenge/barrier or else referred to or described in the article for implementation or adoption of the EHR and in this authors’ view was a lesson. The same approach was taken for identifying facilita-tors for the implementation. Challenges/barriers and facilitafacilita-tors or actions were then grouped under a common category based on similarity. These categories formed the main lessons for example, a lesson regarding governance.

(14)

Organisa on of the results

A number of the identified literature reviews which focused on the implementation of an EHR sys-tem used frameworks or categories for organising their findings [25] [29] [30]. The implementation of an EHR system within a health care organisation involves a series of phases/steps. Kucukyazici et al. describes in their literature review how these phases/steps chart “the journey of an EHR implementation through the health care system over time with respect to strategic and operational levels” [25]. These phases have activities which need to be completed and deliverables/outcomes that mark the completion of the phase and that the next phase can begin [25]. Figure 2.1 shows the implementation phases for an EHR that are used in our study.

FIGURE 2.1 Phases of an EHR implementation including the main activities. Adapted and modified version of a diagram from Jones et al. [29] and based on the description given by Kucukyazici et al. [25] of the different implementation phases.

In addition to organising results by the EHR implementation phases, we also put them into the multi-perspective model. This framework of implementation phases and multi-perspective model is a slightly modified version of that used by Jones et al.in their literature review. It was felt that this framework was the most suitable as their review aimed to gather lessons on tasks, challenges and facilitators for large EHR implementations [29]. The multiple perspectives model is “a systems-based theoretical framework for understanding complex organizational systems” which is made up of the organisational perspective, professional perspective and the technical perspective, all of which have an effect on the implementation or are affected by an implementation. The three perspectives from the multiple perspective model are described below:

• organisational perspective: organisation type, policies and procedures of the organisation, and organisational vision, goals, politics, and culture e.g., leadership that affects the imple-mentation [29]

• Professional perspective: thoughts/behaviours for different groups that are stakeholders in the implementation (management, clinical staff and IT team and the other stakeholders) [29] • Technical perspective: EHR, other information systems and the hardware [29]

2.3 Results

(15)

Of the 26 articles included, nine were from the grey literature, two of which were government/health department supported and 17 were academic. Seven were qualitative studies and one a literature review (which included three studies that looked at academic centres, but the rest of the studies were for health care systems with multiple hospitals). The reminder of the articles were descriptive studies which had either a limited framework or method or none at all. All but one of the studies looked at a multi-hospital EHR implementation and other care levels were also included in some of the studies. The one study which did not deal with a multi-hospital implementation looked at the implementation of the EHR module for outpatient offices and for an inpatient module from the same vendor for a department at a hospital [31] and another study, while a multi-hospital imple-mentation, looked only at the implementation at an outpatient system. One qualitative study [32] had participants from multi-hospital health systems, but also three from large individual hospitals and another qualitative study included 12 clinical nursing informatics leaders from integrated health systems with a range of hospitals of between 1 to 35 [33].

For the studies that focused on a single multi-site implementation they covered 20 different single EHR implementations. four were in relation to implementations that took place in Europe, 12 in the US, one in Australia and three in Canada. Five different EHR vendors were covered while seven of these studies failed to identify the EHR vendor.

Lessons

Lessons learned from the literature on the factors that will aid a multi-site health system’s imple-mentation are listed in the table below by impleimple-mentation phase and the lessons are placed into one of three perspectives of the multiple perspective model.

Pre-implementation

Organisational perspective

• Governance Have in place the correct governance and leadership structure including ensur-ing involvement of different sites/local regions and shared accountability [15] [34] [35].

Professional perspective

• Management readiness There needs to be readiness among the management team to manage such a project [36] [37].

• Clinical engagement Have representative clinical engagement from throughout the health care system in designing the vision for future state of the organisation’s EHR and in selection of vendor, helps create buy-in [38] [39].

Technical perspective

• Hardware The hardware which staff will use for viewing and interacting with the EHR system needs to be considered and planned for during pre-implementation [15].

(16)

Implementation

• Standardisation of build If a level of standardisation is a goal, ensure communication with clinical champions/staff [40] [41] [42]. Realise that standardisation may take longer then planned [43]. Maintaining standardisation across hospitals in a health care system can be an ongoing challenge and balancing act with allowing autonomy to hospitals [33].

• Implementation plan Be prepared to adapt the implementation plan [15] [37] [44] [45] for example, phased to big bang [45] implementation, the order of sites go-live [44] and be ready to reassign resources when needed [15].

Professional perspective

• Vendor engagement Ensure vendor engagement and vendor knowledge are utilised [15] [46] [47] . Have the right number of vendor personnel supporting the implementation project [15]. • Clinical involvement Have involvement of and communicate with clinical staff (including nursing staff) from across the system (different sites) [15] [36] [40] [48] and specialties during the design and build. [15] [40] [48] This allows for an acceptable standardised system to be built [15]. Helps to avoid clinical staff feeling that their first interaction with the system is at go-live or to avoid serious workflow issues at go-live [36]. Clinical leadership and transformation is required for the implementation to be successfully executed [15].

• Workflow analysis Carry out a workflow analysis across the sites to identify gaps between the current workflows versus the future workflows that will be introduced [40] [48]. It was implied that current workflows should be analysed during development and design of the future version [36]. A multi-hospital system will have variation in workflows between hos-pitals for the same activities in at least some areas of care before the implementation of a new EHR [49]. Carry out comprehensive pre-go live simulations testing the environment and functionally at hospitals, this may allow for identification of system issues. (Context for this last recommendation, a health system with a mix of paediatric and adult care) [48]. • Dissatisfaction of some staff Implementing modules with standardised workflows across

the system can lead to dissatisfaction of clinical staff at some sites, at least early on after the implementation [31] [50]. The transition from an existing EHR to a new comprehensive EHR in a health system is likely to lead to some dissatisfaction [51].

• Training If there is a standardisation goal, it helps to have training material developed cen-trally and to use a central training team that engages with the local site leaders and trainers to develop the training programme, this combats variation in how the workflows are taught at different hospitals [34]. Training delivered on a system different to the version at go-live can lead to staff lacking familiarly at go-live [52]. It is implied that super users should receive additional training to the end-users [52]. It is recommended that a lot of the training should be tailored to the care roles [15] and focus on their workflows [36] [53]. It is recommended a mix of training methods should be used; online, face to face and at the elbow and clinical staff should have dedicated time for training pre go-live [15].

• Go-live support for staff During go-live and afterwards having clinical informatics staff with an understanding/expertise of the system who can support their colleagues at each site

(17)

• Modification of the vendor’s solution Vendor’s EHR solution will need to be configured [41] [50] and customised [15] [53] to create workflows that will fit with the health care system. Some health systems have had to balance level of standardisation and some local modifications may happen or be needed to the health systems build [40] [46]. An EHR that is poorly configured will lack a level of standardisation and will allow the same tasks to be completed many ways. Poor configuration was linked in the case of an implementation to lack of clinical involvement especially with those with clinical informatics expertise and it being the first implementation of the vendors solution in the country, with an underestimation of the level of effort required to localise it [15].

• Integration of the system Integration of multi-site EHR with existing information systems can be a challenge [50] [54]. Without the EHR system integrating with existing information systems there may be workarounds [15] [50].

Post implementation

• Governance Put in place or plan for the correct governance structure for the EHR system across the sites post implementation [46] [55], and for management of the data [56].

• Balance support There should be a balance in the support between sites live with an EHR and those about to go-live. If not enough ongoing support is offered to live sites, staff can feel dissatisfied with the system [15] [52] and may not use the system most effectively [15] .

• Carry out optimisation After go-live optimisation of the system will need to take place in order to improve its function [38]. For ongoing standardisation/optimisation, including dedicated time and teams formed to focus on the topic is noted as key enabler for success by stakeholders from a number of health systems [32].

• Measure metrics Reviewing performance of sites that go-live can help to identify issues and errors. [40].

Listed below are decisions and actions which may be unique to multi-site implementations. The primary author felt these were implied given they were not reported in three reviews focused on implementations of an EHR at a single hospital [11] [23] [26] yet were found in this study. They are: selecting the first sites for go-live [38]; deciding spacing between sites [45] (but at an individal hospital there may be phasing by department or modules [26]); reacting to delay in the implemen-tation for one site [15] [44]; the level of standardisation of build between regions [46]; carrying out the optimisation across sites; and setting up the governance structure for post implementation management which again takes account of the fact that the system is used across a multi-hospital organisation [46] .

2.4 Discussion

Main ﬁndings

The results of this study provide an overview of lessons from the literature which can aid in the implementation of a multi-site EHR. In total, 18 broad lesson areas were identified.

(18)

The literature search showed that ensuring and keeping clinical engagement from across the health care organisation is important during the pre-implementation and implementation phases in order to ensure clinical adoption and the design of an effective EHR system. In Europe, there are already examples of this being recognised during pre-implementation. Hertzum et al. 2019 [57], reporting on a multi-site EHR implementation process taking place in Norway, noted an emphasis placed by senior management on ensuring there was representative clinical staff input during the drawing up of the requirements for the system [57] as well as during design and build [20].

We believe a multi-site implementation needs more involvement from across the organisation and responsibility across the system when compared with a single hospital implementation. These factors are mentioned in both the grey literature and academic literature. There will be a difference in work practices between hospitals and one study noted that different hospitals in the health system present a geographical challenge during the design making communication more complicated [41].

It is likely large European implementations will need to follow some of the lessons listed in the results in this chapter in order to achieve a successful implementation. Decisions that are unique to multi-site implementations will have to be made including: agreeing a design for the EHR across hospitals as well as departments, deciding the spacing of go-live, and managing the support of live sites and those still to go-live with the EHR system.

The need for technical integration with existing information systems is likely to be important for European health systems where a number of countries already appear to have a base of interoperable solutions in place [58], any new EHR implemented in hospitals will need to integrate with these as well as existing information systems in the hospital.

There are studies showing that in health systems that transition between EHRs there is a decrease in clinical satisfaction. We would argue for any European Health systems which are mostly paper based (many European nations hospitals have already implemented information systems) this will not be an issue as long as the changes in workflows and buy-in are ensured during the project. As Solomon et al. [15], a report on a large implementation across a region which has faced challenges noted, clinical staff still preferred the EHR system to paper.

Methodological strengths and weaknesses

A strength of this study is that some of the search strategies employed allowed a wide net to be cast for identifying studies with lessons on multi-site implementations by: looking at both the grey and academic literature, using a large set of keywords including abbreviations for the common names used in industry for health record information systems and also searching a number of academic databases. A weakness of this study is that most of the included studies are descriptive, describing an implementation or component of an implementation which the authors are recalling, therefore they may suffer from recall bias. Also, these descriptive case studies which do not have a set of methods or do not use a framework make judgement more difficult on the quality of the lessons and what evidence backed-up the authors’ assertions. Additionally, the search did not include keywords which focused on the patient and staff experience of the EHR post implementation, this may mean that valuable lessons are missed on the implementation from the articles which were not considered.

(19)

This study alongside similar work

As mentioned, a number of literature reviews have been carried out regarding the process of im-plementing and adopting an EHR. Scott et al. [26] carried out a literature review in the context of a regional implementation in Australia along with a qualitative study. They organised their results as a checklist/set of questions for a health care organisation that is preparing to implement and they noted: the need for leadership from the c-suite and clinical staff during the implementation, the importance of selecting a suitable vendor, that the EHR system must integrate with the clinical workflow of staff and existing information systems, and the need to strike a balance between stan-dardisation and customisation of the EHR within the departments. They also noted that a decision needs to be made on big bang or phased for a hospital when implementing an EHR [26]. But Scott et al. considered the topic from the angle of supporting a single hospital with their implementation which would explain while there are similarities in the lessons, there is a lack of lessons for the multi-site context that our paper covered.

Jones et al., 2013 [29] aimed to present the lessons from the literature which could support a large health system in their implementation therefore, as noted in the methods section above the frame-work from their study was used by us. They carried out a wide search (in that they used a large set of keywords and searched both the grey and academic literature) and carried out a detailed analysis of their results placing them into the framework of a hospital implementation. Jones et al. did iden-tify a number of similar lessons from the literature but there were some differences; they placed less emphasis then this paper on lessons for managing a multi-site implementation in relation to governance and accountability between sites and the program. They also stated that EHR systems need extensive customization when implementing in a health care organisation to fit the workflows of that organisation [29]. Our study adds the following to their research: they searched 2010-2013, while our first search included studies from 2007 onwards and for both of our searches, studies published after 2015 were included. Also, we used different databases and our inclusion criteria were focused on only selecting lessons from studies describing multi-site implementations, while Jones et al. also included independent primary care group and single hospital implementations as they had a broader definition of a health system.

A limitation of our study when compared to some of the other literature reviews is the fact that we in-cluded descriptive/commentary pieces on implementations or the step of an implementation rather than only studies which reported results from a scientific study. But these descriptive/commentary pieces are usually written by those who supported or held a leadership role during the implemen-tation, so although the evidence is anecdotal it could be argued that the lessons are still highly relevant.

Impact of the results

This study has collected lessons on multi-site implementations of EHRs and could therefore help to aid health care organisations which are planning to undertake the implementation of such an information system to identify broadly what helped organisations in their implementation and what presented as challenges as recorded in the literature.

(20)

Next step

With a basic set of lessons on multi-site implementations gained from the literature, it would now be valuable, if possible, to gain the perspective of those who took on leadership roles to discover which of these lesson areas presented the greatest challenges and what facilitators they found. It would also be useful to identify additional challenges which may not be recorded in the literature, but have been experienced by those who worked on such implementations. Also, a common reason given for the implementation of an EHR system across sites is to support standardisation of care, but it appears from the literature that this can be a more challenging goal to deliver than expected. It would be valuable to learn about the level of standardisation that was achieved after implementa-tions and how long it took and what other benefits were seen. Therefore, the next chapter describes a questionnaire and follow-up interview study which seeks to gather these lessons first-hand from those who took on a leadership role during a multi-site EHR implementation.

(21)

15

CHAPTER 3

Lessons learnt by the leadership of US

health systems from mul -site EHR

implementa ons

3.1 Background

It is important to learn about the experience of and perspective on multi-site EHR implementations from those who took on leadership roles, both clinical and non-clinical, in order to expand on the lessons taken from the literature. This chapter describes how this was done through an online questionnaire and follow-up interviews.

3.2 Methods

The online questionnaire sought to establish: a basic understanding of the background of the health care organisation, the record system in place pre-implementation, the most important hoped for benefits as stated in the business case of the health care system, what the participant viewed as the greatest challenges during the implementation and finally, to what extent standardisation was achieved. The semi-structured follow-up interviews (which were carried out using an interview guide) sought to complement and expand on the results of the survey. This approach allowed for the results from the questionnaire and interviews to be integrated and analysed together.

Development of ques ons

The questions for the online questionnaire and for the interview guide were developed from the results of the literature review using the lessons identified in this study as well as those identified in other literature reviews. Also experts from the field kindly gave their input into the development of the questions and answer options. After the first drafts of the questionnaire and interview guide were created, they were reviewed by two researchers from Amsterdam Universitair Medische Cen-tra as well as by two Deloitte Ireland staff. Following these reviews, a former hospital CIO and a former EHR vendor employee provided their input on the questionnaire, modifications were made based on their feedback. Finally, the questionnaire was reviewed by a professional in the UK who had several years’ experience in the field of EHR implementations. The questionnaire was then piloted with a stakeholder who would be a target for the survey to test for timing and understand-ing, and modifications were made based on their feedback. After initial deployment a few small modifications were made based on the answers given by the first two participants, this involved

(22)

adding explanation to answer options for two of the questions and adding an extra answer option for one of the questions. While the core of the interview guide stayed the same, it was adapted for each interview. See appendix A.7 for a copy of the online questionnaire.

Recruitment of the par cipants

The target participants for this research were chosen based on leadership roles, both clinical or non-clinical, held by them during a multi-site implementation. These roles were identified through the literature as well as via desk-based research. Recruitment of participants and the viewing of data was only carried out by the primary author. Identification of suitable multi-site health care systems by the primary author was via HIMSS website, Health IT media and vendors’ websites. The primary author decided to contact US stakeholders only as he felt that US health systems in general are further along the digital maturity pathway and US vendor solutions which have been implemented in the US appear to be actively considered and selected by European health systems which are carrying out multi-site implementations. Contacting of stakeholders was via linkedin and email.

Those who completed the questionnaire were invited to take part in a follow-up interview con-ducted over skype or by phone. The interviews were recorded and transcribed. A recording of one interview failed to be made due to technical difficulties, for this interview notes made during and directly after the interview were used.

Results analysis

The aim was to present the results of the questionnaire and follow-up interviews together and, therefore, after first analysing each separately the results were presented in an integrated form.

The responses in the follow up interviews were analysed by thematic analysis as described by Braun and Clarke, 2006 [59]. Deductive analysis was followed using the questions from the interview guide to develop the codes into themes. The same analytic approach was used by A Clark et al., 2015 for analysing interviews given by NHS CIOs on EHR implementations [8]. One of the phases that Braun and Clarke, 2006 identify is the reviewing of the initial themes, this can result in some themes being eliminated as they are not prevalent enough in the data [59]. Because of our study’s topic and the roles held by the target participants, points raised in only a single interview were still included.

(23)

3.3 Results

One hundred and fifty people who held leadership roles during the implementation of an EHR across a multi-site health system in the US were invited to participate in the survey. Of these, 17 people completed the online survey and 11 took part in a follow-up interview. The respondents represented 14 health care systems. Two of the 14 were multi-hospital health systems with be-tween two to five hospitals, at least one of which they classified as a tertiary hospital, and eight were a multi-hospital health systems that had over five hospitals. All of the multi-hospital health care systems also provided ambulatory care (sites that deliver outpatient care which includes ur-gent care centers, primary and specialist offices/practices and clinics in hospitals). The remaining four participants came from four different paediatric health systems that are made-up of one paedi-atric hospital and multiple free standing outpatient sites. Two participant health care systems had yet to have any of their hospitals go-live (the results from these latter two only covered the point their health system was at in its implementation: training and build respectively). The year of last hospital go-live for the EHR system had a range between 2001 and 2019, but 13 had experience of a hospital implementation with go-lives that took place after 2015. Some participants who had implemented the EHR noted that some of their care sites were still to go-live.

Job category/area at the time of implementation Number Chief Clinical/Medical/Nursing Informatics Officer (including regional) 8

Nursing leadership 3

Physician leadership 2

CIO, VIP Informatics, IT Manager 3

Information Management Director 1

All the implementations included vendor modules for supporting inpatient and outpatient care. The health care systems since implementation could now be labelled as a single vendor solution for their core EHR. In the online questionnaire and in the follow-up interviews, four participants noted that additional specialised modules are planned or were added after the first implementation.

Comment made by a participant in the online questionnaire

“Standardization of documentation format and content is ongoing. Module additions continue incrementally ([VENDOR MODULE A]/ [VENDOR MODULE B ]) and pocket of thir[d] party systems remain as future conversion targets when [VENDOR] becomes best solution...”

It could be ascertained from replies to the questionnaire and follow-up interviews that the health care systems’ EHR deployments were mostly replacing disparate systems of: a mix of paper and information systems, different EHRs for outpatient versus inpatient, different single vendor solu-tions for different hospitals, some hospitals being best of breed, custom built EHRs, mostly paper based, and one health system was transitioning from a mostly single vendor solution to another.

The results of both the questionnaire and follow-up interviews will be presented together in the following order:

(24)

• Implementation approach: What approach was taken on spacing the implementation between the different hospitals as well as between the inpatient and outpatient care settings. The reason(s) for this approach.

• Benefits: Benefits expected and delivered from the EHR implementation.

• Challenges: Areas that presented as challenges during the different phases of the implemen-tation.

• Tackling and overcoming the challenges: Covering in more detail the challenges as well as how they were overcome.

• Standardisations: Participants’ view on the EHR build and level of standardisation achieved.

Implementa on approach

The approach to spacing of the EHR implementation between the health system’s inpatient and outpatient care settings will be described, then the spacing of go-live between hospitals.

It was apparent from the research (questionnaire, follow-up interview and desk-based research on the care systems) that a mix of approaches to spacing of the outpatient and inpatient EHR mod-ule go-lives took place. In the US setting, a health system’s outpatient care services (specialized outpatient care and primary care) can be referred to as ambulatory care. Centers for Medicare and Medicaid Service states that ambulatory care refers to any health service delivered which does not require an overnight stay [60]. Health systems’ ambulatory care can include primary and outpa-tient specialist care practices/offices or clinics located in hospitals or free-standing sites and also same-day surgery sites and urgent care sites. From the research, some of the health systems will distinguish between hospital outpatient departments and other outpatient care sites.

• Nine of the health systems split at least some of their outpatient care services and inpatient hospital go-lives. This was done to a different extent with some having the hospitals go-live at once, but separating their connected primary care and specialists’ offices.

• Five health care systems implemented the hospitals and outpatient care sites at the same time. One of these still split their health system into separate regions for go-live.

Included in the above 14 health systems were four paediatric health systems which adopted the following approaches:

• Two of the health care systems employed a staggered approach to go-live between hospital and outpatient care services, one with a gap of only a few months between.

• Two health care systems went live with the hospital and all outpatient care sites at the same time.

From the online questionnaire and follow-up interviews plus desk based research, the implementa-tion approach adopted by the non paediatric health care systems for their hospitals’ inpatient care is varied.

• Six of the health care systems employed a staggered approach to go-live between hospitals, usually in clusters.

(25)

The responses in the interviews provided more detail on the reasons for the approaches taken in the implementation by some of the health care systems and on whether they would have adopted the same approach again. For those of the interviewees whose health care systems selected to go-live system wide or with a very limited spacing between sites (including paediatric systems), the influencing factors were: the availability of resources to be able to support such an implementation (n = 1), the view that the vendor had the experience to support such an implementation approach (n = 1), the wish to move onto optimisation as quickly as possible (n = 1) and the wish to avoid unnecessary temporary integrations (n = 2). One of these interviewees noted that in health systems with a large geographical expanse it is possible to stagger the go-live as they are not dependant on each other.

Some of the health care systems which opted for staggering between hospitals by clusters noted that influencing factors were: the requirement for resources (n = 3), vendor support and third party consulting advice (n= 1), and the clusters of hospitals having similar resource requirements for their project (n = 1). One interviewee believed a reason for a pilot hospital being implemented first was to allow the system to be tested for any glitches. For the two health systems that had a pilot hospital, the hospital was selected for reasons which were unique to those health care systems e.g the pilot hospital was located further away from the other hospitals or the hospital’s contract with its old EHR vendor was the earliest to expire, but both of them noted that the hospitals were medium size compared to the others in their health system. After the pilot, both health care systems aimed to have similar bed numbers for each hospital cluster that went live at the same time.

Implementa on at individual sites

According to replies to the questionnaire, all but two of the participant health care systems went for a big bang implementation for inpatient hospital sites and one of these two varied from phased (split between core clinical modules, and admin and billing modules) to big bang.

According to two interviewees, the reason for a big bang implementation for the modules being introduced per hospital go-live was in order to avoid unnecessary temporary workflows and inte-grations.

Argument against phased implementation given by one of the interviewees.

“...if you’re not able to go live .. big bang … all the systems up within a short period of time and by that I mean … about eight to 12 weeks then that transition often becomes months or years.” PJV2531

One of the interviewees whose health system carried out a phased implementation at its hospital noted it was necessitated due to spacing between vendor modules being developed, but now feels that big bang is more appropriate as US hospital staff are ready for such levels of change to take place. They did state that for paper based practices which are now going live on their ambulatory EHR system, they would follow a phased implementation (billing and admin first, then clinical) because of change capacity at the sites. One interviewee stated there has been a blending of the distinction made between big bang and phased implementations with a short spacing still viewed as big bang.

(26)

Reﬂec ons of interviewees on implementa on approach

Two interviewees noted that they would carry out the same implementation approach again, but one of these said with slight modifications such as bringing up different care settings in a different order.

Motivation given for implementing the same EHR across multiple care sites included the wish to create a unified record across the health system (n = 4) and with partner health care systems (n = 1). While this was the primary motivator other factors acted as instigators: the old vendor solutions were underperforming (n = 2), the EHR used in some of the care levels or sites was soon no longer to have vendor support (n = 2).

“And so we wanted to consolidate all of the … hospitals onto one record and that was … in my opinion the critical decision. And so the decision was made to proceed with [VENDOR] across the [NUMBER] hospitals ..” HEF6248

Of those interviewed who noted a variation in difficulty for the implementation process at different hospitals, reasons given included: different clinical staff levels of experience with EHRs prior to implementation (n = 1), a relatively new health system’s culture and governance not fully estab-lished across the hospital sites (n = 2), and staff from some hospitals demanding what they view as their requirement for unique workflows which prevented full standardisation (n = 2). One inter-viewee noted how the EHR implementation process meant there needed to be a limiting of other projects. Factors that aided in the implementation were: hospitals using the same EHR, even if dif-ferent instances before implementation (n = 1), first go-live of hospitals allowing for lessons to be learnt for later go-lives (n = 3), and the staff at subsequent go-live sites able to experience the live EHR and benefit from sharing super users (n = 1). Lessons learned at the first go-live can allow for a significant reduction in go-live minor issues for later sites as the unexpected issues which arose at the earlier go-live can be dealt with (n = 1).

Beneﬁts

All the participants in the questionnaire noted that the benefits which had been hoped for in the busi-ness case had started to appear or improve, at least in part. Table 1 below shows participants’ views on what were the most important goals the organisations identified for the EHR implementation.

(27)

Benefits Most important 2nd _most

important

3rd_most

important

Improving patient safety and

care 11 1 2

Establishing standardised

work-flows across sites 2 8 3

Delivering an improved patient

experience 1 1 5

Improving the efficiency of care

delivery 6 2

Implemented in order to com-ply with external policy require-ments and guidelines

1 1

Improvement in IT operations

(allowing for cloud hosting etc) 1

Integrating with existing

exter-nal information systems 1

Remove data silos 2 1

Improving data security and

pri-vacy of health information 1

Other 1“Improved

revenue capture”

TABLE 3.1 Benefits highlighted in blue were selected most frequently by participants

(28)

The following comments were added by participants in the questionnaire in relation to the benefits and whether metrics for these benefits had improved:

• “Marked improvement in Financials”

• “Primary driver was improving patient safety and quality” • “Achieved information sharing across the organization”

In the interviews, the following was noted in respect of benefits:

• The benefit alone of care providers having access to the full record held at their health care system was important (n = 5)

• Data silos were not just removed from across the health system and different care levels, but from within each hospital that was coming from a best of breed approach (n = 1)

• The elimination of unwarranted clinical variation in care, now instead following standardised care around best practices (n = 2)

• For improvement in patient safety, there is need to wait until after the EHR has reached a steady state, introduction of the new system can initially increase safety issues as staff adjust to new system and workflows (n = 1)

• The EHR system allowed for improved efficiency in the care processes (n = 2) • Improvement in revenue metrics is and was an important goal (n = 2)

• The new EHR system allowed for drilling down on clinical staff’s interaction with the system so that those requiring extra support could be identified (n = 1)

• A benefit was the ability to roll-out clinical decision support across the health system (n = 2) • Integration of physiological monitoring systems with the EHR so there is no longer a

require-ment for manual data entry (n = 1)

Two interviewees noted that it takes time for the full benefits to appear and that there is the need for improved metrics measurement and analytics to be able to measure the benefits and be able to present them. Also, any improvement in the metrics needed to be considered over time to see the cause and effect (n = 1). One interviewee stated how benefits can already be achieved pre-go live due to analysis and improvement of workflows that take place during the implementation process.

Network eﬀect

Two of the interviewees noted as a benefit of the implementation the fact that the vendors’ EHR was also implemented in the surrounding health care systems, which meant that the exchange of patient information was far easier.

Challenges during the implementa on

The table below sets out what the respondents to the questionnaire viewed as the most important challenges prior to commencing training and first go-live.

(29)

The biggest challenge 2nd_biggest challenge 3rd_biggest challenge Governance 9 1 Communication 4 2 3

Vendor selection and

procure-ment manageprocure-ment 2

Engaging clinical staff from

across the system 1 7 1

Building the system 1 2 1

Data migration 2 1

Accountability/responsibility

between levels 2 1

Agreeing on a design 1 2

Resource management 6

Other: “change management” 2

TABLE 3.2 Areas that were considered challenges by (n) participants during the project prior to commencing training and first go-live. In blue most frequently selected. Some challenges were not selected by any participant, see appendix A.7 Q18 for the full list.

(30)

The following comments on the challenges were added by respondents into the online question-naire:

• “We had operated as largely a federated group of hospitals. The system implementation forced a lot of standardization for which there had not been adequate prep work.”

• “Vendor Choice was enhanced by prior experiences and desire for single system (Acute and OP).” [NOTE: In relation to the challenge of vendor selection]

• “Critical that Clinicians engaged and the decision makers on build/workflows vs the IT team” • “Difficult to assess - all were important”

(31)

Respondents to the questionnaire were also asked what were the greatest challenges for training and go-live phases of their projects.

The biggest challenge 2ndbiggest challenge 3rdbiggest challenge

Managing the change of work

practices 7 4 1

Training 5 2 4

Dealing with requests for change and issues following go-live

4 6

Staff resourcing for the central

programme 1 1

Establishing level of account-ability between sites and the central programme

1 1

Use of workarounds 1 1

Balancing of support of live and

non live sites 1 1

Clinical staff finding the system

difficult to use and interact with 1 1

Reduced throughput/clinical

ca-pacity during go-live 1 1

Ensuring super users had the

right skill levels 1

Communication 1

Other: “Establishing a

pro-gramme for ongoing support” 1

TABLE 3.3 Areas that were considered challenges by (n) participants during training and go-live. Most fre-quently selected in blue. Some challenges were not selected by any participant, see appendix A.7 Q20 for the full list. Note one participant did not answer this question.

(32)

The following comments on the challenges were added by respondents into the online question-naire:

• “We did most of the training in the 2-3 months before go-live - so some staff forgot what they learned before go-live.”

• “Post implementation, users consistently request at the elbow support for personalization and efficiency tips. Defining a programme that could accommodate multiple sites has been a challenge.”

Overcoming and tackling the challenges

In the interviews more detail was provided on the challenges and how they were overcome.

Challenges in relation to governance mentioned in the interviews were:

• Dealing with establishment of a shared culture in a recently formed health system (n = 2) • And following on from this, a challenge for another system was that it was the first large

health system wide project, so a governance structure needed to be established (n = 1). • Managing the resistance around movement to common workflows between sites (n = 3), e.g.

bigger hospitals versus smaller hospitals and departments.

• The need to manage different demands which were being placed on the programme from a set of different stakeholders (n = 1)

• The need to ensure that focus is kept on the programme and that the message is correctly communicated (n = 2)

Factors that helped or would have helped with dealing with these challenges included:

• Strong communication and engagement with leadership (n = 3) • Clinical and department leadership support and buy-in (n = 2) • Constant communication to staff about changes (n = 2) • C-suite buy-in and leadership (n = 2)

• Robust governance structure; having in place a governance matrix with a clear pathway for decision making and escalation of issues (n = 5)

• Previous efforts at standardising clinical workflows across hospitals helped with the move-ment to common workflows (n = 1). Leading on from this, in another health system a pro-gramme that worked on standardising workflows (order sets) pre-implementation aided the design and build of the system (n = 1)

Quotes from interviews which are examples of acknowledging challenges which can present around governance.

“Having the right governance and the right stakeholders. I think we knew that. But even I was impressed the extent that I thought our governance structure was pretty good but we still had challenges.” YXJ3939

“...So first we need to get our informatics and the leadership persuaded that change is necessary. Then we had to go out to each of the hospitals leadership groups and convince them that changes

(33)

One interviewee noted that hospitals that had newly joined the health system and which imple-mented the health care systems’ EHR in a short space of time had challenges around communica-tion and governance. These challenges were overcome by intensive interaccommunica-tion with the hospitals’ leadership and by having in place a structured protocol for implementing the EHR. This structured protocol was based on the implementation at the hospitals that originally made up the health system.

Mul -site governance and culture

Three interviewees identified the absence of an established overarching enterprise governance structure and corporate culture for the health system around standardised workflows and EHR sys-tems as leading to difficulty and some variation in the implementation between hospital sites (2 of these interviewees). This was found in more recently formed health care systems at the time of implementation (n = 2) while two interviewees stated having this structure and culture in place helped the implementation process.

The fact that leadership from across the health system supported the EHR implementation process was stated as important with them acting as change champions (n = 3). This support was maintained through strong communication and engagement (n = 1). One interviewee noted how having hospital leadership support helped overcome some resistance to the implementation at one hospital. Six of the interviewees noted the need for a governance structure and a clear decision-making pathway for the design phase, this included a health system which was now focused on implementing its ambulatory EHR.

The governance structures and groups stayed in place after the EHR system had been implemented and managed the optimisation of the EHR system, however one interviewee noted how the make-up of the committees may change. Optimisation is an ongoing process with the improvement of the EHR continuing following go-live (n = 4).

Data migra on, Resources and Training

Three interviewees had noted data migration as presenting a challenge in the survey in relation to moving data from the existing information systems to the new EHR. One interviewee noted the process of data migration was more time consuming when manual data entry was required for moving information from the outpatient EHR to the new EHR. For another health system, the challenge was migrating the data of newly joined hospitals to the health system’s EHR. A key factor that helped was the decision to integrate straight away the old lab systems into the health care system’s EHR, this aided providers when documentation was added later to be able to view lab results which were months old. There was also a merging and testing of the master patient index to ensure existing patients from the old system were identified. With some difficulty the radiology PAC system was integrated so that the results were viewable in the health system’s PACs. For another interviewee issues with cost, time and ensuring validation of the data migration step led to the decision to transfer only basic demographic data with a link instead to the old EHR used for accessing the remaining data.

Resource management constraints mentioned by interviewees were: recruiting of staff who have experience required for an implementation (n = 1), finding facilities that would allow for the train-ing of all staff who would need it (n = 1), the requirement to plan for staff rotattrain-ing onto traintrain-ing (one interviewee with a nursing leadership background), old ICT infrastructure which needed to

Lessons from the literature and stakeholders