Stakeholder issues that emerge during the implementation
process of Information Systems in hospitals: a systematic
literature review
University of Groningen, Faculty of Economics and Business MSc. Business Administration, Change Management
23-06-2014
Vasiliki Gkiaouri
v.gkiaouri@student.rug.nl
Student number: 2525011
First Supervisor: dr. J.F.J. Vos Second Supervisor: prof. dr. A. Boonstra
Words: 10.898
Acknowledgment:
1 | P a g e
Stakeholder issues that emerge during the implementation process of
Information Systems in hospitals: a systematic literature review
Abstract
Background: The main objective of this research is to identify, categorize and analyze the stakeholder issues that emerge during the implementation process of Information Systems in hospitals.
Methods: A systematic literature review based on 18 research papers, concerning issues of three basic groups of frontline professionals inside the hospital- physicians, nurses,
administrators- that play a significant role during the implementation process of Health Information Systems in hospitals was conducted. Four databases, “Web of Science”,
“EBSCO”, “PubMed”, and the “Cochrane Library” were used in the literature search. Studies were included in the analysis if they met the following requirements: 1) written in English, 2) based on primary empirical data, 3) full text should be available online, 4) focused on
hospital-wide implementation of multiple Information Systems, 5) based upon actual health IS implementation experiences, 6) meeting established quality criteria.
Results: Using an inductive approach based on a review of 18 articles we identified 5 main categories of stakeholder issues, including a total of 16 subcategories. The 5 main categories are: 1) workflow issues, 2) technical issues, 3) financial issues, 4) ethical issues, 5) change
process issues.
2 | P a g e
Background
Health information systems (HIS) are defined as those systems of processing data,
information and knowledge in health care environments (Haux et al., 2004). At the beginning, HIS were oriented to collect information on diseases and on health service outputs. In the meantime, there has been a tremendous progress in medicine as well as in informatics. In contemporary times, HIS were transcended to the domain of modern health practices, they hold great significance in the planning and decision-making of health delivery services and thus they are implemented by numerous hospitals around the world (Reinhold, 2006). Although HISs offer many opportunities to improve health care, basically through cost reduction and quality, safety, and efficiency enhancement, challenges such as the complexity of the systems implementation, the magnitude of investment and possible changes in users’
workflow are often accompanied with the adoption of those systems (Goldschmidt, 2005). Medical informatics scholars have sought to understand how information systems are diffused
in organizations and the barriers to such processes. Although there has been many research focusing on the implementation process of information systems and its critical issues (Bingi et al., 1999; Hong & Kim, 2002; Umble et al., 2003), much of the research has focused on issues with very limited or no regard to stakeholder perspective (Finney & Corbett, 2007).
Successful Information Systems in different areas can only be developed in conjunction with a range of ‘interested parties’, and the benefits of doing so become obvious when the systems move away from being experiments with technology and attempt to become integrated in an organizational setting (Whitley, 1991).The interests and perspectives of these ‘interested parties’ to which an organization has a responsibility should be taken into account. Burchell & Cook (2006) state that issues that stakeholders consider as important often do not surface, or are not taken into account during formal decision-making of the change project, because implementers may not see issues stakeholders perceive as important. Many scholars argue that implementers should take the time to identify, understand, involve, and manage project stakeholders (Schwalbe, 2009; Parmar et al., 2010; Laplume et al., 2008).
Bates (2005), indicates that many of the main barriers to the use of HIS (and especially of Electronic Health Records) are not only technical, but other issues including stakeholder issues such as risk tolerance, fears about privacy, physicians’ concerns related to time burden, the number of vendors in the marketplace and the transience of vendors. Issues should be analyzed in order to gain a thorough understanding of how stakeholders react and why they react in such ways to the implementation of a health information system. However, scholars often only focused on a specific aspect of the implementation process or a specific issue instead of the multiple classifications of the issues (Tarafdar & Roy, 2003; Robey et al., 2002; Grant, 2003).This is the challenge for this study and leads to the following research aim. In the following sections a systematic literature review will be carried out, aiming to contribute to our understanding of the stakeholder issues that may arise during the
3 | P a g e the nurses and the administrators. Also, regarding the broad range of HIS implementations, we mainly focus on Electronic Health Records in the organizational level and other similar types of those such as Electronic medical records, Electronic patient records, Personal health records and Computerized patient records.
Although there is already a body of literature on stakeholder issues, to our knowledge there has been no systematic overview of these studies. By filling this gap we hope this overview provide future HIS implementers with a better understanding on these issues and a greater insight on how to prevent those that is likely to pose a threat to the ultimate success of each IT project implementation.
Terminology
In order to explain the rationale behind of this study it is essential to understand what an issue actually entails. According to Oliver and Donelly (2007), an issue can be considered an unsettled matter that has the potential to change the status quo in an organization’s business environment. An issue generally involves some degree of conflict with external parties in which emotions rather than data or fact dominate, resulting in stressful, anxious, and vague decision-making situations for individuals (Jaques, 2007; Oliver and Donnelly, 2007). Issues emerge from a variety of sources: corporate publications, newspapers and magazines,
managers from other functional areas, and simply rumors and concerns of the internal IS staff. Issues may represent threats, opportunities or both. Thus, a possible issue management
perspective is to minimize the damage or to maximize the opportunity (Benjamin et al., 1987). Issue management can be defined as the capacity to understand, mobilize, coordinate, and direct all the executive functions of problem solving, public relations, and long-range strategic and policy planning as well as communication of the determined strategies/policies internally and externally (Bowen, 2005; Heath, 1997). This interdisciplinary by nature subject called “issue management” incorporates concepts from various management-related domains, such as public relations and policy, information management, business ethics and communications, and it is closely related to crisis management, since an issue can develop into a crisis if it is not properly managed (Conway et al., 2007; Heath, 1997; Jaques, 2007).
One of the dominant perspectives currently seen in the literature regarding issue management is the stakeholder theory. The basic idea of this theory is to identify and monitor the
individuals or groups who affect or are affected by an issue, perform stakeholder prioritization, and then make strategic choices on related functional domains, including corporate
governance, communications, and policy making for the purpose of reducing the tensions between organizations and their stakeholders in order to resolve the focal issues and establish long-term mutual trust (Oliver and Donnelly, 2007; Wartick and Heugens, 2003).
Researchers are now paying more attention to the diversity of stakeholder issues—that is, the explicit concerns and requests raised by individuals or groups that can affect or be affected by the firm (Freeman, 1984)—and the process by which managers interpret, balance, and
4 | P a g e affected by the achievement of the organization’s objectives’ (Freeman, 1984, p.46). In the literature, there are three main groups of IS stakeholders: users, management and IS
professionals. In case of an IS implementation in the health care industry a wide range of stakeholders can be identified based on three basic categories: 1) Immediate stakeholders inside the hospital (Hospital management, IS management, Doctors, Nurses), 2) Immediate stakeholders outside the hospital (GPs, Pharmacies), 3) Other stakeholders (Patients, Insurers, Government agencies) (Boonstra et al., 2008). Issues are considered to be something that stakeholders may be able to influence through their actions and interpret it in different ways which essentially exist independently of their involvement.
Electronic health records, Electronic medical records, Electronic patient records, Personal health records and Computerized patient records are some of the most often viewed terms in the available literature used as interchangeable synonyms. This study has included most of them in the search process in order to provide a broad insight for all the types of health information systems. According to Hodge (2011), EHRs differ from electronic medical records (EMRs) and personal health records (PHRs) most notably on the basis of the completeness of the information the record contains and the designated custodian of the information. EMRs can be defined as partial health records under the custodianship of a health care provider(s) that hold a portion of the relevant health information about a person over their lifetime, EHRs can be defined as complete health records under the custodianship of a health care provider(s) that holds all relevant health information about a person over their lifetime, whereas PHRs can be defined as complete or partial health record under the
custodianship of a person(s) (e.g. a patient or family member) that holds all or a portion of the relevant health information about that person over their lifetime (McGinn et al., 2012). In addition, Electronic patient records (EPRs) and Computerized patient records (CPRs) are other frequently used by authors terms, to indicate all kinds of electronic documentation, independent of the degree of structuring and the amount of information (Stausberg et al., 2003).
Methods
Search strategies
5 | P a g e concern and hospital we ensured that all variations of the words and both the plural and singular forms would be retrieved. Moreover by adding “ ” around the words we ensured that the complete terms were searched for.
Because of the relatively large set of keywords needed to ensure articles were not missed in the search, we finally came up with 16 different search strategies as shown in the table below.
Table 1 Overview of the search strategies
Search
strategy
Terms used
1. Stakeholder* + “health information system*” + concern* + hospital* 2. Physician* + “health information system*” + concern* + hospital* 3. Nurse* + “health information system*” + concern* + hospital*
4. Administrator* + “health information system*” + concern* + hospital* 5. Stakeholder* + “electronic health record*” + concern* + hospital* 6. Physician* + “electronic health record*” + concern* + hospital* 7. Nurse* + “electronic health record*” + concern* + hospital*
8. Administrator* + “electronic health record*” + concern* + hospital* 9. Stakeholder* + “electronic medical record*” + concern* + hospital* 10. Physician* + “electronic medical record*” + concern* + hospital* 11. Nurse* + “electronic medical record*” + concern* + hospital*
12. Administrator* + “electronic medical record*” + concern* + hospital* 13. Stakeholder* + “electronic patient record*” + concern* + hospital* 14. Physician* + “electronic patient record*” + concern* + hospital* 15. Nurse* + “electronic patient record*” + concern* + hospital*
16. Administrator* + “electronic patient record*” + concern* + hospital*
In order for this systematic literature review to provide comprehensive and valid results, a systematic search was conducted on four basic relevant databases. These were: “Web of
6 | P a g e
Data analysis
In order to assess the quality of the articles that survived the filtering process, we used the Standard Quality Assessment Criteria for Evaluating Primary Research Papers (Kmet et al., 2004). In particular, we evaluated to what extent specific criteria had been addressed by each paper, resulting in a rating of 2 (fully addressed), 1 (partly addressed), or 0 (not addressed). 10 questions were scored with the above strategy for the qualitative studies, 14 questions for quantitative studies, while for the mixed-method studies both questionnaires were used. In case that questions were not applicable to a particular study design were marked “N/A” and were excluded from the calculation of the summary score. After this process, only the papers that received at least half of the possible summary score were finally approved for further
analysis. The next phase of our research consisted of extracting and analyzing the data that the
approved articles included. Because of the lack of a clear overview of the identified
stakeholder issues during the HIS implementations in hospitals in the existing literature, we did not start our analysis by applying a pre-established set of stakeholder issue categories. Instead of that, we followed an inductive method of analysis that gave us the opportunity to code our own categories based on the information emerged from the selected articles. Our analysis followed several basic steps in order to provide us with systematic and reliable coding results. Firstly, all the articles were read several times in order to give us a general view of the HIS implementation process in different types of systems and the perspectives of the three examined stakeholder groups. Afterwards, each article was screened carefully and the text segments that contained meaning units for analysis were underlined. Next to that we kept a notebook that was filled gradually with labels of the segments of information taken from each article. Quotations from the raw texts of the articles were also included in order to justify the emerging labels and make it easier for a further verification process of the final results. Some of the reviewed articles provided results classified in specific named categories (e.g. Ash et al., 2004; Lyons et al., 2005; Mannan et al., 2006).That just gave us a first insight on how to develop our own perspective on how to categorize our findings. However we were mainly searching for ideas and words that were repeated in most of the studies in order to decide upon the most important categories of issues. In that way, after gradually adding all the relevant information, merging and excluding some of them due to overlapping, we came up with our final categories and subcategories of issues.
Each of the studies has also been characterized in terms of the country or region of data collection, the research aim, the type of research (qualitative/quantitative/mixed), the data collection method, the number of cases/sample size and the type of hospital (Table 3).
7 | P a g e
Results
Included studies
By applying the 18 different search strategies shown in table 1 with the various search engines, 600 articles were identified (Figure 1). After carrying out an initial screening, 260 articles excluded through manual check because they were duplicates, both within and between the search engines. Of the remaining 340 articles, based on the title and abstract, 303 were excluded because they did not meet all the inclusion criteria described above. Of the 37 articles remaining, 20 were commentaries or literature reviews, therefore lacking fresh empirical data and thus excluded. This screening process resulted in just 17 articles for possible inclusion to the study. These 17 articles were checked for completeness through a systematic search of reference lists, which identified three more relevant articles. In that way we concluded to 20 empirical studies for a final quality assessment. The results of the quality assessment can be found in Appendix A. The results suggested the exclusion of two articles that failed to meet the quality threshold. Thus our final sample amounted to 18 papers that met all the inclusion criteria and sufficiently passed the quality assessment. Of these 18 finally included papers, 13 were qualitative, 2 were quantitative and 3 were
mixed-method studies. As indicated in table 2, the majority of the studies were published after 2010 (11 out of 18 papers). This undoubtedly reflects the recent increased interest of research community in Information Systems Implementation projects in health care industry and the subsequent emerging stakeholder issues. Most of the studies were conducted in USA, while others were conducted in multiple cultural contexts such as in China, Australia, Canada, Israel and some European countries. This let us to include results from diverse fields in diverse contexts and countries and thus emphasizing the ubiquity of the issues that emerge to various HIS implementations. Regarding the types of Information Technology examined per study we counted 5 EHR (Electronic Health Records) implementations, 4 EMR (Electronic Medical Records), 3 CPD (Computerized Patient Documentation), 1 Point-of-Care documentation system and 5 generally clinical IS implementations. This gave us again the ability to find common stakeholder issues that emerge in cases of HIS implementation process regardless of the specific features of each type.
Table 2 Number of papers published by period
Year Paper no.
2000-2009 1,17,34,35,36,50,57 (n=7)
8 | P a g e Figure 1 Flowchart of study selection process
Taxonomy of issues
It is worth mentioning that we ended up with the following results, not only by focusing especially on problematic HIS implementations but also on successful ones since according to Freeman (1984), issues represent the explicit concerns and requests raised by individuals or groups that can affect or be affected by the firm regardless of the ultimate success or failure of a specific project implementation or meaningful organizational change. The results identified after the inductive data analysis of 18 papers have been classified into five categories of
Web of Science: 73 potentially relevant articles EBSCO: 416 potentially relevant articles PubMed: 90 potentially relevant articles
The Cochrane Library: 19 potentially relevant articles
600 articles retrieved for initial screening
260 excluded (duplicates)
340 articles for further screening
303 excluded, based on title and abstract
37 relevant articles for further assessment
20 excluded, based on study design
17 empirical articles
20 empirical studies
3 included, based on search of reference lists
2 excluded, based on quality assessment
9 | P a g e issues: workflow issues, technical issues, financial issues, ethical issues and change process
issues.
An overview of the taxonomy of issues and the corresponding articles can be found in Table 4.
Table 4 Taxonomy of issues
Workflow issues refer to some changes in the daily working routines of the users of an electronic health system that may raise concerns regarding the users’ own interaction with the system and the subsequent influence on the execution of their clinical tasks. Workflow is a matter that frequently seen in the reviewed articles in multiple ways. Lyons et al. (2005) refer to changes that resulted in poor clinical workflow, Embi et al. (2013) argue about workflow alteration and disruption, while Li et al. (2013) discuss about HIS implementations and
possible misalignments with the workflow. Technical issues refer to specific software and hardware features of the HIS that may raise
concerns to users because of their lack of user-friendliness and efficiency on their daily tasks. Many studies refer to technical dysfunctions, challenges and concerns for multiple reasons, as analyzed in the following parts. Li et al. (2013) argue about users’ complaints due to technical errors, user-unfriendly interfaces and unmet requirements, while Ash et al. (2004) attribute
these commonly seen issues to the high-risk field of medicine. Financial issues refer to the costs of HIS implementations and the ability of hospitals to
overcome them through the acquirement of adequate financial resources. Chao et al. (2013) discuss about high implementation and maintenance costs while Lyons et al. (2005) refers to insufficient budgets to purchase the technological equipment as a commonly cited concern. Administrators and managers were mostly seen in the reviewed articles to express concerns about this category of issues.
Ethical issues category was created to include the very common users’ concern of
confidentiality of patients’ information, seen in most of the reviewed articles. About 8 out of the 18 reviewed articles included the critical issue of keeping patients’ information
confidential, which entails an ethical duty for medical community. Concerns about access of unauthorized personnel and possible hackers to the private patients’ data, are commonly
expressed by nurses and physicians and represent a risk of electronic health care systems. Change process issues refer to concerns may arise during the adoption of a new change
10 | P a g e Category A: Workflow issues
A1. Computer tasks
The computer tasks domain outlines activities that participants perform with computers. According to Lyons et al., (2005), computer tasks is the domain most frequently discussed by administrators, physicians and nurses in the case of a HIS implementation in hospitals. These tasks are data retrieval, documentation and order entry. All stakeholder groups are concerned about the type, amount and accuracy of the medical information need to be documented in the systems (Ash et al., 2004). Weir et al. (2007) identify as the physicians’ main concern the information overload that makes it difficult for them to deal with the process of extracting useful data and making proper decisions. Next to that, Embi et al. (2004) refer to the critical issues of accessibility, comprehensibility and legibility of information during the coding and documentation process that represent physicians main concern in case of HIS implementation. Many studies have shown that these aspects of physicians’ workflow are strongly related with the time burden and their difficulty to support their decisions (Embi et al., 2013; Weir et al., 2007; Boyer et al., 2010; Embi et al., 2004).
A2. Performance evaluation
Performance evaluation programs to monitor patient outcomes and assess physicians and nurses adherence to the health electronic systems is considered a useful tool for improving organizational and clinical outcomes for administrators. However physicians and nurses voice concerns that these electronic surveillance features would allow unfavorable and unfair judgments about their clinical practices (Ash et al., 2004; Lyons et al., 2005). In this context of performance evaluation, Ash at al. (2004) refer to the decision support overload that is usually integrated to electronic systems in hospitals and to the overdose of reminders, alerts or warning messages that entails. In that way physicians and nurses may feel supervised,
distrusted or resentful of being constantly interrupted. A3. Lack of computer literacy
All stakeholder groups consider low computer literacy or the degree of comfort and capability with information technology as an important issue that pose difficulties to the proper and successful use of HIS (Lyons et al., 2005; Likourezos et al., 2004; Piscotty et al., 2011). Chao et al. (2013) also argue that the lack of knowledge and skills of users impedes the transition from paper-based to electronic practices. It is commonly deemed that users need to receive considerable education on informatics and on the specific requirements of a computer based working system before being called to integrate it in their daily working routines.
A4. Interaction with patients
11 | P a g e of the patient-professional relationship’ as a matter of crucial importance when the presence of computers mediates patient and clinicians relations. Less eye contact and face-to-face communication may pose a threat to these relations by making patients feeling underestimated and suspicious about physician’s actions. It is worth mentioning that according to Noblin et al. (2013), the communication between physicians and patients in this case represents a negative/positive duality. That means that on the one hand, HIS use impedes the
communication while the patient is in the office whereas, on the other hand HIS use improves communication after the visit. This happens because physicians while being with patients in their office decrease eye-contact and direct interaction with them in order to be entirely focused on the information entry, while after the visit there is a beneficial interaction due to the improved organization and easy accessibility to information. Goldman et al. (2012) argue that the matter of interaction with patients represents also one of the main nurses’ concerns, while Kohle-Ersher et al. (2012) add that even in cases of electronic point-of care
documentation, nurses look the location of the computers in the patients rooms as a problem because of the necessity to turn their backs to patients to chart.
A5. Loss of communication
In the work practice of the health care industry which is characterized by contingencies, proper communication is an important matter. However, physicians and nurses seem to be concerned about the reduced direct interaction and communication that the HIS
implementation may create between them (Ash et al., 2004; Embi et al., 2013; Goldman et al., 2012; Embi et al., 2004). Embi et al. (2013) argue that the problem of less face-to-face communication between the members of clinical staff is addressed more in electronic health systems implementations, since staff can access records from anywhere while in a paper based system the central location of paper charts mitigate this threat. According to Ash et al. (2004) and Weir et al. (2011), physicians may assume that ‘entry’ into the computer system replaces their previous ways of communicating their plans and that their orders would be carried out without further action from their part. This is strongly related to the problem of loss of feedback that usually arises during HIS implementations between physicians and nurses. The use of computerized systems usually make physicians to mistakenly assume that after sending information, the computer will take care of notifying other users, such as the nurses. In this case however, this is not always possible, especially due to the mobile nature of nurses’ work that makes it for them difficult to be constantly physically appeared nearby a printer device in order to receive timely and correctly physicians’ orders (Ash et al., 2004). In other words, it is a matter of transferring of information in the right place and at the right time which cannot always be guaranteed by HIS and may pose threats to the effective completion of clinicians’ tasks. Also, this problem of communication is strongly related to the time burden that physicians may experience because usually they need more time to review a patient’s history on the screen than eliciting key information from an experienced clinic nurse (Goldman et al., 2012).
A6. Health-related concerns
12 | P a g e Category B: Technical issues
B1. System design
Health information systems are hi-tech systems that include complex hardware and software, programming requirements. Software and hardware have to be designed in such a way to optimally fit the nature of work practices in health care settings. However, physicians and nurses usually talk about impractical or outdated system interfaces that fail to meet their needs of retrieving and handling necessary information, adding costly time to their daily routines (Ash et al., 2004; Li et al., 2013; Goldman et al., 2012; Kohle-Ersher et al., 2012; Shield et al., 2010; Chao et al., 2013). Physicians and nurses usually complain about slow systems and their shut·down in the middle of charting, user-unfriendly interfaces, technical errors and bugs. It is worth mentioning that nurses are mostly affected and subsequently concerned about specific system design features in the workplace that are not harmonized with their mobile and team-based work nature (Lyons et al., 2005; Embi et al., 2013; McGinn et al., 2012). While physicians mostly attribute their additional workload to software features directly related to the process of documentation and accessibility to important information for supporting their decisions, nurses attribute their workload to environmental design features that render their workflow inefficient. Embi et al. (2013) argue specifically that when nurses care patients, they need to report recording information as many as three times: at the bedside, in pocket notes and finally at a computer terminal. Since computer workstations are located far from patients’ wards and rooms, nurses are not able to complete their extensive
documentation in a timely manner. Consequently nurses usually complain about poor clinical workflow and undesirable delays in patient care. However, Kohle-Esher et al. (2012) argue that even in cases of electronic point of documentation systems, the stationary nature of computers may pose difficulties to nurses’ workflow because of ineffective communication with the patients. For this reason mobile devices such as tablets and netbooks could provide nurses with a better option for their clinical practices.
B2. Standardization
13 | P a g e B3. Fragmentation
Physicians and nurses by using HIS usually need to switch between different screens in order to complete a documentation task (Ash et al., 2004). This results in fragmentation of the cognitive images that health professionals construct and subsequently their workflow becomes more complicated and time consuming. According to Chao et al. (2013), one of the challenges that physicians face during the implementation of a HIS is the existence of numerous interfaces or settings that affect negatively their work efficiency.
B4. Lack of reliability
Many studies have shown that specific features integrated in electronic health information systems might pose concerns to users regarding the reliability of the document contents (Chao et al., 2013; Embi et al., 2013; Weir et al., 2007; Embi et al., 2004; Weir et al., 2011; Mannan et al., 2006; Li et al., 2013). The users’ ability of copying and pasting clinical assessments and treatment plans is one of those features most frequently referred by physicians, nurses and administrators as possible to generate less trustworthy medical information. As Embi et al. (2004) argue, physicians occasionally tend to copy history and examination information from previously created documents without thinking critically about certain problems and adjust the information when is needed. Next to copy-paste feature many studies refer to the automation of the texts generated by the systems and render physicians skeptical about their veracity. Moreover, according to Chao et al. (2013) and Mannan et al. (2006), physicians are also concerned about the possible data loss due to unknown software errors and are worried about IT department’s ability to provide them with back up and data security solutions.
Category C: Financial issues C1. Lack of financial recourses
An issue mostly directed by administrators is that of finding sufficient financial resources/ funds to support the adoption of HIS in hospitals. Insufficient budget to purchase computers and meet the needs for training purposes is a commonly cited concern of administrators (Lyons et al., 2005; Mannan at al., 2006).
C2. Costs and return on investment
14 | P a g e Category D: Ethical issues
D1.Lack of confidentiality
One of the most frequently seen concerns regarding HIS in hospitals is the ability of the users to maintain the confidentiality of patients’ private, clinical information (Likourezos et al., 2004; Mannan et al., 2006; McGinn et al., 2012; Lyons et al., 2005; Shield et al., 2010; Boyer et al., 2010; Kohle-Ersher et al., 2012; Chao et al., 2013). As Likourezos et al. (2004) argue about this issue, users of HIS look doubtful regarding confidentiality of patients’ information due to possible illegal information leakage. Hackers or other unauthorized people and non-medical workers could have access to this information and threat patients’ right to privacy. Lyons et al. (2005) also refer to the intrusive nature of computer technology that makes nurses and other clinicians to express their misgivings about using computers to deliver patient care.
Category E: Change process issues E1. Lack of involvement
Several articles refer to the need of physicians and nurses to participate in the development of the health information systems and the general implementation strategy (Mannan et al., 2006; McGinn et al., 2012; Goldman et al., 2012; Piscotty et al., 2011). As Mannan et al. (2006) argue, physicians usually express their desire to be consulted by managers before the implementation of a new project since they are the ones who will help to roll it out and who will use the system. Boyer et al. (2010) also refer implicitly to this issue by mentioning the importance of a participative and flexible culture in HIS cases and particularly to the creation of multidisciplinary work groups.
E2. Lack of training
Physicians and nurses have mentioned in several studies the need for better and additional training before the implementation phase of a HIS (Mannan et al, 2006; McGinn et al., 2012; Noblin et al., 2013; Boyer et al., 2010; Chao et al., 2013). In many cases, the transition from a paper-based to an electronic system requires appropriate assistance, sufficient training and specific guidelines and especially for physicians who might be not computer literate (Chao et al., 2013).
E3. Lack of motivation
Several studies include lack of motivation as an important issue in case of a HIS
15 | P a g e
Discussion
The main findings of our systematic review suggest that 5 main categories of issues, following by 16 subcategories are the most important ones that emerge during the
implementation process of HISs. As it is depicted in table 4, the mostly cited issue categories in the reviewed articles are those of workflow (category A) and technical (category B) following by change process (category E), ethical (category D) and financial (category C). However, from the results reviewed, we also saw that our issue categories and some of our
subcategories could be interpreted as highly interrelated. We can argue that, change process issues (category E) are strongly related to the workflow
issues (category A) and the technical issues (category B). In particular, the lack of sufficient training provision and motivation from managers to the users of a HIS may lead to inability of users to integrate the electronic systems in their daily workflow. Computer illiteracy of users for example in this case may fail to be addressed and the computer tasks of documentation, order entry and data retrieval may be executed by users with insufficient expertise and commitment. Also, lack of users’ involvement to the development of HIS project may lead to the issue of impaired communication between nurses and physicians. On the other hand the change process issues of inadequate training, lack of involvement and motivation may also lead to technical issues such as problematic system designs and subsequently lack of
reliability to the special features of those systems. Next to that, we can argue about a possible causal sequence between technical issues (category B) and workflow issues (category A). When the complexity of the systems design is high, its features are considered no user-friendly and there is lack of reliability to the systems, users’ workflow is affected negatively. That means that users may encounter problems with the execution of the computer tasks and the effective interaction with colleagues and patients. Technical issues of standardization and fragmentation also may lead to workflow concerns regarding effective communication and interaction with patients. Another possible relation between categories is that of financial issues (category C) to change process and technical issues. This is explained, when lack of financial resources (C1) in case of a HIS implementation may lead to insufficient provision of training for users and designers of the system (E2) and this may subsequently lead to
ineffective technical systems and problematic daily execution of the computer tasks. Finally we can also mention interrelations between the subcategories within a specific category issue. In case of workflow issues for example we can argue that computer literacy (A3) affect directly the execution of computer tasks (A1), the requirements of the execution of computer tasks affect the users’ interaction with patients (A4) and the communication between clinicians (A5), and finally the daily execution of computer tasks may be directly related to the issue of health concerns (A6). Moreover in case of technical issues we can argue that the issues of standardization (B2) and system design (B1) may lead to lack of reliability to the HISs (B4).
16 | P a g e standardization of the systems and the impaired communication between them. Time is also discussed within the issue subcategory of interaction between patients, physicians and nurses. As analyzed more extensively in the results part the physicians and nurses seem to spend more time in front of their computer interfaces at the expense of their interpersonal
relationship with patients. Although most of the issues analyzed in the results, were found to represent unanimous
concerns of physicians, nurses and administrators, we can distinguish some of them that referred in different ways by our groups. First of all, we found that administrators are mostly concerned about the financial issues of HISs implementation, such as costs and needs for adequate initial financial resources while physicians and nurses referred less to these issues. For physicians and nurses workflow and technical issues that affect most directly their daily clinical tasks are most important and thus frequently mentioned. The issue subcategories of performance evaluation and standardization for example, were explicitly presented as physicians’ and nurses’ complaints to the results of the reviewed studies. In particular, these two groups seem to raise concerns regarding unfavorable surveillance features and restrictive templates that are usually embedded in computerized systems, while administrators seem to value the existence of such features. Finally, one interesting distinction found between physicians and nurses refers to their additional workload through the use of HISs. While physicians found to attribute it mostly to the software features of the electronic systems that affect their ability to make proper and timely decisions, nurses attribute it mostly to the hardware features (environmental design of the systems), because of the mobile nature of their daily work. However, we need to mention that the lines between the three analyzed stakeholder groups are not absolutely explicitly distinct. Despite our initial preference to include studies based on one group in order to gain the opportunity to establish comparisons between them, most studies involved multiple user groups and generally gave overall group results rather than information specific to each individual group. As such, it is possible that the results presented are not completely mutually exclusive across each stakeholder group.
Theory implications
Our findings indicate that Information Systems are much more than computers and telecommunications equipment, as they also involve people and their actions in the organizational settings in which they work. Despite the positive effects of IS in health care practices, the adoption of such systems entails issues that may impede the progress and the ultimate success of them. The categorization of issues that may emerge during HIS
implementations through a stakeholder perspective, is provided by this study and comes to verify the concern of many scholars from the socio-technical approach (Cherns, 1987; Clegg, 2000; Land, 2000; Mumford, 1995), about the need for a better balance between the human and the technical aspects in the design of systems and workplace. Viewing processes from a purely technical or organizational perspective limits our understanding of them.
17 | P a g e practitioners, pharmacies, vendors of IS, government agencies and insurers, since their varying interests, power and attitudes to the HIS could provide more useful information.
Practice implications
This study suggests, that it is important for future HISs implementers, such as hospital managers, project leaders and change managers, to understand how key stakeholders affect and be affected by the use of these systems in order to be able to prevent or manage effectively issues that might arise during the implementation process. Our results give implementers the direction to manage HISs as technochange situations and not as merely IT projects or organizational change programs. According to Markus (2004), a technochange involves both IT and organizational changes and thus differs both from IT projects and from organizational change programs. In that way, if implementers in a future HIS implementation, focus just on the technological aspects of the system but fail to address the critical issue of training staff, this may have negative effects on the proper use by clinicians and the
subsequent success of the project. Our change process category of issues accompanied by the technical and workflow ones, may propose to implementers that each HIS implementation project should be aligned and consistent with the organizational structure and culture. This review suggests that HISs are rather complex systems that require high degree of interaction
between the users and high degree of flexibility. We hope that the analysis of the issues and their interrelations provided by this paper will be a
18 | P a g e Table 3 Overview of studies included in the systematic literature review
Article nr. Author Country/ region of data collection Main objective of study Type of research
Data collection Participants Type of hospital
1. Ash et al. 2004 USA, Australia, Netherlands To describe and interpret the nature of Patient Care Information System-related errors. Qualitative Semi-structured interviews, observations
Physicians, Nurses 24 hospitals
8. Boyer et al. 2010
France To examine health care professionals ’ opinions of the critical events (opportunities and barriers) surrounding EMR implementation. Qualitative Semi-structured interviews 115 Psychiatrists, nurses, psychologists and social assistants, secretaries and administrative professionals
Psychiatric teaching hospital
12. Chao et al. 2013
Macao To investigate the utility of
19 | P a g e 17. Embi et al.
2004
USA To determine the perceived impacts of Computerized Physician
Documentation among faculty and house staff in a Veterans Affairs Medical Center.
Qualitative Semi-structured interviews
10 Faculty physicians and 10 resident physicians
Teaching hospital
18. Embi et al. 2013
USA To assess subjects’ perceptions and usage of
Computerized Provider Documentation at five distinct veterans affairs sites of varying
size, complexity and geographical location across the USA.
Qualitative Focus groups 54 Practitioners, 38 Nurses, 37 Administrators
5 Veterans Affairs medical centers
21. Goldman et al. 2012
Canada To investigate the process of development and implementation of the Colposcopy IS and its initial impact on clinicians’ professional practice. Quantitative and qualitative Survey and semi-structured interviews Quantitative: 15 nurses, 5 physicians
Qualitative: 10 interviews with nurses, 8 interviews with physicians and 6 interviews with IT team members
Teaching hospital
30. Kohle-Ersher et al. 2012
USA To evaluate the
barriers that nurses and nurse aide/clinical technicians
encounter for electronic point-of-care documentation. Quantitative and qualitative Survey and semi-structured interviews Quantitative:24
participants( nurses and nurse assistants)
Qualitative: 14 interviews with nurses and 6 interviews with nurse assistants
20 | P a g e 33. Li et al.
2013
China To assess the
preparedness status of a hospital in Beijing, for implementation of an e-Health system in the context of a pandemic response.
Qualitative Interviews Clinicians, IT managers, Chief Information Officer
1 Hospital
34. Likourezos et al. 2004
USA To assess physician and nurse satisfaction with an Emergency Department (ED) EMR.
Quantitative Survey 44 clinicians (23 physicians and 21 nurses), 38,3% response rate
Teaching hospital
35. Lyons et al. 2005
USA To compare the perceptions of three stakeholder groups regarding information technologies as barriers to and facilitators of clinical practice guidelines (CPGs).
Qualitative Focus groups 102 Administrators, 103 physicians, 117 nurses 18 Veterans Affairs Medical Centers. 36. Mannan et al. 2006 England To investigate the perceptions of primary care staff towards
e-health initiatives in the NHS Connecting for
Health programme and whether front-line staff are ready to implement such changes. Qualitative Semi – structured interviews 20 Participants(doctors, nurses, practice
managers and receptionists)
21 | P a g e 38. McGinn et
al. 2012
Canada To assess the applicability, the importance, and the priority of the results of the systematic review of international literature concerning users’ perspectives of the factors influencing EHR implementation.
Quantitative Survey 64 Participants(non-physician healthcare professionals, health information professionals, managers, and physicians
More than 20 national and provincial e-health and healthcare professional associations. 41. Noblin et al. 2013 USA To investigate clinicians perceptions towards the implementation of an EHR in a new clinic.
Qualitative Semi-structured interviews
23 Participants( Physicians and medical assistants) Academic medical center 44. Piscotty et al. 2011 USA To
examine the Clinical Information System implementation readiness activities adopted by Chief Nurse Executives in hospital settings.
Qualitative Interviews 6 Chief Nurse Executives Teaching and community hospitals
50. Shachak et al. 2008
Israel To describe physicians’ patterns of using an Electronic Medical Record (EMR) system.
Qualitative Semi-structured interviews and field
observations
22 | P a g e 51. Shield et al.
2010
Greece To examine the effects of EHR implementation, especially regarding physician-patient communication and behaviors and patients’ responses. Quantitative and Qualitative Semi-structured interviews, focus groups and observation
Nurses, nurse’s aides, office staff, physicians 1 Residency-based family medicine outpatient clinic 57. Weir et al. 2007
USA To report on a pilot conducted across multiple VA sites and provider
roles to identify current concerns, prevalent practices,
and user perceptions regarding CPD in the VA(Veteran affairs) outpatient setting.
Qualitative Interviews 78Participants(14 nurses, 53 ordering providers, 3 clerks, and 8 pharmacists)
10 primary care sites
56. Weir et al. 2011
USA To explore the experience of experienced users of computerized patient documentation for the purpose of
collaboration and coordination.
Qualitative Focus groups 116 participants( nurses, physicians and administrative staff)
23 | P a g e Table 4 Taxonomy of issues
Category Issues References (Article no.) References
per issue
References per category A)Workflow A1.Computer tasks
A2.Performance evaluation A3.Lack of computer literacy A4.Interaction with patients A5.Loss of communication A6.Health-related concerns 1/8/17/18/35/57 1/35 12/34/35/44 1/8/12/17/21/30/34/35/38/41/51 1/17/18/21/56 35 6 2 4 11 6 1 30
B)Technical B1.System design B2.Standardization B3.Fragmentation B4.Lack of reliability 1/12/18/21/30/33/35/38/51 1/12/17/18/21/30/35/56 1/12 12/17/18/33/36/56/57 9 8 2 7 26
C)Financial C1.Lack of financial resources C2.Costs and return on investment
35/36 12/33
2 2
4
D)Ethical D1.Lack of confidentiality 8/14/30/34/35/36/38/51 8 8
24 | P a g e
Appendix A – Quality assessment
Table 1: quality assessment results of qualitative studies
Criteria qualitative studies
[1]
[35]
[50]
[36]
[18]
[41]
[8]
[12]
[33]
[11]
[44]
[57]
[17]
[56]
[49]
Question/objective sufficiently described?
2
2
2
2
1
1
2
2
2
1
2
1
2
2
1
Study design evident and appropriate?
2
2
2
2
2
2
2
2
2
0
2
2
2
2
2
Context for the study clear?
2
2
2
2
2
1
2
2
2
1
2
2
1
1
0
Connection to a theoretical framework/wider body of knowledge?
1
1
2
2
1
1
2
1
1
0
2
2
2
2
1
Sampling strategy described, relevant and justified?
0
2
1
2
1
1
2
1
2
1
1
2
2
1
1
Data collection methods clearly described and systematic?
1
2
1
1
2
1
2
1
2
1
2
1
2
1
1
Data analysis clearly described and systematic?
1
2
2
1
2
1
1
0
1
1
2
1
2
1
1
Use of verification procedure(s) to establish credibility?
0
0
0
1
2
1
0
0
0
0
2
0
2
0
0
Conclusions supported by the results?
2
2
2
2
1
2
2
2
2
2
2
1
2
2
2
Reflexivity of the account?
1
0
0
1
1
0
0
0
1
0
0
0
0
0
0
Total score/possible maximum score
12/
25 | P a g e
Table 2: quality assessment results of quantitative studies
Criteria quantitative studies
[34]
[38]
Question/objective sufficiently described?
2
2
Study design evident and appropriate?
2
2
Method of subject/comparison group selection or source of information/input variables described
and appropriate?
2
2
Subject (and comparison group, if applicable) characteristics sufficiently described?
2
1
If interventional and random allocation was possible, was it described?N/A
N/A
If interventional and blinding of investigators was possible, was it reported?N/A
N/A
If interventional and blinding of subjects was possible, was it reported?N/A
N/A
Outcome and (if applicable) exposure measure(s) well defined and robust tomeasurement/misclassification bias? Means of assessment reported?
2
1
Sample size appropriate?
2
2
Analytic methods described/justified and appropriate?
2
1
Some estimate of variance is reported for the main results?
0
2
Controlled for confounding?
1
1
Results reported in sufficient detail?
2
2
Conclusions supported by the results?
1
2
26 | P a g e
Table 3: quality assessment results of mixed methods studies
Qualitative criteria mixed methods studies
[51]
[21]
[30]
Question/objective sufficiently described?
2
2
2
Study design evident and appropriate?
2
2
2
Context for the study clear?
2
2
2
Connection to a theoretical framework/wider body of knowledge?
0
0
1
Sampling strategy described, relevant and justified?
1
1
1
Data collection methods clearly described and systematic?
2
1
2
Data analysis clearly described and systematic?
2
1
1
Use of verification procedure(s) to establish credibility?
0
0
0
Conclusions supported by the results?
1
2
2
Reflexivity of the account?
1
0
0
Quantitative criteria mixed methods studies
Question/objective sufficiently described?
2
2
2
Study design evident and appropriate?
2
2
2
Method of subject/comparison group selection or source of information/input variables described and appropriate?
1
0
1
Subject (and comparison group, if applicable) characteristics sufficiently described?
1
0
1
If interventional and random allocation was possible, was it described?N/A
N/A
N/A
If interventional and blinding of investigators was possible, was it reported?N/A
N/A
N/A
If interventional and blinding of subjects was possible, was it reported?N/A
N/A
N/A
Outcome and (if applicable) exposure measure(s) well defined and robust tomeasurement/misclassification bias? Means of assessment reported?
0
0
0
Sample size appropriate?
1
1
2
27 | P a g e Some estimate of variance is reported for the main results?
0
1
1
Controlled for confounding?
N/A
N/A
N/A
Results reported in sufficient detail?
1
1
1
Conclusions supported by the results?
1
2
2
28 | P a g e
References
1.Ash J, Berg M, Coiera E, Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. J Am Med Inform Assoc. 2004; 11:104–112.
2.Bates WD, The quality case for information technology in health care. BMC Medical Informatics & Decision Making. 2002; 2:7.
3.Bates WD, Physicians and Ambulatory Electronic Health Records. Health Affairs, 24, no.5 (2005):1180-1189.
4.Benjamin D. et al., Issues management in the information planning process. June 1987. 5.Bingi, P., Sharma, M.K, Godla, J.K., Critical issues affecting ERP implementation. Information Systems Management. 1999;16(3), pp. 7 – 14.
6.Boonstra A, Boddy D, Bell S, Stakeholder management in IOS projects: analysis of an attempt to implement an electronic patient file. European Journal of Information Systems (2008) 17, 100-111.
7.Bowen, S.A. (2005), A practical model for ethical decision making in issues
management and public relations. Journal of Public Relations Research, Vol. 17 No. 3, pp. 191-216.
8.Boyer L, Samuelian J, Fieschi M, Lancon C, Implementing electronic medical records in a psychiatric hospital: A qualitative study. International Journal of Psychiatry in Clinical Practice, 2010; 14: 223–227.
9.Burchell, J. and Cook, J., It’s good to talk? Examining attitudes towards corporate social responsibility dialogue and engagement processes. Business Ethics: A European Review, 15(2), pp. 154 – 170, (2006).
10.Canada Health Infoway and Health Council of Canada: Beyond good intentions: accelerating the electronic health record in canada. Policy Conference Montebello, QC, Canada; 2006.
11.Carnicero J, Rojas D, Lessons Learned from Implementation of Information and Communication Technologies in Spain's Healthcare Services – Issues and Opportunities. Appl Clin Inf 2010; 1: 363–376.
12.Chao W, Hu H, Ung C, Cai Y, Benefits and Challenges of Electronic Health Record System on Stakeholders: A Qualitative Study of Outpatient Physicians. J Med Syst (2013) 37:9960.
29 | P a g e 15.Conway, T., Ward, M., Lewis, G. and Bernhardt, A. (2007), Internet crisis potential: the importance of a strategic approach to marketing communications. Journal of Marketing Communications, Vol. 13 No. 3, pp. 213-28.
16.Eesley, C., & Lenox, M. 2006. Firm responses to secondary stakeholder action. Strategic Management Journal, 27:765–781
.
17.Embi P, Yackel T, Logan J, Bowen J, Cooney T, Gorman P, Impacts of Computerized Physician Documentation in a Teaching Hospital: Perceptions of Faculty and Resident Physicians. J Am Med Inform Assoc. 2004; 11:300–309.
18.Embi P, Weir C, Efthimiadis E, Thielke S, Hedeen A, Hammond K, Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators. J Am Med Inform Assoc 2013; 20:718–726.
19.Finney S. & Corbett, M., ERP implementation: a compilation and analysis of critical success factors. Business Process Management Journal, 13(3), pp. 329-347(2007).
20.Freeman, R. Strategic management: A stakeholder approach. Cambridge, MA: Cambridge University Press, 1984.
21.Goldman J, Abramovich A, Sadovy B, Murphy K, Rice K, Reeves S, The Development and Implementation of an Electronic Departmental Note in a Colposcopy Clinic. CIN: Computers, Informatics, Nursing & Vol. 30, No. 2, 91–96, 2012.
22.Goldschmidt P, HIT and MIS: Implications Of Health Information technology And Medical Information Systems. Evaluating the potential advantages and considering the risks associated with electronic health care records. Communications of the ACM, October
2005/Vol. 48, No. 10.
23.Grant, G.G., Strategic alignment and enterprise systems implementation: the case of metalco. Journal of Information Technology, 18, p. 159, (2003).
24.Haux R. , A. Winter, E. Ammenwerth, B. Brigl, Strategic Information Management in Hospitals. An Introduction to Hospital Information Systems. Springer, New York, 2004. 25.Heath, R.L. (1997), Strategic Issues Management: Organizations and Public Policy Challenges, Sage, Thousand Oaks, CA.
26.Hodge T: EMR, EHR, and PHR – Why all the confusion? Canada Health Infoway; 2011.
27.Hong, K-K., & Kim, Y-G,The critical success factors for ERP implementation : an organizational fit perspective. Information & Management, (2002), 40, p. 25.
28.Jaques, T. (2007), Issue or problem? Managing the difference and averting crises. Journal of Business Strategy, Vol. 28 No. 6, pp. 25-8.
30 | P a g e 30.Kohle-Ersher A, Chatterjee P, Osmanbeyoglu H, Hochheiser H, Bartos C, Evaluating The Barriers To Point-Of-Care Documentation For Nursing Staff. CIN: Computers,
Informatics, Nursing & Vol. 30, No. 3, 126–133, 2012.
31.Land F, Evaluation in a Socio-Technical Context. Organizational and Social Perspectives on Information Technology,(2000).
32.Laplume, A.O., Sonpar, K., & Litz, R.A. (2008). Stakeholder theory : reviewing a theory that moves us. Journal of Management, 34(6), pp. 1152 – 1189.
33.Li J, Seale H, Ray P, Wang Q, Yang P, Li S, Zhang Y, Maclntyre C, e-Health
preparedness assessment in the context of an influenza pandemic: a qualitative study in China. BMJ Open 2013.
34.Likourezos A, Chalfin D, Murphy D, Sommer B, Darcy K, Davidson S, Physician And Nurse Satisfaction With An Electronic Medical Record System. The Journal of
Emergency Medicine, Vol. 27, No. 4, pp. 419–424, 2004.
35.Lyons S, Tripp-Reimer E, Sorofman B, Dewitt J, Bootsmiller B, Vaughn T, Doebbeling B, VA QUERI Informatics Paper Information Technology for Clinical Guideline
Implementation: Perceptions of Multidisciplinary Stakeholders. J Am Med Inform Assoc. 2005;12:64–71.
36.Mannan R, Murphy J, Jones M, Is primary care ready to embrace e-health?
A qualitative study of staff in a London primary care trust. Informatics in Primary Care 2006;14:121–31.
37.Markus ML, Technochange management: using IT to drive organizational change. Journal of Information Technology, (2004) 19, 3-19.
38.McGinn C, Gagnon M, Shaw N, Sicotte C, Mathieu L, Leduc Y, Grenier S, Duplantie J, Abdeljelil A, Légaré F, Users’ perspectives of key factors to implementing electronic health records in Canada: a Delphi study. BMC Medical Informatics and Decision Making 2012, 12:105.
39.Mitchell, R. K., Agle, B. R., Chrisman, J. J., & Spence, L. J. 2011.Toward a theory of stakeholder salience in family firms. Business Ethics Quarterly, 21: 235–255.
40.Mumford, E. (1995). Effective systems design and requirements analysis. London: Macmillan.
41.Noblin A, Cortelyou-Ward K, Cantiello J, Breyer T, Oliveira L, Dangiolo M, Cannarozzi M, Yeung T, Berman S, EHR Implementation in a New Clinic: A Case Study
of Clinician Perceptions. J Med Syst (2013) 37:9955.
42.Oliver, G.R. and Donnelly, P.J. (2007), Effective use of a strategic issue management system (SIMS): combining tools and approach. Journal of Public Affairs, Vol. 7 No. 4, pp. 399-406.
31 | P a g e 44.Piscotty R, Tzeng H, Exploring The Clinical Information System Implementation Readiness Activities To Support Nursing In Hospital Settings. CIN: Computers, Informatics, Nursing & Vol. 29, No. 11, 648–656, 2011.
45.Reinhold H, Health information systems - past, present, future. Int J Med Inform. 2006;75(3-4):268-81.
46.Robey, D., Ross, J.W., and Boudreau, MC., Learning to implement enterprise systems: an exploratory study of the dialectics of change. Journal of Management Information Systems, 19, p. 17, (2002).
47.Rowley, T. J. 1997. Moving beyond dyadic ties: A network theory of stakeholder influences. Academy of Management Review, 22: 887–910.
48.Schwalbe, K. (2009). An introduction to project management. Boston: Cengage learning.
49.Scott J, Rundall T, Vogt T, Hsu J, Kaiser Permanente’s experience of implementing an electronic medical record: a qualitative study. BMJ 2005; 331:1313–1316.
50.Shachak A, Dayagi M, Ziv A, Reis S, Primary Care Physicians’ Use of an Electronic Medical Record System: A Cognitive Task Analysis. J Gen Intern Med 2008, 24(3):341– 348.
51.Shield R, Goldman R, Anthony D, Wang N, Doyle R, Borkan J, Gradual Electronic Health Record Implementation: New Insights on Physician and Patient Adaptation. Ann Fam Med 2010;8:316-326.
52.Stausberg J, Koch D, Ingenerf J, Betzler M, Comparing Paper-based with Electronic Patient Records: Lessons Learned during a Study on Diagnosis and Procedure Codes. J Am Med Inform Assoc. 2003;10: 470–477.
53.Tarafdar, M. and Roy, R.K., Analyzing the adoption of enterprise resource planning systems in Indian organisations: a process framework. Journal of Global Information Technology Management, 6, p. 31, (2003).
54.Umble, E.J., Haft, R.R., Umble, M.M., Enterprise resource planning in a large
construction firm: implementation analysis. Construction Management and Economics, 2, p.511. (2003).
55.Wartick, S.L. and Heugens, P.P.M.A.R. (2003), Guest editorial: future directions for issues management. Corporate Reputation Review, Vol. 6 No. 1, pp. 7-18.
56.Weir C, Hammond K, Embi P, Efthimiadis E, Thielke S, Hedeen A, An exploration of the impact of computerized patient documentation on clinical collaboration. International Journal of Medical Informatics 80 (2011) xxx.e62–xxx.e71.
57.Weir C, Nebeker J, Critical Issues in an Electronic Documentation System. AMIA 2007 Symposium Proceedings.
