Information or Complication: A Qualitative Study on How Health Information System Implementation can Contribute to the Prevention of Technology-Induced Medical Errors

Information or Complication:

A qualitative study on how health

information system implementation can

contribute to the prevention of

technology-induced medical errors

Author:

Youssef Assal

10854533

Faculty of Science

Master Information Studies: Business Information

Systems

May 25, 2021

Supervisor

Ms. E. Takacs

Examiner

Dhr. drs. A.W.

Abcouwer

Abstract

Digitization, as a result of the information age, has meant a shift in thinking for society, which led to redesigning commonplace practices. The medical field has not been an exception. Increasingly comprehensive infor-mation systems are developed, aiming to make vast amounts of informa-tion manageable. However, wide adopinforma-tion of informainforma-tion systems in the medical field has, in some contexts, added further complexity to already high-pressure work environments. This added complexity can effectively lead to unintended consequences, which contribute to medical errors. To combat this trend, corrective actions were designed to mitigate these un-intended consequences of information systems. Fundamentally, however, implementation of corrective actions is troublesome because they have not been challenged on their feasibility, acceptance, and beneficiality in real-world contexts. The main objective of this research is to find out in which manner health information systems should be implemented to contribute to the prevention of technology-induced medical errors. This was achieved by determining if corrective actions would be effective in improving a health information system by investigating their beneficial-ity and investigating if, when applied, they were feasible within a medical context, accepted, and clarifying why. Finally, the research classifies types of actions and reveals arising conflicts which are then interpreted.

Contents

1

Introduction

1

1.1

Relevance of research . . . .

1

1.2

Research question . . . .

1

2

Review of Literature

1

2.1

Unintended consequences & medical errors . . . .

2

2.2

Corrective actions . . . .

3

2.3

Conceptual model . . . .

3

3

Methodology

5

3.1

Approach . . . .

5

3.2

Data Collection . . . .

5

3.2.1

Desk research . . . .

5

3.2.2

Field research . . . .

6

3.3

Data Analysis . . . .

7

3.3.1

Desk Research . . . .

7

3.3.2

Field Research

. . . .

7

3.4

Validation and Reliability

. . . .

7

4

Results

8

4.1

Results of Desk Research . . . .

8

4.2

Results of Field Research . . . .

8

4.2.1

Monitoring and reporting

. . . .

8

4.2.2

Policies

. . . .

9

4.2.3

Reassesment and redesign . . . 11

5

Discussion

12

5.1

Which corrective actions can be proposed to counter technology-induced medical

errors? . . . 12

5.2

Which types of corrective actions are feasibly implementable within a medical context? 12

5.3

Will corrective actions conflict with user acceptance and, if so, why?

. . . 13

5.4

Which types of corrective actions offer a net benefit to the organizations that adopt

them? . . . 14

6

Reflection

15

7

Conclusion

15

7.1

Future work . . . 16

1

Introduction

Digitization, as a result of the information age, has meant a shift in thinking for us as a society. It has even made us redesign commonplace practices. The medical field has not been an exception. It has seen a great shift not only in modernization but also in sharing and process-ing of information. With today’s accessibility to modern medicine, it has become necessary to track large groups of patients. Information needs to be processed on, among other things, their identity, complaints, health, and indi-vidual progress. For these types of purposes increasingly comprehensive information systems are being developed, aiming to make vast amounts of information manageable for medical professionals. An incidental consequence, however, is that information systems can be perceived as increasingly complex by end-users. In high-pressure work environments like hospitals, this complexity can be at the expense of the utility of the information system and pos-sibly even lead to a medical error. This counterproduc-tive consequence of information system implementation is not unique. Actually, a wide array of so-called unin-tended consequences exists, resulting from the adoption of information systems within the medical field. These consequences are unintended by-products of information systems, which in some cases can even be technology-induced medical errors. As a society, we should not tol-erate that a system that is, among other things, designed to increase safety also contributes to patient unsafety via a different route. For this reason, this research investi-gates in which way medical information systems could be applied to contribute to the prevention of medical errors.

1.1

Relevance of research

With the widespread adoption of health information sys-tems, logically more descriptive researches were done on the issues resulting from the use of IS systems. While it is well documented which unintended consequences can arise with health information system implementation, no adequate implementable solutions are offered. There are few researches done on the corrective actions applied to health information systems, especially specifically tar-geting the collection of unintended consequences that constitute as technology-induced medical errors. Cor-rective actions are actions to eliminate the cause of a

non-conformity or undesirable situation, in this case un-intended consequences, and prevent recurrence (Interna-tional organization for standardization [ISO], 2005). Also in this research strictly unintended consequences related to human-computer interaction are targeted, making the results very usable for medical practitioners. The al-ready existing researches on corrective actions are not concrete guidelines but stay general, often without dis-cussing feasibility. Furthermore, these corrective actions can not be implemented due to the lack of being chal-lenged on their feasibility, accessibility, and beneficiality in real-world contexts. These three dimensions are key in adding actual value within the medical field. This re-search instead aims to offer willing institutions informa-tion on which types of corrective acinforma-tions to apply. This is achieved by investigating if they are feasibly imple-mentable within a medical context, accepted, beneficial, and clarify why.

1.2

Research question

In which manner should health information systems be implemented to contribute to the prevention of technology-induced medical errors?

To be able to answer the main research question correctly, this research needs to address various sub-questions in the discussion section. The following ques-tions need to be answered.

• RQ1: Which corrective actions can be proposed to counter technology-induced medical errors?

• RQ2: Which types of corrective actions are feasibly implementable within a medical context?

• RQ3: Will corrective actions conflict with user ac-ceptance and, if so, why?

• RQ4: Which types of corrective actions offer a net benefit to the organizations that adopt them?

2

Review of Literature

In the last decades, increasing familiarity and usefulness of technology has meant a growing implementation of

information technology within the medical field, affect-ing both patients and professionals (Winter et al, 2011). Because of this a significant amount of scientific liter-ature aims to identify flaws in the form of unintended consequences as a result of IS implementation. In this research, we focus on applying corrective actions to a subset of unintended consequences of health information systems. This subset consists of the unintended conse-quences that originate from human-computer interaction and are significant enough to constitute a technology-induced medical error or cause one. The existing litera-ture, however, focuses on unintended consequences as a whole.

2.1

Unintended consequences & medical

errors

Within the literature, various categorizations are cre-ated to summarize the vast amount of unintended con-sequences. For instance, one paper describes a catego-rization in three manners. Firstly is the design and de-velopment of technology. Secondly, the implementation and customization of technology and lastly the interac-tions between the operation of new technology and the new work processes that arise from its use (Coiera, Ash & Berg, 2016). It is important to explain that a type of these unintended consequences are technology-induced errors (Borycki et al., 2016). They differ from other med-ical errors due to them originating from the technology, in this case, a health information system, itself. These three categories largely depict the nature of the exist-ing unintended consequences. For this research, however, the focus is strictly on unintended consequences caused by human-computer interaction. This human-computer interaction corresponds with the cognitive level of the framework in figure 2.

Consequently, the question arises of what unintended consequences occur? From the start of this millennium, researches proved the occurrence of more obvious un-intended consequences. Sittig, Wright, Ash & Singh (2016) showed for instance that the adoption of an elec-tronic health record can increase the workload, due to the employees dealing with alerts, entry fields, and de-tails. Other research showed an increasing workload due to a clinical documentation burden, causing employees to document after working hours (Moy et al., 2021). At the

same time efforts are being made to measure clinician burnout (Kannampallil et al., 2020). Also, nurse well-being has become worse with the electronic health record, compared to when paper charts were used (Nguyen et al., 2020). Additionally, the digitization of information processing resulted in increasing difficulty to keep the system up to date, forced changes in communication be-tween personnel, and created a new set of usability er-rors. Also, a persistence to use paper for data, even when temporary and occasionally, remained. This can cause confusion as to where to locate information. Other unin-tended consequences were a loss of power because of the inflexibility of an electronic health record requiring em-ployees to comply with its methods, an overdependence on an electronic health record, and negative emotions triggered by the electronic health record.

More recently, however, because of rapid adoption, more unforeseen unintended consequences have taken place. According to Ash, Berg & Coiera (2004) some could possibly constitute or cause a medical error. One has been the unavailability of access to data, as a result of a lack of data sharing (Sittig et al., 2016). A wide array of technological as well as organizational issues prevent sharing (Adler-milstein & Pfeifer, 2017). There-fore this prevents patient data to be available at all times and every point. Another consequence is the increasing complexity of using an information system for the end-user (Sittig et al., 2016). This issue stems from a larger trend being, the lack of usability innovations (American Medical Association [AMA], 2014). This can be partly attributed to electronic health record certification de-manding continuous changes to an information system and new interface expectations due to tablet computing (Sittig et al., 2016). A third consequence is the fact that information system based quality measurements have led to complex clinical workflows, because of the need to collect structured data. These workflows can interfere with tasks and have adverse events like errors as a result (Upadhyay, Sittig & Singh, 2014). Also, interactions between patient and provider can suffer (Street Jr et al., 2014). The fourth final consequence is an overload of information due to the vast amount of information collected per patient nowadays (Murphy, Reis, Sittig & Singh, 2012). Information systems add to this problem by collecting and presenting more information than be-fore digitization. Employees can be overwhelmed with

information causing them to overlook crucial elements of it, leading to errors (Singh et al., 2013).

Table 1: Examples of Unintended Consequences originating from human-computer interaction

Overdependence & Loss of power Cognitive overload & Increased workload New set of Usability errors Alert fatigue

2.2

Corrective actions

Campanella et al. (2016) show information systems are primarily an advantage to significant medical organiza-tions. Improvements could be in quality of care, costs or even communication if there is interoperability (Hoff-man, 2016) or standards between systems (Balgrosky, 2019). However, they clearly have these unintended con-sequences as side effects (Ash et al., 2004). Part of these unintended consequences of an information system can be technology-induced medical errors (Ash et al., 2004). Naturally, this raises the question of which changes must be made to the implementation of health information sys-tems, to minimize unintended consequences and there-fore minimize the risk of medical errors occurring. And if changes were to be applied to combat technology-induced medical errors, in which area would that be?

Fortunately the existing literature points in certain directions. Corrective actions are actions to eliminate the cause of a non-conformity or undesirable situation, in this case, unintended consequences, and prevent re-currence (Iso, 2005). Corrective actions to counter unin-tended consequences are likely to be, among other things, in the form of software change, training, process change, and policy change (Jones et al., 2011). Several exam-ples of more concrete changes are as follows, carefully re-viewing rejected information system alerts, staff involved reassessment of the information system and obliging de-partmental permission by sign off for a subset of IS orders (Jones et al., 2011).

Altogether, a wide array of corrective actions could be proposed by using literature, even when sometimes unspecific, as a base. This leads to the need of exploring which types of corrective actions should be selected by investigating which are viably implementable, accepted, beneficial, and why? The answers to these questions can

only be reached through thorough qualitative research in-volving medical professionals who make frequent use of the very systems that form the core of the problem. Col-lectively these results allow us to then answer the main research question of this research.

2.3

Conceptual model

This research has constructed a conceptual model (fig-ure 1) to illustrate the variables and the relationships between them. The first relationship is between the “cor-rective action” variable and “unintended consequences” variable. The “corrective action” variable has an effect on the “unintended consequences” variable because cor-rective actions aim to mitigate or lessen unintended con-sequences. The relationship is reciprocal because the amount of unintended consequences directly affect the corrective action variable. This is because growth of un-intended consequences will lead to more corrective ac-tions being applied within an organization. The relation-ship between the two variables is affected by an interac-tion effect of two other variables called “user acceptance” and “organizational constraints”.

Firstly, the “user acceptance variable” shall be dis-cussed. A corrective action is applied to a health infor-mation system with the intention of mitigating or lessen-ing the information system its unintended consequences. Within the context of a health information system, how-ever, the corrective action is often dependent on compli-ance by the end-user. If the corrective action is a policy, for example, it requires compliance from the employees. Therefore acceptance of the corrective action by the end-user influences the relationship between the “corrective action” variable and the “unintended consequences” vari-able. If there is a high acceptance of a corrective action, the action is more likely to be successful in affecting the unintended consequences because of correct implementa-tion. Employees stated that, for example, if a corrective action is too disruptive, it could be disregarded in its en-tirety by the staff. Literature also shows that when it comes to alerts, for example, they are often overridden which damages their effectiveness (Van Der Sijs et al., 2006). To answer the third research question the user acceptance of corrective actions and what motivates it will be investigated.

variable. The success of a corrective action is logically dependent on its practical feasibility within an organiza-tion. A corrective action requires effort and can only be implemented if the organization can practically facilitate it. Organizations however have their constraints, and not every action will be feasibly implementable. For exam-ple, a corrective action that revolves around monitoring human-computer interaction could not be feasible due to privacy concerns or lack of flexibility of the IT depart-ment. Employees, for example, stated they would protect their patient’s privacy when it comes to corrective ac-tions involving observation. The literature has also iden-tified some of these constraints or barriers when it was related to electronic health record implementation, and financial costs appeared most often but also insufficient time, technical concerns, and privacy concerns (Kruse et al., 2016 ). The negative impact that organizational constraints can have are depicted in the model by the negative effect of the “organizational constraints” vari-able on the relationship between the “corrective action” and “unintended consequences” variable. The feasibility of corrective actions will be investigated to answer the second research question.

Thirdly, although a corrective action can be success-ful in mitigating unintended consequences, it does not

necessarily mean that this result forms a net benefit to the organization. This is because every corrective action has costs and will be at the expense of other resources within the organization like staff, time and finances. If the costs or consumption of implementing a corrective action are too high, there will be no net benefit because it will be at the expense of other tasks within healthcare. In the model, this effect is depicted by the negative effect that the “resource constraint” variable has on the rela-tionship between the “unintended consequences” variable and the “net benefit variable”. Furthermore, added com-plexity because of corrective actions can lead to cognitive overload, and therefore lead to mistakes. That means that the costs are too high for staff, which is also a re-source for the organization. Literature has shown that electronic health record design and use factors are asso-ciated with stress and burnout (Kroth et al., 2019) and that information overload can lead to practitioners miss-ing test results (Smiss-ingh et al., 2013). The fact that high costs of resources can cause additional unintended conse-quences is depicted by the arrow between the “resource costs of action” variable and “unintended consequences” variable. The net benefit of the research its selected cor-rective actions will be investigated to answer the fourth research question.

3

Methodology

3.1

Approach

The methodology consists of desk research and field re-search. Firstly, desk research was done in the form of a literature search. Secondly, field research was done in the form of a qualitative interview. Based on existing lit-erature a wide array of changes to medical information systems could be proposed. Therefore the first research question was answered by carrying out a literature search which focuses on what the possible corrective actions are. The resulting corrective actions were used as a base for the question design of the qualitative interview. The data resulting from the qualitative interviews were needed to answer the second, third and fourth research question. The results should collectively form an answer to the main question by determining what types of cor-rective actions are viably implementable, accepted, why, and what is to gain by doing so.

Within this research, the selected corrective actions are strictly intended to target a subset of unintended con-sequences. The subset consists of the corrective actions that are based in human-computer interaction, and are significant enough to constitute or cause a technology-induced medical error. By focusing on this subset of corrective actions the research is kept feasible and adds more specific value to the existing research.

The information desired from the qualitative inter-view can only be gathered by consulting medical profes-sionals that have daily experience in dealing with medical information systems as end-users. Also to keep the re-search as structured as possible the proposed corrective actions were categorized into three categories. The first being monitoring & reporting, the second being policies and the third being reassessment & redesign.

3.2

Data Collection

3.2.1 Desk research

The qualitative interview was based on the findings of a literature search on, corrective actions that prevent a subset of unintended consequences and counter med-ical errors. The corrective actions were distilled from a literature search and categorized. When two of the collected corrective actions were too similar, they were

merged into one. The corrective actions were collected from researches that comply with the inclusion criteria and do not conflict with the exclusion criteria

From each research strictly the corrective actions that can be applied to health information systems and counter unintended consequences that occur from human-computer interaction were collected. This means an interaction between a user and system. In the frame-work of figure 2, the intended interaction is shown and defined as an interaction on the cognitive level. This level centers around cognitive aspects of user interaction like information seeking, reasoning, and decision making (Borycki & Kushniruk, 2010). Under Appendix A the results of the literature search are shown.

Search Process

The search was conducted using various web search engines that have access to scientific publications, among which Google Scholar, Embase and PubMed. The search was limited to researches in the English language. A variety of search terms were used in alternate combina-tions. By using boolean operators the search system-atically combined terms like “corrective actions”, “un-intended consequences”, ”mitigation”, “health informa-tion systems”, “electronic health record” and “corrective policies”.

Inclusion Criteria

• Researches discussing corrective actions to health medical information systems.

• Researches containing policies that can be qualified as corrective actions.

• Researches containing methods that can be quali-fied as corrective actions.

Exclusion Criteria

• Researches discussing corrective actions that can not counter unintended consequences that occur from human computer interaction.

• Researches discussing corrective actions that are unspecific.

• Researches discussing corrective actions that are abstract.

• Researches discussing corrective actions that do not affect the end-user in any way.

• Research focusing on corrective actions strictly for the basic workflow level.

• Research focusing on corrective actions strictly for the organizational level

Figure 2: Framework for conducting cognitive-socio-technical analyses at three levels, by E.M Borycki Kushniruk AW, 2010.

3.2.2 Field research

The qualitative interview was a semi structured inter-view conducted among professionals within the medical industry. The interview consists of a set of questions re-lated to the, through desk research, collected corrective actions.

Setting

The qualitative research was conducted within the Netherlands. This means the data was derived ex-clusively from Dutch medical institutions. Within the Netherlands the most comprehensive and widely imple-mented information system within a medical context is the electronic health record. Clinical decision support systems for instance are not as widely used in

prac-tice within the Netherlands (Weda, de Bruijn, Alves, de Vries, 2019). For this reason the main focus was on the electronic health record, as a health information system.

Participants

There were two conditions for the selection of par-ticipants for this research. Firstly they must be working within a medical institution within the Netherlands. Sec-ondly they must have extensive experience with using a health information system as an end user. This research did not select on specific functions or positions within the medical field, because every experience with health information systems is valuable.

The questions were answered for each category of corrective actions. The categories of corrective actions are “Monitoring & Reporting”, “Policies” and “Reassess-ment & Redesign”. Each of these categories entails a set of corrective actions, which can be seen in the results section. The questions were designed based on the sec-ond, third and fourth research questions which revolved around feasibility, acceptability, and beneficiality.

When it comes to feasibility its set of questions were designed to find out the respondent’s opinion on the feasibility of corrective actions. Examples are finding out what organizational conflicts are, how the organiza-tion could help facilitate an acorganiza-tion, and determining the amount of time available. Secondly, when it comes to ac-ceptability the questions are designed to try to gauge the respondent’s attitude to a certain action and gauge their compliance. Thirdly the questions regarding the bene-ficiality were designed to find out which actions would constitute a net benefit to the organization implementing them.

3.3

Data Analysis

3.3.1 Desk Research

The data analysis consisted of classifying the collected corrective actions to health information systems on the basis of their nature. They were classified into three cat-egories, as can be seen under Appendix A. The cate-gories are as follow. Monitoring & reporting, policies and reassessment & redesign. These categories were the com-mon themes noticed during the search process. Therefore the corrective actions were clustered into these three cat-egories. This categorization also allowed us to explore if knowledge can be found about a cluster of corrective ac-tions, and not strictly isolated corrective actions. The corrective actions are presented under Appendix A, and were used to answer the research questions.

3.3.2 Field Research

The qualitative interview was thoroughly transcribed, af-ter which, it was coded. A grounded theory method was applied using first open, then axial and lastly selective coding. The ultimate goal is to find the relevant results and answer each research question within the discussion

section. A sample of the codes that were used can be seen in the annex.

3.4

Validation and Reliability

In order to maximize validity a set of choices were made. A qualitative interview was selected as the research method because it provides the research with in-depth data. For instance, where a quantitative interview could give us data about the feasibility, acceptance, and net benefit of corrective actions, it can not answer the why. Qualitative research provides us with data of the un-derlying reasons that lie at the base of why a certain corrective action is successful and another is not. Ad-ditionally, the interview was semi-structured. By doing this the validity can be improved through preparation, while at the same time in-depth data can be collected by asking more questions than were prepared. The ques-tion design of the interview was transparent by being tied into the research questions and therefore revolving around feasibility, acceptance, and net benefit.

Also, the selection of interview participants and the condition of them having to have worked with the elec-tronic health record contributes significantly to the qual-ity of the collected data. A special attribute of my participant selection is that nearly each of my partici-pants has additional management experience within a medical institution. Meaning most of them also lead medical employees. As a result of this, my participants could speak from their perspective but also have exten-sive knowledge of different perspectives from colleagues. This meant each participant had the capacity of giving a diverse set of views which has certainly led to a more diverse dataset, possibly being more representative. The research recruited respondents to the point that the de-rived information was consistent and reliable. At the point where there was clearly no additional or new infor-mation added by respondents, the research had collected its desired dataset.

Thirdly in order to also maximize reliability, a set of choices were made. The transparency of the qualita-tive interview allows others to reproduce it in a similar fashion. Instead of a general interview about abstract corrective actions, this research conducted a literature

search instead. This contributes to the reliability be-cause a limited set of corrective actions collected through a transparent literature search was used as an example for the interviewees. Also, the interviews were semi-structured, which allowed transparency into the question design and therefore increased consistency of results. A well-defined selection of participants, who had the de-sired expertise, and additional management experience within the medical field adds to this. Furthermore, the research selected the electronic health record, which is widely implemented within the Netherlands, and focused strictly on human-computer interaction.

4

Results

4.1

Results of Desk Research

In the three tables under Appendix A, the collected corrective actions are presented. They are categorized in three categories based on their nature. Additionally the actions are adjusted and merged when necessary for this research, effectively making them unique. The categories are as follows. Monitoring & reporting, policies and re-assessment & redesign. On the left side of the table, the corrective actions are shown. On the right side of the table, one can find the corresponding sources related to a specific corrective action.

4.2

Results of Field Research

4.2.1 Monitoring and reporting

Beneficiality

Generally, the respondents clearly did not doubt the capacity of monitoring and reporting systems to detect unintended consequences or errors. Half of the respon-dents specifically characterized their work within the information system as often being on automatic pilot, meaning that the reporting is done fast and efficiently, which can cause them to be unaware or unattentive of flaws and mistakes made within the system. A respon-dent for example would want a monitoring system to intercept early mistakes, like medication errors. This work on autopilot is a result of time constraints as a

result of high work pressure and distractions by patients and colleagues during their contact with the system. For example, a majority of the respondents regularly did their reporting within the information system after work hours, due to the lack of time during scheduled work hours. A threat for beneficiality however, as multi-ple respondents stated, is employees being aware of the monitoring, and adjusting their behavior as a result of it. One respondent mentioned that she would be forced to spend more time on reporting and less time on contact with patients. This behavior can negatively affect the net benefit of monitoring.

When it comes to implementing an issue desk during the first phase of implementing a system, the respondents en-dorse its beneficiality specifically during the introduction of a system. However multiple respondents had specific requirements for an issue desk. One requirement is being able to get help directly through the system, for example by a chat or call function. Another requirement was to be able to contact the issue desk outside of working hours because that’s when an employee is normally not pre-occupied with patients. Safety alerts were only seen as beneficial if they were never blockades or hard stops, and if they would not be issued too often. Under these terms, safety alerts were valued, for preventing the very time-consuming process of fixing an error after it happens. Pursuing errors and doing failure mode analysis was val-ued and seen as beneficial, but if the employees needed to be consulted, this would preferably happen outside of working hours. Lastly, most respondents already had an internal reporting system for adverse events in place and stated it should be easy and accessible to lower the high threshold that exists to report. Reporting volun-tarily, confidentially, and non-punitive, was surprisingly not seen as beneficial by most. This is because it would damage the ability to pursue the cause of an adverse event, and speak to the employees responsible.

Feasibility

One respondent specifically doubted the capacity of monitoring systems to actually detect errors in a timely manner. If not convinced this would affect her accep-tance of the system. Nearly all respondents characterized safety alerts that would be hard stops, meaning alerts

that would actually block the system and stop the work done by the employee, as absolutely not feasible. This was because of the nature of the medical world, they deal with emergencies, and a hard stop would be a li-ability in these situations and very time consuming. It would also not be feasible to have safety alerts take place often, as this would be too interrupting and employees could become insensitive to them. This would defeat the purpose of implementing safety alerts.

Acceptance

Even though the beneficiality is largely accepted by respondents, its acceptance is not. Respondents state privacy concerns and fear of the consequences of their results derived from monitoring, as the greatest con-cerns. For example, one respondent worried about the consequences if not enough time would have been spent on a specific task. These two concerns threaten the at-titude of acceptance of the employee. Another threat to acceptance is the doubt of if the implementation of a monitoring system is feasible, however, only one re-spondent was clearly doubtful and in need of convincing before being accepting of a monitoring system. When it comes to safety alerts there are interesting results. Even though safety alerts can be seen as interruptions of their work, most respondents stated that the process of fixing an error is a much longer and difficult process that they would like to avoid. Because of this, respondents were very accepting of safety alerts, as long as they would not be a blockade or hard stops and they would not happen too often. A balance is definitely needed.

Constraints

Throughout the research, some constraints were iden-tified for monitoring and reporting. One was the high threshold that exists for reporting and another the time constraint. Because of the high workload employees do not have time to report and often do this outside of working hours. Also, reports are often too extensive, with too many questions and seemingly irrelevant ques-tions, causing employees to only report when absolutely necessary. Thirdly a majority of respondents stated that it is too difficult to get in touch with IT help when an information system is not functioning. When possible, this takes too much time, goes through too many third

parties, and is not available after working hours. An-other constraint was the rigidness of the information system itself, causing exceptional cases to be a problem. Lastly, the age gap within medical institutions was often stated as a constraint. Younger employees seem to pick up faster on changes within an information system, and technology in general. One respondent specifically stated this was also due to the difference in education between older employees and young employees.

4.2.2 Policies

Beneficiality

According to respondents, policies and protocols specifically should aim to provide clarity to employ-ees while not being too extensive, meaning too many rules, or restrictive. To be beneficial they should not be held too strict to the protocol but have some personal freedom in dealing with healthcare and specifically emer-gencies. Under these terms, it was seen as beneficial due to the clarity it provides. Nearly all respondents did not have satisfying training when they started working but saw the beneficiality of having one. Multiple protocols, however, were seen negatively by respondents, because they would create great confusion while working and as one respondent stated cause chaos on the working floor. Additionally, it would be too time-consuming and pro-tocols could even conflict with each other. This would defeat the purpose of implementing the protocol and not results in a net benefit. All respondents preferred to have one protocol in place, even if this meant it would be less specific. When it comes to being protected from distractions, all respondents saw this as beneficial but stated this was only partially possible. Complete pro-tection from distraction would not be feasible because of the nature of medical work. The employees need to be available to patients and colleagues at all times because unexpected events take place and this is part of their responsibility.

Another measure was having backup methods to transmit information, which all respondents saw as ben-eficial. Multiple of the respondents complained, however, about not receiving a confirmation when transmitting

in-formation the first time, causing them to doubt if it was done correctly. Ensuring information is complete prior to transmitting it would only be beneficial if it would stim-ulate completeness but not completely disallow trans-mitting partial information. As two respondents stated, in some cases, you just do not have the luxury of having all the information and still have to transmit essential information. Additionally, requiring third-party sign offs in certain situations was largely seen as negative. It would make an individual employee too dependent on the third party and could be very time-consuming if the party would not respond swiftly. A respondents that had this measure already implemented within the med-ical institution, stated mistakes still take place due to the communication with the third party, for example, a pharmacist. Another respondent stated it could be beneficial if done digitally in a way that would guarantee speed and a fast response. Lastly limiting or prohibiting copying text was rejected and seen as not beneficial, be-cause it would slow the reporting down significantly. It is a very time-consuming process to retype every impor-tant letter or message while reporting. Obliging source attribution, however, was received positively and seen as beneficial by all. By doing this, the source of the copied text could be retrieved without limiting efficiency too much.

Feasibility

A protocol would only be not feasible in the case that it is too extensive or if employees would be held too strict to the protocol. In that case employees, their lack of acceptance would lead to the protocol to not be used in practice. A too extensive protocol would be time-consuming, because it would need to be consulted often throughout the day, taking time away from essen-tial care. Implementing multiple protocols for different applications due to their difference in safety risk is not feasible in practice because it would be too confusing for an individual employee. Only one respondent saw this as a feasible measure. In practice implementing this would not be possible, within the high-pressure work en-vironment of a medical worker. When it comes to being protected from distraction, respondents stated this was not completely feasible because it is absolutely necessary they must remain available for patients and colleagues

at all times. This is for example important when an emergency takes place. Also, one respondent stated that the feasibility of creating environments that shield from distraction is dependent on the financial capabilities of the medical institution. Disallowing partial transmis-sion of information would be not feasible because you do not have all the information all the time as a health-care worker. The same goes for demanding a minimum amount of words. While this could be helpful, the length of a report is dependent on the situation and therefore differs a lot. Lastly, limiting or prohibiting copying text was seen as not feasible by all, due to the amount of of-ficial letters and messages that need to be retyped while reporting if that measure is implemented. This would be time-consuming to the point of damaging productivity and efficiency when reporting.

Acceptance

The acceptance of protocols is dependent on two terms. One is how strict an employee is held to the proto-col while working. If the employee is held to the protoproto-col too strict, they characterize a protocol as restrictive and difficult. According to the respondents, multiple proto-cols would create more confusion than clarity and not be accepted due to the lack of feasibility. When it comes to creating environments that shield from distractions while using an information system, employees would not have acceptance if this would mean they would be iso-lated from their patients or colleagues. The acceptance of having backup methods for transmitting information is clearly dependent on if the employee is convinced of it improving safety and of how often it happens, and how time-consuming it is. Additionally, the majority of re-spondents did not have acceptance for disallowing trans-mitting partial information, due to them often dealing with lack of availability of information. Requiring third-party sign off in certain situations was not accepted by most, because employees would not want to be dependent on a third party to that extent and some were not con-vinced of its beneficiality. Lastly limiting or prohibiting copying text is completely not accepted by respondents, due to the damage it does to efficiency and productivity on the working floor. This is because a vast amount of official letters and messages would have to be retyped each time if the measure were to be implemented.

4.2.3 Reassesment and redesign

Beneficiality

Continual assessment and redesign are certainly seen as beneficial because the information system iteratively improves as a result of it. As one respondent stated, the medical world is dynamic and changing therefore an in-formation system should grow along with this trend. The beneficiality however is dependent on how often changes are actually implemented, communication about the im-plemented changes, and by what speed the changes get implemented. The respondents want changes to be im-plemented at a pace that is manageable and offers them the required time to adopt these changes. Where one respondent would not want more than one large update a year, another would find one update each six months ap-propriate. It is also important to cluster changes together in large updates so employees can anticipate the arrival of changes, and it does not occur too often with small updates. When it comes to actively involve employees in the reassessment and redesign process, respondents characterize this as key to it’s success, because feedback from end-users is vital to implement correct changes that do not damage efficiency. Also, this contributes to the acceptance of the changes that will be implemented and therefore heightens the chances of success. Ideally, however, the employees should be consulted outside of working hours.

Additionally, modular and user composable design was seen as beneficial by all because a common complaint about information systems was that they are too general and rigid. This means the system does not accommo-date the needs of a specific medical institution. One example is an information system that is unnecessar-ily complex for smaller medical institutions and could be adjusted to be made more minimalistic. One re-quest made by two respondents is that an information system is not adjusted for internal departments, but a medical institution as a whole to ensure communication between departments remains easy and clear. Another measure is observing employees through clinical simula-tions and usability testing which is seen as not beneficial by half the respondents. They state the reason for this is, employees can be expected to behave differently when observed. The other half of respondents however

char-acterize simulations as beneficial, meaning that the way the simulations and tests are organized will determine if they will be successful. Exploring the impacts of changes on the workflow of users is seen as a beneficial measure but time-consuming if done too often. Lastly unambigu-ously wording system messages and labels are seen as beneficial to the point that nearly all respondents would also be willing to commit to consistently using certain terminology in reports if it would improve safety. They believe this wording can achieve clarity and simplicity when dealing with multiple information systems. This would ultimately lower the chances of errors, constitut-ing a benefit for all.

Feasibility

The priorly mentioned age gap within employees is specifically mentioned as a problem when it comes to re-assessment and redesign. Because older employees take longer to pick up how to work with the information sys-tem they are disproportionately affected by changes to the system. If these changes occur too often, it would not be feasible to reassess and redesign a system in practice. Also, employees would need to be informed of the changes that are made. For the sake of feasibil-ity, it would be recommended to spend more time on older employees, or employees that struggle with change. Actively involving staff in the redesign process will be feasible if it occurs outside of working hours, evading the high-pressure work environment within the medical world. Modular and composable design is feasible as long as an information system is not adjusted for each department but a medical institution as a whole. Other-wise, it would be too complex to communicate between departments. Another request for the sake of feasibility was that all information remains accessible even if it is through a different route outside the information system. This is because exceptional cases occur, which an infor-mation system might not be able to accommodate. Also not too many people should be authorized to adjust the information system, therefore organizing this within the medical institution is also of great importance. observ-ing employees through clinical simulations and usability testing is feasible if done in a way that does not burden the employee too much and does not interfere with reg-ular tasks. Lastly exploring the impacts of changes on

the workflow of users is as feasible but time-consuming, leading to the conclusion that it should not be done too often to remain feasible within a high-pressure environ-ment.

Acceptance

Respondents state their acceptance of reassessment and redesign is dependent on how often changes are im-plemented. If it happens too often and goes too fast employees, especially older ones, will not be able to keep up and therefore not have acceptance for the redesign measures. If employees are not correctly informed about each change this will also damage acceptance and cause the employees to not adopt changes. Older employees specifically are at risk of not having acceptance and not adopting change as a result of the redesign, because of its complexity. Even though staff should be actively in-volved in the redesign of the information system, the way in which it is done affects acceptance. One respondent states that a survey would not be sufficient because the high-pressure work environment would lead a lot of staff to possibly ignore it. Also, it should take place outside of working hours to increase collaboration. When it comes to modular and composable design all respondents had an attitude of acceptance and saw value in especially a more minimalist information system. Additionally, ob-serving employees through clinical simulations and us-ability testing is accepted only if it does not interfere with the work that needs to be done and if respect is shown for the privacy of patients. Acceptance is clearly dependent on the approach and the way the simulation is organized. For example, if employees would get feedback on what has been done or achieved with the collected information, it would contribute to acceptance. Explor-ing the impacts of changes on the workflow of users is accepted if it is not done too often. Once a month, for example, would be experienced as a burden. Lastly un-ambiguously wording system messages and labels are ac-cepted because it will provide clarity and simplicity when using multiple systems.

5

Discussion

5.1

Which corrective actions can be

pro-posed to counter technology-induced

medical errors?

The answer to this question, are my collected and ad-justed corrective actions that were the result of my desk research. These corrective actions can be found under Appendix A at the end of this thesis. The actions are adjusted and merged when necessary specifically, for this research. They are clustered in three categories based on their nature. The categories are as follows. Monitoring reporting, policies and reassessment & redesign

5.2

Which types of corrective actions

are feasibly implementable within a

medical context?

Where a certain corrective action might be expected to be feasible in theory, in practice it is actually not. This is due to the unique character of medical work environ-ments being demanding, high pressure, time-consuming, and carrying great responsibility, which leaves little room for error. The type of corrective actions that are feasi-ble have characteristics in common which increase the likelihood of feasibility. These characteristics are as follows. Firstly, corrective actions that are not time consuming. The most scarce resource within medical institutions seems to be time. Respondents for example, overwhelmingly report after working hours on their own time on a consistent basis. A corrective action that is too time-consuming is destined to fail because even if it has moderate success the cost of time as a resource will cause it not to be a net benefit. This can be seen in the model, where the net benefit variable is affected by the resource costs. Secondly, corrective actions that are clearly beneficial and need little convincing. Through-out the research, we have learned that the feasibility of a corrective action is often largely dependent on the acceptance of the measure, by individual medical work-ers. Each corrective action requires a medical worker to perform certain actions, and therefore sacrifice resources like time. Because of time and flexibility constraints, these sacrifices are very costly and not always possible. In the case that a corrective action is clearly beneficial,

the sacrifices will be seen as worth it and this will in-crease the likelihood of a successful implementation.

Thirdly, corrective actions that do not interrupt the fast pace within a medical work environment. Employees of a medical institution are dependent on each other in multi-ple ways. They have to communicate their observations to colleagues because care is given collectively. When one employee is occupied, other employees take over, showing collaboration, solidarity, and collective responsibility for the care of each individual patient. However, this also means that if this fast pace is interrupted it will disrupt the individual medical worker, after which it will disrupt colleagues and lastly the patient. An example of this are the alerts when making errors. The most important aspects of these alerts were that they absolutely could not be hard stops, and should be shown briefly, in order to not disrupt the pace. Fourthly, corrective actions that aren’t too restrictive. The medical working environment is unique due to the possibility of medical emergencies and its chaotic nature as a result of that. Often medi-cal workers will need to improvise and will need to be available around the clock. Restrictive corrective actions directly conflict with this, because they occupy them and are often too specific or compulsory, not allowing them to improvise and adjust. This can lead to disregarding the protocol in its entirety. Lastly, corrective actions that are not too resource-intensive. Medical institutions differ from small to large, and the same goes for their financial capabilities. This was characterized as the biggest prob-lem in healthcare by one respondent. Some corrective actions are simply not feasible for smaller medical insti-tutions and vice versa. Two examples of costly corrective actions are employing IT support that is reachable after working hours and creating environments that shield em-ployees from distractions. The resource costs will cause the corrective action to not be a net benefit to the med-ical organization. Again, as can be seen in the model, where the net benefit variable is affected by the resource costs.

5.3

Will corrective actions conflict with

user acceptance and, if so, why?

As can be seen within the results, corrective actions can conflict with user acceptance in several cases.

Accep-tance however is an important building block, to have a corrective action constitute a net benefit. As seen in the model, without acceptance you can not ensure cor-rect implementation of a corcor-rective action, as it is likely to be disregarded and therefore fail. Firstly, corrective actions conflict with acceptance, when medical workers are aware or even intimidated by the consequences of the corrective action. A large part of corrective actions collect data to find flaws and errors, which ideally will be passed on to the people organizing the medical in-stitution. Because of the hierarchical structure within medical institutions employees are afraid of not doing their work correctly, and flaws being intercepted by their superiors. This can lead to two scenarios. Employees will disregard the corrective action and actively work against its implementation or, more interestingly, employees will be fully aware of the corrective action and behave dif-ferently to show the desired behavior. If users behave differently only when data is collected, it will ultimately make the corrective action useless and defeat its pur-pose. Secondly, the results show us that acceptance is highest when the perceived benefit is clear. For example, employees were very accepting of corrective actions that could prevent them from landing in a longer process like preventing an error, even if the costs of the corrective action were high. Thirdly, user acceptance conflicts with corrective actions that are perceived to be restrictive or too complex. The restrictiveness conflicts with the medical worker’s ability to improvise and the complexity conflicts with the limited time employees have to act on a corrective action. Older employees especially will not be able to keep up, when there is too much digital complexity.

Another case is when a corrective action that collects data, for example by observing, conflicts with privacy. Medical workers worry about the privacy of the patients dependent on them and their own privacy. This can how-ever be solved by making clear agreements beforehand, to ensure no boundaries will be overstepped. Fourthly, medical workers were not accepting of corrective ac-tions that would introduce a dependency. Dependencies conflict with acceptance. Medical workers are highly independent and carry great responsibility. They impro-vise due to emergencies and being understaffed, meaning at times they are even providing care alone. Because

of these reasons they do not want to be dependent, as that introduces an extra liability within their workflow. For example, one corrective action introduced a third-party sign-off. By most respondents, this was seen as possibly more time-consuming and a source of confusion instead of helpful. Finally, corrective actions that are cumbersome, interfering, or constitute a shrinkage of efficiency conflict with acceptance. Efficiency is a top priority, therefore the medical worker needs to take the shortest route to a result. An example is updating the information system too often. Each update has a learn-ing curve, so if updates occur too often the acceptance will drop because employees can not efficiently use the updated system for a while. A corrective action will also conflict with user acceptance when there is not a sufficient amount of communication. If an update is im-plemented without informing staff, their acceptance will lower because of the unexpected disruption of their work processes.

5.4

Which types of corrective actions

of-fer a net benefit to the organizations

that adopt them?

A corrective action offers a net benefit to the organi-zation that adopts them when the action mitigates an unintended consequence at a reasonable consumption of resources. This is of great importance in the medi-cal field because every resource is scarce. The types of corrective actions that constitute a net benefit are as follows. Firstly, corrective actions that are beneficial but are not extremely consuming. If an action is time-consuming, that limits the time that can be spent on one-to-one care. Even though unintended consequences may be mitigated, there is a shortage of time as a re-sult of the action. This increases the likelihood of other errors being made because employees need to rush in an already high pressure and understaffed work environ-ment. As a result, extremely time-consuming corrective actions are not a net benefit, while efficient ones are. Ex-amples of the time constraint within the medical field are respondents regularly doing their reporting after work hours and often working on autopilot as they stated. Secondly, corrective actions that are beneficial but are not too costly. Medical institutions have limited finan-cial funds. If an action is too costly, this will mean less

budget for tasks like general care and staff. As a result, the implementation of the corrective action will be at the expense of healthcare and increase the likelihood of a medical error occurring. An example of costly actions is employing IT support with 24/7 availability.

Thirdly, corrective actions that are preventive. As multi-ple respondents have clarified, one of the worst aspects of making an error is correcting it after hand. This can be an extremely costly process because it is time-intensive and interferes with scheduled tasks. Also, employees are often dependent on contacting third parties to con-tribute to fixing the error, which according to them can be unreliable and difficult to reach. If a health informa-tion system for example sends an alert about an error at a late stage, the medical worker can not avoid the costly process of fixing the error. However, if alerts are transmitted in an early stage with the aim of pre-venting larger errors, it is extremely beneficial and can save resources. These preventive corrective actions con-stitute a net benefit. Fourthly, corrective actions that mitigate unintended consequences by adding simplicity instead of adding information. An example of this is the corrective action that creates consistency in system messages and labels to make sure they are unambigu-ously worded. Several respondents complained about a high threshold to report, especially when it’s about adverse events. A corrective action that creates simplic-ity like the one mentioned prior, lowers that threshold and constitutes a net benefit. Corrective actions that provide more information can also mitigate, but in doing so add to the further complexity within the institution and possible information overload. Finally, corrective actions that are iterative. Respondents stated they are often occupied and therefore it is difficult for them to deal with large changes or make great efforts. A solution for this is distributing changes iteratively, so each dose of change is manageable for the individuals and grants them the required time needed for adoption. A balance should be found in this because a lot of small changes are also not received well by respondents and can dam-age the acceptance. This means even though changes can be distributed iteratively it should not result in an overload of small changes but should be balanced and spread out updates. An example of this is implementing changes to the information system, iteratively as a result

of reassessment and redesign.

6

Reflection

During the process of writing a thesis, conflicts and com-plexities arise. As a result of this creative decisions need to be made, for the better of the research. Within this section, some of these decisions will be explored. One is-sue that arose during the desk research was the fact that no exhaustive list of corrective actions seems to exist. Corrective actions are scattered around many sources, and also not formulated in the same consistent manner. Two corrective actions can be formulated differently but aim to mitigate the same unintended consequence in the same manner. For these reasons, this research made its own list of corrective actions, which lies at the base of the field research. This was done by firstly collecting all possible corrective actions intended to mitigate un-intended consequences within the scope of the research, and therefore its medical context. Similar corrective ac-tions were merged, or partially formulated differently to create a comprehensive list of unique corrective actions. This allowed the research to not only have a clear desk research, but also increase the effectiveness of the inter-views by guaranteeing each presented action would not be confused with another one and invoke a new response from the respondent.

Secondly, within this research sub-questions are delib-erately formulated in a comprehensive manner to ensure they are also of value individually and not only collec-tively. Each research question deals with one property, like feasibility, acceptance, or beneficiality. Within the result section, a vast amount of information is provided around each one of these properties. Because of the com-prehensive nature of this research, this provided infor-mation can not only be used as an answer to the main research question of the thesis but also exist separately. If readers would strictly need to inform themself of con-flicts that arise with user acceptance, or concon-flicts that arise with feasibility they could do so, by strictly focus-ing on those properties within the results and discussion sections. I am certain that a large part of the results of this research would also be applicable to other high-pressure work environments, that deal with information systems. This means that some of the corrective actions would be effective as well. Even fields that seem

com-pletely different at first glance, like banking and trading, come to mind. This is because their employees are also under pressure, have to make fast decisions, and interact with information systems to do their work.

Thirdly, within this research, the decision was made to focus largely on the practical implications of correc-tive actions as opposed to the theoretical implications. This is because throughout the literature a large part of the sources seemed to be very positive, leading to a lack of a critical view of unintended consequences in real-world use cases. This turned out to be a great decision as results have shown that there is a certain discrepancy between real-world use cases and some of the prior the-oretical research that revolves around corrective actions. Therefore the worth of first-hand experiences has been proven and further research should definitely focus on practical experiences within medical institutions, possi-bly even creating metrics to quantitatively measure the effects of unintended consequences.

7

Conclusion

Wide adoption of information systems in the medical field has, in some contexts, added further complex-ity to already high-pressure work environments. This added complexity can effectively lead to unintended con-sequences, which contribute to medical errors. To com-bat this trend, corrective actions were designed to miti-gate these new unintended consequences of information systems in the medical field. More importantly, how-ever, these corrective actions can not be implemented because they have not been challenged on their feasibil-ity, acceptance, and beneficiality in real-world contexts. These three dimensions are key in adding actual value within the medical field while minimizing and mitigat-ing the currently existmitigat-ing negative consequences. This research, therefore, aimed to offer willing institutions in-formation on which types of corrective actions to apply, and in which manner.

More concretely, it set out to answer the question of in which manner health information systems should be im-plemented to contribute to the prevention of technology-induced medical errors. This was done by determining if corrective actions would be effective in improving a health information system by investigating their benefi-ciality and investigating if, when applied to health

in-formation systems, they were feasible within a medical context, accepted, and clarifying why.

In terms of feasibility of corrective actions the re-search found this was dependent on time efficiency, ac-ceptance, not interrupting workflows, flexibility, and re-source consumption. When it came to acceptance it found out it was dependent on hierarchy, perceived ben-eficiality, flexibility, complexity, privacy, fear of depen-dencies, efficiency, and communication. In terms of ben-eficiality the research found this was dependent on time consumption, financial costs, being preventive, creating simplicity, and iterative change.

As a result of this research, it is also clear which cor-rective actions can be proposed. It clarified which types of corrective actions are feasibly implementable. It went on to show arising conflicts between corrective actions and user acceptance while interpreting them and

provid-ing an explanation. Lastly, it finished with classifyprovid-ing types of corrective actions that offer a net benefit to the institutions that implement or adopt them.

7.1

Future work

Finally, multiple topics within this research could be rec-ommended for further research. Firstly research should be done on the effect of consumption of resources by cor-rective actions and the consequences of that. Also, addi-tional research should be done on new unintended con-sequences that will arise due to over-regulation or imple-mentation of too many corrective actions. This research should be done through on-site observation in the work-place within medical contexts. Through this method, re-searchers could try to actually measure negative impacts on the work floor as a result of implemented corrective actions, which will lead to new insights.

References

Adler-milstein, J., & Pfeifer, E. (2017). Information blocking: is it occurring and what policy strategies can address it? The Milbank Quarterly, 95 (1), 117–135.

Alert, S. E., Commission, J., et al. (2008). Safely implementing health information and converging technologies. Sentinel Event Alert (42).

Ash, J. S., Berg, M., & Coiera, E. (2004). Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. Journal of the American Medical Informatics Association, 11 (2), 104–112.

Association, A. M., et al. (2014). Improving care: priorities to improve electronic health record usability. 2014.

Balgrosky, J. A. (2019). Understanding health information systems for the health professions. Jones & Bartlett Learning.

Bernat, J. L. (2013). Ethical and quality pitfalls in electronic health records. Neurology, 80 (11), 1057–1061.

Borycki, E. (2013). Trends in health information technology safety: from technology-induced errors to current approaches for ensuring technology safety. Healthcare informatics research, 19 (2), 69–78.

Borycki, E., Dexheimer, J., Cossio, C. H. L., Gong, Y., Jensen, S., Kaipio, J., . . . others (2016). Methods for addressing technology-induced errors: the current state. Yearbook of medical informatics, 25 (01), 30–40.

Borycki, E., & Kushniruk, A. (2010). Towards an integrative cognitive-socio-technical approach in health infor-matics: analyzing technology-induced error involving health information systems to improve patient safety. The open medical informatics journal , 4 , 181.

Bowman, S. (2013). Impact of electronic health record systems on information integrity: quality and safety implications. Perspectives in health information management , 10 (Fall).

Campanella, P., Lovato, E., Marone, C., Fallacara, L., Mancuso, A., Ricciardi, W., & Specchia, M. L. (2016). The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. The European Journal of Public Health, 26 (1), 60–64.

Coiera, E., Ash, J., & Berg, M. (2016). The unintended consequences of health information technology revisited. Yearbook of medical informatics, 25 (01), 163–169.

Fox, J., & Thomson, R. (2002). Clinical decision support systems: a discussion of quality, safety and legal liability issues. In Proceedings of the amia symposium (p. 265).

Hammond, K. W., Helbig, S. T., Benson, C. C., & Brathwaite-Sketoe, B. M. (2003). Are electronic medical records trustworthy? observations on copying, pasting and duplication. In Amia annual symposium proceedings (Vol. 2003, p. 269).

Hoffman, S. (2016). Electronic health records and medical big data (Vol. 32). Cambridge University Press.

Iso, E. (2005). 9000: 2005. Quality management systems-Fundamentals and vocabulary (ISO 9000: 2005), 1 .

Jones, S. S., Koppel, R., Ridgely, M. S., Palen, T. E., Wu, S.-Y., & Harrison, M. I. (2011). Guide to reducing unintended consequences of electronic health records.

Kannampallil, T., Abraham, J., Lou, S. S., & Payne, P. R. (2020). Conceptual considerations for using ehr-based activity logs to measure clinician burnout and its effects. Journal of the American Medical Informatics Association.

Kroth, P. J., Morioka-Douglas, N., Veres, S., Babbott, S., Poplau, S., Qeadan, F., . . . Linzer, M. (2019). Association of electronic health record design and use factors with clinician stress and burnout. JAMA Network Open, 2 (8), e199609–e199609.

Kruse, C. S., Kristof, C., Jones, B., Mitchell, E., & Martinez, A. (2016). Barriers to electronic health record adoption: a systematic literature review. Journal of medical systems, 40 (12), 1–7.

Mangalmurti, S. S., Murtagh, L., & Mello, M. M. (2011). Medical malpractice liability in the age of electronic health records. Survey of Anesthesiology, 55 (6), 317–319.

Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014). An analysis of electronic health record-related patient safety concerns. Journal of the American Medical Informatics Association, 21 (6), 1053–1059.

Moy, A. J., Schwartz, J. M., Chen, R., Sadri, S., Lucas, E., Cato, K. D., & Rossetti, S. C. (2021). Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. Journal of the American Medical Informatics Association.

Murphy, D. R., Reis, B., Sittig, D. F., & Singh, H. (2012). Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. The American journal of medicine, 125 (2), 209–e1.

Nguyen, O. T., Shah, S., Gartland, A. J., Parekh, A., Turner, K., Feldman, S. S., & Merlo, L. J. (2020). Factors associated with nurse well-being in relation to electronic health record use: A systematic review. Journal of the American Medical Informatics Association.

of Medicine (US). Committee on Patient Safety, I., & Technology, H. I. (2012). Health it and patient safety: Building safer systems for better care. National Academies Press.

Siegler, E. L., & Adelman, R. (2009). Copy and paste: a remediable hazard of electronic health records. The American journal of medicine, 122 (6), 495–496.

Singh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., & Sittig, D. F. (2013). Information overload and missed test results in electronic health record–based settings. JAMA internal medicine, 173 (8), 702–704.

Singh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., Smith, M. W., Murphy, D. R., . . . Sittig, D. F. (2013). Primary care practitioners’ views on test result management in ehr-enabled health systems: a national survey. Journal of the American Medical Informatics Association, 20 (4), 727–735.

Sittig, D. F., Wright, A., Ash, J., & Singh, H. (2016). New unintended adverse consequences of electronic health records. Yearbook of medical informatics, 25 (01), 7–12.

Street Jr, R. L., Liu, L., Farber, N. J., Chen, Y., Calvitti, A., Zuest, D., . . . others (2014). Provider interaction with the electronic health record: the effects on patient-centered communication in medical encounters. Patient education and counseling, 96 (3), 315–319.

Upadhyay, D. K., Sittig, D. F., & Singh, H. (2014). Ebola us patient zero: lessons on misdiagnosis and effective use of electronic health records. Diagnosis, 1 (4), 283–287.