• No results found

The Combined Effect of Artificial Intelligence and Electronic Health Records on Hospitals: A Systematic Literature Review

N/A
N/A
Protected

Academic year: 2021

Share "The Combined Effect of Artificial Intelligence and Electronic Health Records on Hospitals: A Systematic Literature Review"

Copied!
47
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

Master Thesis

The Combined Effect of Artificial Intelligence and

Electronic Health Records on Hospitals:

A Systematic Literature Review

Wesley Aalst S3711005

Supervisor: dr. E. Smailhodzic Co-assessor: N. Renting

Word count: 9303

MSc Business Administration: Change Management Faculty of Economics and Business

(2)

Abstract

Methods I conducted a systematic literature review on empirical research of the combined

effect of AI and EHR in hospitals. The following criteria were used to select articles: 1) published in a peer-reviewed journal, 2) written in English, 3) the full text must be accessible to the researcher, 4) contain primary empirical data, 5) a form of AI technology must be clearly linked to an HER, 6) the combined effect on hospitals must be indirectly or directly demonstrable, 7) satisfy the established standard quality criteria.

Results an initial database of 717 articles was used in the selection process. In total 30 articles

were included in this review. Patientcare is affected during diagnosing, risk predication and decision-making. Healthcare professionals are also affected, since they show resistance to new technology, need to train themselves to use the technology and they need to adopt new work processes. Finally, also indirect effects are described. Those effects show an increase and decrease of efficiency and positive effects on usability and privacy.

Conclusions This review provides an overview of effects on hospitals. In particular the

(3)

Introduction

Healthcare has been subject to significant Information Technology (IT) changes (Chen et al., 2020). In the early days, discoveries were made to solve the plaque, after this modern medicine was born, and in more recent years, Electronic Healthcare Records (EHR) made its upcoming. EHR is the IT that makes storing patient data possible (European Commission, 2013). Another influential invention is Artificial Intelligence (AI), described by John McCarthy in 1955 as a form of science and engineering that creates intelligent machines. In addition, Stephen Hawking (2016) stated that this could be the best or the worst thing to happen to humanity. These intelligent machines make use of large databases like EHR to function. The research by Dawes et al. (2016) gives an example of what is possible with the combination of these technologies. They showed that a photo-based AI technology, designed for cardiology, can make accurate predictions on patient health based on data. These relatively new possibilities seem to have the potential to make a considerable impact on processes in hospitals (Neill, 2013).

Jacob (2020) describes EHR as a system with a broad set of functions, including documentation of clinical notes, reviewing results, administering medical procedures, creating data-driven alerts and clinical decision support tools. Garret and Seidman (2011) describe that EHR evolved from Electronic Medical Records (EMR). EMR is no more than a digital version of the paper charts in clinicians’ offices. EMR had many usability and scalability problems. By way of contrast, EHR does all the things EMR does and more by focusing on the patient's total health. These improvements are possible by making better use of data. Likewise, Goldstein, Navar, Pencina, and Ioannidis (2017) argued that EHR could easily operate with many different types of data, such as free-text notes from doctors. Hence, EHR is an improvement to the traditional data analyzing techniques within healthcare (Keyhani et al., 2008).

(4)

it more complicated. Taken together, this indicates the complexity and the challenge of implementing EHR within hospitals.

The excess of data makes EHR ideal for the use of AI techniques to uncover fundamental and new patterns. Nowadays, AI is seen as a branch of engineering that develops new concepts and new solutions to meet complex challenges. According to He, Zhang, Ren, and Sun (2016), this technology can easily contain up to hundreds of layers, millions of neurons, and complex structures of connections between them. As a result, this technology can combine multiple data sources to create accurate insights into complex databases from various sectors and institutions. Hospitals are one of those institutions in which AI is used increasingly. An example is the prediction of mortality of patient with heart failures (Kwon et al., 2019). Another article of Watson, Womack, and Papadakos (2020) show how AI could reduce the workload of nurses. As a result, medical schools increasingly include AI in their study programs to prepare healthcare professionals in applying this technology (Brouillette, 2020). Therefore, the role of AI in hospitals will only increase in the next two decades (Chen, Loh, Kuo, & Tam., 2020).

In line with these trends, there has been an increasing number of studies on the combined use of AI and EHR in hospitals (Kwon et al., 2019; Watson et al., 2020; Brouillette, 2020; Chen et al., 2020). Although many studies contain information about the effects, there still lacks a clear overview of studies on the combined effects that these two technologies have on hospitals. To address this gap in the literature, this study aims to answer the following research question:

"How does the combined use of Artificial Intelligence and Electronic Health Records affect

hospitals?"

(5)

categorizing, and analyzing the present findings on the impact of AI in conjunction with EHR within hospitals. It will not only contribute to the healthcare management literature but also other streams of literature such as change management studies by the focus on change in healthcare processes caused by AI. The systematic literature review also provides practical insights for the practitioners who can use the overview to better understand the new possibilities and their position in this change process.

(6)

Methodology

I conducted a systematic literature review to provide an overview of the combined effect of AI and EHR on hospitals. A systematic literature review is a method in which articles are reviewed and assessed while using pre-specified and standardized techniques. Consequently, the review becomes less biased and more rigorous (Tranfield et al., 2003). The pre-specified method used in this literature review is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyzes (PRISMA) guidelines. The PRISMA guidelines consist of 27-item recommendations with which the systematic literature review must comply. PRISMA guidelines are used to guarantee clarity and transparency in reporting. In addition, PRISMA guidelines facilitate a structure of the literature review. Following the good practice of earlier systematic literature reviews in this field (Radcliffe, Lyson, Barr-Walker, & Sarkar, 2019; Mantelakis & Khajuria, 2020) I used Web of Science (WOS) which is the world's leading scientific citation search and analytical information platform since it contains and provides insights in many academic fields of research (Li, Rollins, & Yan, 2018). Moreover, The WOS is one of the oldest databases that contains 34,000 selected and structured journals from databases such as Medline and SciELO Citation Index (Birkle, Pendlebury, Schnell, & Adams, 2020).

Selection criteria

For the search terms, I focused on three critical terms, namely EHR, AI and hospitals. For the search terms "EHR" and "hospitals", I relied on the good practice from an earlier study of Boonstra, Versluis, & Vos (2014). In their study, all the relevant synonyms and related search terms of "EHR" and "hospitals" are used. In addition, I supplemented the search terms from Boonstra et al. (2014) with abbreviations in order to be more comprehensive and ensure the search results' completeness. The search terms used for “EHR” are: "EHR*" or "Electronic Health Record*" or "EMR*" or" Electronic medical record*" or "Electronic patient record*" or "Computerized patient record*" or "CPR*" or "Personal health record*".

(7)

computational intelligence (Dunjko, 2018; Wolff, 2020). Since I focused on the combined use of AI and EHR in hospitals I decided to only use the most trending search terms of "AI". Following the good practice of Wolff (2020) I used Google trends to confirm these terms were most trending. I supplemented the relevant search terms of Wolff (2020) with two other trending AI search terms to extent the database with more AI technologies. Appendix A shows the Google Trends results (Figure 1) and a comparison of the not used search terms (Figure 2) between January 1st, 2004, until April 1st, 2020. The terms "artificial intelligence", "machine learning", "deep learning", "artificial neural network", and "genetic algorithm" were by far the most trending in this period. This resulted in a search string with comprehensive and relevant search results. The search terms used for AI are: "Artificial intelligence" or "Artificial neural network" or "Deep learning" or "Genetic algorithm" or "Machine learning". I included the whole search string, as stated and used in WOS from April 1st, 2020, until May 15th, 2020, in Appendix B.

Articles that met the following requirements were included in the review: 1) published in a peer-reviewed journal, 2) written in English, 3) the full text must be accessible to the researcher, 4) contain primary empirical data, 5) a form of AI technology must be clearly linked to an EHR, 6) the combined effect on hospitals must be indirectly or directly demonstrable, 7) satisfy the established standard quality criteria (Kmet, Lee, & Cook ,2004). This means that articles were only included if they met certain quality conditions according to the quality assessment of Kmet et al. (2004). This quality assessment is included in Appendix C.

(8)

Figure 3. Flowchart

Data analysis

(9)

Results

Search results

The research started on April 1st, 2020 and lasted until May 15th, 2020. The number of articles, based on the search string, in the WOS, was increasing over time. Therefore, the data collection included the articles that were published up to May 15th, 2020. On April 1st, 2020, the search string found 649 English written articles. On May 15th, 2020, the same search string found 717 English written articles. This increased amount can be explained by the rapid development and research on AI and EHR. All articles between April 1st and May 15th were included in the first screening phase. Because the Web of Science was used, there was no need to remove duplicates.

The 717 articles that were left were screened by title and abstract. Articles were eliminated from the study if data was not relevant. In the end, 577 articles were excluded, and 140 articles remained that required further study. When the relevance of an article was unclear then this article was reviewed in full detail. Articles were removed from the selection when there was no clear effect on hospitals or when it appeared that the form of AI had no connection with any type of EHR. In this phase, another 110 articles were excluded. Next to reviewing articles in full detail, I also used a snowballing technique on the article of Li, Carrell, Aberdeen, Hirschman, and Malin (2014) to get a better understanding of their study. This resulted in one more additional article that was included in the study. Overall this resulted in a list of 30 articles that are included in this review.

(10)

Descriptive results

In order to illustrate some basic information about the articles, I first present the descriptive results, which are followed by the results on the effects of the combined use of AI and EHR on hospitals. The descriptive results provide a better understanding of the selected articles in Appendix D by providing graphical insights.

As described in the introduction and methodology, this field of research has been increasing in popularity over the past two years. This is in line with Figure 4, which shows that since 2018 there have been growing number of articles that focused on AI combined with EHR. Half of the articles are published in 2019. This represents 50% of the selected articles. Five of the selected articles origin form 2020. This lower amount can be explained due to the time when this study was conducted.

Figure 4. Number of articles between 2013 and 2020

Within the selected articles there is a clear distinction in methods. All studies have a quantitative method and two of these studies use a qualitative extension. In total 28 out of 30 articles are based on a quantitative research method and two articles used a mixed method. This simplified distribution can be seen in Figure 5.

Figure 5. Distribution of type of method 0 2 4 6 8 10 12 14 16 2013 2014 2016 2018 2019 2020 0 5 10 15 20 25 30

(11)

The articles in this study are divided over different types of journals. From the selected articles 28% have been published in Medical Informatics journals, followed by 21% in Healthcare Science journals. Also, 13% of the publications are published in Computer Science Information journals. Besides, there are publications in specialized medical journals, such as surgery and medicine. Another number of articles have been published in journals that are focused on information science. In Figure 6 this distribution is shown.

Figure 6. Distribution of journal types

In these journals, research has been done into the different forms of AI. Figure 7 shows that 20 studies are based on machine learning, six on deep learning, two on artificial neural network and two describe AI in general. Thereby machine learning is most trending in the selected database of articles. These results can be explained by the many different forms of machine learning that have been studied in the past years.

Figure 7. Distribution of AI forms

The different types of AI can be found articles containing various medical topics. This distribution is illustrated in Figure 8. Two studies had a more general approach to healthcare since these articles show effects within more than one medical topic. In total nine articles do not have a specific medical topic. These articles do contain research on patient documents and are not especially connected with a medical topic.

(12)

Figure 8. Distribution of medical topics

Effects on hospitals

In this section, the most important findings of the selected articles are discussed. All of these findings contain effects from the combined use of AI and EHR. During the study, it quickly became apparent that there are direct effects on patient care and the management of healthcare professionals. Another stream of the impact, provoked by the combined use of AI and EHR, covers the indirect effects on hospitals. These effects include efficiency, usability and privacy. In the first part of the results, the effects on patient care and management are analyzed. In the second part, the impact on the management of the healthcare professional is shown. In the third part, all the indirect effects are described.

1. Patient management

Patient management in this chapter is limited to the literature that describes the influences that AI, in combination with EHR, has on the way the patient is treated in hospitals, clinics, and comparable healthcare environments. The procedure that patients go through is influenced by the application of AI in conjunction with EHR on various ways. Medical problems can be diagnosed and monitored through the combination of AI and EHR. Also, risk predictions, that

influence therapy decision-making and clinical planning, is affected. Thereby, effects occur in treatment methods, such as prescribing certain medicines or performing surgical procedures. Finally, the risk of mortality can be estimated. When patients go through these different phases, a simplified model of the patient journey is formed. This journey is illustrated in Figure 9. For this I used the good example from the study of Johnson et al. (2012) and made extensions to

0 1 2 3 4 5 6 7 8 9 10

(13)

show the role of AI in combination with EHR. The phases in which the combined use of AI and EHR has effects on patient management are marked in red.

Figure 9. Simplified patient journey

1.1 Diagnosing

Various studies show that diagnosing diseases and medical abnormalities can be done faster and with more accuracy by using AI technologies combined with EHR. In the study by Kate, Pearce, Mazumdar, and Nilakantan (2020) machine learning is used to make a new diagnosis with every data change in the EHR system. Thereby, a new, potentially serious diagnosis is observed significantly earlier. Another effect is described by Tao et al. (2020). According to them, AI based tools (operating on EHR) have clinical benefits for the accuracy of diagnosing and shortening confirmed diagnosis times. While the above studies make use of more complicated structured and unstructured data, the study by Zhao et al. (2020) demonstrated that AI is also excellent to explore medical issues by using only structured data. They show that AI is able to detect skin abnormalities in clinical photos using EHR. This results in a significantly faster diagnosis of diseases. Discovering a medical problem in time is a start but can take several turns during admission. Therefore, it is essential to monitor EHR systems.

1.2 Monitoring

(14)

trauma, AI accurately determined which patient should be monitored in a clinical setting (Hale et al., 2019). The case study of Kate et al. (2020) takes this a step further and shows the possibility of continually monitoring patients. They showed that with every change in EHR, AI could make a new prediction of an adverse medical event. According to Kate el al. (2020), the importance of continually monitoring is caused by the possibility of a significant change in the patient medical status in only a couple of days. Continuous monitoring allows healthcare professionals to make real-time therapeutic interventions. Hale et al. (2019) added to this that real-time data-driven updates in the hands of healthcare providers lead to the most accurate evidence-based care.

Another effect on monitoring described by Hale et al. (2019) is the possibility to provide remote help to inexperienced healthcare providers. These healthcare providers work in remote areas. By AI based monitoring inexperienced healthcare provider could act upon changes in the health status of a patient more adequate and accurate. The accuracy of this feature is also described by Miotto et al. (2016). They describe how clinicians can monitor patients and check if diseases are likely to occur based on the data in EHR. This research is interesting because next to constant monitoring, Miotto et al. (2016) describe the possibility of making risk predictions with their deep learning technology.

1.3 Risk prediction

By using different types of data from EHR, several articles show success in making risk predictions about specific medical conditions with the use of AI. Miotto et al. (2016) describe how their deep learning model can predict the development of certain medical conditions through different types of data (medication, diagnoses, procedures, lab tests and demography). Risk on certain cancers, schizophrenia, and diabetes is predicted in this case. In the above example they used structured data. This is in contrast to the research by Du et al. (2018) in which a form of machine learning is used to make valuable predictions from unstructured data. They consider that nursing notes that include "local concerns" by nurses can be leveraged to capture "general concerns". The machine learning algorithm converts the notes from EHR into phrases of data that allow another AI-technology to predict the risk and severity of disease in real-time (Du et al. 2018).

(15)

the AI technology only depends on raw text from EHR. The evidence presented in the study of Feller, Zucker, Yin, Gordon, and Elhadad (2018) supports this idea. They state that their machine learning tool can also be applied in other areas like virology by showing the ability to predict the risk of having HIV in real-time.

A contradiction that Feller et al. (2018) found that did not recur in the other studies was the accuracy of medical predictions. They concluded that their machine learning model made mistakes with disastrous outcomes in diagnosing. Despite this negative outcome, Feller et al. (2018) are positive about the identification of medical risk. The question now remains how these predictions can be reflected in the decision-making within hospitals.

1.4 AI-based decision-making

The selection of articles shows that EHR and AI have a joint influence on the way decisions are made during patient care. In the past years, many Clinical Decision Support Systems (CDSS) have emerged that are based on AI and expert systems such as EHR (Tao et al., 2020). According to Hu, Bajracharya, and Yu et al. (2018), these systems are a necessity since diseases are becoming increasingly complex to understand while physicians are specializing their profession and focus less on the healthcare in a broader sense. The name ‘CDSS’ implies that these tools are made to support healthcare professionals in making decisions and that this technology does not make decisions autonomously. The selected articles contain insights in decision-making in medicine. In this section adistinction between drug prescription and surgery is made.

1.4.1 Decision-making in medicine

(16)

shows "alerts" when wrong or dangerous medicine is described to a patient. Also, Wunnava et al. (2019) describe that adverse drug events are important for effective pharmacovigilance. In their case, AI is used on patients' notes who shared relevant data on drug use to discover valuable new data about adverse drug reactions. The AI technology paraphrases this valuable data and include this in new medicine research (Wunnava et al, 2019). By this effect, negative side-effects become better administrated in EHR. Eventually, this gives healthcare professionals the possibility to include these insights while considering a specific drug prescription.

Miotto, Li, Kidd, and Dudley (2016) went a step further and described the possibility of personalized prescriptions and treatment recommendations through the combined use of AI and EHR. This is also stated by Reddy, Delen, and Agrawal (2018). The AI-based decision support system in their research made real-time predictions about inflammation values in patients more accurate. Healthcare professionals were able to improve the patient's condition using AI-based tools through early tailor-made interventions and therapeutic adjustments in medicine. According to Reddy et al. (2018), this effect is the first step towards personalized or precision medicine. In the case that medicine is not sufficient for patient treatment, another medical intervention can be chosen, namely, surgery.

The use AI in the context of surgery and using EHR databases provides various effects. A first effect gives insight before the surgery has taken place. It concerns the prediction of last moment cancellation of surgery and the loss of preparation time and resources (Liu, Ni, Zhang, & Pratap, 2019). The last mentioned research described that machine learning allows assessment of cancellation risk by uncovering patterns in historical data from EHR. The research of Tuwatananurak et al. (2019) shows agreement on this by showing the possibility of a time-saving effect. By the use of AI in combination with EHR, it was possible to calculate the required time in an operating room with high accuracy. Therefore, more efficient use of the operating room became possible.

(17)

models used within hospitals. Additionally, Bihorac et al. (2019) claim that predicting surgery is complicated. However, their machine learning algorithm can predict the risks of future complications (including mortality) after surgery.

1.5 Managing patient mortality

Ultimately, it is essential that the healthcare professional's decision ensures that the living situation for patients improves and that possible health damage is kept to a minimum. The study by Parikh et al. (2019) shows how machine learning in combination with EHR provides the opportunity to identify high mortality risks within a short period of time. A third study by Samad et al. (2019) also includes risk indications about the short-term mortality of heart patients. They used data from echocardiography and combined this with the clinical data from EHR systems. In addition to Parikh et al. (2019), they emphasize the superior precision over standard linear regression models that typically predict mortality. AI, in combination with EHR, provides valuable data about the course of a patient's life. Wang et al. (2019) explain how the prediction of mortality could be used as a proxy for selecting patients who benefit from particular forms of care like palliative care. Foreseeing premature death in cancer patients is an insight that leads to a bad news conversation. Earlier interventions and conversations with these patients about the course of their illness is a possibility that would only become possible by using AI on EHR.

(18)

Patient management Effect Article no.

Diagnosing Faster diagnosis [27] [26]

More accurate diagnosis [28] Monitoring Decision for in hospital

monitoring

[11] Continuous monitoring [27] [3] Remote assistance [11] Risk prediction Predicting risk medical

issues

[8] Discovering concerns [3] [9] Decision-making Improve complex

decision-making [28] [4] Improve decision-making in medicine [12] [23] [3] [22] Improve decision-making in surgery [13] [21] [14] [19] [15] Managing patient mortality Faster identification of

mortality risk

[16] [5] Earlier interventions and

(19)

2. Management of healthcare professionals

Next to the effect on patient care, there is also an effect on the healthcare professionals themselves. The included articles show various effects of the combination of AI and EHR on healthcare professionals within hospitals. Some of the described effects concern the replacement of healthcare professionals, resistance by healthcare providers and, doctor-patient relationship.

2.1 Replacement of healthcare professionals

The included articles do not advocate for complete replacement of healthcare professionals. However, some articles show that AI, in combination with EHR, can be a valuable extension. In the study by Hill et al. (2019), the current way of clinical risk scores was compared with the risk scores that machine learning could predict. Despite the speed that machine learning entails, Hill et al. (2019) state that the best score is achieved in combination with a healthcare professional's judgment. Hale et al. (2019) support this statement and describe how synergy arises between the hospital administrators (who manage and edit the EHR) and the clinical teams (healthcare professionals). Because of this synergy, the AI-tool provides faster predictions, enabling clinical teams to perform medical interventions faster. According to Hale et al. (2019) this is the way to provide the highest quality of clinical care. In short, according to the literature, a combination of the medical skills of the healthcare professional and AI will result in the best medical solution.

An argument why the technology is used as an extension concerns verification of correctness. AI statements should be reviewed by healthcare professionals (Tao et al., 2020). The research of Tao et al. (2020) shows that machine learning achieves good results in making diagnoses and using these diagnoses in CDSS. The last mentioned researchers also explain that these AI systems have good diagnostic accuracy, but are in need of verification by healthcare professionals before this technology can be applied in a clinical context. Therefore, a healthcare professional does not become abundant.

(20)

AI-enabled systems which can deal with all kinds of medical problems the way thathealthcare professionals can (Wu, Liu, Zhang, He, and Lv, 2018).

2.2 Need for training and education

The combined use of AI and EHR in a hospital environment has complex aspects. By this complexity a side effect regarding the education of healthcare professionals occurs. Different studies show the necessity for training and educating healthcare professionals to better understand the AI-tool in clinical care. There is a fear of doctors not being trained and skilled enough to use this technology in future care (Williams, Mekhail, Williams, Mccord, & Buchan, 2019). Research by Tao et al. (2020) also advocates training, causing a correct adoption of AI-based CDSS. Tao et al. (2020) explain that there is no standard restriction to use AI-AI-based technology. Therefore, training healthcare professionals is required before AI-based CDSS could be adopted in clinical care. The concerns are not without any reasoning since various articles show forms of resistance from healthcare professionals in the implementation of AI-techniques.

2.3 Resistance

(21)

confidence they have in their own clinical reasoning and prognostication (Ginestra et al., 2019). In addition to resistance, the literature also shows effects on patients' opinions and knowledge.

2.4 The relationship between healthcare professionals and patients

The AI-tools powered by EHR provide insights that give reason to start early conversations with patients on how to deal with a medical situation. The study by Parikh et al. (2019) showed that healthcare professionals considered it a reason to enter into a conversation about treatment and end-of-life preference if the AI technology discovered a high risk of dying in the short term. A suitable solution can be found in the co-operation between the patient and healthcare professionals. This conversation would also have taken place without the AI tool, but later and possibly too late. This co-operation between the patient and healthcare professionals increases by the use of NLP, a machine learning technology that makes EHR more accessible and understandable for patients (Zheng & Yu, 2018). Zheng and Yu (2018) emphasize that while EHR notes are better accessible and easier to read, they can still be incorrectly understood by patients who do not have a medical background/training. However, patients with the right knowledge could get a deeper understanding and a stronger opinion on their current health situation (Zheng & Yu, 2018).

Another effect on the relationship is the need for co-operation in creating an impactful database. Collecting data from patients is essential. By supplying data from the patient side, a higher performance database is created from which AI can obtain better diagnoses (Zhao, 2020). For example, reviews patients share on specific drugs (Wunnava et al., 2019). Therefore, the patient will play an increasingly valuable role in this process of enriching the EHR.

(22)

Management of healthcare professionals

Effect Article no.

Replacement of healthcare professional Supporting Healthcare professionals [19] [11] [28] [10] Outperforming clinicians [4]

Need for training and

education Training for healthcare professionals [24] [28] Resistance Resistance by uncertainty [25] [20]

Resistance by Alert fatigue [20] Resistance by unclarity of

action

[20] Relationship between

healthcare professional and patients

Earlier interaction with

patient [16] [7] [26] [23]

(23)

3. Indirect effects

In addition to the direct effects that are noticeable in the treatment of the patient and the management of the healthcare professionals, the selected articles also include indirect effects on hospitals. These indirect effects are diverse and can be divided into efficiency and costs, usability and privacy.

3.1 Efficiency and cost

Due to the increasing amount of insightful information from EHR through AI analysis, there are various ways of acting more efficiently in healthcare. A first cost advantage is described by Miled et al. (2020). The last mentioned researcher showed that combining three valuable data flows (drug prescription, diagnoses, and medical notes) in a machine learning algorithm is sufficient to make predictions and to do plan-based interventions. Early interventions can delay the severity of the disease and, on average, reduce 14% of the costs. This statement about increasing efficiency is reinforced by the research of Brisimi, Xu, Wang, Dai, and Paschalidis (2019). They concluded that early detection of diabetes using AI on EHR leads to savings of $ 34 per patient. In the US alone, this saves up to 29.1 million dollars per year (Brisimi et al., 2019).

A second cost advantage comes from the more efficient handling through early recognition of surgery cancellation. The research conducted by Liu et al. (2019) reveals this potential. They explained that the use of AI in combination with EHR has great potential to prevent cancellation in advance, which results in healthcare cost benefits. These cost benefits in surgery are also described by Tuwatananurak et al. (2019). They describe how machine learning calculations reduce the average operation by 7 minutes, which is an insignificant amount per case, but a significant decrease in costs cumulatively.

(24)

Despite the above results Simon et al. (2019) argue that higher costs are involved with the implementation of the AI in consisting EHR in hospitals. Simon et al. (2019) state that the development, integration and implementation of AI that is compatible with EHR is costly. Furthermore, Simon et al. (2019) explain that training of healthcare professionals in the use of AI-tools involves a lot of expensive time investments from clinical experts, who cannot use this time to take care of patients. Another reason they put forward is the idea that AI-technologies are not mature enough and are not an off-the-shelf commodity. This reduces the degree of usability for general use.

3.2 Usability

The usability of AI-tools plays a role in the maximum achievable knowledge-gathering out of EHR. However, healthcare professionals often lack the skills to analyze and understand the outcomes of AI technologies. Miotto et al. (2016) also anticipated on the lack of expertise in informatics by healthcare professionals. They developed a feature selection tool in their algorithm so that clinicians better understand the underlying drivers of the different predictions. In addition, Brennan et al. (2019) showed that physicians generally respond positively to the use of AI technology. The positive effect is attributed to the built-in confidence that emerges through the transparent design of the AI technology. The usability of EHR increases due to the development of such tools. However, this does not yet solve data anomalies between different EHR systems. A solution to deal with data anomalies can be a large-scale patient data warehouse (Miotto et al., 2016). With this, hospitals can exchange data with each other so that the accuracy of outcomes at different locations does not deviate significantly. However, privacy plays a major role in such large central databases.

3.3 Privacy

(25)

process may cause a loss of valuable clinical information. This makes AI the solution to the problem it creates itself. However, it is possible to lose useful data by increasing levels of privacy.

To conclude, three indirect effects can be observed within the selected articles. Much is written about costs and efficiency. Authors showed cost advantages, although also cases of increased costs are noticed. The usability of the technology plays a role when healthcare professionals lack expertise. Privacy is cited as an important variable in several studies. AI in combination with EHR causes privacy issues but could also solve those issues. Table 8 shows these effects and the corresponding articles.

Indirect effects Effect Article no.

Efficiency and cost Reduced cost [30] [18] [21] [6]

Increased cost [25]

Usability Increased usability of data [3] [14]

Privacy More privacy issues [29] [2]

Loss of data [1]

(26)

Discussion

This review provides insights into the effects caused by the combined use of AI and EHR in hospitals. After an extensive selection procedure, 30 articles have been selected to be included in this systematic literature review. During the selection procedure, one article was rejected for quality reasons. It is interesting that all of the articles have been written in the past seven years. This trend can be explained by the increasing developments in AI making effects in data-driven sectors increasingly clear and useful (Müller & Bostrom, 2016).

The articles were first subdivided in different forms of effects the combined use of AI and EHR has on hospitals. The results are centered around three main topics, namely the effects on patient management, the effects on healthcare professionals and additional not health-related effects on hospitals.

(27)

1. Relationship between data usability and effect: The usability of data and the effect on improved patient health

When the AI technology makes complex data from EHR more usable, new insights for healthcare professionals become available (Miotto et al., 2016). At the same time, healthcare professionals need training or experience to make optimal use of these insights (Tao et al., 2020). This data insights occur in the areas of monitoring, diagnosis, risk and decision-making (Kate et al., 2020; Hale et al., 2019; Miotto et al., 2016). Patients are constantly monitored, and when a small change occurs in real-time EHR, decisions can be made more quickly by healthcare professionals who are supported by decision-making AI technology. The slightest change in a patient's health is rapidly and accurately detected, allowing the risk of deterioration and even short-term mortality to be addressed quickly (Kate et al., 2020). Although the complexities of diseases are increasing, the combined use of both technologies allows healthcare professionals to make better decisions by taking more variables than previously thought possible into account (Hu et al., 2018; He et al., 2016). This could include prescribing medication that has less side-effects or whether it is substantiated to perform a certain surgery regarding the survival of a patient (Wunnava et al., 2019; Brennan et al., 2019). Due to the progress in usability of healthcare data massive improvements in abruptness and accurate decision-making is possible. This results in a higher quality of patient care and patient well-being. Using data from EHR is essential but complex at the same time. A problem arises when healthcare professionals are not sufficiently trained in making correct interpretations out of the EHR. Williams et al. (2018) described that healthcare professionals are not skilled enough to use the technology in an efficient manner. Additionally, Tao et al. (2020) advocate for training healthcare professionals with the aim to use the technology correctly.

Proposition 1: The combined use of AI and EHR makes complex data better usable for healthcare professionals. This effect is stronger when healthcare professionals are trained to make more sense of the insights the data generates.

2. Relationship between data overview and efficiency: Obtaining an overview of hospital data and reducing costs

(28)

processes and the required resources can be predicted, which means that scarce resources are used more efficiently. Liu et al. (2019) describe that the costs of unused operating rooms and length of hospital stay decreases, which ultimately reduces costs. Two other selected studies also illustrate direct positive effects in the aspect of cost reduction (Brisimi et al., 2019; Tuwatananurak et al., 2019). However, some articles also illustrate the potential of higher costs (Simon et al., 2019). This is caused by the complexity of the data and the time investments of healthcare professionals to make proper use of this complex data. In this case, cost of education and training is taking away time from healthcare professionals that can not be spent on patientcare. Moreover, it is of great importance to guarantee privacy (Korach, 2020). By ensuring privacy, potential valuable data could be lost due to data de-identification (Delegger, 2013). The technology can potentially become less accurate, resulting in potential inefficiencies that result in more costly decision-making. Another drawback that reduces efficiency is the anomalies between EHR databases. An extensive central database can be the solution for this (Miotto et al., 2016). However, the implementation of a central database is associated with high development costs and more significant privacy concerns. Despite the fact that this development involves many problems, it is crucial to know the effects on cost and if making investments causes significant efficiency advantages.

Proposition 2: The combined use of AI and EHR increases efficiency of hospital care. This effect becomes stronger when the cost of education remains minimal and when ensuring privacy does not cause loss of valuable patient data.

3. Relationship between effect and resistance: New possibilities and work processes that lead to resistance of healthcare professionals

(29)

as most important factor during decision-making (Ginestra et al., 2019). This causes a form of resistance that has the possibility to disable the maximum potential of the combined use of AI and EHR in hospitals. Resistance is seen as the main reason for failure and can occur at different times during the process of change (Bateh, Castaneda, & Farah, 2013). The findings in this study are consistent with this change process, as described by Duck (2001). In the first stage of his process, he describes how stagnation ensures that those involved do not want to know anything or little about the change. This fits in well with the preference of healthcare professionals to put their own judgment in the first place (Ginestra et al., 2019). Despite the fact that this is an indicator that hospitals are at the beginning of the change process, it is very important to identify this resistance force early on so that new developments are not hindered and possibly improved by it. Resistance itself has been studied intensively, but resistance in hospital environments, where the importance of a patient's health is central, may provide new insights into the management of change processes within hospitals.

Proposition 3: The implementation of AI combined with EHR causes resistance by healthcare professionals.

Conclusion

(30)

becomes more important. One article described how AI is the solution for its own created privacy issue (Miotto et al., 2016). To conclude, the combination of AI and EHR has positive effects on usability of complex patient data. The improved processes in hospitals have positive effects on cost, but these effects are only achieved when resistance of healthcare professionals is better understood.

Future research and limitations

This review shows different effects of AI on EHR. In the descriptive section of this review it became clear that this topic is trending and developing rapidly. Therefore, the first recommendation for future research is to repeat the study to discover a pattern in the developments. A second future research recommendation covers the effects of AI and comprehensive databases on management theories. AI is getting more capable of using massive amounts of data and making quick accurate decisions. Thereby a new autonomous form of pressure becomes visible. It is of importance to know what this impact on management models and theories is, because in most of these theories and models people play a key role in final decision-making. Therefore, these effects should be studied to better understand the influences these effects will have on academic management theories and eventually society.

(31)

References

1. Deleger, L., Molnar, K., Savova, G., Xia, F., Lingren, T., Li, Q., … Solti, I. (2013) Large scale evaluation of automated clinical note de-identification and its impact on information extraction. Journal of the American Medical Informatics Association, 20(1), 84–94.

2. Li, M., Carrell, D., Aberdeen, J., Hirschman, L., & Malin, B. A. (2014). De-identification of clinical narratives through writing complexity measures. International Journal of

Medical Informatics, 83(10), 750–767.

3. Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016). Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records. Scientific Reports, 6(1).

4. Hu, B., Bajracharya, A., & Yu, H. (2020). Generating Medical Assessments Using a Neural Network Model: Algorithm Development and Validation. JMIR Medical

Informatics, 8(1).

5. Samad, M. D., Ulloa, A., Wehner, G. J., Jing, L., Hartzel, D., Good, C. W., … Fornwalt, B. K. (2019). Predicting Survival From Large Echocardiography and Electronic Health Record Datasets. JACC: Cardiovascular Imaging, 12(4), 681–689.

6. Wang, H., Cui, Z., Chen, Y., Avidan, M., Abdallah, A. B., & Kronzer, A. (2018). Predicting Hospital Readmission via Cost-Sensitive Deep Learning. IEEE/ACM Transactions on

Computational Biology and Bioinformatics, 15(6), 1968–1978.

7. Zheng, J., & Yu, H. (2018). Assessing the Readability of Medical Documents: A Ranking Approach. JMIR Medical Informatics, 6(1).

8. Du, L., Xia, C., Deng, Z., Lu, G., Xia, S., & Ma, J. (2018). A machine learning based approach to identify protected health information in Chinese clinical text. International

Journal of Medical Informatics, 116, 24–32.

9. Feller, D. J., Zucker, J., Yin, M. T., Gordon, P., & Elhadad, N. (2018). Using Clinical Notes and Natural Language Processing for Automated HIV Risk Assessment. JAIDS Journal

of Acquired Immune Deficiency Syndromes, 77(2), 160–166.

10. Wu, J., Liu, X., Zhang, X., He, Z., & Lv, P. (2018). Master clinical medical knowledge at certificated-doctor-level with deep learning model. Nature Communications, 9(1).

11. Hale, A. T., Stonko, D. P., Lim, J., Guillamondegui, O. D., Shannon, C. N., & Patel, M. B. (2019). Using an artificial neural network to predict traumatic brain injury. Journal of

(32)

12. Segal, G., Segev, A., Brom, A., Lifshitz, Y., Wasserstrum, Y., & Zimlichman, E. (2019). Reducing drug prescription errors and adverse drug events by application of a probabilistic, machine-learning based clinical decision support system in an inpatient setting. Journal of the American Medical Informatics Association, 26(12), 1560–1565.

13. Liu, L., Ni, Y., Zhang, N., & Pratap, J. "N. (2019). Mining patient-specific and contextual data with machine learning technologies to predict cancellation of children's surgery. International Journal of Medical Informatics, 129, 234–241.

14. Brennan, M., Puri, S., Ozrazgat-Baslanti, T., Feng, Z., Ruppert, M., Hashemighouchani, H., … Bihorac, A. (2019). Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: A pilot usability study. Surgery, 165(5), 1035–1045.

15. Bihorac, A., Ozrazgat-Baslanti, T., Ebadi, A., Motaei, A., Madkour, M., Pardalos, P. M., … Momcilovic, P. (2019). MySurgeryRisk. Annals of Surgery, 269(4), 652–662.

16. Parikh, R. B., Manz, C., Chivers, C., Regli, S. H., Braun, J., Draugelis, M. E., … O'Connor, N. R. (2019). Machine Learning Approaches to Predict 6-Month Mortality Among Patients With Cancer. JAMA Network Open, 2(10).

17. Wang, L., Sha, L., Lakin, J. R., Bynum, J., Bates, D. W., Hong, P., & Zhou, L. (2019). Development and Validation of a Deep Learning Algorithm for Mortality Prediction in Selecting Patients With Dementia for Earlier Palliative Care Interventions. JAMA

Network Open, 2(7).

18. Brisimi, T. S., Xu, T., Wang, T., Dai, W., & Paschalidis, I. C. (2018). Predicting diabetes related hospitalizations based on electronic health records. Statistical Methods in

Medical Research, 28(12), 3667–3682.

19. Hill, B. L., Brown, R., Gabel, E., Rakocz, N., Lee, C., Cannesson, M., … Halperin, E. (2019). An automated machine learning-based model predicts postoperative mortality using readily-extractable preoperative electronic health record data. British Journal of

Anaesthesia, 123(6), 877–886.

20. Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., … Umscheid, C. A. (2019). Clinician Perception of a Machine Learning–Based Early Warning System Designed to Predict Severe Sepsis and Septic Shock*. Critical

Care Medicine, 47(11), 1477–1484.

21. Tuwatananurak, J. P., Zadeh, S., Xu, X., Vacanti, J. A., Fulton, W. R., Ehrenfeld, J. M., & Urman, R. D. (2019). Machine Learning Can Improve Estimation of Surgical Case Duration: A Pilot Study. Journal of Medical Systems, 43(3).

(33)

in Crohn's disease patients using predictive analytics methods and electronic medical record data. Health Informatics Journal, 25(4), 1201–1218.

23. Wunnava, S., Qin, X., Kakar, T., Sen, C., Rundensteiner, E. A., & Kong, X. (2019). Adverse Drug Event Detection from Electronic Health Records Using Hierarchical Recurrent Neural Networks with Dual-Level Embedding. Drug Safety, 42(1), 113–122.

24. Williams, A., Mekhail, A.-M., Williams, J., Mccord, J., & Buchan, V. (2018). Effective resource management using machine learning in medicine: an applied example. BMJ

Simulation and Technology Enhanced Learning, 5(2), 85–90.

25. Simon, G., Dinardo, C. D., Takahashi, K., Cascone, T., Powers, C., Stevens, R., … Chin, L. (2018). Applying Artificial Intelligence to Address the Knowledge Gaps in Cancer Care. The Oncologist, 24(6), 772–782.

26. Zhao, S., Xie, B., Li, Y., Zhao, X., Kuang, Y., Su, J., … Chen, X. (2019). Smart

identification of psoriasis by images using convolutional neural networks: a case study in China. Journal of the European Academy of Dermatology and Venereology, 34(3), 518–524.

27. Kate, R. J., Pearce, N., Mazumdar, D., & Nilakantan, V. (2020). A continual prediction model for inpatient acute kidney injury. Computers in Biology and Medicine, 116, 103580.

28. Tao, L., Zhang, C., Zeng, L., Zhu, S., Li, N., Li, W., … Ji, H. (2020). Accuracy and Effects of Clinical Decision Support Systems Integrated With BMJ Best Practice–Aided Diagnosis: Interrupted Time Series Study. JMIR Medical Informatics, 8(1).

29. Korach, Z. T., Yang, J., Rossetti, S. C., Cato, K. D., Kang, M.-J., Knaplund, C., … Zhou, L. (2020). Mining clinical phrases from nursing notes to discover risk factors of patient

deterioration. International Journal of Medical Informatics, 135, 104053.

30. Miled, Z. B., Haas, K., Black, C. M., Khandker, R. K., Chandrasekaran, V., Lipton, R., & Boustani, M. A. (2020). Predicting dementia with routine care EMR data. Artificial

Intelligence in Medicine, 102, 101771.

31. Bateh, J, Castaneda, ME, and Farah, JE (2013) Employee resistance to organizational change. International Journal of Management & Information Systems, 17(2), 113–16.

(34)

33. Birkle, C., Pendlebury, D. A., Schnell, J., & Adams, J. (2020). Web of Science as a data source for research on scientific and scholarly activity. Quantitative Science Studies,

1(1), 363-376.

34. Boonstra, A., Versluis, A., & Vos, J. F. (2014). Implementing electronic health records in hospitals: A systematic literature review. BMC Health Services Research, 14(1).

35. Brouillette, M. (2019). AI added to the curriculum for doctors-to-be. Nature

Medicine, 25(12), 1808–1809.

36. Chen, C., Loh, E., Kuo, K. N., & Tam, K. (2019). The Times they Are a-Changin’ Healthcare 4.0 Is Coming! Journal of Medical Systems, 44(2).

37. Cook, D. J. (1997). Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions. Annals of Internal Medicine, 126(5), 376.

38. Data quality and artificial intelligence – mitigating bias and error to protect fundamental

rights. (2020, February 3rd). Retrieved from

https://fra.europa.eu/en/publication/2019/data-quality-and-artificial-intelligence-mitigating-bias-and-error-protect

39. Dawes, T., Marvao, A. D., Shi, W., Fletcher, T., Watson, G., Wharton, J., … Oregan, D. (2016). Use of artificial intelligence to predict survival in pulmonary hypertension. The

Lancet, 387.

40. Duck, J. D. (2001). The change monster: The human forces that fuel or foil corporate transformation and change. New York: Crown Business.

41. Dunjko, V., & Briegel, H. J. (2018). Machine learning & artificial intelligence in the quantum domain: A review of recent progress. Reports on Progress in Physics, 81(7), 074001.

42. Electronic Health & Medical Records. (2011). P. Garret, J. Seidman. Retrieved from https://www.healthit.gov/buzz-blog/electronic-health-and-medical-records/emr-vs-ehr-difference

43. Goldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2016). Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. Journal of the American Medical Informatics Association, 24(1), 198–208.

44. Grimson J, Grimson W, Hasselbring W: The SI Challenge in Health Care. Commun ACM

2000, 43(6):49–55.

45. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition.

(35)

46. Howe, J. L., Adams, K. T., Hettinger, A. Z., & Ratwani, R. M. (2018). Electronic Health Record Usability Issues and Potential Contribution to Patient Harm. Jama, 319(12), 1276.

47. Jacob, P. D. (2020). Management of patient healthcare information. Fundamentals of

Telemedicine and Telehealth, 35–57.

48. Johnson, J. K., Farnan, J. M., Barach, P., Hesselink, G., Wollersheim, H., Pijnenborg, L., Arora, V. M. (2012). Searching for the missing pieces between the hospital and primary care: Mapping the patient process during care transitions. BMJ Quality &

Safety, 21(Suppl 1), I97-I105.

49. Keyhani, S., Hebert, P., Ross, J., Federman, A., Zhu, C., & Siu, A. (2008). Electronic Health Record Components and the Quality of Care. Medical Care, 46(12), 1267-1272. Retrieved June 18th, 2020, from www.jstor.org/stable/23053782

50. Kmet LM, Lee RC, Cook LS. Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Alberta Herit Found Med Res. 2004; 13:1–22.

51. Kwon, J., Kim, K., Jeon, K., Lee, S. E., Lee, H., Cho, H., . . . Oh, B. (2019). Artificial intelligence algorithm for predicting mortality of patients with acute heart failure. Plos

One, 14(7).

52. Li, K., Rollins, J., & Yan, E. (2017). Web of Science use in published research and review papers 1997–2017: A selective, dynamic, cross-domain, content-based analysis.

Scientometrics, 115(1), 1-20.

53. Li, L. (2018). China's manufacturing locus in 2025: With a comparison of “Made-in China 2025” and “Industry 4.0”. Technological Forecasting and Social Change, 135, 66-74.

54. Mantelakis, A., & Khajuria, A. (2020). The applications of machine learning in plastic and reconstructive surgery: Protocol of a systematic review. Systematic Reviews,9(1).

55. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31st, 1955. AI

Magazine, 27(4), 12.

56. Menachemi, N., & Collum. (2011). Benefits and drawbacks of electronic health record systems. Risk Management and Healthcare Policy, 47.

57. Müller, V. C., & Bostrom, N. (2016). Future Progress in Artificial Intelligence: A Survey of Expert Opinion. Fundamental Issues of Artificial Intelligence, 555-572.

58. Neill, D. B. (2013). Using Artificial Intelligence to Improve Hospital Inpatient Care. IEEE

(36)

59. Radcliffe, K., Lyson, H. C., Barr-Walker, J., & Sarkar, U. (2019). Collective intelligence in medical decision-making: A systematic scoping review. BMC Medical Informatics and

Decision Making,19(1).

60. Stephen Hawking warns of dangerous AI. (2016, October 20th). Retrieved from https://www.bbc.com/news/av/technology-37713942/stephen-hawking-warns-of-dangerous-ai

61. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British

Journal of Management, 14(3), 207–222.

62. Tyllinen, M., Kaipio, J., & Lääveri, T. (2019). Usability Analysis of Contending Electronic Health Record Systems. Studies in health technology and informatics, 257, 430–435.

63. Watson, D., Womack, J., & Papadakos, S. (2020). Rise of the Robots. Critical Care Nursing

Quarterly, 43(3), 303-311.

64. Wolff, J., Pauling, J., Keck, A., & Baumbach, J. (2020). Systematic Review of Economic Impact Studies of Artificial Intelligence in Health Care. Journal of Medical Internet

(37)

Appendix A

Figure 1. Google trends analyse of Trending AI search terms (01-01-2004 - 01-04-2020)

Figure 2. Google trends compared analyse between less trending terms (01-01-2004 -

(38)

Appendix B

Table 1. The complete search string

Subject Search term

Artificial Intelligence "Artificial intelligence" or "Artificial neural network" or "Deep learning" or "Genetic algorithm" or "Machine learning"

Electronic Healthcare Records "EHR*" or "Electronic Health Record*" or "EMR*" or" Electronic medical record*" or "Electronic patient record*" or "Computerized patient record*" or "CPR*" or "Personal health record*"

(39)

Appendix C

Table 2. Quantitative studies 1-3, 5-19, 21-22

Criteria quantitative studies [1] [2] [3] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [21] [22]

Question/objective sufficiently described? 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

Study design evident and appropriate? 2 2 2 2 2 2 2 2 1 2 2 1 2 2 2 2 2 1 2 1

Method of subject/comparison group selection or source of information/input variables described and

appropriate?

1 2 1 2 1 1 2 1 2 1 2 2 1 1 2 2 1 1 1 1

Subject (and comparison group, if applicable) characteristics sufficiently described?

1 2 1 1 2 2 1 1 2 2 1 2 1 2 2 1 2 1 1 1

If interventional and random allocation was possible, was it described?

n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a

If interventional and blinding of investigators was possible, was it reported?

n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a

If interventional and blinding of subjects was possible, was it reported?

n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a

Outcome and (if applicable) exposure measure(s) well defined and robust to measurement/misclassification bias? Means of assessment reported?

2 2 2 1 1 1 1 2 1 1 2 2 2 1 1 1 2 2 1 1

Sample size appropriate? 2 2 2 1 2 2 2 1 1 2 2 2 1 2 2 1 2 2 1 2

Analytic methods described/justified and appropriate? 2 1 1 2 1 2 2 1 1 2 2 1 1 1 2 1 1 1 1 1

Some estimate of variance is reported for the main

results? 2 1 1 2 1 2 2 1 2 2 1 2 1 2 2 2 1 2 2 n/a

Controlled for confounding? 1 1 1 0 n/a 1 2 1 1 2 2 2 0 2 2 1 1 2 2 0

Results reported in sufficient detail? 1 2 2 2 2 2 2 2 1 2 2 1 2 1 2 2 2 2 2 2

Conclusions supported by the results? 2 2 1 2 2 2 1 1 1 2 2 1 2 2 2 2 2 1 1 2

Total score/possible maximum score 20/

(40)

Table 3. Quantitative studies 23-31

Criteria quantitative studies [23] [24] [25] [26] [27] [28] [29] [30]

Question/objective sufficiently described? 2 2 2 2 2 2 2 1

Study design evident and appropriate? 1 2 2 2 2 1 2 1

Method of subject/comparison group selection or source of information/input variables described and appropriate?

2 2 2 2 2 1 2 1

Subject (and comparison group, if applicable) characteristics sufficiently described? 1 2 2 1 2 1 1 2 If interventional and random allocation was possible, was it described? n/a n/a n/a n/a n/a n/a n/a n/a If interventional and blinding of investigators was possible, was it reported? n/a n/a n/a n/a n/a n/a n/a n/a If interventional and blinding of subjects was possible, was it reported? n/a n/a n/a n/a n/a n/a n/a n/a Outcome and (if applicable) exposure measure(s) well defined and robust to

measurement/misclassification bias? Means of assessment reported? 2 2 2 1 2 2 1 2

Sample size appropriate? 0 2 1 1 2 1 2 1

Analytic methods described/justified and appropriate? 1 1 2 2 1 1 2 1

Some estimate of variance is reported for the main results? 2 1 2 2 2 2 2 1

Controlled for confounding? 1 1 2 0 1 1 2 1

Results reported in sufficient detail? 2 2 2 2 2 2 2 2

Conclusions supported by the results? 1 2 2 2 1 1 1 2

(41)

Table 4. Mixed studies

Mixed method studies [4] [20]

Qualitative criteria mixed methods studies

Question/objective sufficiently described? 1 2

Study design evident and appropriate? 2 2

Context for the study clear? 2 1

Connection to a theoretical framework/wider body of knowledge? 1 1 Sampling strategy described, relevant and justified? 1 2 Data collection methods clearly described and systematic? 2 2 Data analysis clearly described and systematic? 1 2 Use of verification procedure(s) to establish credibility? 0 2

Conclusions supported by the results? 1 1

Reflexivity of the account? 1 1

Quantitative criteria mixed methods studies

Question/objective sufficiently described? 1 2

Study design evident and appropriate? 2 2

Method of subject/comparison group selection or source of information/input variables

described and appropriate? 1 1

Subject (and comparison group, if applicable) characteristics sufficiently described? 1 2 If interventional and random allocation was possible, was it described? n/a n/a If interventional and blinding of investigators was possible, was it reported? n/a n/a If interventional and blinding of subjects was possible, was it reported? n/a n/a Outcome and (if applicable) exposure measure(s) well defined and robust to

measurement/misclassification bias? Means of assessment reported?

1 2

Sample size appropriate? 1 1

Analytic methods described/justified and appropriate? 2 1 Some estimate of variance is reported for the main results? 1 1

Controlled for confounding? 0 1

Results reported in sufficient detail? 2 2

Conclusions supported by the results? 2 1

(42)

Appendix D

Table 5. Selected Articles

Article number

Year Journal Main objective Type of

research

Data collection

Sample size Type of

AI

Medical topic Impact on hospital [1] 2013 JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION Machine learning as automated de-identification tool without losing valuable data in EHR Quantitative Analysis of EHR (notes) 3503 documents Machine

learning Patient data Privacy

[2] 2014 INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS Machine learning based de-identification tool to solve privacy issues in EHR Quantitative Analysis of EHR 4500 documents of varying document types Machine learning

Patient data Privacy

[3] 2016 SCIENTIFIC

REPORTS

Deep learning was used to better predict diseases then raw EHR data Quantitative Analysis of EHR 762.14 patients records Deep learning General Monitoring / Risk / Decision-making / Privacy [4] 2018 JMIR MEDICAL INFORMATICS

Use deep learning to create a software that helps make clinical decisions based on EHR data Quantitative and Qualitative Analysis of EHR (notes) and survey

Not defined data amount and 2 Physician evaluations

Deep learning

(43)

[6] 2018 IEEE-ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS

Use deep learning for early possible readmission and thereby lowering the cost of life threating medical issues Quantitative analysis of EHR Undefined amount of data from a medical University Datawarehouse Deep learning Readmission efficiency / decision-making [7] 2018 JMIR MEDICAL INFORMATICS Use machine learning on EHR to make personal notes and documents readable and accessible Quantitative Analysis of EHR (notes) 20.865 medical sentences Machine learning

Clinical Notes Patient relationship [8] 2018 INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS A machine learning technique that takes care of patient data by de-identifying data Quantitative Analysis of EHR 14.719 discharge summaries Machine learning

Patient data Privacy / Risk [9] 2018 JAIDS-JOURNAL OF ACQUIRED IMMUNE DEFICIENCY SYNDROMES Use AI to create an automated risk assessment that has cost saving properties Quantitative Analysis of EHR 724 patient records Machine learning HIV Risk [10] 2018 NATURE COMMUNICATION S A deep learning AI tool that has the potential to replace doctors or help when a shortage of doctors is occurring Quantitative Analysis of EHR

Not defined Deep

(44)

[11] 2019 JOURNAL OF NEUROSURGERY-PEDIATRICS Use artificial neural networks to faster diagnose traumatic brain injury Quantitative hospital data set 14.969 patients Artificia l Neural network Neurology Monitoring / Replacemen t of healthcare professional [12] 2019 JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION The development of a Machine learning tool that is able to describe drugs and to collect errors in drugs description Quantitative Analysis of EMR 4.533 patient admissions Machine learning Medicine Decision-making [13] 2019 INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS Testing how machine learning could predict last minute

cancellation in surgery saving time and money

Quantitative Analysis of EHR 17.000 scheduled surgeries Machine learning Surgery Decision-making / Efficiency

[14] 2019 SURGERY Using Machine

(45)

in the first 6 months [17] 2019 JAMA NETWORK OPEN Predicting mortality of people with dementia by using deep learning Quantitative Analysis of EHR 24.229 patients records Deep learning Mortality Mortality [18] 2019 STATISTICAL METHODS IN MEDICAL RESEARCH Saving significant amounts of money by machine learning based prediction on hospitalization of patient with diabetes Quantitative Analysis

of EHR 40.921 patients records Machine learning Diabetes Efficiency

(46)

[22] 2019 HEALTH INFORMATICS JOURNAL Use machine learning to predict and monitor level of inflammation in crohn disease Quantitative Analysis of EMR 30.150 patients records Machine learning gastroenterolog y Decision-making

[23] 2019 DRUG SAFETY Deep learning as

tool to provide insights in adverse drug events Quantitative Analysis of EHR 1.089 de-identified EHR notes from 21 cancer patients Artificia l Neural network Medicine Decision-making [24] 2019 BMJ SIMULATION & TECHNOLOGY ENHANCED LEARNING Effective collaboration between clinicians and machine learning Quantitative Analysis

of dataset 7.988 patient records Machine learning Clinical resources Efficiency

[25] 2019 ONCOLOGIST Use machine

learning to support practitioners in decision-making and also contain implementation effects on cost Quantitative Analysis of EMR 1.000 patients records General AI Oncology Efficiency [26] 2020 Journal of the European Academy of Dermatology and Venereology The application of deep learning on medical images to identify and diagnose psoriasis Quantitative Analysis of EMR 8.021 clinical images Deep learning Dermatology Diagnosing / patient relationship [27] 2020 COMPUTERS IN BIOLOGY AND MEDICINE Use machine learning to predict kidney failure and to predict the time

Referenties

GERELATEERDE DOCUMENTEN

17 (Weir et al., 1994) A facilitating factor associated with successful implementation of a CPOE is an interdisciplinary, effective implementation

The development of the user participation theory could benefit if, besides the more widely presented views of physicians and nurses ((end-) users) in the user participation

Dit laatste betekent dus in de eerste plaats de ontwikkeling van een in wezen nieuw element in de sociologische theorie - ook al zijn hier zeker heel wat verspreide aanzetten voor

De keuze voor een gangbaar ras in de derde teelt is gebaseerd op het feit dat voor de najaarsteelt nog geen goede meeldauw tolerante rassen beschikbaar zijn, op de ongunstige

Despite the National Assembly’s wish to go about matters of state in a radically different way than the States General had done, the characteristic elements – reaching

Een eerste verkenning van het bronnenmateriaal maakte al duidelijk dat men in het negen- tiende-eeuwse Maastricht misschien wel sociale lagen zou kunnen identificeren met een voor-

De creatieve broedplaatsen in Noord hebben het stadsdeel aantrekkelijker gemaakt voor deze nieuwe bewoners, echter spelen ook andere redenen mee waarom zij zich in

LENNART: Want je zegt, ik ga of een aanverwante technologie, ik zeg niet dat het blockchain moet zijn, maar dat er dus trusted partner is, die zeg ik meet die energie, ik deel dat