USER PARTICIPATION IN RELATION TO THE
DEVELOPMENT AND USE OF AN EHR SYSTEM :
A SYSTEMATIC LITERATURE REVIEW
Laura Kaal (s3214451)
University of Groningen
Faculty of Economics and Business Msc BA – Health
July 2020
Supervisor: Dr. M.A.G. van Offenbeek
Second assessor: Dr. E.I. Metting
ABSTRACT
Introduction: Electronic Health Records (EHR) have become an important tool to increase the quality of care. To utilize an EHR system and to benefit from gained advantages, user participation is crucial.(End-) Users play an important role as they must integrate the EHR system. Therefore, this literature review focuses on the degree, importance and influence of user participation. The aim is to understand the concept of user participation by reviewing the existing empirical evidence of user participation. This might be beneficial to the better use and development of an EHR system.
Methods: A literature search was conducted on five electronic databases. Studies were included if they reported on user participation in relation to the development of
implementation of an EHR system in a healthcare context. The quality and relevance of the studies were assessed. Therefore, five inclusion/exclusion criteria are used.
Results: Of a total of 3500 research studies identified, only 19 of them met all our criteria and were included in the literature review. Three challenges towards user participation within the healthcare context were distinguished: integrating different perspectives, usability and customizability and continuous development.These challenges revealed the divide between the clinical and IT stakeholders and was the starting point for a stakeholder analysis. This analysis showed that the perspectives of nurses and physicians are the most studied. The super users, managers and IT professionals are at the other hand undervalued.
Conclusions: This literature review presents the divide between the clinical and IT stakeholders as the most important aspect of the concept of user participation. Similarities between the different stakeholders are highlighted, but the differences demonstrate that every stakeholder has a different, but unique and valuable, perspective to the development and use of an EHR system. Therefore all stakeholders should be taken into account, also the less presented but perhaps just as important perspectives of the super users, managers and IT professionals.
TABLE OF CONTENTS
1. INTRODUCTION ... 4
1.1 Research process ... 6
2. METHOD ... 7
2.1 Databases / Information sources ... 7
2.2 Search terms ... 7
2.3 Search strategies ... 8
2.4 Selection criteria and the selection process ...10
3. RESULTS ...12
3.1 Search Results ...12
3.2 Overview of the included studies ...15
3.3 Analysis of the results ...15
3.4 Challenges regarding user participation ...17
3.4.1 Papers perspectives – relationship between (end-) users and IT professionals ....17
3.4.2 Integrating different perspectives ...19
3.4.3 Understanding and evaluation of usability of the EHR system ...20
3.4.4 Continuous design and development of the EHR system ...22
3.5 The need for new approaches regarding user participation ...23
3.6 Stakeholder analysis ...23
3.6.1 Physicians ...25
3.6.2 Nurses ...25
3.6.3 Nurses AND physicians ...26
3.6.4 Super users ...27
3.6.5 IT professionals and managers ...30
1. INTRODUCTION
The electronic health record (EHR), the digital version of the paper-based patient’s medical record, is increasingly recognized as one of the most important tools for improving the quality of care (Yuan, Bradley & Nembhard, 2015). The key objective of an EHR system is to render better care and services to the patients. The system contains information related to the patients including diagnosis, test results, treatment (details), monitoring (details), information access, billing, and patient handling. The expectations are high and the benefits of the use of this technology within healthcare can be considerable. At the same time, implementing EHRs, similar to other healthcare information systems, is complex and unpredictable in most of the times. An early study estimated that 45% of the EHR implementations fail because of user resistance (Ward, Vartak, Schwichtnberg and Wakenfield, 2011). When a new EHR system is implemented, (end-) users of the EHR system often show resistance to the use of it. The system does not meet their needs and requirements and utilizing the system use took more time and effort than expected. Previous studies, for example, have revealed that physicians tend to be reluctant to adopt new
technologies. The medical literature discloses that doctors have negative outlook towards the EHR system due to technology anxiety and ineffective doctor-patient communications (Hossain, Quaresma andRahman, 2019). These kind of difficulties, illustrate that it takes time for an (healthcare) organization to move along with system related changes. Healthcare organizations need to reserve time to adapt to these changes (Lin, Lin & Roan, 2011).
To utilize (new) EHR systems, (end-) user participation is important (Boonstra, et al., 2010). The study of Bajwa, Singh and Kumar (2017), showed that active (end-) user participation contributes to use the intended benefits of EHR systems to the fullest. The aim of (end-) user participation, when implementing EHR systems, is to enhance both the quality of the
developed system and the acceptance of this system by the (future) users. By gaining this quality and gaining user acceptance by participating (end-) users, effective use of the EHR system will be achieved (Offenbeek and Koopman, 1996). The group of (end-) users investigated in this review are limited to the physicians and nurses of the hospital. Patients are not included because they are a less important stakeholder in this review.
This wide acceptance of EHR systems will only be achieved when all organizational
organization) participate together. Not only physicians and nurses (the clinical side) but also the administrative staff, IT professionals, managers and other organizational members.
Management for example, should be aware of the social sector characteristics that may hinder the successful and optimal use of a new EHR system and encourage active user participation (Lim, 2003). These sector characteristics of healthcare (end-) users, like for example a high degree of communication and collaboration among (healthcare) professionals, diverse and dynamic working practices and governmental and professional regulations, ensure that the participation of healthcare (end-) users is more difficult than in a "normal" setting. Those sector characteristics are therefore crucial to understand (Martikainen, et al. 2012).
The degree, importance and influence of user participation are illustrated and discussed in various ways in the literature, but the published studies target many different user groups and represent a wide range of disciplines. Many initiatives are driving the development of
knowledge about research with and about user participation forwards, but the scientific knowledge within the healthcare context is weak. There are only a few studies which focus specifically on the degree, importance and influence of user participation within the
healthcare context (Najaftorkaman, et al,. 2015). This despite the fact that research about(user participation is gaining importance in the health sector. Many researches support the
argument that EHR development should involve users, and emphasize the need for understanding the contextual aspects behind the EHR system design and involving (end-) users in the (development) activities (Hartwick and Barki, 1994). However, making sure that healthcare (end-) users participate and are fully involved in the (development of the) EHR systems, seems to be difficult.
This study focuses on the weak represented healthcare context, and thereby the research about user participation will be strengthened and value will be added to the existing EHR literature. The goal is to understand the most important part(s) of user participation in relation to the (development of) EHR systems and the most optimal/efficient way of using it
part(s) of the concept of user participation is/are considered as the most important and seems to be beneficial regarding the optimal use and development of the EHR system?”
1.1 Research process
To answer the research question, a systematic literature review is used. The aim of the
literature review was “to map and to assess the existing intellectual territory, and to specify a research question to develop the existing body of knowledge further” [Tranfield, Denyer and Smart, 2003]. Reviewing literature is an important supporting tool for other stages of the research process as well. Anchoring one’s first ideas in the existing body of literature (while this usually does not imply a systematic review though) will most probably be of great help for formulating and clarifying the research topic in the first place. Furthermore, unexpected insights from one’s own data analysis may induce the researcher to dive into literature at a later stage again in order to re-conceptualize the findings (Seuring and Gold, 2012).
In this systematic literature review, the most important and outstanding part(s) of user
participation which is/are beneficial to the (optimal) use and development of an EHR system, is/are investigated. This is conducted by analyzing different articles describing user
participation in relation to the (development of the) EHR system. This results in
2. METHOD
This chapter outline the research design and the used literature review method. A systematic literature review is a key element of evidence-based healthcare and means that the results in existing studies in a specific field are gathered, combined, analyzed and in the end
summarized. The guidelines on conducting a systematic literature review as prescribed by the Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA) have been followed (Moher et al., 2009).
2.1 Databases / Information sources
The sources used in this study are limited to EBSCOhost, WEB OF SCIENCE, ACM Digital Library and GOOGLE SCHOLAR. In EBSCOhost, the healthcare databases “CINAHL” and “MEDLINE” and the business perspective database “Business source premier” were
included. The irrelevant databases within EBSCOhost were excluded. By including the healthcare databases in EBSCOhost, there was no further need to use the database PubMed because PubMed select the same articles as EBSCOhost (by focusing on the healthcare databases). In addition to EBSCOhost, WEB OF SCIENCE, ACM Digital Library and GOOGLE SCHOLAR were used. GOOGLE SCHOLAR in particular was used, in addition to the other databases, to help find additional and grey articles that are not represented in the other databases. So, while GOOGLE SCHOLAR is a very general and broad database with a lot of extra (not represented within the other databases) articles, it helped in our case to expand the search regarding the subject of user participation within the healthcare context.
2.2 Search terms
Table I – Search terms and synonyms used as search terms in the literature search
User participation EHR system
“User participation” “EHR”
“User involvement” “Electronic Healthcare records”
“Health Information Systems” “Hospital Information Systems” “Electronic Patient Record systems” “Health Information Technology” “HIS”
“Health IT” “HIT”
2.3 Search strategies
Search strategies were determined to collect the actual data and therefore, combinations of two search terms were used. To optimize the search, several synonyms were used. Eight for the keyword “EHR” and one for “user participation”. Doing this, made sure that the search would not be limited. For an overview of all the search strategies used (26 in total), see Table II and Table III.
A difference is made between the number of search strategies for EBSCOhost, WEB OF SCIENCE and the ACM Digital Library (Table II) and GOOGLE SCHOLAR (Table III). GOOGLE SCHOLAR gave enough information/articles already when only using four of the synonyms of EHR systems, “EHR”, “Electronic health records”, “Health information
systems” and “Health information technology” (Table III). When using all the other synonyms (as shown in Table II) a lot of overlap would occur and too many articles would show up to make an accurate selection (using the selection criteria). Furthermore, in
EBSCOhost, WEB OF SCIENCE and the ACM Digital Library articles were selected with a publication date from 2000 and later. In GOOGLE SCHOLAR a selection of articles was made on the publication date from 2015 and further. This to select more strictly.
Table II - Search strategies for EBSCOhost, WEB OF SCIENCE and the ACM Digital Library
Search strategy User participation EHRs
[1] “User participation” + “EHR”
[2] “User participation” + “Electronic Health
Records”
[3] “User participation” + “Health Information
Systems”
[4] “User participation” + “Hospital
Information Systems”
[5] “User participation” + “Electronic Patient
Record systems”
[6] “User participation” + “Health Information
Technology”
[7] “User participation” + “HIS”
[8] “User participation” + “Health IT”
[9] “User participation” + “HIT”
[10] “User involvement” + “EHR”
[11] “User involvement” + “Electronic Health
Records”
[12] “User involvement” + “Health Information
Systems”
[13] “User involvement” + “Hospital
Information Systems”
[14] “User involvement” + “Electronic Patient
Record systems”
[15] “User involvement” + “Health Information
Technology”
[16] “User involvement” + “HIS”
[17] “User involvement” + “Health IT”
Table III - Search strategies for GOOGLE SCHOLAR
Search strategy User participation EHRs
[19] “User participation” + “EHR”
[20] “User participation” + “Electronic Health
Records”
[21] “User participation” + “Health Information
Systems”
[22] “User participation” + “Health Information
Technology”
[23] “User involvement” + “EHR”
[24] “User involvement” + “Electronic Health
Records”
[25] “User involvement” + “Health Information
Systems”
[26] “User involvement” + “Health Information
Technology” 2.4 Selection criteria and the selection process
Once the articles were obtained, (inclusion) selection criteria were applied to filter out the irrelevant articles. This selection process was carried out in five steps:
Step [1]: General screening. The document types needed to be articles and reviews, the articles needed to be published in a peer-reviewed journal (full text available), the written language needed to be English to establish deep understanding of the articles and in the end, duplicated articles needed to be removed and excluded.
Step [2]: Screening the titles of the articles left after applying step one. By doing this the totally irrelevant articles were filtered out manually. Only articles which could be of potential use for to this study remained in the selection. (End-) Users in this review were limited to the physicians and nurses, studies in which the patient was the main (end-) users were excluded because of the irrelevance for this study.
was excluded in case it was not specifically about information systems within the healthcare context.
Step [4]: Considering the relevance of the articles for this study. First of all it was checked whether the concept of user participation was an independent variable. After this check it was important to consider whether an article provided additional information about the effect of user participation. Articles had to meet the relevance of this study by gaining insight in the most important aspect(s) of user participation .
3. RESULTS
The results of the systematic literature review are presented in this chapter. In the first section the search results towards the final 19 selected articles are presented and an overview of the included articles is given. Furthermore, the results showed the challenges to user
participation, followed by the need for new approaches and ends with a stakeholder analysis.
3.1 Search Results
Through this search strategy, 3500 articles were identified. The search for these articles was performed from the 20th until the 27th of April 2020 and all five databases were searched in this period of time. In performing the first selection step, a general screening of the articles, 630 duplicates were identified and removed by using the Mendeley application for
identifying exact and close duplicates. The remaining 2897 articles were all articles and reviews, published in a peer-reviewed journal and written in English. Out of these remaining 2897 articles, the second selection step, screening the titles, was performed and a total of 210 articles were left. A huge part of the articles did not add any additional value to this study based on the corresponding title. For example, articles in which participation of the patient was the main topic instead of the health professionals. The next selection step, step 3, was screening and reading the abstracts. Articles that did not explicitly concern user participation in relation to a EHR system and were not related to the healthcare context were excluded. Furthermore, user participation should be the main topic of the article and articles in which user participation was only a little part of a bigger picture, were excluded. From here, after performing steps one, two and three, a total of 55 articles remained. In the next selection step, step 4, these 55 articles were screened by their relevance. Articles in which user participation was a dependent variable were excluded. Furthermore it was important that the article really adds information to the concept of user participation. For this study it was only interesting to see which parts of user participation were the most important and outstanding. After
performing step 4, a total of 21 articles were left. Finally, a closer look was given to the 21 articles regarding their quality. This was done by using the Standard Quality Assessment Criteria for Evaluating Primary Research Papers by Kmet et al., (2004). In here two scoring systems (quantitative and qualitative) to evaluate the quality of the studies potentially eligible for inclusion in a review were developed.
and “no” = 0 points). A (summary) score was calculated for each paper by summing the total score obtained across relevant items and dividing by the total possible score. Items that were not applicable to a particular study design were marked with a “n/a” and were excluded from the calculation of the total summary score. When this score was below the threshold of 50%, the article was assumed to be of too low quality and was removed from the selection. This way of exclusion is seen as a good assessment tool (Kmet et al., 2004). Appendix A shows an overview of the completed (scoring) checklists for the 21 qualitative, quantitative and
qualitative & qualitative studies selected after applying the four selection steps. Two of the 21 articles scored below the 50%, respectively 40% and 35%. These two articles were excluded and this resulted in a total of 19 useable articles taking the Standard Quality Assessment Criteria for Evaluating Primary Research Papers in mind.
Figure I – Total selection process EBSCOhost – 204 potentially relevant articles WEB OF SCIENCE – 146 potentially relevant articles ACM Digital Library – 331 potentially relevant articles GOOGLE SCHOLAR – 2819 potentially relevant articles Total of 3500 potentially relevant articles Total of 2897 potentially relevant
articles left for screening Total of 210 potentially relevant articles Total of 55 potentially relevant articles Total of 19 relevant and included articles
left
1) 603 removed duplicates
2) 2687 articles excluded based on step
1 & 2
3) 155 articles excluded based on step 3
4) 34 articles excluded based on step 4 & the
3.2 Overview of the included studies
To gain more insight in the nature and objectives of the 19 selected articles, an overview is provided in Table IV. The articles were numbered and an overview was given of the names of the authors, the year of publication, the journal the article is published in, the main objective of the article, the type of research and finally the way in which the specific data was collected and which sample size was used.
The final selection of articles consists of 11 qualitative studies, 4 quantitative and 3 mixed methods. Among the qualitative articles, most of the them used a combination of (different kind of) interviews and (different kind of) observations. Furthermore, two literature studies were conducted, one article used a usability evaluation (heuristic walkthrough method) and within the quantitative articles the authors used a lot of surveys and (different kind of)
questionnaires. All the articles were published from the year 2001 and further, and even more than 50 % was published in the last 5 years. The topic seems to be very “new” and that makes this study interesting. Furthermore, almost 75% of the articles were published in a
medical/health related journal. More research is needed regarding the management side to broaden the subject and the existing literature around this (health related) subject.
3.3 Analysis of the results
Table IV – Overview of the included studies (given) Number Author (s) Year of publicatio n
Journal Main objective of the article Type of
research
Research design and/or data
collection Sample size
1 Abugabah & Alfarraj 2015 Electronic Journal of Health Informatics
To identify key issues that should be considered when designing, developing, and
implementing Healthcare Information Systems at the user level Qualitative Literature review 26 studies
2 Cresswell, et al. 2017 HSR : Health Services Research
To explore longer-term perspectives on user engagement through investigating the range of ways in which health care workers and organizations accommodate new
technologies over time
Qualitative
(semi-structured) Interviews, (non-participant) observations & collected organizational documents
173 interviews, 24 observations & 17
documents
3 Cresswell, et al. 2011 Informatics in Primary Care
To build on the existing user engagement literature in relation to IT in health care by reflecting on the approaches to and experiences of user engagement in a national
implementation of complex EHRs
Qualitative (semi-structured) Interviews, (non-participant) observations 138 interviews & 43 hours of observations 4 Høstgaard, Bertelsen & Nøhr 2011 BMC Medical Informatics and Decision Making
To develop an improved method for identifying, studying and understanding end-user participation in HIT development in order to collect more detailed information
on social groups participating in HIT development and their interaction during the development.
Qualitative
Observations, (semi-structured) interviews & insight gathered from
documents and relevant literature
15 interviews
5 Ivatury, Jeena &
Prakash 2017
ACM International Conference Proceeding Series
To present a case for using participatory approaches in the design and development of information and communications technology or ICT-based systems to incorporate the complexities embedded in the Indian public health sector and bring desired
changes
Qualitative (participant) Observation & interviews
Not mentioned / not clear
6 Ju, Wei & Tsai 2016
Studies in Health Technology and Informatics
To understand the relationship of socialnetwork, user participation and the success of system implementation and to discover the role of social network in
post-implementation and in the enhancement of user participationto accomplish the system post-implementation successfully.
Quantitative Questionnaires 211 questionnaires
7 Kushniruk & Nøhr 2016 Evidence-Based Health Informatics
To show that the participatory approach goes beyond user-centered design and co-operative design approaches to include end users as more active participants in
design ideas and decision making
Qualitative Conceptual comparison 3 Models
8 Litwin 2011 Industrial and Labor Relations Review
To present the first empirical evidence of the potential of employee involvement to enhance the effectiveness of health IT
Quantitative & Qualitative
Employee and patient surveys, interviews, archival data & (clinic)
observation
Not mentioned / not clear
9 Martikainen, Kaipio & Lääveri 2020
International Journal of Medical Informatics
To examine the participation experiences of physicians and nurses with HIS
development. Quantitative Cross-sectional surveys
4018 physicians & 3607 nurses
10 Martikainen, et al. 2012 International Journal of Medical Informatics
To learn (1) about the kind of experiences that physicians have with participation in healthcare IT development; (2) whether physicians are interested in participating in IT development activities, and if so, how; and (3) the visions that physicians have
regarding future IT systems
Quantitative Web-based questionnaire 3929 physicians
11 Mclean, Frisch &
Roudsari 2015
Canadian Journal of
Nursing Informatics To explore this gap in our understanding, to find ‘Nursing’s Voice’ in this process Qualitative Survey methodology & interviews
5 nursing informatics, 7 Health Authorities & 14
senior executives
12 Pilemalm & Timpka 2008 Journal of Biomedical Informatics
To report participatory action research on the development of a (participatory design) PD framework for large-scale system design. Which modifications are needed in order to make PD of (health information systems) HIS applicable in large
health service organizations
Qualitative (active and passive participant) Observation
Not mentioned / not clear
13 Rahimi, Safdari & Jebraeily 2014
Acta Informatica Medica
To investigate the participation rate of users in different stages of HIS development
as well as to identify the factors affecting it. Quantitative Self-structured questionnaire
140 individuals in various job
14 Saleem, et al. 2017 Health Care Manager
To examine the paradigms underlying the concept of user participation and what user participation must entail to achieve its desired effective and beneficial results.
So, further emphasizing the importance of effective user participation to ensure system success.
Qualitative Case study Not mentioned / not clear
15 Shah & Robinson 2007
International Journal of Technology Assessment in
To investigate the benefits of and barriers to the involvement of users in medical
device technology development Qualitative Literature review 25 studies
16 Van der Meijden, et
al. 2001 Health Care To describe in more detail the participation of our future users
Quantitative & Qualitative
Questionnaires & (in-depth) interviews
128 questionnaires & 14 interviews
17 Vehmas & Kaipio 2018 Finnish Journal of EHealth and EWelfare
To find out if a heuristic usability evaluation method can produce plausible results when carried out by a group of primary care physicians Qualitative
Heuristic walkthrough method / usability evaluation
6 evaluators/novice users & 3 usability
experts
18 Vimarlund & Timpka 2002 Methods of Information in Medicine
To explore the value of end-users participation in information system development projects
Qualitative &
Quantitative (face-to-face) Interviews
Not mentioned / not clear
19 Yuan, Bradley &
Nembhard 2015
BMC Medical Informatics and Decision Makin
To identify super users’ mechanisms of influence and examine their effects on EHR implementation outcomes.
Qualitative & Quantitative
Observations, (in depth) interviews & longitudinal survey data
3.4 Challenges regarding user participation
Three categories of challenges concerning user participation within the healthcare context were identified out of the selected articles. The challenges are subdivided and three categories are determined: a) design challenges, b) system challenges and c) organizational challenges. Each of these challenges are associated with a suitable change in the future design method and are suited to the emerging networked structures used to build modern health service organizations (Pilemalm and Timpka, 2008).
3.4.1 Papers perspectives – relationship between (end-) users and IT professionals Ten of the included articles identified challenges in relation to (end-) user participation. Within all the identified challenges, the relationship between (end-) users and IT
professionals appears continuously. Therefore when differentiating the three different categories of challenges recognized, this relationship is used and seen as the basis for understanding:
• Design challenges: Integrating different perspectives
Based on working together with all the stakeholders within the healthcare organization and more specifically, working with both the (end-) users and IT professionals.
• System challenges: Understanding and usability of the EHR
Based on the fact that the (end-) users want to be involved in the development of the EHR system. (end-) Users want to see that their needs and whishes are taken into consideration by the IT staff (into the development of the EHR system).
• Organizational challenges: Continuous design and development of the EHR To develop and improve the system in the future and to expand the EHR system in their possibilities, (end-) users and IT professionals need to be on the same page and continuing closer collaboration and participation is needed.
Next to the fact that the relationship between (end-) users and IT professionals is presented in the identified challenges, also other articles are describing this relationship. Some examples of phrases in where these articles emphasize this importance and interest in thisrelationship: “Further research is needed to find out more about developers’ attitudes towards
succeed, considerations of usability and user needs should be a part of the requirement elicitation process” [Martikainen, et al., 2012]; “One of the most important things physicians and nurses reported, was the fact that they were not able to positively influence EHR
development without being listened to by IT developers” [Martikainen, Kaipio & Lääveri, 2020]; “System failures are caused by limited understanding of IT professionals towards human factors” [Abugabah & Alfarraj, 2015].
To understand the identified challenges, they are discussed in more depth taking the relationship between (end-) users and IT professionals in mind. Next to the identified
Table VII – Challenges to user participation in a healthcare context
3.4.2 Integrating different perspectives
Within the selected articles this challenge refer to the differences in perspectives of the IT professionals on the one hand, and the (end-) users (nurses and physicians) on the other hand. Pilemalm & Timpka (2008) observed a public health setting and this observation showed them that the relationship between those two parties is in most of the time complicated and sometimes even difficult. Differences in thoughts, interests, positions, status, education level
Dimension Challenges Solving aspects Paper
no. Design Integrating
different perspectives
• Involve (end) users to recognize, evaluate and solve problems (17) • Self-interest and ownership (5,15) • Interdepartmental collaboration (5) 5, 15, 17 System Understanding and usability of the EHR system
• Usability & customizability (3, 16) • Introducing early evaluation of highly
functional tools existing technologies in contextual settings (12)
• Perspectives, values and priorities have to be included in the design process (2) 2, 3, 12, 14, 16 Organizational Continuous design and development of the EHR system R ela tio n sh ip b etw ee n ( en d -) u sers & I T p ro fes sio n a ls
• Assessment and addressing user requirements, attitudes and concerns
(3)
• (end-) users should be partners in the design process, empowering of the (end-) users (2, 17)
• Effective communication (13) • Effective leadership (3) • Evaluation and monitoring of
progress (3)
• Analyze related (external) factors (2) • Creating key user / super users as
translators between multiple world (3, 16, 17, 19)
• Time and labor hours (12)
and norms and values are difficult to overcome. Nerveless, it is necessary to consider these differences in order to optimize the performance and the effectiveness of an EHR system (Pilemalm & Timpka , 2008). Furthermore, Ivatury, Jeena & Prakash (2005), investigated public health programs in India and measured the involvement and participation in a specific EHR system. By doing a case study and attending monthly meetings about the development of the EHR system, they confirmed the fact that different perspectives should be taken into consideration when implementing or developing an EHR system, especially those of the (end-) users and the IT professionals. An ‘unequal power relationship’ between the (end-) users and the IT professionals occurs. IT professionals seem to have more power in comparison to the (end-) users, concerning the development of the EHR system. This is because of their technical knowledge which is necessary to keep the system running in the first place (Høstgaard, Bertelsen & Nøhr, 2011).
The participatory design approach, discussed and evaluated by, among others, Ivatury, Jeena and Prakash (2017) and Vimarlund and Timpka (2002) is an approach to include the (end-) user more into the development process of EHR system and thereby give them more “power” to influence this process. The primary objective of participatory approaches is to include the ‘perspectives, values and priorities’ of the beneficiaries and the stakeholders with any development initiative or project. By doing so, it aims to create a sense of ‘ownership and control’ amongst (end-) users in the development process of the EHR system. One step further is the user-centered design approach. In this approach (end-) users are the most important sources of design information and are recognized as equal partners in the design process. Doing so ensures that the EHR system is useful, usable, elegant, and desirable (Cresswell, et al., 2017). This approach is totally referring to the relationship between (end-) users and IT professionals because of the total integration, and thereby participation of the (end-) users within the design of the EHR system. The aim of the user centered approach is to build and develop the EHR system around the (end-) user and consider the user as the main source for developing and using the EHR system.
3.4.3 Understanding and evaluation of usability of the EHR system
understood their needs or the needs of their patients. The developers are more focused on the technological aspects and the efficiency of the EHR system. The technological features should be more tailored to the (end-) users and their specific needs. This need for more technological features based on the needs and wishes of the (end-) users is also mentioned within the results of Cresswell, et al. (2011), Rahimi, Safdari & Jebraeily (2014) and Van der Meijden, et al. (2001). Customizability and evaluability of the EHR system is important to give the (end-) users a voice, a vote into the design, development and implementation of an EHR system. (End-) Users want to be heard by the IT professionals but mostly, they want their opinion, knowledge and expertise being reflected into the technology. Using evaluating tools in an early stage can help to reach this goal and prevent resistance and/or disaffection among (end-) users (Pilemalm & Timpka, 2008).
Vehmas & Kaipio (2018) explained the need for customizability, evaluability and usability problems, by investigating and discussing different type of evaluators and their
influence/impact (the usability evaluation method). Three types of evaluators described were: a) Evaluators that exist of novice users (physicians or medical students with moderate (2-5 years) working experience and no prior experience in using the EHR in question or in the usability evaluation), b) experienced users (physicians or medical students with moderate (2-5 years) working experience and a minimum of one-year previous experience in continuous use of the EHR in question and no experience in usability evaluation) and c) usability experts ( usability engineering master students or researchers with knowledge and practical
3.4.4 Continuous design and development of the EHR system
A lot of healthcare organizations consist of many organizational layers and several vertical managerial levels. (End-) Users who participate effectively in the development of an EHR system, easily communicate with the managers and IT professionals at the closest level, but less with the managers and IT professionals at higher levels (Pilemalm & Timpka, 2008). Communication between the multiple levels of an healthcare organization needs more attention to eventually stimulate (end-) user participation. Problems related to the EHR system should reach the whole healthcare organization and not only a part of it. The EHR related problems should be solved together as a united organization. This will (Ivatury, Jeena & Prakash, 2005).
To optimize this communication, collaboration and integration between all the multiple levels of the healthcare organization, time and labor hours need to be available (Shah & Robinson, 2017). Unfortunately, within the healthcare context the lack of time is a big organizational challenge. One of the positions/persons who can improve the communication, collaboration and integration are the super users, key champions or key users. In this study the name super users will be used to prevent confusion. Super users are key success factors and a very new and important concept to take into consideration when implementing an EHR system or developing it further. Appointing super user can help to bring the clinical and managemental world together (Van der Meijden, et al., 2001). But, despite the importance and benefits of the super users, Pilemalm & Timpka (2008) also mentioned the challenges towards the deployment of super users. Their study pointed out that it is important that super users keep developing their knowledge and expertise and thereby keep having positive influence during every phase of the EHR development process. Appointing someone as a super user in the pre-implementation phase, does not mean that this super user still is “useful” in the
3.5 The need for new approaches regarding user participation
Within the healthcare context the main focus is on training (end-) users to adapt to poorly designed EHR systems instead of designing systems or approaches to fit their needs and characteristics. Most of the focus within the articles lies on the difficulties to support (end-) users, give (end-) user IT training and manage the process change (Abugabah & Alfarraj, 2015). Those difficulties and the continues focus to adapt to poorly design systems, has in general led to delays in the further development of EHR systems and the related user participation. (End-) User resistance still exists and there is a strong need for a new multidisciplinary approach or framework in where a fresh method will be introduced
(Martikainen, et al., 2012; Mclean, Frisch & Roudsari, 2015; Abugabah & Alfarraj, 2015). It is known from the literature that a successful design and development of EHR systems can increase efficiency and productivity and therefore a more inclusive approach or framework, in the retrospective investigation of previous EHR development approaches, is considered necessary (Abugabah & Alfarraj, 2015).
3.6 Stakeholder analysis
Before the relationship between (end-) users and IT professionals can be optimized, it is important to start with understanding all the parties involved. A stakeholder analysis was used. The aim of the stakeholder analysis process is to develop a strategic view of the human and institutional landscape, the relationships between the different stakeholders and the issues they care about most. The same goes for the need for new approaches. The stakeholders mentioned the most in the different selected articles are the physicians, nurses, physicians AND nurses, super users, managers and IT professionals. The stakeholders are involved in the relationship between the (end-) users and the IT professionals. Their involvement may also be needed in future development of a new approach or framework. A start is made with the stakeholder analysis in this literature review (Figure VIII). It shows that a lot of boxes are not completed yet. The analysis should be completed in the future by gathering more
24
Table VIII – Stakeholder analysis including physicians, nurses, super users, managers and IT professionals views
Power Experiences Vision Interests Informatics knowledge &
experiences Informatic attitude
Physicians High clinical
power (4)
Strong dissatisfaction with the physicians’ abilities to have an impact on system development (10)
Positive vision (10)
(definitely and surprisingly) interested in participating in IT systems development in several ways (10)
Lack of technical/informatics knowledge made them incapable of excerting real
influence (4)
x
High exert real influence on which system to choose (4) A lack of user-centeredness in healthcare IT systems development (10)
Have to accept the new (increasingly standardized version of the) electronic health records (4)
Preferred direct feedback, connection to discussions with developers, and an opportunity to develop IT systems in close collaboration with developers as the best methods of collaboration (10) Little power regarding the participation in developing the EHR system - technological side (4) Physicians’ experiences regarding their participation were quite negative, developers do not understand their needs (10)
To ensure positive clinical benefits (4)
Experiencing a lot of time pressure (4) Frustrating experiences, their feedback and input never received a response or notification that it had reached the IT staff/developers (10)
Nurses High clinical
power (4)
Nurses felled welcome except for a few rejections, even more when the IT staff and nurses began to work as a team (11)
x x
Nurses indicated this work has real meaning for them and (extra) informatics knowledge is for them a purposeful activity (11)
Nurses seek/need/want support from their expert IT colleagues (11)
Nurses feel like they have personally influenced the upgrading of the EHR system when actively being involved in the evaluation stage of the project (11)
Nurses seek/need/want the IT staff/developers to believe them, to believe in their expertise, input and knowledge (11)
Physicians AND nurses
High clinical power (4)
Many willing clinicians remain underutilized and more working time dedicated to and reducing earlier negative experiences with EHR development is necessary (11)
Both end users think that processes should be more visible to the end users (9, 11)
Physicians were more critical In
their feedback than the nurses (9) x
Both end users think that developers must consider how their changes impact the whole system and the (end-) users (9) Nurses were more knowledgeable
about how to provide EHR feedback (9)
Both end users feel ignored if the timespan between providing feedback and possible correction is too long
Physicians are (more) willing to participate in user groups (11) Older physicians & nurses are more knowledgeable on how and to whom they could send their feedbackand they particpate more often in EHR development (9)
Younger physicians & nurses are more willing to participate in EHR development but 70.% of the younger physicians and nurses had never participated in EHR development (9) Super users High, when being an good translator between the clinical and IT world (19) x x
High, when voluntarily registered for the position and seeing the position as a oppertunity (19)
x x
Low, when technological based registered for the position and seeing the position as an burdensome (19)
IT staff/develop
ers
High power to support (the end-users) (4)
x x Optimizing administrative functions
in the EHR (4) High (4) x
High technical knowledge (4)
High
organizational x x No attempts to learn from other
3.6.1 Physicians
The interviews, conducted by Martikainen, et al. (2012), showed that almost half of the responding physicians disagreed with the statement, “When I want to give feedback I know to whom and how I can send it”. Only 13.3% of all respondents (physicians) in the study of Martikainen, et al. (2012) thought that IT professionals were interested in (end-) users’ feedback. A significant number of the physicians were even disappointed with the ability to produce corrections and changes rapidly and in a desired manner. Some physicians even mentioned the word dissatisfaction and referred back to the relationship with the IT
professionals and the corresponding (managerial) actions they take regarding the feedback, opinions and ideas they submit. For most of the physicians it feels like they have no impact on the development of the EHR system or the efficiency of it. This despite the fact that they are very interested in the way the EHR system is working and developing. They want to ensure that positive clinical benefits are reached and that the system works in “their favor”, taking their clinical position in mind. But it seemed that despite their high clinical power, they have very little participatory power to gain positive clinical benefits. The lack of technical knowledge made the physicians incapable of exerting real influence on several important decisions made during the development process (Høstgaard, Bertelsen & Nøhr, 2011).
3.6.2 Nurses
Nurses also want to be involved as much as possible in the development process of the EHR system. They want to be part of the development and personally influence the upgrading and success of the system (Mclean, Frisch & Roudsari, 2015). Only about the half of the
high clinical power but they sometimes felt overwhelmed by the physicians. They felt
underrated and this makes that they are more willing to understand the informatics and the IT professionals better and in more depth. Nurses indicated that understanding the IT
professionals and their work better, has real meaning for them and therefore (extra)
informatics knowledge is for them a purposeful activity. They are more willing to learn the other side of the EHR system. More than the physicians (Mclean, Frisch & Roudsari, 2015).
3.6.3 Nurses AND physicians
To continue on the influence of age, Martikainen, Kaipio & Lääveri (2020) investigated the impact of age on the experiences of both the nurses and physicians regarding the use and development of an EHR system. Out of their surveys it became clear that the older (> 45 years) physicians and nurses were more knowledgeable than the younger ones, on how and to whom they could provide feedback about the EHR. In the youngest age group (< 35 years) at the other hand, 70.4 % of the (youngest) physicians and nurses had never participated in the EHR development, but they are the most willing to participate. An assumption is that the younger ones do not get a chance to make themselves heard because the older nurses and physicians are in the lead. Maybe their (work-) environment is not developed in such a way that every nurse and physician, with every age, experience and knowledge, get the same chance to participate within the (development of the) EHR system. A survey also showed that the youngest age group prefer communicating with a person in charge of EHR development to provide feedback and to participate more and better in the EHR system. This is interesting because the other option the authors presented to the young respondents, namely providing feedback through a website, was less popular. So even younger people do still prefer communicating face to face despite the world wide digitalization and the assumption that they want to do everything online. It seems that they need the face to face approach and that they want to be heard and respected by the organization.
the table”? It seems that nurses wants to make their voice heard. Also the physicians in the article of Martikainen, et al. (2012) showed that they wanted to be more involved by the IT professionals. The physicians argued that the IT professionals do not understand their needs because the IT professionals never observed their work in the hospital. They also mentioned the fact that, although they gave feedback and improvement ideas, they never received a response or notification that the message had reached the IT professionals. This makes that a lot of physicians have a negative experience regarding the relationship with IT professionals. The physician's requests more communication between them and the IT professionals
because momentarily, this communication is lacking. They want to recognize their given feedback back into new applications or improvements within the EHR system. Martikainen, Kaipio & Lääveri (2020), in which they compare the nurses against the physician's view, are also investigating this subject. Physicians and nurses reported similar experiences regarding their feedback to EHR developers. A few (learning) points they gave were the fact that the IT professionals need to do more with the given feedback and the time between providing feedback and possible correction should be shorter. Nowadays this time slot is too long by which the (end-) users may feel ignored. Furthermore, they must consider more how their changes impact the whole system and other users. At the end the main “vision” of both the physicians and nurses is that processes should be more visible to (end-) users.
3.6.4 Super users
Yuan, Bradley & Nembhard (2015) zoomed in more on a more specific view, the one of the super users. Regarding to those authors, it is a common recommendation to enlist super users or individuals, which already work within the organization. They should receive additional training on the use and functionalities of the selected EHR system so that they can provide frontline technical support to their peer users, to foster the successful implementation of EHRs. Also, Cresswell, et al. (2015), emphasized the fact that there is evidence that the appointment of clinical leads and ‘boundary spanners’ can be effective as these individuals often have an insight into ‘both worlds’: management and clinical. Super users can be very effective and efficient when “using them right”. If their additional value will be appreciated and exploited more, their power to link and let the (end-) users and IT professionals
communicate, can be really high.
show the importance, influence but most of all the added value of the super users to improve the relationship between (end-) users and IT professionals. Out of those (existing) key roles, the needed key behaviors of super users to be influential in the development of EHR systems, are developed. Bradley & Nembhard (2015) observed, interviewed and gained survey data of two different units of a large, academic hospital which was implementing an EHR system.
Point 1 in Table IX showed the key roles a super users must own in order to be a “good and valuable” super user. Point 2 in Table IX presents the general and overlapping behaviors which occurs in both units. Those behaviors are “normal” and general behaviors for super users. Furthermore, those behaviors support the implementation process. Both point 1 and 2 should be part of the selection process in order to function as super users in a healthcare organization. If someone meets both the key roles and supportive behaviors, he or she is a potential candidate for the job and attention can be given to the influential behaviors. These are behaviors which can ensure that a super user is successful and even is the EHR system. Point 3 in Table IX, described these influential behaviors. The green marked behaviors are the ones that are influential and occur in the unit in which the EHR implementation and development is successful. The non-green-marked behaviors are behaviors of super users that occur in the units in which the EHR implementation and the development process are not successful. Those behaviors should be excluded in the selection process towards the best super user.
The first point why those differences occur, stemmed from differences in role engagement. Interviewees’ comments suggested that super users’ role engagement was primarily shaped by two contextual factors related to managers’ actions at the frontlines of the EHR
them. The second contextual factor is the implementation climate. When gaining a climate where actively supporting the hospital’s implementation policies, rewarding extra role behaviors and modeling proactive behaviors, a lot more engagement can be achieved.
So, there seems to be four key behaviors which should be considered in the selection process of super users: proactiveness, depth of explanation (emphasizing), positive framing and constantly information sharing (see the green highlights in Table IX). Those behaviors are presented within the units with the greatest implementation success, in which super users are voluntary selected and the hospital is given those super user the opportunity to participate and to help develop the EHR system (regarding the work-environment).
Table IX – Key roles and behaviors of super user (1,2 and 3 respectively should be considered in the selection process)
1 Key roles of Super users [19] 2 Super user behaviors that support implementation [19]
3 Super user behaviors that creates greater
implementation success [19] 1.1 Serving as project liaison
between the EHR developer and clinical team
2.1 Reporting problems with
the EHR to someone in a position to fix it
3.1 Proactivity
3.1.1 Proactively (key behavior) 3.1.2 Reactively
1.2 Facilitator during EHR
training sessions
2.2 Employing teaching
strategies that promoted “learning by doing”
3.2 Depth of explanation
3.2.1 Emphasizing(key behavior)
3.2.2 Demonstrating
1.3 Role model for their
colleagues
2.3 Providing extra support to
individuals struggling with the change 3.3 Framing 3.3.1 Positive frames(key behavior) 3.3.2 Neutral frames 1.4 Technology “ambassador”
who helps foster acceptance of change
3.4 Information sharing
3.4.1 Constantly sharing(key behavior)
3.4.2 Limiting the spread of sharing
3.6.5 IT professionals and managers
Only in one article information is found about the perspectives of the IT professionals and the managers (Bertelsen & Nøhr (2011). Their results shows that the biggest interest of IT
professionals is the administrative side of the EHR system. They live in their own “IT-world” and do not see a lot of other interests, for example the ones of the nurses and physicians. Furthermore, the IT professionals do have a really high share of power because of their technical knowledge. Because the IT professionals nowadays are only doing their part of the job, and physicians and nurses do not have the time to participate enough to for example attend meetings, IT professionals with a past clinical background (non-physicians) developed most of the medical demands in the requirement specifications. This can be a good start, but it is also a great risk. By not adding the value of the physicians and/or nurses, the medical requirements do not reflect the clinical reality, because experience shows that professional knowledge has to be presented by the professionals. At the same time, it implies a great risk in that the physicians’ interests were not, or only partially, met during the process. Because of the lack of time related sources and the lack of influence within the development of the EHR system regarding the physicians and/or nurses, most of the time the IT professionals made the final choice regarding the EHR system according to technical and economic criteria.
Furthermore, it seemed that the will and drive to learn and to develop the EHR system is missing among the managers within the (investigated) hospital. During the planning process, for example, the management made no attempts to learn from the experiences of EHR
planning-processes in the other Danish counties and hospitals. The fact that no sharing of past experiences at any level (strategic, tactical and operational) took place before or during the EHR planning process could indicate that the management had underestimated the workload associated with the planning process. So the conclusion, made out of this very little evidence and research to the influence and importance of managers, is that the management should be more actively support end user participation throughout the process. They must provide the resources necessary in terms of time and personnel, but most of all, they should be the heart of the development and the link between the clinical and the IT world because those
4. DISCUSSION
The main findings of this literature review showed a large discrepancy between the perspectives of the clinical (end-) users and the technical IT professionals. This causes a situation where the role and effectiveness of (end-) users participation is less beneficial to the use and development of an EHR system. The organizational relationship and the planned collaboration between the clinical (end-) users and the technical IT professionals, who design, build and maintain an EHR system, is therefore the most valuable part of the user
participation concept to consider.
Results from the studies included in this literature review indicated that collaboration and communication between the clinical (end-) users and IT stakeholders is difficult. (End-) Users have a different view on the processes incorporated in the EHR system and the information derived from it. Their expertise, skills, values and education are related to a clinical working environment, in where the patient has the highest priority. The IT
professionals on the other hand, are working from a technical point of view, where an cost and time effective, user friendly IT system has the highest priority. Because of those different perspectives, the two different stakeholders are not working in the same environment, do not have the same priorities and education and may even have different interests. This can result in miscommunication, misunderstanding and not working into the same direction.
Furthermore, the results of this literature review indicated that (end-) users feel a lack of understanding and appreciation by the IT professionals towards their commitment, knowledge and participation (Kaipio & Lääveri, 2020; Martikainen, et al., 2012). The commitment of clinical (end-) users to develop and constantly improve the EHR system is high, and should be valued more by the IT professionals (Martikainen, Kaipio & Lääveri, 2020). (End-) Users know that their input can lead to better results if there would be a better understanding and collaboration with the IT professionals. In order to establish this
collaboration more emphasis is needed onprofessionally supervising the process. This can be achieved bymaking a project leader or manager responsible for this important task. The effect and importance of such an additional managerial role in user participation is not
indirectly to create more collaboration between the clinical and the IT stakeholders, is missing among the managers in healthcare organizations (Høstgaard, Bertelsen & Nøhr, 2011). It has not become clear whether this managerial responsibility is missing because they do not have the skills and knowledge or because they are not well instructed and trained.
An important aspect of the concept of user participation are the group of super users. Super users act as translators between the clinical and IT environment (Yuan, Bradley & Nembhard, 2015). Managers should value the role of superusers more, because they are able to decrease the divide between the different perspectives of the clinical (end-) users and IT professionals more. Our findings showed that empowering super users can be valuable to stimulate and support user participation more. To achieve this empowerment capable super users need to be selected. The most important selection criteria are that super users are selected on a voluntary base, are motivated, have useful skills and expertise and in the end feature the key roles and behaviors necessary to be valuable to the healthcare organization. Furthermore, a stimulating climate in where super users are valued and encouraged should be created. An important aspect of the role of super users to consider as a healthcare organization, is the different added value that a super user can bring at consecutive stages of the EHR system development and implementation. The personal development and skills of the super users need to be closely monitored, observed and evaluated. The super users need to develop together with the EHR system. It could be, that the super user at the beginning of the implementation are very useful to guide the (other) (end-) users, but that at a later stage, they no longer have the right skills, knowledge, time or willingness to keep learning and developing alongside the EHR system.
Our literature review showed that the specific roles of super users, managers and IT professionals need more attention and clarification. In all the studied articles these stakeholders are seen as a “black box”. The development of the user participation theory could benefit if, besides the more widely presented views of physicians and nurses ((end-) users) in the user participation literature (Høstgaard, Bertelsen & Nøhr, 2011; Ivatury, Jeena & Prakash, 2017; Martikainen, Kaipio & Lääveri, 2020; Martikainen, et al., 2012; Mclean, Frisch & Roudsari, 2015), the role of the super users, managers and IT professionals are taken into account. Furthermore, although the development process of an EHR system still mostly evolve around the technical aspect of the system, (end-) users are more and more becoming an integrated part of the development. To gain even more benefits from the (end-) users and to optimize and utilize the EHR system even better in the future, a refreshing and applicable framework, that fit (end-) users’ needs and characteristics, is needed and
recommended. A new user-participation-focused framework can gradually change the practice and understanding of user participation, and the multi-disciplinary collaboration in the EHR design and development. Such a framework can serve as a design process that (maybe) will lead to better results and more benefits of the EHR system what will express itself into higher quality of care, more acceptance of the (end-) users, less pressure on health professionals and a united healthcare organization. To develop such a framework, time and labor need to be available and a cooperation should be initiated with (end-) users who are the most willing to learn (the younger ones) and the ones with the biggest influence (the older ones). Our findings show, that the youngest (end-) users prefer to learn about the EHR system and the use of it, but that they do not get the chance in the current healthcare organizations. Theyounger (end-) users are not respected and valued within the organization. It seemed that the older (end-) users are positioned higher in the “healthcare-hierarchy” and thereby gaining more benefits and power over the younger (end-) users. Older (end-) users have little interest in the system's development process, but do have a lot of influence among the IT
4.1 Limitations
There was a limited number of articles, which described user participation in the healthcare context, available for this literature review. Next to the limited number of studies, the content and the used methods and objectives of the selected studies differed a lot and made it difficult to draw a clear conclusion. They were not compatible regarding the interpretation and
investigation of user participation within the healthcare context. This limitation can be resolved in the future by analyzing, discussing and comparing the concept of user
participation within the healthcare context to articles outside the healthcare context. Doing so, the differences between the “regular” organizations and the healthcare organization will become more visible.
4.2 Future research
5. CONCLUSION
EHR implementation is a complex and multi-dimensional process. In this literature review we studied the role of user participation in this process. User participation seems to be essential in optimizing and utilizing the EHR system. Collaboration and communication between the different stakeholders (clinical vs. IT) involved in the EHR system development is decisive for effective user participation. Power of both stakeholders need to be in more balance in order to optimize and utilize the EHR system. The existing divide between the clinical and IT stakeholders weaken the effective use and the associated benefits of the EHR system. It also delays the development of it. This literature review presented this divide as the main problem and therefore the most important part of the concept of user participation. To minimize this divide by involving other participants could play an important role. More attention should be paid to the collaborative tasks and actions of managers, super user and IT professionals. Decision-makers should draw upon the findings of this review in order to improve the concept of user participation within the healthcare context. Healthcare
organizations must overcome a lot of (additional) challenges and participating of (end-) users requires time, money and energy of all stakeholders before it brings benefits to the
organization. EHR systems have proven to be very innovative and important for the healthcare system. Continues implementation and improvement of an EHR system is
REFERENCES
[1] Abugabah, A. and Alfarraj, O. (2015). Issues to consider in designing health care information systems: A user-centred design approach. Electronic Journal of Health Informatics, 9 (1), 2-15.
[2] Hossain, A., Quaresma., R. andRahman, H. (2019). Investigating factors influencing the physicians’ adoption of electronic health record (EHR) in healthcare system of Bangladesh: An empirical study. International Journal of Information Management, 44, 76–87.
[3] Barki, H. and Hartwick, J. (1994). Measuring User Participation, User Involvement, and User Attitude. MIS Quarterly, 18(1), 59-82.
[4] Bleich, H.W. and Slack, W.V. (2010). Reflections on electronic medical records: When doctors will use them and when they will not. International journal of medical informatics, 79, 1-4.
[5] Boonstra, A. and Broekhuis, M. (2010). Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Services Research, 10 (231), 1-17.
[6] Cresswell, K. M., Lee, L., Mozaffar, H. and Williams, R., Sheikh. (2017). Sustained User Engagement in Health Information Technology: The Long Road from Implementation to System Optimization of Computerized Physician Order Entry and Clinical Decision Support Systems for Prescribing in Hospitals in England. Health Services Research, 52 (2), 1928-1957.
[7] Cresswell, K., Morrison, Z., Crowe, S., Robertson, A. and Sheikh, A. (2011). Anything but engaged: User involvement in the context of a national electronic health record
implementation. Informatics in Primary Care, 19, 191-206.
[8] Hartwick, J. and Barki, H. (1994). Explaining the Role of User Participation in Information System Use. Management Science, 40 (4), 440-465.
[9] Høstgaard, A., Bertelsen, P. and Nøhr, C. (2011). Methods to identify, study and understand End-user participation in HIT development. BMC Medical Informatics and Decision Making, 11 (57), 1-11.
[10] Ivatury, P., Jeena, A. and Prakash, A. (2017). Relevance of participatory approaches in creating ICT systems for public health programmes. ACM International Conference
