• No results found

Rijksuniversiteit Groningen Faculteit

N/A
N/A
Protected

Academic year: 2021

Share "Rijksuniversiteit Groningen Faculteit"

Copied!
35
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

Student: Peter Reezigt

Studentnummer: 1656236

Titel afstudeeropdracht: An Intervention Model for Physicians’ Barriers towards Electronic Medical

Record Adoption

Maand en jaar: juni 2008

Nummer afstudeeropdracht: NB

(2)

An Intervention Model for Physicians’ Barriers

towards Electronic Medical Record Adoption

By

Peter Reezigt

University of Groningen

(3)

CONTENTS

1 INTRODUCTION _____________________________________________________________________ 4 2 INTERVENTIONS AND RELATED BARRIERS ___________________________________________ 7 3 MEASURING AND LOCATING BARRIERS IN PRACTICE _______________________________ 14 4 INTERVENTION PLAN_______________________________________________________________ 23 5 DISCUSSION ________________________________________________________________________ 28 6 CONCLUSION_______________________________________________________________________ 30 7 REFERENCES_______________________________________________________________________ 31 APPENDIX A: Pre-requirement checklist ___________________________________________________ 34 APPENDIX B: Barrier list for practice _____________________________________________________ 35

ABSTRACT

Objective: To introduce interventions and to link these interventions to the potential presence of barriers from

physicians towards the use of EMRs and synonyms.

Data sources: Articles (2002 to 2008) which displayed studies that evaluated potential barriers from physicians,

medical specialists and general practitioners towards the use of EMR and synonyms, were searched in 3 databases. Articles were found in the following databases; Google Scholar, Science-Direct and EBSCO. Articles were selected based upon a search criterion which contained the introduced target group, one of the introduced Information Systems (EMR) and the following subject terms; barriers, resistance, obstacles and interventions towards EMR.

Method: Articles which were approved by the search criterion were read and analyzed with an extensive search

on potential interventions.

Results: There were a number of 41 articles found and used for this literature study, based on barriers and

interventions towards EMR from physicians. Most paper handled different research methods and discussed more than one barrier and intervention. In total, 5 areas of interventions were found along with 20 interventions, recognized by several researchers. These areas consist of Organizational Interventions (3), Motivational Interventions (5), Selection Interventions (3), Technical Interventions (4) and Financial Interventions (5). Each intervention in each area is linked to a barrier. The presented material and the practical use of it is introduced in a so-called intervention model.

Conclusion: Two surveys can be used to locate presence and the importance of statements linked to barriers.

(4)

1 INTRODUCTION

(5)

Areas Interventions Supported Barriers

1. Redesigning workflow to get a better organizational fit F2, F3

2. Select product champions and provide managerial support A3, A6, C1, C2

O rg a n iz a t. In te rv en ti o n s

3. Establish a conflict / open culture before implementation A3, A5, D1 4. Educate employees and support ongoing trainings A1, A3, E2, F3 5. Performance incentives and mandates A3, A4, A7, F1, G2 6. Dictation and voice recognition for non-typers A1, A3, F1, G1

7. Let employees participate and inform them about their roles A3, A4, A5, D1, F1, G3

M o ti v a ti o n a l In te rv en ti o n s

8. Present suspected positive outcomes of EMR to physicians A3, A4, A5, B4, D2, E1, F1, G1, H3, H7

9. Choose a system based on an evaluation form A2, A3, B1, B2, B3, B4, E1, E4, F2, H1

10. Analyze and select vendor with the best and unique support B1, B2, E4

S el ec ti o n In te rv en ti o n s

11. Acquire third party consult for selection of EMR / vendor A3, A5, B1, B4, D1, E1, F2, F3, G3 12. Communitywide data exchange and network standards A3, D2, E1, E4, F1

13. Optimize software, upgrade hardware and tune networks E3, E4, F1, F2, F4 14. Provide internet access to the EMR A3, A4, F1, F4

T ec h n ic a l In te rv en ti o n s

15. Hiring a computer company A1, E1, F3 16. Purchase additional components in a later stadium F2, H2, H3 17. Documented ROI B1, B3, H3, H4 18. Accelerate Implementation Time H5

19. ASP technology H6 F in a n ci a l In te rv en ti o n s

20. Additional “backfill personnel” / patients reduction H2, F3

(6)

Areas Barriers Supported Interventions

A1) Lack of individual technical expertise 6, 15

A2) Lack of experimentation and piloting abilities 9

A3) Individual / Cultural resistance 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 14

A4) No personal benefits 5, 7, 8, 14

A5) No participation in selection / implementation process 3, 7, 8, 11

A6) Groupsize – adoption rate 2

(A ) H u m a n

A7) Lack of incentives 5

B1) Difficulty to choose vendor / system 9, 10, 11, 17

B2) Vendor’s support / product deliveries 9, 10

B3) Lack of financial model / data from vendor 9, 17

(B ) S el ec ti o n

B4) Uncertain about quality improvements of EMRs 8, 9, 11

C1) Lack of project champions / management techniques 2

(C

)

L

S

C2) Lack of management support 2

D1) Lost of autonomy 3, 7, 11 (D ) L eg a l

D2) Privacy & security issues 8, 12

E1) Interoperability issues / Lack of standards 8, 9, 11, 12, 15

E2) Complexity of system 4

E3) Slow system speed 13

(E ) T ec h n ic a l

E4) Unreliable technology 9, 10, 12, 13

F1) Decrease in accuracy, unreliability and prod. of work 5, 6, 7, 8, 12, 13, 14

F2) Misfit with current working routines 1, 9, 11, 13, 16

F3) Complex to migrate from paper-based to EMR 1, 4, 11, 15, 20

(F ) P ro ce d u ra l

F4) Lack of unavailability computer 13, 14

G1) Time to enter data 6, 8

G2) Time to learn program / doing trainings 5

(G

)

T

im

e

G3) Time to select, contract & implement the system 7, 11

H1) Doubts about financial return of system 9

H2) High initial medical specialist time costs 16, 20

H3) High initial costs 8, 16, 17

H4) Most of suspected benefit goes to payers / consumers 17

H5) Long timescale system’s suspected payoff 18

H6) Costs to maintain the system 19

(H ) F in a n ci a l

H7) Responsibility for purchase costs 8

(7)

2 INTERVENTIONS AND RELATED BARRIERS

In the first article, articles were analyzed based on the presence of barriers and relevant interventions for those barriers. This resulted in a list of 20 interventions, divided in five areas. These areas are being introduced as the following; Organizational Interventions, Motivational Interventions, Selection Interventions, Technical Interventions and Financial Interventions. The interventions are described in this section. The barriers that are supported by each intervention can be found in table 1 and table 2, which are displayed in the previous two pages. When a part of the described intervention recognizes the support for a barrier, this barrier is presented between anchors in the described section of each intervention. The support of interventions for barriers is mostly recognized from the literature. In some situations, it is also the case that interventions are in a logical way coupled to their barriers. Note that the description of each intervention should give managers an impression about how to handle and implement these interventions in practice. Each intervention can not be seen as a direct solution; it functions as a tool to give an impression about how to handle problematic issues. Handling these issues with interventions should be seen as an extensive activity, characterized by its unique environment. It could also be the case that some interventions are detailed described and others are less detailed described. The exact steps that managers need to take for intervention falls outside the boundaries of this study and is, therefore, limited discussed in this study.

Organizational Interventions

1. Redesigning workflow to get a better organizational fit: Before technology will be implemented, workflows

need to be redesigned. The leading idea behind this is that it should have a better fit with the purchased EMR. According to Stefan, staff needs to redesign all relevant workflows which will fit correctly with an EMR environment (Stefan, 2005). This means an organization needs to correct problems and eliminate work-arounds1. In addition, after implementation workflows need to be redesigned further, in order to generate the highest fit with the new EMR. Pizzeferri et al. argue that it is wise to let consultants support this process, since this can be an extensive and specialized activity. It is not only a difficult and hard process to realize, but also a process with a huge impact on the organization (Pizzerferri et al, 2004). This intervention should support the fit with current working routines (Barrier F2) and it should make the migration process to an electronic environment with EMR simpler, since workflows are redesigned to have a better fit with EMR (Barrier F3).

2. Select product champions and provide managerial support: A change situation is successful when participants

could make use of managerial support and when there is a so-called product champion present. EMR champions are selected and eventually during the entire process, created. These EMR product champions have positive attitudes towards solving problems and try to motivate other physicians. They are wiling to bear initial financial and time costs to generate benefits. According to Miller & Sim, clinics without the presence of EMR product champions may flounder in their efforts to generate quality or financial benefits from EMRs (Miller & Sim, 2004). According to Keshavjee, EMR product champions are the right actors with good skills like planning and evaluating EMR, along with a participative management style. When there are conflicts, this champion should direct and solve them. However, in large practices, Keshavjee argues that it should be better to set-up a form of an EMR committee. He argues that this committee could have more impact in the entire project since we talk

1

(8)

about multiple members instead of a single champion. These members should enable a positive and stimulating working environment (Keshavjee, 2006). This issue is relevant for the barrier “group-size adoption rate”. Besides introducing this form of stimulating leadership, every participant should get managerial support from the top. Before the actual implementation of an EMR, there are major software, hardware and professional role change decisions where clinics are not even aware of. Keshavjee argues that commitment from the top should support these difficult issues, as well as resource allocation, support of redesign and the facilitation of implementation steps. This argument is supported by Pizziferri et al., where they acknowledge the fact that managerial support is the key to an organized and successful EMR implementation (Pizziferri et al., 2004).

3. Establish a conflict / open culture before implementation: A strong form of leadership should translate the

vision of an EMR and its desired goals to a high rate of success. Together with a culture based on conflict, open relationships and trust, this should support the adoption of EMR (Barrier A6). Note, that conflict in this context can be seen as culture with discussions, rather than arguments. Stefan argues that is it important to select the right people for the rights job. Teams should be created in order to achieve the highest adoption ratio of EMRs. According to Stefan, it is essential to build the right MIS project staff. This staff should consist of people with recognized and respected clinical experience and leadership skills. In addition, technical experience can be seen as a bonus. Building teams with these skills and experience will help manage the most difficult aspects of winning over physicians and other users (Stefan, 2005). Keshavjee (2006) recognizes 4 critical points which need attention in order to be successful. These points present the desired attitude and culture towards EMR implementation:

 Regular staff meetings

 Opportunities for discussing and venting (conflict / open culture)  Evaluation, monitoring and tracking progress of implementation

 Systems to track issues and problems with a process in place to solve them

Notice that the first two points represent an open culture which may lead to discussions and having conflicts between participants. This should enable positive feedback where physicians are feeling committed to the ERM project. The 4th point represents generally the basic idea which is discussed in the implementation plan. An open culture also recognizes each participant’s opinion, which results in the fact that each participant has influence in the process and that his / her opinion counts and is evaluated. The organizational culture, however, can also be a major obstacle when it does not match with desired attitude required for the implementation phase of EMR. In the Kaisler case, Øvretveit et al. argued that a no-conflict culture led to feedback not being openly expressed. This automatically resulted in resistance to the EMR project, since physicians felt they had a lack of influence and power in the project. Since these cultural aspects can be different and they have influence on the success of EMR implementation, EMR implementation can be seen as a “conditional intervention”, since the success of EMR, seen as an intervention is dependent on many factors (Øvretveit et al, 2007). By the participative attitude of this intervention, physicians become involved (Barrier A5), have the ability to influence the project with their visionary opinion (Barrier D1), which could result in less resistance to the adoption of EMR by physicians (Barrier A3).

Motivational Interventions

4. Educate employees and support ongoing trainings: In order to adopt and to use an EMR, a comprehensive

(9)

functionality. Stefan recognizes a training plan with competences involved which each physicians needs to pass to be able to work with EMRs (Stefan, 2005). Trainings and education should stimulate the commitment and understanding of physicians towards EMRs. When physicians performed some training assessments, there is a good chance that they may be less fearful and less focussed on what they must know. In addition, they enlarge their technical expertise and become more connected with EMR and its functionality (Barrier A1). This should generate a proactive attitude from each physician towards to the EMR system. It is also useful to let physicians work with EMRs in practice, while guided by super-users or professional trainers. In this way, the training should become more effective, since the training is performed in the actual practice. Furthermore, professional trainers can provide direct feedback to each physician, which should increase the understanding and the commitment of the physician towards the complex EMR system (Barrier E2). It is important that trainings are not only initial but

on-going. On-going trainings sharpen physicians’ knowledge and make them aware of software add-ons and upgrades provided by vendors. It should be clear that proper trainings enabled by a flexible and attractive trainings plan, is one of the key success factors for smooth transition to a paperless patient care system (Keshavjee, 2006; Tang et al., 2006; Boorady, 2006) (Barrier F3).

5. Performance incentives and mandates: When physicians experience a lack of personal benefit from an

implemented EMR, individual and cultural resistance will give rise which result in not using the EMR in its optimal form (Clayton et al., 2005; Pizziferri et al., 2004; Anderson 2007). Therefore, it is extremely important to stimulate the use of EMR by providing financial incentives, mandates and rewarding systems (Barrier A4 & A7). These forms of stimulation should increase the adoption and use of EMR for quality improvement (Barrier F1). Miller & Sim introduce in their article the “pay-for-performance” initiative which started in 2003 in California. They argue that adoption of pay-for-performance programs by both Medicare and private payers could have a powerful effect on accelerating EMR adoption and use (Miller & Sim, 2004). However, these rewarding systems are difficult to design, since it is quite difficult to define a form of performance. Many clinics and other business units struggle with this issue. Managers in a clinic should decide which sort of financial system fits best in their organization. Priorities and goals for this system need to be set in order to measure its performance. It should be clear that a financial compensation which can be earned for each stakeholder in an EMR project should stimulate the adoption of it. Even after implementation financial incentives are also important. Continuing incentives should further stimulate the use of EMRs (Keshavjee, 2006).

6. Dictation and voice recognition for non-typers: There are multiple methods in the market which could be used

(10)

7. Let employees participate and inform them about their roles: Achieving a high level of commitment from

physicians towards an EMR project can be achieved by informing physicians. In addition, it can be further stimulated by participating them in the entire process (Barrier A3 & A5). According to Stefan, this means managers need to involve physicians in all aspects of EMR implementation, including the selection of the actual EMR system, system build design, workflow redesign and trainings and education (Stefan, 2005). The adoption rate is a key factor for defining success of the project. By informing and participating physicians, they should get a feeling of ownership regarding this project (Barrier A4), which should lead to a higher adoption rate of the systems. It is recognizable that physicians are sensitive for this aspect, since they are afraid of loosing their status of autonomy which was recognized as a barrier towards adoption (Barrier D1). Participation can be translated to giving physicians typical roles, such as managerial functions and product champions. In the Kaisler case handled by Øvretveit et al., physicians reported they needed better information about how much time and money they should set aside for this project (Barrier G3). Education and information was not as good as it could have been. Many physicians were not aware on how to use project techniques. By the lack of information, physicians found it hard to know what their position was in the project and what managers expected from them. These aspects hindered implementation (Øvretveit et al, 2007). Therefore, physicians need to know what kind of role they will have in the project and what their responsibility is. There should be an potential presence of personal commitment to the EMR project, when this intervention is addressed in practice. Each level in the organization should be involved in different ways, with clear parameters about which decisions can be made locally and which require higher-level. In addition, managers should present their working methods and planning to physicians. Finally, everyone should commit to going live with an EMR program in a specified time period (Boorady, 2006). This is supported by Øvretveit et al., where they argue that the decision about the system should be participatory, but once made, implementation should be direct and driven (Øvretveit et al, 2007).

8. Present suspected positive outcomes of EMR to physicians: An obvious but still important intervention can be

the presentation of the suspected advantages and outcomes of EMR. First of all, this provides physicians with a general idea about the system and its potentials (Barrier A3 & A5). Physicians become more attracted to the project and should have a better understanding what reasons the management have with introduction of the EMR system. If managers promote these advantages in a positive manner, this should help with the adoption of this system by physicians. Typical advantages of EMR, recognized by Winn (2002), Hier (2002) and Pondrom (2007) are:

 Increased access / efficiency of information (Barrier F1 & H7)

 Secure patient information with authority levels to view information (Barrier D2)  Improved documentation since less errors will be made (Barrier B4 & F1)  Decision support add-ons

 After transformation to EMR there will be space left for other rooms

 Most insurance companies reduce malpractice premiums by 10 percent (Barrier H3)  Connectivity with other IT systems (Barrier E1)

 No re-entry but re-use of data (Barrier G1)

(11)

Selection Interventions

9. Choose a system based on an evaluation form: Selecting a system which is suitable with organizational

workflows will have positive outcomes (Barrier F2). It results in minor modifications of the system and higher adoption rate of physicians’ EMR acceptance (Barrier A3), since physicians will not need to change their working habits and routines. Therefore, it is wisely to choose a system which allows a wide range of needs to be met. In addition, it is also wisely to choose an already tried and tested EMR, which worked good for other organizations in the market (Barrier B4, E1, E4 & H1). Physicians need to have the ability to read evaluations of others who are using EMR so they can try to imagine whether these particular systems will work for their working methods (Barrier A2). Furthermore, the system should be easy to modify when needed (Øvretveit et al, 2007). Before this selection, it is wise to set-up a list of features that an EMR should provide and which are important for practice. Since EMRs provide a wide range of abilities and functionality, priority between each function and ability needs to be made. In a later stadium, an organization should have the ability to expand their EMR in functionality. With this list of demanded features, a clinic has the opportunity to use this list as a tool to compare vendor’s different products (Barrier B1, B2 & B3), which should help them with the selection of the right EMR (Mehta & Partin, 2007). Holbrook also mentions this advisable step. When an evaluation form or list of demanded features is created, clinics should do a broad search of available EMR systems, make a brief review of each available EMR system and they should visit the vendor’s site. Eventually, the best suitable EMR system with has the highest organizational fit should be chosen, in order to have the highest rate of success after the implementation phase (Holbrook, 2003).

10. Analyze and select vendor with the best and unique support: It is well recognized in the literature that vendor

mostly provide only technical support (Miller & Sim, 2004). Physicians will benefit from vendors who also provide expert knowledge, value and a customized approach besides technical support. For example, Stefan argues that vendors, who understand healthcare issues like HIPAA and patient care, will bring value to the EMR implementation (Stefan, 2005). A vendor is a wisely choice when a vendor can translate healthcare needs to suitable IT functions and systems (Barrier B1 & B2). Since many hospitals, clinics and pharmacies are different in size and other characteristics, vendors should provide a customized support towards each organization. ENH, an healthcare agency described in the case of Stefan, argues that the success of EMR implementations is related to two technological factors; the selected vendor needs to be willing to modify and enhance the application to meet the organization’s needs and a vendor needs to provide support over the long term. This could be an obvious aspect, however, it is recognized from several researchers that many vendors have not been financial successful, which resulted in bankruptcy (Tang et al., 2006). Moreover, a well established partnership can also be useful to implement an EMR based on strategic visions, where a vendor can play a good part in by thinking along with the organizational vision. Besides these aspects, Boorady addresses that physicians need to make sure that vendors have a large user base to ensure regular upgrades that will keep the EMR up to date (Barrier E4). Furthermore, vendors should provide adequate trainings and ongoing support for the staff (Boorady, 2006).

11. Acquire third party consult for selection of EMR / vendor: Selection of a suitable EMR system as well as a

(12)

architecture and they can display the wishes they have for the new EMR (Barrier F2). A consultant will analyze this profile and consult the customer within 24 hours with a handful of recommendations of systems to investigate as most likely to be suitable (Barrier B4, E1, F3 & G3). HIMSS (The Health Information and Management Systems Society) provides an online EMR evaluation tool. This tool can be found at www.ehrselector.com/ehrselector/EMRToolkit/ASP/Default.asp. It requires, however, a $149 annual practice subscription fee. Specialists and researchers found this tool useful, since it provides options to locate a suitable vendor as well as a function to compare the different vendors based on similarities and distinctions. Pondrom advices to go to www.buyerzone.com/software/electronic-records/buyers_guide1.html where many EMRs are described and can be compared (Pondrom, 2007). These sources are examples of third parties that organizations may use to help them with this selection process. Especially when a clinic or hospital lacks technical expertise to make a wise selection criterion, they have plenty of opportunities for a third party consult. It also should clear up uncertainties about vendors and provide a clearer overall picture of the large amount of vendors and the services they have to offer. In addition, physicians themselves should have the ability to use these selection systems in order to generate higher participation. This would hopefully lead to a higher adoption rate since the participants are involved in the selection process (Barrier A3 & A5). In this way, physicians have influence in these organizational decisions which should stimulate the barrier “lost of autonomy” (Barrier D1).

Technical interventions

12. Communitywide data exchange and network standards: Secure electronic data exchange of clinical

(13)

13. Optimize software, upgrade hardware and tune networks: Before the actual implementation of EMR,

physicians should analyze their current technical conditions of the organization. Since EMRs are known as complex software packages, they require computer equipment which is well maintained and is of good quality. Hier argues that running EMRs on outdated computer systems will decrease the overall performance of the EMR (Hier, 2002) (Barrier E4). In addition, physicians are recognized from the literature as individuals to be obsessed with system speed. The survey which was taken by Hier in his case among several physicians, reported that physicians listed system speed as their greatest concern. System speed is running parallel with the working speed of a physician. Thus, it also reasonable to place enough workstations, so each physician does not need to wait on a workstation when a colleague is working on it (Barrier F1, F4 & E3). Therefore, organizations need to ensure that there are plenty of working stations which are customized by the latest standards of soft- and hardware, to provide an acceptable form of system speed and accessibility (Barrier F1 & F2).

14. Provide internet access to the EMR: EMRs should be accessible anywhere, anytime. This is an important

point recognized by many researchers, plus it is a huge selling point to physicians (Hier, 2002). This vision can be realized by connecting EMRs to the internet (Barrier A3, A4, F1 & F4). With this option, physicians can access the EMR form anywhere they want to. Even when they are not at the office, they can use the internet at home to access the EMR. Accessibility for physicians is important, since they do not like constraints which can have impact on their working processes.

15. Hiring a computer company: Many physicians and managers lack technical skills when it comes to the

installation of an office network. This network is a pre when it comes to the implementation of EMR. Therefore, when physicians lack the skill, it is wise to hire a professional and reputable computer company, in order to install a stable technical network (Barrier A1, E1 & F3). Hier reports a computer company asks between the $1000 and $3000 for this service (Hier, 2002).

Financial Interventions

16. Purchase additional components in a later stadium: A relative simple but good working intervention could

be to purchase additional components of the EMR in a later stadium (Miller et al., 2003) (Barrier F2, H2 & H3). Besides the actual purchase, organizations also have the ability to rent extra components. For this option, managers need to find a vendor who accepts this form of delivery. Extra components can be data exchange interfaces or hardware components like extra notebook computers. In this way, managers have the possibility to exact measure what is still needed to make the missing aspects of the process complete, so it can run perfect.

17. Documented ROI: Before EMR implementation, it is wise to locate organizational problems which EMR

(14)

18. Accelerate Implementation Time: Many EMR implementations are planned over several years. The total cost

of ownership is 1/3 capital technology and 2/3 on manpower costs (Pizzeferri et al., 2004). These facts announce variable costs which will increase more when EMR projects take more time. Therefore, researchers and physicians seek ways to accelerate the implementation time of EMR projects (Barrier H5). Besides the positive idea, it is hard to realize. Organizations who try to achieve this objective, should try to set-up a matrix structure, a joint partnership with a vendor consultant or should use techniques such as automated mapping to realize the idea behind this intervention (Pizzeferri et al., 2004). A matrix structure is an organizational structure where every actor has two managers; one project manager (EMR) and one process manager (current working practices in the organization). In this way, each actor is influenced by two managers with different goals. This connection can stimulate the outcomes for the process goals and the project goals. This idea should create a better fit of the EMR project and the working processes. However, this idea can be confusing for employees, since they need to report to two managers with (maybe) different objectives.

19. ASP technology: Application Service Provider (ASP) is provider who rents hardware and software programs

to companies. APS can also be used with using EMR, where users rent access to software and vendors provide access to data applications (Miller et al., 2003; Boorady, 2006). This option could decrease cost like maintenance costs, hardware costs and other technical costs (Barrier H6). These savings together can represents 30% compared to a “normal” EMR project (Miller & Sim, 2004).

20. Additional “backfill personnel” / patient reduction of 30% during transition period: The transition period is

recognized from the literature as a slow-going process with much to learn for physicians. Physicians argue that they can not see much patients in this period, compared with previous periods, since physicians need additional time to learn and get familiar with the EMR. When physicians do not see the potential benefits of EMR and experience a form of resistance to it, this may be coupled with the other disadvantage regarding the system; 30% patient reduction. These two aspects can play an important role in the attitude of a physicians related to the adoption of the implemented EMR. Experienced from the Kaisler case, Øvretveit et al., argue that clinics should be aware of this transition period and the resistance of physicians towards it. They argue to plan additional “backfill” personnel for this initial period to reduce impact on workload (Øvretveit et al, 2007). Another advice would lie on the other end of this situation. It is also possible to reduce the patient schedule with 30% (Boorady, 2006) (Barrier H2). Besides these advices, vendors are also experiencing these difficult initial periods for clinics. Many EMR vendors have responded by providing tools to transfer paper records into their systems, easing the transition process (Da’Ve, 2004) (Barrier F3). These advices together should make the transition process easier to perform. Nonetheless, physicians as well as managers and vendors should not forget this initial period by paying less attention to it, since the actual performance of this implementation phase is crucial for a successful outcome.

3 MEASURING AND LOCATING BARRIERS IN PRACTICE

(15)

using a 5 point Likert Scale, which is scaled with -2, -1, 0, +1 and +2. The given opinion for each statement is linked to the presence or non-presence of a barrier. For example, two statements have been set-up for barrier A1: Lack of individual technical expertise. To test this barrier in practice, two relevant statements regarding this barrier have been set-up, which are:

I have novice computer skills Disagree AgreeUsing computer systems is new to me Disagree Agree

When a physicians agrees with the first statement and agrees with the second statement, it is quit noticeable the physicians is new to using computer systems and has novice computer skills. With this construction of statements, there is an ability to test if each barrier is present in practice. For this example, we may assume that barrier A1 is present. It could also be the case that some physicians have neutral thoughts towards this issue, or they even may think they are expert.

Besides measuring the presence of barriers with statements, which is shown in survey 1A, an extra dimension is needed to display the relative importance of each stated subject in each statement. It could be the case that physicians see a subject as a barrier towards the adoption of EMR, but it could lack importance related to their working practice and the EMR system. This makes the discussed barrier less relevant to deal with in practice, since physicians see this barrier as unimportant, thus, irrelevant towards their practices. Any reason for this could be possible, since every situation has is own and unique characteristics. With the presence of this second dimension which is measured in survey 1B through so-called “importance statements”, both dimensions enable a matrix which is shown in figure 1.

Figure 1: Barrier matrix

Unimportant non-barriers: Barriers which fall in this category are practically unimportant. Although barriers may be present in other situation recognized by the literature study, respondents feel this barrier lacks presence in their working practice. Besides the lack of presence, respondents see this barrier as unimportant. With these conditions, barriers in this area should get less to no attention.

L o w P re se n ce o f b a rr ie r H ig h

Low Relative importance of barrier High

(16)

Important non-barriers: This area represents barriers which also lack presence related to their working practice. Physicians do not recognize these potential barriers as serious present barriers in their working conditions. However, they do recognize these potential barriers as important barriers regarding EMR and the organizational situation. Therefore, it could be wise to analyze the reported potential barriers in this area and to handle them with the related interventions.

Unimportant barriers: Unimportant barriers are present barriers recognized in the working conditions by physicians. The opinions by physicians regarding to the statements in survey 1A represent a serious present of a potential barrier. Besides this condition, barriers in this area lack importance. This should not be seen as an obstacle to attend these issues. Physicians may see these issues as unimportant, but it could be the case that these barriers are very important towards the sketched situation.

Important barriers: Each barrier which falls in this category should be seen as the most important ones that need the most attention. Each barrier is present in the working practice and is seen by physicians as an important aspect regarding the implementation of EMR. It is obvious that barriers in this unit need the most attention regarding the implementation process of EMR.

(17)

Survey 1A: Statements linked to potential presence of barriers

Instructions: Results of this survey should locate important issues regarding the EMR project. Please color one dot between the words “Disagree” and “Agree”, which best reflect your opinion regarding the presented statement next to it.

A1.1 I have novice computer skills Disagree Agree

A1.2 Using computer systems is new to me Disagree Agree

A2.1 I think that most EMR vendors provide abilities to Disagree Agree

experiment EMR

A2.2 I think that most EMR vendors provide a pilot Disagree Agree

version of EMR

A3.1 The current way of recording medical data is Disagree Agree

adequate

A3.2 Changing work will enforce/create resistant Disagree Agree

behaviour in my organization

A4.1 EMR will provide me individual benefit Disagree Agree

regarding my work process(es)

A5.1 I assume I will be involved in the entire Disagree Agree

implementation process

A6.1 I think that the more users EMR has, the better it Disagree Agree

will run

A7.1 When performing good regarding the EMR project, Disagree Agree

I can get financially rewarded

B1.1 I think that finding a right EMR vendor is easy Disagree Agree

B2.1 I think that EMR vendors will provide support Disagree Agree

during the implementation process

B2.2 I think that EMR vendors will provide support Disagree Agree

after the implementation process

B3.1 I think that EMR vendors offer proper financial Disagree Agree

information to use for EMR selection

B4.1 I have doubts whether EMR will improve quality Disagree Agree

C1.1 During projects, my organization pays attention to Disagree Agree

the selection of product champions

C1.2 During projects, my organization uses project Disagree Agree

management techniques

C2.1 When I have question during projects, my manager Disagree Agree

is there to answer them

D1.2 With the introduction of EMR, I assume that I Disagree Agree

will keep my current status of autonomy

D2.1 I think that EMR will provides good security and Disagree Agree

privacy regulations

E1.1 I think that EMR will technically fit with other Disagree Agree

systems in the organization

E2.1 EMR is a complex system to use and to understand Disagree Agree

E3.1 I think that EMR is a fast system Disagree Agree

E4.1 I think that EMR is a reliable system Disagree Agree

F1.1 EMR would increase the quality of my work Disagree Agree

(18)

F3.1 I assume that it will be complex to migrate from the Disagree Agree

current situation towards a EMR environment

F4.1 There are enough working stations where I can use Disagree Agree

the EMR system

G1.1 I assume it will take much time to enter data in Disagree Agree

EMR

G2.1 I like to spend time on learning EMR Disagree Agree

G3.1 My manager will clearly communicate to me about Disagree Agree

how much time I need to spend on EMR

H1.1 I am certain about the financial return of EMR Disagree Agree

H2.1 During implementation, I assume there will be a Disagree Agree

productivity loss

H3.1 I think that purchase costs of EMR are acceptable Disagree Agree

H4.1 Other parties like payers and consumers will Disagree Agree

benefit more from EMR than we will do

H5.1 The suspected time scale of EMR’s financial return Disagree Agree

is too long

H6.1 I think that maintenance cost of EMR are acceptable Disagree Agree

H7.1 I think that my organization is responsible for Disagree Agree

(19)

Survey 1B: Statements linked to potential presence of barriers

Instructions: Please make sure survey 1A is finished properly, since this survey continues on the previous introduced statements presented in survey 1A. These statements are constructed to decide your relative importance on several subjects, which were introduced in survey 1A. Each code displayed between the anchors at each statement displays the related statements of survey 1A. Please color one dot between the words “Disagree” and “Agree”, which best reflect your opinion regarding the presented statement next to it.

1. Having experience with computers is important for me when working with EMR (A1.1, A1.2, G1.1)

Disagree Agree

2. Testing EMR with for example a pilot version is important for me regarding EMR (A2.1, A2.2)

Disagree Agree

3. Holding and maintaining the current working environment is important for me regarding EMR (A3.1, A3.2, F2.1)

Disagree Agree

4. Benefiting individually by using EMR in practice is important for me (A4.1)

Disagree Agree

5. Participating and being involved during the

implementation process regarding EMR, is important for me (A5.1)

Disagree Agree

6. It is important for me that many physicians in the organization are working with EMR (A6.1)

Disagree Agree

7. Rewarding physicians financially when performing good is important for me regarding EMR (A7.1)

Disagree Agree

8. It is important for me to select a right vendor regarding EMR (B1.1)

Disagree Agree

9. It is important for me that vendors provide support after implementation (B2.1, B2.2)

Disagree Agree

10. Providing financial information about a vendor’s EMR is important for me (B3.1)

Disagree Agree

11. It is important for me that EMR will lead to quality improvements (B4.1, F1.1)

Disagree Agree

12. The presence of a product champion is important for my motivation regarding EMR (C1.1)

Disagree Agree

13. Working with qualitative project techniques is important for me regarding EMR (C1.2)

Disagree Agree

14. During the EMR project, I find it important to have managerial support (C2.1)

Disagree Agree

15. Having an autonomous status in the organization is important for me regarding EMR (D1.1)

(20)

16. Security and privacy issues concerning patient information is important for me (D2.1)

Disagree Agree

17. Having a technical fit with other IT systems in the organization is important for me (D2.1)

Disagree Agree

18. The easiness of use of EMR is important for me (E2.1, G2.1)

Disagree Agree

19. It is important for me that the system speed of EMR is fast (E3.1)

Disagree Agree

20. It is important for me that EMR will work stable (E4.1)

Disagree Agree

21. Establishing a good migration plan to support the transition period is important for me (F3.1)

Disagree Agree

22. It is important for me that I can get access to a workstation when needed (F4.1)

Disagree Agree

23. It is important for me that my boss communicates about how much time to set apart for selecting, contracting and implementing EMR (G3.1)

Disagree Agree

24. Financial issues concerning EMR are important for me (H1.1, H2.1, H3.1, H4.1, H5.1, H6.1, H7.1)

(21)

Practice example concerning the introduced material

Survey 1A has the function of locating the presence of barriers. Survey 1B provides statements which are related to the introduced statements in survey 1A. The function of this survey is to find the relative importance of each statement. Data from both surveys can be presented in the introduced barrier matrix (figure 1). This should provide the management a general overview of the different types of barriers. Along with this matrix, managers have the ability to see which barriers are in the more serious units, like “unimportant barriers” and “important barriers” and which barriers are in the less serious units like “unimportant barriers” and “important non-barriers”. A serious unit in this field of context stands for barriers which should be handled extensively, since the presence and importance of it should be obvious. This should lead to a general impression about how much effort it takes for an organization to link interventions to so-called “serious” barriers. To make this material more understandable, this section provides a practical example about how to use the material.

Statements from survey 1A

A1.1 I have novice computer skills Disagree Agree

A1.2 Using computer systems is new to me Disagree Agree

In this practice example, we see that a respondent answers a “+1” on statement A1.1. The respondent basically agrees with the statement. Since he / she can be seen as quit an novice when it comes to computer skills, we may assume that the referred barrier A1: Lack of individual technical expertise is present, measured with this statement. Note that assumptions are being made in this section to make this material clear for practical use. The respondent also scores a “+1” on statement A1.2, which represents the assumption that he / she is not quit familiar with computer systems.

Statement from survey 1B

1. Having experience with computers is important for me when working with EMR (A1.1, A1.2, G1.1)

Disagree Agree

(22)

Figure 2: Barrier list to use in practice

A1.1 and A1.2 both represents the potential presence of barrier A1. Like introduced before, sometimes multiple statements are constructed for one barrier, when a barrier is introduced too abstract. Both dotted lines in the above figure distinguish the 4 units of the barrier matrix. Note that barrier A1, represented by the linked statement A1.1 and A1.2 falls in the unit “Important non-barrier”. To make this section complete, several practical issues are introduced which serve as important factors:

 The result of a statement sometimes needs to be reversed interpreted. This is related to the related barrier of a statement.

 Get the mean for each statement outcome, which is filled in by multiple respondents. It is too much work to analyze each individual survey.

 The more respondents for the survey, the reliable the survey results are.

 Survey 1A measures the presence of the barriers. Survey 1B measures the relative importance of each statement in survey 1A, which is eventually linked to the barriers recognized from the literature study. +2 +1 0 -1 -2 -2 -1 0 +1 +2

L

o

w

P

re

se

n

ce

o

f

b

a

rr

ie

r

H

ig

h

Low Relative importance of barrier High

(23)

 When an outcome has “0” vertical and “0” horizontal, the outcome is neutral. It does fit in each unit of the barrier matrix. The dotted lines represent this neutral area.

 When the mean outcomes of the survey are analyzed, the management team should have a filled in barrier matrix which displays potential presence of the potential importance of barriers in different units. The next step is to consider which units (and barriers in that unit) are important for the management to handle. One advice would be to handle barriers which fall in the unit “important barrier” and the unit “unimportant barriers”.

4 INTERVENTION PLAN

A comprehensive and detailed plan is needed to use the introduced theory, tables and the survey in an effective manner in practice. Therefore, this section displays a step-by-step intervention plan which should guide each physician in the right direction. Before the introduction of each intervention step is presented, take into account that these steps need to be performed by managers, who are responsible for a successful EMR implementation. Within this scope, the main focus is still on the barriers of physicians towards EMR adoption and the related interventions. Each step is presented in a model which is displayed in figure 3. However, note that this plan can be seen as an intervention on its own. In addition, this plan already consists on several interventions. For example, take step 1 where physicians are being informed and step 5 where a presentation and discussion should be given. These steps are crucial to perform a good intervention plan for each situation, since informing, participating and involvement are highly recognized interventions from the literature. It are basic the ingredients for bringing managers to physicians with a shared bundle ideas towards the intervention plan, which is suitable for each practice situation. For now, see figure 3.

(24)

Informing physicians about organizational changes is crucial for the rate of acceptance of physicians towards that particular change. When physicians are informed, they can create a better understanding of the desired change which may lead adopting and supporting it. Therefore, managers should present the EMR and its initial intentions. Since the literature reports there are many problems with the adoption of EMR from physicians, managers should explain why they want to use a survey. Even if physicians have positive attitudes towards the new idea, it is still wise to use this survey since it may contain aspects where physicians were not aware of. By introducing the EMR in general, its potential advantages and intentions, along with a presentation of the surveys’ intentions, this should stimulate physicians’ support and their understanding towards the organizational change. It is important that the “barrier” is presented in a positive way, since this term could have a negative influence on physicians. It could be the case that physicians feel responsible when the project fails, since it is widely recognized from the literature that adoption lacks by the barriers of physicians. Another important aspect for the management is to explain several terms which are introduced in the survey. It could be possible that physicians cannot fill in some statements, because they do not have the required information to answer the statement. One unknown aspect is finance. For this reason, managers can use the results of a cost-benefit research which was conducted in a study by Wang et al (2003). Their data came primary from their electronic medal record system, from other published studies and from expert opinions. Therefore, this data can be used to provide physicians with the average costs on EMR in the market (Figure 4). Besides the financial aspect, managers should provide general information about EMR and practical issues concerning EMR implementation. Furthermore, managers should inform physicians about the diversity of vendors who sell EMR. The amount of information that needs to be provided is related to the respondent’s knowledge about this subject. In the figure below, the finance model of Wang et al. is presented, which can be used by managers to give physicians a general impression of average cost regarding EMRs.

Figure 4: Overview of average EMR cost Step 1: Inform physicians about the introduction of an EMR and a survey which should highlight

(25)

Now physicians are informed about the organizational change and have created an informal attitude towards the EMR, they should get the chance to do the survey. It is important for management to support physicians when they need help or when they do not understand the survey. Managers need to be focussed on the fact that physicians should fill in the survey in the most honest and motivated way to gain better results. This can be achieved by explaining the use of the survey, helping them with problems and unclear statements and by informing them of possible physicians’ participation in the next intervention steps. For this task, it also could be wise to hire a consultancy company who is expert in this field. Even if physicians resist the survey because they do not see the use of it, explain them that it is necessary since the survey results can provide output which may prevent implementation problems concerning the EMR. Also think of your most important stakeholders to let them participate in the survey. It is a hard job to analyze a lot of results when less could also do the job. Besides the survey and the limitations of this study, it could also be a wise option to take interviews in order to generate extra information for the project.

This step is very important when an organization uses this material as an intervention towards the barrier problem. It is already described in detail in the practical example of section 3. The results should be interpreted correctly so managers can link the right interventions to barriers. It could be a good advice to perform this step with multiple persons. Be aware of the potential presence of patterns which can be discovered by analyzing the survey results. Note that all information can be useful. It is wisely to display all the located barriers in the barrier matrix for practice. A clean template of this model can be found in appendix B.

The barrier list displays each potential barrier in the analyzed and unique working practice. Each barrier that needs to be dealt with should be present and important, recognized by practitioners. Each code of a barrier needs to be linked to an intervention number, using table 1 and if necessary, table 2. Note that some interventions are linked to more than one barrier, which may result in reporting the same intervention. This may be useful, since the more times an intervention is linked by the presence of several barriers, the more attention needs to be attended to that particular intervention. The outcomes of this step should present an entire intervention list, which serves as a general input to step 7; the intervention plan. A good advice would also be to present the barriers, along with the linked interventions to the entire group of actors who are going to deal with the implementation of EMR. This is discussed in step 5.

Step 2: Select the amount of physicians you want to use for the survey and let them take the survey along

with a motivated attitude.

Step 3: Analyze survey results and select important and relevant barriers, recognized by physicians.

Step 4: Select interventions by linking them to relevant barriers displayed by the survey results, using table

(26)

It is wise to present the complete result list to all actors who are participating in the implementation process of EMR. In this way, everyone feels like getting involved in the entire process. In addition, the results can be discussed which should provide extra insight regarding the project. Physicians can give their opinion about the presence of barriers, the interventions and they inform managers about their beliefs towards the EMR project. Be aware that physicians still can provide useful input after the survey, since it can be possible that physicians were unaware of several aspects when filling in the survey. Furthermore, besides a positive feeling of getting involved for physicians during the EMR project, physicians also feel they have influence in the decision-making process of further steps that needs to taken. Their opinions should be heard and discussed. As a team you should perform pro-active and critic. Physicians like this sort of attitude. The outcomes of this presentation and discussion also forms a general input for the final intervention plan.

The literature recognizes some pre-requirements for the organization, which are in this article translated to interventions. To use all the interventions introduced in this article could be a wise step. However, it will take much time to analyze and apply each intervention and many interventions are costly. Therefore, organizations may want to use the tool presented in this article which links interventions to recognized barriers. However, there are some interventions which can be used for every working situation and which are independent on the fact if barriers are present or not. Think of organizational culture, structure, technical conditions etc. Therefore, a pre-requirements checklist has been made which an organization can use to check the current status of the organization. Paying attention to each of the aspects should be useful for any organization, since the literature recognizes these aspects factors that should promote the implementation process of EMR. Besides this fact, organizations can also use it as a reminder tool. Since many barriers can be present, linked with an additional list of interventions, managers can check whether they did not miss anything. The idea behind the pre-requirement checklist is that managers should be able to at least answer one A or B question with a “Yes”. This checklist recognizes seven important categories, displayed with two questions / statement. If a manager is unable to answer the first statement with a “Yes”, the second statement should be a “Yes” in order to let the manager know they are aware of that category. Thus, a “Yes” displays the fact that the organization paid attention to that pre-requirement condition. The function of the checklist can be seen as a last check, before implementing the EMR. This tool should be the last general input for the creation of the intervention plan. The pre-requirement checklist can be found in appendix A.

Step 5: Present the outcomes of the survey so it can be discussed and criticized

Step 6: Analyze the status-quo of the environment where EMR will be implemented and check its conditions

(27)

When each step is performed, step 4, 5 and 6 provide the general input for making the intervention plan. The intervention plan displays each intervention that needs to be handled regarding the presence of barriers. Each intervention is now analyzed so information is available for the required cost, available and required man hours, involved employees and a time-table. Each intervention should be seen as an extensive activity. Stakeholders need to research the details of each intervention. Furthermore, it could be wise to make product champions / product teams responsible for each intervention. Besides these aspects, organizations should consider whether they need professional help from a consultancy company, for guidance during the implementation phase. Desired outcomes of the interventions need to be displayed in order to measure the success of each intervention’s outcome. Again, involve employees during the set-up of this plan, to let participate so they have the ability to criticize the project conditions, and thereby influence the project. The intervention plan should be the main guidance for project managers and employees during the actual implementation phase of EMR. There needs to be a continuous interaction between these two aspects. When physicians experience new difficulties or new opportunities, these issues also need to be placed in the intervention plan. Physicians can use this plan when the organization experiences another technical change.

Step 7: Set-up the intervention Step 8: Implement EMR

(28)

5 DISCUSSION

This article presents a way to recognize physicians’ barriers in practice and to handle each barrier with relevant interventions, both recognized from an extensive search through literature. Since the literature study is based on multiple case studies, interviews and other research methods performed in real time practice situations, it is arguable that each barrier could be present in each working practice. This makes the study relevant to practice situations, since the study is primary based on it. Furthermore, since each barrier was found on research outcomes linked to practice, this makes the validity of the actual presence of these barriers in practice relevant. The strength of this data also comes from the multiple researchers who did recognize the same barriers and interventions. However, practitioners need to keep in mind that each situation has its own characteristics, which may lead to different outcomes. It is possible that there can be more barriers found in practice, which should lead to a useful extension for this research. This article can also be used to make physicians aware of barriers they do not experience, or as a reminder tool to evaluate if each barriers is analyzed based on its potential presence. If you make physicians aware of the presence of barriers before EMR implementation, there is still time left to search for interventions to handle these barriers. When physicians experience obstacles that prevent use of the EMR after implementation, it could be too late to handle these issues with interventions.

(29)

a way to measure the perceptions of physicians towards the EMR. Physicians have the ability to agree or disagree with a statement, which should give managers the ability to recognize the potential presence and / or importance of a barrier.

(30)

6 CONCLUSION

(31)

7 REFERENCES

Ash, J.S. and Bates, D.W. (2005) Factors and forces affecting EHR system adoption: report of a 2004 ACMI discussion. J Am Med Inform Assoc 12(1), 8-12.

Assar, K. (2002) The Need for Electronic Medical Records in Primary Care. Avaible at: http://www.medscape.com/viewarticle/446702

Anderson, J.G. (2007) Social, ethical and legal barriers to e-health. International Journal of Medical Informatics 76, 480-483.

Bar-Lev, S., Shirly, H. and Michael, I. (2005) Localization of Health IT: How Users “Repair” Electronic Medical Record Systems. Academy of Management Proceeding, 1-5.

Baron, R.J., Fabens, E.L., Schiffman, M. and Wolf, E. (2005) Electronic health records: just around the corner? Or over the cliff? Ann Int Med 143, 222-226.

Bates, D. W. (2005) Physicians and ambulatory electronic health records. Health Aff 24(5), 1180-1190.

Blair, J. (2004). Survey of Electronic Health Record Trends and Usage for 2004. EHR Summit III. Medical Records Institute. Available at:

http://www.medrecinst.com/conferences/seminar/july04/proceedings/pdfProceedings/JB.pdf. Accessed on 10/5/2004.

Boonstra, A., Boddy, D., Bell, S. (2008) Stakeholder management in IOS projects: analysis of an attempt to implement an electronic patient file. European journal of information systems 17(2), 100-111.

Boorady, J. (2006) The EHR fear factor. Journal of the American Optometric Association 77(6), 317-318. Clayton, P.D., Narus, S.P., Bowes, W.A., Madsen, T.S., Wilcox, A.B., Orsmond, G., Rocha, B., Thornton, S.N., Jones, S., Jacobsen, C.A., Udall, M.R., Rhodes, M.L., Wallace, B.E., Cannon, W., Gardner, J., Huff., S.M. and Leckman, L. (2005) Physician use of electronic medical records: issues and successes with direct data entry and physician productivity. AMIA symposium, 141-145.

Da’Ve, D. (2004) Benefits and Barriers to EMR Implementation. Caring 11, 50-51.

Gans, D. (2005) Off to a slow start. Available at: www.rpiusa.com/Con_05_Oct_slow_start%5B1%5D.pdf. Gans, D., Kralewski, J., Hammons, T. and Dowd, B. (2005) Medical groups’ adoption of electronic health records and information systems. Health Aff 24, 1323-1333.

Hier, D. B. (2002). Physician buy-in for an EMR. Healthcare informatics. Available at: http://www.healthcare-informatics.com/issues/2002/10_02/commentary.htm

Hier, D. B., Rothschild, A., LeMaistre, A., and Keeler, J. (2004) Differing faculty and housestaff acceptance of an electronic health record one year after implementation. Medinfo 11, 1300-1303.

Hodge, R. (2002). Myths and realities of electronic medical records, The Physician Executive, 14-19.

Kemper, A.R., Uren, R.L. and Clark, S.J. (2006) Adoption of Electronic Health Records in Primary Care Pediatric Practices. Pediatrics 118(7), 20-24.

Keshavjee, K., Bosomworth, J., Copen, J., Lai, J., Kucukyazici, B., Lilani, R. and Holbrook, A.M. (2006) Best Practices in EMR Implementation: A Systematic Review. Available at:

Referenties

GERELATEERDE DOCUMENTEN

The first thank you goes without a doubt to Raffaella for having, in the first place, relied on me to take a piece of her beautiful project ‘Radiolife’ forward. Thank you for

With its unprecedented sensitivity and resolution, LOFAR has allowed us to perform the first systematic search for remnant radio galaxies at 150 MHz, and much more will

Using the PACS 70 μm emission as an indicator of the possible regions of young star formation, and thus the locations of objects contributing to such free –free emission, we can be

To test our hypothesis that, compared with the other three groups, workaholic residents score less favorable on the various correlates of wor- kaholism, a multivariate analysis

Compilers for modern object-oriented programming languages generate code in a platform independent intermediate language [LY99, CLI06] preserv- ing the concepts of the source

For example, A6 represents category A (Human Area) and the number 6 represents barrier 6 of that category (Groupsize – adoption rate). The reason for this approach is to give

In order to strengthen the communication with teachers, a contact person could be designated to maintain contact with the teachers (e.g. every week) about how the matter stands

In deze paragraaf wordt aan de hand van verschillende wetenschappelijke artikelen gekeken of de factoren die van invloed zijn op de implementatie van een MACS ook van toepassing