Lourens Huizinga S1785141 Bsc MA Change Management First assessor: dr. M.A.G. van Offenbeek Second assessor: dr. J.F.J. Vos Date: February 16, 2016 Total word count: 15.601

(1)

Insights in implementation success: applying normalization process

theory to the implementation of a physical activity stimulation

program in rehabilitation care

Lourens Huizinga

S1785141

Bsc MA Change Management

First assessor: dr. M.A.G. van Offenbeek

Second assessor: dr. J.F.J. Vos

Date: February 16, 2016

Total word count: 15.601

Abstract

(2)

(3)

2

1. Introduction

The field of healthcare has never suffered from a lack of interest: the results of medical research are likely to have an impact on one’s health and wellbeing at some point in life. Although there is no shortage of knowledge being produced, the usage of the knowledge is far from optimal. There is a gap between the outcomes of research and practice of healthcare, and uneven uptake of research findings – and thus suboptimal care – occurs across settings, specialties and countries (Eccles and Mittman, 2006; Strauss, Tetroe and Graham, 2009).

The transitioning phase from knowledge to implemented practice has seen an increase in attention by researchers over the last decades. An indicator of this is the founding of a dedicated journal in 2006 (Implementation science, Eccles and Mittman, 2006). A meta study on the transfer from knowledge to practice by Graham and Tetroe, (2007) uncovered a total of 31 frameworks guiding this transfer, which had many common elements and action plans. The article leads to the construction of the knowledge-to-action framework, which recognizes the complexity of implementing innovative practices in the healthcare sector as the stakeholders include not only patients, but also insurance companies, managers and clinicians who are all considered to be end-users of the knowledge that is being implemented (Strauss et al., 2009).

An example of the gap between knowledge and practice is visible in the treatment procedures of the rehabilitation care. There is ample evidence that physical exercise has a positive influence on health and a physically active lifestyle is strongly recommended for persons with a disability or chronic health condition (Rimmer and Rowland, 2004). However, previous research has shown that the transfer of evidence-based programs regarding the stimulation of physical activity from study phase to actual implementation is often not without hurdles or failure (Fallon, Wilcox and Laken, 2006; Glasgow, Eakin, Fisher, Bacak and Brownson, 2001; VanWormer, Pronk and Kroeninger., 2009).

The example of rehabilitation care is especially interesting because of the complex composition of the stakeholder group involved in the implementation of innovations as rehabilitation care is multidisciplinary by nature. New work methods call for behavior change by the involved parties, which further increases the complexity of the intervention (Connell et al., 2015).

(6)

5 importance for the success of a physical activity stimulation program (Huijg et al., 2013; Wierenga et al., 2014). The specific relevant factors are further discussed in the literature chapter. Huijg et al. (2013) and Wierenga et al. (2014) also underpin that identical intervention may lead to different implementation success rates in different contexts. An intervention in setting A can lead to high usage of the new program and usage of the program as intended, whereas the same intervention in setting B might not be used as intended or have low usage.

Another theory that might explain success of implementation is May and Finch’s (2009) Normalization Process Theory. The theory was developed to help explain and better predict the success of implementation of complex practices, for example business processes or healthcare interventions. The theory explains this success by introducing four components that are influencing implementation, coherence, collective action, cognitive participation and reflexive monitoring. Although the theory has the potential to be a valuable tool, it needs to be tested more in practice (Murray et al., 2009).

One of the main conclusions of a broad systematic review on how innovations can be spread and sustained in health service delivery and organizations by Greenalgh, Robert, Macfarlane, Bate and Kyriakidou (2004), is that there is a clear lack of easily generalizable knowledge on how innovations become embedded in organizations. There is also a lack of agreement regarding concepts hypothesized to affect implementation success and identifiable measures of these concepts (Chaudoir, Dugan and Barr., 2013). This paper, therefore, focuses on finding an answer on the following research question:

“Which implementation process factors influence the implementation level of an evidence-based physical activity stimulation program in the rehabilitation care, and how do these factors influence the continuation of the implementation.”

(7)

6 By answering the research question, this paper aims to better predict the factors influencing the success of implementation. By enriching the current knowledge about implementing evidence-based innovations in rehabilitation care, healthcare administrators are able to more easily recognize what restricts successful implementation and stimulate those factors that increase the likelihood of successful implementation. In short, this means that future innovations could be implemented with greater success. The research can also determine whether previous physical activity stimulation program implementation research conducted in primary healthcare can be transferred to secondary healthcare providers by applying the literature by Huijg at al. (2013). The longitudinal aspect of this research allows to identify factors in the implementation process that are most influential over the course of the implementation, which is important as the focus of this research is to a large extent on the continuation phase.

A preliminary research model was developed to show the focus of this research (figure 1). The model shows that the implementation process influences the success of implementation. That what is being implemented (implementation content) can directly influence healthcare performance, but successful implementation increases the impact. This research focuses on the central part of the model: how the implementation process influences the implementation success.

Figure 1. Preliminary research model

(8)

7 Chapter 3 explains the methods used in this research. It deals with the selection of the cases and the collection of the data, as well as the methods used to analyze the data and the reasoning for choosing these methods.

Chapter 4 describes the results of the analysis of the RSE case, gives an overview of the collected data and shows the results of the data analysis.

Chapter 5 discusses how the results relate to the propositions made in the literature chapter, and what the implications of the results are. Furthermore, the limitations of the research are discussed and recommendations for further research are proposed.

The paper ends with a short conclusions, which contains the final remarks.

2. Literature review

As proposed by Greenhalgh et al. (2004), the factors to be included in the research are derived from the relevant change management and implementation literature. In this chapter, we discuss the body of relevant literature. The literature review starts with a closer look at the phases of implementation and explain on which of the phases this paper focuses. The ‘phases in change’ section also includes a review of change management literature with the focus on what is important in which phase. Secondly, the state-of-the-art knowledge regarding implementation in healthcare organizations is presented and compared, focusing on the for this research relevant phases during the implementation. Thirdly, a new theory on the process by which procedures becomes embedded in organizations – normalization process theory – is introduced. This new theory is developed specifically for complex interventions in healthcare organizations and is compared to classical change management literature and the state-of-the-art literature on healthcare implementation, which leads to twelve propositions and a refined research model.

2.1 Change management literature: phases in change

(9)

8 that change is necessary, breaking down the status quo. This is followed by moving, which is the step where the change is implemented and new routines are adopted. Finally, refreezing is what needs to happen after change is embraced. The change has to be internalized and become part of daily routine.

Table 1 shows that different scholars discussed in this research use alternative terms for different phases in the literature that are discussed in this section. The scholars, who are discussed in detail later in the literature section, all have in common that the first phase is where the change is initiated. The second phase is where the change is implemented. The third phase is where new practices are routinized and become the new standard. As explained earlier, this research focuses on the factors influencing the implementation success of an evidence-based physical activity stimulation program in rehabilitation care, and how these factors influence the continuation of the implementation. The scope of the research is on the period after the decision to implement the innovation has been made and after the implementation process has started, so when the physical activity stimulation program is already in use. The rest of this paper refers to those phases as the implementation and continuation phase.

Table 1. Comparison of terminology about phases used by different scholars in implementation Lewin (1947) Glasgow et al. (1999) Fleuren et al. (2004) Huijg et al. (2013) Unfreeze: ‘Overcome

inner resistance (to change)’ (p. 32), ‘unfreezing the present level’. (p. 35)

Adoption: ‘proportion of settings, practices, organization

and plans that will adopt this intervention’ (p. 1324)

Dissemination: not specified in article Adoption: not specified in article

Adoption: ‘the decision to work with an

innovation’(p. 1)

Moving: ‘moving to the new level’ (p. 35)

Implementation: ‘extent to which the intervention is implemented as intended in the real world’ (p. 1324)

Implementation: not specified in article Implementation: ‘deliver it (the innovation) as intended’ (p. 1) Refreeze: ‘freezing

group life on the new level’(p. 35)

Maintenance:

‘extent to which a program is sustained over time’ (p. 1324) Continuation: not specified in article Continuation: ‘continue to use it over a longer period of time’ (p. 1)

(10)

(11)

10 Table 2. Comparison of steps and stages of change management models

Lewin Kanters 10 commandments Kotter’s 8 stage process for succesful org transformation Luecke’s seven Steps for Change

Beer’s six steps for change

Unfreeze Analyze the organization and its need for change Establish a sense of urgency Mobilize energy, commitment through joint identification of business problems and their solutions Mobilize commitment to change through joint diagnosis of

problems

Create a vision and a common direction Create a guiding coalition Develop a shared vision of how to organize and manage for competitiveness Develop a shared vision of how to organize and manage for competitiveness Separate from the

past

Develop a vision and strategy

Identify the leadership

Foster consensus for the new vision, competence to enact it, and cohesion to move it along Create a sense of urgency Empower broad-based action Support a strong leader role Line up political sponsorship Move Craft an implementation plan Communicate the change vision Focus on results, not activities Spread revitalization to all departments without pushing it from the top Develop enabling structures Generate short-term wins Start change at the periphery, then let it spread to other units, pushing it from the top Communicate,

involve people & be honest

Consolidate gains and produce more change Refreeze Reinforce and

institutionalize change Anchor new approaches Institutionalize success through formal policies, systems, and structures Institutionalize revitalization through formal policies, systems, and structures Monitor and adjust strategies in response to problems in the change process.

(12)

11

2.2 Specific healthcare implementation literature

Glasgow et al. (1999) presented the public health impact of an intervention as the function of two factors and three stages; reach, efficacy, adoption, implementation and maintenance. Reach measures the impact by the number of patients that is possibly impacted by the intervention. Efficacy measures the success rate if the intervention is implemented according to protocol. The stages adoption, implementation and maintenance show a parallel with the three phases of Lewin (1947). Adoption refers to the setting in which the decision is made to adopt the intervention, similar to the unfreezing stage and therefore not relevant in this research. Implementation refers to the phase in which the organization adapts to the working with the innovation, similar to the move phase. The maintenance phase in which the implementation is sustained and continued is similar to Lewin’s refreeze stage.

In a literature review and Delphi study held in the context of the Dutch healthcare system by Fleuren, Wiefferink and Paulussen (2004), the implementation and continuation steps were further explained by adding factors that are of importance in each of the two stages. The research found that the determinants (factors that facilitate or impede actual change) of innovation could be separated into five groups: (1) determinants related to the characteristics of the socio-political context, such as rules and regulations; (2) determinants related to organizational characteristics, such as the decision-making process in the organization; (3) determinants related to the characteristics of the person adopting the innovation, such as knowledge, skills and perceived support; (4) determinants related to characteristics of the innovation, such as complexity or relative advantage; and (5) determinants related to the characteristics of the innovation strategy. Fleuren et al. (2004) identified a total of 50 determinants that can be placed in one or more of the five groups. The determinants were, however, not ranked on importance as importance of individual determinants is dependent on the type of innovation. The research also did not indicate in which phase of the innovation process the determinants were relevant.

(13)

12 the intervention and accessibility to the target group were indicated as most important for all three stages (adoption, implementation and continuation). For the implementation phase the factors participants’ feedback, presence of the target group within the organization, provider knowledge, skills, motivation to deliver the intervention, and experience with the intervention’s effectiveness were found to be most important. The most important factors in the continuation phase were intervention’s sustainability, network between primary healthcare and local physical activity or sport facilities, participants’ feedback, provider motivation to deliver the intervention, introduction’s success, and availability of a list of local physical activity or sport facilities. The next section introduces a novel theory in implementation science which focuses on the implementation of interventions in healthcare organizations. It also shows the relationship between the literature that is presented in the next section and the literature that was presented in this section and the previous section.

2.3 Novel theory in healthcare implementation: Normalization Process Theory

(14)

13 1995; Luecke, 2003; Beer 1990). The vision component (Kanter, 2003; Kotter, 1995; Luecke, 2003; Beer 1990) also underwrites the importance of the participation component. Other steps that support the importance of participation are the identification and involvement of a strong leader (Kanter, 2003; Beer, 1990). The importance collective action component is underwritten by the writing of an implementation plan and the development of supporting structures (Kanter, 2003) and letting change spread to the organization rather than pushing from the top (Luecke, 2003; Beer, 1990). The importance of reflective monitoring is underwritten through the monitoring and adjusting steps (Luecke, 2003; Beer, 1990).

When the NPT is applied to the implementation of a complex intervention, it might lead to a number of questions that, when answered positively, suggest that the implementation is likely to succeed (Murray et al., 2010). Table 3 shows the NPT components and relevant questions, based on Murray et al. (2010), to consider when implementing a physical activity stimulation program. In the table it is also visible that there is overlap with a number of factors of Huijg et al. (2013), which can be used to answer the questions based on Murray at al., (2010). The factors of Huijg et al. were used as they have been developed specifically for the implementation of physical activity stimulation programs in healthcare. The factors that are presented in table 3 are limited to those factors that directly have a connection to the relevant questions to consider, and are limited to the factors that are relevant in the implementation and continuation stage.

Table 3. NPT components and the factors of Huigh et al. (2013) that correspond to the NPT components NPT Components

(May and Finch, 2009)

Questions to consider regarding NPT when implementing a physical activity

stimulation program (Murray et al., 2010)

Correspondence with factors of Huijg et al. (2013)

Coherence Is it a clear and understandable program? Do healthcare professionals see the benefits of the program?

Provider attitudes towards intervention effectiveness; fit with PHC organizations’ and professionals’ objectives; evidence for intervention effectiveness

Collective Action Does the program fit with overall organizational goals?

Are there financial resources available?

Does the program fit in the current work practices? financial resources for introduction;

Cognitive Participation

Are the patients engaging in, and committed to the program

Provider attitudes towards the intervention;

potential participant enthusiasm Reflexive monitoring Is there a possibility for feedback or

reflection the implementation process? Can the intervention be adapted or improved on the basis of experience?

participants feedback

(15)

14

2.4 Propositions

In the previous sections of the literature review we showed that there is overlap between NPT, change management literature and healthcare implementation literature. The literature review also shows that the NPT components influence the implementation success. It is possible to make this proposition for each of the four NPT components. These propositions are described in table 4 (P1a, P2a, P3a and P4a).

The influence of NPT on implementation success can also be expressed by the proposition that healthcare facilities that maintain or increase their NPT components realize a more positive continuation of the physical activity stimulation program, and healthcare facilities that show a decrease on NPT components have a negative continuation of the physical activity stimulation program. Table 4 shows the propositions expressing these relations for each NPT component (P1b, P1c, P2b, P2c, P3b, P3c, P4b and P4c )

Table 4. Propositions

NPT component coherence

P1a Healthcare facilities that score higher on NPT factor coherence have higher implementation success

P1b An increase in NPT factor Coherence is positively related to positive continuation of the implementation

P1c A decrease in NPT factor Coherence is negatively related to positive continuation of the implementation

NPT component collective action

P2a Healthcare facilities that score higher on NPT factor collective action have higher implementation success

P2b An increase in NPT factor collective action is positively related to positive continuation of the implementation

P2c A decrease in NPT factor collective action is negatively related to positive continuation of the implementation

NPT component cognitive participation

P3a Healthcare facilities that score higher on NPT factor cognitive participation have higher implementation success

P3b An increase in NPT factor cognitive participation is positively related to positive continuation of the implementation

P3c A decrease in NPT factor cognitive participation is negatively related to positive continuation of the implementation

NPT component reflexive monitoring

P4a Healthcare facilities that score higher on NPT factor reflexive monitoring have higher implementation success

P4b An increase in NPT factor reflexive monitoring is positively related to positive continuation of the implementation

P4c A decrease in NPT factor cognitive participation is negatively related to positive continuation of the implementation

(16)

15 Figure 2. Research model II, the presence of NPT factors is proposed to positively influence the

implementation success

3. Methods

In this chapter the methods used for this research are explained and accounted for. The first part of the chapter outlines the research design, including the case selection. The second and third subsections explain the methods used to collect and reduce data. The last part is devoted to the methods by which the data was analyzed.

3.1 Research design and case selection

As explained in previous chapters, the aim of this paper is to expand the knowledge regarding the factors explaining implementation success in rehabilitation healthcare by connecting NPT to change management and healthcare implementation literature, and to find out if NPT can be used to explain differences in healthcare implementation success in the rehabilitation care. The research also aims to find if the components of NPT influence the continuation level over time. The research design used in this paper followed a variety of the theory testing cycle as described by Van Aken (2012), namely the analytic induction approach (Robinson, 1958) in which hypothetical causes for an observed phenomenon are proposed, which are tested in the light of a case study.

(17)

16 implementation of the RSE program was observed in a total of twelve rehabilitation clinics and six rehabilitation departments of hospitals, so the sample size was equal to the total population.

The factors considered to influence the implementation success were the NPT components, and the success of implementation was measured by looking at continuation level and compliance rate. To increase the robustness of the results, the individual facilities that were studied were clustered according to their level of success of implementation. The process by which they were clustered is discussed later in this chapter, as well as chapter 4.

A longitudinal research design was used to measure change in the success of implementation over time during the implementation phase. This makes it possible to observe whether or not NPT components influence the continuation of the implementation of the RSE program. A within-case analysis was performed to research how the NPT components relate to the continuation of the implementation for each cluster, and a cross-case analysis was performed to reveal similarities or differences between the clusters. The data that was used for the analyses was quantitative in nature. The research was executed at an organizational unit level, which means that the data from the different respondents was combined per facility for the different statistical analyses. A holistic multiple-case study design (Yin, 2002) was used in this research.

In the results section we will discuss the implementation change content, context and process to find if it is feasible to discuss the NPT in the light of the RSE case.

3.2 Data collection methods

Two types of data were collected for this research. The first type, used for measuring implementation success consists of data about the number of patients that used the sports counseling center (SCC), which is an integral part of the RSE program, and the percentage that was treated according to protocol. This data was derived from the online registration tool for the RSE program. The online registration tool was used by sports counselors to fill out a form including basic information about the patients. Every patient using the SCC was given a unique registration number which made it possible to quantify the number of patients in every facility. The number of consultations offered to the patient was registered under ‘C_aangeboden_RT’ where C_aangeboden_RT=0 meant no consults, C_aangeboden_RT=1 meant between one and three consults and C_aangeboden_RT=2 meant four or more consults. The protocol prescribes that four consults should be given after the intake in the SCC, so the protocol is followed when C_aangeboden_RT=2.

(18)

17 this research were answered on a 5 point Likert scale, ranging from totally disagree to totally agree, except for the question ‘I stand behind the decision that the program rehabilitation sports and exercise is being implemented in our organization’, which was answered on a 4 point Likert scale ranging from totally disagree to totally agree. Appendix B shows the full questionnaire. The questionnaires have been filled out by managers, physicians, project leaders and sports counselors working at the facilities where the RSE was implemented (see Figure 3). So all roles involved in the implementation were included in the research to ensure reliability. The data collection for the NPT factors took place at three moments during the implementation phase (Hoekstra et al., 2015). The first measurement took place in April 2013 (T0), at the beginning of the program. The second in June 2014 (T1) and the last measurement took place in September 2015 (T2). The multitude in measurement points makes it possible to determine patterns of change and their direction. (Lewis-Beck et al., 2009, p.597). Figure 3 shows the timeline of the different measurements. The blocks in the left column show the different roles that were involved in the implementation process. The columns in the rest of the graph show which roles filled out the questionnaires and when, the arrow shows which role was responsible for entering data in the online registration system.

Figure 3. Timeline showing the measurement moments and methods used at the different moments

3.3 Data reduction methods

To be able to better interpret the data collected, the data was reduced. The first part of this subchapter discusses the data reduction methods, starting with the independent variables, on which a factor analysis was performed. In the second part, we discuss the treatment of the data regarding implementation success.

3.3.1 Independent variables

(19)

18 questions that were expected to measure a certain component, did indeed measure what they intend to measure. To ensure this internal validity, a confirmatory factor analysis was performed on the data. The factor analysis was carried out using the varimax rotation method. Variables that were attributed to more than one factor, without a sufficient difference in their loading for the different components, were subtracted, after which the factor analysis was performed again without the subtracted variable. The results of the factor analysis is presented in the results chapter. The Cronbach’s alpha of the constructs that resulted from the factor analysis can be found in table 11.

3.3.2 Dependent variable

The dependent variable, success of implementation, consisted of two components. The continuation level, which was measured by the absolute number of patients registered in the online registration tool over time. The second component was the compliance rate, measured by the number of patients that was treated according to protocol relative to the total number of patients registered in the online registration tool. To increase the robustness of the results of this research, the rehabilitation facilities were clustered according to success of implementation. The facilities were put in one of three clusters: positive, instable or negative. The clusters allow a better comparison between rehabilitation facilities with successful implementation of the RSE program, and facilities that did not successfully implement the RSE program.

The data that was collected with the online registration tool in each of the four periods can be found in the results chapter. The data was then displayed in graphs, showing the number of patients using the SCC and the percentage of patients treated according to protocol. Based upon the trend lines in the graphs, the facilities were clustered. The entire clustering process can be found in the results chapter.

3.4 Data analysis methods

(20)

19 aggregated to create an NPT component score per facility per measurement. From this data, the median and range were calculated. The facility type was attributed a value of 1 for rehabilitation clinics and 2 for hospital departments. For implementation success, a value of 1 was attributed to the positive cluster, 2 for the instable cluster and 3 for the negative cluster. The results of the Spearman correlation test can be found in table 11.

Next to the correlation test, both a cross-case analysis and a within-case analysis were performed. The cross-case analysis focused on testing whether there was a difference in the factors influencing the success of implementation in the cluster that successfully implemented the RSE program, and the cluster that did not successfully implement the RSE program. This was tested using the non-parametric Mann-Whitney U-Test, as this specific test is used to compare the difference between two independent groups, in this case different clusters. The dataset for this test consisted of the combined aggregated data from T1 and T2 of the positive and negative cluster.

The within-case analysis focused on how the response on the questions regarding the different factors influencing the success of implementation corresponded with the continuation of the implementation. The results of this test are presented in table 12. To test this, responses from different measurement moments were compared by performing the non-parametric Wilcoxon signed-rank test. The Wilcoxon signed-rank test is used to compare related samples, so it tests whether the facilities scored significantly lower or higher on the different NPT in T2 compared to T1. The results of this test can be found in table 12.

Because the study focuses on a low number of observations, the significance level was set at 0.1 for all tests. All tests and analyses in this research were performed by using IBM SPSS Statistics version 23.

4. Results

(21)

20

4.1 Case fit: implementation of a physical activity stimulation program

To get a better insight on what the RSE program entails, and get a better view of whether or not NPT is applicable to the RSE case, the RSE case will now be further reviewed. The change content, context and process framework proposed by Pettigrew (1987) is used to introduce the various aspects of the RSE program.

4.1.1 Implementation content

The

RSE program is an evidence-based program that was developed on the basis of the results of the study of Van der Ploeg (2006) and Van der Ploeg et al. (2007). The studies by Van der Ploeg et al. showed that patients who participated in the combined sports and active lifestyle stimulation program developed better daily physical activity and sports behavior, compared to patients who participated in the sports stimulation program and the control group. The RSE program was developed to improve the transition from rehabilitation clinics and hospital departments, where physical activity is often part of the treatment, and physical activities outside the rehabilitation facility. The program consists of six main components:

1. Intake session on exercise and sports. During the intake session the individual goals of the patients are discussed and an individual training plan is made, tailored to the needs of the patient.

2. Exercise and sports are standard components of the rehabilitation treatment in the clinic or hospital department.

3. Referral to Sports Counseling Center. Each clinic or hospital department in the program has set up a Sports Counseling Center (SCC) where the consultations for the RSE program take place. Patients are referred to a sports counsellor working at the SCC three to six weeks before the end of the rehabilitation process. The sports counsellor is responsible for supporting and stimulating the patient in physical activity in the home setting.

4. Face-to-face consultation in the SCC. Each patient receives a face-to-face consultation in which the counsellor gives advice about physical activity in the home setting.

5. Telephone-based consultations. Four telephone-based counseling sessions follow after the end of the rehabilitation process to support patients and further stimulate physical activity after the rehabilitation process.

(22)

21 ability to advise patients about the possibilities regarding physical activities in the region, which is an important part of the program. If patients experience that there are insufficient external sports facilities in the region, it is suggested that the clinic or hospital organizes activities for those patients.

4.1.2 Implementation context

Within each facility, multiple people are involved in the implementation of the program. The research focuses on four groups: managers, physicians, project leaders and counselors. The implementation of the program was coordinated by the Stichting Onbeperkt Sportief, a non-profit organization that strives to increase the participation in sports and physical activities for people with a handicap. The program is financially supported by the Ministry of Health, Welfare and Sport. The results of the program on patient level are measured by research group ReSpAct (Alingh et al., 2015) The ReSpAct study is a national research on the RSE program (Hoekstra et al., 2014; Hoekstra et al., 2015; Alingh et al., 2015). It follows and monitors the participating rehabilitation clinics and hospital departments during the implementation and execution of the RSE program. The participating clinics and hospital departments were selected on the basis of three criteria:

1. sufficient support for the RSE program from the professionals in the organization; 2. sufficient ambition to integrate exercise and sports into the rehabilitation treatment;

3. sufficient intention to continue the RSE program after the project period. (Hoekstra et al., 2014, p.3)

The participating clinics and hospitals then signed a declaration in order to formalize the adoption of the program. After formally adopting the program, the participants were instructed to make a plan for the implementation and continuation of the RSE program. Participation was then formalized by the signing of an agreement to participate by the head of the participating organization. The agreement included the following points

1. willingness to implement the RSE program according to the protocol during a period of three years (2012 - 2015);

2. willingness to participate in the ReSpAct study until December 2015;

(23)

22

4.1.3 Implementation process

During the implementation process, program coordinators of the Stichting Onpebeperkt Sportief support the adoption, implementation and continuation of the RSE program. The following points together form the most important aspects of the implementation process:

- the rehabilitation clinics and hospital departments involved in the implementation received a handbook guiding the implementation of the RSE program. The handbook gives an overview of steps that ought to be taken in order to successfully implement the RSE program;

- a three-day training course for Motivational Interviewing (MI), which is an important method in the consultation, is part of the program for the physicians and sports counselors. The MI course is followed by a yearly return-day to fresh up their MI skills.

- professionals from the clinics and hospital departments involved in the implementation wrote project plans, annual plans and annual reports which were being reviewed by program coordinators;

- to facilitate communication between the organizations involved in the implementation, the program coordinators and the ReSpAct team organized national and regional meetings where participating organizations have the opportunity to share knowledge and experiences; - the program coordinators and the ReSpAct research team are also available for support

during the implementation period.

(24)

23 multiple people are involved in the implementation and the implementation process consists of multiple steps. Therefore the RSE case is considered to be a complex intervention in a healthcare setting, and a good fit for this study.

Now that the sufficient background information has been provided on the case studied, and why it is suitably to use this case in this research, we turn to the results of the data reduction, collection and analysis methods.

4.2 Data reduction results

In this subchapter the results of the data reduction methods are discussed. The results of the factor analysis are presented first, followed by results of the clustering process.

4.2.1 Reducing the independent variables; factor analysis

In the literature review, four NPT components were proposed to influence the success of implementation. Therefore, the factor analysis was performed to reduce the data to four independent variables.

(25)

24 Table 5. NPT components and the questions by which they were measured

Factor (Huijg et al., 2013) Corresponding Question NPT component coherence Fit with PHC organizations’ and

professionals’ objectives

The program Rehabilitation, Sport and Exercise is in line with my vision on the rehabilitation treatment of my patients.

Provider attitudes towards intervention effectiveness

I can see the added value of the program Rehabilitation Sport and Exercise. NPT component collective action

Does the program fit in the current work practices?

The program Rehabilitation, Sport and Exercise fits well in the current situation of our organizations.

Financial resources for introduction There is enough money available in our organization to carry out the program Rehabilitation, Sport and Exercise according to protocol. NPT component cognitive participation

Provider attitude towards the intervention

I stand behind the decision that the program Rehabilitation, Sport and Exercise is being implemented in our organization.

Potential participants enthusiasm How do you estimate the support for carrying out an active policy towards exercise and sports among you colleagues? Colleague group sports counselors

How do you estimate the support for carrying out an active policy towards exercise and sports among you colleagues? Colleague group movement specialists

4.2.2 Implementation success: a search for clusters

(26)

25

Figure 4. Graph showing the data regarding the success of implementation of rehabilitation facility 1

The horizontal axis shows the four different observation periods that we defined. The left vertical axis gives the percentage of patients who were treated according to protocol, which is shown by the green line. The right vertical axis gives absolute number of patients that were registered in the SCC, which is shown by the blue bars. Trend lines were added to see how the number of patients and the percentage of those patients that was treated according to protocol developed over time. The black line is the trend line for the number of patients, the red line is the trend line for the percentage of patients being treated according to protocol.

The rehabilitation facilities were clustered based on the graphs. In order to make the trend lines for both the number of patients and the extent to which protocol was followed comparable, the graphs were converted to show the increase in the number of patients relative to the first period. All data was now shown as percentages and the derivatives of the functions of the trend lines now showed average percentage in- or decrease of the number of patients and the extent to which they were treated according to protocol. For example, the function for the trend line regarding the number of patients for example facility 1 was as follows:

𝑦 = −0,02𝑥 + 1,07 The derivative of this function is as follows:

(27)

26 This derivative predicts that the number of patients treated in each following period will be 2 percent point lower than the previous period. The derivatives of the trend line functions of facilities 1 till 17 were then used as input for a hierarchical cluster analysis using Ward’s method. facility 18 was not used as input because of missing data from the first period. The result of the hierarchical cluster analysis suggested the use of two clusters.

Based on the absolute number of patients, the relative number that received treatment according to protocol and the difference between those two, one cluster was made where the trend lines show that the level of adoption is stable or improves over the four periods and one cluster was made where the trend lines show a decline in the level of implementation over the four periods. This resulted in the following two clusters (see table 6)

Table 6. Initial clusters

Stable or positive cluster Negative cluster

Rehabilitation facility 2 Rehabilitation facility 4 Rehabilitation facility 5 Rehabilitation facility 9 Rehabilitation facility 10 Rehabilitation facility 13 Rehabilitation facility 14 Rehabilitation facility 15 Rehabilitation facility 16 Rehabilitation facility 17 Rehabilitation facility 1 Rehabilitation facility 3 Rehabilitation facility 6 Rehabilitation facility 7 Rehabilitation facility 8 Rehabilitation facility 11 Rehabilitation facility 12 Rehabilitation facility

(28)

27 spectrum are compared (Yin, 2002). Because the R2 of the trend lines regarding theprotocol were low overall, the inclusion criterion regarding whether or not protocol was followed was based on the percentage of patients treated according to protocol. The average number of patients treated according to protocol was 49% (4714 patient using the SCC, 2321 according to protocol). Healthcare facilities who treated 49% or more of their patients according to protocol were labeled as positive, below 49% as negative. In summary, the inclusion criteria for dividing the cases among the clusters were as follows:

1. rehabilitation facilities with positive or stable patient number trend lines are in the positive cluster, rehabilitation facilities with negative patient number trend lines are in the negative clusters;

2. rehabilitation facilities that treat more than the average of 49% of patients according to protocol are in the positive cluster, rehabilitation facilities that treat less than the average of 49% of patients according to protocol are in the negative cluster;

3. only rehabilitation facilities with an R2>0.4 are in the positive or negative cluster, the rest of the rehabilitation facilities is in the instable cluster.

(29)

28 Table 7. Data used for the clustering of the healthcare facilities

Healthcare facility

Number of patients using the SCC trend line function

Percentage of patients using the SCC according to protocol trend line function Percentage treated according to protocol R2 Number of patients R2 Protocol 1 Y=-0,02x+1,08 Y=-0,09x+0,78 53% 0,022 0,28 2 Y=0,23x+0,72 Y=-0,02x+0,55 49% 0,95 0,03 3 Y=-0,14x+0,98 Y=-0,00x+0,01 0% 0,53 0,60 4 Y=0.01x+1,14 Y=0,03x+0,85 91% 0,32 0,01 5 Y=0,26x+0,54 Y=-0,01x+0,96 94% 0,51 0,01 6 Y=-0,07x+1,03 Y=0,01x+0,97 98% 0,00 0,45 7 Y=-0,02x+0,86 Y=-0,02x+0,13 7% 0,03 0,34 8 Y=-0,04x+1,07 Y=0,01+0,11 13% 0,11 0,00 9 Y=0,51x+0,43 Y=0,03x+0,49 59% 0,71 0,11 10 Y=0,26x+0,80 Y=0,19x-0,22 38% 0,11 0,20 11 Y=-0,14x+1,10 Y=-0,01x+0,09 6% 0,66 0,04 12 Y=-0,24+1,48 Y=-0,14x+1,19 88% 0,41 0,88 13 Y=0,06x+0,90 Y=0,11x+0,40 68% 0,41 0,68 14 Y=0,10x+1,33 Y=0,01x+0,62 59% 0,08 0,00 15 Y=0,17x+1,48 Y=0,16x+0,28 67% 0,08 0,80 16 Y=0,04x+1,17 Y=0,01x+0,90 91% 0,03 0,03 17 Y=0,57x+0,34 Y=0,01x+0,26 28% 0,91 0,02 18 Y=6x-1,3333 Y=-0,21x+0,74 25% 0,6 0,94

Data highlighted green places a healthcare facility in the positive cluster, data highlighted red places a healthcare facility in the negative cluster. Healthcare facilities with a combination of red and green were placed in the instable cluster.

Table 8. Final clusters

Stable or positive cluster Instable cluster, low R2 Negative cluster Rehabilitation facility 2 Rehabilitation facility 5 Rehabilitation facility 9 Rehabilitation facility 13 Rehabilitation facility 1 Rehabilitation facility 4 Rehabilitation facility 6 Rehabilitation facility 7 Rehabilitation facility 8 Rehabilitation facility 10 Rehabilitation facility 14 Rehabilitation facility 15 Rehabilitation facility 16 Rehabilitation facility 3 Rehabilitation facility 11 Rehabilitation facility 18

4.3 Descriptive data analysis

In this subchapter, the data that was collected through the questionnaires and the online registration tool are presented, starting with the data from the questionnaires.

4.3.1 NPT data

(30)

29 scores of the different clusters. Within the clusters, the differences between the medians of T1 and T2 are also very small. The differences are smallest for cognitive participation, the largest differences can be found in NPT component collective action, but here it can also be said that the differences are very small. Because the entire population was part of this study, the descriptives show us, even without statistical tests, that there is little to no relationship between NPT and the cluster in which a rehabilitation facility is placed.

Table 9. Descriptive statistics NPT data

Measurement moment Positive Cluster Instable Cluster Negative Cluster All rehabilitation facilities median / range median / range median / range median / range

Coherence T1 + T2 4.71 / 0.47 4.56 / 0.71 4.58 / 1.42 4.60 / 1.42 T1 4.65 / 0.42 4.63 / 1.00 4.83 / 1.67 4.65 / 1.67 T2 4.75 / 0.42 4.70 / 0.70 4.33 / 1.17 4.68 / 1.83 Collective action T1 + T2 3.36 / 1.58 3.58 / 1.75 3.00 / 1.50 3.54 /1.83 T1 3.50 / 2.00 3.67 / 1.56 3.67 / 1.83 3.75 / 2.67 T2 4.00 / 2.50 3.67 / 3.33 4.00 / 2.83 3.75 / 3.33 Cognitive participation T1 + T2 4.28 / 0.31 4.28 / 0.43 4.37 / .20 4.28 / 0.49 T1 4.43 / 0.25 4.33 / 0.67 4.39 / .11 4.42 / 0.37 T2 4.17 / 0.67 4.27 / 0.50 4.33 / .33 4.22 / 0.67

The table shows three columns with the result of the positive, instable and negative cluster and a column with results for all rehabilitation facilities. The rows show the median and range of the responses on the NPT components for the different clusters. For each of the NPT components and each cluster, the table shows the median and range for all measurements in a certain cluster, as well as the mean and range for T1 and T2 separately.

4.3.2 Implementation success data

(31)

30 Table 10. Number of patients using the SCC per period and compliance rate per rehabilitation facility

Rehabilitation facility Number of patients period 1 Number of patients period 2 Number of patients period 3 Number of patients period 4 Total Number of patients Percentage of patients enrolled in the RSE treated according to protocol 1 84 104 64 91 343 53% 2 18 21 24 31 94 49% 3 196 87 100 95 478 0% 4 117 151 140 117 525 91% 5 29 28 26 55 138 94% 6 49 35 27 33 134 98% 7 136 88 91 125 440 7% 8 58 52 68 44 222 13% 9 51 57 124 115 347 59% 10 42 31 124 48 245 38% 11 46 32 39 23 140 6% 12 29 44 13 16 102 88% 13 86 78 101 95 360 68% 14 78 169 133 118 498 59% 15 28 81 51 54 214 67% 16 40 66 46 52 204 91% 17 28 34 65 71 198 28% 18 0 2 16 14 32 25%

Now that we have given an overview of the collected data, we continue with how the data on NPT components and the data on implementation success relate to each other.

4.4 Correlation matrix

(32)

31 Table 11. Correlation matrix of NPT components and implementation success

Variables N Median Range 1 2 3 4 5 6 7

1 NPT1 T1 16 4.65 1.67 .34✝ 2 NPT1 T2 16 4.68 1.33 .303 .39✝ 3 NPT2 T1 16 3.67 2.67 .393 .094 .60✝ 4 NPT2 T2 16 3.75 3.33 .351 .603** .114 .74✝ 5 NPT3 T1 13 4.42 .67 .382 .435 .019 .421 .27✝ 6 NPT3 T2 13 4.22 .67 .571** .533* -.147 .753*** .604** .74✝ 7 Facility type 16 1 2 -.430* -.015 -.293 -.176 -.057 .057 .61✝ 8 Impl. Success 16 2 2 .122 -.349 -0.86 .024 -.158 .217 .277 *Significance <.1 **Significance<.05 ***Significance<.01

✝

Cronbach’s Alpha NPT1 = Coherence NPT2 = Collective Action NPT3 = Cognitive participation

4.5 Within- and cross-case analyses

Table 12 shows the results of the cross-case analysis, for which a Mann-Whitney U-test was performed, and the result of the within-cluster analysis of the positive and negative cluster for which a Wilcoxon signed rank test was performed.

The results of the Mann-Whitney U test showed that there were no difference between coherence in the positive (N=8) and the negative (N=6) cluster (p=.852). Neither was there a difference for collective action (p=.662) and cognitive participation (p=.808).

For the within-case analysis, a Wilcoxon signed rank test was performed. This test compares the rehabilitation facilities in T1 with T2, to find if there is a significant difference in their NPT scores in the two measurements. The results on the test of the positive cluster (N=4) showed that there were no difference between T1 and T2 for coherence (p=.109), and collective action (p=1.000). There was a significant difference for cognitive participation (p=.066). Further investigation into this result, however, revealed that T2 – T1 gave a negative result. This means that the NPT score for cognitive participation was higher in T1 than in T2 in the positive cluster, which implies that a decrease in cognitive participation leads to positive continuation of implementation. This is opposite to what is proposed based on the literature.

(33)

32 Table 12. Results Mann-Whitney U-test, performed for the cross-case analysis

Cross-case analysis N positive cluster N negative cluster significance

Coherence 6 8 .852

Collective Action 6 8 .662

Cognitive Participation 6 8 .808

Table 13. Results Wilcoxon signed rank tests, performed for the within-case analysis of the positive and the

negative cluster

Within-case analysis, positive cluster N significance

Coherence 4 .109

Collective Action 4 1.000

Cognitive Participation 4 .066

Within-case analysis, negative cluster N significance

Coherence 3 .593

Collective Action 3 .593

Cognitive Participation 3 .593

4.6 Testing of the propositions

A look at the descriptive statistics table 9 shows that there is no significant difference in the results of the different clusters. This is confirmed by the correlations between implementation success and the clusters, which can be found in the correlation matrix (table 11). There are no significant correlations between the NPT factors and the implementation success. This indicates that there is no relationship between the score on NPT components and being in the positive, instable of negative cluster. This was also confirmed by the cross-case analysis (table 12), which did not show a significant difference in NPT scores between the different clusters. This means that the propositions that rehabilitation facilities that score higher on a certain NPT factor have a higher rate of implementation cannot be supported.

The within-case analysis only showed a significant result on cognitive participation in the positive cluster, and this result had an opposite relation to what was expected. The rest of the NPT components in the positive cluster, and all NPT components in the negative cluster, did not show significant difference in the score on NPT components between T1 and T2 (table 13). This means that none of the NPT factors seems to be related to the continuation level of the RSE program. The propositions that an increase or decrease in the score on NPT factors influence the continuation can therefore not be supported. The results regarding the propositions are summarized in table 14.

(34)

33 is no longer present, as the results did not indicate a relationship between NPT components and success of implementation in the RSE case. In the next chapter, the results are further discussed.

Table 14. Propositions and tests result on the proposition

Propositions Result

NPT component coherence

P1a Healthcare facilities that score higher on NPT factor Coherence have higher implementation success

Not Supported P1b An increase in NPT factor Coherence is positively related to positive

continuation of the implementation

Not supported P1c A decrease in NPT factor Coherence is negatively related to positive

Not supported NPT component collective action

P2a Healthcare facilities that score higher on NPT factor collective action have higher implementation success

Not supported P2b An increase in NPT factor collective action is positively related to positive

Not supported P2c A decrease in NPT factor collective action is negatively related to positive

Not supported NPT component cognitive participation

P3a Healthcare facilities that score higher on NPT factor cognitive participation h have higher implementation success

Not supported P3b An increase in NPT factor cognitive participation is positively related to

positive continuation of the implementation

Not supported P3c A decrease in NPT factor cognitive participation is negatively related to

positive continuation of the implementation

Not supported

(35)

34

5. Discussion

In this chapter, the results of the different tests presented in the previous chapter are discussed, interpreted and reflected upon. The results of the statistical analysis are also compared to existing literature. Theoretical and managerial implications are presented, and the chapter concludes by addressing the limitations and offering suggestions for further research.

5.1 Discussion of results

Some of the results of this study are in line with what is expected based on current knowledge. Huig et al. (2013) and Fleuren et al, (2004) found that there were much more factors influencing the success of implementation than just factors related to the innovation, or the support during the innovation. This is also visible in the RSE case. Table 10 shows the different rehabilitation facilities have a large difference in the number of patients using the SCC and in compliance rate, even though all facilities implement the same program and receive the same support. So differences in implementation success were expected, and did indeed occur. However, there are also big differences in expected outcomes and actual outcomes in this paper.

As shown in the previous chapter, the test results do not provide any evidence to support the propositions. Rather, the results suggest that there is no relation between NPT and successful implementation. Hence, all none of the propositions are supported (table 14). With regards to the research question: “Which implementation process factors influence the implementation level of an evidence-based physical activity stimulation program in the rehabilitation care, and how do these factors influence the continuation of the implementation?”, this means that a conclusive answer could not be drawn based on this research.

(36)

35

5.2 Theoretical implications

From a theoretical perspective, this results of this research do not support the importance of coherence, collective action, cognitive participation and reflexive monitoring. The results are in stark contrast with what was proposed based on the literature. This leads to the theoretical implication that either previous research drew incorrect conclusions about the relationship between NPT components and success of implementation, or that the causes of the discrepancy between the outcomes of this research and previous literature lie in the characteristics of this research or in de specificities of the RSE case. If there are indeed more important factors at play in the RSE case, that override the effects of NPT factors, the theoretical implication would be that further investigation should be done in the RSE case to find those specificities. However, we find it more likely that this research did not measure and did not test the relation between NPT components and success of implementation correctly. In that case, the theoretical implication is that the methodology used in this paper should not be replicated.

5.3 Managerial implications

Based on the results of this research and previous literature it is hard to make firm statements about the managerial or theoretical implications of this paper. One could suggest, based on this research, that it is unnecessary to consider NPT as a guiding theory during the implementation of an evidence-based program, such as the RSE, in healthcare. However, as stated in the theoretical implications, we find it likely that the results are influenced by characteristics of this research. Therefor we discuss the limitations of this research and make a number of recommendations for further research. Further research can be used to support the results of this research, or lead to different results with regard to the relationship between NPT and success of implementation, consequently refuting the outcomes of this research.

5.4 Limitations

(37)

36 Also, there are limitations with regard to the measurement of implementation success. Implementation success was measured as a combination of continuation level and compliance rate. The continuation level was measured by the number of patients registered in the online registration tool. Those numbers are absolute numbers (table 10), rather than relative numbers. A rehabilitation facility that shows high continuation level might, therefor, in reality only treat a very low percentage of the eligible patients with the RSE program. This means that facilities in the positive cluster might in reality have very low implementation success, and vice versa. This is important, because the method by which the clusters were made has a big influence on the test results, it is well possible that other cluster compositions would result in other results. The absence of data on the size of the institution also makes it hard to judge the different facilities because it is possible that enrolling a large percentage of the total patient population in the RSE program makes it more difficult to treat patients according to protocol due to resource allocations. This makes it possible for large facilities to show positive implementation with the measurement method in used in the paper, while actually only applying the RSE program to a very small part of the patient population eligible to use the SCC.

Then, there are some problem with the usage of NPT. This research attempts to measure the relationship between the four components of NPT and success of implementation. However, the component reflexive monitoring was not measured, as it was subtracted in the factor analysis process. Therefore, not all components of NPT were measured. Also, the other three NPT components, that were formed based on the factor analysis, showed low Cronbach’s alphas. The Cronbach’s alphas had a range of .27 and .74, which implies that there is a low relationship between the different questions measuring an NPT factor, which means there is low construct validity.

Problems also arise from testing the presence of NPT components by asking questions based on the factors of Huijg et al. (2013). This method is not optimal, because although the factors were designed to enhance the development of effective introduction strategies for physical activity interventions, they were not designed to measure NPT components. Also, the research of Huijg et al. (2013) was conducted in a primary healthcare setting, whereas this study found place in a secondary healthcare setting.

(38)

37

5.5 Further research

A number of recommendations can be made for further research. As the research methods used in this research did not lead to satisfactory results it would be too early to propose research that follows up on this research. Rather, we recommend to carry out a similar research, using the same research question and the same case study, but with alternative research methods than used for this research, rather than give direction for research that follows up on this research.

The limitations to this research are to a great extent related to the quantitative nature of the research. As the sample size is small, which limits the statistical power, we recommend further research to use a qualitative approach when studying the RSE case. Interviews are less restrictive than questionnaires, which would enable an inductive research design. Interviewing participants allows the researcher to create deeper insights in the presence of NPT components, but also the possibility to find other important factors at play in the RSE case, that may have overriden the effects of NPT factors. We recommend to create an interview guide that is based directly on NPT components, rather than through factors of Huijg et al. (2013), as they are not developed specifically for measuring NPT components. The interview should open with open and broad questions, followed by questions based upon the theory. It is also recommendable to ensure that the participants in the research do so on a voluntary basis, rather than being required to participate, as the former may well increase the reliability of the interviews.

Clustering was difficult, because between the four periods the facilities showed large differences in the numbers of patients using the SCC and in compliance rate. Therefore, we suggest to no longer cluster the facilities, but rather threat the facilities as individual cases, hereby applying an embedded case study design. Interviews can then be held in some facilities where the implementation was successful, and some in facilities where implementation was not successful. The interviews can be used to compare the institutions, and identify the cause of implementation success.

(39)

38

6. Conclusion

(40)

39

Bibliography

Alingh, R. A., Hoekstra, F., van der Schans, C. P., Hettinga, F. J., Dekker, R., & van der Woude, L. H. (2015). Protocol of a longitudinal cohort study on physical activity behaviour in physically disabled patients participating in a rehabilitation counselling programme: ReSpAct. BMJ Open, 5(1), e007591.

Andersson, U., Cuervo-Cazurra, A., & Nielsen, B. B. (2014). From the Editors: Explaining interaction effects within and across levels of analysis. Journal of International Business Studies, 45(9), 1063– 1071.

Beer, M., Eisenstat, R. A., & Spector, B. (1990). Why change programs don’t produce change.

Bouamrane, M.-M., Osbourne, J., & Mair, F. S. (2011). Understanding the implementation & integration of remote & tele-health services… an overview of Normalization Process Theory. In Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2011 5th International Conference on (pp. 300–307). IEEE.

Bullock, R. J., & Batten, D. (1985). It’s just a phase we’re going through: a review and synthesis of OD phase analysis. Group & Organization Management, 10(4), 383–412.

Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., ... & Kinmonth, A. L. (2007). Designing and evaluating complex interventions to improve health care. Bmj, 334(7591), 455-459.

Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci, 8(1), 22.

Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3-4), 327–350.

Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implement Sci, 1(1), 1–3.

(41)

40 Fallon, E. A., Wilcox, S., & Laken, M. (2006). PEER REVIEWED: Health Care Provider Advice for African

American Adults Not Meeting Health Behavior Recommendations. Preventing Chronic Disease, 3(2)./

Finch, T. L., Mair, F. S., O’Donnell, C., Murray, E., & May, C. R. (2012). From theory to'measurement'in complex interventions: Methodological lessons from the development of an e-health normalisation instrument. BMC medical research methodology, 12(1), 1.

Fleuren, M., Wiefferink, K., & Paulussen, T. (2004). Determinants of innovation within health care organizations. International Journal for Quality in Health Care, 16(2), 107–123.

Glasgow, R. E., Eakin, E. G., Fisher, E. B., Bacak, S. J., & Brownson, R. C. (2001). Physician advice and support for physical activity: results from a national survey. American Journal of Preventive Medicine, 21(3), 189–196.

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health

promotion interventions: the RE-AIM framework. American Journal of Public Health, 89(9), 1322– 1327.

Graham, I. D., & Tetroe, J. (2007). Some theoretical underpinnings of knowledge translation. Academic Emergency Medicine, 14(11), 936–941.

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations. Milbank Quarterly, 82(4), 581– 629.

Hoekstra, F., Alingh, R. A., van der Schans, C. P., Hettinga, F. J., Duijf, M., Dekker, R., & van der Woude, L. H. (2014a). Design of a process evaluation of the implementation of a physical activity and sports stimulation programme in Dutch rehabilitation setting: ReSpAct. Implementation Science, 9(1), 127.

Hoekstra, F., Hettinga, F. J., Alingh, R. A., Duijf, M., Dekker, R., van der Woude, L. H., & van der Schans, C. P. (2015). The current implementation status of the integration of sports and physical activity into Dutch rehabilitation care. Disability and Rehabilitation, 1–6.