Change interventions that influence adoption behaviors of mental healthcare professionals during e-therapy implementations

Change interventions that influence adoption behaviors of mental healthcare professionals during e-therapy implementations

March, 2018

MARTIJN VINCENT FABER Student number: S2810530

m.v.faber@student.rug.nl +31 (0) 652646277

Master Thesis

MSc. BA - Change Management Faculty of Economics and Business  

University of Groningen    

Supervisor RUG Dr. M.A.G. van Offenbeek

Co-assessor RUG Dr. J.FJ. Vos Supervisor Lentis

Dr. M.R. Dekker Word count: 14.702

ABSTRACT

This research analyzed the implementation process of an e-therapy technology to provide insights into how change interventions influence adoption behaviors of mental healthcare professionals. This research is organized in two phases. The first phase encompassed a quantitative research in the form of a survey, where different user-groups, drawing on a model of four behavioral categories, are identified. In the second phase, interviews with mental healthcare professionals of different user-groups were conducted in order to gain a deeper understanding of the professionals’ experiences with the e-therapy technology and the implementation of this technology. Moreover, this study focused on group-level technology adoption and analyzed if user-groups perceptions towards the effectiveness of change interventions differed. The findings show that management support, organizational support and training are important change interventions in the post-implementation phase to increase and sustain adoption behaviors of mental health professionals.

Keywords: healthcare, eHealth, technologies, implementation, adoption behaviors, change,

interventions

Table of contents

1. INTRODUCTION  ...  6  

2. THEORETICAL BACKGROUND  ...  8  

2.1 E-therapy  ...  8  

2.1.1 Minddistrict  ...  8  

2.2 E-therapy implementations in healthcare organizations  ...  9  

2.3 The implementation model of Cooper and Zmud  ...  10  

2.4 Users’ reactions  ...  10  

2.5 Antecedents of acceptance and resistance behaviors  ...  12  

2.6 Change interventions  ...  13  

3. METHODS  ...  16  

3.1 Context  ...  16  

3.2 Research part I: Survey  ...  17  

3.2.1 Measurements  ...  18  

3.2.2 Data analysis  ...  21  

3.3 Research part II: Interviews  ...  21  

3.3.1 Data analysis  ...  22  

4. RESULTS  ...  22  

4.1 Results part 1  ...  22  

4.1.1 Correlation analysis  ...  23  

4.1.2 Regression analysis  ...  24  

4.1.3 Adoption reactions of professionals  ...  26  

4.2 Results part II  ...  28  

4.2.1 User group 1: supporting users  ...  29  

4.2.2 User group 3: resisting non-users  ...  31  

4.2.3 User-group 4: supporting non-users  ...  32  

4.3 Cross user-group analysis  ...  33  

5. DISCUSSION  ...  36  

5.1 Variables that influence use and support behaviors  ...  37  

5.2 Change interventions that were valued by the user-groups  ...  37  

5.3 Change interventions that influence adoption behaviors  ...  38  

5.4 Theoretical discussion  ...  39  

5.5 Practical implications  ...  40  

5.6 Limitations and suggestions for further research  ...  41  

6. CONCLUSION  ...  41  

References  ...  43  

Appendix A: Invitation  ...  48  

Appendix B: Survey  ...  49  

Appendix C: Descriptives of teams and functions  ...  60  

Appendix D: Measurement items and scales  ...  61  

Appendix E: Factor analysis  ...  64  

Appendix F: Interview protocol Lentis (Dutch)  ...  65  

Appendix G: Correlation analysis  ...  69  

Appendix H: Adoption reactions on division level  ...  70  

Appendix I: Coding scheme  ...  71  

1. INTRODUCTION

Healthcare organizations have to deal with a greater external pressure to manage the access, quality, and costs of healthcare provision (Embertson, 2006; Wu, Kao & Sambamurthy, 2016). Furthermore, the tendency of the western world’s ageing also puts pressure on the healthcare sector (Murray et al., 2011). This demographic trend does not only cause economic problems due to the decreasing ratio of tax paying workers, but also because higher life expectancies in the end will lead to increased costs in the healthcare sector as well

(Meulendijk et al., 2011; Scanaill et al., 2006). Besides that, there is also a dramatic decrease of available young professionals in this sector (Chan et al., 2009). Consequently, as stated by Owens et al. (2011), further effort is needed to control healthcare costs and to maintain or improve the quality of care.

Technological innovations are a potential solution to the above mentioned challenges in healthcare (Meulendijk et al., 2011). An example of such an innovation is the use of health- related internet applications (eHealth), which is expected to help increase efficiency, decrease costs, and empower individuals to actively manage their health by adopting healthy behaviors (Wicks et al., 2014; Wilson & Lankton, 2004). Therefore, a deeper understanding is needed into the opportunities and concerns of implementing eHealth in healthcare.

Oh et al. (2005) found in their systematic review that the definition of Eysenbach (2001) of the term eHealth is often used and is used in this research as well:

E-health is an emerging field in the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced through the Internet and related technologies. In a broader sense, the term

characterizes not only a technical development, but also a state-of-mind, a way of thinking, an attitude, and a commitment for networked, global thinking, to improve healthcare locally, regionally, and worldwide by using information and

communication technology (Eysenbach, 2001, p. 1).

The expectations about eHealth are mixed. On the one hand, today’s clients are willing to

have a greater involvement in their own treatment process and, therefore, it can be stated that

the use of eHealth applications will meet in this request (Wilson & Lankton, 2004). Besides,

eHealth applications have the potential to stimulate clients in their commitment to the

treatment process (Andreassen et al., 2007). On the other hand, according to the literature,

there are also some concerns when it comes to using eHealth applications about the concepts

of client information and inappropriate use (Andreassen et al., 2007). Furthermore, in the

Netherlands is large-scale adoption of eHealth technologies lacking (eHealth monitor, 2016).

Although the last few decades studies created a better understanding in the usefulness of eHealth applications, healthcare organizations are still not very successful in adopting and gaining benefits from eHealth (Wu et al., 2016). Project delays, budget deficits and negative impacts on the quality and effectiveness of healthcare are problems being widely reported caused by implementation of eHealth technologies (Murray et al., 2011). Besides, eHealth technologies are most strongly resisted by that group of people which is expected to gain the greatest benefits from it (Bhattacherjee & Hilkmet, 2007). Health professionals have

expressed concerns that eHealth applications will not fully meet their needs, which can result in decreased effectiveness and efficiency in the delivery of healthcare (Goh, Geo & Agarwal, 2011). Implementing eHealth applications in the right way could prevent for the problems mentioned before. Despite extensive studies about the value of eHealth solutions in healthcare organizations, limited research has been conducted on the mechanisms underlying successful implementations of eHealth applications (Goh et al., 2011). A better understanding of the change interventions that influence adoption behaviors during eHealth implementations in the mental healthcare sector will lead to more successfully eHealth implementations and

eventually to healthcare organizations that can gain the benefits from eHealth. This study, therefore, contributes the to existing literature by refining the understanding of the change interventions that underlie acceptance and/or resistance and result in differences in adoption behaviors between user-groups. As such, the aim of this study is to develop a more thorough understanding of why health professionals adopt a new system in the context of an eHealth implementation by exploring how different user-groups, drawing on a model of four behavioral categories (Van Offenbeek, Boonstra & Seo, 2013), valued the effectiveness of different change interventions. Therefore, this study intended to answer the following research question: ‘How do change interventions influence the adoption behaviors of mental healthcare professionals during e-therapy implementations?’

This study focused on adoption of eHealth implementations in mental healthcare settings. Furthermore, the scope of this study was limited to research focusing on work practices. The change interventions that influence the adoption behaviors of eHealth in current work practices were assessed. Additionally, we only chose to focus on one user group.

Only healthcare professionals as users of the eHealth application were included in this study.

Looking at the managerial interest, this study advances our understanding of adoption behaviors during eHealth implementations. Moreover, this study focused on group-level technology adoption and analyzed if user-groups perceptions towards the effectiveness of change interventions differed.

This paper is structured as follows. The next chapter gives a theoretical background

of relevant concepts. Thereafter, the methods section describes which approaches are used for

this case study. A result section then presents the study’s findings. Finally, this study ends

with a conclusion and discussion section that outlines theoretical and practical contribution, limitations and suggestions for further research.

      2. THEORETICAL BACKGROUND

In this section, we elaborated on state of the art knowledge in literature of eHealth

implementations in the healthcare sector and, more specifically, on e-therapy as an eHealth technology. Further, we discuss why and how the behavioral oriented framework of Van Offenbeek et al. (2013) serve as appropriate theoretical framework for this study.

2.1 E-therapy

The Internet provides opportunities towards the provision of online mental health services and the use of eHealth has shown promising results in various mental health treatments (Wentzel et al., 2016; Wells et al., 2007). Online mental treatment- or, more specifically, e-therapy- services provide opportunities for private communication between health professionals and their clients and are coupled with relatively low cost (Wells et al., 2011). Further, the provision of e-therapy improves accessibility to high quality care and empowers clients (Boonstra et al., 2011). E-therapy covers a broad range of mental health services such as electronic provision of individual counseling, group and family therapy and professionally facilitated online support groups (Wells et al., 2011). For this study, we focused on the use of e-therapy that supports an individual’s personal rout to recovery by means of technology (Minddistrict, 2018).

E-therapy is often portrayed as a promising eHealth technology in terms of reducing costs and waiting lists (Richards, 2009). Moreover, it can be a solution to the social problem of the decrease in young professionals working in the healthcare sector (Chan et al., 2009).

Despite the increasing attention, the diffusion of e-therapy remains limited, mainly because healthcare organizations are still not very successful in adopting and gaining benefits from eHealth (Wu et al., 2016). In this study, the implementation process of an e-therapy application is analyzed in order to get an understanding of why and how mental health professionals adopt and integrate it in their work practices.

2.1.1 Minddistrict

At the case organization, a large mental healthcare organization in the Netherlands, we

analyzed the implementation of one particular e-therapy application. The case study focused

on 23 teams of three departments where health professionals work with the so-called

Minddistrict application. Minddistrict is a product that supports an individual’s treatment

process by means of technology, from prevention to aftercare (Minddistrict, 2018). In

literature, the e-therapy application used at the case organization is known as a tool that

provides online interventions in a structured way through text, audio, or explanatory videos

(Doherty, Coyle, & Sharry, 2012). The different modules of Minddistrict are based on relevant theories and knowledge from healthcare providers, but especially on the needs of clients. Examples of modules are building confidence, dealing with thoughts and feelings, finding balance (Minddistrict, 2018).

2.2 E-therapy implementations in healthcare organizations

The implementation of an e-therapy technology is often seen as a simple, safe, cheap and effective solution. However, previous research shows that the interplay between the new e- therapy technology and existing work practices is complex and may lead to unintended outcomes (Farre & Cummins, 2016). The implementation of an e-therapy technology alone is not enough to accomplish the intended result. Boonstra et al. (2011) mentioned the

importance of strategic alignment, which is based on the view that it is only when IT is effectively aligned with the strategy of the organization, processes and practices will IT enable organizations to achieve goals. This alignment seems to be difficult in the healthcare sector, where managers, health professionals, and the IT department follow different rationalities and have different domain knowledge (Heeks, 2006). Besides, managers often experience difficulties in achieving alignment by relating the new technologies, such as e- therapy, with their business strategies and structures (Boonstra et al, 2011).

Adoption of technologies by individuals is always subject to management support (Battilana et al, 2010). Top-down management support is not enough to accomplish a change in the whole organization, especially in healthcare organizations where the systems are complex, power is distributed among professional groups, and professions have their own norms and values (Chreim et al., 2010). Therefore, Best et al. (2012) mentioned that leadership must be designated and distributed. Distributed leadership, i.e., teams and health professionals must share responsibilities for mobilizing the efforts and delivering project components, is necessary to realize the change (Best et al., 2012).

To summarize, effective diffusion of e-therapy requires adaptations of current work

practices, reorientation, and organizational change, especially in the knowledge-intensive

mental healthcare sector (Tsiknakis & Kouroubali, 2009). Previous research mentioned the

important factor of the collaboration with professionals during e-therapy implementations

(Kaye et al., 2010). Professionals in healthcare have a high degree of autonomy, which is

difficult to change (Tsiknakis & Kouroubali, 2009). Subsequently, it is a challenge for mental

healthcare organizations to successful implement the e-therapy technologies and gaining the

benefits from it. According to Murray et al. (2011), professionals will only use a new e-

therapy application when they perceive a positive impact on interactions between

professionals and client.

2.3 The implementation model of Cooper and Zmud

The model of the IT implementation process from Cooper and Zmud (1990) is used to structure this study. Cooper and Zmud (1990) propose a stage model of IT implementation founded on Lewin’s (1952) change model. In this model, IT implementation is defined as: ‘an organizational effort directed toward diffusing appropriate information technology within a user community’ (Cooper & Zmud, 1990, p. 124).

Looking at the six-phase model, it can be noticed that the first phase is focused on awakening the organization and prepare the organization for the new system. In the stage theory of change (Lewin, 1952) this is mentioned as the ‘unfreezing’ stage. Adoption and adaption are linked with the ‘change’ stage and acceptance, routinization, and infusion are associated with the ‘refreezing’ stage. In the ‘refreezing’ stage, users will need to adapt to the changes in the organization and develop new patterns and habits (Cawsey et al., 2016).

The ultimate goal of an IT implementation is to accomplish the infusion phase. Infusion in this study is defined as a state when the organization obtained increased effectiveness by using the technology in a more comprehensive and integrated manner to support higher levels aspects of organizational work (Cooper & Zmud, 1990).

As mental healthcare organizations are still not very successful in gaining benefits from realized e-therapy implementations, the scope of this study is directed at the ‘refreezing’

phases to gain a better understanding of the adoption behaviors of mental healthcare

professionals. These three phases from the IT implementation model are displayed in Figure 1. Furthermore, since the case organization has passed the adaption phase and is now working on individual users’ acceptance leading to new organizational routines and infusion of e- therapy in the teams, this study focused on the post-implementation phase. The results are analyzed at the group level. In this study, the term users intends to describe the mental health professionals of the case organization that have to work with the e-therapy technology.

E-therapy implementation

Figure 1. IT refreezing phases, based on Cooper and Zmud (1990)

2.4 Users’ reactions

Many change projects that deal with the implementation of IS never realize the intended benefits. Therefore, the awareness and understanding of users’ acceptance is important (Lauterbach & Mueller, 2014). A lot of research is done on the determinants of IS adoption

Acceptance   Routinization   Infusion  

behaviors and use. Most studies to IS adoption behaviors look at either user acceptance (Compeau & Higgens, 1995; Davis, 1989; Davis et al., 1992; Venkatesh et al., 2003) or user resistance (Markus 1993; 2004; Lapointe & Rivard, 2005). Acceptance and resistance are associated with a range of behaviors such as use, appropriation, misuse, and non-use, which are related to various impacts for users and organizations (Bagayogo, Beaudry & Lapointe, 2013). However, this traditional one-dimensional view fails to acknowledge that such behavior can be ambivalent (Seo, Boonstra & Van Offenbeek, 2011). In their research to managing adoption behaviors in ambivalent groups during IS implementations Seo et al.

(2011) mentioned that user behaviors might consist of both user resistance- and user acceptance aspects. Behaviors such as: ‘supporting but no or low usage’ and ‘resisting but high usage’ are ambivalent (Seo et al., 2011, p. 68). Therefore, Van Offenbeek et al. (2013) introduced a framework with a two-dimensional view were acceptance and resistance are considered as two different dimensions instead of one behavioral dimension.

In this study, resistance is defined as ‘opposition by an actor, or a group of actors, to the change associated with IS implementation’ and acceptance as ‘user’s employment of a system to perform a task’ (Van Offenbeek et al., 2013, p. 438). The behavior-oriented framework (Figure 2) shows that the behavioral component is positioned on a unipolar continuum from high use to non-use, whereas resistance is positioned on a continuum from enthusiastic support to aggressive resistance (Van Offenbeek et al., 2013). The behavior- oriented framework is used in this study to identify and categorize different user-groups at the case organization.

Figure 2. Behavior oriented framework (Van Offenbeek et al., 2013).

2.5 Antecedents of acceptance and resistance behaviors

Users’ reactions can be identified by measuring the antecedents of acceptance and resistance behaviors (Van Offenbeek et al., 2013). For this study, we used the antecedents from the acceptance and resistance literature as identified by Van Offenbeek et al. (2013). We included antecedents that are identified by the case organization as most influential. After measuring users’ reactions, professionals can be categorized in one of the four behavioral user groups (see Figure 2): (1) supporting-user, (2) resisting-user, (3) resisting non-user), and (4) supporting non-user.

Use- / non use behaviors

Perceived usefulness: The degree to which a user believes that using a particular system would enhance his or her job performance (Davis, 1989).

Perceived ease-of-use: The degree to which a user believes that using a system would be free of effort (Davis, 1989).

Self-efficacy: The belief that one has the capability to perform a particular behavior (Compeau & Higgens, 1995).

Previous studies showed the relationship between these antecedents and use- / non-use behaviors (Davis, 1989; Davis et al.,1992; Compeau & Higgens, 1995; Venkatesh et al., 2003). Therefore the following hypothesis can be formulated:

Hypothesis 1: Perceived usefulness, perceived ease-of-use, and self-efficacy are positively related to use behaviors.

Support-resistance behaviors

Facilitating conditions: The degree to which a user believes that an organizational and technical infrastructure exists to support use of the system (Venkatesh et al., 2003).

Compatibility in norms and values: The degree to which the implementation of a system is felt to be consistent with one’s norms and values (Kappos & Rivard, 2008).

Subjective norm: The degree to which an user perceives that most people who are important to him think he should or should not use the system (Fishbein & Azjen, 1975;

Venkatesh & Davis, 2000).

Previous studies showed the relationship between these antecedents and support-resistance behaviors (Kappos & Rivard, 2008; Venkatesh et al., 2003; Venkatesh & Davis, 2000).

Therefore the following hypotheses can be formulated:

Hypothesis 2: Compatibility in norms and values and subjective norm are positively related

to support behaviors

Hypothesis 3: Facilitating conditions is positively related to both acceptance and support behaviors.

Figure 3. Conceptual model for sub-question 1

In order to develop a more thorough understanding of why mental health professionals adopt an e-therapy application, we must first identify the different user-groups at the case

organization. The case study aims to answer the following sub-question:

Sub-question 1: Which user-groups in terms of adoption behaviors emerge after the implementation of e-therapy and how can these be explained?

2.6 Change interventions

The adoption of technology can be studied on organizational, group and at the individual level (Venkatesh, 2006). As mentioned before, this study focused on group-level technology adoption and analyzed if user-groups perceptions towards the effectiveness of change

interventions differed. Venkatesh and Bala (2008) classify change interventions as follows: ‘a set of organizational activities that can potentially lead to greater acceptance of the system’

(Venkatesh & Bala, 2008, p. 292).

Identifying change interventions that could influence adoption behaviors of a new

system support managerial decision-making on successful strategies for IS implementations

(Venkatesh & Bala, 2008). In this study, we asked professionals for their perceptions towards various types of change interventions which are mentioned in the IS implementation literature (Markus, 1983; Venkatesh & Bala, 2008; Rivard & Lapoint, 2012; Van Offenbeek et al., 2013). The following change interventions were mentioned by the professionals during the interviews and, therefore, are discussed below.

First, coercion refers to forcing use of the system using coercive power (Rivard &

Lapointe, 2012). For instance, next time that users tries to use the old system, tell them that they cannot (Rivard & Lapointe, 2012). However, the use of coercive power can negatively influence the relationship with the user and therefore can harm the long-term relationship (Allen et al., 2000).

The second intervention, management support, refers to the degree to which users believe that management shows commitment for the implementation and support the use of the new system (Venkatesh & Bala, 2008). Management must make the need for the change and share their vision of the need for change with the users (Battilana et al., 2010). Moreover, Van der Vaart et al. (2016) mentioned that support from management is an essential factor among non-users to start using the system.

Third, organizational support refers to informal or formal activities or functions to assist users in using a new system effectively (Venkatesh & Bala, 2008). Previous research shows that organizational support will positively influence user adoption (Venkatesh et al., 2003). Jasperson, Carter, & Zmud (2005) mentioned the importance of internal or external experts who can support users when they encounter problems with the new system. After the implementation it is important to continually monitor and evaluate the progress. Therefore, achievement should be measured against the project goals of the implementation (Fuih-Hoon, Lee-Shang & Kuang, 2001).

Fourth, training is a type of intervention which can positively affect the adoption process (Sharma & Yetton, 2007). It is widely accepted that training interventions can have an important influence in enhancing adoption behaviors (Venkatesh, Speier, & Morris, 2002).

The emphasis of training has been on educating users to work with the new system (Markus, 1983). Eventually, the goal of educating the users is that technologies are used as designed and create successful outcomes beyond just use (Venkatesh, 2006).

Fourth, discussing issues is an intervention to increase usage and support (Rivard &

Lapointe, 2012). The intervention refers to discussing issues that arise during the implementation process. Discussing issues is important for involving users in the

implementation process, which increase the probability of adoption (Aubert & Hamel, 2001).

Moreover, the involvement in the implementation process enables users to take ownership of

working with the new system (De Weger et al., 2013).

The last intervention included in this study is peer support. Peer support is a type of intervention where different activities performed by co-workers may help a user effectively use a new system (Venkatesh & Bala, 2008). Co-workers can give instructions through formal and informal training about the system which can enhance users’ understanding of a system (Venkatesh & Bala, 2008).

To sum up, we focus on these six types of interventions summarized in the Table 1:

Table 1. Change interventions Change

intervention

Definition Sub-types Example from literature

Literature

Coercion Use of coercive power during an intervention.

“The next time the user tries to use the old process, tell them that they can not”

(Allen et al., 2000;

Rivard & Lapointe, 2012)

Management support

The degree to which users believe that management has committed to the implementation and use of the new system.

Top-management support

“My manager has a clear vision for the change and motivates me”

(Venkatesh & Bala, 2008; Battilana et al., 2010; Van der Vaart et al., 2016)

Middle-management support

Lower-management support

Organizational support

Informal or formal activities or functions to assist users in using a new system effectively.

Role of super user “There is an internal or external expert available who can support me when I encounter

problems with the new IS”

(Venkatesh et al., 2003; Venkatesh &

Bala, 2008;

Jasperson, Carter, &

Zmud, 2005;

Fuih- Hoon, Lee-Shang

& Kuang, 2001

of online contacts were discussed in team meetings”

Training Educating users to work with the new system.

“Training can help users decide how to cope with or adapt a new IT”

(Sharma & Yetton, 2007; Venkatesh et al., 2002; Markus, 1983; Venkatesh, 2006)

Discussing issues Discussing issues that arise during the implementation process.

“Discussing issues to gain our understanding would enhance my use”

(Rivard & Lapointe, 2012)

Peer-support Activities performed by co- workers may help a user effectively use a new system

“My co-worker gives me clear instructions about how to use the system”

(Venkatesh & Bala, 2008; Jasperson et al., 2005)

To summarize, this section discussed the usefulness of the behavioral-oriented framework of Van Offenbeek et al. (2013) for this study and introduced sub-question 1:

Sub-question 1: Which user-groups in terms of adoption behaviors emerge after the implementation of e-therapy and how can these be explained?

Furthermore, different change interventions that can influence adoption behaviors during e- therapy implementations are discussed. This study focused on group level technology adoption and analyzed if user-groups perceptions towards the effectiveness of change interventions differed. The second sub-question for this study is:

Sub-question 2: Which change interventions are valued by these different user-groups and why?

            3. METHODS

This study consisted of a case study design. A case study is appropriate because it is a research approach that focuses on understanding the dynamics present within single settings (Eisenhardt, 1989). The literature about implementation of eHealth technologies in healthcare organizations is mature, however it lacks some context-specific knowledge of why mental health professionals adopt an e-therapy technology and how change interventions influence the adoption behaviors.

Exploring this phenomenon in the mental healthcare sector is relevant because e- therapy is an emerging field and despite the extensive research to the value of eHealth for mental healthcare organizations, limited research has been conducted on the mechanisms underlying successful adoption of e-therapy technologies. In order to understand the dynamics of the implementation of an e-therapy technology, and more specifically why different user-groups manage to adopt (or not) an e-therapy application and integrate it in their work practices, data was collected using a survey at 23 teams and subsequently twelve interviews were conducted with professionals in order to analyze which change interventions are valued by the different user-groups.

3.1 Context

Data for this study was collected at a large mental healthcare organization in the Netherlands.

Data was collected over 23 teams, where the investigated e-therapy technology was

implemented. These teams belong to three divisions. Division A, basic mental healthcare,

helps adults with mild to moderate psychological problems. Division B, offers treatment to

children and young adults who are experiencing mental health issues. Division C, specialist

mental healthcare, helps adults with mental disorders. An overview of the teams is given in

Appendix C.

Over the past eight years, the organization has implemented several eHealth-enabled changes. The aim of the organization for establishing eHealth programs is to create

sustainable change, improve and renew healthcare by the use of smart technologies. The organization created four main goals: to improve the quality, accessibility, affordability of healthcare and to provide more possibilities for empowerment and self-management of the client (BigPicture, iLentis). The implementation of e-therapy is one of the most recent eHealth innovations currently initiated by the organization. This application is an online treatment method, which offers a platform for both clients and their professionals. Clients can work on their problems and personal goals together with their professional, by using this online platform. E-therapy is always combined with face-to-face sessions, and together they form the so-called ‘blended treatments’. In 2012, the organization started implementing e- therapy in pilot teams. This implementation led to a great deal of change for some of the professionals. The professionals needed to interact with the clients by making use of this online platform. Prior to this change, the professionals were used to interact with clients only through face-to-face sessions, by making use of e-mail and phone calls. Since the introduction of e-therapy, clients can play a more active role in their treatment process and have more responsibilities, which is in accordance with the goals set in the organization concerning eHealth. This particular implementation is selected because the organization faces a serious lack of use and support from the professionals. Due to the scope of this research, the clients are only researched by questioning the professionals about the interaction with the clients by making use of e-therapy, not researching the clients themselves.

Due to the two sub-questions, a distinction was made in how data related to each part of this question was collected. As such, research part I and II are described separately in the following subsections.

3.2 Research part I: Survey

Research phase I intended to answer the following sub-question: “Which user-groups in terms of adoption behaviors emerge after the implementation of e-therapy and how can these be explained?”

The first research part encompassed a survey to identify different user-groups in the

organization. To analyze the current situation a combination of survey results, log data and an introductory interview was used. Data was collected on group-level. In the early stages of this study, an introductory meeting took place with the project manager where we discussed the different antecedents from the acceptance and resistance literature. The project manager identified the antecedents, which are most relevant for this particular e-therapy

implementation (see conceptual model, Figure 3). A survey was developed, to include these

antecedents, and was send to the professionals. The survey was carried out using the web-

enabled survey tool Qualtrics. From the selected teams, all professionals (total 348) are invited by mail (see Appendix A for invitation message, see Appendix B for survey questions). A total of 77 professionals completed the survey and 18 partial responses are received (response rate: 27.3%). Division C is the largest division in this case study (51,58%

of total participants). The results were compared with log data from the organization to guarantee construct validity (Yin, 2003) and to increase reliability (Van Aken et al., 2007). As can been seen in Appendix C, Team B_4: 43.8% (overall 9.1%), team C_12: 36.4% (overall 5.2%), Team C_1: 30.4% (overall 9.1%), and Team C_5: 30.0% (overall 7.8%) are the teams with the highest response rate (related to team size). These results are interesting, because the teams C_1 (2.0%) and C_5 (1.9%) score very low in the central registration of online contacts related to total client contacts (see Appendix C for complete overview with scores of all participating teams). This input is important for the second part of this study where we try to gain a deeper understanding of the professionals’ feelings, thoughts and experiences with the e-therapy technology and the implementation of this technology.

3.2.1 Measurements

For the survey, multiple measures and scales are gathered from earlier studies. For the complete list of measurement items, see Appendix D.

Independent variables

Perceived usefulness. Davis (1989) presented six different items to measure perceived usefulness. In our study, perceived usefulness is measured by means of four items: (1) using Minddistrict in my job increased my productivity, (2) using Minddistrict improves my job performance, (3) using Minddistrict simplifies to do my job, and (4) I find Minddistrict useful in my job. Professionals were asked to use a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The scale was highly reliably (α =.90)

Perceived ease-of-use. Davis’ survey (1989) contains six items by means of which perceived ease-of-use can be measured. In this study, perceived ease-of-use was measured by means of three items, which are also measured on a 7-point Likert scale. The measurement items are: (1) learning to operate Minddistrict was easy for me, (2) I find Minddistrict flexible to interact with, and (3) I find Minddistrict easy to use. The Cronbach alpha was highly reliably (α =.87).

Self-efficacy. Compeau and Higgens (1995) presented ten items in which respondents indicate their level of confidence when using a software package under a variety of

conditions. For this study, only four items were selected to determine the confidence level

ranging from 1(not at all confident) to 5 (totally confident). The measurement items are: (1) I

am able to complete the job using Minddistrict even if there was no one around to tell me

what to do, (2) I am able to complete the job using Minddistrict if I only had the software manual for reference, (3) I am able to complete the job using Minddistrict if I could call someone for help if I got stuck, and (4) I am able to complete the job using Minddistrict if someone else may help me to get started. The Cronbach alpha for these items together is α=.81. The factor loadings of these factors are .71, .77, .81 and .73. However, the factors are loaded onto the construct of perceived ease-of-use. Considering the content of these questions the items of perceived ease-of-use and self-efficacy are related. The items of self-efficacy are measured on a different Likert scale and therefore removed for further analysis.

Facilitating conditions Thompson, Higgens and Howell (1991) presented three items to measure the extent to which an individual believes that the organizational and technical infrastructure exist to support the use of Minddistrict. In this study two items were used, which are measured on a 7-point Likert scale. The measurement items are: (1) Specialized instruction was available, (2) A specific person (or group) was available for assistance with system difficulties. In this study, three extra items were added: (3) I had the opportunity to follow a training before using Minddistrict, (4) professionals are involved in the process of development, planning and implementation of Minddistrict, and (5) I am involved with the development, planning and implementation of Minddistrict. The Cronbach alpha for these items together is α=.66. The Cronbach alfa is not sufficient to use for further analysis.

Considering the content of these questions, item 5 was deleted because it loaded to more than one factor. Items 1 & 2 and items 3 & 4 were both loaded on different factors. The Cronbach alpha for items 1 & 2 seemed not sufficient for further analysis. The explanation for this may be that items 1 & 2 are questions about the feelings of the professional if they are involved during the project. Moreover, the questions are focused on multiple points in time

(development, planning and implementation), which makes these questions too broad. Items 3

& 4 are questions about facts (i.e. the possibility to follow a training). Items 1, 2 and 5 were removed from analysis, which resulted in a higher Cronbach alpha (α=.76).

Compatibility in norms and values is the degree to which the implementation of Minddistrict is felt to be consistent (or conflicting) with one’s norms and values (Kappos &

Rivard, 2008). Four items were created and measured on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree).. The measurement items are: (1) I support eHealth innovations in healthcare, (2) Using online treatment applications contribute to the quality of professional healthcare, (3) Using the online treatment applications is beneficial for the client, and (4) I am proud that my organization uses an online treatment application. The Cronbach alpha was highly reliably (α =.87).

Subjective norm is the users’ perception that most people who are important to them think they should or should not perform the behavior in question (Venkatesh et al., 2003).

Venkatesh et al. (2003) presented two items. In this study only one of them is used: (1)

People who are important to me think that I should use Minddistrict. For this study, three extra items were developed: (2) My clients think that I should use Minddistrict, (3) My colleagues think that I should use Minddistrict, and (4) My manager think that I should use Minddistrict. The four items are measured on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). After the factor analysis item 1 has been deleted because of an extraction value below .500. Extraction values above .500 were desired as then at least 50%

of the variance in the item is explained by the latent factor. The Cronbach alpha for these items together is α=.76.

Dependent variables

Use- / Non-use behaviors. For measuring use-/ non-use behaviors two items were developed:

(1) I consider myself an active user of Minddistrict, which is measured on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree) and (2) At the moment I use Minddistrict…, which is measured a 7-point Likert scale ranging from 1 (not at all) to 7 (multiple times a day). The Cronbach alpha for these items together is α=.85.

Support-Resistance behaviors. The questions investigating resistance were formulated in a reversed way, to guarantee neutrality and avoid pushing professionals in a specific direction. Therefore, the data on resistance needed to be recoded in order to ensure that all items are headed in the same direction. Two items were developed to investigate support behaviors: (1) I actively contribute to a successful implementation of Minddistrict and (2) I motivate my colleagues and clients to use Minddistrict. And two items to investigate resistance behaviors: (1) I am against the use of Minddistrict and (2) I express my objections agianst the use of Minddistrict. The four items were measured on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). After the factor analysis we deleted the item I express my objections to the use of Minddistrict because of an extraction value below .500. The Cronbach alpha for these three items together is α=.80.

Control variable

In this study the dependent variables are (1) use – non-use behaviors and (2) support-

resistance behaviors. There turned out to be substantial differences regarding the level of use (high use vs. low use) and level of support (enthusiastic support vs. active resistance) between the professionals and teams at the case organization.

The integration of an e-therapy technology in the daily work routines is not always straightforward and may create unexpected change when users interact with it (Farre &

Cummins, 2016). As work routines change, users have to learn new ways of working, which can also result in resistance to use to e-therapy application. According to Kanfer and

Ackerman (2004) it is more difficult for older people to learn new working methods. Besides,

the older people are more reluctant to adopt new technologies (Kanfer & Ackerman, 2004).

Age was, therefore, added as control variable in this research.

3.2.2 Data analysis

First, after data reduction, the descriptive statistics were calculated in order to represent the means, standard deviations and Pearson’s correlations of all items. A principal factor analysis (PCA) determined if the questions from the variables to measure use- / non-use behaviors (i.e.

perceived usefulness, perceived ease-of-use, and facilitating conditions) empirically reflect the constructs intended to measure. The closed factor analysis is based on the predefined number of factors, instead of an open factor analysis based on Eigenvalues. Items of the variables that load onto multiple factors or load to the wrong factor were removed. For the results of the factor analysis, see Appendix E. After that, a reliability analysis was conducted with the remaining items. Subsequently, a correlation analysis was used to calculate the relation between the independent, dependent and the control variable. Afterwards, a

regression analysis determined the effect of perceived usefulness, perceived ease-of-use and facilitating conditions on acceptance (use) behaviors and facilitating conditions, compatibility in norms and values and subjective norm on support behaviors.

3.3 Research part II: Interviews

Research phase II intended to answer the following sub-question: “Which change interventions are valued by these different user-groups and why?”

Semi-structured interviews with professionals who were categorized in the different user- groups of the behavioral oriented framework were conducted in order to gain a deeper understanding of the professionals’ feelings, thoughts and experiences with the e-therapy technology and the implementation. Moreover, we focused on group-level technology adoption and analyzed if user-groups perceptions towards the effectiveness of change interventions differed.

In total, twelve interviews were conduced and each interview took approximately 40- 60 minutes. Interviewees represented the major stakeholders: professionals, project manager, and former director of the eHealth program. The professionals were selected based on the type of user-group in order to gain sufficient information form the different user-groups. We were mainly interested in ‘how’ and ‘why’ different user-groups adopt (or not) the e-therapy technology. Further, professionals were asked which actions and interventions were taken by the organization and how they experienced the implementation project. Semi-structured interviews were therefore chosen so that participants had the opportunity to tell their own story, and thus to collect data about the underlying motivations, beliefs and attitudes

(Blumberg, Cooper, & Schindler, 2010). An interview protocol was used in order to enhance

the controllability and construct reliability (Corbin & Strauss, 1990; Van Aken et al., 2012).

The interview protocol (Appendix F) is based on survey results, theory, and information gained from the case organization.

3.3.1 Data analysis

All interviews were recorded and transcribed with ExpressScribe. The interview transcripts were coded by using deductive and inductive coding strategies (Miles & Huberman, 1994).

Based on existing literature, deductive codes were created. Subsequently, inductive codes were created based on issues raised by the professionals. Thereafter, data was compared and patterns were identified. For this study, the results were analyzed separately for the different user-groups. Within the context of the specific case study, the level of analysis is dual. First, we measure users’ reaction to the implementation of the e-therapy technology (individual level) in order to categorize the professionals in one of the four user-groups of the behavior- oriented framework. Second, the different user-groups (group level) are analyzed by focusing on the ‘how’ and ‘why’ professionals adopt (or not) the e-therapy technology. This study is built on three data sources: survey results, semi-structured interviews and log data from the organization. By triangulating multiple data sources, a stronger foundation is created for the results of this research (Eisenhardt, 1989; Yin, 2003).

          4. RESULTS

Following the two sub-questions that guided our research in order to answer the research question, this section is divided into two parts. In the first part we will answer the first sub- question which user-groups in terms of adoption behaviors emerge after the implementation of e-therapy and how can these be explained? To answer this question, we analyzed the survey results and categorize the professionals into the behavior-oriented framework of Van Offenbeek et al. (2013). As mentioned before, in this first part the focus was on the

individual’s acceptance and resistance behaviors to illustrate how the professionals are divided over the framework. In the second part, we addressed the second sub-question which change interventions are valued by these different user-groups and why? Here, we use the interview data to gain a deeper understanding of the professionals’ feelings, thoughts and experiences with the e-therapy technology and the implementation of this technology. The results of the second part are presented at user-group level.

4.1 Results part 1

To begin, the amount and patterns of the missing data is analyzed. The overall pattern is that

the last questions of the survey have the highest amount of missing data. For a complete case

analysis, 13 cases (13.83%) are excluded from the database. The items were excluded when it

was impossible to categorize the professional in one of the four user-groups of the behavior-

oriented framework or to use the data for further analysis. After data reduction, the

descriptive statistics were calculated in order to represent the means, standard deviations and Pearson’s correlations of all items. After that, the hypotheses were tested using the regression analysis, starting with the main effect variables followed by adding the control variable. The last step of results part I was to categorize the professionals in the behavior-oriented

framework.

4.1.1 Correlation analysis

For the main variables the means, standard deviations and Pearson’s correlations were calculated. For this purpose, the relevant items of each construct were combined and divided by the number of items in order to create a sum variable. In addition, data on age is rescaled to nominal level. Table 6 and 7 present the final results (see Appendix G). The results of Table 6 show relatively high levels of perceived usefulness and facilitating conditions. The average level of perceived usefulness was 5.98 (sd=1.72), and for facilitating condition this was 5.94 (sd = 1.18) on a 7-point Likert scale. This indicates that most professionals believed that using the e-therapy technology would enhance their job performance. Also, the

professionals reported an average level of 5.50 (sd=1.13) for perceived ease-of-use, which indicates that they somewhat agree on the fact that using the e-therapy technology is free of effort. Furthermore, the results indicate, as expected, that perceived usefulness (r=0.60, p<0.01), and perceived ease-of-use (r=0.45, p<0.01) have a significant positive relationship with use vs. non-use behaviors. This means that an increase or decrease in one of the independent variables will lead to an increase or decrease in use vs. non-use behaviors.

However, not every independent variable seems to have a significant correlation with other independent variables. The results show a significant relationship between perceived ease-of- use and perceived usefulness (r=0.53, p<0.01), but reveal no significant relationship between facilitating conditions and perceived usefulness and between facilitating conditions and perceived ease-of-use. Since the independent variables do not correlate with each other, it is not likely that the items represent one and the same construct. The control variable age (r=- .39, p<0.01) negatively correlates with perceived ease-of-use. This result indicates that the level of perceived ease-of-use will decrease when professionals become older. Therefore, learning how to operate the e-therapy technology is, on average, more difficult for older professionals.

The results of Table 7 show a relatively high level for facilitating conditions. The

average level was 5.90 (sd = 1.25) on a 7-point Likert scale. This indicates that most of the

professionals believed that the organizational and technical infrastructure exists to support use

of the system. Also, professionals reported an average level of 4.26 (sd = 1.22) for subjective

norm. This indicates that most professionals neither agree nor disagree with the fact that most

people who are important to them, e.g., co-workers or clients, think they should use the e-

therapy technology. Further, the results indicate, as expected, that compatibility in norms and values (r=0.71, p<0.01), and subjective norm (r=0.33, p<0.01) have a significant positive relationship with the variable support vs. resistance. Again, this means that an increase or decrease in one of the independent variables will lead to an increase or decrease in support- resistance behaviors. Despite the high level for facilitating conditions, there is no significant relationship with the support vs. resistance variable. The results show a significant

relationship between subjective norm and compatibility in norms and values (r=0.31,

p<0.01). This result indicates that an increase in subjective norm, which entails the perception of others about using the e-therapy technology or not, is significantly related with an increase in the degree to which the implementation is felt to be consistent with one’s norms and values.

4.1.2 Regression analysis

The different hypotheses are tested by a regression analysis. First, the direct relationship

between perceived usefulness, perceived ease-of-use, facilitating conditions and use vs. non-

use are tested. This test is repeated for the direct relationship between compatibility in norms

and values, subjective norm, facilitating conditions and support-resistance behaviors. After

that, the control variable age is added in both analyses to calculate this direct effect. Before

testing, the data was checked for multi-collinearity and outliners. The results indicate no

multi-collinearity problems (Variance Inflation Factors are below 3). When the values of the

VIF are below 3 there is no need for the elimination of one or more independent variables

from the analysis (O’Brien, 2007). The check for Mahalanobis distance (outliners) resulted in

the identification of one outlier, which is removed for further analysis. The final results of the

regression analysis are presented in Tables 8 & 9.

Table 8. Results regression analysis for use vs. non-use behaviors (n=80)

Model 1 Model 2

B SE B SE

Intercept 5.1

Control variable

Age -.03 (.02) -.01 (.02)

Predictor variable

Perceived usefulness .48

(.12)

Perceived ease-of-use .23 (.17)

Facilitating conditions .08 (.13)

R

.04 .40

Corrected R

.04 .35

F-value 2.85 10.85

Dependent variable: Use vs. non-use behaviors

* Correlation is significant for p<0.001 (2-tailed)

Hypothesis 1: Perceived usefulness, perceived ease-of-use, and self-efficacy are positively related to use behaviors.

Table 8 reports the results of the regression analysis for acceptance vs. non-acceptance behaviors. Model 2 includes the control variable and the independent variables. Further, Model 2 shows that the control variable in combination with the independent variables can account for 40% of the variation in use behaviors. However, the R

of model 2 is 0.40, and the F-statistic (10.85) is not significant. The results in Table 8 only illustrate a positive significant relationship for perceived usefulness (β=0.48, p<0.001) with use behaviors. This means that an increase in perceived usefulness will lead to an increase in use behaviors. As mentioned in subsection 3.2.1, the variable self-efficacy was removed for further analysis. Therefore, hypothesis 1 can be partly accepted.

Hypothesis 2: Compatibility in norms and values and subjective norm are positively related to support behaviors.

Table 9 reports the results of the regression analysis for support-resistance behaviors. Model 2

includes the control variable and the independent variables. Further, model 2 shows that the

control variable in combination with the independent variables can account for 57% of the

variation in support behaviors. However, the R

of model 2 is 0.53 and the F-statistic (22.35) is not significant. The results in Table 9 only illustrate a positive significant relationship for compatibility in norms and values (β=0.88, p<0.001). This means that an increase in compatibility in norms and values will lead to an increase in support behaviors. Therefore, hypothesis 2 can be partly accepted.

Hypothesis 3: Facilitating conditions are positively related to both use and support behaviors.

The results in Table 8 and 9 indicate no empirical support for a positive effect of facilitating conditions on use or support behaviors. In other words, the results shows that facilitating conditions does not contribute to use or support behaviors in this case. Therefore, hypothesis 3 can be rejected.

Table 9. Results regression analysis for support vs. resistance behaviors (n=80)

Model 1 Model 2

B SE B SE

Intercept 6.1

Control variable

Age -.02 (.01) -.02 (.01)

Predictor variable

Facilitating conditions .14 (.08

Compatibility in norms and values .88* (.11)

Subjective norm .09 (.09)

R

.04 .57

Corrected R

.04 .53

F-value 2.62 22.35

Dependent variable: Support vs. resistance behaviors Correlation is significant for p<0.001 (2-tailed)

4.1.3 Adoption reactions of professionals

In this subsection, we analyzed and categorized the adoption reactions of the professionals

towards the implementation of the e-therapy technology. In total, 80 reactions are analyzed

and categorized in Figure 4. The differences in their positioning are based on the degree of

acceptance and resistance behaviors resulting from the survey results. The numbers of the respondents is consistent with Table 2 (Appendix C). The adoption behaviors are presented on division level in Figure 5 (Appendix H).

N=80

Figure 4. Adoption reactions of professionals  

Remarkably, there are no professionals mapped in category 2, resisting users. This result indicates that the professionals did not experience much pressure from to use of the e-therapy technology.

The largest group according to Figure 4 is group 1, supporting users. As illustrated, most supporting users showed moderate use on the acceptance dimension and are between neutral and constructive cooperation on the support dimension. Further, the results show that there is a large group of 31 professionals (39%) that are categorized in category 3 (resisting non-users) and category 4 (supporting non-users). A non-user is a professional who has no clients participating in the e-therapy technology. Professionals categorized in these categories affirmed they use the e-therapy technology rarely or do not use it at all. Further, Figure 4 shows us that no more than 6 out of 31 non-users are categorized on the passive-active resistance dimension.

Clearly, this result indicated low usage and adoption behaviors of the professionals,

and confirmed the business problem as mentioned in the method section. To summarize, the

results of Figure 4 show that professionals are distributed over three quadrants of the

behavior-oriented framework. Despite the large group of non-users, there are only a few

professionals who explicitly indicate to resist against using the e-therapy technology. The

findings indicate that perceived usefulness and compatibility in norms and values are

important variables to consider in understanding use and support behaviors respectively.

In research part II of this study we try to gain a deeper understanding of the professionals’ feelings, and experiences with the e-therapy technology and the

implementation of this technology. Moreover, we discussed different change interventions that can influence adoption behavior. At this point, we can conclude that the case organization provided an online platform for the professionals and clients; one, however, the professionals have on average made little use of until this point in time.

4.2 Results part II

In this section, the results from the interviews will be presented. Figure 6 shows how the selected professionals for the interviews are distributed over the behavior-oriented framework. In total, 10 interviews were conducted (LEN001-LEN010). Interviewees represented the major stakeholders: psychiatrists, psychologists, and nurses. Initial

interviewees were conducted with the former director of the eHealth program, and a middle manager. The results are presented separately for the different user-groups.

  Figure 6. Adoption reactions of participants for the interviews

First of all, it should be mentioned that there were substantial differences in how the

professionals experienced the implementation process. In general, most of the professionals

were positive about the development of e-therapy. One professional explained: “I think this

development is really good. It is good because we become more digital in this society. We

have to deal with it. Further, one of the strengths of Minddistrict is that the responsibility for

success of the treatment is partly delegated to the client” (LEN010). Nevertheless, the results

of research part I showed that not all professionals feel the need to make use of the e-therapy

technology. The survey results indicate that the average acceptance level is moderate (mean =

3.94). This means that the professionals use the e-therapy technology on average 2-3 times a week.

Below, the qualitative results are presented separately for each of the identified user- groups in the behavior-oriented framework. Quotes from the interviews were given to demonstrate the perceptions of the professionals. In Appendix I there is an overview of the coding scheme.

4.2.1 User group 1: supporting users

As mentioned in research part I, the supporting user-group is the largest group (61,25% of total group). The supporting user group experienced perceived usefulness of the e-therapy technology as high (mean 6.63). The professionals perceived the e-therapy as useful because they experienced the added value during treatment. One professional explained: “More engagement during treatment, more interaction. Blended treatment is nice also” (LEN005).

Additionally, the professionals use e-therapy to deal with no-show of clients. One

professional explained: “We always have to deal with no-show. This is indirect client time, free space in the agenda. When someone (client) sends you something, we have the possibility to score direct client time. That is where we are accountable for” (LEN007). Moreover, the supporting user-group mentioned that e-therapy empowers clients to participate actively in the treatment process. One professional explained: “It is also a part of awareness and diagnostics. It is not only about the doctor in the uniform who decides what treatment the client needs. The client is able to participate and Minddistrict is suitable for this. Minddistrict gives a lot of structure and support” (LEN005).

Although professionals in this group are already categorized as supporting users, there is still room for improvement since 35 professionals (71,43% of supporting users) use e- therapy on average only 2-3 times a week.

Interventions

All supporting users were already motivated to use e-therapy before implementation of the technology. One of the professionals stated: “At first, I thought it was fantastic that Lentis did something to offer online healthcare. I really needed it” (LEN005). The supporting users perceived training, organizational support, and peer support as effective interventions.

The supporting user-group mentioned that the training provided by the external party

was very well organized. Further, they mentioned that the use of external trainers was

important to introduce and learn how to interact with the system. The training sessions were

organized in classroom settings. In general, all professionals, who participated during training

sessions, were satisfied with this type of training. One professional explained: “When they

started with the implementation of Minddistrict our team was divided in two and we followed

the training. The Minddistrict guru organized the training. […] They really did a good job in the first round” (LEN010). Interestingly, not every professional had the opportunity to follow the training organized by the external party. A professional that recently joined the

organization: “I needed to follow an e-learning program. [...] It was not about the content. I can remember that the program was not very useful” (LEN003).

Besides training, the supporting user-group valued the support from the organization in the early phases of the project. More specifically, the project team appointed a group of supportive users as super user. Super users were responsible for giving updates about e- therapy, were available for questions, and educate new colleagues in the use of e-therapy. The super user understands the whole picture and therefore can help to see the potential value of e-therapy. Further, super users can support other professionals and help to get used to it. One professional explained: “That (super user) is really an added value. Someone with a great spirit and which is able to contribute ideas, someone who is familiar with the program”

(LEN003). Super users provided guidance and direction for the professionals. Therefore, it can provide a powerful ‘pull-factor’ on professionals to adopt e-therapy, especially in the post-implementation phase.

Another intervention that was valued was peer support. Peer support activities performed by co-workers may help a user effectively use a new system. Actually, the super users were performing this role. The super users can help other professionals by showing them the functionalities of e-therapy in practice. One professional explained: “Show what you have to offer by doing it. No only the technical part, but also show the content” (LEN003).

The supporting user group mentioned that they are willing to give their opinions and ideas about e-therapy and discuss this with others. However, the supportive user-group mentioned that the evaluation phase was skipped by the project team after the

implementation. It was missing, but really needed. One professional explained: “No, there was not anything. Sometimes, there was a person that was somewhere in the building available for questions. But that is already a year ago. I noticed nothing further” (LEN002).

Besides the evaluation phase, the supporting user group missed the possibility to discuss issues. The supporting user-group mentioned that they are willing to share their ideas and discuss the functionalities with other teams. One professional explained: “Are we ready for the second step? What about the content? What are we going to do with all of this? Who is using what? We are not there yet. We have to do it with the things that are already there”

(LEN005). Additionally, according to the interviewees, there was a lack of management

support in division C. Management of division C did not establish a sense of urgency for

using e-therapy, which does not motivate professionals to change their behavior. Further,

there was a lack of a clear strategy. Consequently, professionals experienced difficulties from