The Extended Equity Implementation Model: An Analysis of User Reactions

(1)

Student: Paul M.A. Werker

University Supervisor: Prof. Dr. Albert Boonstra

University Co-Supervisor: Dr. Cees Reezigt

Company Supervisor: (…)

(2)

(3)

The Extended Equity Implementation Model: An Analysis of User Reactions

Master thesis, MSc. Business Administration, Specialization: Change Management

University of Groningen, Faculty of Economics and Business

December, 2009

PAUL M.A. Werker Student number: 1477366

Supervisor: University Prof. Dr. Albert Boonstra

Co-Supervisor: University Dr. Cees Reezigt

Supervisor: Field of Study (…)

(4)

A

BSTRACT

(5)

T

ABLE OF

C

ONTENTS

INTRODUCTION ... 7

RESEARCH OBJECTIVE ... 8

RELEVANCE ... 8

MAIN RESEARCH QUESTION ... 9

2. THEORY ... 10

2.1THEORIES OF RESISTANCE ... 10

2.2TECHNOLOGY ACCEPTANCE MODEL ... 11

2.3UNIFIED THEORY OF ACCEPTANCE AND USE OF TECHNOLOGY ... 12

2.4EQUITY IMPLEMENTATION MODEL ... 13

3. THE EXTENDED EQUITY-IMPLEMENTATION MODEL ... 18

4. BACKGROUND ... 21

4.1THE HYDROCARBON RESOURCE VOLUMES MANAGEMENT SYSTEM ... 21

4.2STAKEHOLDERS ... 23

5. METHODS ... 26

5.1DATA COLLECTION ... 26

5.2DATA ANALYSIS ... 28

6. RESULTS ... 30

6.1FIRST LEVEL OF ANALYSIS ... 30

6.2SECOND LEVEL OF ANALYSIS ... 35

6.3THIRD LEVEL OF ANALYSIS ... 39

7. DISCUSSION ... 42

7.1FIRST LEVEL OF ANALYSIS ... 42

7.2ACROSS-LEVEL ANALYSIS ... 45

7.3PULLING IT ALL TOGETHER ... 46

8. CONCLUSION ... 48

(6)

10. RECOMMENDATIONS ... 51

10.1MANAGEMENT RECOMMENDATIONS ... 51

10.2RESEARCH RECOMMENDATIONS ... 51

BIBLIOGRAPHY... 53

APPENDIX A: INTERVIEW TRANSCRIPTS ... 56

(7)

I

NTRODUCTION

The successful implementation of information systems (ISs) poses a significant challenge to organizations (Joshi, 1991). While ISs offer the promise of increasing productivity (Lauer, Joshi & Browdy, 2000) and in that way improving an organization‟s competitiveness in the marketplace, such productivity increases are often blocked by users‟ unwillingness to accept and use available systems (Bowen, 1986; Young, 1984; cited by Davis, 1989). Indeed, “because of the persistence and importance of this problem, explaining user acceptance has been a long-standing issue in MIS (ed. management information system) research.” (Davis, 1989: p. 319) Considering that investing in ISs can be costly in both financial and human terms (Martinko, Henry & Zmud, 1996), having (in-depth) knowledge on why users react in the ways they do becomes all the more important. This paper will use an extension of the equity implementation model1 (EIM) to generate further understanding of user reactions to a new IS. For this purpose, a case study will be used: the implementation of a new management information system (MIS) at Alpha Inc.

As part of a drive for continuous process improvement, Alpha Inc. has developed a new management information system, the so-called Resource Management Information System (R-MIS), set for worldwide implementation. The current use of spreadsheets makes the results of the existing (standardized) workflow difficult to audit. Also, the transparency of the process is suboptimal. On the other hand, the R-MIS will reduce flexibility on the part of engineers and local management in terms of handling the resource management process. All the while, differences in local legal requirements toward governments, regulators (e.g. SEC) and partners also need to be taken into account.

In response to a global increase in controls and reporting regulations (e.g. the Sarbanes-Oxley Act adopted in 2002 in the United States) Alpha Inc. has had to tighten internal procedures and business controls. Next to better training around the requirements of external regulatory bodies (e.g. SEC), Alpha Inc. also increased its efforts to audit results before publishing. However, one concern remained, which was the dominant role of spreadsheets in the resource management process. This made version control (i.e. the ability to track changes to documents or electronic files as they are being made), transparency of results and the ability to audit difficult. Furthermore, the use of spreadsheets made it unwieldy to generate aggregated views, or analyze data from a global perspective.

In the period of 2005-2006 a feasibility and requirements analysis was executed on a central data gathering system for resource maturation data with management information system (MIS) functionality

(8)

and very tight security controls. In 2007 a first design was completed, which resulted in two prototypes that were evaluated in different operation areas during 2008. At present (March 2009) the first release of the final design is nearing completion and will be deployed globally during the third quarter of 2009. Because of local data export restrictions in some countries, the system is implemented on a distributed infrastructure which installation is in progress. At the end of 2009, Alpha Inc. plans on running the R-MIS system in parallel to the conventional method to determine whether the system is able to meet performance demands. Ease, repeatability and accuracy of the results of R-MIS are factors that will determine its global operational use from 2010 onwards.

Research objective

The objective of this research paper is to carry out an analysis of the implications of the implementation of the R-MIS, in order to gain insight into what users‟ reactions are in terms of acceptance and resistance. In more general terms, the analysis contained in this paper will generate further understanding of user reactions by identifying factors that influence user reactions. Managers and practitioners can use knowledge obtained on these factors to predict and respond to adverse user reactions to change. The importance of user acceptance and resistance in relation to information system implementation has been discussed extensively in the literature in a myriad of ways (see for example Venkatesh, Morris, Davis & Davis: 2003). Indeed, according to Venkatesh et al. (2003), “explaining user acceptance of new technology is often described as one of the most mature research areas in the contemporary information systems literature.” (p. 426) This paper is different in that it uses an improved version of the EIM, the so-called Extended EIM (E-EIM). Because the E-EIM considers a broad range of variables, it is able to generate a more complete understanding of the nature of user reactions (i.e. user acceptance) than is possible using other theories and models2.

Relevance

This research paper is relevant for four reasons. First, it offers Alpha Inc. insight into key users‟ perceptions with regard to the R-MIS in terms of acceptance and resistance: Alpha Inc. can use this knowledge to facilitate the implementation process of the R-MIS. Second, practitioners (i.e. individuals concerned with the design and implementation of information systems) may draw lessons from this research paper in that it contains an outline of how key users perceive and react to the implementation of a (system-wide) information system (IS). Third, Joshi (1991) proposed that the EIM may be used to explain user reactions. This paper contributes to the literature by refining the categories suggested by Joshi (1991) by applying these in the context of a new management information system of a global

2

(9)

company. Lastly, this research paper is relevant from a change management perspective in that it formulates an analysis of how certain agents (in this case key users) respond to a change in their organization.

Main Research Question

In light of the research objective mentioned above, my thesis will strive to answer the following main research question:

“What are users’ reactions to the implementation of the new resource management information system in terms of acceptance or resistance?”

(10)

2. T

HEORY

In order to answer this paper‟s main research question, I require a suitable theory to guide my analysis. I will now formulate three requirements that this theory –in my opinion- has to meet. Taken together, these requirements ensure a good fit with the case study used in this paper and will therefore result in a high-quality analysis. First, the theory should allow for different types of reactions on the part of users. Rather than (for example) assuming that users will resist any type of change, a broader range of possible reactions to the change (in this case an IS implementation) should be possible. This requirement follows from the fact that the implementation of the new IS (i.e. the R-MIS) is looked forward to by a number of stakeholders: this will be explained further on in this paper. For this reason, the theory should be able to examine the specifics of why (individual) users has a favorable opinion of the new IS. Second, the theory should consider a broad range of possible inputs and outcomes. In this way, the issue of possibly omitting important variables is solved. Third, the theory should take into account that direct interaction (and perhaps conflict) between different groups may or may not occur. In the case study used in this paper, some stakeholders interact directly with one another, while other stakeholders may operate independently (this will be explained further on in this paper). In light of these requirements, and the need for an appropriate theory, I will now discuss a number of theories available in the literature which are of particular interest.

2.1 Theories of resistance

MIS researchers have adopted a wide range of different approaches and theoretical perspectives to examine and explain user reactions to MIS implementation. (Joshi, 1991) One way in which the dynamics of implementation have been explored is by defining it as a political process. In doing so, the “sequence and direction of implementation can be explained in terms of the conflicting interests of different user groups.” (Joshi, 1991) These different user groups compete with one another for power, for the purpose of (for example) controlling information (Markus, 1983) and securing a greater share of computer resources (Keen, 1981; Robey & Markus, 1984).

(11)

that individuals or groups resist systems due to the interaction that occurs between characteristics of the people and the system, respectively. For example: systems that centralize control over data are resisted in organizations with decentralized authority in structures. Markus (1983) employs a political version of this so-called interaction theory, in which “resistance is explained as a product of the interaction of system design features with the intra-organizational distribution of power.” (p. 432) Using a case study, Markus (1983) shows that of the three theories, the political version of interaction theory has the most to offer in terms of accuracy of predictions derived from it. In addition, it also offers the greatest utility to its implementers in terms of additional facts and data that can be uncovered and explained, which may be useful in designing an implementation effort.

Lapointe & Rivard‟s (2005) article makes an interesting contribution to the extant theory on resistance. Building on earlier efforts, Lapointe & Rivard (2005) put forward a process model of resistance to IS implementation that sheds light on how resistance arises, develops, and concludes. Lapointe & Rivard (2005) posit that resistance behaviors will follow if threats are perceived between the object of resistance and initial conditions. Using three case studies, Lapointe & Rivard‟s (2005) analysis shows that resistance behaviors may increase in intensity over time: initially starting out as apathy or indifference, only to change into aggressive behaviors in later episodes. Interestingly, in addition to an increase in the severity of resistance behavior, a change in the level of perceived threats was observed. While perceived threats manifested themselves at the individual level at first, in later episodes the perceived threats characterized entire groups. By taking a multi-level approach, Lapointe & Rivard (2005) are able to show how resistance behaviors develop over time, and at different (i.e. individual vs. group) levels.

2.2 Technology acceptance model

(12)

FIGURE 1

The Technology Acceptance Model

Adapted from Boddy et al. (2005)

The TAM has been added to and modified in a number of ways since it was first introduced. Researchers have combined the TAM with a whole range of different variables; examples include gender (Gefen & Straub, 1997), experience (Venkatesh & Davis, 2000) and trust (Pavlou, 2003). Legris, Ingham and Colerrette (2003) provide a comprehensive overview of research in the literature involving the TAM. Interestingly, even after modifying (i.e. improving) the TAM, the model still explains only 40% of the variance in use of information systems (Legris et al., 2003). Another significant limitation of the TAM is that it does not take into account organizational dynamics (i.e. the manner in which people interact with one another). Legris et al. (2003) point out that research in the field of innovation and change management provides evidence of a relation between technological implementation and organizational dynamics.

2.3 Unified theory of acceptance and use of technology

An interesting theory in IS acceptance research is the Unified Theory of Acceptance and Use of Technology (UTAUT). This model represents an attempt by Venkatesh et al. (2003) to combine eight prominent models in IS acceptance research (such as the TAM), based on empirical and conceptual similarities across models. The UTAUT may be displayed graphically as follows:

Perceived usefulness

(13)

FIGURE 2

The Unified Theory of Acceptance and Use of Technology Model

After reviewing eight prominent models in the literature on user acceptance of information systems, Venkatesh et al. (2003) concluded that four constructs “will play a significant role as direct determinants of user acceptance and user behavior” (p. 447): „performance expectancy‟ refers to the degree to which an individual expects an information system to yield improvements in job performance; „effort expectancy‟ is related to the degree of ease of use associated with the system; „social influence‟ concerns the degree to which an individual believes „important others‟ believe he or she should use the new system; „facilitating conditions‟ involve the degree to which an individual perceives a technical and organizational infrastructure exists which supports use of the system. (Venkatesh et al., 2003) Key moderators influence these four constructs: gender, age, voluntariness of use, and experience. Interestingly, Venkatesh et al. (2003) found that the UTAUT outperforms the eight individual models: the model explains 69% of the variance in user intention to use an IS.

2.4 Equity implementation model

While early researchers who studied IT implementation have acknowledged resistance as a critical variable (Keen, 1981, cited by Lapointe & Rivard, 2005), most research has treated it as a black box (Lapointe & Rivard, 2005). Joshi (1991) has written one of the few articles which has opened the black box of user resistance by proposing a theoretical explanation of how and why resistance occurs. (Lapointe & Rivard, 2005) Using equity theory, Joshi (1991) formulates the EIM, which “provides an understanding of issues useful in determining acceptance of or resistance to a new system, technology,

Adapted from Venkatesh et al. (2003)

Performance Expectancy Social Influence Effort Expectancy Facilitating Conditions Use Behavior Behavioral Intention

Gender Age Experience

(14)

work practice, or other change in their work environment.” (Lauer et al., p. 92) Equity theory poses that individuals are constantly concerned with fairness in all exchange relationships, “constantly comparing themselves with others in their reference group to assess whether the relative gains are the same”. (Joshi, 1991)

(15)

FIGURE 3 Conceptual Model3

In this figure, „user reactions‟ refers to the way in which changes in inputs and outcomes add up for each user group (taking into account all three levels of the framework). If users experience the distress of inequity or loss of equity, they are likely to resist the change. Alternatively, if users perceive an increase in their equity they are likely to accept: in this case the R-MIS implementation. In that sense, acceptance and resistance may be viewed as opposites on a single continuum. Depending on how users perceive their net equity to have changed, they will be either on the resistance, or on the acceptance end of the continuum. Alternatively, some users may on average be indifferent towards the implementation: perceived increases in equity might be balanced by a sense of perceived inequity. The continuum of acceptance vs. resistance may be displayed as follows:

FIGURE 4

Continuum of Resistance and Acceptance

(16)

TABLE 1

Possible Changes in Outcomes and Inputs on Account of Implementation

Increase in Outcomes Increase in Inputs

 More pleasant work environment  Less tension, more job satisfaction  More opportunities for advancement  Better service to customers

 Recognition, better visibility

 Salary increase, grade increase, or higher-level title  Increase in power and influence

 Learning a marketable skill  Reduced dependence on others  Usefulness of the system

 More work in entering data  More tension

 Bringing higher level skills to the job  Effort in learning a new system  Assignment of additional tasks

 More effort in performing tasks in view of increased monitoring

 Need to spend more time

 Fear of unknown (e.g. failure) and the resulting anxiety

Decrease in Outcomes Decrease in Inputs

 Reduced job satisfaction  Reduced power

 Reduced bargaining power relative to the employer of others

 Threat of loss of employment  Loss of value of marketable skills  Reduced importance, control  Increased monitoring

 Reduced scope for advancement  More role conflict and ambiguity

 Potential failure in learning and adopting the new system

 Ease of usage  Less effort

 Reduced search for solutions or information  Reduced manual effort

 Reduced cognitive effort  Less rework due to fewer errors

Adapted from Joshi (1991)

The following table outlines how each of the respective articles written by the authors discussed relates to the three requirements I have formulated earlier:

3

(17)

TABLE 2 Theory Requirements

Author Requirement I Requirement II Requirement III

Markus (1983)

Focuses on resistance, which suggests a focus on negative reactions.

Power (an outcome) User groups will interact with at least one other group: user groups compete with one another in an attempt to increase their power, in order to control information.



Lapointe & Rivard (2005)

Focuses on resistance, which suggests a focus on negative reactions.

Group dynamics (an input) among others

All stakeholders interact with one another (in varying degrees).



Davis (1989, 1993)

Model allows for both positive and negative reactions on the part of the user, depending on what the user‟s perception is of the new IS.

Ease of use (reduction of inputs) and usefulness (an outcome)

Considers the perspective of the individual user and his or her perception of the IS only. In doing so, it does not consider any kind of interaction between stakeholders.





Venkatesh et al. (2003)

Broad range of inputs and moderators considered (see Figure 2)

Considers the perspective of the individual user and his or her perception of the IS only. In doing so, it does not consider any kind of interaction between stakeholders.





Joshi (1991)

Broad range of inputs and outcomes considered (see Table 1).

Describes a process of comparison in which direct interaction (and perhaps conflict) between different (groups of) users may or may not occur.



The three requirements mentioned in the table above are as follows: firstly, the theory should allow for different types of reactions on the part of users; secondly, the theory should consider a broad range of possible inputs and outcomes; thirdly, the theory should take into account that direct interaction (and perhaps conflict) between different groups may or may not occur.

(18)

3. T

HE

E

XTENDED

E

QUITY

-I

MPLEMENTATION

M

ODEL

This section will outline the E-EIM that I will use for my analysis. As mentioned earlier, Joshi (1991) has made some suggestions of possible changes in inputs and outcomes, respectively (see Table 1). In my opinion, it is difficult to conduct research using such a list of possible changes. Therefore, in an effort to improve the EIM, I will formalize this list of possible changes by formulating a number of variables which may be used instead. The benefit of formulating these variables is that they provide a more solid foundation from which to conduct research. In addition, it offers the possibility of testing variables for statistical significance. These variables are outlined, and defined, in the following tables:

TABLE 3A

Input and Outcome Variables

Inputs Outcomes

(19)

TABLE 3B

Input and Outcome Variables Definitions

System use complexity The degree to which a user finds it easy to use and interact with a particular IS.

Change effort The amount of effort involved in transitioning from previous work methods, to the work methods associated with the new IS.

Time The amount of time involved in carrying out particular work tasks.

Job performance The degree to which a user feels that using an IS will enhance his or her job performance.

Information value The degree to which the value of information extracted through use of the IS is raised or decreased.

Impact on work The degree to which an IS fits within the daily work, and the resulting impact on work.

Autonomy The degree to which a user has the perception of autonomy (i.e. independence) in conducting his or her work tasks.

Power & influence The degree to which an IS changes a user‟s power and influence over others (e.g. being able to

enforce adherence to data input standards).

Taken together, Figure 3 and Table 3 constitute the E-EIM. The E-EIM is similar to the EIM in that it also represents a three-level framework of analysis, as explained in section 2.4.

These variables have been derived by closely studying the possible changes in outcomes and inputs outlined in table 1, and formulating a set of variables that may „capture‟ these changes. In a further refinement of the model, I have added the variable „change effort‟, which further improves the model‟s ability to analyze change initiatives. All in all, these variables constitute an attempt to identify a set of generic variables that may also be applied in other case studies. Indeed, it is my opinion that –with slight modifications- this model may also be used in other contexts, such as case studies involving other types of organizational change.

In order to make tangible the possible sense of equity or inequity that may exist among users vis-à-vis other users/user groups or Alpha Inc. as a whole, the variable „benefit‟ is used. While not an input or outcome variable in itself, this variable will serve to analyse if users feel that benefits (or the opposite, disadvantages) are distributed equally across all three levels of analysis. If users feel the benefits and disadvantages are distributed equally, he or she will have a sense of equity and is therefore likely to accept the change (in this case the introduction of R-MIS). If the opposite holds true, users will have a sense of inequity, and are likely to have a more negative attitude towards (i.e. resist) the system.

(20)

reaction. The final individual user reaction is determined by taking all variables into account, across all three levels of analysis.

Using the E-EIM as a frame of reference, I will now formulate six sub research questions that will serve to investigate this paper‟s main research question in a more detailed way:

I. What changes do user groups perceive in terms of a decrease in input in relation to the implementation of the R-MIS?

II. What changes do user groups perceive in terms of an increase in outcome in relation to the implementation of the R-MIS?

III. What changes do user groups perceive in terms of an increase in input in relation to the implementation of the R-MIS?

IV. What changes do user groups perceive in terms of a decrease in outcome in relation to the implementation of the R-MIS?

V. What changes do user groups perceive to their relative outcomes, in comparison to Alpha Inc.?

VI. What changes do user groups perceive to their relative outcomes, in comparison to other user groups?

(21)

4. B

ACKGROUND

This section will provide some background information on the R-MIS project to further an understanding of the context of the project. First, the background of the R-MIS will be discussed, followed by an exploration of the stakeholders involved in the project.

4.1 The Hydrocarbon Resource Volumes Management System

As mentioned earlier in the introduction, the R-MIS has been developed by Alpha Inc. as part of a drive for continuous process improvement. The current use of spreadsheets for resource management has been deemed undesirable for a number of reasons. First, it is inefficient: gathering all the relevant resource information from each individual spreadsheet, and aggregating these figures across the group is very time-consuming. Second, the use of spreadsheets is fault-prone: variations exist between spreadsheets, which can result in data being corrupted when it is being transferred from one spreadsheet to another incorrectly. Third, the use of spreadsheets makes the resource management process difficult to audit. The R-MIS will serve to remedy these shortcomings. The system itself is standardized, and will be deployed in all Alpha Inc. locations across the globe. Because of its standardized format, the R-MIS is able to aggregate resource data with a single click of a button as soon as all the resource information is keyed in. In effect, this solves the issues related to efficiency, and the risk of making mistakes when aggregating the data from numerous spreadsheets. Also, the R-MIS is a major step forward in terms of auditability: the system automatically creates an audit log of who has filled in what information at what time, while also demanding that resource information keyed in is approved at management level at certain intervals. The analysis contained in this paper is based on user experiences with version 1.0 of R-MIS. Future versions of R-MIS will offer users more benefits in terms of added functionality. Examples include the ability to carry out analyses of the data contained in R-MIS, and generating user-defined reports.

The system itself operates inside an internet browser, and is run from a centralized server. Certain exceptions exist here, with a number of regions preferring to store their resource data locally (due to sensitivities related to the import and export of resource data) before synchronizing the data with the central server. Users can log in from any location using their Alpha Inc. log-in credentials, and start using the R-MIS.

(22)

FIGURE 5 R-MIS Hierarchy

Data is entered at the primary (ground) level by engineers and is subsequently aggregated through the different hierarchical levels (indicated by the different icons). Before data may be aggregated to higher levels in the hierarchy, the figures will need to be approved by the appropriate managerial level.

(23)

In summary, an in-depth understanding of user reactions is warranted, as user reactions are expected to vary. The reason for this is that (currently) the advantages and disadvantages of using R-MIS are unevenly distributed. Because users are interdependent, it is critical that they use R-MIS in the correct manner. In knowing how users will react, and which variables influence these reactions, management can make adjustments to the system itself and the process of introduction to remove or neutralize any objections users might have. In this way, the transition to R-MIS can take place in the smoothest way possible.

4.2 Stakeholders

(24)

TABLE 4 R-MIS Stakeholders

Stakeholder Description Direct users

Business planners Use the R-MIS to generate regional and global business plans. In this way, cost/benefit and timing aspects of projects may be analyzed.

Resource maturation staff Owners of the resource maturation (RM) process

4

. The RHCM staff will use the resource data contained in the R-MIS to manage the RM process.

Reporters Uses the data stored in the R-MIS to generate reports, including ARF reports.

Engineers Enter resource data using the R-MIS (in effect replacing the current use of spreadsheets).

Other

Core team Responsible for managing the R-MIS project and getting the system fully operational across the Alpha Inc. Group.

Data loading activities team During the final stages of the development phase of the R-MIS project, the data loading team will load resource data from the 2008 Annual Report of Petroleum Resource (ARF5) into the system. Decision review board Composed of members from different work areas: responsible for making decisions on

investment and resource allocation for the R-MIS.

Deployment community Global community assisting in the world-wide deployment of the R-MIS through a range of (short-term) activities.

Development team Dedicated to designing the R-MIS in such a way that it can meet its business objectives.

Resource figures reporting staff

Use the R-MIS for the purpose of annual resource reporting, both internal (which influences management decisions) and external (to investors in accordance with SEC and government regulation).

IT community Set the restrictions of the applications that are run within the R-MIS, and provide a range of necessary services and resources (e.g. a central server storing the resource data).

Engineering leadership team Oversee the RM process: any change to this process resulting from the R-MIS has implications for the operations they have to oversee.

Support community Provide a range of support services for the R-MIS (e.g. a help-desk).

Referring back to this paper‟s main research question (what are users‟ reactions to the implementation of the new resource management system in terms of acceptance or resistance?), I will limit my analysis to

4

The RMHCM process concerns the management of Alpha Inc. Group‟s global portfolio of (potential) resource volumes into commercially feasible development projects. The aim is to realize as high a return as possible.

5

(25)

the following four stakeholders: RM staff, Business Planners, Reporters, and Engineers. The reason for limiting the analysis to these stakeholders is that they constitute the direct users of the R-MIS. With lack of an appropriate definition in the literature, I will define direct users as follows: users for whom a particular IS is a critical enabler and/or facilitator of his or her other work flows. The stakeholder groups who do not fit this definition have been classified as 'other' in Table 4.

(26)

5. M

ETHODS

This section of the paper will focus on the design of the empirical part of the research. First, the manner of data collection will be discussed. Second, the process involved in analyzing this data will be explained.

5.1 Data collection

The data used in the empirical part of this paper has been obtained from a number of sources. First, available documentation at Alpha Inc. has been consulted to further develop an understanding of the R-MIS project. Second, a stakeholder section was formulated, describing the main parties who are involved in and impacted by the deployment of this new system. In this paper, Freeman‟s (1984) definition of stakeholders is used in the context of an inter-organizational system (IOS) (in this case the R-MIS): “a stakeholder is any individual, group, organization, or institution who can affect or is affected by the IOS system under study.” (Boonstra & De Vries, 2008) Third, a combination of surveys and interviews has been used to measure user reactions to the R-MIS.

One reason for using a survey includes that a respondent feels anonymous (Cooper & Schindler, 2003). I will assume that respondents feel more comfortable giving truthful answers when they can do so anonymously. Another reason for using the survey method of data collection is the fact that it is flexible in its deployment, allowing for rapid data collection (Cooper & Schindler, 2003). In this research, surveys results were collected in three ways: via e-mail, over the phone, and via a survey that was made available online. Due to the limited time frame associated with this research, it is convenient to be able to gather a lot of data in a short period of time. Regarding the interviews, these are used to gather information that is high in level of depth and detail (Cooper & Schindler, 2003), something that is difficult to achieve using solely surveys as a method of data collection.

The reason for using a combination of surveys and interviews is that, according to Kaplan & Duchon (1988), “collecting different kinds of data by different methods (…) provides a wider range of coverage that may result in a fuller picture of the unit under study than would have been achieved otherwise.” (p. 575). In addition, using multiple methods of data collection increases the robustness of results.

(27)

Jick (1983) argues that “blending and integrating a variety of data and methods, as triangulation demands, may be seen on a continuum that ranges from simple to complex designs.” (p. 603) Ranging from simple design to complex design, the research method employed in this paper tends to the complex design side of the spectrum. In a complex design, “elements of the context are illuminated.” (p. 603) Through the use of triangulation, I will be able to enrich my understanding by allow new and deeper dimensions to emerge in my research (Jick, 1983).

Using the variables contained in the E-EIM, I have formulated nineteen Likert-scale questions, and six interview questions. The results of the survey and interviews were obtained by means of face-to-face interviews (where possible), and also a number of telephone interviews. During these interviews the respondents first filled in the surveys, whereafter I posed a number of open questions. Interviews were taped using a voice recorder, and were transcribed at a later time. In all, eleven interviews were conducted in this manner. In order to obtain more survey results, the survey was also made available online: in this way an additional ten filled-in surveys were attained.

Taken together, eleven interviews were conducted, and twenty-one filled-in surveys were obtained. The population of users that had access to and sufficient experience with R-MIS totaled 65. As such, the response rate was approximately 32% for the survey, and 17% for the interview.

A draft of the survey and interview has been tested on employees closely involved in the deployment of the R-MIS project to ensure all questions were clear and accurate. Because it was difficult to make tangible the difference between power and influence, I decided to combine these two variables in a single set of questions.

A Likert scale is an example of a so-called summated scale, which consists of “statements that express either a favorable or unfavorable attitude toward the object of interest.” (Cooper & Schindler, 2003: p. 253) In my survey the Likert scale has the following range: strongly disagree, somewhat disagree, somewhat agree, and strongly agree. Respondents could also opt for „no opinion‟. The questions were worded positively, which means that higher scores are better. For example: a high score on system use complexity indicated that users find R-MIS easy to use.

(28)

The sample included 21 individuals from four different stakeholder groups: Resource Maturation Staff, Business Planners, Reporters, and Engineers. In order to allow respondents to speak freely, all interviews and surveys were conducted on the basis of strict anonymity.

5.2 Data analysis

During the course of gathering data for this research, and also while working on the R-MIS project, I got an indication of possible significant differences in user reactions on the part of Engineers vis-à-vis other user roles contained in R-MIS. For this reason, my analysis will focus on identifying possible noteworthy or important differences between these two groups: engineers, and non-engineers. For the survey, the distribution of users between these two groups would be as follows: five engineers, and sixteen non- engineers. For the interviews, the distribution is as follows: two engineers, and nine non-engineers.

Due to the limited sample size, it was not possible to test my findings for statistical significance using statistical techniques. Considering that my survey sample included 21 individuals, which I subdivided into two user groups, I was unable to reliably use statistical techniques such as a Student-t Test or a chi square test. While there were a few instances where it was possible to apply (for example) a Chi Square test, I was unable to do so for the whole dataset. Instead, I will construct tables outlining means and standard deviations for the variables contained in my survey for each of the two user groups separately. In order to arrive at a single score for each of the variables contained in the E-EIM, individual questions were taken together to calculate the mean, resulting in a so-called subscale. This was done for the following variables: „system use complexity‟, „change effort‟, „time‟, „job performance‟, „information value‟, „impact on work‟, „autonomy‟, „power & influence‟. Furthermore, to facilitate an understanding of the values of the standard deviation, these will be classified as follows: low (ranging from 0 to 0.5), medium (ranging from 0.5 to 1.0) or high (1.0 and greater). While these numbers are, in a way, somewhat arbitrary, they do serve to provide an instant understanding of the degree of variation for each of the variables, and also allow for easy comparison between the two different groups.

The variables „change effort‟ and „power & influence were chosen to measure if differences exist between the three levels of the E-EIM in the perceptions of the user6. The survey results of the first level of analysis for these two variables were incorporated in the subscale calculations outlined in Table 5, which will in effect be used as scores for this first level of analysis. For this reason, the scores obtained

6

(29)

from the individual questions related to Table 6 and 7 for the first level analysis will not be discussed, as to prevent confusion. Furthermore, in a further attempt to measure if users feel benefits (or disadvantages) are equally distributed between users themselves, Alpha Inc. as a whole, and other users, the variable benefit is used. This variable should not be confused with the other input and outcome variables: it is merely a means of exploring the three-level framework of the E-EIM.

The interview transcripts will be used to add to the survey results, looking for ways in which they are similar and/or different. A set of tables containing all relevant information related to my research model obtained from the interviews is outlined in Appendix C (tables 13 through 16). In an effort to summarize these transcripts, an additional set of tables was formulated looking for similarities between interviewee inputs (tables 9 through 12). Also, these summaries have been categorized in accordance with the variable and level of analysis they refer to, and whether they comment on variables in either a positive, neutral or negative way.

All information contained in both sets of tables has been coded. Depending on whether a respondent belongs to the group of engineers or the non-engineer group, he or she is either an „E‟ or an „O‟ (for other). Next, „R‟ plus the respondent‟s number indicates which respondent said what. Lastly, the final number indicates how many times a respondent is quoted (i.e. included in the tables).

(30)

6. R

ESULTS

This section of the paper will outline the results obtained from analyzing the survey and interview data for all three levels of analysis of the E-EIM. As mentioned earlier in the methods section, the survey data was analyzed by composing tables outlining values for mean (ranging from 1 to 4), standard deviation (which are either low, medium, or high), and number of observations (i.e. „N‟). The non-engineer group is designated as „Other‟ in tables 6, 7 and 8.

All questions contained in the survey were phrased in a so-called positive manner. Therefore: for all mean scores included in tables 5, 6, 7, and 8, higher is better. For example: if a user gives a low score to the variable time, then this particular user does not expect the use of R-MIS to save him or her time. Another example: system use complexity. If this variable is rated highly by a user, then he or she feels the system is very easy to use, and is likely to make few mistakes when using it.

The interview results have been put in tables that summarize the key findings. These summaries have been categorized in accordance with the variable and level of analysis they refer to, and whether they comment on variables in either a positive, neutral or negative way.

6.1 First level of analysis

The E-EIM‟s first level of analysis concerns the user‟s relative outcomes. Using the subscales I calculated for each of variables contained in the E-EIM, I derived the following table:

TABLE 5

Input and Outcome Variables, Subscales

Interestingly, the non-engineer user group scored higher on almost all variables, save two: job performance, for which it had a lower score, and autonomy, which outlines similar scores.

A number of elements in this table stand out. First, the number of observations per variable varies: this may be attributed to users feeling unsure how to rate certain questions, and option for the „no opinion‟ answer. Also, there are a number of instances where the standard deviation for a particular

Input Variable Outcome Variable User role System use complexity Change effort Time Job performance Information value Impact on work

Autonomy Power & influence Engineers

(N = 5)

Mean 2.60 2.50 2.50 3.00 2.42 2.38 2.67 2.00

N (observations) 5 5 2 2 4 4 3 3

Std. Deviation Medium Medium Medium Zero High High Medium Medium

Other (N = 16)

Mean 2.87 2.88 2.85 2.75 2.79 3.32 2.63 2.78

N (observations) 15 16 10 8 12 11 8 9

(31)

variable is high: this suggests there may be significant differences between how individual users value a certain variable. For example: in the case of the time variable, „Other‟ has a fairly high mean, but also a high standard deviation. This suggests that while some might feel R-MIS will save them time in their work, others might expect the opposite to be true. Indeed, when looking at the individual scores, a number of users rate the variable time with a 4 (strongly agree), while there is also a user that rates time with a 2 (somewhat disagree), and even a user that rates it with a 1 (strongly disagree). Similar results may be derived for the variables „information value‟ and „impact on work‟ for the engineer group, and „job performance‟ and „power & influence‟ for the non-engineer user group. Another interesting outcome is job performance for the engineer group. Both observations here suggest that engineers expect the system to improve their job performance. However, taking into account that three observations are missing, other engineers might not be so sure about that. Lastly, I will have a close look at „autonomy‟. Notably, for the non-engineer group, half of the observations are missing. Also, when looking at the individual scores assigned, it seems that the average score is weighed down by two respondents, who assigned a score of 1 and 2, respectively.

As mentioned earlier, the variables „change effort‟ and „power & influence‟ were incorporated in the subscale variables outlined in Table 5. Therefore, I will now continue with the survey results for the variable „benefit‟.

TABLE 8

Benefit, Three Levels of Analysis

User role R-MIS benefits me R-MIS benefits Alpha Inc. R-MIS benefits other users Engineers (N = 5) Mean 2.50 3.50 3.67 N (observations) 4 4 3

Std. Deviation High Medium Medium

Other (N = 16)

Mean 3.25 3.85 3.54

N (observations) 12 13 13

Std. Deviation Medium Low Medium

In this table, for the first level of analysis, the group of non-engineers has both a higher mean score and a lower standard deviation in comparison to the group of engineers. This provides some indication of non-engineers feeling they benefit more from R-MIS than non-engineers feel they do as a group. Interestingly, when looking at the individual scores assigned by the engineer group, the high standard deviation is easy to explain: the scores range from 1 to 4, each individual user rating it differently.

(32)

TABLE 9

Summary Key Points Interviews, Input & Outcome Variables

Variable Negative Neutral Positive

Inp

u

t

System use complexity

„The process to enter data is not user-friendly.‟

O-R9-2, O-R10-1

„The application is very easy to use.‟

O-R2-1, O-R5-1

„The interface is user-friendly.‟

O-R8-1, O-R9-1

Time „Data input costs more time.‟

O-R2-2, O-R4-1, O-R9-2, E-R11-1

„R-MIS takes the same amount of

time.‟

O-R10-2

„It will save time.‟ O-R1-1, E-R3-1, O-R5-2

O u tc o m e Job performance

„Job performance will not improve.‟

E-R11-2

„Down-time of the web application poses a risk.‟

O-R8-2, O-R9-3

„Job performance will stay the same.‟

0-R10-3

„Job performance will initially worsen, but after a while it will greatly improve.‟

O-R6-1

„Job performance will improve, because R-MIS gives me a single tool to do all the exercises.‟

O-R9-4

Information value

„Some types of data cannot be stored.‟

O-R4-3

„R-MIS allows for a better understanding of the data.‟

O-R1-2, O-R6-2

„Documents may be added to forecasts (…).‟

O-R4-2

Impact on work Autonomy

First, „system use complexity‟. In terms of positive remarks, two interviewees of the „Other‟ group commented that they found the application very easy to use. Of these two, respondent 2 (R2) argued that R-MIS‟ standardization made it a consistent, and therefore –over time- easier to use for people. Two other respondents of the „Other‟ group found the system‟s interface to be very user-friendly. R10 attributed this to the program‟s web-interface.

When looking at negative aspects associated with system use complexity, two users specifically commented that the (current) process to enter data is not user-friendly. Indeed, R9 described it as „almost a show-stopper‟ for the general population to start using R-MIS.

(33)

Third, „job performance‟. Here, R9 felt that R-MIS might improve job performance, because it will provide a single tool for all resource management related tasks. However, this user was also quick to point out that with R-MIS being a web-application, possible down-time might pose a (significant) risk: this issue was also raised by R8. As was the case for the variable time, R10 takes on a more neutral standpoint, venturing that job performance is likely to stay the same. R10 put it succinctly by stating that: “it is an application, still in the development phase. It looks promising (…), but there are quite some things that need to be solved and procedures to be put in place.”

Another interviewee, R6 distinguished between the short term and the long-term effect of R-MIS on job performance by arguing that while job performance may initially worsen, it will greatly improve after a while. The reasoning behind this is that in the short term there is a degree of effort involved in rolling out the system. After this initial period, the benefits will materialize, improving job performance as a result.

One of the more interesting comments related to job performance came from R11, who argued that with R-MIS, even “if all the functionality is there, it will be the same as Excel.” Put differently, even if all the functionality that he feels should be in there, and the functionality that is promised for future versions, R-MIS can only be ever be as good as the previous use of spreadsheets, according to R11.

Fourth, information value. R1 and R6 argue that R-MIS allows for a better understanding of the data. It is interesting to note that both users belong to RM staff. For these two users, R-MIS offers the possibility of increasing information value in a significant way. For example: R1 argues that the R-MIS will provide him with a better understanding of the portfolio (i.e. the whole of producing assets and projects under development). R4 also feels that information value is improved with R-MIS, because the system includes the added functionality of being able to attach documents to forecasts: something previously not possible with spreadsheets. On a more critical note, R4 did comment that currently not all types of data can be stored in R-MIS, although this user did feel hopeful that the development team will pick up the issues he has raised.

(34)

TABLE 10

Summary Key Points Interviews, „Change Effort‟ per Level of Analysis

Variable Negative Neutral Positive Switch to R-MIS required

little effort from me

„We will have to change our workflows.‟

O-R1-1, O-R2-1

„Training, and the transfer from the current system will

take time.‟

O-R2-2, E-R3-1

„The application is easy to learn.‟

O-R5-1, E-R11-1

Switch to R-MIS required little effort from Alpha Inc.

„It will take time for Alpha Inc. to incorporate the

R-MIS.‟

O-R1-2, E-R3-2

Switch to R-MIS required little effort from other users

„Some will have to change their workflows more than

others.‟

O-R1-3

„Other regions have it easier.‟

O-R2-3

On the positive side, two users commented that the application is easy to learn. R1 and R2 commented that there is a need to change workflows, following the implementation of R-MIS. Both respondents emphasized the relation between benefits and the need to change work flows. On the one hand, R1 mentioned that “eventually it will be very positive, but it will require a lot of change management in work flows.” In a slightly different vein, R2 argued that “for Alpha Inc. it is only benefits, while we also have to change our work methods.” Both respondents are non-engineers, and therefore have to process the information that is keyed in by engineers. It would seem that for these users, the different manner in which the information resource information is delivered to them requires (significant) changes in work flows. Furthermore, R2 and R3 noted that the training involved in the deployment of R-MIS and the transfer to the current system will also take time.

(35)

TABLE 12

Summary Key Points Interviews, „Benefit‟ per Level of Analysis

Variable Negative Neutral Positive R-MIS

benefits me

„We are the business owners of the tool, so we will have an extra effort in the rollout.‟

O-R6-1

„I will benefit from R-MIS‟

O-R6-2, O-R9-1

R-MIS benefits Alpha Inc.

„There are no differences in terms of benefits or

disadvantages.‟

O-R1-1, O-R6-3, O-R7-1, O-R10-1

„The Alpha Inc. Group will benefit.‟ E-R11-1 R-MIS benefits other users

disadvantages.‟

O-R7-2, O-R10-2

„Other users are impacted more.‟

E-R11-2

„Everyone will benefit.‟

O-R1-2, E-R3-1, E-R3-2, O-R5-1

„Engineers will benefit less.‟

O-R1-3, O-R1-4, O-R2-1, O-R2-2, O-R2-3, O-R9-2

In terms of personal benefits, R6 explained that he enjoyed both benefits and suffered disadvantages from using R-MIS: “because we are the business (ed. i.e. responsible for monitoring the interests of the business in this project) owners of the tool, we will have an extra effort in the roll-out.” On the other hand, R6 stated: “we will have an extra benefit in its use because we will be the only one using it on a global scale.” R9 posited that while in the long term benefits are the same for everyone, she expected to gain more benefits in the short term than others from using R-MIS.

6.2 Second level of analysis

(36)

TABLE 6

Change Effort, Three Levels of Analysis

User role Switch to R-MIS required little effort from Alpha Inc. Switch to R-MIS required little effort from other users Engineers (N = 5) Mean 2.00 2.33 N 2 3

Std. Deviation Zero Medium

Other (N = 16)

Mean 2.58 2.67

N 12 15

Std. Deviation Medium Medium

For the variable „change effort‟, even though the mean score for „Other‟ is higher, engineers have a standard deviation of exactly zero. It should be noted, however, that there are only two valid observations for the engineer group. For the „Other‟ group, while there are a few outliers (i.e. two scores of 1, one score of 4), most users assigned a score of 3, resulting in a fairly neutral overall mean score.

TABLE 7

Power & Influence, Three Levels of Analysis

User role R-MIS increases Alpha Inc.’s power and influence over others R-MIS increases other users’ power & influence over me Engineers (N = 5) Mean 2.67 1.67 N (observations) 3 3

Std. Deviation High High

Other (N = 16)

Mean 3.00 2.10

N (observations) 12 10

(37)

TABLE 8

Other (N = 16)

Mean 3.25 3.85 3.54

For the „benefit‟ variable, engineers have a (somewhat) lower mean score, and also a higher standard deviation. Looking more closely at the individual rating for the engineer group, it appears that scores vary between 3 and 4 suggesting that both user groups feel the R-MIS is beneficial to Alpha Inc.

Next, I will analyze the interview results for the second level of analysis, starting with the variable „change effort‟:

TABLE 10

O-R1-1, O-R2-1

take time.‟

O-R2-2, E-R3-1

O-R5-1, E-R11-1

R-MIS.‟

O-R1-2, E-R3-2

others.‟

O-R1-3

O-R2-3

(38)

TABLE 11

Summary Key Points Interviews, „Power & Influence‟ per Level of Analysis

Variable Negative Neutral Positive R-MIS increases my power &

influence over others

R-MIS increases Alpha Inc.’s power & influence over others

„R-MIS enforces consistency of resource data.‟

O-R4-1, O-R6-2

„Engineers will face higher accountability standards.‟

O-R1-1

R-MIS increases other users’ power & influence over me

For the variable „power & influence‟ R4 and R6 point out that the R-MIS enforces consistency of resource data. Due to the standardized nature of R-MIS, and the fact that it is an across-the-group application, Alpha Inc. is able to exert a far greater degree of control of the resource management process. This also translates into engineers facing higher accountability standards, according to R1.

TABLE 12

benefits me

O-R6-1

O-R6-2, O-R9-1

disadvantages.‟

O-R1-1, O-R6-3, O-R7-1, O-R10-1

disadvantages.‟

O-R7-2, O-R10-2

E-R11-2

O-R1-2, E-R3-1, E-R3-2, O-R5-1

O-R1-3, O-R1-4, O-R2-1, O-R2-2, O-R2-3, O-R9-2

(39)

6.3 Third level of analysis

The E-EIM‟s third level of analysis concerns the user‟s relative outcomes in comparison to other users or user groups. For the survey results the „change effort‟, „power & influence‟ and „benefit‟ variables have been used to explore the third level of analysis.

TABLE 6

Change Effort, Three Levels of Analysis

User role Switch to R-MIS required little effort from Alpha Inc. Switch to R-MIS required little effort from other users Engineers (N = 5) Mean 2.00 2.33 N 2 3

Std. Deviation Zero Medium

Other (N = 16)

Mean 2.58 2.67

N 12 15

Concerning the variable „change effort‟, both groups rate it about neutral, although „Other‟ does have a higher mean score: both groups have a medium standard deviation. As such, there appears to be no clear indication of either user group feeling that other users had to put in little effort to switch to R-MIS, in comparison to the amount of effort they had to put in themselves.

TABLE 7

Power & Influence, Three Levels of Analysis

User role R-MIS increases Alpha Inc.’s power and influence over others R-MIS increases other users’ power & influence over me Engineers (N = 5) Mean 2.67 1.67 N (observations) 3 3

Std. Deviation High High

Other (N = 16)

Mean 3.00 2.10

N (observations) 12 10

(40)

TABLE 8

Other (N = 16)

Mean 3.25 3.85 3.54

For the variable benefit, both groups feel that R-MIS benefits other user, with engineers rating it slightly higher than non-engineers. When looking at the survey results, all respondents (i.e. both groups) rate this question with either a three (somewhat agree), or a four (strongly agree). This provides some evidence that both user groups feel that other users benefit from R-MIS. Also, perhaps engineers feel that other users benefit more.

Next, I will analyze the interview results for the third level of analysis, starting with the variable „change effort‟:

TABLE 10

O-R1-1, O-R2-1

take time.‟

O-R2-2, E-R3-1

O-R5-1, E-R11-1

R-MIS.‟

O-R1-2, E-R3-2

others.‟

O-R1-3

O-R2-3

(41)

Considering that the interview did not provide any input for the third level of analysis for „power & influence, I will move on to the variable „benefit‟:

TABLE 12

benefits me

O-R6-1

O-R6-2, O-R9-1

disadvantages.‟

O-R1-1, O-R6-3, O-R7-1, O-R10-1

disadvantages.‟

O-R7-2, O-R10-2

E-R11-2

O-R1-2, E-R3-1, E-R3-2, O-R5-1

O-R1-3, O-R1-4, O-R2-1, O-R2-2, O-R2-3, O-R9-2

(42)

7. D

ISCUSSION

In this section, the results outlined in the previous section will be discussed more in-depth. I will discuss the results of each level of analysis separately. For first level of analysis, I will start with the most notable findings of the survey, which I will relate to the input and outcome variables of the E-EIM (see Table 3) so as to explore whether users perceive increases or decreases, respectively. In addition, through the use of triangulation (i.e. cross-validating the survey and interview results) I will investigate whether the survey and interview results7 converge or diverge. Next, I will look at the variables „change effort‟, „power & influence‟, and „benefit‟ to explore whether users feel a sense of equity or inequity when comparing all three levels of analysis. Again, I will start with a discussion of the survey results, and proceed with the use of triangulation to investigate whether the survey and interview results converge or diverge. Lastly, I will at the general picture that arises when considering the research findings from a holistic point of view.

7.1 First level of analysis

The EIM‟s first level of analysis concerns the user‟s relative outcomes. The input variables of the E-EIM included „system use complexity‟, „time‟, and „change effort‟.

First, „system use complexity‟: both user groups rate the variable higher than neutral (with the non-engineer group rating it the highest), which translates into a perceived decrease of inputs. When looking at the interview findings, there is evidence of both convergence, and divergence. On the one hand, a number of respondents of the „Other‟ group confirmed (i.e. provided evidence of convergence) the (moderately) positive score, by describing the application as being easy to use, and having a user-friendly interface. Two other respondents criticize the process of entering data. In a way, this might explain why a system that is described as being easy to use, and user-friendly, only has a marginally positive score for „system use complexity‟.

Second, „time‟. The survey results for the engineer group provide very little information: not only are three observations missing, the medium standard deviation suggests users rate this item differently. The interview results show an equally disparate picture, with one engineer arguing that the system will save time, and another the opposite. As such, the interview findings confirm the survey results. For the non-engineer group, the story is similar: even though the mean score is higher than that of the non-engineer group, its standard deviation is higher. This may be explained when looking at the interview results: whereas two

7