Through The Pearly Gates: Post Migration Experience Analysis of Software Modernization

(1)

MASTER THESIS

Through The Pearly Gates: Post Migration

Experience Analysis of Software Modernization

Author: Prajan Shrestha M.Sc. Software Engineering Student number: 10476075 e-mail: prajan.shrestha@student.uva.nl Supervisor:

Ravi Khadka (Utrecht University) e-mail: r.khadka@uu.nl

2nd Supervisors:

Magiel Bruntink (University of Amsterdam) e-mail: m.bruntink@uva.nl

Slinger Jansen (Utrecht University) e-mail: slinger.jansen@uu.nl

(2)

i

ABSTRACT

Software modernization has been extensively researched, primarily focusing on observing the associated phenomena, and providing technical solutions to facilitate the modernization process. Software modernization is claimed to be successful when the modernization is completed using those technical solutions. Academia and software vendor- publicize such ‘technically’ successful modernizations but very limited research, if any, is reported with an aim at documenting the post-modernization impacts, i.e., whether any of the pre-modernization business goals are in fact achieved after modernization.

Several studies have investigated software evolution and maintenance listing numbers of claimed benefits, however, with no empirical evidence. This research attempts to address the relative absence of empirical study through three retrospective software modernization case studies. This research uses an exploratory case study approach to document the pre-modernization business goals and to decide whether those goals have been achieved. Practitioners from the industry are interviewed in order to get a good overview of pre- and post-modernization business goals. The claimed benefits from the academic literature are also scientifically documented, which are later used to cross analyze with the business goals from industry cases.

The expected modernization benefits for each of the three industry cases were achieved. Moreover, all industry cases exhibited a number of unintended benefits, and some reported detrimental effects of modernization. By investigating claimed benefits from academia and performing cross analysis against the business goals of case studies, this thesis finds no significant gap between academia and industry. However, some unintended benefits of modernization are ignored in the academic literature.

(3)

ii

PREFACE

This master research is the final deliverable of M.Sc. Software Engineering program at the University of Amsterdam.

First, I would like to thank my external supervisor Ravi Khadka, Utrecht University, for his support and guidance throughout this research. Besides him, I would also like to thank my internal supervisor Dr. Magiel Bruntink, University of Amsterdam, and Dr. Slinger Jansen for being the second examiner of this research.

Furthermore, I would like to thank all case companies for their participation who shared their valuable knowledge and insights to make this research happen.

Finally, I would like to thank my family and especially my wife Priyanka Pradhan for everything they have done for me in the past.

(4)

iii

A. TABLE OF CONTENTS

Abstract ... i

Preface ... ii

A. Table of contents ... iii

B. List of figures ... v C. List of tables ... v 1. Introduction ... 1 1.1 Problem area ... 2 1.2 Research questions ... 3 1.3 Contribution ... 4 1.3.1 Scientific contribution ... 5 1.3.2 Societial contribution ... 5

2. Research method and approach ... 7

2.1 Snowballing based literature study... 8

2.2 Case study ... 10

2.2.1 Case study type and design ... 11

2.2.2 Preparation and collection of data ... 13

2.2.3 Analysis ... 15

2.2.4 Reporting ... 19

2.2.5 Research validity ... 19

3. Literature study... 21

3.1 Legacy systems ... 21

3.2 Legacy modernization strategies ... 21

3.3 Legacy modernization benefits ... 23

4. The three industry cases ... 26

4.1 Case I: The Electrical Company ... 27

4.1.1 Case description ... 27

4.1.2 Benefits (pre- and post-modernization) ... 27

4.2 Case II: The Aviation Company ... 35

(5)

iv

4.3 Case II: The Government Office ... 44

4.3.1 Case description ... 44

5. Discussion ... 48

5.1 Modernization objectives are case dependent ... 48

5.2 Expected benefits are achieved in each case company ... 49

5.3 Unintended benefits after modernization ... 50

5.4 System performance not a primary objective, but remained satisfactory after modernization ... 50

5.5 Casual relationships among benefits (Business goals)... 51

5.6 Organizational perspective of modernization ... 53

5.7 Claimed benefits from academia vs business goals from industry ... 53

5.8 Threats to validity ... 54 5.8.1 Construct validity ... 54 5.8.2 Internal validity ... 55 5.8.3 External validity ... 55 5.8.4 Reliability ... 55 6. Conclusion ... 58 6.1 Future research... 60 References ... 61

Appendix A - Documented claimed benefits using backward snowballing... 65

Appendix B - Definition of claimed benefits ... 69

Appendix C - Code collection ... 70

Appendix D - Participants quotations ... 72

Appendix E - Index codes of participants ... 85

(6)

v

B. LIST OF FIGURES

Figure 1: Research model (Yin, 2003)... 8

Figure 2: Backward snowballing (Wohlin, 2014) ... 10

Figure 3: Basic types of designs for case studies (Yin, 2003) ... 12

Figure 4: General principles for interview sessions (Runeson & Höst, 2008) ... 14

Figure 5: Data transcribing using NVivo10 ... 16

Figure 6: Coding concepts by (Corbin & Strauss, 1990) ... 17

Figure 7: Coding representation using NVivio10 ... 18

Figure 8: Modernization strategies compared with respect to cost and reuse ... 23

C. LIST OF TABLES

Table 1: Details of the interviewees... 14

Table 2: Case study research four validities (Yin, 2003) ... 20

Table 3: Claimed benefits from academia in technological perspective ... 24

Table 4: Claimed benefits from academia in organizational perspective ... 25

Table 5: Case analysis summary of The Electrical Company ... 35

Table 6: Case analysis summary of The Aviation Company ... 43

Table 7: Case analysis summary of The Government Office ... 47

Table 8: Cross-Case Analysis of three case studies ... 48

(7)

1

1. INTRODUCTION

Over the last few years, software systems have become one of the core components to successfully run an organization. A software system not only consolidates business information but also supports the information flow in an organization (Bisbal, Lawless, & Grimson, 1999). Various changes occur such as business requirement changes, mergers and acquisition changes, laws and regulation changes, supporting information technology changes, etc. To adapt these changes, software systems should also be changed, if not adopted, they become useless.

The first law of Software Evolution, known as the law of continuing change (Lehman, 1968), describes that a software system becomes a core component by adapting to changes in a real-time operational environment. The organization requires changes in a software system to adapt to changing business requirements, enabling faster-time-to-market, intra- and inter-organizational changes, and business collaboration via mergers and acquisition (Khadka, Batlajery, Saeidi, Jansen, & Hage, 2014). Organizations are in constant pressure to adapt their systems to these changes. Such changes, if not adopted properly, can increase the complexity of software. The unmanaged adaptations in software lead to increased dependencies and interactions between software components resulting in an unstructured pattern, and hence increasing the complexity of the software system. Research indicates that changes are adopted in ad-hoc manner which in long-term brings issues related to maintainability and flexibility, thereby making software system legacy- software system that resists modification and incurs high maintenance cost (Bisbal et al., 1999).

Such legacy systems are built more than a decade ago and are written in obsolete programming languages such as COBOL or FORTRAN (Brodie & Stonebraker, 1993) and often incur higher cost to maintain (Bisbal et al., 1999). Additionally, such legacy systems are not flexible and resist modification, documentation is often missing and experts are rare to maintain these systems (Khadka et al., 2014). However, legacy systems are crucial to an organization as they are mission critical and their failure can have a serious impact in business (Bennett, 1995). With years of testing and optimization, legacy systems are fit-for-purpose. Hence, these systems pose a dilemma, often termed as the “legacy dilemma” (Bennett, 1995).

On the one hand, legacy systems are fit-for-purpose, perform crucial tasks for the organization, and are vitally important to the continuity of the business. On the other hand, such systems are expensive to maintain, resist modifications and are not sufficiently flexible to cope changes in a business environment. An approach to deal with this dilemma is to modernize such legacy software systems. Software modernization is the process of evolving existing software systems by replacing, re-developing, reusing, or migrating the software components and platforms, when traditional maintenance practices no longer achieve the desired system properties (Khadka et al., 2014).

(8)

2 Software modernization primarily aims at reducing operation and maintenance cost and increase flexibility by evolving monolithic architecture to distributed architecture. Software modernization pays off by reducing in operation and maintenance cost as there is a good availability of skilled resources and vendors around new technology. Software vendors also positively supported organizations’ modernization objectives. After modernization, vendor publicized successful modernization but there is no empirical evidence in real-life situations that explain the objectives of modernization have been achieved. There is room to investigate if expected objectives are really achieved after modernization.

1.1 PROBLEM AREA

There is a tough competition in a business market with evolving innovative business ideas and every enterprise wants to stay competitive. In general, the software system has become one of the core parts of any organization. Implementing business ideas such as business requirement changes, mergers and acquisition changes, laws and regulation changes, etc., require a business process re-engineering which will lead to a change in the software systems. If the software system is unable to handle the required business process, an enterprise seeks for a new software system. For a sustainable growth and an increase in revenue, an enterprise always investigates to reduce enterprise’s cost. One of the primary areas where cost can be reduced is IT (Information Technology) and in general software maintenance cost. In the research performed by Forrester Consulting (Forrester, 2011), the three primary drivers for modernization are high cost (47%), outdated functionality (36%), and transformation program (31%). Therefore, an organization seeks to have a modern portfolio of applications that primarily enables cost reduction and is fit-for-purpose.

In academia, several researchers have indicated the benefits of legacy software modernization, including, but not limited to, increased maintainability, re-usability of legacy assets, increased flexibility and so no. However, there is a lack of empirical validation indicating that such benefits are met after modernization.

Further, it is also a common understanding within the software evolution community that software modernizations are labeled as “successful” once they are technically completed (Khadka & Idu, 2013; Nasr, Gross, & Deursen, 2013) . However, such (technically) successful software modernization does not necessarily indicate that post-modernization business goals such as reduced cost, increased flexibility, increased productivity, and re-usability (Nasr et al., 2013) have been achieved.

There are limited, if any, empirical evidences that explain if pre-modernization business goals are met after a (technically) successful modernization. The two primary reasons are: firstly, in order to analyze a successful modernization, case study research needs to be performed when a modernized system reaches to a level of maturity. However, it is difficult to define the level of maturity, as no one knows how long a software system takes to materialize. Secondly, existing system to compare is not available anymore after modernization and that makes it further

(9)

3 difficult to analyze reduced cost, increased flexibility, increased productivity, and re-usability (Nasr et al., 2013).

In this research, we aim at empirically documenting this knowledge gap around the post-modernization effects. We take an initial step to perform retrospective case studies of successful software modernization with an aim to bridge the knowledge gap around the lack of empirical evidences of benefits of software modernization.

In this research, we distinguish between the following types of modernization benefits (business goals). Modernization benefits reported in academia without sufficient empirical evidence are termed “claimed benefits”. Pre-modernization benefits that any organization aims to achieve by software modernization are hereinafter known as “expected benefits”. Such pre-modernization business goals will be validated if they are met after modernization. Such post-modernization benefits are termed as “observed benefits” in this research. Finally, we use “unintended

benefits” to indicate if any unexpected benefits are observed as a result of software

modernization.

1.2 RESEARCH QUESTIONS

This research aims at investigating the expected benefits of modernization. The benefits of software modernization are repeatedly discussed in various studies (Almonaies, Cordy, & Dean, 2010; Khadka & Idu, 2013; Nasr et al., 2013), but less attention from research community is provided on empirically validating if the expected benefits are met by the organization. A careful empirical validation might lead to new insights on understanding the impacts of modernization. Successful technical modernization does not necessarily indicate that the expected benefits are met. There can be a difference between what is aimed for and what has been observed after modernization.

This research addresses the following research question:

The research is expected to gather empirical data regarding expected and observed benefits of modernization from three case studies. The collected data are analyzed to understand if expected benefits are really observed after modernization. The result is further analyzed with claimed benefits that may further provide empirical evidence in academia. The main research question is divided into the following sub-research questions.

Main Research Question:

(10)

4 The claimed benefits of software modernization have been discussed in academic research. With this research question, we gather claimed benefits from the existing scientific literature. We perform a systematic literature study to identify the relevant research. Furthermore, these claimed benefits will be further categorized as technological benefits and organizational benefits. The claimed benefits are further empirically validated with the results of this research.

It has been widely accepted within academia and practitioners that software modernization can lead to various expected benefits such as maintenance cost reduction, increased flexibility, increased productivity, and re-usability. In the research, three retrospective case studies are conducted to collect empirical data about the expected (pre-modernization) benefits of the software modernization. During the analysis of collected data, benefits discussed in a pre-modernization situation will be documented.

In the research, empirical data are collected and analyzed. The analysis of pre-modernization phase answers SRQ2. During the analysis of post-modernization phase, observed benefits are documented and is compared with the expected benefits from SRQ2.

The results of SRQ1 and SRQ4 are further analyzed to get an answer to this research question. It further provides empirical validity to SRQ1.

1.3 CONTRIBUTION

This section elaborates the contribution of this research. Firstly, the scientific contribution explains why the research has been carried out, what problems led to the research, and how the research will contribute towards academia. Secondly, a societal contribution explains how the research will contribute to the industry, what problems still exist, and how the research will address those problems.

Sub- Research Question 3 (SRQ3):

Which of the expected benefits are observed and are those expected benefits met? Sub- Research Question 2 (SRQ2):

What are the benefits expected by an organization after a successful modernization? Sub- Research Question 1 (SRQ1):

What are the claimed benefits of software modernization?

Sub- research Question 4:

(11)

5

1.3.1 SCIENTIFIC CONTRIBUTION

Academic research such as the one carried out by Nasr (Nasr et al., 2013) mentions legacy modernization to be successful once it is (technically) feasible. However, such (technically) successful modernization does not necessarily indicate that it has been successful on achieving the business goals (benefits). Hence, the impact of modernization should be validated in real-life situations where comparisons between what business goals are aimed at and which of them indeed have been achieved (Nasr et al., 2013). Despite, a plethora of research, there are limited, if any, empirical evidences that explore the relative knowledge gap in terms of validating if pre- and post-software modernization business goals are met (Khadka, Saeidi, Idu, Hage, & Jansen, 2012)

Legacy software modernization is a widely discussed topic in academia. Several areas such as modernization strategies, software modernization models, and claimed benefits are discussed, but empirical evidence of modernization benefits are researched only by a few researchers. This research aims to fill this gap by making the following contributions to the scientific knowledge base:

 There are academic papers discussing claimed benefits of modernization. However, such claimed benefits discussed in academia are not systematically documented yet. This research helps to systematically document claimed benefits of modernization by performing a systematic literature study.

 To our knowledge, there are relatively less academic studies that discuss the benefits of software modernization in real-world industry cases after a modernized system reaches some maturity. Therefore, the contribution of this paper is to provide real-world industry cases where a clear understanding can be created between the expected benefits and the observed benefits. The analysis, thus, helps to provide an understanding of software modernization benefits in the organization.

 Using multiple real-world industry cases, the research explores if the expected and observed benefits of all organizations are similar or do they depend on an organization’s niche market.

1.3.2 SOCIETIAL CONTRIBUTION

The importance of legacy modernization is not only a research topic, but is also a main agenda in the industry. In the year 2013, “Improving IT applications and infrastructure” and “Legacy modernization” have been both listed at 5th

positions out of top 10 business priorities and top 10 technical priorities consecutively (Gartner, 2013). It is surprising that not many real-world industry case researches have been done to document benefits of modernization though software modernization is a well-established research domain.

(12)

6 Using multiple cases, the research contributes to a better understanding of the benefits of software modernization. The artifacts presented in this research not only contribute to the scientific knowledge base, but also helps organizations to examine the evidence practically. More specifically, the industry contributions of this research are as follows:

 Using a systematic literature study, claimed benefits from academia have been documented in this research. Software vendor publicizes various business goals of software modernization. The industry can cross-validate software modernization benefits mentioned by software vendors against the claimed benefits documented in this research.

 An organization that falls under same niche market can use this research to generalize their software modernization planning. This helps to understand if business goals planned in pre-modernization phase can really be achieved in post-modernization phase.

 During this research, a number of unintended benefits after software modernization have been observed. Thus, this research is a guideline to indicate that software modernization is not only about expected and observed benefits but also about unexpected outcomes.

(13)

7

2. RESEARCH METHOD AND APPROACH

This chapter explains the research background and research method used in this research. First, a literature study is performed to understand the legacy modernization from an academic perspective. This literature study provides a theoretical background explaining the need of legacy modernization for an organization and what goals (technical and business) are expected after a successful modernization. In academia, the modernization drivers include reduced cost, increased flexibility, increased productivity, and re-usability of the existing (legacy) assets. As indicated earlier in Chapter 1, such benefits are largely claimed without sufficient empirical evidences. Hence, in this research, such academic benefits are termed as “claimed benefits”, hereinafter. In Section 2.1, we document claimed benefits reported in academia using a backward snowballing literature study method.

As for the second part of the research, three case companies are opportunistically selected where legacy software modernization was successfully performed in the past. Based on their successful software modernization experiences, we aim at documenting their pre-modernization business goals, hereinafter known as “expected benefits” followed by post-modernization experiences to validate if those “excepted benefits” were observed. Such post-modernization benefits are termed as “observed benefits” in this research. Furthermore, we use “unintended

benefits” to indicate if any unexpected benefits are observed as a result legacy software

modernization. For each case company, data were gathered by conducting interview sessions with the resources involved in the modernization process. In addition to interview, documents, if available, were analyzed to obtain or support the findings from interviews. The data collection and analysis focus on pre- and post-modernization scenarios, including expected benefits and observed benefits of legacy system modernization.

Figure 1 depicts the overall research model used in this research. It starts with a literature study that provides relevant knowledge to construct and strengthen the theoretical framework for the research. The second step relates to a case study research method, which starts with identifying relevant case companies. To prepare for data collection, a case study protocol and an interview protocol are prepared. Such protocols contain a short introductory summary of the research, personal information about the interviewee, information about the modernization project and set of representative questions that will be discussed during the interview sessions. The interview s are then conducted and are further analyzed and refined during several steps that include transcribing the interview, coding, and memoing. If necessary, the interviewee is contacted for further explanation of any needed information. The details of the case study research are presented in Section 2.2.

(14)

8 Case study and

data collection protocol Interview Memoing Coding Transcribing Refinement Validation

Literature Study Case companies

Figure 1: Research model (Yin, 2003)

2.1 SNOWBALLING BASED LITERATURE STUDY

Systematic studies of the literature can be conducted in different ways. Some of the widely used literature study methods include systematic approaches such as systematic literature review (SLR), systematic mapping studies (SMS) and snowballing approach (Jalali, 2012).

SLR is used to find relevant literature and aims at establishing the state of evidence through systematic searches in different databases with a well-defined search string (Jalali, 2012). This approach starts with formulating a well-formed search string. The search string is then used against different databases\indexing services to identify relevant literature. The formulated search string might vary while using different databases. Finally, before including in the list, relevance and quality of found literature are checked.

SLR is not the only method to systematically gather and analyze existing literature. Systematic mapping study (SMS) is also a widely used approach which is a suitable approach when there is a little research that requires more theoretical and empirical research or the research topic is fairly broad and scattered (Kitchenham et al., 2009). Moreover, SMS is particularly used when the research is to uncover new trends and that very less literature study has been conducted in such specific areas.

(15)

9 Snowballing is a procedure to identify additional relevant research papers by using the reference list of a paper or the citations to the paper (Wohlin, 2014). This procedure starts with identifying a initial set of papers. From initial set of papers, its reference link, or citations are checked to identify a new set of papers and the iteration continues until no new papers are found. This saturation stage concludes snowballing. A snowballing method can be carried out in two different ways: forward snowballing and backward snowballing.

Snowballing procedure has two major steps: i) identifying an initial set of papers, and ii) iterations. The main challenge in snowballing procedure is to identify an initial set of papers. However, there is no silver bullet for a good initial set papers, but are likely to have characteristics such as should be from keywords in the research question; should be highly cited papers; should come from different communities.

The second step of snowballing is iterations that can be either forward or backward. Forward snowballing identifies new papers based on those papers citing the initial set papers. Google scholar that uses Abstracting and Indexing Services is used for citations to obtain relevant information about the paper (Wohlin, 2014). If the obtained relevant information is insufficient to decide on exclusion or inclusion then citing paper’s abstract is read and if necessary, the whole paper. The iteration for inclusion or exclusion continues until a saturation stage is reached.

One of the main reasons to choose snowballing is that it requires lesser time and effort in searching new papers since it starts from the reference list or citation of initial set of papers and moves further. Using snowballing, papers can be efficiently identified either from the reference list or from citations while SLR requires formulating separate search strings for separate databases. As identified in a research, SLR generates more irrelevant papers (85% noise) compared to snowballing (32% noise) (Jalali, 2012). On the other hand, SMS against snowballing, SMS is inappropriate as there are abundant studies published and this research aims to provide a complete overview of the research domain, based on all papers published on legacy software modernization. In snowballing, inclusion is based on the title of the paper and sometimes reading the abstract while using an SLR, both title and abstract are read before deciding for inclusion. It is also worth to note that snowballing is easy to understand and follow compared to an SLR or SMS.

This research uses backward snowballing as compared to forward snowballing. The backward snowballing is preferred because the research topic is based on a paper included in one the initial set of papers and reference link is easier to follow and lesser time consuming compared to citations. Furthermore, if the initial set of papers are from recent years, relevant papers might be missed due to fewer citations.

The backward snowballing approach is depicted in Figure 2. Backward snowballing uses reference list of initial set papers to identify new papers to include. Only those papers are

(16)

10 included that meet basic criteria such as language, publication year, type of publication, etc. A list is further examined to remove duplicate papers that were already found during forward or backward snowballing. The iteration continues until a saturation stage is reached.

During the iteration, when a new paper is found, its abstract is read and if required, other sections are read as well to make a decision on inclusion or exclusion. If a wrong paper is selected for inclusion, the selected paper should be excluded. The whole process is sometimes time-consuming, so extra care should be taken during inclusion or exclusion. The iteration of exclusion and inclusion will lead to the saturation stage when no new information is found by reading new papers. This will conclude the snowballing procedure.

Start literature search

Identify a tentative start set of papers and evaluate the papers for inclusions or exclusions. Included papers enter the snowballing procedure

Backward snowballing

Final inclusion of a paper should be done based on the full paper, i.e. before the paper can be included in a new set of papers that goes into the snowballing procedure

Iterate:

1. Look at title in reference list 2. Look at the place of reference End iterate

3. Look at the abstract of the paper referenced 4. Look at the full references paper

In each step, it is possible to decide to exclude or tentatively include a paper for further consideration

If no new papers are found then snowballing

procedure is finished

Iterate until no new papers are found

Figure 2: Backward snowballing (Wohlin, 2014)

The objective of using backward snowballing is to identify claimed benefits of software modernization in academia. The results that identify claimed benefits using backward snowballing are presented in Chapter 3, Section 3.3.

2.2 CASE STUDY

The research model in Figure 1 requires a proper research method to fulfill the research objectives. The widely discussed empirical research method includes controlled experiment, case studies, survey research, ethnographies and action research (Easterbrook, Singer, Storey, & Damian, 2002). The most appropriate empirical method for this research is a case study research method.

A number of reasons behind choosing case study as a research method includes: case study examines contemporary phenomena with its natural setting, helps to collect data by multiple means, examines one or few entities (person, group or organization), and no variables definition

(17)

11 in advance that affect the study results (Benbasat, Goldstein, & Mead, 2014). Additionally, the ‘why’ and ‘how’ questions are best answered via case study research as they are traced over time rather than its frequency or incidence.

Case study research is the most widely used qualitative research method and is appropriate to investigate a real-life situation. This research is about investigating real-life scenarios in a number of case companies and is, therefore, the most suitable approach to solving research questions. A case study is defined as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.” (Yin, 2003). When using a case study, the research is open to the use of theory or conceptual categories that guide the research and analysis of data (Yin, 2003).

Considering the aforementioned real-world case based research, case study research method is deemed appropriate for this research.

A typical case study research consists of the following steps: i. Case Study type and design

ii. Preparation and collection of data iii. Analysis

iv. Reporting

In the following sub-sections, the steps of the case study are detailed.

2.2.1 CASE STUDY TYPE AND DESIGN

Case study design and planning is a phase in which different elements are studied to get an answer of a research question. The planning includes what methods to use for data collection, what department of an enterprise to visit, what documents to read, which persons to interview, etc (Runeson & Höst, 2008).

Case studies can be either exploratory, explanatory or confirmatory (Easterbrook et al., 2002; Runeson & Höst, 2008). The research is not confirming to any existing theories but deriving hypotheses, therefore exploratory case study will be the best suitable research method. The exploratory case study is about “finding out what is happening, seeking new insights and generating ideas and hypotheses for new research” (Runeson & Höst, 2008). Thus, to document pre- and post-modernization scenarios, exploratory case study fits well in this research.

Figure 3 depicts single- and multiple-case design including two variants: a single unit of analysis and multiple units of analysis, as illustrated by (Yin, 2003). Figure 3(A) represents holistic single-case design where data collection and analysis is done in a single case. Figure 3(B) illustrates holistic multiple-case design in which data collection and analysis is done in multiple cases. Both Figure 3(A) and 3(B) are considered a single unit of analysis. This research

(18)

12 implements holistic multiple-case design approach for data collection and analysis. Similarly Figure 3(C) and 3(D) represents embedded single-case and embedded multiple-case design that contains single or multiple cases within multiple units of analysis.

Context Context Context Context Context Context Case Case Case Case Case Context Context Context Context Case Case

Embedded Unit of Analysis 1

Embedded Unit of Analysis 2

Embedded Unit of Analysis 2 Case Embedded Unit of Analysis 1 Embedded Unit of Analysis 2 Case Embedded Unit of Analysis 1 Embedded Unit of Analysis 2 Case Embedded Unit of Analysis 1 Embedded Unit of Analysis 2 (A) Holistic single-case design

(D) Embedded multiple-case design (C) Embedded single-case design

(B) Holistic multiple-case design

Figure 3: Basic types of designs for case studies (Yin, 2003)

A holistic multiple-case studies (Yin, 2003) is used in this research in which multiple case companies are chosen to collect and analyze multiple sources of data. In holistic multiple-case studies, a unit of analysis is investigated under multiple cases. The unit of analysis, in this case, is ‘expected and observed benefits of software modernization’. The reasons for choosing multiple case companies are due to the fact that they provide sufficient information for research results, helps for cross-case analysis of the subject information and offers greater validity (Easterbrook et al., 2002; Runeson & Höst, 2008).

An effective case study research requires a proper guide for data collection, concrete research plan, feedback on created plans from relevant stakeholders so that collected data is richer and robust. One of the ways to have an effective case study is to create a case study protocol (Runeson & Höst, 2008). A case study protocol contains design decisions that are shared with

(19)

13 the interviewee of case companies prior to the interview session. The protocol contains guidelines for data collection that includes a short introduction of the topic, questions that help to answer the main research question, related terminologies and research approach for data collection and analysis. The interview guideline is updated when the plans for the case study are changed. The use of case study protocol helps to increase construct validity as case protocol is used to obtain data from one case study and later the same protocol is used to obtain data from other cases studies. The case study protocol can be found in Appendix F.

The research analyzes three Dutch software companies that have performed software modernization. The case companies were opportunistically selected but at least keeping in mind that successful modernization was observerd at least two years prior to this research. The case companies come from different domains: one from the public sector (Government service office) and two are equipment production companies.

2.2.2 PREPARATION AND COLLECTION OF DATA

Data collection method can be divided into three categories namely first degree, second degree, and third degree. In the first degree data collection, the researcher is in direct contact with the subject and collects data in real time whereas second and third degree are indirect ways of data collection (Runeson & Höst, 2008). Therefore, first-degree data collection is appropriate for this research. First degree data collection includes interviews, focus groups, Delphi surveys and observations (Runeson & Höst, 2008). Among these methods, interview suits best for this research as the interviewer is in direct contact with the subject and data is collected in real time (Runeson & Höst, 2008).

Interview is further categorized into:- structured interview in which all questions from interview protocol are asked exactly in planned way; unstructured interview in which no interview protocol is prepare and interviewee is the source of both questions and answer; and semi -structured interview in which questions from interview protocol are asked based on on-going conversation during the interview session (Runeson & Höst, 2008). The main advantage of a semi-structured interview is that it helps to improvise and explore the research question until the interview ends (Runeson & Höst, 2008). Thus, semi-structured interveiw provides highly robust and reliable quality information. This research, hence, adopts semi-structured interviews.

Furthermore, the interview session can be structured in three principles: funnel model, pyramid model, and time-glass model (Runeson & Höst, 2008). Figure 4(a) represents funnel model where the interview starts with open questions and later towards specific ones. Figure 4(b) is a pyramid model that begins with specific ones and end towards open questions. Finally, Figure 4(c) depicts a time - glass model where the interview begins with open questions, gets more precise in the middle, and opens up again at the end of the interview. The research follows a time-glass principle that allows going deeper into the context while finally provides answers to all questions from the interview protocol. During an interview session, the interviewee goes back to the history and provides random data. Hence, it can be said that specific sequenced

(20)

14 questions will degrade data retrieval during the interview. The interview stops when it reaches saturation and no more new information can be extracted.

Figure 4: General principles for interview sessions (Runeson & Höst, 2008)

The interview session can be recorded in an audio or video format. The research utilizes an audio format to record the session.

In this research, the interviewees typically include a project manager and/or a technical manager. In two of the cases, the researcher interviewed an IT director and a finance manager. The researcher aimed at involving different roles for interviews to get diverse information. For instance, an IT Director potentially provides a better rationale/business case for software modernization while a technical manager can elaborate on the technical issues and impacts of modernization.

The case study protocol design results into interview guidelines that are provided to the interviewee. This helps interviewee to gather historical data from memories and thus helps research to collect accurate and richer data. Prior to the interviews, each interviewee was introduced to an interview protocol, a document detailing the objectives of the interview with relevant questions grouped into themes, and a glossary of the technical terms to attain a common understanding. The interview protocol included a brief introduction to the research, questions regarding the personal background of the interviewee (name, current position and responsibilities, expertise and experience), and interview questions categorized into three themes: pre-modernization situation, modernization process, and post-modernization situation. All the interviews were conducted in English and lasted between 60 – 120 minutes. Table 1 depicts the details of the interviewees with anonymized name and company.

No. Role Experience Company

P1 Technical Lead 34 The Electrical Company P2 Developer 29

P3 Finance Manager 28

P4 ICT Business Coordinator 13 The Aviation Company P5 Business System Analyst 34

P6 IT Director 30

P7 Project Manager 10 The Government Office

(21)

15

2.2.3 ANALYSIS

The next step after data collection is to analyze it. Data analysis falls under two categories: hypothesis generation and hypothesis confirmation (Seaman, 1999). This research seeks for generating hypothesis from the collected data and thus the hypothesis generation analysis method is the best suitable approach. Normally, the data analysis approach is tedious, and time-consuming.

Hypothesis generation further contains two different techniques: constant comparison and cross-case analysis (Seaman, 1999). Both techniques are used during this research for data analysis. This research contains three real-world case studies. Among three case studies, two cases have three interviewee, thereby leading to three data sets. Hence, the constant comparison is well suited in each case study within three participants. The constant comparison applied in each case study, firstly helps to generate a proposition or observation, and secondly provides facts to support or refute developed propositions.

The constant comparison method (Seaman, 1999) was originally developed by Glaser and Strauss. The analysis process begins with coding the collected data. Coding contains codes/sub-codes that are prepared by the researcher and during the analysis codes/sub-codes/sub-codes/sub-codes is connected to the collected data. It is then followed by grouping codes and generates patterns and trends. The final step is to write a proposition or an observation from coded data. The process is repeatedly carried out once new data has been collected. This provides an opportunity to review codes/sub-codes, bring new insights in researcher’s mind, and help to make clear patterns and trends. This repeated process of coding converts observation to a proposition if any or helps to support the defined proposition or refute it (Seaman, 1999).

As there are three case studies in this research, cross-case analysis best suits to find out similarities and differences among the cases. Firstly, two case studies are compared to determine similarities and differences. Secondly, conclusions from these two cases are li sted in the form of propositions. Finally, the third case study is examined to determine whether the third inspection support or refute the proposition formulated from the first two case studies. The cross-case analysis was first developed by (Eisenhardt, 1989). There are three useful strategies for cross-case analysis (Eisenhardt, 1989):

 The cases can be partitioned into two groups based on some attributes(e.g., number of people involved, type of product, etc.) and then examined to see what similarities hold within each group

 Compare pairs of cases to determine variations and similarities

(22)

16

2.1.3.1 TRANSCRIBING

The first step of the analysis is transcribing the recorded conversation into text. The text consists of questions from interview protocol and answers provided by the participants. For easiness, the text is first written down using Microsoft Word and is later copied to NVivo101- an instrumentation tool to facilitate the interview analysis process.

New insights can be generated during transcribing, thus it has been recommended that the researcher himself transcribe all the recordings (Runeson & Höst, 2008). As the transcribing process continues, the researcher gets an eagle view of analysis and supports the defined hypothesis. After the first case study interview, transcribing is immediately started, followed by second case study interview and its transcribing and same goes for the third case study.

Figure 5: Data transcribing using NVivo10

The final transcripts of each case study can be found in Appendix D.

2.1.3.2 CODING

The next step after transcribing is coding. Coding is an interpretive process where words, sentences, or even paragraphs are broken down and assigned to a code that can represent theme, area or construct, etc. Qualitative data can be coded into two ways: preformed codes and postformed codes (Seaman, 1999). Preformed codes are generated when the research objective is very clear and codes can be constructed before data collection begins. The codes are generated from the goals of the study, the research question, and predefined variables of

1

(23)

17 interest. Postformed codes, in other hand, are created during the coding process where the research objectives are still open and unfocused. This research implements postformed coding since the research objective is open and unfocused.

(Corbin & Strauss, 1990) mentions three basic types of coding: open, axial and selective. Open coding helps to understand and generates new insights through standard ways of thinking or interpreting phenomena reflected in the data (Corbin & Strauss, 1990). The generated codes/sub-codes are compared with others to identify similarities and differences. A single code can contain multiple pieces of text and represents certain theme, area, construct, etc.

Axial coding helps to understand relationships among the open codes. All hypothetical relationships created deductively during axial coding must be considered provisional until verified repeatedly against incoming data (Corbin & Strauss, 1990). The logically combined codes then represent concept and are more abstract that code itself.

In selective coding, common concepts are combined together to form a core category and categories that further explications are filled-in with descriptive details (Corbin & Strauss, 1990). The central phenomenon of the study is a core category. A core category is identified when researcher is able to answer the questions such as: “What is the main analytic idea presented in this research?, If my findings are to be conceptualized in a few sentences, what do I say?” etc (Corbin & Strauss, 1990). In this research, an identified core category should be able to support the hypothesis. Figure 6 shows a visual representation of (Corbin & Strauss, 1990) coding types.

Code Code Code Code Code Code Code Code

2.1.3.3 MEMOING

During the analysis process, new thoughts, categories, and propositions evolve regularly. It is thus required to keep a good track and a note of such information before they get lost when the

(25)

19 researcher goes to another topic. This process of keeping comments to the code is known as memoing. Memoing is the vehicle for the researcher to articulate the findings (Seaman, 1999). It can be in the form of a remainder list or simply bulleted list but should express the idea clearly, which later helps the researcher to come back to the original idea when jumping from topic to topic.

Proposition is actually developed through memoing. It can be in the form of phrases where similar concepts are grouped together, patterns are identified, and differences are noted. The phrases are iteratively analyzed as case study analysis runs in parallel. Finally, this provides a body of knowledge that will be used to finalize the answers to the research question.

2.1.3.4 SATURATION STAGE

The case study analysis process is stopped when no new or valuable information is generated. The researcher then stops searching for new case studies or participants. The saturation stage provides enough information to a draw a conclusion for the research.

2.2.4 REPORTING

A linear-analytical reporting structure will be used as the most widely accepted format in academic (Runeson & Höst, 2008).

2.2.5 RESEARCH VALIDITY

In case of qualitative case study research, trustworthiness of data and its analysis are very important. Hence, assessing the validity of qualitative research is a challenging task. The quality of case studies based on exploratory research should be judged on the basis of four aspects of validity: construct validity, internal validity, external validity and reliability (Yin, 2003).

 Construct validity

Construct validity is defined as “establishing correct operational measures for the concepts being studied” (Yin, 2003). The correct operation measures include interview questions, the terminology used, etc. It is a clear threat to this research as maintaining consistent terminologies and their definitions throughout multiple and diverse case companies are a challenge. To minimize this threat, multiple sources of evidence are created by conducting an interview at three case companies. Furthermore, at two case companies, the research includes three interviewees. Besides using multiple sources, the same interview protocol is provided that includes a brief introduction to the research, interview questions, and explanation of the terminology used. Hence, this further establishes a chain of evidence.

 Internal validity

Internal validity is defined as “establishing a causal relationship, whereby certain conditions are shown to lead to other conditions, as distinguished from spurious relationships” (Yin, 2003). The casual relationships between events are identified by

(26)

20 breaking the empirical evidence into smaller chunks known as codes. In addition to codes, memos are created during analysis that further helped to minimize the threat.

 External validity

External validity is defined as “establishing the domain to which a study’s findings can be generalized” (Yin, 2003). The research does not claim comprehensiveness of the findings; it is rather indicative. It is not possible to generalize the results completely, however, the external validity can be guarded by using three case companies of a different nature. Furthermore, cross-case analysis method to analyze and synthesize the findings arguably increases generalizability.

 Reliability

Reliability is defined as “demonstrating that the operations of a study- such as the data collections procedures-can be repeated, with the same results” (Yin, 2003). To ensure reliability, the case study protocol and data collection protocol are properly documented in case study database. The case study database consists of anonymized interview transcripts, interview protocols, and cross-case analysis reports. Thus, the case study database enables reliability of the research.

Aspects of validity Guard validity Research phase Construct validity Use multiple sources of evidence

Establish chain of evidence

Data collection Data collection

Internal validity Data analysis uses coding approach Data analysis

External validity Three different case companies Research design

Reliability Case study/data collection protocol created and updated

Data collection

Table 2: Case study research four validities (Yin, 2003)

 Hindsight bias

One of the problems encountered in this researcher is hindsight bias. Hindsight bias occurs when people believe that an event is more predictable after it becomes known than it was before it became known (Roese & Vohs, 2012). The hindsight bias and outcome have an opposite effect. That means a positive outcome will result to decreased hindsight bias and vice-versa. However, hindsight bias can be eradicated after a successful implementation of debiasing strategies.

(27)

21

3. LITERATURE STUDY

The chapter discusses academic literature related to this research. The main objective of this chapter is to understand legacy systems, legacy modernization strategies and to answer sub-research question “What were the benefits expected by an enterprise after a successful

modernization?”. The chapter is further divided into 3 sub-sections. The first section covers

understanding of legacy systems and its definitions. The second section discusses legacy modernization strategies together with their strengths and weaknesses. Finally, the third section explores claimed benefits of modernization from an academic perspective.

3.1 LEGACY SYSTEMS

There are several definitions of a legacy system from various authors. One of the early definitions is “large software systems that we don’t know how to cope with but that are vital to our organization” (Bennett, 1995). Both academic and industry communities acknowledge the importance of legacy system as “the backbone of an organization’s information flow and the main vehicle for consolidating business information” (Bisbal et al., 1999). Sneed defines a legacy system as “jobs, transactions, programs, modules or procedures within existing application systems which are more than five years old” (Harry M Sneed, 2000). The legacy system is a business critical system. In case of failure of such system, the organization suffers heavy loss.

Features such as monolithic architecture, outdated and obsolete technology (assembly or early version of third generation languages such as Fortran-66, Cobol.), high maintenance cost, increased application backlog, size of programs (hundreds of thousands of line of source code or more) are useful symptoms to identify legacy system (Bennett, 1995). Desipte being vital to any organizations, legacy system significantly resists modification and evolution (Bisbal et al., 1999).

Legacy systems have also been observed through various viewpoints namely developmental, operational, organizational and strategic (Alderson & Shah, 1999). A clear separation of concerns has been identified from those viewpoints. The developmental viewpoint considers development and maintenance of a legacy system. The operational viewpoint supports the business by operating its systems. The organizational viewpoint relates defining and operating the business processes in support of business strategy. The strategic viewpoint is concerned with financial cost and benefits of running the organization.

3.2 LEGACY MODERNIZATION STRATEGIES

There are several modernization strategies proposed in academia such as wrapping, redevelopment, migration, and replacement (Almonaies et al., 2010; Bisbal et al., 1999), (Comella-Dorda, Wallnau, Seacord, & Robert, 2000), (Ali & Abdelhak-Djamel, 2012). All these strategies have their own strengths and weakness. Wrapping is defined as a way to provide

(28)

22 a new interface to a component that makes it easily accessible by other software components. A well-known example of wrapping is a service oriented architecture (SOA) interface that it provides to a legacy system. Organization adopts wrapping if the legacy system is too expensive to re-write, is reusable, is comparatively small, and is a cost-effective solution. However, wrapping does not solve well-known problems of a legacy system such as maintenance and upgrading.

Redevelopment, also known as Big Bang or Cold Turkey, redevelops the legacy information system from scratch using a new hardware platform and modern architecture, tools, and database (Bisbal et al., 1999). It is an approach that involves reverse engineering, restructuring, redesigning, and re-implementing. The major problem with redevelopments is its high cost and risky implementation.

Migration is a technique that takes a legacy system to a more flexible environment while retaining its original system’s data and functionality (Bisbal et al., 1999). The strategy can incorporate both wrapping and redevelopment. Migration is a complex process while it has better offerings such as greater flexibility, easier system understanding, easier maintenance, and reduced cost.

Replacement involves replacing a legacy system with an off-the-shelf package or a complete rewrite of a legacy system from scratch (Almonaies et al., 2010). Replacement strategy is adopted by an organization when business process rules are very clear and easy to implement or if difficult or obsolete technology has been used to maintain legacy system. An organization may choose replacement strategy if wrapping, migration, and redevelopment will incur more cost that cannot be justified (Almonaies et al., 2010). The main risk with replacement strategy is the maintenance of a new system will not be familiar as a legacy system and lack of guarantee that the new system will function similar to the legacy.

Implementing any of the above four mentioned strategies could possess several risks to an organization. Therefore, a proper risk assessment needs to be carried out before legacy modernization. The discussed risk assessment in literature includes performance loss during migration, unanticipated architectural mismatch, the testing bottleneck and lack of test data, falling to attain quality goals as they are poorly defined and finally user program rejection (H.M. Sneed, 1999). Figure 8 represents how migration strategies are compared with respect to cost and reuse of the existing application. There are variety of factors that play roles while choosing a proper strategy such as available budget, time constraint, resources, etc.

(29)

23

Wrapping Migration

Replacement Redevelopment

Figure 8: Modernization strategies compared with respect to cost and reuse

3.3 LEGACY MODERNIZATION BENEFITS

Several researchers in academia have discussed claimed benefits of software modernization. To establish a theoretical background, this research documents claimed benefits from academia using a systematic literature study. As discussed in Chapter 2, Section 2.1, the suitable systematic literature study for this research is backward snowballing.

After a careful investigation, a good set of papers has been chosen as a initial set of papers namely (Almonaies et al., 2010; Khadka et al., 2014; Nasr et al., 2013; Razavian & Lago, 2011). A research work by Nasr et al. (2013) mentions that research in the real-world situation needs to be done to understand if modernization delivers what it has promised. Based on the mentioned good set of paper’s reference list, other new papers have been identified and a list is created making sure to have no duplicate papers.

After identifying a new paper, its abstract is read. If the abstract is clear enough for inclusion then the paper is included as an added good set of papers, else the paper is then thoroughly read to make a final decision for exclusion or inclusion. The process has been iterated until some level of saturation has been reached. In the research, 26 papers are chosen to reach a saturation stage. The documented table of claimed benefits can be found in Appendix A. During the literature study, definition of the claimed benefits has been proposed as mentioned in Appendix B. Later, common phrases that fall under same definition and have the same meaning are grouped together in claimed benefits.

Claimed benefits from academia are further classified into technological and organizational benefits. The highly cited top five technological claimed benefits from academia are maintainability, re-usability, agility, flexibility, performance, availability, interoperability, and system correctness as mentioned in Table 3. Maintainability is claimed in ten academic papers, re-usability, agility, and flexibility in eight academic papers, performance, and availability in seven academic papers and finally interoperability in six academic papers.

Reuse e

(30)

24 Similarly, the top five organizational claimed benefits are cost reduction, business strategy to remain competitive, lack of knowledge/expertise/documentation, faster time to market, business efficiency, and taking advantage of newer technology as seen in Table 4. It has further been observed that cost reduction is claimed in seventeen academic papers, business strategy to remain competitive in ten academic papers, lack of knowledge/expertise/documentation in six academic papers, faster time to market and business efficiency in three academic papers and finally taking advantage of newer technology in two academic papers.

Nr Claimed benefits Academic papers (first column

of above table) Common phrases

1 Increased maintainability 1,8,10,11,12,13,14,18,22,25

Improve turnaround time in resolving defects, Brittle (easily broken down when modified for other purpose), Reduced complexity of current infrastructure, Possible to reorganize database and removing operating restrictions, Support moving to a distributed and more efficient environment, Reduce complexity hidden in a product

2 Re-usability 1,2,9,10,11,19,21,23,24

Reuse of shared business services, System usability, Clean interface to integrate with other systems, Improve business communication, Advanced flexible user interface

3 Increased agility 1,2,3,9,10,16,21,24 Business agility, Better respond to rapidly changing technologies, Extendibility

4 Increased flexibility 1,2,3,4,8,15,23,26 5 Improved performance

(Time behavior) 9,16,14,18,19,21,23

Resolve performance memory limitations, Improve system performance, System efficiency

6 Reduce prone to failure

(Availability) 4,7,10,11,18,22,24 Reliability, Robustness 7 Interoperability 5,7,10,11,17

Open systems (standard interfaces or architectures) , Transparent multi-vendor interoperability, Improve business communication

8 Functional correctness 19,22,24 Correct inappropriate functionality 9 Scalability 7,10

10 Better understandability 13,21 System understanding 11 Use of proper developmental methodology with relevant standards 14

Loose coupling, Abstraction of underlying business logic, Agility, Re-usability, Autonomy, Statelessness,

Discoverability (it’s a programming standard and a framework achievement of SOA)

Table 3: Claimed benefits from academia in technological perspective

Nr Claimed benefits Academic papers (first column

of above table) Common phrases 1 Cost reduction 1,2,4,5,6,8,9,10,11,15,18,19,20,2

1,22,23,25

Maintenance cost, Reduce operation and maintenance cost (integration cost as well / cost of ownership), reduce communication cost, Gain control over maintenance cost,

2 Business strategy to

remain competitive 5,7,10,12,16,18,19,22,23,26

Reduce business risk, Keep pace with the changing demands of business, Change in business process, Handle accelerating changes in market demands and opportunities, Better respond to marketplace needs, Inflexible (Difficult to adapt to changing business needs), Change or re-engineering of business processes, Escalating expectations of customers for new enterprise standards, new products and system features. 3 Lack of knowledge /

expert / documentation 4,9,10,21,22,26 Lack of knowledge / skills

(31)

25 5 Business efficiency 1,6,7 Added value, Meet business objectives

6 To take advantage of

new technology 18,22 Have latest technology 7 Reduce risk of lack of

supplier 4, 19 Ending of technological support 8

Handle accelerating intra-organizational changes

4, 5 Create business opportunities via mergers and acquisition 9 Improve

development-staff-productivity 14

(32)

26

4. THE THREE INDUSTRY CASES

This chapter describes three industrial case studies where legacy system modernization had been done. The two case companies belong to the private sector, while the third case is a Government office.

This chapter starts with first case followed by its three sub-chapters: Case description, Benefits (pre- and post-modernization), and other observations. Second and third cases follow the same study reporting pattern as the first one. The first sub-chapter, Case description, provides a brief introduction to the case company, including its niche market, IT platform details (before and after modernization), and other general information. The second sub-chapter discusses pre and post benefits of modernization where the benefits that look similar are logically grouped and discussed. All the benefits discussed in this sub-chapter are expected benefits, which are well observed after modernization. The third sub-chapter discusses other observations. Other observations include: unintended benefits- benefits that the case company does not expect to improve after modernization, but do expect no deterioration as well; and expected but remains unobserved after modernization.

In the first case, The Electrical Company, benefits that were expected and well observed after modernization are enhanced usability, product consolidation, increased maintainability, long-term vision and planning, and functional suitability. The other observation includes positive unintended benefits such as reduced operation and maintenance cost, increased transparency and collaboration and increased business productivity while negative unintended observation such as user resistance. Performance is also listed in other observation as the case company di d not expect it to increase but not to deteriorate as well. Forward thinking and stay competitive were expected from modernization, but remains unobserved.

In the second case, The Aviation Company, expected and well-observed benefits are listed as increased flexibility, increased maintainability, enhanced usability, increased functional suitability, increased vendor support and stay competitive. The other observation, such as reduced operation and maintenance cost, increased availability, increased organizational flexibility and transparency, increased business productivity and faster time to market are seen as positive unintended benefits. Objectives such as performance and user resistance are the ones which the case company did not expect to improve after modernization.

The Government Office is the third case company where reduced operation and maintenance cost, phase out legacy technology, increased organizational flexibility, and IT streamlined business process are seen as expected and also observed benefits after modernization. Two negative effects, namely: user resistance and user productivity were observed.