Post-Project Evaluation of the IT Business Case

report

Post-Project Evaluation of the IT Business Case

A post-project evaluation model for business cases of IT projects at Essent Energy

By Peter Schuurman

March 2006

University of Groningen, The Netherlands Faculty of Economics

Supervisors

Prof. Dr. E.W. Berghout

Faculty of Economics, University of Groningen

Dr. E. Harison

Faculty of Economics, University of Groningen

Drs. G.L. Donkersteeg

Manager AQAD, Essent

Thesis post-project evaluation of the IT business case i

Preface

“Rivers know this: there is no hurry. We shall get there some day.” – A.A. Milne (1926)

September 1

last year, a research was initiated to create a method for the post-project evaluation of the IT business case at Essent. Six months and 713 cups of coffee later, this report is the result.

I would like to express my gratitude to Essent, and especially my supervisor Geert Donkersteeg, for offering me this research opportunity. Geert, thank you very much for all your guidance and support during the research.

Furthermore, the support I received from Egon Berghout was of infinite value to the progress and quality of the research and report. His feedback and insights have made this report to what it is.

Also, I would like to thank Frans Tiel, Carel Ilgen, Robin Roodenburg and Jeroen Havenaar for sharing their experience and showing me project management in progress; as well as my friends for their support and feedback.

But without a doubt, my most gratitude goes to my parents and brother, who have supported me no matter what. Thank you.

Peter Schuurman

February 2006

Thesis post-project evaluation of the IT business case

iii

Management summary

In the process of project management Essent uses business cases to justify their IT projects;

however, just like many other companies (Berghout, Nijland, 2002, Gwillim, Dovery, Wieder, 2005, Kumar, 1990) Essent does not explicitly evaluate the business cases after realization . Due to this absence Essent could miss out on lessons that may be learned and used for future benefits. These future benefits include improving the quality of future business cases, improvement of systems development practices, improvements in the effectiveness and productivity, and making IT tangible for managers and users so that they can recognize, if and how, the existing information systems need to be modified. This research is initiated on behalf of Essent’s Energy division to design a method which will fill this gap for their IT projects. The problem statement that will be addressed in this research is formulated as follows:

Develop a method for the post-project evaluation of the information technology business cases at Essent and apply this method on the Power Nomination System project; this method should compare the elements of the business case after implementation of the project with the elements of the business case at the time of initiation.

The problem statement is divided in several sub-questions which cover the different elements of the main question, these are:

1 What is a business case, what elements should it contain for IT and how should these be post-project evaluated according to literature?

2 Considering the theoretical business case, what information do the IT business cases and project management documents at Essent contain?

3 How should Essent conduct post-project evaluations on their IT business cases?

4 Based on the Power Nomination System pilot project, what should Essent considered when using the post-project evaluation method?

The business case

Based on a combination of elements recognized in multiple theories the business case is defined as a

description of the reasons for the IT project and the justification for undertaking the project. This

justification is explicitly not only based on the financial consequences of a project, but also includes

other elements that have to be accounted for; together these elements are the foundation of the

Peter Schuurman

iv

business case. This foundation consists of six elements, being the estimated costs of the project, the expected business benefits and savings, the considered scenarios, a risks analysis, the project planning and the stakeholders of the project.

Post-project evaluation

According to literature, a post-project IT evaluation is the assessment of the value, worth or usefulness of an IT project after project closure (see Appendix A). It is conducted to determine if the expected situation has been obtained. Furthermore the post-project evaluation also considers the future exploitation of the business case. The main advantage of the post-project IT evaluation is the contribution the results can be for individual and organizational learning.

The process of assessing a business case to a post-project evaluation resembles the continuation of the business case evaluations which are conducted during a project. It is the comparison of the initial business case elements with the updated business case elements which is created on the basis of the updated information which comes available in time.

For each of the six elements the theoretical issues to take into consideration, when executing a post- project evaluation (PPE), are determined. These issues include the PPE questions, required information, time frame, roles and possible actions. Combined the elements and the accompanying issues are combined in the theoretical PPE matrix.

The business case at Essent

In order to create a post-project evaluation method for Essent, the theoretical PPE matrix is adjusted to the situation at Essent. For this the results of an examination of thirty-one project initiation documents and thirty accompanying business cases of projects at Essent in parallel to the five business case elements are analyzed.

From this analysis emerged that at Essent, the business case provides information on the costs and benefits elements of the business case. The costs are provided as expenditures in three dimension;

the activities, owners and efforts. Whereas the benefits consider both financial yields and savings as non financial improvements distributed between the classes of risk reduction or avoidance, cost reduction and efficiencies, business opportunities and intangibles. These benefits are provided either by solely their origin, or the origin and a total amount, or the origin, a calculation and an amount.

This business case information is complemented in the project initiation documents with the business

case elements of risk, project planning and stakeholders. Here, information on risks is provided by

management of exception and risk analyses. Project planning information is complementary to the

Thesis post-project evaluation of the IT business case v costs of the project and consists of which deliverables are available when, by whom and at what costs. Subsequently, the stakeholders are described in communication matrices and an overview of the project organization. The whole of this information, together with the economic metrics justifies a project.

Post-project evaluation at Essent

With the overview of the available information in the business cases of Essent being available the theoretical post-project evaluation matrix is tailored to fit the situation at Essent. The PPE matrix is provided with information on possible sources for Essent to gain information on the initial business case. Furthermore, it is determined in what way the post-project information is required at Essent to keep it in line with the initial business cases.

To provide a practicable solution for the implementation of a post-project evaluation process a template is created on the basis of the adjusted matrix. The template contains a general description of its purpose and use, the monitoring and forecasting part of post-project evaluation each handling the five elements of the business case and in conclusion the lessons learned.

Pilot project

In order to test the PPE template and recover issues to be considered for enabling a good post-project evaluation environment a pilot PPE was implemented on the Power Nomination System project. In the pilot three major issues are identified, which are described next.

The main problem in the execution of the post-project evaluation is the lack of clearly stated initial information. The second aspect which came forward in the pilot evaluation is the general difficulty of executing profound benefit and cost tracking, stakeholder and risk management and planning analysis. The management and evaluation of each of these elements is a profession on its own.

Further research on each of these elements could increase the clarity and practicability of their post- project evaluation. Thirdly, the business case is not a one time document, but a technique to continually assess the justification of a project. All elements have to be updated throughout and after the project for management with the business case and to enable valuable post-project evaluation.

Conclusion and recommendations

For Essent the issues which came forward in the pilot imply that in order to create the learning loop

from the business case there is a need to improve the provided information. This will describe what

Peter Schuurman

vi

needs to be evaluated post-project and will increase the usefulness of PPE since information can be provided in the same context. Quick wins can be established by declaring methods of calculation and assigning costs and benefits to responsible owners. Furthermore, to obtain the benefits of post-project evaluation, the process needs to be embedded in the organization enforcing the management of projects with the business case. Not only needs the process of post-project evaluation to be embedded, the assimilation of the lessons learned is of great importance.

The template which is provided in this research could serve as a starting block in the process of

implementing post-project evaluation and the gaining of the accompanying benefits.

Thesis post-project evaluation of the IT business case vii

Table of contents

Preface ...i

Management summary...iii

1 Introduction...1

1.1 Essent...1

1.1.1 General information ...1

1.1.2 Organization...2

1.2 Research approach...3

1.2.1 Problem Definition ...4

1.2.2 Concepts ...6

1.2.3 Sources, measurement and observation methods, and analysis and reporting...7

1.3 Report outline...8

2 Theory on the IT business case...9

2.1 Introduction...9

2.2 Office of Government Commerce – The PRINCE2 business case... 10

2.2.1 Introduction... 10

2.2.2 Development... 11

2.3 Remenyi – The IT investment business case... 12

2.3.1 Introduction... 12

2.3.2 The business outcome... 13

2.3.3 The stakeholders ... 14

2.3.4 Strategic alignment and IT benefit identification... 15

2.3.5 Technology issues... 15

2.3.6 Risk – Project and systems... 16

2.3.7 The IT investment business case as a process... 16

2.4 Schmidt – Building blocks for the business case ... 17

2.4.1 Introduction... 17

2.4.2 Building blocks A: Introduction and overview ... 18

2.4.3 Building blocks B: Assumptions and methods ... 18

2.4.4 Building blocks C: Business impacts ... 18

2.4.5 Building blocks D: Sensitivity and risk analysis... 19

2.4.6 Building blocks E: Conclusions and recommendations... 19

Peter Schuurman

viii

2.4.7 Building the business case ... 19

2.5 Findings on IT business case theories ... 20

2.5.1 Definition... 20

2.5.2 Content ... 21

3 Post-project evaluation...25

3.1 Introduction... 25

3.2 Estimated costs... 26

3.3 Expected business benefits and savings ... 29

3.4 Considered scenarios ... 31

3.5 Risk analysis ... 31

3.6 Project planning ... 34

3.7 Stakeholders... 34

3.8 Findings on IT business case evaluation... 36

4 The business case and post-project evaluation at Essent ...39

4.1 Introduction... 39

4.2 Estimated costs... 40

4.2.1 Business case... 40

4.2.2 Post-project evaluation... 43

4.3 Expected business benefits and savings ... 45

4.3.1 Business case... 45

4.3.2 Post-project evaluation... 47

4.4 Considered scenarios ... 48

4.4.1 Business case... 48

4.4.2 Post-project evaluation... 48

4.5 Risk analysis ... 49

4.5.1 Business case... 49

4.5.2 Post-project evaluation... 50

4.6 Project Planning ... 50

4.6.1 Business case... 50

4.6.2 Post-project evaluation... 51

4.7 Stakeholders... 51

4.7.1 Business case... 51

4.7.2 Post-project evaluation... 52

4.8 Findings on the business case at Essent... 53

4.8.1 Business case... 53

Thesis post-project evaluation of the IT business case ix

4.8.2 Post-project evaluation... 55

5 Pilot project: Power Nomination System...57

5.1 Introduction... 57

5.2 The project... 58

5.3 The post-project evaluation ... 58

5.3.1 Estimated costs ... 58

5.3.2 Expected business benefits and savings... 59

5.3.3 Risk analysis... 60

5.3.4 Stakeholders ... 62

5.3.5 Project planning... 62

5.3.6 Conclusions... 63

5.4 Findings on the pilot project ... 63

6 Conclusions and recommendations...65

References...71

Appendix A: Concepts...75

IT projects ... 75

PRINCE2... 76

Evaluation... 79

Appendix B: Outline of Projects ...83

Appendix C: Conversion Table Projects...95

Appendix D: Template business case post-project evaluation for Essent ...97

Thesis post-project evaluation of the IT business case

1

1 Introduction

In the process of project management Essent uses business cases to justify their IT projects;

however, just like many other companies (Berghout, Nijland, 2002, Gwillim, Dovery, Wieder, 2005, Kumar, 1990) Essent does not explicitly evaluate the business cases after realization. Due to this absence Essent could miss out on lessons that may be learned and used for future benefits. These future benefits include improving the quality of future business cases (Von Zedtwitz, 2003), improvement of systems development practices, improvements in the effectiveness and productivity, and making IT tangible for managers and users so that they can recognize, if and how, the existing information systems need to be modified (Kumar, 1990). This research is initiated on behalf of Essent’s Energy division to design a method which will fill this gap for their IT projects.

In this chapter an overview of Essent in general and the Energy division in particular is provided in paragraph 1.1. This overview is followed in paragraph 1.2 by a description of the research approach, which is based on De Leeuw’s method of research (De Leeuw, 2001). Subsequently, the report outline is given in paragraph 1.3.

1.1 Essent

1.1.1 General information

Essent is a vertically integrated power company which concentrates on the production, transport,

trading, and delivery of gas and electricity to its domestic and business customers. In addition the

Peter Schuurman

2

company offers other products and services, such as heat, the maintenance of central heating and hot water equipment, and cable company products, like cable television and radio, broadband internet access and telephony. Essent also handles waste management and handling. For the production of energy multiple methods are used; for instance coal-fired plants, garbage incineration plants, wind energy and biomass.

The company has its home market in the northern and southern parts of The Netherlands, but is also active in Belgium and Germany. The enterprise is owned by Dutch provinces and other local authorities and obtained a turnover of €7.4 billion in 2004, serving 2.6 million consumers and companies and employing 12.000 people. In The Netherlands it is the largest company on the energy market measured by turnover and the number two cable television company in number of customers.

The ambition of Essent is to be part of the top-5 of North-West European energy companies in the field of customer satisfaction and profitability (Essent, 15-09-2005).

1.1.2 Organization

Until the first of October 2005 the Board of Management led seven divisions; these are Holding, Network, Retail, Energy, Waste, Kabelcom and International. Each division had its own business units (BU) and service units (SU). After that date the divisions were transformed into business units and the division layer was removed between the business and service units of the Energy division and the Board of Management.

The service units of the Energy division were respectively Human Resource Management and Finance

& Business Services and the business units were Business Development, Generation, Projects, and Energy Management Group (EMG). Until the first of January 2005 the business unit Business to Business was also part of Essent Energy, but this BU was then transferred to the Retail division.

Since this research makes use of information from the past, the former Energy layer is shown in the

organizational chart in Figure 1. The BU Business to Business is illustrated in shades due to its earlier

removal from the Energy layer.

Thesis post-project evaluation of the IT business case 3

Figure 1: The Organizational Chart of Essent Energy (Essent, 16-09-2005)

The exact location of this research is the Essent Energy Information Management (EEIM) sub-unit of Finance & Business Services. The EEIM department is responsible for managing and giving advice about processes that aim at the creation of required information for divisions and departments. This includes the guiding and coaching of the employees that work in these processes. Its mission is to be the logical choice for business and employees concerning information management and ICT solutions.

IM’s vision is to be an integral part of the business and be a key-player in realizing business goals. Its strategy is to live up to the expectations, have business knowledge, realize ICT solutions, be efficient and effective and operate cost-conscious on a basis of trustworthy and stable IM-organization, information systems and infrastructure.

The Power Nomination System pilot project which will be evaluated during this research was carried out by the EMG department. They are the intermediary between production and retail and are responsible for disposing primary products, like gas and electricity, in the market against the best price and the right conditions.

1.2 Research approach

For the research approach of this study De Leeuw’s method of research is used (De Leeuw, 2001).

This method was chosen because it has its origins in the field of management and organization.

Therefore it is presumed that this method provides a good foundation for this research.

The method uses five components to describe a research approach; these are respectively the problem definition, concepts, sources, measure and observation methods and analysis methods. The first two components are the core of the method, where the remaining three mainly state the part concerning data. All components will be briefly explained.

Essent Energy

BD Generation Projects EMG B2B HRM F&B

Peter Schuurman

4

The problem definition describes what you want to know and why, while also stating the relevance of the research. Secondly, the concepts express the theoretical framework in which the research will take place. Third, the sources describe where data will be collected. Next, how these data will be gathered is explained in the measure and observation methods. Last, the analysis methods describe how the gathered data will be analysed and how the results will be used. The method and its components are graphically represented in Figure 2.

Figure 2: The research method of De Leeuw (De Leeuw, 2001)

In the next section, the research method is applied to this research. The approach is divided in the method’s three main parts; the problem definition is described in paragraph 1.2.1, paragraph 1.2.2 consists of the theoretical concepts which will be used, and the approach to the research, the data approach, is defined in paragraph 1.2.3.

1.2.1 Problem Definition

1.2.1.1 Research Objective

The IT projects at Essent are managed in accordance with the PRINCE2 methodology. This is a project management method based on best practices which covers the managing, controlling and organizing of a project. In this context, a project is a temporary organization, created to deliver one or more business products in accordance with a specified business case (OGC, 2002). This business case is established for the first time at the work out of the project idea and evolves to a detailed estimation

Problem Definition

Concepts

Sources Analysis Methods

Measure and Observation Methods

Thesis post-project evaluation of the IT business case

5 in the initiation phase of the project. During the execution of the project the business case is updated and evaluated at every milestone until the project is finalised. It is likely that in many projects not all the issues included in the business case are fully realized at that moment. An additional evaluation of the business case should take place sometime after the closure of the project. This evaluation could provide the organization with information about the actual realized situation after the project. This information can then be compared with the initial estimates and the evolved business case for evaluation purposes. With the results of this comparison a good opportunity arises for improving the decision making process for IT investments by increasing the accuracy of the business cases that will be created in the future. In the case of Essent’s IT projects no such post-project evaluation takes place at the moment. Therefore Essent misses out on these possible benefits.

In connection with this issue the management of the Information Management (IM) business unit at Essent requires a method for the post-project evaluation of their business cases. This research is initiated to create a method which gives an insight in the actual realization of the business cases of the IT projects at Essent. Such insights could then be used to increase the quality of IT project management.

1.2.1.2 Problem Statement

The problem statement that will be solved in this research is formulated as follows:

Develop a method for the post-project evaluation of the information technology business cases at Essent and apply this method on the Power Nomination System project; this method should compare the elements of the business case after implementation of the project with the elements of the business case at the time of initiation.

In addition to the method, the research can provides recommendations on how Essent can improve their design of the business case to improve the practicability of the method.

The problem statement is divided in several sub-questions which cover the different parts of the main question. By answering these sub-questions the solution to the main statement is constructed. The sub-questions are defined as follows:

1 What is a business case, what elements should it contain for IT and how should these be post-project evaluated according to literature?

2 Considering the theoretical business case, what information do the IT business cases and project management documents at Essent contain?

3 How should Essent conduct post-project evaluations on their IT business cases?

Peter Schuurman

6

4 Based on the Power Nomination System pilot project, what should Essent considered when using the post-project evaluation method?

1.2.1.3 Constraints

Two types of constraints are identified, successively the constraints having relation with the research method that will be used, the so called logical constraints, and constraints in respect to the feasibility of the project, the feasibility constraints.

For this research, three logical constraints are identified. First, full insights in the IT business cases at Essent are expected in order to have a sound foundation for the research. Second, because the main subject of this research is the post-project evaluation of the business cases and not the writing of a complete and well estimated business case, the content of the IT business cases at Essent is assumed right. Third, full cooperation and sincerity of the, new and former, people involved is necessary when applying the model at the Power Nomination System pilot project because only full and correct information will lead to a good post-project evaluation which provides useful information for the developed method.

The feasibility constraints are twofold. Firstly, the project will take place between the first of September 2005 and the 31

of January 2006. Secondly, complete access to all necessary literature for the desk research is taken for granted in order to execute a solid study.

1.2.2 Concepts

The main concepts of this research are project management, IT projects, the business case and post- project evaluation (PPE). The IT project is controlled by means of project management. In this management the business case justifies the project and serves as a guard of this justification during the project execution as well as during the exploitation of the project outcome, i.e. post-project. To fulfil this task, the business case needs to be evaluated regularly.

For the concept of project management the PRINCE2 methodology has been adopted, since this is the

project management method currently being used by Essent. An overview of PRINCE2 is included in

Appendix A. Furthermore, on the basis of a literature review, also provided in Appendix A, the

concepts IT project and post-project evaluation are defined as follows:

Thesis post-project evaluation of the IT business case

7 IT project

An IT project is a temporary organization that is created for the purpose of delivering one or more business products with the use of IT facilities according to a specified business case (OGC, 2002).

Post-project evaluation

A post-project evaluation is the assessment of the value, worth or usefulness of a project after project closure. It is conducted to determine if the expected situation has been obtained and therefore compares the previous conditions with the actual results achieved by the project.

Furthermore, it determines what is expected in future exploitation and how this will be achieved. The main advantage of the post-project IT evaluation is the input the results can be for individual and organizational learning (see Appendix A).

The concept of post-project evaluation is illustrated in Figure 3. The remaining concept, business case, will be addressed in the next chapter.

Figure 3: The monitoring and forecasting parts of PPE in time

1.2.3 Sources, measurement and observation methods, and analysis and reporting

Table 1 offers an overview of the approach that will be used for this research and the methods used in the research.

Project

PPE¹ PPE²

Closure

Time

Monitor Forecast

PPEⁿ Initiation

Project outcome life

Peter Schuurman

8

Research methods

Research question Research method Additional information The IT business case Desk research

Post-project evaluation of the IT

business case Desk research

Initial sources: Library of Economics / Management &

Organization, Business Source Premier, Econlit. PiCarta and the references included in the found literature.

Initial keywords: project management, PRINCE2, justification, initiation, realization, exploitation, business case, post-project, ex-post, review, evaluation, project outcome, control, post-implementation.

The IT business case at Essent Field research The business cases of a number of approved projects at Essent will be compared to the results of the preceding desk research to determine their quality.

Post-project evaluation at Essent Field research & case

study This part will be executed on the basis of the business case research and the literature research on post-project evaluations. A method will be developed for practice post- project evaluations of business cases of IT projects at Essent. Afterwards the model will be tested, for which the Power Nomination System project will serve as a pilot.

Table 1: The research methods per sub-question

1.3 Report outline

The report will follow the structure of the sub-questions. Insights on the theoretical business case are given in Chapter 2, followed by the desk research about post-project evaluation of the IT business case in Chapter 3.

The quality assessment of the IT business cases at Essent and the practice side of post-project

evaluations are then made in Chapter 4. The model of how Essent should assess post-project

evaluations of business cases is defined in this chapter and implemented on the Power Nomination

System pilot in Chapter 5. The report ends with the conclusions and recommendations in Chapter 6.

Thesis post-project evaluation of the IT business case

9

2 Theory on the IT business case

2.1 Introduction

The target of this chapter is to answer what a business case is and what elements it should contain.

For this purpose first three different theories of what a business case should look like and how it should be developed are discussed. This is done because these theories provide different conception on the business case. These points of view and the multiple elements identified by the theories contribute to an accurate insight into the business case.

These theories are respectively, the PRINCE2 business case by the Office of Government Commerce (OGC, 2002) which is discussed in paragraph 2.2, Dan Remenyi’s IT investment business case (Remenyi, 1999) in 2.3, and the building blocks for business cases of Marty Schmidt of Solution Matrix Ltd. (Schmidt, 2002) considered in paragraph 2.4.

The PRINCE2 business case is included since it represents the business case as is expected to be used in project management in general, as well as project management at Essent. The other two theories were chosen because these were the only two theories that were found in the desk research, as described in paragraph 1.2.3, having the business case as central point of attention. The first two theories have a theoretical view, whereas the third has a more practical one. The findings on the theoretical IT business case, specifying the business case and its elements are provided in paragraph 2.5.

It should be noted that all theories share one point of attention in regard to the measurable; it is

important that the benefits are made tangible as much as possible, since this tangibility contributes to

Peter Schuurman

10

the evaluation of the business cases. If an issue is intangible it is worth the try to transform it to a tangible one in order to increase measurability and decrease subjective elements. This issue comes back in the post-project evaluation analysis of the business case elements in the next chapter.

Although there is little specific literature about the business case post-project evaluation, there is a lot of information available which is closely related to this research; for instance project management (Hof, 2004, Cadle, Yeates, 2004), managing software processes (Harmon, 2003), measurement of (software) products (Van Solingen, 2000, Humphrey, 1989, Fenton, Pfleeger, 1996, Gentleman, 1994) and processes (Van Solingen, 2000, Hatton, 1995, Basili, Weiss, 1984, Lynch, Cross, 1991) and organizational learning (Agryris, Schön, 1978, Huber, 1991).

2.2 Office of Government Commerce – The PRINCE2 business case

2.2.1 Introduction

The business case is the central point of attention when managing according to PRINCE2. In PRINCE2, the business case is developed continuously during the lifecycle of a project. Multiple times during its lifecycle the business case is reassessed to monitor the validation of the project’s justification. Since the business case has such a large influence on the project it is one of the most important components for the project. It drives the decision-making process and is used to steer the project’s progress to the business objectives that are defined within the business case. In its most simple form the business case is defined as the justification of a project by stating its financial consequences; that is:

A description of the reasons for the project and the justification for undertaking the project, based on the estimated costs of the project, the risks and the expected business benefits and savings.

Its scope always is the total influence a project has on the business. To contain sufficient information

for effective management of the project the business case should contain at least six points of

information; these are the reasons why, the various options which can be used to create the desired

project outcome, the benefits, the risk issues, the cost and timescale, and the investment appraisal

(OGC, 2002). To achieve agreement on the content of these points stakeholder involvement is

essential. The six elements are described next. Note that PRINCE2 is a general management method

which can be used for all kinds of projects and does not provide a lead on what the elements should

contain, such as specific metrics that can be used, therefore the description of the business case

content stays very broad.

Thesis post-project evaluation of the IT business case 11 The first element is the reasons why the project outcome is needed. This information should already be clear in the project mandate.

Second, a brief description of the various options that have been considered to reach the required outcome should be noted. This includes a motive for the chosen option. Maintaining the status quo should always be regarded as an alternative.

Third, the benefits a project claims to achieve have to be comprised; each has to be described in clear measurable terms. Besides these measurable benefits the current situation should be described in the same measurable terms to be able to apply a post-project evaluation. Of course the benefits can also be the (negative) effects of not running the project; this describes what will happen if the project is not carried out.

The fourth point is an overview of the key risk issues, their effects in case of occurrence, and the chance of occurrence. Details of how these risks will be managed are contained in the Risk Log.

The cost and timescale estimation should be inserted next. This information should already be present in the project plan, which describes the major products, timescale and costs for the project. If the project plan is not yet completed, it may be necessary to outline the project’s costs and timescales in the business case and refine them when the project plan is completed.

The last element is the investment appraisal. This illustrates the balance between the development, operational, maintenance and support costs against the financial value of the benefits over a period of time. The length of this period depends on the content of each project itself. In order to see the project in the right context, the information of the investment appraisal has to be compared with the

‘do-nothing’ variant. This variant shows the costs and benefits for the current situation if no project is executed at all.

2.2.2 Development

In PRINCE2 the executive is eventually owner of the business case, but the development will often be

done by the project manager, or a small team under his supervision, on the basis of information

provided by the business. The information provided by the project mandate is the foundation for the

first business case. During initiation, the business case is further crystallized from an outline business

case to an enhanced and approved version using the updated information. While executing the project

the business case is evaluated continuously to confirm that the project remains on track and to check

that the business case remains valid within the business context. The business case requires change

control and configuration management to ensure any changes in the project’s environment. At the

Peter Schuurman

12

closing a project process a post-project evaluation plan (PPEP) is submitted to the PCD. Figure 4 gives an overview of the PRINCE2 business case development path (OGC, 2002).

Figure 4: Business case development path in PRINCE2

2.3 Remenyi – The IT investment business case

2.3.1 Introduction

The second theory that will be addressed is the IT investment business case of Remenyi. In this theory Remenyi defines the business case as (Remenyi, 1999):

A justification for pursuing a course of action in an organizational context to meet stated organizational objectives or goals.

Remenyi identifies three main issues to which a well produced business case can contribute during the entire lifecycle of the project which is the subject of the business case, these are

•

Corporate learning of what really is expected from the IT investment and how to manage the project better in order to achieve its objectives,

•

Acquiring commitment of the principal stakeholders and improving stakeholder management,

•

Understanding the risks involved in making the project deliver the anticipated benefits.

Project

Mandate Project

Brief

Business Case

PPEP End Stage

Report Project

Issues

Reasons for the Project Outline BC

Updated BC Reviewed BC

Expected benefits

PID

Enhanced & approved BC

Thesis post-project evaluation of the IT business case 13 Next, he states that the business case should be viewed more as a corporate culture process than a strictly technical event. Culture determines the length and format of the IT business case, who initiates the business case and how it will be presented and processed in the organization; therefore house rules for the IT investment business case have to be established first. Furthermore corporate culture will determine what issues are important to the organization and which issues are not.

Remenyi identifies five elements which need to be considered to produce a complete business case.

These elements are the business outcome, stakeholder management, strategic alignment, technology issues, and project and system risks. The connection between the elements with the risk factors at the centre of the project is visible in Figure 5 (Remenyi, 1999).

Figure 5: An overview of the IT business case (Remenyi, 1999)

2.3.2 The business outcome

The outcome of an IT investment is defined as the desired effect of an intervention or change to a business process or practice. The business outcome is a business result, which has a measurable impact on the performance of the organization and it directly follows from the business problem. A good outcome statement creates the possibility for the maximisation of IT investment effectiveness.

To make sure that there is understanding of how the IT investment will be implemented and how the outcome and associated benefits will be generated it is essential to develop the outcome thoroughly.

Remenyi distinguishes three levels of detail; the macro model, meso model, and micro model. The connection is shown in Figure 6.

Business Outcome Stakeholders

Risk – Project and systems

Technology Strategic Alignment

Peter Schuurman

14

Figure 6: The three steps in the business outcome (Remenyi, 1999)

The macro level is the highest level, it should contain the problem statement, what will be done by whom, when to take advantage of it, and what the expected outcome or business result will be. The meso model is a half-way stage between the high level overview and the detailed financial statement from the micro model. An essential feature of a meso model is that all the possible benefits are listed and that each benefit has a specific metric matched with it. The micro model is presented as a detailed financial representation of how the project will be implemented, in which the set-up costs, the on-going cost, the on-going benefits and the net benefits are specifically stated. The micro model includes the investment statistics, such as the return on investment (ROI), net present value (NPV), and internal rate of return (IRR). The micro model takes the issues from the meso model and describes them in quantified financial terms.

2.3.3 The stakeholders

To have a successful project, the principle stakeholders have to be satisfied. A stakeholder is defined as any individual or group with an involvement in the project interested in improving the business processes or practices being supported by the proposed IT investment. Stakeholder relationships have a large influence on the chance on a successful project. If for any reason stakeholders are in conflict, the likelihood of a successful project notably decreases. Three major categories of stakeholders are identified; the users/owners, the IT professionals and the finance and administration staff. According to Remenyi, these stakeholders are concerned with the IT business case in at least two ways, namely at the proposal of the project, and in the stage of project approval.

Macro model

Meso model

Micro model

Thesis post-project evaluation of the IT business case 15

2.3.4 Strategic alignment and IT benefit identification

An IT investment should support the corporate strategy, if this is not the case the possibility can arise that the investment is pulling the organization in a different way to the overall corporate strategy. This support should be found in the business case of the investment. To make this strategic alignment between the corporate strategy and the IT strategy relatively easy to see Remenyi uses the following functional definition of strategy (Remenyi, 1999):

Corporate strategy is how an organization finds, gets and keeps it clients.

Before possible alignment between the two strategies can be identified first the actual applications and the benefits the IT investment is expected to deliver; seven possible applications and benefits are distinct:

•

Group productivity tools,

•

Individual or personal productivity tools,

•

Provide specific information for decision making (informate the organization),

•

Reduce time and space in business processes,

•

Create a corporate memory,

•

Bind the organization closely with clients and suppliers,

•

Introduce discontinuities by business process improvement.

After both the corporate strategy and the applications and benefits the project is expected to deliver have been recognized the alignment can be measured. An IT investment is aligned with the corporate strategy if the processes or practices supported by the investment directly assist or contribute to the organization in achieving its strategy.

2.3.5 Technology issues

This element checks the viability of the proposal via a technology report. Three terms are central in this report; technobility, doability and achievability. The report provides much of the detail required for the actual management of the tasks and the deliverables of the IT project. Especially the next six issues have to be considered:

•

IT infrastructure,

•

Hardware,

•

Software,

Peter Schuurman

16

•

People,

•

Budgets,

•

Timing of the project.

2.3.6 Risk – Project and systems

A review of the key risk issues should be included in the business case. Two central categories have to be listed. First, the risks associated with the implementation process that might cause the project to fail, and second, the risks of the system failing to deliver the type of improvements to the business processes and practices that had originally been desired.

2.3.7 The IT investment business case as a process

The development of the IT investment business case can be viewed as the business process which is shown in Figure 7.

Three key activity groups are identified in this process:

1 The identification of the business process or practice opportunity,

2 The identification of the stakeholders who will work on the development of the business case, 3 The production of the business case itself which requires reiteration.

Once the business case is developed a decision has to be made, the business case has to be evaluated. This is not a one time event. Remenyi distincts three phases in a successful IT project management programme; these are setting the course, formative evaluation and moving forward.

These phases will be explained next.

The setting the course phase involves the development of the IT investment business case using the

elements of the business case; that is business outcomes, stakeholders, strategic alignment,

technology and risk. The second phase, formative evaluation, involves closely assessing the progress

of the project. The last phase, moving forward, provides a feedback loop which should be available

not only during development, but also throughout the entire life of the project (Remenyi, 1999).

Thesis post-project evaluation of the IT business case 17 Figure 7: The IT business case as a process (Remenyi, 1999)

2.4 Schmidt – Building blocks for the business case

2.4.1 Introduction

In the last theory that will be described Marty Schmidt of Solution Matrix Ltd. defines the business case as (Schmidt, 2002):

A tool that supports planning and decision-making; they are written to answer the question of what the likely financial and other business consequences are if an action or decision is taken.

This includes decisions about whether to buy, which vendor to choose, and when to implement. He states that a single outline or template business case for all projects can not be created. However building blocks for a good business case with a logical structure can be identified. The building blocks appear in five general categories; introduction and overview, assumptions and methods, business impacts, sensitivity and risk analysis and conclusions and recommendations. In every business case all five categories have to be represented, how this is done depends on each project (Schmidt, 2003).

Yes Identification of process

or practice opportunity

Identification of primary stakeholders Business Outcome

Stakeholder Planning Strategic Alignment

Technology Issues Risk – System and

Project

Summative evaluation IT project declined

IT project commences

No

Peter Schuurman

18

2.4.2 Building blocks A: Introduction and overview

This part contains important components for the remainder of the business case design, for instance the subject, purpose and situation. These building blocks need to represent the entire case in terms that are terse, clear, and accurate. Introduction and overview can contain the following blocks:

•

Title and subtitle

•

Authors and recipients

•

Date

•

Subject

•

Purpose

•

Executive summary

•

Disclaimer

•

Situation

2.4.3 Building blocks B: Assumptions and methods

Schmidt states that case credibility and effectiveness depend heavily on the explanation of where the data and results come from. This explanation should be provided by the description of the assumptions and methods used in the business case. Assumptions and methods can contain the following blocks:

•

Financial metrics

•

Assumptions

•

Scope and boundaries

•

Scenarios

•

Full value vs. incremental data

•

Cost model

•

Benefits rationale

2.4.4 Building blocks C: Business impacts

This section predicts the business consequences of the proposed action. It is important that data summaries and analysis are presented objectively and directly, to achieve this interpretations and explanatory text should be kept to a minimum. Business impacts can contain the following blocks:

•

Financial model / Cash flow statements

•

Analysis of results

Thesis post-project evaluation of the IT business case 19

•

Non financial results

2.4.5 Building blocks D: Sensitivity and risk analysis

The earlier blocks represent the most likely outcome in the eyes of the business case writer. However this outcome is uncertain since it projects results in the future. Sensitivity and risk analysis can contain the following blocks:

•

Sensitivity analysis

•

Risk analysis

2.4.6 Building blocks E: Conclusions and recommendations

This section interprets the content of the earlier described parts and connects this interpretation to objectives, decisions, and actions. Conclusions and recommendations can contain the following blocks:

•

Conclusions

•

Recommendations

2.4.7 Building the business case

Schmidt provides the creators of the business case with a three level overview on the business case, what he calls a 39,000 feet overview, a 3,000 feet fly over, and a ground-level walkthrough. The highest level provides a view on the best and worst kinds of business cases. The six key points a business case should live up to are (Schmidt, 2002):

•

Are subject, purpose and scope up front and clear?

•

Are cash flow projections organized along a time line?

•

Does the case present the assumptions and methods for identifying benefits and costs?

•

Does the case include all important benefits and costs – even those that are non financial?

•

Does the case discuss critical success factors?

•

Does the case identify and measure risks?

The middle level covers the main points of the business case, but it stays on a high enough level to

keep everything into perspective. The last, and lowest, level offers a walkthrough through even the

smallest elements of the business case.

Peter Schuurman

20

Some of the identified methods for filling in the content of a business case at this level are included in the next chapter.

When an organization uses the views on its business cases and has made a decision of which projects to implement Schmidt states that (quarterly) review reports on the financial measures outlined in the business cases as well as the progress toward the important business objectives as outlined in the introduction of the business case should be made.

2.5 Findings on IT business case theories

2.5.1 Definition

For the answer to the ‘what is a business case’ part of the sub-question which is answered in this chapter the three different theories will be compared on the basis of their definition of the business case. The three definitions that are provided in the preceding paragraphs are recollected in Table 2.

The business case is Theory Definition

PRINCE2 A description of the reasons for the project and the justification for undertaking the project, based on the estimated costs of the project, the risks and the expected business benefits and savings.

Remenyi A justification for pursuing a course of action in an organizational context to meet stated organizational objectives or goals.

Schmidt A tool that supports planning and decision-making; they are written to answer the question of what the likely financial and other business consequences are if an action or decision is taken.

Table 2: Definitions of the business case

Although in comparison with other business investments IT investments can have some different characteristics, IT business cases, just like IT projects in relation to general projects, have no different function than normal business cases. They too are created to overview the consequences, get project approval, and support in project management. It is also likely that IT business cases are also created with the same drivers as general business cases.

When comparing the three definitions, and the earlier noted extended descriptions, a few conclusions

can be drawn. The purpose of the business case creation is for all three roughly identical, being the

justification for the start up or continuation of a project or action. This indicates that the business case

can be very well used for the support of project management. The three descriptions of the use of the

Thesis post-project evaluation of the IT business case

21 business case support this by indicating respectively the continuous evaluation, the feedback loop and the review reports. Not surprisingly PRINCE2, Remenyi, as well as Schmidt all make this justification on the basis of the consequences for the business.

The scope of the theories however differs. Whereas PRINCE2 covers literally all kinds of projects and remains on the surface of the business case, Remenyi focuses on solely the IT investments and already applies some economical principles, and Schmidt offers a method for business projects in far more detail and with more hold for the person(s) who set(s) up the business case.

Due to the nature of this research the IT investment business case should provide the best foundation because it emphases most on the strategic and other IT issues of the projects. In addition, the Remenyi and Schmidt theories offer most insights into the actual content of the business case; it is probable that this information can contribute to the post-project evaluation of the business case further on. However in the view that Essent Energy already manages under PRINCE2 it could be so that the practice part of this research will connect best to the PRINCE2 business case theory. For now the business case will therefore be regarded as (OGC, 2002):

A description of the reasons for the project and the justification for undertaking the project, based on the estimated costs of the project, the risks and the expected business benefits and savings.

2.5.2 Content

Now that is determined what a business case is, the content part remains. To determine the content

of a business case, the different content of the three theories will be put side by side. An overview per

theory is offered in Table 3.

Peter Schuurman

22

The business case contents

PRINCE2 Remenyi Schmidt

Content Reasons Business outcome Intro & overview

Options Stakeholders Assumptions & methods

Benefits Strategic alignment & IT

benefit identification Business impacts Cost & timescale Technology issues / Business

Outcome Sensitivity, risks & contingencies Risks Project & System risk Conclusions & recommendations

Investment appraisal

Table 3: Contents of a business case per theory

Some similarities between the theories can be identified. These resemblances are listed in Table 4.

The business case content similarities

PRINCE2 Remenyi Schmidt Similarities

Content Reasons Business outcome /

Strategic alignment Intro & overview Necessity & desired outcome

Benefits / Cost Business outcome / IT

benefit identification Business impacts (Measurable) outcome / Cost-benefit consideration

Options - Assumptions and

methods Considered scenarios

Stakeholders influence

in management process

Stakeholders Stakeholders influence in management process

Stakeholder identification and roles

Timescale Technology issues /

Business outcome - Planning

Risks Project & System risk Sensitivity, risks &

contingencies Risk Analysis Investment appraisal Summative evaluation

(BC process) Conclusions &

recommendations Investment consideration Table 4: Similarities between the contents of the business case per theory

Five elements are found in all three business case theories, these are: the necessity and desired

outcome, cost-benefit consideration, risk analysis, stakeholder identification and roles and investment

Thesis post-project evaluation of the IT business case 23 consideration. The presence of considered scenarios, besides the chosen solution, and project planning is found at two out of three theories. It is noticeable that Remenyi is the only one to explicitly recognize the stakeholders as a class in the business case. The other two recognize the importance of the stakeholders according to their influence on the management process.

Of the recognized elements the five which are acknowledged three times are already included in the adopted PRINCE2 definition of the business case. For the answer of the question handled in this chapter this definition is extended with the other elements; this results in the following conclusion which will be used for the remainder of this research;

What is a business case and what should it contain for IT?

The business case is a description of the reasons for the IT project and the justification for undertaking the project. This justification is explicitly not only based on the financial consequences of a project, but also includes other elements that have to be accounted for;

together these elements are the foundation of the business case. This foundation consists of the estimated costs of the project, the expected business benefits and savings, the considered scenarios, a risks analysis and the project planning and is established in cooperation with the stakeholders of the project.

Next, in order to complete the theoretical survey of post-project evaluation of the business case, all

identified elements are considered in regard to PPE.

Thesis post-project evaluation of the IT business case 25

3 Post-project evaluation

3.1 Introduction

A post-project evaluation has been determined in paragraph 1.2.2 as

(…) the assessment of the value, worth or usefulness of a project after project closure. It is conducted to determine if the expected situation has been obtained and therefore compares the previous conditions with the actual results achieved by the project. Furthermore, it determines what is expected in future exploitation and how this will be achieved.

How such a post-project evaluation should be carried out therefore depends on how the previous,

expected, actual and future situations are described. For the assessment of a business case to a post-

project evaluation a separation has been made in these descriptions in the previous chapter; namely

the six elements of the business case. Each of these elements needs to be evaluated separately

(Remenyi, 1999); therefore an overview of the elements and the issues to take into consideration

when executing a post-project evaluation taken from the literature will be addressed in the following

paragraphs. Working towards a method of how business cases should be post-project evaluated, the

central points of attention in these overviews are: what and when should be evaluated and who

should evaluate. The findings on ex-post evaluation of the business case are then summarised in

paragraph 3.8 in a matrix.

Peter Schuurman

26

3.2 Estimated costs

The first element to be assessed is the estimated costs. Costs are the negative implications of a project, either financial or non-financial. To facilitate their PPE, they can be categorized into different types. Often used classifications are for instance the fixed and variable costs, which indicates the measure to which a cost is depend of the amount of products, and the direct and indirect costs, which points out the measure to which a cost is tied to a product (Renkema, Berghout, 1997).

Another way to classify costs, which is aggravated to IT costs, is in the three categories of technological, human and organizational costs (Remenyi, Money, Sherwood-Smith, 2000, Willcocks, 1994). In this distribution the technological costs are mainly direct, whereas the human and organizational costs are mostly indirect and are likely to occur mainly after implementation of the project. Table 5 provides an overview of common forms of these costs associated with the adoption of IT; these include for instance the costs for hardware, software and support. The table gives an impression of what costs should be accounted for in the business cases and could be recovered in a post-project evaluation.

Common costs in IT projects

Costs Direct Indirect

Technological Environmental operating costs Hardware costs

Software costs

Installation and configuration costs Training costs

Maintenance costs

Human Management resources

Management time

Management effort and dedication Employee training

Personnel issues Employee motivation Employee time Cost of ownership Organizational Losses in productivity

Strains on organizational resources Opportunity, cost, and risk Organizational restructuring Covert resistance

Business process re-engineering Table 5: Common costs in IT projects (after Remenyi, Money, Sherwood-Smith, 2000)

Thesis post-project evaluation of the IT business case 27 With the evaluation of the involved costs it is established whether or not the project and its outcome are staying within their outline of costs. Differences can occur at identified cost items as efficiency or price differences or come from unforeseen cost types. To determine the differences, there is a need for information on all actual costs made during the observed stage.

When addressing a post-project evaluation to the costs of a project, as drawn up in the initial business case, the case is thus to identify all actual costs and to determine the degree to which these were the result of the project. Due to the illusive nature of the (indirect) costs however, the identification and control however is very difficult to connect to a certain project outcome (Remenyi, Money, Sherwood- Smith, 2000). This connecting of consequences is identified as one of the major difficulties of post- project evaluations (Farbey, Land, Targett, 1993).

This aspect comes forward in a special form of indirect costs, namely the hidden costs. These are the costs of the impact of a project which are not so obvious and may appear in other non-project budgets (Willcocks, 1994). They lead to a distorted image of projects and could severely influence the results of a PPE. It is said however that due to the better understanding of IT costs in the past years there is less scope for these costs then there used to be (Remenyi, 1999).

Besides the illusive nature of costs, a next issue that plays a role in the difficulty of establishing the costs picture, and therefore in the evaluation of business cases, is that of political reasons. A coloured costs picture could significantly increase the change for project approval, therefore costs are subject to underestimation (Berghout, Nijland, Grant, 2005, Farbey, Land, Targett, 1993). On the other hand, also for political reasons, anticipated costs may be overstated in order to keep the project inside the budget. This can have positive side-effect for the project manager, like a reputation of being successful and effective (Willcocks, 1994).

Looking at the moments in time on which a post-project evaluation can be executed. The first time costs are evaluated is at project closure; with this evaluation the project costs are monitored and the exploitation costs are forecasted. At this time, the project manager, being the individual responsible for the day-to-day management of the project, is still responsible for the project, and therefore the responsibility for the project costs also lies with the project manager. During a PRINCE2 project, if expectations are it will fall outside its costs borders, an exception report is submitted for acceptance of the budget violation. With this report consequences on the business case are continuously observed. Therefore, under normal circumstances, it should be unlikely that the project costs differ a lot from the permitted budget.

After the project, in the exploitation phase, the costs are included in the operational costs. With that,

the task of PPE of the costs changes to line management, i.e. the business (case) owner. As said, the

major challenge here is allocating costs to a specific project result. The line manager can be

Peter Schuurman

28

supported in this task by (key) users, as an important source of information in this question and have to be consulted.

There is no particular indicated time to execute these following evaluations; therefore it is advisable to evaluate the costs simultaneously with the benefit evaluations (which should be planned, as will emerge in the next paragraph). Furthermore a non-standard, or not planned, evaluation should take place when the business case is reassessed of necessity due to arising exceptions. These are for instance, occurring risks, costs which are expected to get out of hand and benefits which are unlikely to be reached.

The information obtained from the costs evaluations should be analysed; after which the information and its consequences can be used as input for updating the business case in the forecast part of a PPE. Large deficiencies in costs should receive special attention in this process to make its continued effects to the business case clear. These continued effects are subject to the control paradox; that is, the possibility to influence the costs, as well as the benefits, decreases in time, however, in time the cumulative costs and benefits rise (Berghout, 2005). Therefore the deficiencies and their effects should be identified as soon as possible. The paradox is illustrated in Figure 8.

Figure 8: Costs and benefits cumulative vs. possible influence (after Berghout, 2005)

In the forecasting part of PPE the business case is updated and the exploitation of the project

outcome is re-justified. The emphasis for the cost elements lays on the expected exploitation costs,

with the sunk costs bared in mind and new cost items considered. Sunk costs are any costs that have

already been incurred and that cannot be changed by any decision made now or in the future, they