Cost estimation, the development of an estimation method for The Portal Company

(1)

Cost estimation, the development

of an estimation method for The

Portal Company

(censored version)

Jorjan Mulder

July 2009

University of Groningen, The Netherlands

Master Technology Management

Sponsor: R. Lanjouw

The Portal Company

Supervisors:

Bakker, drs. K.F.C. de

Business & ICT, Faculteit Economie en Bedrijfskunde, Rijksuniversiteit Groningen

Wortmann, prof. dr. ir. J.C. (Hans)

(2)

Preface

In front of you lies the final result of my master thesis which has been carried out at The Portal Company in Rotterdam. The main reason for this research about making cost estimates are some large budget overruns during the implementation of software projects by The Portal Company. All my findings and experiences have been described in this report. I have gained a lot of new practical and theoretical knowledge during this research and have worked with great pleasure on the final result.. My gratitude goes out to The Portal Company, particularly Ronald Lanjouw, who has made it possible for me to carry out this study. In spite of some setbacks Ronald always continued supporting me in conducting the research. I would also like to thank all the other employees of The Portal Company for their participation whilst conducting the interviews. Their contribution has ensured a clear view on several occasions. Furthermore I would like to thank my supervisor

(3)

Management summary

In this research a new estimation method for The Portal Company has been developed. The Portal Company wants to concentrate more on the execution of Portal projects, but without a structured estimation method these projects are an increased (financial) risk. The cost estimates for those projects are an important part of this risk. The current method of cost estimation is

unstructured and is done by experts. The cost estimates are subjective and prone to optimism. In addition, the knowledge of the experts are not easy to apply and difficult to control. Therefore the estimates are less reliable. The current method of estimation does not contribute to reducing the financial risk, but will rather increase it.

A direct reason for the research are a number of significant budget overruns on projects. Flyvbjerg (2007) emphasizes in his research the importance of an 'outside view' when creating cost estimates. The outside view is

established on the basis of information from a class of similar projects. Use of the outside view does not involve trying to forecast the specific uncertain events that will affect the particular project, but instead involves placing the project in a statistical distribution of outcomes from this class of reference projects (Flyvbjerg, 2007). The new estimation method for The Portal Company will enable the project manager to adopt an ‘outside view’.

There are several ways to make cost estimates: based on an expert, based on analogies and based on parametric models (Heemstra, 1992). For the development of a new estimation method, estimates based on analogies are the best suitable. Before, estimates based on analogies are possible in projects, three problems must be solved:

1. How can the project be characterized in the best way? 2. How do you decide if projects are similar?

3. How to use the information from a similar project for the estimation of the new project?

To characterize the project there has been an inventory of potential cost factors. The research is conducted within the framework of a practical case, therefore many cost factors will remain constant. This study shows that for the project characterization the product-related factors software size,

software complexity and degree of reuse are dominant. It is assumed that

the impact of the experience factor is negligible, because the practical case will take place in a short period (about one year). Other research has shown that the transition point from low to a lot of experience is one year.

(4)

The estimate of a target case can be based on information from a reference case. For this should first determine which projects are comparable. This will be done on the basis of a Euclidean distance. Then one or more reference case(s) are selected for making an estimate. Based on the reference case(s) an initial estimate is created. This estimate can then be adjusted in the light of the differences between the cases.

Analysis shows that the new estimation method produce reasonably accurate estimates for a number of cases. The experience factor is apparently of significant influence on the required effort of a project. Even if the time span of the project is less than one year, which is the transition point from low to a lot of experience.

A weak link in the estimation method is the selection of the number of

analogies. The literature indicates there is no 'best' method for selection. The estimator must determine the number of cases himself.

The study demonstrated that the dominant factors exert influence on the effort required of the project. When making cost estimates, it is important to take into account these factors. It is illustrated how this can be done in an estimation method based on analogies. The estimation method enables The Portal Company to take an "outside view" in making cost estimates.

A recommendation directed to The Portal Company is to preserve

(5)

1 Introduction

1.1 Situation

The Portal Company is a fictitious company name. The Portal Company is a small to medium company and provides IT consultancy services to its customers. The company is a so-called system integrator and operates on both project and detachment base. The projects covered include software development, system integration issues, architecture and business process management. The customers are mainly large companies. In many projects of The Portal Company there is a direct business need and applications are being developed and implemented where the customer is based.

The Portal Company wants to be a market leader in developing and

implementing a portal. A portal presents information from various sources in a unified way. A portal provides an organization the ability to use a

consistent “look and feel” with access control and procedures for multiple applications. The Portal Company has developed their own vision for the portal.

<part is removed>

The Portal Company tries to contribute this vision to customers and it has become part of the mission of the company.

Besides projects The Portal Company also develops detachment activities. The detachment activities of The Portal Company contain relatively low risk taking for the organization. The projects that The Portal Company carry out for customers are on the other hand highly-risk activities. To pursue the vision of the High Performance Workplace The Portal Company is more committed to carrying out portal projects instead of developing the

(8)

1.2 Problem description

Because The Portal Company is increasingly implementing projects the company runs a greater risk. This risk, among other, lies in closing

agreements with customers, which are often connected with the carrying out of projects. In the project agreement commitments are made in respect to the cost of a project. Here underlies a cost estimate.

A cost estimate is an important part of a project, but is also a risk component. The customer expects a cost estimate of The Portal Company at an early stage of the project. This is one of the aspects of a project where The Portal Company runs a (financial) risk. Making a low cost estimate has a negative impact on the relationship with the customer. When there is a fixed price agreement, a low estimate could even lead to a loss. This is an undesirable situation for The Portal Company. Obviously, a high estimate of the cost is also an undesirable situation for The Portal Company, because there is always the risk that the project (in the early stage) will go to a cheaper competitor (Lederer & Prasad, 1992). The Portal Company already faced a number of projects with an estimate of the costs which was to low. In these projects, there has been a significant budget overrun. Project budget overruns are not new in the IT sector. Many are due to a low cost estimate because of optimistic plans, underestimated of scope and complexity (Genuchten, 1991). The cost of a project consist for a large part out of the cost for the necessary effort to realize the project.

The Portal Company makes use of the knowledge and experience of the consultants (or experts) for estimating the effort required. This method of estimation is frequently used in the software development industry, although is has some disadvantages:

- Estimates of experts are subject to subjectivity (Heemstra, 1992). This makes them susceptible to e.g. optimism of the expert (Genuchten, 1991).

- The implicit knowledge that is used during the estimation is difficult to transfer. This makes it difficult for someone else to acquire and use the knowledge and experience of an expert. This can lead to a misleading situation in which rules of thumb of one expert are

becoming general applicable rules and applied in the wrong situations (Heemstra, 1992).

- Estimates of experts are difficult to control. In some cases, a project proposal is completely based on the estimate of an expert. If this estimate is not easy to check (and thus less reliable), there is an increased (financial) risk for The Portal Company.

(9)

scenarios of their coming progress and extrapolated current trends into the future.

In the outside view the details of the project at hand are completely ignored, and there is no attempt at forecasting the events that would influence the future course of the project.

Instead, the experiences of a class of similar projects are examined, a rough distribution of outcomes for this reference class are laid out, and then the current project is positioned in that distribution.

Flyvbjerg (2007) shows that the resulting forecasts, even the most

conservative ones, were overly optimistic in case of an inside view approach. In case of the outside view approach the resulting forecast was much more accurate.

The contrast between inside and outside views has been confirmed by systematic research (Gilovich et al, 2002).

When both forecasting methods are applied with equal skill, the outside view is much more likely to produce a realistic estimate. That is because it

bypasses cognitive and political biases such as optimism bias and strategic misrepresentation and cuts directly to outcomes.

By estimating the cost from an 'inside view' approach, so just by looking at its own capabilities, a cost estimate could be subject to optimism. The adoption of an outside view can ensure that they become aware of this phenomenon. The cost estimate can be corrected.

Heemstra (1993) shows in his research the importance of registering data from realized projects. This historical information is important for the success of cost estimates within an organization. The Portal Company does not store information about estimates in a consistent way, hereby limits the ability to learn from incorrect estimates.

The current way in which The Portal Company estimates is certainly not reducing the financial risk in the execution of projects. It will rather increase this risk for The Portal Company. The problems and disadvantages

mentionend above about the current way of estimating costs are the reason to launch an investigation. In this research the development of a new

estimation method for The Portal Company is central.

1.3 Theoretical framework

There are many methods known for making cost estimates. In general, these methods can be divided into three main categories (Heemstra, 1992):

- Estimate based on an expert - Estimate based on analogies

- Estimate based on parametric models

(10)

organization to make successful cost estimates. These directives include that a combination of two or three methods will give the best results. At present The Portal Company estimates are based on an expert. For the development of a new estimation method it is therefore interesting to look at methods from the other two categories: analogies and parametric models.

The current method for creating estimated will not be replaced, but supplemented with another method.

For the development of a new method there has been chosen for the estimate based on analogies. Given the small number of projects in the portfolio of The Portal Company estimating based on analogies is more suitable opposed to the use of parametric models. Being able to apply a parametric model requires a large number of historical project data against which the model must be calibrated (Heemstra, 1992). Estimates based on analogies does not requires this. If a similar project can be found it should be possible to create an estimate. To find analogies there are many projects necessary. Having a database covering a large number of historical projects increases the chance of finding a similar project. Moreover, the estimate based on analogies provide the possibility to compensate for the differences between projects.

The new estimation method will be based on the method of estimate based on analogies.

Estimating based on analogies is also known as Case-Based Reasoning (Delany, 2000). With CBR statements are made about the future based on information from the past. The method is explained briefly below using figure 1.

(11)

A problem may be for any new project (case) where the cost of implementing the project are not known. To overcome this problem a historical project whose costs are known may be used. It is important that both projects (new and historical) are somewhat similar to obtain a reliable estimate of the cost. In order to determine this similarity information that is known about the new project is used to find (retrieve) a similar project from a historical project database (previous cases). If a similar project (retrieved case) is found, the cost is an estimate (reuse) of the cost of the new project. The problem is thus solved (case solved). The CBR cycle is not yet completed. After the new project is completed, the (actual) cost known. The project can be adjusted (revise) and saved (retain) in the historic project database. This allows the saved project (learned case) to be used for future problems.

According to the literature (Shepperd, 1996), the following problems need to be solved to be able to apply CBR:

1. How can the project be characterized in the best way? 2. How do you decide if projects are similar?

3. How to use the information from a similar project for the estimation of the new project?

For the development of a new estimation method (based on analogies) this study will answer the above problems.

1.4 Research objective

(12)

1.5 Research model

Figure 2: research model Description of research model:

Since an uniform method to implement projects at The Portal Company is non excitant there has been decided to develop the new estimation method within the framework of a practical case. In the practical case there is one project. A description of this project will be given and the way current estimates are made.

Literature will be researched on factors that influence the duration of a project. This could be factors like system size, complexity, environment, technology, etc. These factors play an important role in making an estimate of the effort required for the project.

On the basis of these factors there will be a selection of factors in which a project of The Portal Company is best characterized. These are the

characteristics of the project. In a next step, the characteristics are

quantified. An aspect such as complexity will be made measureable in a few variables. The literature will be searched for starting points to operationalize the various project features. Here, the information available at the time of estimating will play an important role. The practical case will also be used as example.

(13)

1.6 Research question

Which factors affect the required effort in the development phase of a Portal project and how can these factors be applied in a new estimation method for The Portal Company?

The sub-questions are derived from the research model:

1. What does a Portal project of The Portal Company looks like and how are (current) cost estimates made?

2. Which features can characterize the project taking into account dominant cost factors that have an effect on software development? 3. Which variables can quantify the characteristics taking into account the

information available at the time of estimate?

4. What does a new estimation method based on these variables look like?

1.7 Practical case

The research will take place within the framework of a large-scale

automation project at a Dutch railway company. The railway company is currently working to (further) automate their human resources department. The Portal Company is involved as implementation partner. This means that The Portal Company is responsible for the development and implementation of the software. The large-scale automation project consists of several stages. Each stage can be seen as a separate project. Within the project portfolio of The Portal Company the railway company seems to be an ideal candidate for this research because of the following reasons:

- At the start of the study one phase of the large-scale automation project is finished

- There has been a budget overrun which is an important reason for the investigation

- The automation project includes a typical portal implementation which is in line with the strategy of The Portal Company. The Portal Company wants to carry out more of these type of projects in the near future - The completion of a subsequent stage, will be within the time frame of

the research

(14)

2 Projects and cost estimation

Sub-question 1: What does a Portal project of The Portal Company looks like and how are (current) cost estimates made?

2.1 Portal projects

The projects where a portal is developed and implemented for the customer are very important for The Portal Company. In these projects system

development is taken place within the layers of a service-oriented

architecture.1_{The projects have a duration of several months to about one}

year. For the technical realization of SOA different products and techniques are used. The portal is one of them. Besides the implementation of a portal there are also custom applications being developed. These applications are implemented within the context of a portal. Because of this context all applications have the same consistent look-and-feel and the user has one central access point for all

applications. The custom applications are being developed in appropriate software and they make use of services for their operation. The services are important components of an SOA. The services are used for retrieving

database data, modify database data and perform complex calculations. Services can be developed, but there are also many existing services in the backend systems which can be reused. The client has these backend

systems usually in use for some time. Examples of

backend systems are CRM and HRM systems.

Figure 3: a typical SOA design (Lublinsky, 2004) In a portal project often one or more key business processes are central. With these business processes in mind a portal is developed and

1

(15)

implemented. The portal will be established based on these business processes. The business processes will be supported by the applications in the portal and for some parts even automated by the system. To establish this the business processes are being fixed with the help of workflow management software.2 In addition to that integration software or middleware are also used. Finally there is also software for developing services. Usually, these three applications (workflow, middleware, service development) are unified in one software solution. In figure 3 is a typical SOA design.

2.2 Project phasing and cost estimates

In the projects of The Portal Company a clear phasing can be identified in which various cost estimates are identified. In figure 4, the phases of a typical software project are shown. This is sufficient for the projects of The Portal Company.

Phase 1: initiation

In the initiation phase of a project, a project is more or less born. In this stage a business case is developed based upon a common business problem. Generally there will be thought of possible solutions. Subsequently, a project plan for a solution is developed. In the initiation phase the order magnitude of the project will be clear.

Phase 2: analysis

In the analysis phase the business problem will be analyzed in detail. The business processes of the customer are central here. For original (sub) problems, solutions are considered and written out. The result of an analysis phase is a solution which solves the business problem and meet customer needs. Often this is described in a specification document.

Phase 3: design

In the design phase of a project, the solution will be described in detail. The specifications are translated into a detail design. In this phase it will become clear what exactly should be built. A functional and technical design are known in this phase.

Phase 4: development & test

This phase actually consists of two phases, but are often performed iteratively. In this phase, the solution is actually built on the base of the design. The construction is completed if there are no errors found in the test and the solution meets the requirements in the design. The development and

2

(16)

testing phase may consist of several cycles and is often the largest stage in the project.

Stage 5: acceptance

In the acceptance phase, the product (the solution) is actually accepted by the customer. In particular, the original business problem is consider here and if the developed solution solves this problem. If a solution does not meet the specifications or it won’t solve the company’s problem, the product is not accepted.

Step 6: go-live

Finally in the go-live phase, if the product is accepted, it will be taken into production use. There are often a number of specific tasks for the product to actually work in a production environment of the customer.

Figure 4: project phases

In the above figure there are three moments of making an estimate indicated:

rough order of magnitude, budgetary estimate and definitive estimate

(Schwalbe, 2007). The customer often expects an estimate on the moment the project is initiated. At this moment, there is little information available about what exactly needs to be developed. Therefore this estimate is very unreliable. As in the analyzing phase more information becomes available, the estimate will become more reliable. This is the second estimate. Both estimates have the objective to be an indication of the cost of the project and are often used for important decisions. For example, whether the benefits of the solution does outweigh the costs. Often there has been top-down

reasoning for making these estimates. In a top-down approach, the cost estimate for the entire project is derived from the characteristics of the product. The total estimated cost is then divided among the various components (Heemstra, 1992). The estimate based on analogies and / or parametric models are both examples of top-down approaches (Phillips, 2004).

In the design stage it will become clear what is actually going to be

(17)

together to estimate the cost of the entire project (Heemstra, 1992). This estimate is often the basis for detailed planning activities.

2.3 Current situation

To understand how cost estimates within The Portal Company are created the current situation is briefly described. This is done on the basis of the practical case (chapter 1.7) where the research will take place.

In the practical case, the above mentioned project phases are recognized. Currently the initiation phase of the project has already passed. In addition, The Portal Company is as implementation partner not responsible for the analysis phase of the project. The specifications are drawn up by a third party. Both phases will therefore not be involved in the investigation. In Figure 5, the scope of the research is indicated in relation to the project phases.

Figure 5: research scope Two moments are identified where an estimate is made:

- Estimate A: is made from the information available from the analyzing phase. Depending on whether this information is complete and precise enough some assumptions must be made. This estimate can be

classified as a budget estimate. In the practical case a budget overrun has taken place. Here, an estimate of type A was made initially.

- Estimate B: can be seen as a definitive estimate and will be made at a later stage in the project. The individual parts in the project are

assessed and estimated. To make this estimate more information is available from the design phase. Based on estimate type B a detailed plan is drawn up.

For the creation of the two estimates experts are asked for an assessment of the effort required of the project. This is done in a unstructured manner and is usually performed in groups. The project manager and the experts come together to make an estimate.

(18)

3 Cost factors and project characterization

Sub-question 2: Which features can characterize the project taking into account dominant cost factors that have an effect on software development?

A part of the estimate based on analogies is to characterize the project in variables. In the literature cost factors are discussed that affect the software development. For a characterization of the projects the cost factors are an important starting point. Earlier research identified many cost factors which influence software development and therefore should be included in the estimation of development costs (Lederer & Prasad, 1992).

3.1 Important cost factors

When an estimate is made, you have to know which cost factor are the most important for a specific situation, what is the value of these factors and which impact they have on effort and duration (Heemstra, 1992). There are many variables that affect effort and time. North & Krerzschmar (1984) have found more than 1200 different factors that could have an affect. It is

impossible to take all these factors into account in making an estimate. It is therefore important for an organization to determine the most dominant factors (Heemstra, 1992). In the literature there are different opinions about what the dominant factors are. This is evidenced by research of Boehm (1984) showing that in different estimation models, different factors play a role. Heemstra has divided these factors into categories and defined for each category the important factors (from his point of view). This subdivision is displayed below in table 1.

Table 1: important cost factors (Heemstra, 1992)

Heemstra makes in his structuring of cost factors distinction between the following cost factors:

- Product

(19)

developing software products will increase if the size and complexity of the product increases. Also the degree of reuse is a cost factor. The case here is that with reuse the final product can be developed faster which result in cost savings (Heemstra, 1987).

- Technical resources

Limitations of the technical systems (such as response time) and the user devices are cost factors here. A rapid response time during software development leads to significant improvements in productivity (Walston & Felix, 1977).

- Staff

In this category, the quality of the personnel is a cost factor. This concerns the quality of the project team. Besides quality are experience and availability of personnel cost factors. Research (Heemstra, 1987) shows that the experience factor is an important cost-determining factor. Roughly can be said: the more experience, the lower development costs. After a period of five years there is a transition point from low to a lot of experience. This is different for the factor of experience in programming. Here the transition point will be after about one year. Moreover, after two to three years this

experience factor has reached a saturation point (Heemstra, 1987). - Project management

This category of cost factors include: requirements on the project duration and control of the project. In general, the influence of the two factors are small.

- Customer organization

The cost factors are involvement of the user and the number of users. Also stability and experience of the client organization with automation are cost factors. Research of Heemstra (1987) shows that the

productivity decreases more than 50% if the participation of the user in drawing up requirements increase. If the same software product needs to be developed for multiple users, the development time and costs increase (Heemstra, 1987).

(20)

3.2 Project characterization

The creation of the project characterization will be based on the

(sub)projects of the practical case. In the practical case occurs (at the time of research) the following situation:

- One project (phase 1) has been completed and a second project (phase 2) is about to begin.

- Implementation of the two projects are within the context of the same (customer) organization.

- Implementing the two projects is completed within one year. - The project team remains for a large part unchanged.

- The project management methodology (Prince2) and software development method is unchanged

- The projects are implemented in an existing software architecture therefore the technical resources for both projects are fixed.

Based on the above situation it can be observed that for both projects (phase 1 and 2) the cost factors related to the customer organization, project

management method and / or technical resources will remain unchanged. These cost factors may be constant within the framework of the investigation. It has no use to consider these factors in making a cost estimate for Phase 2. Here it is assumed that the influence of these factors is zero.

The experience factor Heemstra's category Staff plays a possible role in both projects. Research has shown that the effect is only noticeable after a period of one year. This is the turning point of little or much experience. Both projects are implemented within the time frame of one year and the project team remains unchanged. The factor experience is assumed to have little or no influence on the final estimate. The fact that some (project) team

members reach their turning point halfway through the project is neglected. The above finding is that only the product related factors are dominant in creating a cost estimate within the framework of the practical case. For the characterization of the projects the cost factors related to the product being developed will be used. Virtually the factor required quality will also remain unchanged as in the two projects this should be on the same level.

From the above selection of major cost factors of Heemstra (1987) are the following factors the most dominant in the practical case:

- Size of the software - Software complexity - Degree of reuse

(21)

4 Variables and quantify

Sub-question 3: Which variables can quantify the characteristics taking into account the information available at the time of estimate?

To measure the features: - size of the software; - software complexity and; - degree of reuse;

in a project the concepts will be operationalized. For each characteristic variables will be defined through which the attribute can be measured. For this the literature will be consulted. The choice of variables should be limited to the information available at the time of estimate (Shepperd, 1996). Heemstra (1992) explains that cost estimates are often hurriedly executed. This is confirmed by the project managers of The Portal Company. Its often the case that a customer requires an estimate which needs to be carried out in a short timeframe of about five days. An estimation method that cannot comply to this will not be workable in practice. For this reason, quantifying the characteristics of the selected variables should be carried out in less then five days.

4.1 Size of software

4.1.1 Theory

In the literature there are several techniques to determine the volume of software (Heemstra, 1987). Known techniques are lines of code and function points. The result of these techniques is the size of the software expressed in the number of lines of source code or the number of function points.

Lines of code as an indication of the size of a product works less well in an early stage of development (Heemstra, 1989). To make an estimate in the early stage of a project an indication of the size is also required in this early stage. Since in this early stage, no line of code is written, this technique is not suitable as an indicator of the size.

(22)

points analysis are suitable for application within the practical case of the research.

In the following section the method is briefly explained.

4.1.1 Method COSMIC-FFP

The COSMIC measurement process consists of three steps (Abran, 2007):

1. Scope definition

This concerns identifying the Functional User Requirements (FURs) and functional users. In addition, the establishment of a decomposition of the software to be developed in smaller (software) components. This decomposition includes the layering of the software. In the next

section (4.1.2) a decomposition of the software in the practical case is described.

2. Identification of functional processes

Here the functional processes and data groups are identified. A functional process is an elementary component of a series FURs consisting of a series of data movements.

3. Data movement count

First, the data movements between the functional processes and functional users are identified. A distinction is made between four different types of data movement: entry, exit, read and write. Finally, all data movements are added together.

The result is a measurement of the functional size of a piece of software expressed in COSMIC Function Points (or CFP).

4.1.2 Decomposition

In chapter 2.1, the Portal projects of The Portal Company are described which are using an SOA. An SOA has the property that the (software) components (representing the final product) can be developed in different development environments. Each development environment has its own productivity aspects (Santillo 2007).

In the practical case a distinction is made between three development environments:

- Front-end

The front-end development consist mainly of development of screens in the Portal using services (chapter 2.1 Portal projects).

- Integration

The integration includes the creation of business processes in a workflow management system and defining services.

- Back-end

(23)

The software in different development environment have to be measured separate from one another (Santillo 2007). Each development environment has a different development language and requires different skills and expertise of a developer.

In figure 5 the relations of the (software) components are shown.

Process step A

Service A _{Service B}

Interface B

Building block A Building block B

Interface A

Interface B`

Screen A Screen B Screen C

Interface C`

Building block C FRONT-END

INTEGRATION

BACK-END

Figure 5: decomposition into (software) components

In the decomposition the boundaries of the software to develop are visible. These boundaries also represent the layers of software in an SOA.

4.1.3 Functional users and processes

(24)

Figure 6: cross-boundary relations

Figure 6 also shows the different types of data movements available. An entry (E) and exit (X) data movement will always occur between a functional and user process.

A read (R) and write (W) are always between a functional process and a persistent storage (database).

4.1.4 Example: declaration request

Below is an example explained of a (sub)process for requesting a declaration by an employee to illustrate the COSMIC-FFP method.

Functional proces

An employee wants to submit a declaration. He completes the required declaration data on a screen and hits 'send'. The system checks if the declaration amount is not more than 25 euro. If the declaration does not exceed 25 euro’s, then it is automatically approved. The claim is filed and the employee receives a confirmation. If the declaration amount exceeds 25 euro’s, a confirmation request is sent to the appropriate manager. The manager receives the request in his inbox.

Datamovement

In figure 7, the beginning of the functional process of requesting a

(25)

Request declaration

Process declaration request

Produce declaration SAP HR W: create DECLARATION E: {declaration data} X: {declaration data} E: {declaration data} X: {declaration data} E: {declaration data} X: {error reports} SAP HR Retrieve manager X: request manager Inbox (Backend) Produce message E: request manager R: EMPLOYEE X: EMPLOYEE

E: EMPLOYEE X: {message data}

E: {message data}

W: create MESSAGE

Figure 7: data movements

(26)

4.2 Complexity

4.2.1 Theory

Complexity consists of distinctions (components) and connections (links) which maintain a relationship. Distinctions are the number of appearances of an object. Connections are the links between distinctions. A distinction may also have a connection with the same distinction.

One way to determine complexity is to look at the size of the system. The larger the size, the more complex it is. This assumption does not apply in all situations (Konieczny, 2004).

The best definition so far to determine how complex something is, is the length of the shortest description of the system, better known as the

Kolmogorov method (Konieczny, 2004). The idea behind this is: how complex the system, the larger and harder it is to describe in its entirety. The aspect information plays an important role here. If it is not known what the system does, there is insufficient information about the system, then also nothing meaningful could be said about the level of complexity. There is also a risk that things are omitted (because information is missing) so the system seems less complex (Konieczny, 2004).

The above describes the complexity in general. For this research the complexity of software in particular is considered.

4.2.2 Software complexity

In the literature there is often a qualitative assessment of software

complexity to grade the product (to be developed) into a complexity class (Heemstra, 1987). This grading can be based on different criteria. This qualitative assessment has some drawbacks. during assessing a certain degree of subjectivity is involved. Heemstra (1987) argues that there is a lack of quantitative criteria.

In studies of Tran-Coa (2004) there is a software complexity model developed which is based on earlier research by Wood (1986) on task complexity. In this model we distinguish between two categories of complexity: component and system complexity.

Component complexity refers to the internal complexity of a functional process. It can be characterized by both the input and output data and by manipulation of the input data processed to produce output.

System complexity refers to the complexity of the relationship between functional processes. This kind of complexity is characterized by the data communication between functional processes.

Both forms of complexity contribute to the overall software complexity: - Component complexity refers to how complex a component is . - System complexity refers to how the complex inter-relationship

between components is.

(27)

Figure 8: software complexity

The first measure is the number of data groups. Based on the idea of Wood (1986) the number of data groups (NOD) is a good measure of data group complexity.

The second measure is the number of conditions on the input to produce various output. This measure aims to obtain the relationship between input and output. The complexity in data manipulation, as measured by the number of conditions, is interpreted as the complexity of the relationship between input and output. This can be seen as a Cyclomatic number. In the literature the Cyclomatic complexity of McCabe (1976) is accepted as a good measure of the complexity of program structure. Cyclomatic complexity is defined based on a flow chart of a program. It can be seen as the number of conditions knots in a flow chart plus one and can be interpreted as the number of independent paths of the flow chart.

The third measure takes into account the relationship between functions. This complexity is characterized by the data groups in the communication between two functions. This means that data connections between functions are obtained. There may be an entropy measure to quantify this kind of complexity. Entropy is a measure of disorder in a system (Tran-Cao, 2004). In the next paragraph (4.2.3) is described how to measure entropy.

(28)

4.2.3 Entropy measure

To measure the entropy of a system the data dependency between functional processes can be modeled as a directed weighted graph. Two processes have a data dependency if an output of one process becomes an input of another. If one process sends a signal to trigger another process without sending data, then there is no data dependency (Tran-Cao, 2005).

More precisely, a node on the graph is a process. An arc represents a data group in data dependency between two processes, and the weight of arc represents the number of data groups in

the communication.

In figure 9 an example of a directed-weighted graph representing

the relationships between functions. To calculate entropy, all functional processes are classified in

equivalent classes. Two processes belong to the same equivalent class if and only if they have the same in-degree and out-in-degree, which is the sum of the weight of arcs coming into and out of the process,

respectively. Figure 9: a directed-weighted graph

To calculate entropy, all functional processes are classified in equivalent classes. Two processes belong to the same equivalent class if and only if they have the same in-degree and out-degree, which is the sum of the weight of arcs coming into and out of the process, respectively (Tran-Cao, 2005). For example, the graph in figure 9 has the following equivalent classes:

{a} with in-degree = 0 and out-degree = 3 {b,f} with in-degree = 1 and out -degree = 2 {c,e} with in-degree = 2 and out -degree = 3 {d} with in -degree = 4 and out-degree = 2 {g} with in-degree = 5 and out-degree = 0. The entropy of data dependency is:

(29)

4.3 Reuse

4.3.1 Theory

Software reuse is the use of existing software artifacts (or knowledge) to new software development (Frakes & Terry, 1996). Re-usability is the extent to which a thing can be reused (Frakes & Isoda, 1994). Software reuse can be applied to any (project) life-cycle product. Not only to fragments of source code. This means that the developer of the requirements documents, system specifications, design structures and any other development artifact can reuse (Barnes & Bollinger, 1991). Jones (1993) has ten potential aspects of software projects identified that can be reused (see table 2). In addition to these life cycle products are processes and knowledge also potential for reuse.

1. architectures 6. estimates (templates)

2. source code 7. human interfaces

3. data 8. plans

4. designs 9. requirements

5. documentation 10. test cases

Table 2: reusable aspects

For this research the reuse of source code in particular is considered.

4.3.2 Source code reuse

There are several ways in which a source code can be reused: totally, in part or not at all (Leach. 1996).

These can be specified into four categories of reuse:

1. Transported. This means that the code is reused from an existing system.

2. Custom. This means that the code from an existing system can be used. The code has been changed. Between 75 and 100 percent of the code is reused.

3. Converted. This means that the code from an existing system can be used. The code is adjusted more, than in the custom category. Between 50 and 75 percent of the code is reused.

4. New. The code is completely re-developed. This category applies when the code from an existing system can be reused for less than 50

percent.

Different organizations use different breakpoints to determine the type of reuse in a system (Leach. 1996).

Ho (2000) argues that from the perspective of the re-user there are two types of software reuse. This is reuse without modification, called black-box reuse and reuse with modification, called white-box reuse.

(30)

Categorieën van hergebruik (Leach) Soorten hergebruik (Ho) Transported Black-box Adjusted White-box Converted New Table 3: Leach vs. Ho

(31)

4.4 Choice of variables

4.4.1 Available information

Shepperd (1996) argues that the choice of variables should be limited to the information available at the time the estimate is required. Paragraph 2.2.1 decides that using the new estimation method a budget estimate (estimate A in figure 5) must be made. This estimate is needed at a time in the project that the analysis phase has finished and the design phase is about to start. In the practical case various documents are being used. This is shown in figure 10.

Figure 10: output per phase

The output of the analysis phase are process descriptions. A process description contains information about:

- the target group of the process (stakeholders) - sequence of process steps (activities)

- systems that are consulted

- the interaction between process, systems and stakeholders

- input (person, system, etc..) and output (database, document, etc..) - decision points in the process

- functional requirements and wishes

Much of this information is visually displayed in a process model. In the appendix an example of a process model is included.

Based on the process descriptions the functional and technical designs are prepared. In addition, screen design (called mockup) are prepared in the design phase.

A functional design and mockup contains information about: - a description of a use case

- functional requirements - workflow of screens

(32)

- data required per screen - conditions

- possible actions - presentation of data

The information for the functional design is briefly written in a document. The workflow is displayed visually on the basis of a flow chart. Most attention is paid to the design of a mockup. In Appendix II an example of a mockup is included.

A technical design contains information about:

- detailed (technical) descriptions of the process - interaction between systems (per process step)

- necessary information / services and automatic transactions - detailed description of services at data field level

- description of interfaces (data field level)

- definition of mappings between interfaces (data field level) - implementation of conditions

This information is described on detailed level in a document. Again, it uses models to visual support. In Appendix III an example of a model in a

technical design is included.

4.4.2 Measurement of software size

To measure the size of the developed software in the practical case the function point COSMIC-FFP method is applied. Counting the number of lines of code is not applicable, because the estimate is required at an early stage of the project. At this early stage of the project, there are little or no lines of code programmed. To perform a budget estimate (estimate A in figure 9), only the process descriptions are available. Therefore, using COSMIC-FFP to measure the functional size of software look obvious. Based on the

information in the process description it is possible to identify the functional processes and users. The software architecture is established (see figure 5). Therefore it should also be possible to link the functional processes (and users) to the various systems in the architecture. Then the data movements between the different systems can also be identified.

4.4.3 Measurement of complexity

By determining the size of the system the complexity is already taken into account. The size of a system tells us something about the complexity. An increase in size is often associated with an increase in complexity of software (Heemstra, 1987). The number of function points (or CFPs) in different

(33)

Again, the fact that only the information from the process descriptions are available for the determination of the complexity must be taken into account. The process models (flow charts) used to describe the business processes are suitable for determining the Cyclomatic complexity of McCabe (1976). In appendix I an example of a process model is included. The number of conditions in this process model is a measure of the complexity of the program structure.

4.4.4 Measurement of reuse

An SOA is characterized by reuse of services. Services can be reused without knowledge about the functioning of the services. A service is a good example of black-box reuse of Ho (2000). It is a form of reuse without changing the operation of the service. The extent to which use is made of existing services is an appropriate measure of the (black-box) reuse.

(34)

5 Estimation method and analysis

Sub-question 4: What does a new estimation method based on these variables look like?

According to Heemstra (1993), there is no such thing as a generally

applicable approach to estimating. This approach will vary from development to development.

In the practical case two separate phases can be distinguished (chapter 1.7). Each phase can be seen as a separate project. In both phases are a number of key business processes automated.

Phase 1 consists of four business processes: NAW, P-dossier, Declaratie and

Verlof.

Phase 1 is completed at the time of estimate. Therefore the number of actual hours spent is known. `

The automated business processes in phase 1 may be used as analogies in the new estimation method.

Phase 2 consists of automating five business processes: Indiensttreding,

Verplaatsing, Uitdiensttreding, Digitale loonstrook en Bank & Gezin.

Phase 2 is at the time of estimate not yet completed. Using the new

estimation method an estimate can be made of the (actual) hours spent on automating these business processes.

Estimating the effort of a software project with analogies includes the following steps (Walkerden, 1999):

1. Measuring or estimating the values of the project metrics for the target project.

2. The search in a repository of existing projects on a project similar to the target project and select one or more project as a source analogies. 3. Use the effort of the source analogies as an initial estimate for the

target project.

4. Compare the familiar metric values for the target and source project 5. Adjust the estimation of the effort in light of the differences between

the target and source project.

(35)

Figure 11: flowchart

5.1 Estimation method (5 steps)

5.1.1 Step 1: measurement of variables

Measuring the variables for size, complexity and reuse will be done at

business process level. This created nine separate cases, of which four cases can be used as analogy. In table 4 the result of measurement is shown. For each business process separately are the chosen variables (chapter 4.4) measured and registered in a calculation model. This is done based on the information that was described in the process descriptions.

EFFORT SIZE COMPLEXITY REUSE

Process Estimation Actual Front-_end Integration Back-_end Conditions Front-_end Back-_end

NAW - 720 91 6 49 1 0 0 P-dossier - 384 37 2 28 1 20 19 Declaratie - 1842 92 12 44 26 21 27 Verlof - 1477 82 9 52 24 26 27 Indiensttreding ? - 126 40 90 20 18 22 Verplaatsen ? - 107 34 63 21 46 52 Uitdiensttreding ? - 73 36 51 20 38 32 Dig. loonstrook ? - 16 0 15 0 0 3 Bank / gezin ? - 78 6 31 6 18 19

(36)

Measuring the size and reuse is broken down into three different

development environments (chapter 4.1.2) within the software architecture (or SOA) of the practical case.

The size of the software in the business processes is determined by applying the function point method COSMIC-FFP (chapter 4.1.1). For each business process data movements (per development environment) are measured. For example, in the business process ‘Declaratie’ there are 92 CFPs measured in the front-end.

The reuse of software in business processes is determined by identifying the functional processes that are being reused. This figure is created by

multiplying the number of times the reuse occurs with the number of CFPs (of the reusable functional process).

For example, in the business process 'Verplaatsen' there are 46 CFPs

measured in the front-end wherefore existing functional processes could be reused.

Reuse is only measured in the front- and back-end. This is because the functional processes in the integration environment are hardly being be reused. In the integration environment the process logic of the business process is defined with the conditions.

The business process 'NAW' has no reuse, because this business process was first developed.

The complexity of the software in business processes is determined by the number of conditions (Cyclomatic number) in the business process to be measured. In addition, of the software size which also determine the complexity of the software.

For the first four business processes from phase 1 also the number of actual hours spent are registered. These are the reference cases.

For the business processes from phase 2 an estimate of the number of actual hours spent needs to be created. These are the target cases.

5.1.2 Step 2: finding analogues projects

(37)

The Euclidean distance can be calculated using the following formula:

: Euclidean distance between 0 (target case) and 1 (reference case) : value of variable x (e.g. number of function points) of case 0 : value of variable x (e.g. number of function points) of case 1 : value of variable y (e.g. number of conditions) of case 0

: value of variable y (e.g. number of conditions) of case 1

It is important to first normalize the measured variables before applying this formula. Therefore, each variable (or dimension) will contribute proportional to finding analogies (Shepperd, 1996). In table 7 is the Euclidean distance for all target cases to the reference cases are calculated.

The calculated distances from the target case with the reference cases are relatively large compared to distances in other studies. This means that the cases are quite different in terms of size, complexity and degree of reuse. It is possible to use one or more cases as reference cases.

k : number of analogies used to estimate the target case.

Selecting similar cases can be done according Kadoda et al (2000) in two ways:

- A fixed value of k (e.g. k=1, k=2, k=3, etc.)

- A distance-based selection for k (e.g. d<0.10, d<0.15, etc.)

Research (2000) Kadoda et al. show that three analogies (k=3) will have the best results and that a distance-based analogy selection is more effective for small data sets. Li et al. (2006) notes that there are studies where only one analogy (k=1) will have the best results.

In the selection of similar cases (K) for the new estimation method in this study a variant of the distance-based analogy selection of Kadoda et al. is applied.

It will take into account the difference between the Euclidean distance of the reference cases in which the reference case are arranged at a distance (from small to large).

The maximum3 here is: ∆(d1 – d2) < 0.2.

d1 : Euclidean distance between case 0 (target) and case 1 d2 : Euclidean distance between case 0 (target) and case 2

3

(38)

The ordered reference cases whose Euclidean distance to the target case is less than 0.20 apart are used to estimate the target case. In table 7 is for each target case indicated which reference case(s) are used and how many (k). The value 0.20 is determined by the researcher and will provide the best result in this practical case.

5.1.3 Step 3: initial estimate

In this step, a first estimate is made for the number of necessary (effort) hours of the target case. It is an initial estimate. The actual hours spent of the reference case is used here as an (initial) estimate for the target case. In the case of several reference cases a weighted average of the actual hours spent is calculated. The weighting is based on the Euclidean distance of the reference cast with the target case. This reference case with the smallest Euclidean distance will have more influence (high weight). In table 5 the initial estimates for the target cases are calculated. These are the values between brackets.

Process Estimation Actual

Front-end Integration Back-end Conditions Front-end Back-end NAW - 720 91 6 49 1 0 0 P-dossier - 384 37 2 28 1 20 19 Declaratie - 1842 92 12 44 26 21 27 Verlof - 1477 82 9 52 24 26 27 Indiensttreding (1666) - 126 40 90 20 18 22 Verplaatsen (1660) - 107 34 63 21 46 52 Uitdiensttreding (1663) - 73 36 51 20 38 32 Dig. loonstrook (543) - 16 0 15 0 0 3 Bank / gezin (384) - 78 6 31 6 18 19

Table 5: calculation model (with initial estimation)

Because the Euclidean distances of the target cast to the reference cases are relatively large, this initial estimate is not reliable. Here is assumed that when the cases differ in size, complexity and degree of reuse, the also will differ in the number of (effort) hours spent. This (initial) estimate can not be used.

(39)

5.1.4 Step 4: comparing variables

Compare the variables in the reference case(s) with the variables of the target case. This can be done by divided the variables of the target case with the variables of the reference case(s). This is done for each case individually. Again all variables need to be normalized before comparing them. It is

important to know whether the correction should be positive or negative (Kadoda et al, 2000). This depends on the variable itself.

The variables for which the estimation should be positive adjusted are: size (CFPs) and complexity (number of conditions). Here it is assumed that the relationship of size and complexity with required hours is positive. In other words, the greater or complex the software, the more hours are needed for software development.

The variables for which the estimate should be adjusted negatively are the CFPs for the degree of reuse. Here it is assumed that the relationship of the degree of reuse with required hours is negative. In other words, the more can be reused, the less hours there will be needed to develop the software.

5.1.5 Step 5: adjusting estimation

The last step is to adjust the initial estimate in the light of the differences between the variables. This can be done by multiplying the sum of

differences with the initial estimate. In table 6 the final (budget) estimates for the target cases are calculated.

Process Estimation Actual

Front-end Integration Back-end Conditions Front-end Back-end NAW - 720 91 6 49 1 0 0 P-dossier - 384 37 2 28 1 20 19 Declaratie - 1842 92 12 44 26 21 27 Verlof - 1477 82 9 52 24 26 27 Indiensttreding 2907 - 126 40 90 20 18 22 Verplaatsen 2122 - 107 34 63 21 46 52 Uitdiensttreding 2104 - 73 36 51 20 38 32 Dig. loonstrook 575 - 16 0 15 0 0 3 Bank / gezin 917 - 78 6 31 6 18 19

(40)

5.2 Analysis results

In table 7 below the budget estimates (chapter 5.1.5) are compared to the actual hours spent in the target cases.

ESTIMATION Distance Number of analogies (k) Estimate

(Excel) Actual Difference %

Indiensttreding Declaratie 0.962 k=2 2907 4480 1573 35% Verlof 1.026 P-dossier 1.558 NAW 1.638 Verplaatsen Declaratie 1.045 k=2 2122 2120 -2 0% Verlof 1.029 P-dossier 1.714 NAW 1.828 Uitdiensttreding Verlof 0.849 k=2 2104 854 -1250 -146% Declaratie 0.885 P-dossier 1.471 NAW 1.808 Digitale loonstrook P-dossier 1.042 k=2 575 523 -52 -10% NAW 1.164 Verlof 2.056 Declaratie 2.106 Bank / gezin P-dossier 0.597 k=1 917 1005 88 9% Verlof 0.943 Declaratie 1.014 NAW 1.073 Total 8625 8982 357 4%

Tabel 7: analysis results

5.2.1 Short explanation

The third column is the calculated Euclidean distance of the target business process (column 1) with the four reference processes. The analogies used as reference cases are shown in bold figures. These cases are selected as analogies, because the Euclidean distance is smallest and are not more than 0.2 from each other (chapter 5.1.2).

The fifth column shows the budget estimate in number of hours. This has been based on the analogy case(s) and compensated for the differences in variables (chapter 5.1.4).

The sixth column shows the actual hours spent which are calculated based on information obtained from the hour registration system after the completion of the project (phase 2).

(41)

5.2.2 Analysis

The estimates made by the new estimation method are budget estimates. According to Schwalbe (2007) an error rate of between 10% and 25% for an budget estimated is acceptable.

Knowing this, the estimates for the processes Verplaatsen, Digitale

loonstrook and Bank/gezin can be called fairly accurately. The estimates for

these three processes are within the error range. The process Verplaatsen is even estimated 2 hours accurately.

Regarding the Indiensttreding and Uitdiensttreding processes the estimation method is quite wrong. The Indiensttreding process is slightly

underestimated. The Uitdiensttreding process is even 1.5 times overestimated by the new estimation method.

Another observation is that the total of the estimates for all five processes is only 4% off from the actual hours spent in total. The (overall) budget

estimate of 8625 hours for entire phase 2 is also fairly accurate.

(42)

6 Conclusion

The conclusion consists of answering the main and sub questions of the research. Finally, some recommendations towards The Portal Company are given.

6.1 Answers sub-questions

Sub-question 1: What does a Portal project of The Portal Company looks like and how are (current) cost estimates made?

The portal projects that The Portal Company performs at customers have identifiable project phases: initiation, analysis, design, development and test,

acceptance and go-live. The practical case in this research includes the

automation of several business processes within a Service Oriented

Architecture. There are several estimates made within the practical case. To create these estimates, no real method is used. Estimates are established in an unstructured matter and on the basis of expert knowledge. Therefore, the estimates are subjective and prone to optimism.

Sub-question 2: Which features can characterize the project taking into account dominant cost factors that have an effect on software development? The literature lists a large number of factors which may affect the making of (cost) estimates. Heemstra (1992) has the most important cost factors categorized into five categories: product, technical resources, personnel,

project management and client organization.

For the research, only the factors from the product category are of interest, because it is assumed that the factors from other categories will remain constant (or whose impact is negligible) within the time span of the research. The following factors are assumed to be dominant in the practical case: the

size of the software, software complexity and degree of reuse.

Sub-question 3: Which variables can quantify the characteristics taking into account the information available at the time of estimate?

The information available at the time of estimate appears to be decisive for the choice of variables. To measure the size of the software a modern

function point analysis COSMIC-FFP is applied. For measuring the procedural complexity the number of conditions in a business process is set as a

measure. The degree of reuse is measured by multiplying the number of function points of the functional process that is reused with the number of times the process is reused. The process descriptions from the analysis phase are uses as information source.

(43)

The estimation method is based on a Case Based Reasoning model for

estimating based on analogies. The estimation method consists of five steps. In figure 11 the estimation method is shown in a flowchart. The estimation method takes into account the differences between the reference case(s) and target case. The final (budget) estimate is adjusted in the light of these differences.

6.2 Answers research questions

The main question of the research is:

Which factors affect the required effort in the development phase of a Portal project and how can these factors be applied in a new estimation method for The Portal Company?

This research has been able to demonstrate that the factors (size, complexity and degree of reuse) affect the required effort. It is therefore important to take these three factors into account when making estimates. The study also illustrate how these factors can be used in an estimation method by using the estimate based on analogies.

6.3 Recommendations The Portal Company

A major lack of the 'old' way in which The Portal Company estimated was the subjectivity and optimism that the estimates contain. Flyvbjerg (2007) calls this manner the 'inside' view in his research and is a major cause of the low estimates of projects. With this new estimation method, the project manager is enabled to take an "outside" view. Through estimating based on (objective) historical project data instead of (subjective) expertise. This should neutralize the optimism which is currently in the current estimate. What ultimately will result in creating 'better' cost estimates.

The Portal Company should now start with storing the estimation information in a consistent matter. This is also one of the directives Heemstra (1993) advises. The estimation method developed in this study would be a good basis for doing this.

Cost estimation, the development of an estimation method for The Portal Company