The effect of project management methods on project success

(1)

Master Thesis Business Administration – Strategic Innovation Management

The effect of project management methods on project

success

Author: Joël Derks Student number: s1906402

Groningen, April 2014

University of Groningen Faculty of Economics and Business First supervisor: Dr. T.L.J. Broekhuizen

(2)

2

ABSTRACT

This paper analyses how project management methods influence project success. The existing literature on project management does not provide clear evidence for the influence of project management methods on project success. In order to fill this research gap this study focuses on how project management methods influence project success via achieving the critical success factors. Moreover this study analyzes how the project management methods are applied and adapted. A case study is performed among 6 project managers based on one focal firm and six separate cases. The results provide indications that project management methods influence project success via the critical success factors communication, end user involvement, and realistic schedules, but not via the critical success factors clear goals, top management support, adequate funds and resources, effective leadership, flexible approach to change, past experience or effective risk management. Furthermore, project management methods are tools that may have a positive effect on success when applied and adjusted correctly for the project situation, but the methods are not a panacea for success.

Keywords: Projects, Project management, Project management methods, Project success, Critical success factors.

1. INTRODUCTION

Since the late 1960s project management researchers have tried to uncover which factors lead to project success and failure (e.g. Atkinson, 1999; Avots, 1969; Munns and Bjeirmi, 1996; Slevin and Pinto, 1988; Shenhar et al., 2001; Shenhar, 2001; Terry Cooke-Davies, 2002;). Still, recent results of the Standish Group (Standish, 2013) indicate that the project success rate (on time, within budget and with the required features and functions) was only 39% in 2012, which is similar to the success rate of 41% found by White and Fortune (2002). Although success rates slightly improved, after 40 years of research into project management still less than 50% of projects are successful (Standish, 2013).

(4)

4 specific preferences. White and Fortune (2002) found that 72% of the 236 respondents in their study used a project management method. In recent research (including multiple countries) 92% of the 150 respondents used project management methods (Fortune et al., 2011). The increased adoption of project management methods is further evidenced by the increasing certification of PRINCE2 practitioners (in 2012 900,000 certificates were issued (Knowledge Train, 2013)). The increasing use of project management methods, especially de facto methods, may explain why project success rates have increased. Recent studies propose that project management methods are a critical success factor in project management, (Cooke-Davies, 2002; Lehtonen and Martinsuo, 2006; Loo, 2002) and therefore have a positive influence on project success. However, these studies do not explain how project management methods influence project success. A better understanding of this influence is necessary in order to understand whether the use of project management methods causes project success or whether the methods only contribute to project success. Therefore the goal of this research is to study how project management methods influence project success. The research question is:

How do project management methods influence project success?

The research question is answered by following the case study approach. The scope of the research is limited to six IT projects (cases) that involve incremental administrative innovative change. This means that the projects are only new for the client firm (not for the consultancy firm) and only involve technological discontinuity, e.g. installing a new website operating platform. For the client firm the new IT systems will be a major improvement of the current technology and will systematically change the current administrative processes. Additionally, the study only includes ‘de facto’ project management methods.

For both practitioners as well as for academics it is interesting to study the effect of project management methods due to the increasing popularity and use of the methods. From a managerial perspective it is interesting to study how the methods contribute to success and which factors and conditions need to be managed, because it will enable them to increase their project success rate. Additionally it will show the managers the pitfalls of the methods.

(5)

5 (2012) in that it focuses specifically on how effective project management methods are in generating project success, instead of just identifying the overall benefits of project management methods. In addition to that the paper explains why the methods are not a panacea for success and why the methods should be adapted in order to attain project success. This is in line with recent criticism on the Stage-Gate or phased project management methods, the authors argue that the methods are too bureaucratic, controlling and involve too much documentation which negatively influences project success (Becker, 2006; Lenfle and Loch, 2010).

In order to answer the main research question, the study first defines what project management methods, which success measures are used and what the critical success factors for project management are. In addition to that the study explains how project managers adapt the project management methods and why they adapt the methods. Therefore the following sub questions are formulated:

S1: What are project management methods? S2: What success measures are used for projects?

S3: How do project managers’ use the project management methods, when do project managers adapt the methods and why do they make these adaptations?

The remainder of this paper is structured as follows. First, a literature review discusses the current literature on project management to provide a background to the research topic. Second, the methodology section describes the research approach and the data collection method. After that, the analysis section describes how the data are analyzed. Fourth the results are described about the studied cases. Fifth, a conclusion section follows in which the research question is answered. After that a discussion section follows, including the limitations of this research and outlining the future research possibilities.

2. LITERATURE REVIEW

2.1 Projects and project management

Defining projects & project management

(6)

6 and a project can be considered to be “the achievement of a specific objective, which involves a series of activities and tasks which consume resources; that has to be completed within a set specification, having definite start and end dates” (Munns and Bjeirmi, 1996, p.81).

The definitions differ in their inclusion of the parameters cost, schedule and quality. Whether or not to include the parameters cost, schedule and quality can be explained by the fact that project success is often judged by these three factors, the so called “triple constraint”. However, research into the effect of these variables has found them to be incomplete (e.g. Kerzner, 2009; Pinto and Slevin, 1987). Therefore, for this paper the triple constraint variables are not included in the project definition. The definition used is that “a project is a unique temporary venture, involving a series of tasks which consume resources and with the objective of achieving a specific end goal conforming to agree upon specifications”.

Project management has been defined as the planning, controlling and monitoring of a project and the delegation of responsibilities within a project (Burke, 1993; Munns and Bjeirmi, 1996; Olsen, 1971; Hedeman, van Heemst and Fredriksz, 2009). Furthermore the definitions include that the final goal is to control and achieve the projects objectives. Therefore in this paper project management is defined as “the process of controlling, monitoring and planning all aspects of a project to achieve the projects objectives”.

2.2 Project Management Methods

Defining project management methods

A method is defined as “a structured approach for delivering a project that consists of a set of processes, with each process having clearly defined resources and activities” (Turner, 2000). A project managementmethod sets out what an organization regards as bestpractice; improves inter-organizational communication; and minimizes duplication of effort by having common resources, documentation and training (Clarke, 1999). The methods used for this study are ‘de facto ‘project management methods and based on the project life cycle that consists out of four phases: conceptualization, planning, execution and termination (Cleland and King, 1983, Pinto and Prescott, 1988).

The effect of Project Management Methods on Success

(7)

7 project management methods can positively influence project success. Recent research found that 52% of practitioners perceive project management methods as beneficial for their job and that with the use of project management methods satisfaction increases (Martijnse and Noordam, 2007; Wells, 2012). In addition using project management methods is found to be a critical success factor for projects (Cooke-Davies, 2002; Lehtonen and Martinsuo, 2006; Loo, 2002). Schmidt et al. (2001) even defined the lack of adequate project management methods as a major risk for project success. Furthermore, research found that during the different phases of the project life cycle different project tools, which derive from the methods, can be used and that the right selection of tools has a positive influence on project success (Besner and Hobbs, 2006; Patanukel et al. 2010). If tools positively influence project success, this may also apply for methods. However, White and Fortune (2002) found in their study several limitations with project management tool and methods. They found that 36 out of 122 mentioned limitations had to do with project management methods (in-house and de-facto methods), which is nearly 30%.

2.3 Project success

How to measure project success has been extensively discussed in the literature (Cooke-Davies, 2002; Davis, 2014; Pinto and Slevin, 1987; Pinto, 2010; Shenhar et al. 2001). There is however, no consensus as to when a project should be considered successful. Additionally, there are two streams of project success studies. Former research can be classified according to how to define project success (Atkinson 1999; Olsen, 1971) or what factors contribute to successful projects (Bakert et al. 1988; Might and Fischer, 1985; Shenhar et al. 2001; Slevin and Pinto, 1987). This section discusses project success and how it is defined in this paper, followed by a section which discusses ‘critical success factors’ necessary for project success.

Defining project success

(8)

8 (including payments to the project service firm). The last criterion is quality since every project needs to deliver a product or service that fulfills agreed upon requirements. The triple constraint has been criticized for being incomplete (Atkinson, 1999; de Wit, 1988; Pinto and Slevin, 1987). Scholars state that client acceptance should be included in the definition, because projects are developed with customers or clients in mind and that the purpose is to satisfy customer needs (Pinto, 2010; Pinto and Slevin, 1987). Others (Cooke-Davies, 2002; Munns & Bjeirmi, 1996; de Wit, 1988) argue for a differentiation between “project success” and “project management success”. Project management success can be measured based on the original triple constraint variables. Project success on the other hand, should be measured against the overall objectives of the project which have multiple stakeholders and change over the life cycle of a project (Cooke-Davies, 2002; de Wit, 1988). This research takes into account both measures, and measures project success based on project management success and perceived success in order to overcome the limitations of the triple constraint.

Critical success factors

Research on the critical success factors (CSFs) for project management is extensive (Bakert et al., 1987; Clarke, 1999; Cooke-Davies, 2002; Kerzner, 2009; Munns and Bjeirmi, 1996; Scott-Young and Samson, 2007; Slevin and Pinto, 1987; Young and Jordan, 2008; Young and Poon, 2013; White and Fortune, 2002), but has not yet resulted in an exhaustive list of factors.

When comparing the CSFs of several authors it is possible to identify common success factor groups, which need to be present in a project for it to be successful. Appendix A provides an overview of CSFs groups of several authors. This paper focuses on the list of CSFs developed by White and Fortune (2002), because the list is extensive, based on other lists from the literature and empirically tested using a questionnaire among 236 managers. The authors created a ranking by asking project managers what in their opinion are the three most important success factors. Table 1 shows the most frequently mentioned CSFs.

1. Clear goals and objectives 6. Effective leadership

2. Top management support 7. Flexible approach to change

3. Adequate funds and resources 8. Clear communication channels

4. Realistic Schedule 9. Taking account of past experience

5. End user involvement 10. Effective risk management

Table 1: Critical success factor ranking (based on White and Fortune, 2002)

(9)

(10)

10 project managers have to take advantage of their past experience when implementing a new project. Former research found that project team experience has a positive influence on staying on schedule, which is an indication that experience contributes to success (Scott-Young and Samson, 2008). The last factor, effective risk management, consists out of two phases risk assessment and risk control. Risk assessment is identifying, analyzing and prioritizing risks. Risk control includes risk management planning, risk resolution; risk monitoring planning, tracking and corrective action (Boehm, 1991). Following the risk management process (assessment and control) has a positive influence on a timely project delivery and thus contributes to project success (De Bakker et al. 2010; Cooke-Davies, 2000; McGrew and Bilotta, 2000; Ropponen and Lyytinen, 1997).

2.4 Contextual factors

The effect that project management methods have on project success is influenced by several contextual factors. This study limits itself by focusing on the factors: project size, complexity, technological newness, target volatility, and team size.

Project size

Project size involves project length and total budget. Project size negatively influences (on time) project completion (Martin et al. 2007; Sauer et al. 2007; Yetton et al. 2000). Additionally, two studies found that project size has a significant negative influence on budget adherence and project quality (Martin et al. 2007; Sauer et al. 2007). However, Yetton et al. (2000) found in their study no evidence (no significant effect) for the negative effect of firm size on budget adherence. Project size has a negative influence due to increasing complexity with larger projects and thereby also increasing project risk (Martin et al., 2007; Yetton et al., 2000). Additionally, larger projects often include more organizational units which results in ineffective communication, which negatively influences project success (Kraut and Streeter, 1995).

Team size

(11)

11 Technological novelty

Technological novelty is closely related to complexity and often included as a factor of complexity (E.g. Baccarini, 1996). Project risk increases when the implanted technology is new for both the client and supplier firm due to unexpected technical problems and unfamiliarity with the technology (McFarlan, 1981). In addition, research found that the greater the newness of the project to the organization (so the client firm) the more likely it is that project completion will be problematic and that the project goals will be redefined (Yetton et al, 2000).

Target volatility

Target volatility is defined as the number of changes to schedule, budget and project scope (Sauer et al., 2007). Target changes increase the risk of project underperformance (Engwall, 2002; Sauer et al., 2007). Sauer et al. (2007) found e.g. that a change in scope is associated with an increase in time of 0.6% and increase in budget spent of 1.1% (similar results were found for changes to schedule). So there exists a causal relationship between the three factors. What the authors not mention however is that changes to schedule, budget or scope may also happen in order to prevent failure or correct for failure which will increase volatility.

2.5 Research model

Based on the described theory, three propositions are developed that are used to answer the main research question. These propositions are:

Proposition 1: Project management methods have a positive influence on project success.

Proposition 2: Project management methods have a positive influence on project success, which is mediated via realizing the CSFs.

Proposition 3: Project size, team size, technological novelty and target volatility negatively influence project success.

(12)

12 preference of a project manager and therefore differs between project managers. Project management methods provide a structure which project managers can follow, but how they will lead a project is dependent on the personal preference and skills of the project manager. PRINCE2 acknowledges that it does not include leadership qualities (Hedeman, Heemst and Frederiksz, 2009, p.11). Therefore it is very unlikely that project management methods will have an influence on effective leadership and therefore this will not be further studied. Past experience is excluded, because experience results from past events and methods will not have an influence on this. Risk management is excluded because the methods describe risk management procedures, and therefore studying this is unlikely to result in new insights.

Figure 1: Research model

3. METHODOLOGY

This section describes research method followed. As not much is known about the effect of project management methods on project success, this paper performs qualitative research by doing a case study. The next section describes the case study approach in more detail.

3.1 Case study research

To answer the research question this study applies the case study approach. The advantage of using case studies is that it can offer important insights that other types of studies cannot provide (Yin,

Contextual factors: - Project size - Team size - Technological novelty - Target volatility Project management methods: - PRINCE2 - Scrum Project success: - Triple constraint - Perceived success Top management responsibilities

(Client firm):

- Top management support - Resources

CSFs controlled by project manager:

- Clear goals - Realistic schedule - End user involvement - Flexible approach to change - Clear communication channels

(13)

13 2014). This is due to the fact that with case studies it is possible to study ‘how’ or ‘why’ questions instead of effectiveness questions (Shavelson & Towne, 2002, pp 99-106). The reason for choosing a case study approach for this study is to find out how project management methods influence project success and to gain theoretical insights about the effects. In line with the features of the case study approach multiple sources of evidence are collected and propositions are developed (see section 2), which guide the data collection and analysis (Yin, 2014).

3.2 Case selection

This study selected the cases (projects) based on pragmatic grounds and only includes projects from TIC Consulting (Telecommunication and ICT consultancy). TIC has a leading role in the Netherlands in project management, for example it was the first firm that introduced PRINCE2 (a project management method) in the Netherlands. Furthermore, TIC makes use of a wide variety of proven project management standards adjacent to PRINCE2 (e.g. ITIL, MSP and M_O_R). Therefore TIC Consulting is a firm with substantive knowledge and experience with project management methods and the right firm to use for this research. The advantage of selecting a single versus multiple firms is that it reduces the heterogeneity of performance and implementation of the methods due to excluding company differences. This results in a better isolation effect of project management methods on success. Last, the cases within TIC provide sufficient variety to analyze the effect of the different management methods.

Six cases were selected for this research based on three criteria. First, it was necessary that a project management method was used during the project. Otherwise, it would not be possible to answer the research question. However, one case is included where no project management method was used in order to identify why project management methods may not be beneficial in all circumstances. Second, projects needed to have a project manager, since they are the users of project management methods. Lastly, the projects needed to be recent (< 5 years old), because project management methods are constantly updated and by focusing only on the most recent versions the results are better comparable across cases. Also project managers will have more vivid memories of recent projects, increasing the validity of the findings. Over time memories will be replaced by new experiences, making retrieval of the specific event more problematic (Stone et al, 2009, p. 36). Last, the case selection was dependent on the willingness of project managers to cooperate.

(14)

14 Key Case: TIC Consulting

The research took place at TIC Consulting, which is an IT Consultancy company based in the Netherlands with approximately 1100 employees. TIC Consulting is specialized in IT implementation, integration and transition projects and makes frequent use project management methods. TIC has an APMG license and is therefore certified to train project managers the methods of the Cabinet office.

3.3 Data collection

Two sources of data are used: internal documents and interviews. Internal project documentation

The analyzed documents are project initiation documents (PID), end reports, PowerPoint slides, meeting minutes and highlight reports. The PID describes the project plan, scope, method used, team composition, cost estimation and planning. The detail of information in the PIDs differs per project. Project end reports describe the results of the project and often include a section with learning points for future projects. The PowerPoint slides include start up presentations which include the same information as the project initiation documents or progress PowerPoint’s which provide information over the project during a specific stage of the project. Meeting minutes and highlight reports provide information on how the projects progress.

Interviews

Semi-structured Interviews were conducted with six project managers with various levels of experience. These project managers were selected, because of their expertise with project management methods. The interviews lasted approximately fifty minutes and involved six different projects. The interviews were held in Dutch and the answers were translated into English. Interview questions were developed upfront (see Appendix B). Based on the first interview some adjustments were made, because during the interview it became clear that target volatility played also a role during projects and therefore a question related to volatility was included. After analyzing the initial data all project managers were contacted again and asked to fill in an online questionnaire that included seven questions (6 scale questions and 1 open question, which are included in Appendix B). Measurement instruments

(15)

15 average score of 0.33 for each success measure, so that the sum of the score of the three measures (time, budget, quality) is 1 for complete project management success. The reason for scoring the three success measures is used to provide indicative case differences (see Appendix C)

Project size is measured based on the total budget of a project and on the duration of the project in months.

Technological novelty is measured based of the newness of the outcome of the project for the client firm, so whether the project has a small or large impact on the firm.

Target volatility is measured based on the number of changes to budget, schedule and scope (Sauer et al. 2007).

Team size is measured based on the number of people in the project team.

3.4 Case descriptions

Below the six cases used for this study are described, followed by table 2 that gives an overview of the project manager and project characteristics of the different cases.

Project A: Server migration

Project A was an internal TIC project that involved a change in responsibilities of two departments. Department X would remain responsible for the internal application management of TIC and department Y took over the technical management of the server environment of department X. The server environment of department X was transferred to the standard infrastructure of department Y. Therefore, the objective of the project was the migration of 2000 servers from department X to the environment of department Y. The servers were mainly TIC consulting servers with applications in use by TIC Consulting, but it also included servers that were in use by client firms of TIC Consulting. Project B: New Digital Workplaces

(16)

16 Project C: Product integration

The project took place at a funeral insurance company (FIC) and involved the integration of the two services. The project was external and internal driven. External drivers were new legislation and regulations implemented by the AFM (Autoriteit Financiële Markten: Authority for Financial Markets). This legislation put a strained on the planning of the project because the project had to be finished before the legislation would become of power. Internal drivers were cooperation (all clients needed to become members of the cooperation, which was not the case), accessibility, reliability and advice. The advantages of the service integration were meeting legislation, free product choice for clients, best advice for clients, all clients’ members of the cooperation, utilizing market opportunities and acceptable IT costs.

Project D: Service helpdesk

The project was an internal TIC project involving the replacement of the internal service helpdesk. Employees can contact the helpdesk with questions, problems, changes or incidents with e.g. their computer. The helpdesk would then contact the right supplier firm of the software or hardware (e.g. IBM). In the old situation the helpdesk service was provided by an external organization with 45 people working on it. The contract was cancelled and a new system was implemented. The new system is operated internally with only 9 employees working on it. The system has auto dispatch functionality, meaning that every problem, incident, change or question is directly transferred to the right external supplier organization. The new system saves TIC €200,000 a year.

Project E: Tridion upgrade

The project took place at a multinational paint, coatings and chemicals firm. The objective of the project was to upgrade Tridion, which is a web content management system. The aim of project was to have a stable, future-proof and user friendly environment. Reasons for the project were that the client firm was facing the following issues; supplier of Tridion did not provide support anymore for the version currently in use, potential reputational damage due to instable environment, non-technical users find the system to complicated and the firm wants to be able to use new functionalities available in the upgrade version.

Project F: Underpinning Contracts

(17)

17 set up these legal contracts (underpinning contracts). A second part of the project involved creating order in the invoice system of two firms; because during the project it became clear that this was a chaos (e.g. Clients would not receive an invoice or would receive it twice).

Project A B C D E F Project manager characteristi cs: Age 29 43 48 42 48 53 Years of experience 5 15 15 17 25 25 Certificates IPMA D PRINCE2 practitioner Scrum Master PRINCE2 practitioner N/A IPMA C (expired) IPMA C IPMA B (in progress) PRINCE2 practitioner N/A Project Characteristi cs: Project Size: Budget (€) Duration (months) Moderate: 850,000 12 Large: 3,000,000 27 Moderate: 800,000 10 Moderate: 900,000 12 Small: 99,656 5 Small-Moderate: 475,000 7

Team Size Moderate:

14 Large: 19 Large: 20 Moderate: 15 (3 fulltime) Small: 6 Small: 5 Project management method

PRINCE2 PRINCE2 Scrum PRINCE2 PRINCE2 none

Project management success

0.67 1 1 1 1 1

Perceived project success Moderate:

Satisfied project manager and client, however disappointment about the delay

High: Satisfied project manager and client High: Satisfied project manager and client High: Satisfied project manager and client High: Satisfied project manager and client High Satisfied project manager and client Impact on Organization Number of Specialists External parties High: Organization wide 3 5 High: Organization wide 5 (All IT related) 5 High: Organization wide 3 5 High: Organization wide 5 6 High: Organization wide 3 1 Moderate: 2 Divisions 3 120 Technological novelty Low-Moderate:

Only new hardware High: Extremely new, unique in the Netherlands Low-Moderate: Only change in system Moderate: A new automated system, with minimal impact on employee. For clients it was a big change High: Complete new system with new hardware and services. Staff training was necessary Moderate:

(18)

18 Table clarifications

Table 2 includes terms that need clarification. First, the certificates include project management method related certificates and internationally recognized professional project manager certification (IPMA). PRINCE2 practitioner and Scrum Master are certificates that are related to the project management methods. For PRINCE2 there exist three types of certificates: foundation, practitioner and professional and these can only be obtained in chronological order.

For Scrum the certificates are related to the different roles included in the method. The IPMA (International Project Management Association) certifications include four levels A-D (A= low, D=high). These are qualifications that refer to the general skills of a project manager and are not related to any of the methods. Lastly, the certificates may expire if no reexamination is done.

For project management success it should be taken into account that projects A, B and D may seem completely successful, but all included initial budget overruns. Project B was also over schedule. However, these time and budget overruns were discussed with and accepted by the client firm.

3.5 Project management methods being studied

The two project management methods that are studied are PRINCE2 and Scrum. PRINCE2 is a generic structured project management method based on best practices that can be applied to any type of project (Heedeman and Segers, 2009). Its success depends on its adaption to the project type and project environment (Hedeman et al. 2009). The method consists of seven principles (tailor to suit project environment, continued business justification, learn from experience, focus on products, manage by exception, manage by stages and define roles and responsibilities), seven themes (business case, organization, quality, plans, risks, change and progress) and seven processes (starting up, directing, initiating, controlling, managing product delivery, managing stage boundary and closing a project). The seven principles always need to be applied in order to call it a PRINCE2 project. However, themes and processes can be combined or integrated depending on the project. E.g. for a small project the starting up and initiation processes can be combined. In addition to that the different themes and processes include different management products e.g. PID (project initiation document), risk log, daily log and communication management strategy, and it is up to the project manager to decide which products to use and how to adapt them to the situation. In total the method includes 26 management products that all require documentation.

(19)

19 scrum is described as management-as-organizing, which means that action follows from the situation and is not predetermined (Koskela and Howell, 2002).

3.6 Data analysis

For the data analyses I relied on the underlying theoretical propositions to organize the analysis (Yin, 2014). The second technique used is cross-case synthesis (Yin, 2014); where the individual cases were compared in order to identify patterns in the data. To simplify the cross-case synthesis interview write ups were made in Microsoft Office Excel grouped per question, in order to create a clear overview. This resulted in an iterative process in which I tried to identify the similarities and differences between the interviews, by using the CSFs as the leading factors. So I went through the data searching for information related to e.g. clear goals, schedule etc.

4. RESULTS

4.1 The effect of project management methods

How do project management methods influence project success

This section first explains how project management methods influence perceived success. Followed by explaining how the methods influence the CSFs realistic schedule, end user involvement, and communication, and why the influences of the methods on the CSFs clear goals and flexible approach to change are limited. Moreover it is explained that success is largely dependent on how the methods are applied. The methods are a tool that can contribute to success, but when applied in the wrong way they may even have a negative effect.

Perceived success (client satisfaction) improved due to the use of the methods in two ways. According to project manager C satisfaction improved, because the use of Scrum improved the collaboration and cohesion between the IT department and commercial departments. Second, client satisfaction improved according to project manager E, because the client firm constantly knew how the project was progressing. This was due to the go/no go moments and the product breakdown structure (PBS) that is part of the methods. Although, for both examples it can be argued that the improved client satisfaction resulted from clear communication with the client, and so via the CSF clear communication contributes to perceived project success.

(20)

20 Clear goals – The influence of project management methods on the formulation of clear goals is limited. Project managers A, D, and E stated that the project goals were already defined when they started the project. They indicated that the goals are often provided to them based on negotiations between the client firm and TIC , so this is not controllable by the project manager. This indicates that it is not only possible to set up clear goals without using the project management methods, but also more common to do so. This was also the situation in the Scrum project, where goals also were established without using the methods. Although, according to the methods (PRINCE2) project managers need to develop a project plan (PID) in which they clearly describe what the objective of the project is, and what is in and out of scope. Therefore, according to project manager A they have to fine tune the predefined project goals and describe them. Thus, project management methods stimulate the formulation of clear goals; however in practice goals are often predetermined without using the methods.

Realistic schedule – There are indications that both methods have a positive influence on the development of a realistic schedule, and possibly via a realistic schedule have a positive influence on project management success. In appendix C it can be seen that five of the six projects were delivered on time. This indicates that the methods do seem to have a positive effect on project management success. The two methods provide different tools that help with the development of a realistic schedule. PRINCE2 includes the product break down structure and according to project managers B and D this helps them with creating a realistic schedule, because they can now define the time necessary per subpart, resulting in a more precise planning when adding all the parts. With Scrum, planning is a monthly and daily activity, due to these short periods it helps to create realistic schedules. Project manager C stated that creating a planning is part of the Scrum method and that the method helps with prioritizing the actions that are necessary.

(21)

21 Flexible approach to change – There are no indications that the project management methods (PRINCE2 or Scrum) have an influence on a flexible approach to change at all (positive or negative). This depends on the skills of the project managers and his ability to deal with changes. While the methods may not affect a flexible approach to change, according to project manager D it is crucial for project success to deal with and adapt the project to changing circumstances. The use of project a management method (especially PRINCE2) may even reduce the flexibility a project manager has due to the fact that he will need project board approval if solving a situation will for example influence the triple constraint variables. However, no evidence was found in support of this. In addition to that Scrum is more flexible then PRINCE2, because for Scrum project goals and planning or only determined for a maximum of one month, while for PRINCE2 this is done at the start of the project for the whole project. This makes it easier to e.g. adapt project goals.

Communication – The methods have a positive influence on communication, because the methods create a common language, and include “tools” (reports) that improve communication. According to project managers A, B, D, and E the methods create a common language for all the project members, because everybody is familiar with and knows the content of the methods which increases understanding and improves communication. For example project manager E compared it with traffic, he said that when driving onto and intersection everybody knows that traffic coming from right may go first, there is common understanding. According to him this is the same with the methods, because all project members know the steps to be taken. This is supported by Wells (2011) who also found that an advantage of using the methods is a unified language. The second positive influence is that the methods include documentation (PID, highlight reports and exception reports) that improve communication. According to project manager A, B and E with the PID you clearly communicate what the project is going to look like, what the planning is; role division and what will be delivered. With the highlight reports it is communicated to the client how the project is progressing. Exception reports are used to communicate to the project board that the client firm wants the project objectives to be adapted (broader project scope), this needs to be approved by the board. So these reports clearly improve the communication during a project and in combination with the created common language the methods positively influence communication. These results are in line with the definition of Clarke (1999) that project management methods improve inter-organizational communication.

Disadvantages of using project management methods

(22)

22 products included in the method. Managers A, B, and D indicated that when the method is followed dogmatically, all steps are taken, all themes are included and every project management product (like e.g. a risk log, daily log, issue register or configuration management strategy) is used or executed a project will probably fail. Failure will occur because in projects there is not enough time for executing all the small steps included in the themes and writing all the documentation included in the project management products, resulting in projects that run over budget and over schedule. Project manager D stated the following “when the methods were just introduced a lot of projects went controlled into failure, due to dogmatically following the PRINCE2 guidelines”. However an advantage of PRINCE2 is that it includes sufficient flexibility to adjust it to different contextual backgrounds and to the personal taste of the project manager. How managers adapt the methods is discussed in the next section.

Finally, project manager F does not use the methods at all, because he believes that project management methods do not work in too complex situations. He states that when it is uncertain how long it will take to realize a certain step, because of e.g. lack of information it is impossible and useless to make a project plan. This is because you do not know what exactly needs to happen, how the current situation is and when you will be finished. However, this only applies to projects where there is a lack of information and a high level of uncertainty.

How do project managers adapt the methods

(23)

23 managers differ in the extensiveness of their PIDs, Project manager D even stated that often he does not write a PID at all anymore and just make a PowerPoint presentation in which he describes the same information. Adaptation especially takes place with regard to documentation being made, e.g. PRINCE2 describes that an extensive communication strategy needs to written down and documented, while in reality a communication strategy is often described in a few sentences in the PID. Another example is that PRINCE2 describes that in order to write a planning a product flow and product breakdown structure has to be used, while a manager may only use one (or none). Project managers also differ in the extent to which a risk log is kept or activity log is kept. So overall the project managers adapt the methods by only including those management products they find effective and by adapting (not writing) all the documentation PRINCE2 prescribes. What is consistent in every project is the use of the PID (except project manager D, then in other projects), highlight reports, exception reports and project end reports.

The results regarding Scrum should be interpreted with caution, because it is only based on one case. In project C Scrum was adapted substantially by including a project manager and steering committee while, this is not part of the method at all. This was done so that the project manager could deal with the governance regarding the project and the contact with the steering committee. In addition to that an extensive project plan was written similar to a PID, while this is not part of the method. The PID was written for clarity regarding what the project goals were, how communication would go, to describe what Scrum entails and what would be the role and responsibilities of the project team members.

4.2 The importance of the critical success factors

This section describes the results of the role of the CSFs and adds additional factors found by this research. Table 3 provides an overview of the CSFs studied and their importance.

(24)

24 methods are only of moderate importance (M=4.33) in comparison to the other CSFs. This could be an indication that project management methods may contribute to success, but are not a cause for success.

Project:

A B C D E F

How important is/are1:

Mean (M) and Standard deviation (SD)

Clear goals 4 5 5 5 5 5 4.83 (.41) Very high

End user involvement 5 4 5 5 5 5 4.83 (.41) Very high Top Management support 4 5 5 5 5 4 4.67 (.52) High

Team composition 5 4 5 4 5 5 4.67 (.52) High

Clear communication channels 5 5 5 3 5 5 4.67 (.82) High project management methods 5 5 5 4 4 3 4.33 (.82) Moderate Dedication project members 5 4 4 3 5 5 4.33 (.82) Moderate Adequate funds2 and resources 5 4 5 4 - 3.53 4.30 (.62) Moderate Effective Leadership 4 4 5 3 4 5 4.17 (.75) Moderate

Past experience 3 4 4 5 5 4 4.17 (.75) Moderate

Realistic schedule 4 4 4 4 5 2 3.83 (.98) Low

Effective risk management 4 4 4 2 5 4 3.83 (.98) Low Flexible approach to change 4 3 4 3 3 5 3.67 (.82) Low

Team Size 1 1 2 1 2 5 2.00 (1.55) Very low

1_{The scores are on a scale of 1-5, one being unimportant and five being extremely important.} 2_{Project manager E stated that it was impossible to answer the importance of sufficient}

resources on a scale of 1-5, because projects are started based on a contract agreement with a client. Whether the budget or time that has been agreed upon is enough, is unknown.

3 _{Project manager made a distinction between the importance of time (3) and other}

resources (4) therefore average of 3.5

(25)

25 According to the six project members there are additional factors not mentioned by the CSF literature important for realizing project success:

Atmosphere within the project team.

Clear role separation between the assigner (client) and assignee (parent firm), in order to create a distinct separation of tasks.

A clear sense of urgency at the client firm, important because according to project manager D, without a sense of urgency for the project within the client organization, the likelihood of project failure is high due to lack of commitment and cooperation. This is supported by Kotter (1995) who argues that in organizations involved in change almost 50% fail due to a lack of sense of urgency within the firm for the change.

Client firm awareness: meaning that management of the client firm needs to create awareness within the client firm for the project and explain to its employees why the project is taking place and that their cooperation is expected from them when asked by the project manager.

Expectations management, which means that the project manager needs to balance the needs and requirements of the client with the time and budget allocated to the project. If done incorrectly the client firm will not be satisfied (perceived unsuccessful project). This also supported by Khazanchi and Reich (2008) who stated that “failure to manage end-user expectations’ is enormously important for all types of projects”. In addition a lack of expectations management has been defined as one of three major risks (others were no project management method, turbulence business environment) of project management (Schmidt et al. 2001).

4.3 The influence of the external factors on objective project success

Project size

(26)

26 who found no significant effect for project size on budget adherence, but did find a negative significant effect of project size on timely completion of a project.

Team size

In table 3 team size has the lowest score (M=2.00) and there are no significant differences in performance and team size when comparing the six projects. Therefore team size seems of lesser importance. Project manager A, B, D, and E all stated that team size is dependent on the project and the activities necessary for the project, but that it has no effect on project success. Project manager E said that sometimes you will need 10 people for 100 hours of work due to different type of tasks, while for other projects 1 person may fulfill the 100 hours. Project manager D said that he worked with teams of 300 people and teams of 5, but that the impact of the projects was the same. So the effect of team size on success seems to be rather limited. An explanation for the fact that size does not seem to have an effect in the projects discussed here may be that the projects all have rather small teams (see table 2), according to Sauer et al. (2007) the risk of underperformance does not become that high (max. 33%) when team consist out of less than 20 persons.

Technological novelty

Project B and E are the projects that involved the newest technology. In project B this was one of the reasons why the project was delayed, due to unforeseen bugs and constant provider updates of the new system. This indicates that technological novelty has a negative influence on project management success. However in project E, there are no indications that the newness of the technology contributed to the budget overruns. Indeed the project budget overruns were caused by an error at the start of the project, where the offer made to the client was based on a rough estimate of a technical consultant instead of being based on product breakdown structure, resulting in budget overruns. In addition the budget was fixed therefore could not be renegotiated with the client. So project B provides indications suggesting that technological novelty may negatively influence project management success, however as discussed above this may also be due to the size of the project or a combination of both.

Target volatility

(27)

27 Project F was a different kind of project with no upfront budget or planning, due to a lack of information it was not possible to make a realistic schedule. Budget and planning were constantly adapted, based on new information. Therefore changes occurred more often, but the project was a complete success and so the changes do not seem to have had any effect. Therefore the results regarding the influence of target volatility are rather inconclusive and are probably limited. Projects C, D and E were all complete successes, while also including volatility. It seems that volatility is just part of project management, and that when these changes are made with agreement of the client they have no negative effect.

5. CONCLUSION AND DISCUSSION

(28)

28 chance of projects being finished on time and therefore influences project management success. On time project delivery also contributes to client firm satisfaction and therefore contributes to perceived project success.

The list of CSFs (table 3) shows similarities and differences with the work of White and Fortune (2002). The ranking in table 3 is similar to the ranking of White and Fortune (table 1), when excluding the four additional factors (project management methods, team dedication, size and composition). However, a remarkable difference is that realistic schedule is identified as less important and end user involvement is found to be more important. These differences may be explained by the fairly small sample size of this study. Also I found that project management methods only had a moderate mean in comparison to the CSFs, which may indicate that project management methods contribute to project success but are not more important than e.g. clear goals or end user involvement. Furthermore, I found additional CSFs not mentioned by the CSF literature that are important for project success. These are project atmosphere, clear separation of tasks between client and parent firm, clear sense of urgency and the client, client firm awareness and expectations management. Especially, expectations management was mentioned by all project managers to be crucial for project success. When comparing objective and subjective success, there is one factor that determines whether the project will be perceived successfully by the client and that is “expectations management”. This means that the project manager has to assure that the client will have the right expectations of what the project outcome. So discussing with the client what is most important schedule, budget or quality and emphasizing as a project manager what then the possibilities are and what the project deliverables will be, in order to prevent disappointment.

(29)

29 time and is too difficult in complex projects where there is a lack of information at the start of the project. Moreover, developing a plan will create client expectations that may be impossible to fulfill when the situation becomes clearer.

This study contributes to the literature by finding that project management methods influence project success via the CSFs communication, end user involvement and schedule. Furthermore it adds to the literature that using project management methods is likely to lead to improvements in project management success and perceived project success and that the successful implementation of project management methods is strongly dependent on the experience of the project manager. In addition it adds to the literature that project management methods will only lead to project success when the methods are applied pragmatically. Last, it adds to the theory of CSFs that expectations management, project ambiance, sense of urgency, client firm awareness and a clear separation between the client and parent firm are critical for project success.

Managerial implications

Project management methods are tools that can contribute to success, but when badly managed and wrongfully implemented the methods will be ineffective and may even lead to negative results. It is therefore crucial for project managers to spent sufficient time on adapting their methods to the project context. In addition the methods have a positive influence on communication, end user involvement and realistic schedule and therefore the methods will be especially beneficial to use in projects where these three factors play a critical role. Moreover it is important that managers create a positive ambiance in the project team, create client awareness, create a clear separation between the client and parent firm and manage the expectations. It is crucial that project managers discuss with the client which project objective has priority, so whether schedule, quality or budget is more important in order to prevent client dissatisfaction. Last, it may be wise to assist young project managers in the adaptation of their project management methods, to prevent mistakes and project failures.

Limitations and future research

(30)

30 characteristics of the methods contribute to success. Moreover, this study did not include the effect of team composition while there are several studies that found evidence that team compositions have an effect on success (e.g. Steahle, 1999; Roosevelt Thomas, 2001 and Katzenbach and Smith, 1993). Also the role of the project manager and his/her capabilities are not included in this research, while Cheng et al. (2005) and Turner and Müller (2005) found that the capabilities of a project manager and his leadership style are important project success factors.

Future research should include more methods and study a larger sample size, including several industries and multiple companies to provide more support for the effect of project management methods on project success. Additionally, future research should include project members and client firm representatives; and include team composition and the role of the project manager. This could indicate the importance of project management methods for project success in relation to the CSFs. Finally, future research should also take into account technological novelty and target volatility, our research only provided partial support for the effects of novelty and volatility. Future research could try to identify how it influences success or how it increases the importance of other CSFs.

Acknowledgements

(31)

31

REFERENCES

Aladwani, A.M. (2002). IT project uncertainty, planning and success. An empirical investigation from Kuwait. Information Technology and People, 15(3), 210-226.

Atkinson, R. (1999). Project Management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management. 17(6), 337-342.

Avots, I. (1969). Why Does Project Management Fail? California Management Review, XII(1), 77-82. Baccarini, D. (1996). The concept of project complexity – a review. International Journal of Project Management, 14(4), 201-204.

Bakert. B.N., Murphy, D.C., and Fisher, D. (1983). Factors Affecting Project Success. Project Management Handbook. New York: Van Nostrand Reinhold Co.

Bakert, B.N., Murphy, D.C., Fisher, D. (1988). Factors Affecting Project Success. In D.I. Cleland and W.R. King (Eds.). Project Management Handbook (2nd ed.).New York: John Wiley and Sonc, Inc. Bakker, K. de, Boonstra, A., and Wortmann, H. (2010). Does risk management contribute to IT project success? A meta-analysis of empirical evidence. International Journal of Project Management, 28(5), 493-503.

Becker, B. (2006). Rethinking the Stage-Gate process – A reply to the critics. Accessed on April 13, 2014, Management Roundtable: http://www.pd-advantage.com/images/RethinkingtheStage-Gate_Process_AReplytotheCritics.pdfa

Belassi, W., and Tukel, O.I. (1996). A new framework for determining critical success/failure factors in projects. International Journal of Project Management, 14(3), 141-151.

Besner, C., and Hobbs, B. (2006). The Perceived Value and Potential Contribution of Project Management Practices to Project Success. Project Management Journal, 37(3), 37-48.

Boehm, B.W. (1991). Software risk management: principles and practices. IEEE Software 8(1): 32-41. Buchanan, D.A. and Boddy, D. (1992). The expertise of the Change Agent: Public Performance and Backstage Activity. London: Prentice Hall.

(32)

32 Cheng, M., Dainty, A.R.J., and Moore, D.R. What makes a good project manager? Human Resource Management Journal, 15(1), 25-37.

Cooke-Davies, T. (2000). Towards Improved Project Management Practice, Uncovering the Evidence for Effective Practices through Empirical Research (diss.): Dissertation.com.

Cooke-Davies, T. (2002). The “real” success factors on projects. International Journal of Project Management, 20(3), 185-190.

Cooper, R.G. (1994). Perspective third-generation new product processes. Journal of Product Innovation Management, 11(1), 3-14.

Cooper, R.G. (2014). What’s Next? After Stage-Gate. Research Technology Management, 57(1), 20-31.

Clark, A. (1999). A practical use of key success factors to improve the effectiveness of project management. International Journal of Project Management, 17(3), 139-145.

Cleland, D.I., W. R. King. (1983). Systems Analysis and Project Management. New York: McGraw-Hill. Curral, L.A., Forrester, R.H., Dawson, J.F., and West, M.A. (2001). It’s what you do and the way that you do it: Team task, team size, and innovation-related group processes. European Journal of Work and Organizational Psychology, 10(2), 187-204.

Davis, K. (2014). Different Stakeholder groups and their perceptions of project success. International Journal of Project Management, 32(2), 189-201.

Dvir, D., Raz, T. and Shenhar, A.J. (2003). An empirical analyses of the relationship between project planning and project success. International Journal of Project Management, 21(2), 89-95.

Eisenhardt, K.M. (1989). Building Theories from Case Study Research. The Academy of Management Review, 14(4), 532-550.

Engwall, M. (2002). The futile dream of the project goal. In K. Sahlin-Anderson and Søderholm (Eds.). Beyond project management: New perspectives on the temporary-permanent dilemma. Copenhagen: Copenhagen Business School Press.

Fortune, J., White, D., Judgev, K., and Walker, D. (2011). Looking again at current practice in project management. International Journal of Project Management, 4(4), 553-572.

(33)

33 Frame, J.D. (1995). Managing Projects in Organizations, 2nd ed. San Francisco: Jossey-Bass.

Frame, J.D. (2003). Managing Projects in Organizations – How to Make the Best Use of Time, Techniques and People, 3e ed. San Francisco: Jossey-Bass.

Fiedler, F.E. (1965). Engineer the Job to Fit the Manager. Harvard Business Review, Sept-Oct, 115-129.

Flyvbjerg, B. (2007). Megaprojects and risk. Cambridge: Cambrigde Universtiy Press.

Grau, N. (2013). Standards and Excellence in Project Management – In Who Do We Trust? Procedia – Social and Behavioral Sciences, 74, 10-30.

Hedeman, B., van Heemst. G.V., Fredriksz. (2009). Project Management based on PRINCE2TM 2009 edition. Zaltbommel: Van Haren Publishing.

Heegemans, B., and Seegers, R. (2009). PRINCE2TM 2009 edition A pocket Guide. Zaltbommel: Van Haren Publishing.

Ives, B., and Olson, H. (1984). User involvement and MIS success: A review of research. Management Science, 30(5), 586-603.

Johannessen, J., and Olsen, B. (2011). Projects as communicating systems: Creating a culture of innovation and performance. International Journal of Information Management, 31(1), 30-37.

Katzenbach, J.R., and Smith, D.K. (1993). The Wisdom of Teams: Creating the High-Performance Organization. Boston: Harvard Business School Press.

Kerzner, H. (2009). Project Management A Systems Approach to Planning, Scheduling, and Controlling (10th ed). New York: John Wiley and Sons, Inc.

Khazanchi, D., and Reich, B.H. (2008). Achieving IT project success through control, measurement, managing expectations, and top management support. International Journal of Project Management, 26(7), p.699.

Kirsch, L.J., Sambamurthy, V., Ko, D., and Purvis, R.L. (2002). Controlling Information Systems Development Projects: The View from the Client. Management Science, 48(4), 484-498.

(34)

34 Kotter, J.P. (1995). Leading Change: Why Transformation Efforts Fail. Harvard Business Review, 73(2), 59-68.

Kraut, R.E., and Streeter, L.A. (1995). Coordination in software development. Communications of the ACM, 38(3), 69-81.

Lehtonen, P and Martinsuo, M. (2006). Three ways to fail in project management and the role of project management methodology. Project Perspective annual publication of International Project Management Association, 28, 6-11.

Lenfle, S. and Loch, C. (2010). Lost Roots: How project management came to emphasize control over flexibility and novelty. California Management Review, 53(1), 32-55.

Loo, R. (2002). Working towards best practices in project management: a Canadian study. International Journal of Project Management, 20(2), 93-98.

Knowledge Train Ltd (2013). What is PRINCE2 certification. Accessed on February 27, 2014. Virtual project consulting: http://www.virtualprojectconsulting.com/what-is-prince2-certification.

Martijnse, N., and Noordam, P. (2007). Projectmanagement: Lessen uit Falende en Succesvolle ICT-Projecten: Controllers moeten vanaf de start een rol spelen. MCA, 1, 36-45.

Martin, N.L., Pearson, M.J. and Furumo, K. (2007). IS Project Management: Size, Practices and the Project Management Office. Journal of Computer Information Systems, 47(4), 52-60.

McFarlan, F.W. (1981). Portfolio approach to information systems. Harvard Business Review, 59(5), 142-150.

McGrew, J.F., and Billotta, J.G. (2000). The effectiveness of risk management; measuring what didn’t happen. Management Decision, 28(4), 293-300.

Monteiro de Carvalho, M. (2014). An investigation of the role of communication in IT projects. International Journal of Operations & Production Management, 34(1), 36-64.

Morris, P.W., and Hough, G. H. (1987). The Anatomy of Major Projects, New York (1987): John Wiley and Sons.

(35)

35 Munns, A.K., and Bjeirmi, B.F. (1996). The role of project management in achieving project success. International Journal of Project Management, 14(2), 81-87.

Patanukel, P. Lewwongcharoe, B., and Milosevic, D. (2010). An empirical study on the use of project management tools and techniques across project life-cycle and their impact on project success. Journal of General Management, 35(3), 41-65.

Pinto, J.K. and Slevin, D.P. (1987) Critical Success Factors in Effective Project Implementation., Project management handbook (2nd ed.) New York: Van Nostrand Reinhold.

Pinto, J.K., and Prescott, J.E. (1988). Variations in Critical Success Factors Over the Stages in the Project Life Cycle. Journal of Management, 14(1), 5-18.

Pinto, J.K. (2010). Project Management Achieving Competitive Advantage (2nd ed.). New Jersey: Pearson Education, Inc.

Roosevelt Thomas, R. (2001). Management of Diversity. Wiesbaden: Gabler.

Ropponen, J., and Lyytinen, K. (1997). Can software risk management improve system development; an exploratory study. European Journal of Information Systems, 6, 41-50.

Sauer, C., Gemino, A., Horner Reich, B. (2007). The Impact of Size and Volatility on IT Project Performance studying the factors influencing project risk. Communications of the ACM, 50 (11): 79-84.

Scott-Young, C., and Samson, D. (2008). Project success and project team management: Evidence from capital projects in the process industries. Journal of Operations Management, 26(6), 749-766. Schmidt, R., Lyytinen, K., Keil, M., and Cule, P. (2001). Identifying Software Project Risks: An

International Delphi Study. Jounal of Management Information Systems, 17(4), 5–36.

Schwaber, K., and Beedle, M. (2002) Agile Software Development with Scrum. Upper Saddle River: Prentice Hall, p. 158.

Shavelson, R., and Towne, L. (2002). Scientific research in education. Washington, DC: National Academies Press.

The effect of project management methods on project success

Master Thesis Business Administration – Strategic Innovation Management