Lean Six Sigma project selection based on Multi-Criteria-Decision- Making theory

(1)

Lean Six Sigma project

selection based on

Multi-Criteria-Decision-Making theory

A case study at Ziggo N.V.

Master thesis, Technology Management

University of Groningen, Faculty of Economics en Business

Author:

A.J. Mooij

Student number:

s2025345

(2)

(3)

Preface

This thesis was written to complete my Master degree in Technology and Operations Management at the University of Groningen. With much pleasure I have conducted this research at Ziggo N.V. in Utrecht where I especially enjoyed the good company of the employees of the Operations Competence Centre.

First, I would like to thank Mariëlle Weijters for giving me the opportunity to perform my research at Ziggo and provide me with feedback and guidance. Furthermore, I would not have succeeded this research without the help of my colleagues, especially Greet van den Braak.

I would like to thank my supervisors dr. H. Broekhuis and dr. N.D. van Foreest for their useful feedback and comments during this graduation process.

Finally I would like to thank all the respondents who participated in the surveys and all the employees who freed their time for the interviews.

(4)

(5)

Abstract

The objective of this research is to design a structured decision process for project selection for Lean Six Sigma (LSS) at Ziggo N.V. (one of the largest telecommunications companies in the Netherlands). LSS is an increasingly popular process improvement methodology. This methodology was implemented by Ziggo N.V. in June 2011 as part of a strategic program to improve the processes of the Technological cluster (TC). Since LSS is a project-based methodology, selecting the right improvement projects is essential to the early success and long-term acceptance within an organization (Adams et al., 2003; Antony, 2004). Project selection refers to the process of choosing the best among alternative proposals so that the objectives of the organization will be achieved (Hsieh et al., 2012).

By building on academic theory and previous experiences with LSS project selection in the organization, a suited LSS project selection process is designed in this research. This design is based on Multi-Criteria Decision-Making (MCDM) methods because of the many criteria that can be applied and the complexity of the decision-making problem of LSS project selection. MCDM methods are suited since these assist in reaching critical decisions that cannot be made straightforwardly (Lin, Lee, & Wu, 2009). The MCDM selection procedure for Ziggo consists of the following steps:

1) Identify improvement opportunities

Improvement opportunities are deducted from day-to-day business by using strategic input from the performance measurement systems available in the departments. A line-manager is responsible for the collection of project ideas and can apply various techniques such as Brainstorming, focus groups etc. The result of this step is a long-list of candidate projects.

2) Select the best alternative

After a pre-selection and prioritization based on strategic relevance and compatibility with the LSS methodology a short-list of candidate projects is evaluated using a fuzzy multi-rater MCDM tool developed in this research by combining the approaches of Wang and Chen (2008) and Chang et al. (2008).

(6)

(7)

1. Introduction

Organizations face increasingly global and competitive environments which require a firm to identify new opportunities and sustainable excellent performance. A strategy for companies to avoid competitive shortcomings is to increase customer satisfaction and company performance by eliminating operational inefficiencies using process-quality improvement techniques. Lean Six Sigma (LSS) is the latest generation of these process-quality improvement methodologies (Snee, 2010)(see Appendix I for a description of LSS).

Despite the broad popularity of LSS, it seems that many programs still fail to deliver on expectations at an organizational level (Chakravorty, 2009). Antony and Banuelas (2002) and Banuelas, Coronado and Antony (2002) presented the key ingredients for the effective introduction and implementation of Six Sigma in UK manufacturing and services organizations as the following:

 Management commitment and involvement.

 Understanding of Six Sigma methodology, tools, and techniques.  Linking Six Sigma to business strategy.

 Linking Six Sigma to customers.  Project selection, reviews and tracking.  Organizational infrastructure.

 Cultural change.

 Project management skills.  Liking Six Sigma to suppliers.  Training.

Among these critical success factors, selecting the right improvement projects is essential to the early success and long-term acceptance within an organization (Adams et al. 2003; Antony, 2004). Project selection is the process of evaluating individual projects or groups of projects, followed by making a choice to execute a subset of them, so that the objectives of the organization will be achieved (Padhy and Sahu, 2011; Hsieh et al., 2012) and is one of the most critical and challenging activities faced by Six Sigma companies (Chaki, 2004; Pande et al., 2000; Snee, 2010). “For drawing on both lean and Six Sigma as effectively as possible – that is, to achieve and sustain desired improvements – nothing is more critical than project selection” (Snee, 2010).

It is paradoxical that LSS, being a highly structured methodology, often relies on unstructured approaches and past managerial experience for project selection (Yusof an Aspinwall, 2000). This leads to lost improvement opportunities (Carazo et al., 2010) and employee frustration (Pande et al., 2000). There has long been a call in literature for a more objective approach to directing process improvement at an organization or enterprise level (Snee and Rodenbaugh, 2002; Garvin, 1993). While Six Sigma research suggests a number of project selection approaches, these approaches usually provide only general guidelines for project selection (Antony 2006). Researchers consider project selection as a universal challenge for Six Sigma (Antony et al. 2007; Heckl et al. 2010).

This research will be conducted at Ziggo N.V., the largest network communication company in the Netherlands. At Ziggo N.V. a suited research environment has been found where project selection for LSS is a relevant subject. Various Greenbelts (LSS improvement specialists) indicated that they think that the problems they have experienced in their projects could have been obviated if project selection would have taken place more accurately. A Greenbelt at Ziggo stated the following: “Projects have been selected of which I could already predict their failure” another Greenbelt confirmed the relevance of project selection: “I think that project selection should be executed with more strict criteria, and that evaluating these should be linked to actual consequences, this has not yet been the case here at Ziggo”. Therefore the objective of the research is to:

(10)

The selection process consists of two phases: the ‘identification of improvement opportunities’ and the ‘selection of the best alternative’ and results in the assignment of one project per Greenbelt.

This report is structured as follows:

 The next chapter will present the theoretical framework and research questions.  The methodology of this study will be explained in chapter 3.

 A description of how project selection for LSS has occurred previously at Ziggo, and the requirements posed on a selection process by the organization are discussed in chapter 4.  The degree to which these requirements were met is analyzed in chapter 5.

 A suggested redesign and description of the implementation context are described in chapter 6.

 This is followed by an in-depth description of the suggested project selection tool in Chapter 7.

(11)

2. Literature review

In the introduction it is argued why the selection of appropriate improvement projects is important for organizations that perform LSS projects. The following chapter provides a description of the process steps, criteria and structural elements that make up an LSS project selection process. After this review, the research questions are developed which lead to the design of a structured decision process for LSS project selection at Ziggo.

2.1 Project selection process

A systematic approach for LSS project selection should incorporate the appropriate selection criteria applied by decision makers in a controlled decision-making process. Multi-Criteria Decision Making (MCDM) methods seem suited since these assist in reaching critical decisions that cannot be made straightforwardly (Lin, Lee, & Wu, 2009). MCDM methods are mainly used in operations research and aim at supporting decision makers (DM) in structuring and solving decision problems with multiple criteria. Often no unique optimal solution exists for such problems (Dyer et al., 2008). Formal MCDM techniques seek to make a DM’s search as effective and efficient as possible, thereby attempting to objectify a subjective process (Stewart, 1992). A number of authors have suggested MCDM methods for the selection of Six Sigma projects, with the first notable publication by Yang and Chen (2003), who aim to integrate qualitative and quantitative criteria in selecting projects by applying MCDM. This line of research has later received more attention in Six Sigma literature; the steps used by the various authors are listed in Table 1. The selection processes are generally applicable in the sense that the steps of project selection remain mainly the same, although the tools and criteria can differ from industry to industry (Kumar et al., 2009).

Table 1 - MCDM process-steps for Six Sigma Project selection

Author Step A Step B Step C Step D Step E

Yang and Chen (2003) List the potential projects Define the qualitative and quantitative criteria Determine the criteria weights Evaluate the alternative projects Determine the project priority Su and Chou (2008) Project generation Project benefits evaluation Project risk evaluation Project mapping Yang and Hsieh (2009) Select criteria Determine weights of criteria Evaluate project candidates Rank projects Kumar et al. (2009) Select criteria Identify projects Determine weights of criteria

Map projects Evaluate prioritization Büyüközkan and Öztürkcan (2010) Project generation Determine criteria inter-dependence Calculate weights of decision criteria Select best alternative Perçin and Kahraman (2010) Define

criteria weights of Calculate criteria

Determine ideal

solution Calculate similarities to ideal solution and preference

Although these authors have suggested different selection processes, they generally consist of the following two steps:

1) Identify improvement opportunities: Generate a set of potential projects

(e.g. step A – Yang and Chen, 2003; step A - Büyüközkan and Öztürkcan, 2010; etc.) 2) Select the best alternative:

Define selection criteria and evaluate alternatives

(12)

Identify improvement opportunities

A set of alternatives needs to be created before any MCDM tool can assist in the selection of a project. When this set of alternatives is created on the basis of managerial experience alone it is most likely that it is neither optimal nor complete (Archer and Ghasemzadeh, 1999, 2004). In order to create a good set of alternatives the right input should be applied while employing the right tools. Banuelas et al. (2006) have reviewed the selection of Six Sigma projects by practitioners in the UK. Respondents were asked which tools they use to identify Six Sigma projects (see Appendix II). It was noticed that 80 per cent of the companies employ more than one tool to identify potential projects.

Norrie and Walker (2004) pointed out that the best organizations leverage projects to implement their business strategies. Another important input for project selection is alignment with the “voice of the customer” (Griffin and Hauser 1993). Strategic and customer input are related because any firm’s strategy must reflect the customer perspective to ensure long-term success (Zhang et al., 2008). Kaplan and Norton (2004) assert that strategy flows through the “customer perspective” to generate and select projects. The flow-down of strategy into objectives and the system of reviewing these objectives and the customer input (e.g. customer survey’s, customer visits) should serve as input for the generation of a set of candidate LSS projects (Kaplan and Norton, 2004; Jolayemi, 2009). These candidate projects should be evaluated to select the most suitable improvement project for the organization.

Select the best alternative

In practice, the selection of LSS projects in many organizations is still based on subjective judgment (Raisinghani 2005; Antony 2006). Management has difficulty making project go/no go decisions and projects are generally initiated because management thinks they will make a contribution to quality (Antony 2006). In order to design an objective project selection method, the selection criteria and a tool to evaluate the alternatives on these criteria need to be defined.

Define selection criteria

A first selection can be made by classifying the identified improvement opportunities into two categories: problems with a known solution and that with an unknown solution (Snee, 2002; Pande et al., 2000; Pyzdek, 2003). For an “unknown solution”, the problems are ill-structured and LSS methodology may be needed to solve them. Only project ideas concerning the enhancement of recurring processes of which the improvement solution is unknown and require thorough analysis should be considered in the selection procedure (Pande et al, 2000).

In order to select improvement projects systematically they should be evaluated on a number of criteria, the criteria applied by various authors are listed in Table 2. Some authors have categorized the selection criteria into two categories to facilitate easy comparison, e.g. Value vs. Complexity (Hsieh et al., 2012), Effort vs. Impact (Kumar et al., 2009) or Risk vs. Benefits (Su and Chou, 2008). Such categories are useful to determine the expected value of a project for an organization.

Table 2 - Criteria for Six Sigma project selection

Author(s) Project selection criteria

Does et al. (2002) Relation to business and customer problems Snee (2002) Business priority, importance to firms, and scope

Antony (2004) Financial return, customer satisfaction, resource required, risks, and alignment of strategic business goals and objectives

Gijo and Rao (2005)

Alignment with firm’s goals and objectives

(13)

Antony et al.

(2007) Cost of poor-quality, risk, expertise required for project, project alignment to business goals in strategic terms Chakrabarty and

Tan (2007) Financial benefits, customer satisfaction, employee satisfaction, service quality, and reduced variation Chakrabarty and

Tan (2008)

Measurable financial benefits, linking to company’s business strategy, high probability of success, and far reaching impact on business

Heckl et al. (2010) Data availability, clearly defined goals, milestones, timelines, and budgets Given the many criteria that are available for project selection and the different conditions organizations face, it seems unlikely that a fixed list of criteria should suffice in selecting the best projects for every organization. To evaluate potential projects, a relevant selection of these criteria needs to be performed. This can be realized by conducting multiple rounds of surveys among a group of experts similar to the Delphi method (Perçin and Kahraman, 2010). The Delphi method is based on the assumption that group decisions are usually more valid than decisions made by a single person and even more so if the group is comprised of experts in the field. The modified Delphi method that can be used for developing the list of criteria is described in detail in Appendix III.

Evaluate alternatives

When the criteria have been determined, a suited method to evaluate the alternatives is required. Most authors suggest the Analytical Hierarchy Process (AHP) to do this (Table 3). Since the criteria on which the projects are selected are often ambiguous, it is difficult for experts to evaluate the alternatives and assign an

absolute judgment. The AHP method deals with this by asking experts to compare two elements and provide a relative rating. The AHP method is known in research as an MCDM method. AHP is developed to solve complex MCDM problems involving multiple qualitative and quantitative criteria.

The purpose of the AHP is to provide weights for each criterion and alternatives by incorporating the evaluations of all decision makers into a final decision. AHP requires three steps: (1) identifying evaluation hierarchy, (2) assessing the decision-maker evaluations by pairwise comparisons, and (3) calculating the weights for criteria and alternatives. It allows decision-makers to specify their preferences using the Saaty’s 1-9 scale (Saaty, 1990). Saaty's scale is a relative scale used to represent the relative importance of one criterion over another with respect to the main criterion. The scale consists of numbers one through nine. Number one represents equal importance and number nine represents extreme importance. Pair-wise comparison between the selection criteria is done by using this scale (Saaty, 1988). This scale can be very useful in helping a group of experts or an individual to make a decision (Perçin and Kahraman, 2010).

Conclusion

A selection process for LSS projects consists of two main steps: (1) the identification of improvement opportunities from customer and strategic input, and (2) the evaluation of alternatives using a systematic evaluation on company-specific criteria using MCDM theories. Research exposes several criteria for the process of LSS project selection, by using the Delphi method a company-specific list of criteria can be identified. Fundamentally, firms should choose projects in accordance with the firm’s goals and objectives (Gijo and Rao 2005; Antony et al., 2007) that tackle their business and customer problems (Does et al., 2002).

Table 3 - Evaluate alternatives

Author(s) Evaluation of alternatives

Yang and Chen (2003) AHP Kumar et al. (2007) None Kumar et al. (2009) AHP Su and Chou (2008) AHP, FMEA

Yang and Hsieh, (2009) Aggregated fuzzy evaluation Perçin and Kahraman

(14)

By using a project selection process that complies with the theoretical framework (Figure 1) it is expected that resources will be allocated accurately to achieve the most important strategic objectives of an organization. A project selection method is considered useful when it (1) possesses the capability to wisely allocate scarce resources to achieve the most important strategic objectives (Jamieson and Morris 2004) and (2) yield projects of which the goals can be achieved within time. The perceived usefulness of the selection procedures is operationalized by assessing these two elements.

Figure 1 – Suggested project selection process

2.2 Research context

This research takes place at Ziggo N.V., the largest network communication company in the Netherlands. Ziggo as an organization is in the midst of a transformation process. Its inception took place in 2008 by the merging of three Dutch cable companies and it was stock listed in 2012 on NYSE Euronext Amsterdam. The organization is described in the interviews to be at a “low maturity” level, meaning that both decision- and business-processes are still unstructured. The company experienced much growth and intends to follow this path in the future whilst at the same time becoming a ‘high performance organization’ (Ziggo annual report, 2012). “The changing nature of our business requires the ability to continuously adjust our processes” (Ziggo annual report, 2012). LSS is part of this strategic program in which the business processes need to be optimized. A very concrete target for 2013 is a required saving of €16 million on the current operational processes.

The problem owner, one of the directors at Ziggo and involved with the implementation of LSS, states that in the 1½ years LSS has been applied at Ziggo, it has proven difficult to execute LSS projects and that a more structured project selection process is required. At the time of the research, two rounds (referred to as ‘waves’) of LSS projects were already performed at Ziggo while projects were being selected for a third wave. Partly owing to involvement of the researcher in the case study company, the way projects were selected in the third wave differed from the selection procedure during the first two waves and will therefore be assessed separately. The project selection procedures of both these waves are evaluated in terms of perceived usefulness in this study.

Figure 2- Timeline of consecutive 'waves' and researcher involvement

STEP 1:

Use strategic and customer-input to generate a set of candidate projects

STEP 2:

Evaluate the set of candidate projects

using MCDM techniques

Best suited project

Select project selection criteria

(15)

2.3 Research questions

The literature review has provided a framework by which the selection processes previously adopted by the organization are studied and evaluated. Since wave 1&2 took place prior to the research whilst project selection for wave 3 occurred during the time of research (see Figure 2) they are described and evaluated in separate research questions. Furthermore, the requirements posed on an LSS project selection procedure by the stakeholders from the Ziggo organization are identified and validated. This leads to the achievement of the research goal: The design of a structured decision process for LSS project selection at Ziggo.

RQ1. How can the process of project selection at Ziggo N.V. in the prior waves (1&2) be described and how is this evaluated in terms of usefulness?

RQ1.1. How was step 1 (Figure 1, page 6) of the project selection process performed in the first two waves in terms of input, tools & actors?

RQ1.2. How was step 2 (Figure 1, page 6) of the project selection process performed in the first two waves performed in terms of input, tools & actors and which criteria have been used? RQ1.3. How is this process evaluated in terms of usefulness?

RQ2. How, and based on which input was the process of project selection at Ziggo N.V. in the third wave adapted and how is this evaluated in terms of usefulness?

RQ2.1. Based on which input was the project selection procedure for wave 3 adapted compared to that of wave 1&2?

RQ2.2. How was step 1 (Figure 1, page 6) of the project selection process performed in the third wave performed in terms of input, tools & actors?

RQ2.3. How was step 2 (Figure 1, page 6) of the project selection process performed in the third wave performed in terms of input, tools & actors?

RQ2.4. How is this process evaluated in terms of usefulness?

RQ3. Based on their experiences, what does the Ziggo organization consider important elements for the identification and selection of LSS projects?

RQ3.1. Which requirements are posed on step 1 (Figure 1, page 6) of the project selection process by the Ziggo organization and does the presence/absence of these requirements influence the perceived usefulness of the project selection procedure?

RQ3.2. Which requirements are posed on step (Figure 1, page 6) of the project selection process by the Ziggo organization and does the presence/absence of these requirements influence the perceived usefulness of the project selection procedure?

RQ4. How do the project selection procedures of the three waves at Ziggo compare with the requirements and the theoretical framework?

RQ4.1. How does step 1 (Figure 1, page 6) of the project selection process of the three waves compare with the requirements and the theoretical framework?

RQ4.2. How does step 2 (Figure 1, page 6) of the project selection process of the three waves compare with the requirements and the theoretical framework?

RQ5. How should the LSS project selection processes at Ziggo be adapted?

RQ5.1. How should step 1 (Figure 1, page 6) of the project selection process be adapted? RQ5.2. How should step 2 (Figure 1, page 6) of the project selection process be adapted? RQ5.3. What are the change conditions for the implementation of the new selection process?

(16)

(17)

3. Methodology

A case study research at Ziggo N.V. is conducted in order to answer the research questions. Case studies are helpful in identifying the causes that lead to the success or failure in organizational situations (Cooper & Schindler, 2006). A single-site case study is deemed to be the appropriate study design because of the opportunity to study the single case at two points in time (longitudinal case study, Yin, 2003). The time intervals reflect the stages at which changes should reveal themselves, in this research these are the consecutive ‘waves’ of projects in which the process of project selection is the unit of analysis. The project selection method used during the first two waves is studied in retrospect whilst the project selection process of the third wave is assessed in real-time during the time of the research. Through participative research, the researcher learns about the processes and about the prescriptive actions that are studied. The ‘criteria for judging the quality of research designs’ as listed by Yin (2003, pp. 40 – 45) are used as a guideline in developing the research method. The methods, sources and analyses used to answer the research questions are discussed in the subsequent sections.

3.1 Description and evaluation of the project

selection processes (RQ1, RQ2).

The retrospective part of the research describes and evaluates how the process of project selection was executed at Ziggo N.V. during the first two waves of projects (RQ1). The project selection procedure in the third wave is investigated through participative research (RQ2). To construct validity, multiple sources of evidence are used (Yin, 2003 pp. 102) and key informants have reviewed the report (Yin, 2003).

Description

Data on the selection procedure in the different waves is gathered by reviewing relevant documentation (e.g. internal email communication, training material and project documentation). Additionally thirteen Greenbelts (LSS improvement specialists), three managers, four champions (LSS project managers) and the LSS program manager are interviewed. Furthermore, the project selection process in the third wave is observed directly by the researcher by attending planning sessions. Data regarding the selection procedures in the three waves collected from these sources is coded along RQ1.1-RQ1.3. Qualitative analysis of the data takes place by systematically reviewing the data in each of these categories to provide answers to the research questions. Subsequently, the description of the selection process is reviewed and validated by the LSS program management. Moreover the results of this qualitative description are validated through a survey among the Greenbelts and Champions involved in selecting projects during the consecutive waves.

Evaluation

(18)

7=Strong positive influence). These likert-type characteristics (Clason and Dormody, 1994) fall within the ordinal level of measurement (Blaikie, 2003). Since this concerns ordered ordinal data, it is described using the median as the measure of central tendency (Clegg, 1998).

3.2 Identifying the requirements for the project

selection process (RQ3)

A list of possible elements (actors, type of input used, selection method) for the different steps in the selection process is formulated based on the literature review and the interviews. Each of the elements are rated on the degree to which they are considered important for the design of a selection process using 5-point likert-type questions. The survey has two main purposes: 1) Identify the required input & actors for the identification of improvement opportunities and 2) Identify the required structure & actors for the selection of projects.

A 5-point likert-type is deemed appropriate since this sufficiently allows the respondent to provide a judgment on the level of importance of the elements for the project selection process. Since the data is ordinal the results will be described by the median and spread of response (using boxplots) in chapter 4 (Clegg, 1998). The respondents are asked whether they agreed/disagreed that an element should be present in the selection process (1=Strongly disagree,…, 5=Strongly agree). Quantitative data, like survey responses, provide more anonymity (Cooper & Schindler, 2006). Every respondent gives his/her opinion without individual consequences. In addition to the quantitative ratings, open-ended questions are used to provide additional comments on each of the elements. The data of the survey were gathered between March 15th and March 26th in 2013. The survey was sent to 46 Ziggo employees, with a response of 67.4%. The survey was conducted among all Greenbelts and Champions involved in one of the three waves at Ziggo. Prior to the survey, a pilot with six respondents was performed before it was sent to the whole respondent group.

The descriptions and evaluations of the selection procedures are further analyzed in order to verify whether the presence/absence of the required elements indeed resulted in different evaluation. This is tested using Mann-Whitney U test (α level of 0.05 to test for significance) which is a test for ordinal data often used when small samples (<25) are involved (Baarda et al., 2004). From this, a number of requirements are listed for the final design of the LSS project selection process (RQ5).

3.3 Comparing the selection procedures with

requirements and theoretical framework (RQ4)

The description of the project selection methods of wave 1&2 and wave 3 are compared with the theoretical framework and the requirements that were identified in RQ3. Depending on the extent to which the selection procedures complied with the theoretical framework and the requirements it is determined whether or not the selection procedures needed to be further adapted in the final design.

3.4 Final design of the LSS project selection process

(RQ5)

(19)

4. Case study

The previous chapters have described LSS project selection from a theoretical perspective and outlined the research method. In this chapter RQ1, RQ2 and RQ3 are answered by conducting a case study at Ziggo N.V.. Both qualitative and survey data are used to describe the project selection methods and outline the requirements for a suited selection method.

4.1 RQ1: Description and evaluation of selection

procedure wave 1&2

LSS at Ziggo N.V. started in June 2011 as part of a strategic program initiated by the company board (High Performance Ziggo, HPZ). The LSS program was assigned to the Technology Cluster (TC) of the company and was commenced by its VP operations. Thus far, two so called, ‘waves’ of projects have been initiated, comprising a total of 26 LSS projects. Of these 26 projects, only six projects have been successfully completed with an average project lead time of eleven months while the average prospected lead time is five months.

“Sometimes the improvement proposals of a project become irrelevant because they are caught up by reality due to the long project lead times” (program management). A review of the performance of the first waves based on Ziggo intranet documentation yielded the information on the performance of the projects depicted in Table 4.

The process of selecting these projects in wave 1&2 is described and the perceived usefulness is analyzed in this section. The goal of the project selection process in the first two waves was to identify a suitable project and candidate Greenbelt for each department in the TC to gain experience with LSS. Project selection for LSS has been performed in a process that can be described by three steps: Identification of improvement opportunities, pre-selection and final selection. The first step involves the identification of improvement opportunities (step 1, RQ1.1) and the latter two encompass the selection of projects (step 2, RQ1.2). A description of the selection procedure at Ziggo during wave 1&2 is provided in this section.

RQ1.1 Description of the method for the identification of improvement

opportunities in wave 1&2

Project-ideas were collected throughout the organization by asking the director of every department within the Technology cluster. During a kick-off meeting led by the CTO, a complete list of criteria (Table 22, Appendix IV) to which a project idea should comply was presented to all of the managers and directors of the departments. Each of the managers and directors of the Technology cluster were asked to deliver at least one project idea via a web-based portal on the intranet. In this portal, the manager and directors were asked to provide further information regarding the project-idea (Table 23, Appendix IV). However, the urgency to start LSS projects was not experienced by every department: “LSS is not anchored in every department” (GB1; Program management). An indicator of this varying degree of enthusiasm is the fluctuating number of project ideas delivered by the different departments via the web-portal (Table 24, Appendix IV).

Generating the project ideas was performed in a decentralized fashion. “Each director was expected to deliver project ideas and candidate Greenbelts for his/her department, each director has generated project-ideas differently following his/her own preferences” Some directors would delegate this to the candidate Greenbelt: “I (Greenbelt) have started the project from my own experiences in my daily work”; others consulted their departments’ line-level employees: “The

(20)

project-idea stems from one of the field engineers who pointed out that this might be an interesting process improvement project” (GB3); yet other directors came up with project ideas themselves: “I (director) have some experience with LSS projects from working in another company, therefore I think I can make a good judgment in selecting the right project for my department” (Champion 1).

A survey among the Greenbelts and Champions involved in wave 1 and 2 (n=20) indicates that project ideas were mostly collected by multiple actors, often using customer problems (81%) and strategic goals (75%) to identify problems. A summary of this step is provided in Table 5.

Table 5 – Step 1: Input, tools and actors in generating project ideas

Action Input Tools actors

Gather project

ideas Greenbelts daily work experience Problems indicated by the line Director preferences

Intranet

portal Director Greenbelts Line

RQ1.2 Description of the method for the selection of projects in wave 1&2

The selection of projects leads to the final choice for one project per GreenBelt, the criteria that were used and the method for the selection are described in this section.

Pre-selection of projects

The Program Management selected only those ideas that matched with the LSS methodology. These project-ideas were prioritized by the program management based on their “impact on the business” and “feasibility”. Different criteria have been used during the project selection process. The criteria, that were developed from training material based on prior experience of the consultant, are grouped in three categories (see Table 23, Appendix IV): (1)Impact; these criteria relate to the impact of the project on the business, (2) Feasibility, these criteria relate to the likeliness that a project can be completed, and (3) Match with LSS, these criteria relate to the compatibility of the problem with the LSS tools. This last set of criteria is relevant since it is not always necessary to set up an entire LSS project. Or, as one Greenbelt put it: “Some problems are just more suited to be solved by more Lean-like approaches, simply structuring the processes sometimes suffices and measurements to provide statistical data are superfluous”(GB1).

The selection of projects consisted of a pre-selection and a final selection. The pre-selection was performed by further discussing the input collected from the portal in a meeting with program management, the candidate belt and his/her manager in order to deduce candidate projects. The project ideas were assessed evaluated on the “match with the LSS methodology criteria (Appendix IV)” by the program management, based on the information provided in the portal and the meeting. Only those ideas that were found to be suitable LSS projects were subject to further consideration in step 2 and 3 of the selection process. The remaining project ideas were described in more detail by the program management with the assistance of an external consultant from a training institute. After this, the program management rated each project-idea on a five-point scale on each of the ‘impact’ and ‘feasibility’ categories criteria and prioritized the project ideas for each department.

Table 6 - Step 2a: Summary of input, tools and actors in the pre-selection of projects

Asses project ideas on match with LSS and formulate candidate projects

Ideas from intranet portal. Additional information from discussion. Meeting. Scoring list. GB Manager of GB Program management External consultant Prioritize candidate

projects Candidate projects + background information from the intranet portal and prior meetings.

Scoring

list. Program management

Final selection of projects

(21)

The final selection of the projects took place in a meeting with the Management team (MT) of the Technology Cluster (MT-TC consisting of the clusters’ directors & the Vice President operations (VP)) and the LSS program management. The various prioritized project ideas formed the input of a discussion. “Selecting the projects in the MT-TC meeting has not occurred very systematically, projects were mainly selected by judging whether the projects would form logical combinations with each other and whether they matched with the daily activities of a Greenbelt” (Program management). However, the ‘impact’ and ‘feasibility’ criteria (Table 23, Appendix IV) were not further discussed during this discussion. As one of the directors mentioned: “I (director) have never seen the list with impact and feasibility criteria” and “the choice between project ideas was very limited since most directors provided only a few project ideas and some directors delivered only one idea. The actual decision was thus not made by the program management or the MT-TC, but by the directors themselves” (Champion 1). (The final project selection is summarized in Table 7)

Table 7 - Step 2b Summary of input, tools and actors in the final selection of projects

Final project

selection Prioritized projects Discussion MT-TC Program management A survey among the Greenbelts and Champions involved in wave 1&2 (n=20) indicated that sometimes more than one project idea was considered in the evaluation (58%), that the evaluation has not been performed systematically (84%) and that multiple actors were involved in selecting the projects.

RQ1.3 Evaluation of wave 1&2

An evaluation of the selection method based on qualitative and quantitative data is outlined below.

Step 1: Identification of improvement opportunities

The selection process of the first two waves was performed “very much on an ad hoc basis since there was no experience with the LSS methodology” (Program manager). The method of project identification in wave 1&2 has differed between the departments, it is therefore not possible to provide a generic evaluation of this step. As one of the directors put it: “I have performed the idea generation as I thought was best. Other directors might have done this differently but I cannot evaluate the methods used by other directors”. Because of the use of different methods for step 1, the approaches used for the identification of eight different project-ideas have been compared in order to analyze the differences between them (Table 25, Appendix IV). It was observed that when projects were not explicitly formulated in line with a department’s strategy, some projects would at some point lack commitment from the management. As a Greenbelt mentioned: “More management commitment to the projects is required. It would be good if projects were related to one of the department’s KPI’s. People (managers/directors/belts) are judged based on these KPI’s, this creates commitment at the right level” (GB2). When the projects were formulated by (or in accordance with) the relevant directors, they possessed more strategic relevance resulting in more commitment of the management.

Step 1 of wave 1&2 was evaluated using a survey among the Greenbelts and Champions involved in wave 1&2 (n = 20; Appendix IV). The respondents indicated the perceived effect of the project identification method on the accurate allocation of resources to be slightly positive and the likeliness that the projects would achieve the goals within time to be neutral. The results are depicted in more detail in Appendix V.

Step 2: Project selection

(22)

Management; GB1, GB3, GB7) and “Sometimes it is just not necessary to improve the processes via the LSS methodology, simply structuring the processes can be enough” (GB5). Projects that were not considered suitable LSS projects were removed from the selection and not further evaluated in step 2 and 3 of the selection process.

The program management has elaborated the project ideas into potential projects. “This was a lot of work but it was necessary since there was no LSS knowledge available within Ziggo” (Program Management). A first evaluation of the projects by the program management on the ‘feasibility’ and ‘impact’ criteria was carried out with the aim “to provide focus for the discussion in the MT”. However, for some departments this was not considered as relevant since only one project idea was delivered and no prioritization could take place.

The goal of making a final selection of projects in the MT-TC was to select the best set of projects for the entire cluster. This goal was not met since “the directors decided which project was selected for their own department and were not really involved in the discussion about potential projects in other departments” (Champion 1). The LSS program management stated that “the directors had a lot of freedom in choosing their preferred project for their department” (Program management).

The final selection of projects was therefore performed by the directors of the departments in which the projects were going to take place. “Since the directors were forced top-down to start up an LSS project, some directors were not very motivated to start an LSS project. Some directors selected projects that they were not going to provide with active support” (Program management). This lack of management support often leads to problems during the execution of the projects and implementation of the suggested improvements: “The engagement of the director (being champion) in an LSS project is of utmost importance. When this is not available the projects is almost bound to fail. One of the belts who I championed in a project experienced this during a previous project. Her project was not considered a priority by the champion and was stopped eventually” (Champion 2).

Step 2 of wave 1&2 was evaluated using a survey among the Greenbelts and Champions involved in wave 1&2 (n = 20, Appendix IV). The respondents indicated the perceived effect of the project identification method on the accurate allocation of resources to be slightly positive and on the likeliness that the projects would achieve the goals within time to be neutral. The results are depicted in more detail in Appendix V.

4.2 RQ2: Description and evaluation of selection

procedure wave 3

The third wave of LSS projects at Ziggo N.V. started in March 2013 and has taken place in three departments of the organization. Only one of these departments possessed prior LSS experience (participated in the second wave). In this section, the adaptation of the process of selecting LSS projects for the third wave is described, the procedure for the third wave is described and the perceived usefulness of this process is analyzed. Six Greenbelt-projects have been studied in two departments. In each of these departments three projects were initiated.

RQ2.1 Adaption of the selection procedure for wave 3

The researcher’s involvement in the case study company took place just before and during the project selection process of the third wave. Part of the role of the researcher was to advise and assist the program management in the selection procedure for the third wave of LSS projects.

(23)

Table 8 - Description of researcher involvement

Suggestion: Who: Provided by:

Use customer and strategic input Program management continuous discussion and feedback

Project evaluation model (AHP) Program management presented during meetings Use customer and strategic input Candidate Greenbelts presentation during introduction

meeting of candidate GB’s Use a list of criteria for evaluation

Candidate Greenbelts presentation during introduction meeting of candidate GB’s

The process suggested from literature proposes generating project ideas using customer and strategic input. Since projects are performed within different departments in the case study company, the researcher suggested generating projects at department level. This was suggested to the program management during continuous presence of the researcher in the case study company and during the introduction meeting of the candidate Greenbelts. The use of an AHP evaluation model has been presented to the program management during formal meetings. The candidate-Greenbelts were suggested to evaluate their project ideas systematically on the criteria during the introduction meeting of the candidate Greenbelts.

RQ2.2 Description of the method for the identification of improvement

opportunities in wave 3

The goal of the project selection process in the third wave was to identify a suitable project for each Greenbelt. This section describes the method used for the identification of improvement opportunities in the third wave.

(24)

A survey among the Greenbelts and Champions involved in wave 3 (n=10) indicates that project ideas were mostly collected by multiple actors, often using customer problems (67%) and always using strategic goals (100%) to identify potential projects.

Table 9 - Step 1: Input, tools and actors

Gather project

ideas Problems indicated by customers KPI dashboards GB/Director/Line work experiences Strategic themes

Criteria (Table 28 , Appendix VI))

Multiple

brainstorms Management-representative Greenbelts

RQ2.3 Description of the method for the selection of projects in wave 3

The selection of projects leads to the final choice for one project per GreenBelt, the criteria that were used and the method for the selection are described in this section.

The list of potential projects was discussed in each of the departments to make a final selection of LSS projects for the third wave. “The long-list of projects was further discussed with the directors and managers to make a final selection” (GB7). The criteria that were used and the evaluation method employed by both departments are discussed separately.

Other than the ‘Match with LSS’ set of criteria, no predefined lists of criteria have been utilized during the selection procedure in this wave. While no checklists with criteria were used, certain norms were taken into account when generating and selecting the projects. These criteria have not been documented by the organization but were mentioned in interviews and observed by the researcher during planning sessions. These criteria were based on experiences with other types of projects within the department and from LSS experience within other companies. The criteria are listed in Appendix VI. The criteria have been applied in the different stages of the selection process, albeit not always formally. A manager of one of the departments commented on this: “It is difficult to set up a structured evaluation of project ideas in this maturity phase of the LSS initiative. Although we have tried to create some structure in generating project ideas from our strategic goals, we have not used checklists with criteria”.

In department 1 the potential projects were discussed during a meeting with the director and managers of the department. The project-ideas have been discussed on their contribution to the strategic goals and how the projects could support each other in the achievement of these goals. During this meeting the Greenbelts were assigned a project and asked to elaborate the projects into project proposals. In department 2 the departments’ director evaluated the long-list of projects, the director also evaluated the candidate projects’ contribution to the strategic goals and how the projects could support each other in the achievement of these goals. The Greenbelts have been assigned a project by the department’s director and were asked to elaborate the projects into project proposals. The project proposals have been formulated with the assistance of the program management and/or experienced BB’s who have provided guidance and assured that the projects complied with the ‘Match with LSS criteria’ (Table 28).

In department 1 the project proposals were presented during a gathering of the director and managers of the department. In department 2, this occurred during a meeting of the ‘steering committee continuous improvement’ which consisted of multiple directors. In both meetings the project proposals were challenged on the contribution to the strategic goals, project scope and financial benefits. All of the projects were accepted and received a final go.

(25)

not been performed systematically (80%). Furthermore, multiple actors were involved in making the final selection (LSS program management; Greenbelt; Champion; Department’s MT).

RQ2.4 Evaluation of wave 3

An evaluation of the selection method based on qualitative and quantitative data is outlined below.

Step 1: Identification of improvement opportunities

Identifying the improvement opportunities with the method that was used in wave 3 was considered valuable by the respondents. By explicitly identifying improvement opportunities from strategic input, the LSS projects contribute to the achievement of a department’s most important objectives. One of the Greenbelts mentioned: “From the KPI-dashboard I can easily deduce 30 projects. Based on the strategic goals the department’s MT can develop a shortlist and request the Greenbelts to elaborate these into project proposals. From these proposals a final selection can be made” (Greenbelt).

The structure in which a manager was made responsible for the identification of improvement opportunities, was valued by the respondents: “A manager who was not going to be personally involved in the projects was responsible for collecting the project ideas and consulted us (belts) and the director. This was very useful since the manager was able to analyze the project-ideas from different levels” (Greenbelt)

Step 1 of wave 3 was evaluated using a survey among the Greenbelts and Champions involved in wave 3(n = 10). The respondents were indicated the perceived effect of the project identification method on the accurate allocation of resources to be positive and on the likeliness that the projects would achieve the goals within time to be positive. The results are depicted in Appendix VII.

Step 2: project selection

The structure of considering multiple options on a strategic level (MT/steering committee) and elaborate on the projects on a tactical level (by the Greenbelts) was valued by the respondents. Some respondents stated that a more systematical approach to the selection of projects would be useful. A manager mentioned: “In a later phase it would be helpful to use a more systematic approach to the selection of projects. We have not done this now because I think this not our organizational routine” (Manager). Another manager indicated: “A bit more structure in this phase would have been useful, for our department this is the first time that we have selected LSS projects and we had to reinvent the wheel”.

Step 2 of wave 3 was evaluated using a survey among the Greenbelts and Champions involved in wave 3(n = 9). The respondents were asked to indicate the perceived effect of the project identification method on the accurate allocation of resources to be positive and on the likeliness that

Table 10 - Step 2: Input, tools and actors

Input Tools actors

Evaluate project ideas and

assignment to Greenbelts

Long-list of project ideas Meeting Department 1: Department director and managers Department 2: Department director Formulate project

proposals Assigned project-ideas ‘Match with LSS’ criteria - Greenbelts Experienced BB’s Final selection Project proposals (1 per GB)

(26)

the projects would achieve the goals within time to be positive. The results are depicted in Appendix VII.

4.3 RQ3: Ziggo’s requirements for an LSS project

selection process.

From the literature review (chapter 2) and the description of the different selection processes at Ziggo (paragraph 4.1-4.2), a number of elements that make up the selection process can be identified (which input to use in step 1, how to make a selection in step 2 and a description of the actors that should be involved). These elements were rated by 31 respondents from the Ziggo organization who indicated whether they agreed/disagreed that the elements were important for the design of a project selection process.

Step 1: Identifying improvement opportunities

The theoretical framework suggests deducting improvement opportunities from the day-to-day operations of the business by using strategic and customer input. The respondents strongly agree (90% agrees, 10% neutral) that strategic input was important and strongly agree (69% strongly agrees, 21% neutral, 9% disagrees) that customer input is important. During the case study it was observed that different actors were involved in identifying improvement opportunities. The respondents strongly agree (100% agrees) that line-employees and line-managers should be involved, agree that management (directors) should be involved (86% agrees, 3% neutral, 10% disagrees) and are neutral in their opinion whether LSS program management should be involved (34% agrees, 48% neutral, 17% disagrees). (the results are further outlined in 51Appendix VIII).

To validate the requirements indicated by the organization for step 1, these requirements are further analyzed based on the descriptions and evaluations provided by the actors involved in previous project selection, described in detail in Appendix IX. This analysis of the descriptions and evaluations of previous rounds of project selection at Ziggo shows that identifying improvement opportunities from strategic input is perceived to result in projects that are more likely to be completed within time and in a more accurate allocation of resources. These findings correspond with the suggestions from the theoretical framework. The data shows no significant difference for presence/absence of certain actors during step 1 of the selection process.

Step 2: Selecting projects

Theory suggests evaluating multiple project ideas using a systematic MCDM method. The respondents agree that multiple options should be considered (80% agrees, 10% neutral, 10% disagrees) and agree that these options should evaluated systematically (90% agrees, 7% neutral, 3% disagrees).During the case study it was observed that different actors were involved in selecting projects in the first three waves. The respondents agree (65% agrees, 24% neutral, 10% disagrees) that LSS program management should be involved, strongly agree that the Greenbelt should be involved (72% agrees, 24% neutral, 3% disagrees), strongly agree that the Champion should be involved (90% agrees, 7% neutral, 3% disagrees) and agree that project selection is an MT decision (93% agrees, 7% neutral). (the results are further outlined in Appendix VIII).

(27)

5. RQ4: Analysis

The selection processes applied at Ziggo in the first two waves, and the third wave are compared with the theoretical framework and with the requirements identified in the previous chapter. When the selection procedures did not comply with the requirements, the selection process needs to be adapted. A suited project selection process is designed in chapter 6

5.1 RQ4.1 Comparison: Step 1

A summary of the suggestions from literature (chapter 2), previous selection processes (chapter 4.1) and the requirements by the organization for Step 1 (chapter 4.3) is provided in Table 11. The percentages specify the number of respondents who indicated that an element was present in the selection process. The requirements indicate the degree to which respondents agree that the element should be included in the selection process.

In wave 1&2, the identification of improvement opportunities was performed on an ad hoc basis; the survey respondents indicated that customer problems and strategic goals were used as input to

identify improvement

opportunities. However, the feedback of Greenbelts collected during the interviews shows that

projects were not always explicitly formulated in line with a department’s strategy. These projects would at some point lack commitment from the management, as a Greenbelt stated: “More management commitment to the projects is required. It would be good if projects were related to a department’s KPI’s. People (managers/directors/belts) are judged based on these KPIs, this creates commitment at the right level” (GB2). Where the projects were formulated by (or in accordance with) the relevant directors, they possessed more strategic relevance resulting in more commitment of the management. This is in line with the findings from the statistical analysis in the previous chapter. The theoretical framework suggests a systematic review of customer/strategic input from performance measurement systems or customer feedback reports. The method used in wave 1&2 did not comply with this suggestion; the interviewees stated that improvement opportunities were mostly identified based on ‘personal experience’ and ‘gut-feeling’. Each department has identified the improvement opportunities differently, involving different actors. The survey results show that this was mostly performed by the director and line, which is consistent with the requirements posed by the organization. In wave 1&2 the theoretical framework was not followed although the actors

(28)

involved in the identification of improvement opportunities were consistent with the requirements by the organization.

During the project selection procedure of wave 3, the performance of the departments on strategic goals was reviewed systematically in a manner similar to the theoretical framework. Customer input has also been taken into account (albeit not systematically), but projects were primarily aligned with strategic goals. The survey results show that this was primarily performed by the director and line, a structure was used in which a line-manager coordinated the identification of improvement opportunities which is consistent with the requirements posed by the organization, although these requirements indicate that the line should play a larger role than the directors in identifying improvement opportunities. In wave 3 the theoretical framework was followed for step 1 and the actors involved were in line with the requirements by the organization.

From the above analysis of wave 1&2 and wave 3 it can be concluded that they differed in their method used to identify improvement opportunities. To test whether these differences resulted in any significant changes in the evaluations of step 1, these have been compared using Mann-Whitney U tests. It has been demonstrated with significance (p<0.05) that step 1 of wave 3 is perceived to result in a more accurate allocation of resources than step 1 of wave 1&2. Moreover, it was demonstrated with strong significance (p<0.01) that the method used for step 1 during wave 3 is perceived to result in a higher likeliness that projects are completed within time that the method used for step 1 during wave 1&2.

5.2 RQ4.2 Comparison: Step 2

The selection processes applied at Ziggo in the three waves are compared with the theoretical framework and evaluated on the extent to which they meet the requirements identified in the previous chapter. Again, a summary of the theoretical framework, previous selection

processes and the

requirements by the organization for the selection of LSS projects is provided in Table 12.

Only just over half of the respondents indicate that different project options were considered during wave 1&2, the rest of the respondents did not evaluate multiple project ideas. Furthermore, the evaluation has not been performed systematically. This is neither in line with the theoretical framework nor with the requirements posed by the

(29)

However, interviewees who were involved in wave 1&2 commented on this by stating that the actual decision was primarily made by the directors. This is in line with the requirements by the Ziggo-organization.

In wave 1&2 the theoretical framework was not followed, although the actors involved in the identification of improvement opportunities were in line with the requirements by the organization.

Multiple options were considered in wave 3, although the evaluation has not been performed systematically. This is only partly in line with the theoretical framework and the requirements posed by the Ziggo-organization. The results of the survey indicate that projects were selected by multiple actors. In department 1 this was done through discussion among multiple actors and in department 2 this was mainly a director’s decision. This is in line with the requirements by the Ziggo-organization. In wave 3 the theoretical framework was not entirely followed. The actors involved in the identification of improvement opportunities were in line with the requirements by the organization.

From the above analysis of the three waves it can be concluded that they differed in the manner in which LSS projects were selected. To test whether these differences resulted in any significant changes in the evaluations of step 2, these have been compared using Mann-Whitney U tests It has been demonstrated with strong significance (p<0.01) that step 2 of wave 3 is perceived to result in a more accurate allocation of resources and in a higher likeliness that projects are completed within time than the method used for step 2 of wave 1&2.

5.3 Conclusion

It was observed that LSS project selection at Ziggo in the third wave was performed more in line with the theoretical framework and the organization’s requirements than in wave 1&2. Although in wave 3 the theoretical framework was followed for step 1, it lacked a systematic evaluation method in step 2 thereby not complying with the requirements.

Additionally, the analysis of the survey responses has provided insight in the effects of different selection methods on the perceived usefulness. The method used for step 1 in wave 3 complies with the theoretical framework and seems to be a suited method for the identification of improvement opportunities. This method is perceived to be more useful than step 1 of the selection process of wave 1&2 (p<0.01).

(30)

Lean Six Sigma project selection based on Multi-Criteria-Decision- Making theory