The MUM Effect: Scrum vs Waterfall

(1)

Master Thesis

The MUM Effect: Scrum vs Waterfall

Lisette M. Pool

10454101

M

ASTER

I

NFORMATION

S

TUDIES

H

UMAN-

C

ENTERED

M

ULTIMEDIA

F

ACULTY OF

S

CIENCE

U

NIVERSITY OF

A

MSTERDAM

July 30, 2017

Supervisor & first examiner: Loek Stolwijk

Second examiner: Arjan Vreeken

(2)

The MUM Effect: Scrum vs Waterfall

Lisette Pool

University of Amsterdam

Science Park 904

lisettepool@live.com

ABSTRACT

The MUM effect has been identified as an important contrib-utor to IS project failure. The main objective of this study is to test if the MUM effect differs in the development environ-ments Scrum and Waterfall. Moreover we focused on four contradictions between these environments and how they af-fect the MUM efaf-fect: development method flexibility, com-munication frequency and formality, review procedure, and team spirit and work responsibility. We addressed this prob-lem with a quantitative approach using a scenario based ex-periment. The data was analyzed using independent-samples t-testing and one-sample t-testing. The results show that there was no statistical significant difference in willingness to re-port between those two development environments. However we found that some of the individual factors could signifi-cantly influence the MUM effect in each environment, espe-cially in the Waterfall development environment. The effects of each factor on the willingness to report bad news in each development method will be discussed. Further more, in the discussion some limitations and implications of this study are mentioned as well as directions for future research on this topic.

Author Keywords

IS project failure, whistle blowing, MUM effect, development method, Scrum, Waterfall

INTRODUCTION

Information Systems (IS) project failure is a common and costly problem [21]. Most IS projects fail to meet budget, time and or functionality criteria. The Standish Group re-ported that only 29% of the IS projects were successful on all three performance measures [6]. Until today the failure rate of IS projects remains high [17].

As we found in our earlier study where we created a de-scriptive model of this complex process using system dynam-ics, there are numerous factors that account for the success and failure of IS projects [27]. One important contributor to project failure is the MUM effect, or so called “code of si-lence” [37] [28]. The MUM effect has been defined by Tesser and Rosen as the reluctance to communicate undesirable in-formation [30]. In the context of IS projects, it can be de-scribed as the situation where a person or team decides to keep silence and hide negative or critical information about a project [28].

It is problematic if bad news about an IS project’s status is apparent to a project member, but withheld from the man-agement [21]. It results in a situation where managers are unaware of the true status of a project [37]. Troubled projects can escalate and have a high chance of failing. If the manage-ment knows about a projects bad ’health’, they can take cor-rective steps that might turn the project around before more resources are wasted [26]. If project failure is foreseen, the organization at least has a chance to prepare for it, which can reduce the size of the loss [18].

We are not aware of any studies about the way different soft-ware development environments affect the MUM effect. In this study we make a comparison between two popular mod-els: the traditional Waterfall method and the Agile software development process Scrum. The main research question is: Does Scrum provide a better environment for minimizing the MUM effect compared to the traditional Waterfall Model? In order to answer this question, we will first look at the main differences between the Waterfall method and Scrum by do-ing a literature study. Based on the literature, we derive four factors that we consider important and could affect the MUM effect. By doing a scenario based experiment we will research whether these factors influence the MUM effect. Using this information, we might get more insight on what development method to use in order to minimize the MUM effect. Besides that, it might be useful to see which factors contribute most to the MUM effect and see how the development methods could be adjusted to minimize this contribution.

The remainder of this document is organized as follows. First, some background material will be discussed, focusing on the MUM effect, the Waterfall method and Scrum. Next, the hypotheses derived from the theoretical background will be presented. After that, the research methodology will be ex-plained. Then we show the results of this study and give an overview of its implications for practice and research. Fur-thermore, we discuss the limitations of this study and finally come to a conclusion in the last section.

THEORETICAL BACKGROUND

This section discusses the literature on the MUM effect in a general context as well as in an organizational context. Be-sides that, the two software development methods Waterfall and Scrum are shortly introduced as well as how they differ. For each aspect that differs, it is also briefly explained how it might influence the MUM effect.

(3)

The MUM Effect and Whistle Blowing

The MUM effect has been defined by Tesser and Rosen as the reluctance to communicate undesirable information [30]. In this study the existents of the phenomenon was tested, which was until then only assumed to exist. After that, several other studies attempted explain the MUM effect. An overview of this literature is given by Miceli and Near [23].

In the organizational context, the research in this area is mostly grounded in the whistle blowing literature. Whistle blowing is contradictory to keeping mum and can be seen as “the disclosure by organizational members of illegal, im-moral, or illegitimate organizational acts or omissions to par-ties who can take action to correct the wrongdoing” [23]. However, according to Jubb [19], there are many definitions used in the literature. Therefore his study attempted to clarify the definition: it all comes down to “sharing a type of in-formation which entails disclosure, accusation and dissent”. Even though whistle blowers can improve the effectiveness of the organization, by pointing out problems or even pro-viding solutions to these problems, whistle blowers are not always seen as heroes [23]. Therefore, the potential whistle blower has to consider whether to share the bad news or to re-main mum. Dozier and Miceli [13] created a whistle blowing model that includes two factors that influences an individuals willingness to report. According to this basic Whistle Blow-ing Model, the individual first assesses if the news ought to be reported and then considers to what extend it is his or her personal responsibility to report. These factors influence how willing the individual is to report the bad news.

The MUM Effect in Software Development Context

Since the MUM effect was established as important contribu-tor to the failure of IT projects [26], additional research within the narrower scope of software development was provided. Most of these studies have built further on the whistle blow-ing literature and are focused on additional factors that might influence someones willingness to report bad news about an IT project.

For example, one study [26] tried to extend the basic whistle blowing model by adding an extra variable that is important in the context of IT projects. It found that fault responsibility of outsourced IT projects involving an external vendor, as well as the variable of time urgency, have a significant influence on a project members willingness to report bad news. This study involved a controlled laboratory experiment.

Another study [33] found some support for two other fac-tors that might influence the reluctance to report: the level of impact from failure and the behavioral wrongdoing. In this research a similar method was used to the previous one. One other example of a study that works on the development of the MUM Effect theory is [22], which proposed a model on top of the basic theoretical model from the whistle blow-ing theory by addblow-ing two factors: information asymmetry and organizational climate. These factors also appeared to indi-rectly influence the willingness to report. Another closely re-lated study [32] tested these relationships in a cross-cultural context.

Another valuable study [32] presented a theoretical model which draws on literature from all different kinds of fields with the goal to provide more insight into the topic. How-ever, this study noted that no single body of the literature pro-vides a direct explanation for all aspects of the MUM effect and shows how complex the phenomenon actually is. They claimed that it would probably be impossible for any one study to consider all the factors.

Previous work on the topic of the MUM effect within IT project has mostly focused on direct extension of the Whistle Blowing Model and hereby failed to address whether there is an actual difference in willingness to report caused by the chosen development method. We assume that the present study is the first one that attempts to shed some light on this new research area by comparing the MUM effect in two dif-ferent software development environments; one working with a Waterfall and the other with a Scrum development method. Therefore we will briefly introduce the traditional Waterfall Model as well as the Scrum approach and make a compari-son between them.

The Waterfall Model

The Waterfall model, illustrated in Figure 1 (see Appendix E), was introduced by Royce in 1970 [24]. It is a traditional se-quential development methodology consisting of six phases; analysis, design, development, testing, implementation and maintenance. Each phase has its own specified time frame and the order of the phases is fixed. One can only continue to the next phase after the previous phase is completed [3]. Going back to a previous phase is possible, but it is not al-lowed to go further back in the process [8]. Even though the Waterfall model is quite a rigid way of developing, it is still widely used in government projects as well as in many major companies [24].

Scrum

The Scrum methodology, illustrated in Figure 2 (see Ap-pendix E) was first published about by Schwabler [31] in 1995. Scrum works with iterations called sprints. A sprint is a short working period that leads to a potentially shippable product increment. All the work that has to be done can be found in the product backlog. During the sprint planning is determined which tasks from the product backlog are most important at that point and have to be included in the Sprint backlog. This sprint backlog contains the items that have to be done during the sprint. Today, this has become the most popular methodology that is consistent with the Agile Mani-festo [4][8].

Comparing Waterfall and Scrum

For the purpose of this study we are interested in the con-tradictory aspects of the development methods Waterfall and Scrum. There are several comparative studies on Waterfall and Scrum, which are quite similar [3] [33]. In the literature, four contradictory aspects have repetitively been mentioned. This section highlights these aspects and gives an explanation on how they might influence the MUM effect.

(4)

Flexibility of the Method

The flexibility of the development method differs in a Water-fall and Scrum environment. When a problem occurs during a IS project, the flexibility might influence the ease with which a problem can be corrected and thus how big the problem can become. We therefore believe that this factor is closely re-lated to project risk. Which according to[33] might influence whether a problem ought to be reported. According to the the-ory of the MUM effect, people find it more difficult to report about bad news than good news [30]. We state that a problem is worse when it has greater chances of failing to meet the functionality criteria or overrunning either the budget or time schedule. Therefore we believe that a flexible development method might positively influence the MUM effect.

The Waterfall methodology is quite rigid. The context and deliverables are defined at the start of the IS project and will not be adjusted during the project [31]. It is known that many problems regarding a particular phase arise after the comple-tion of this phase [3]. When using the Waterfall method this is problematic since it is hard to solve problems in earlier stages. In order to solve this problem you either have to go back to a previous phase, or continue the project and let the problem exist. It will eventually result in a badly structured system [3]. When, for example, it is discovered that extra require-ments are needed, this creates a problem since no additional changes are considered after the completion of the analysis phase [3]. We can therefore conclude that the consequences of problems that arise in Waterfall can become relatively big. Scrum on the other hand is a much more flexible method [31]. It has a high degree of tolerance for changes that have to be made during the project. Adjustments to requirements can be made, even in later development stages [3]. Thus the conse-quences of a problem in a Scrum development environment are relatively small.

Communication Frequency and Formality

Another difference between Waterfall and Scrum is the fre-quency and formality of communication. An earlier study suggests that the the type of communication might influence whether bad news about a project’s status is reported [25]. The Waterfall Method is most used by organizations were re-porting on paper is a must and quality control is a major con-cern [24]. Its intensive documentation is inevitable and there are not that many face-to-face meetings as in Scrum. This is known to reduce informal contact [1].

Scrum, on the other hand, is about reporting less on paper and communicating efficiently, which is in person. Several kinds of face-to-face meetings are frequently organized in or-der to support this [31]. For example the 15 minute Daily Scrum meeting at the beginning of each day during which team members share what they did since the last Scrum meet-ing. Furthermore they share problems they encountered and what they plan to work on before the next meeting [4].

Review Moments

Another difference between the Waterfall method and Scrum can be found in the frequency of test or review moments. We believe that the frequent review moments increase the chance

that someone will report about a problem in the IS project, because the problem might be discovered during the review. It creates some sort of time pressure on the individual who knows about the problem. People might think it is less urgent to report the problem if there is more time to get rid of it. This effect was also suggested in a previous study [26].

In the Waterfall model, there is one testing phase very late in the development process. This is problematic since problems might only be found at that stage and not earlier because the test team is only involved from that moment [3].

When working with scrum the review moments are quite fre-quent, usually one to four weeks. During this review moment, each team has to present their progress with a working demo [31]. If there are any problems with the software that was developed until this point, they might be encountered.

Team Spirit

The team spirit also differs between the Waterfall model and Scrum method. When working with the Waterfall method, the team spirit does not have to be strong to make it work [20]. The Waterfall method is quite costly for small teams [20], therefore the teams are often much bigger than when working with Scrum. Team members also work more indi-vidually instead of collaborative like in Scrum. This individ-ualism has the consequence that people might place their own interests first [14]. Meaning that when a problem arises and people fear the consequences of reporting it, for example get-ting fired, they most likely decide to remain mum.

Scrum teams, on the other hand, are quite small and exist of three to six people who work on a collaborative basis. The team spirit of Scrum teams is known to be high [31]. Peo-ple might be more likely to put the team interest above their own personal interest and decide to report bad news sooner, because it can benefit the team [14]. In this case the team will be able to work on the problems if they know about them.

RESEARCH QUESTION AND HYPOTHESES

As mentioned in the introduction of this paper, the main re-search question is: Does Scrum provide a better environment for minimizing the MUM effect compared to the traditional Waterfall model? The null hypothesis that follows from this research question is:

H0: The willingness to report bad news about an IS project

is equal in projects that work with the Waterfall method and in projects that work with the Scrum method.

and the alternative hypothesis:

HA: The willingness to report bad news about an IS project

is not equal in projects that work with the Waterfall method and in projects that work with the Scrum method.

As indicated by the literature, it is not realistic to look at all the aspects within the development methods at once. There-fore we choose to focus on four contradictory aspects be-tween the traditional Waterfall Model and the Scrum method as described in the theoretical background (see Table 1). These aspects are: flexibility of the project, frequency and formality of communication, review procedure, and team

(5)

spirit and responsibility for work. In this study we also want to discover if these aspects influence the MUM effect indi-vidually. We therefore stated four additional hypotheses re-garding the aforementioned aspects. Table 2 lists their null hypotheses as well as the alternative hypotheses.

RESEARCH METHODOLOGY

The main objective of this study is to test whether the MUM effect differs in the development environments Scrum and Waterfall. Moreover we are interested in four specific con-tradictions between these environments and how they affect the MUM effect. We addressed this problem with a quantita-tive approach.

The idea is to measure the MUM effect whilst one group of participants is working on an IS project in a Waterfall de-velopment environment and an other group is working in a Scrum development environment.

Internal validity would be an issue if we would perform the test in the actual working field. Moreover, in every company the development environment differs and extraneous factors might influence the MUM effect as well. Therefore we ducted a scenario based experiment, in which we could con-trol the variables and perform measurements based on similar settings [12]. Another reason for using scenarios is that this method is less expensive and quicker to carry out than obser-vational studies and that it makes it possible to collect data from a larger participant group [29].

We developed two scenarios for the purpose of this study; sce-nario 1 described a Waterfall development environment and scenario 2 a Scrum development environment. In these sce-narios we manipulated four variables at two levels. These variables are: the flexibility of the development method (in-flexible versus (in-flexible), the communication frequency and formality (infrequent and formal versus frequent and infor-mal), the review moments planned (infrequent versus fre-quent) and lastly the team spirit and responsibility (low team spirit and individual responsibility versus high team spirit and shared responsibility). The dependent variable was the will-ingness to report bad news.

The remainder of the research methodology section is orga-nized as follows; first the scenarios will be described as well as how the variables were manipulated within these scenarios. Next, the experiment procedure is explained. Then, some de-tails about the participants are provided. After that, the mea-sures and tools used are explained and finally the data analysis approach is described.

Scenarios

The two scenarios (see appendix A) developed for this study have a similar construction and consist of four paragraphs. During the creation of the scenarios we were inspired by the methodology of creating scenarios as described in [34]. Even though this method is focused on the development of scenar-ios about the future, this technique is not restricted to this purpose [29].

Paragraph 1: Introduction of the Scenario

In the first paragraph of the scenario, the participant is in-troduced to the organization in which he or she has the role of a software developer. The participant discovered a major problem while working on the development of a new infor-mation system in Project Y and unless the participant decides to shares the bad news, he or she is the only one who is aware of this problem.

This paragraph is a slightly adjusted version of a scenario originally developed by Keil [22]. We decided to adopt the first part of the original scenario and use this as the basis for the scenario used in this study, since it has undergone vari-ous pilot tests and refinements and has proven to be useful to test hypotheses related to sharing bad news in IT settings [22] [21].

Another good property of this introduction is that the partici-pants were asked to assume the role of a software developer, rather than answering the question based on their personal function, which reduces the social desirability effect [5] . The most important differences between the original scenario and the one used in this study is the role the participant has to take. We changed it from development project manager to developer, since we believe it would be easier for the partici-pant to imagine to be in the role of a software developer than a project manager. Besides that, it suits this study better, since we are interested in reporting bad news to the management. For the purpose of this study we also added a cause of the problem to better fit the remainder of the scenario.

We continued the scenario with three new paragraphs devel-oped for the purpose of this study. The content of the new paragraphs are based on the findings from the literature intro-duced earlier in the background section of this paper. How-ever, we did shorten and simplify the explanations to make it easier to read and more understandable for the participants, even for those who do not have any experience with the de-velopment methods Scrum or Waterfall.

Paragraph 2: Flexibility of the Development Method

In the second paragraph of the scenario, the flexibility of the development method is manipulated by giving the participant a short explanation of the software development method used. In scenario 1 the general Waterfall method is described as quite rigid, since the development process is divided into dif-ferent sequential phases. Emphasis is placed on its low degree of tolerance for making adjustments in previous phases. In scenario 2 the Scrum method is explained. It is described as a more flexible method. This time the emphasis lies on its high degree of tolerance for changes, even later in the devel-opment process.

Paragraph 3: Review/Testing Procedure

In the third paragraph we manipulated the testing procedure, by adjusting the time left before a new test procedure was planned and by whom it was performed. This has conse-quences for the amount of work that has to be (re)done to fix the problem.

(6)

Factors Waterfall Scrum

Development Method Flexibility Rigid Flexible

Communication Frequency and Formality Infrequent, formal Frequent, informal Review Moments Planned Infrequent Frequent

Team Spirit and Work Responsibility Low, individual High, shared

Table 1. Four contradictions between Waterfall and Scrum

Null hypothesis Alternative hypothesis Development Method Flexibility

H1: There is no difference in the mean willingness to

re-port bad news when the Scrum development environment would be less flexible.

H1A: There is a difference in the mean willingness to

re-port bad news when the Scrum development environment would be less flexible.

re-port bad news when the Waterfall development environ-ment would be more flexible.

report bad news when the Waterfall development envi-ronment would be more flexible.

Communication Frequency and Formality

re-port bad news when the communication in the Scrum de-velopment environment would be less frequent and more formal.

report bad news when the communication in the Waterfall environment would be more frequent and less formal.

report bad news when the communication in the Waterfall environment would be more frequent and less formal. Planned Review Moments

re-port bad news when the review moments in the Scrum de-velopment environment would be planned less frequent.

re-port bad news when the review moments in the Scrum de-velopment environment would be planned less frequent. H6: There is no difference in the mean willingness to

report bad news when the review moments in the Wa-terfall development environment would be planned more frequent.

Team Spirit and Work Responsibility

report bad news when team spirit would be lower in the Scrum development environment and the work responsi-bility would be with the individual.

report bad news when team spirit would be higher in the Waterfall development environment and the work respon-sibility would be shared.

(7)

halfway the development process when the problem was dis-covered. Besides that, the participant was told that the prob-lem might be discovered during a testing procedure which takes place once all code is written and carried out by a test team which is only involved during this phase.

In Scenario 2 the participant is informed that the team was working on one of many two-week sprints when the problem was discovered. Here the participant was told that the test session takes place at the end of the sprint and is carried out by the team itself and also reviewed by the client.

Paragraph 4: Communication, Team Spirit and Work Respon-sibility

In the last paragraph two variables were manipulated: the variable communication frequency and formality and the variable team spirit and work responsibility.

In Scenario 1 the participant is informed that the communi-cation is very document intensive. There are not that many face to face meeting organized to share problems encoun-tered. Communication among project members and the man-agement is quite formal and done mostly through written doc-uments. Besides that, the participant was informed that he or she is part of a large team of 30 people, who work mostly on an individual basis. The responsibility for the work lies with the individual, thus with the participant.

In Scenario 2 the participant is informed that the communi-cation is done mostly in person. There are many face to face meetings organized to share problems encountered. The com-munication among everyone involved in the project is there-fore informal. Besides that, the participant was also informed that he or she is part of a small team of 6 people who work together on a collaborative basis. The problems faced are mostly shared with each other as well as the responsibility for the work that was performed.

Experiment Procedure

Two surveys were created, one for the Waterfall scenario and one for the Scrum scenario. The participants were randomly assigned to one of the surveys and received a link to the cor-responding online survey. Before the participants enrolled in the experiment, they were presented with an informed con-sent, so they could make an informed decision on whether to take part in this study.

The participants were promised complete confidentiality and were informed that there were no right or wrong answers in the questionnaire as was suggested by Keil [22]. This was done in order to increase the likeliness that participants would share their honest opinion about the statements presented af-ter the scenario and worry less about their answers not being correct or socially acceptable. Subsequently, the participants were asked to fill out the scenario based survey which consists of three parts.

The first part was intended to gain some basic demographic information. The participant was asked about his gender, age, level of education and current employment status.

In the second part of the survey the participant was asked to read a short scenario about an unhealthy IS project.

The last part consists of three types of statements about the scenario. The first statement that was included was to test whether the MUM effect differed between the Waterfall and Scrum scenarios (see Appendix C). The second set of state-ments were used to determine whether each of the four fac-tors influenced the participants willingness to report bad news about the IS projects (see Appendix B). A last set of four statements was included to check if the factors in the scenar-ios were manipulated effectively (see Appendix D).

Participants

A total of 124 subjects participated this study. For this study it is important that the participants have experience - or at least affinity - with the development of ICT projects, since they should be capable of understanding the context of the scenario and assume the role of a software developer in this setting. We therefore selected the participants by convenience and snowball sampling . The participants varied in gender, age, educational level and working status.

Approximately 80% of the participants are male and 20% are female. Among them, 33% is under 25, 41% between 25 and 35, and the rest were between 35 and 65 years old.

The educational level of the participants differed as well. They either were in/or completed university (56%), university of applied sciences (27%) or intermediate vocational educa-tion (15%).

Among the participants, 36% were students in Information Science, Artificial Intelligence or Informatics and half of them also has a part-time job in the ICT field. The other 64 % of the participants had professional working experience in the ICT sector.

Measures

Demographic Information

Several demographic questions were included in order to pro-vide background information about the participant. These were all multiple choice questions, since this made it easier to transform the answers into numerical data.

Scenario Manipulation Checks

Manipulation checks (see Appendix D) were performed to verify whether the independent variables were effectively ma-nipulated. The perceived factors were measured using a seven point Likert scale, as recommended [2], since it allows the participant to give a more precise answer. The Likert scale ranging from 1, strongly disagree to 7, strongly agree.

The MUM Effect in the Development Environments Waterfall and Scrum

The subjects willingness to report bad news given a certain development environment is measured on a seven point Likert scale ranging from 1, very unlikely to 7, very likely. The statement (see Appendix C) and its measurement scale were adopted from Keil [22]. However, we slightly adjusted them to better fit the scenarios.

(8)

Influence of each Individual Factor on the Willingness to Re-port

For each scenario we tested how the willingness to report bad news chances when this factor would be different and the rest of the factors within that scenario stay the same. The state-ments that were used for this are included in Appendix B. The chance in willingness to report was measured on a seven point Likert scale that ranges from 1, strongly disagree to 7, strongly agree. We compared this data to the neutral value of 4, to see how the willingness to report would chance.

Tools

The data was collected using Google Forms [15], an on-line survey tool. We used Google Spreadsheets [16] to put the data in the right format and then analyzed it through IBM SPSS Statistics 24 [35].

Data Analysis

Independent-Samples T-test

The measurements of the manipulation checks and the over-all willingness to report bad news in the development envi-ronments were analyzed using independent-samples t-tests. This test is a common method for determining if a statisti-cally significant difference exists between the means of two independent groups on a continuous dependent variable [36]. We followed the procedure of conduction of independent-samples t-tests as outlined by Laerd Statistics [36] which describes that in order to run an independent-samples t-test, there are six assumptions that needed to be considered. 1. One dependent variable measured at the continuous level.

- This assumption is met for all cases, since we measure the items at a seven point Likert scale, which can be con-sidered a continuous scale, since Likert scaling presumes the existence of an underlying continuous variable [7]. 2. One independent variable that consists of two

indepen-dent groups. - This assumption is met as well since we did all measurements with two groups.

3. Independence of observations - This assumption is met also since there is no relationship between the groups. 4. Preferably no significant outliers because of the effect

they might have on the results. - We produced boxplots to identify the outliers in each test. A few outliers were identified, therefore we ran two tests; one test with the out-liers included and another test without the outout-liers. After that, the results were compared and it was noticed that the two results did not differ sufficiently, since the conclusions would essentially be the same [36]. Based on this we de-cided to keep the outliers in the data. We believe this was possible because of the large sample size per group (N = 62), the effect of the outliers on the results was therefore minimal.

5. The dependent variable should be normally distributed for each group of the independent variable - According to the Central Limit Theorem the means of groups with a sample size greater than 10, are approximately normally

distributed regardless of the original distribution [2]. This assumption was therefore met as well since each group has an N value of 62.

6. Homogeneity of variances - This was assessed by Lev-ene’s test for equality of variances. The independent t-test gives two results that were calculated differently, one for when the assumption is met and the other when it is not.

One-Sample T-test

The measurements of the individual factors on the willingness to report bad news in the development environments were an-alyzed using one-samples t-tests. One-sample t-tests are used in order to compare values from a sample to another value, which is in this case is the neutral value on the Likert scale (4). Again we followed the instructions for conduction of this test by Laert Statistics [36]. The assumptions 1, 3, 4 and 5 mentioned in the in the previous section apply here as well.

Cohen’s d

For the independent-samples t-tests, we also measured the ef-fect sizes with Cohen’s d using an on-line calculator1. For the one-sample t-test on the other hand, we had to calculate this value by dividing the mean difference by the standard de-viation of the difference [36]. According to the guidelines of [9] the value indicates the size of the effect ; 0.2 is small, 0.5 medium and 0.8 large.

Power Analysis

Lastly we performed a power analysis for each statement. There are no formal standards to interpret the value of the power, however a value of 0.80 is know to be acceptable [29]. Again we used an on-line calculator2for the computation.

RESULTS

This section presents the results of the scenario manipulation checks, the influence of the development method on the will-ingness to report bad news and finally the influence of the four individual factors on the willingness to report bad news.

Scenario Manipulation Checks

Table 3 shows the mean values and standard deviations of each perceived factor in each scenario. Here it can be seen that the mean values are in the intended direction. Thus the mean value for perceived flexibility is higher in the Scrum scenario (M = 5.13, SD = 1.00) than in the Waterfall scenario (M = 2.74, SD = 1.62). The mean value of the perceived frequent and informal communication is higher in the Scrum scenario (M = 5.47, SD = 1.08) than in the Waterfall scenario (M = 2.85, SD = 1.70). When we look at the mean value of the perceived factor “frequent review moments” we see that it is higher in the Scrum scenario (M = 5.55, SD = 0.99) than in the Waterfall scenario (M = 2.87, SD =1.52). Also the mean “High team spirit and shared responsibility” perceived is higher in the Scrum scenario (M = 4.82, SD = 1.12) than the Waterfall scenario (M = 2.95, SD = 1.51).

For each of factors the null hypothesis is formulated that:

1

http://www.danielsoper.com/statcalc/calculator.aspx?id=48

(9)

H0: The mean perceivedfactor X is equal in both scenarios.

and the alternative hypothesis is:

HA: The mean perceivedfactor X is not equal in both

sce-narios.

So for example, for the factor flexibility of the development method the null hypothesis is: H0: The perceived flexibility

of the development method is equal in both scenarios. and the alternative hypothesis: HA: The perceived flexibility of

the development method is not equal in both scenarios. We then tested whether the differences in means were statis-tically significant by conducting an independent-samples test for each factor.

Perceived Flexibility of the Development Method

An independent-samples t-test was conducted to compare the mean perceived flexibility of the development method in the Waterfall scenario and in the Scrum scenario. There were two outliers in the data, as assessed by inspection of a boxplot, these were included anyway. Besides that, the homogeneity of the variances was violated as assessed by Levenes test for equality of variances (p = .000032). There was a statistical significant difference in the perceived development flexibility scores for the Waterfall and the Scrum scenario M = −2.39, 95% CI [−2, 87, −1, 91], t(101.69) = −9.88, p = 0.00, d = 1.78, power = 1.00 (see the first row in Table 4). Therefore the null hypothesis is rejected and the alternative hypothesis can be accepted. There also was a high effect size and a high power value. The results indicate that the mean perceived flexibility of the development method is higher in the Scrum scenario than in the Waterfall scenario which is as expected. Thus the manipulation for the factor flexibility of the devel-opment method was effective.

Perceived Communication Frequency and Formality

Another independent-samples t-test was conducted to com-pare the mean perceived frequent and informal communi-cation in the Scrum scenario and in the Waterfall scenario. There were two outliers in the data, as assessed by inspec-tion of a boxplot. Besides that, the homogeneity of variances was violated, as assessed by Levenes test for equality of vari-ances (p = .000057). As can be seen by reading the sec-ond row in Table 4, there was a significant difference in the mean perceived frequent and informal communication scores for the Waterfall and the Scrum scenario M = −2.61, 95% CI [−3.12, −2.10], t(103.57) = −10.22, p = 0.00, d = 1.83, power = 1.00. Therefore the null hypotheses is rejected and the alternative hypothesis can be accepted. This result shows that the mean extent to which the communication is perceived as frequent and informal is higher in the Scrum scenario than in the Waterfall scenario. Thus the manipulation for the factor communication frequency and formality is effective.

Perceived Review Procedure

We conducted a third independent-samples t-test to compare the mean perceived frequency in the Scrum scenario and in the Waterfall scenario. By inspecting a boxplot we discov-ered one outlier in the data, which was included also. Besides that the homogeneity of variances was violated, as assessed

by Levenes test for equality of variances (p = .002). There was a significant difference in the mean perceived review fre-quency for the Waterfall and the Scrum scenario M = −2.68, 95% CI [−3.13, −2.22], t(104.63) = −11.63, p = 0.00, d = 2.09, power = 1.00, as can also been seen on the third row of Table 4. Therefore the null hypothesis is rejected and the alternative hypothesis can be accepted. This result indicates that the mean perceived frequency of the review moments is higher in the Scrum scenario than in the Waterfall scenario. Therefore we consider the manipulation for the factor review frequency effective.

Perceived Team Spirit and Responsibility

We conducted fourth independent-samples t-test to compare the mean perceived team spirit and shared responsibility in the Scrum scenario and in the Waterfall scenario. There were two outliers in the data, as assessed by inspection of a box-plot and there was homogeneity of variances, as assessed by Levenes test for equality of variances (p = .06). There was a significant difference in the mean perceived team spirit and shared responsibility for the Waterfall and the Scrum scenario M = −1.87 , 95% CI [−2.34, −0.08], t(122) = −7.83, p = 0.00, d = 1.41, power = 1.00 (see the last row in Table 4). Therefore the null hypothesis is rejected and the alternative hypothesis can be accepted. This indicates that the mean per-ceived team spirit and shared responsibility is higher in the Scrum scenario than in the Waterfall scenario. Thus the ma-nipulation for the factor team spirit and responsibility is ef-fective.

Since it is now shown that the scenarios were successfully manipulated based on the four factors we can continue to the actual hypotheses.

Development Method and Willingness to Report Bad News

An independent-samples t-test was run to determine if there were differences in the mean willingness to report bad news about an IS project between Waterfall projects and Scrum projects. There were no outliers in the data, as assessed by inspection of a boxplot and there was homogeneity of vari-ances, as assessed by Levenes test for equality of variances (p = .286). Table 5 gives an overview of the descriptive statis-tics of the Scrum group and the Waterfall group. Here it can be seen that the mean willingness to report score was higher in Scrum projects (M = 5.82, SD = 1.15) than in Waterfall projects (M = 5.77, SD = 1.38). However this difference was not statistically significant M = 0.05, 95% CI [−0.40, 0.50], t(122) = 0.21, p = 0.83, d = 0.04, power = 0.04, as can also be seen in Table 6. These results indicate that we failed to reject the null hypothesis. Important to note is that when we look at the Cohen’s d value, it can be seen that the effect size is rather small. Besides that there is a low power. This provides extra information about this result, because a non-significant result in combination with a low statistical power to detect this effect suggest that another study, or a higher sample size is required before we can draw any further conclusions [29].

Flexibility of the Software Development Method and Will-ingness to Report Bad News

(10)

Scenario N Mean SD Waterfall 62 2.74 1.62 Flexible Development Method

Scrum 62 5.13 1.00 Waterfall 62 2.85 1.70 Frequent and Informal Communication

Scrum 62 5.47 1.08 Waterfall 62 2.87 1.52 Frequent Review Moments

Scrum 62 5.55 0.99 Waterfall 62 2.95 1.51 High Team Spirit and Shared Responsibility

Scrum 62 4.82 1.12

Table 3. Perceived factors within the scenarios

Variable t df Sig. Mean difference CI lower CI upper d power Flexible development method −9.88 101.69 0.00 −2.39 −2, 87 −1, 91 1.78 1.00 Frequent and Informal Communication −10.22 103.57 0.00 −2.61 −3.12 −2.10 1.83 1.00 Frequent Review Moments −11.63 104.63 0.00 −2.68 −3.13 −2.22 2.09 1.00 High Team Spirit, Shared Responsibility −7.83 122 0.00 −1.87 −2.34 −0.08 1.41 1.00

Table 4. Results of the Independent-samples t-test, Cohen’s d and Power for the perceived factors in the Scrum and Waterfall development environ-ments. CI refers to the confidence intervals of the mean difference, which is set at 95%.

If the Scrum Development Environment Would be Less Flexi-ble

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the development method in the Scrum environ-ment would be less flexible (H1). There were no outliers in

the data, as assessed by inspection of a boxplot. The mean willingness to report score in the sample (M = 4.05, SD = 1.79) was higher than the ’neutral’ score of 4.0 (see Table 7). However this is not a statistically significant mean difference, 0.05, 95% CI [−0.41 to 0.50], t(61) = 0.21, p = 0.83, d = 0.03, power = 0.03, as can be seen in Table 8. Therefore we failed to reject the null hypothesis. Based on this result we can say that if the Scrum development environment would be less flexible it would not significantly influence the willing-ness to report score. Besides that, there is a small effect size and a low power, this indicated that a higher sample size is required in order to draw any further conclusions [29].

If the Waterfall Development Environment Would be More Flexible

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that flexibility of the development method in the Wa-terfall environment would be more flexible (H2). There were

no outliers in the data, as assessed by inspection of a boxplot. The mean willingness to report score in the sample (M = 5.21, SD = 1.69) was higher than the ’neutral’ score of 4.0 (see Ta-ble 7). This is a statistically significant mean difference of 1.21, 95% CI [ 0.78 to 1.64 ], t(61) = 5.64, p = 0.00, d = 0.72, power = 0.80, as can be seen in Table 8. Therefore, we can reject the null hypothesis and accept the alternative hypothe-sis. When we look at the Cohen’s d value, we see that there is a medium effect. The probability that we correctly lead to rejection of a false null hypothesis is expressed by the power value, which is in this case it is high. This result indicates that the willingness to report bad news in the Waterfall

develop-ment method could be increased by making the project more less rigid in some way.

Frequency and Formality of Communication and Willing-ness to Report Bad News

If the Communication in the Scrum Environment would be Less Frequent and More Informal

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the communication in the Scrum environment would be less frequent and more formal (H3). There were

no outliers in the data, as assessed by inspection of a boxplot. The mean willingness to report score in the sample (M = 3.79, SD = 1.75) was lower than the ’neutral’ score of 4.0 (see Ta-ble 7). However this is not a statistically significant mean difference, −0.21, 95% CI [−0.65 to 0.23], t(61) = −0.95, p = 0.35, d = 0.12, power = 0.07, as can be seen in Table 8. Therefore we failed to reject the null hypothesis. This indi-cates that when Scrum communication would be less frequent and more informal it would not lead to a lower willingness to report bad news. However, there is a small effect size and a low power, again this indicates that no further conclusions can be drawn at this point, and another study, or a higher sample size is needed to further research this[29] .

If the Communication in the Waterfall Environment would be more Frequent and less Formal

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the communication in the Waterfall environment would be more frequent and less formal (H4). There were 5

outliers in the data, as assessed by inspection of a boxplot, but these were included anyway. The mean willingness to re-port score in the sample (M = 5.34, SD = 1.49) was higher than the ’neutral’ score of 4.0 (see Table 7). This is a statis-tically significant mean difference of 1.34, 95% CI [ 0.96 to 1.72 ], t(61) = 7.06 p = 0.00, d = 0.90, power = 0.94, as can

(11)

Categories N Mean SD Scrum 62 5.82 1.15 Development Method

Waterfall 62 5.77 1.38

Table 5. Willingness to report bad news about a IS project, in two development environments Scrum and Waterfall. More over, each row in the table presents statistics on the dependent variable ”Willingness to report”, for the different categories, Waterfall and Scrum, of the independent variable ”Development Method”. ”SD” stands for Standard Deviation.

Independent Variable t df Sig. Mean difference CI lower CI upper d Power Development method −0.21 122 0.83 0.05 −0.40 0.50 0.04 0.04

Table 6. Results of the Independent-samples t-test, Cohen’s d and Power for the willingness to report bad news in the Scrum and Waterfall development environments. CI refers to the confidence intervals of the mean difference, which is set at 95%.

be seen in Table 8. Therefore, we can reject the null hypoth-esis and accept the alternative hypothhypoth-esis. When we look at the Cohen’s d value, we see that there is a large effect. The probability that we correctly lead to rejection of a false null hypothesis is expressed by the power value, which is in this case it is very high. This result suggests that the willingness to report bad news in the Waterfall development method could be increased by communicating on a more frequent basis in an informal way.

Frequency of Planned Review Moments and Willingness to Report Bad News

If the Review Moments in the Scrum Environment would be less Frequent

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the review moments in the Scrum environment would be planned on a less frequent basis (H5). There were

no outliers in the data, as assessed by inspection of a boxplot. The mean willingness to report score in the sample (M = 4.53, SD = 1.84) was higher than the ’neutral’ score of 4.0 (see Ta-ble 7). This is a statistically significant mean difference of 0.53, 95% CI [ 0.06 to 1.00 ], t(61) = 2.27, p = 0.03, d = 0.29, power = 0.20, as can be seen in Table 8. Therefore, we can reject the null hypothesis and accept the alternative hypothe-sis. When we look at the Cohen’s d value, we see that there is a small effect. The power value is also low. This result indicates that the willingness to report bad news in the Scrum development method could be increased by planning review moments on a less frequent basis.

If the Review Moments in the Waterfall Environment would be more Frequent

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the review moments in the Waterfall environ-ment would be planned on a more frequent basis (H6). There

were no outliers in the data, as assessed by inspection of a boxplot. The mean willingness to report score in the sample (M = 4.90, SD = 1.69) was higher than the ’neutral’ score of 4.0 (see Table 7). This is a statistically significant mean difference of 0.90, 95% CI [ 0.48 to 1.33 ], t(61) = 4.22, p = 0.00, d = 0.53, power = 0.54, as can be seen in Table 8. Therefore, we can reject the null hypothesis and accept the alternative hypothesis. When we look at the Cohen’s d value, we see that there is a medium effect. The power value is on

the low side. This result suggests that the willingness to re-port bad news in the Waterfall development method could be increased by planning review moments on a more frequent basis.

Team Spirit and Work Responsibility and Willingness to Report Bad News

If the Team Spirit Would be Lower in the Scrum Environment and Work Responsibility would be with the Individual

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the team spirit in the Scrum environment would be lower and the work responsibility would lie with the indi-vidual (H7). There were no outliers in the data, as assessed

by inspection of a boxplot. The mean willingness to report score in the sample (M = 4.13, SD = 1.74) was higher than the ’neutral’ score of 4.0 (see Table 7). However this is not a statistically significant mean difference, 0.13, 95% CI [−0.31 to 0.57], t(61) = −0.60, p = 0.56, d = 0.07, power = 0.05, as can be seen in Table 8. Therefore we failed to reject the null hypothesis. This result indicates that the willingness to re-port bad news in a Scrum development environment would not change when the team spirit would be lower or when re-sponsibility would shift to the individual. But there is also a small effect size and a low power in combination with a non-significant result. This suggests that another study, or more sampling is required before any further conclusions can be drawn [29].

If the Team Spirit would be Higher in the Waterfall Environment and Work Responsibility would be Shared

An one-sample t-test was run to determine whether the will-ingness to report score was different to neutral score of 4.0 in the case that the team spirit in the Scrum environment would be lower and the work responsibility would be with the indi-vidual (H8). There were no outliers in the data, as assessed

by inspection of a boxplot. The mean willingness to report score in the sample (M = 4.79, SD = 1.55) was higher than the ’neutral’ score of 4.0 (see Table 7). This is a statistically significant mean difference of 0.79, 95% CI [ 0.40 to 1.18 ], t(61) = 4.02, p = 0.00, d = 0.51, power = 0.50, as can be seen in Table 8. Therefore, we can reject the null hypothesis and accept the alternative hypothesis. When we look at the Cohen’s d value, we see that there is a medium effect. The power value is on the low side. This result indicates that the willingness to report bad news in the Waterfall development

(12)

Independent Variable Scenario and condition (if it would be..) N Mean SD Scrum: Less flexible 62 4.05 1.79 Development Method Flexibility

Waterfall: More flexible 62 5.21 1.69 Scrum: Infrequent and formal 62 3.79 1.75 Communication Frequency and Formality

Waterfall: Frequent and informal 62 5.34 1.49 Scrum: Infrequent 62 4.53 1.84 Review Moments Planned

Waterfall: Frequent 62 4.90 1.69 Scrum: Low team spirit, individual responsibility 62 4.13 1.74 Team Spirit and Work Responsibility

Waterfall: High team spirit, shared responsibility 62 4.79 1.55

Table 7. Willingness to report bad news about a IS project, measured for each independent variable in two different conditions. More over, each row in the table presents the descriptive statistics of the independent variable for every category on the dependent variable ”Willingness to report”. Development Method”. ”SD” stands for Standard Deviation.

Variable t df Sig. M difference CI lower CI upper d power Development method flexibility

Scrum: Less flexible 0.21 61 0.83 0.05 −0.41 0.50 0.03 0.03 Waterfall: More flexible 5.64 61 0.00 1.21 0.78 1.64 0.72 0.80 Communication Frequency and Formality

Scrum: Infrequent and Formal −0.95 61 0.35 −0.21 −0.65 0.23 0.12 0.07 Waterfall: Frequent and informal 7.06 61 0.00 1.34 0.96 1.72 0.90 0.94 Review Moments Planned

Scrum: Infrequent 2.27 61 0.03 0.53 0.06 1.00 0.29 0.20 Waterfall: Frequent 4.22 61 0.00 0.90 0.48 1.33 0.53 0.54 Team Spirit and Work Responsibility

Scrum: Low team spirit, individual responsibility 0.60 61 0.56 0.13 −0.31 0.57 0.07 0.05 Waterfall: High team spirit, shared responsibility 4.02 61 0.00 0.79 0.40 1.18 0.51 0.50

Table 8. The variable would be different how would it influence the Willingness to report. These are the results of the One-sample t-tests (with a test value of 4), Cohen’s d and Power for the willingness to report bad news in the Scrum and Waterfall development environments. CI refers to the confidence intervals of the mean difference, which is set at 95%.

environment could be increased by working on a higher team spirit and a shifting the work responsibility from the individ-ual towards the team.

DISCUSSION

The main purpose of this study was to discover if there is a difference in the willingness to report between the develop-ment environdevelop-ments Scrum and Waterfall and which of these methods thus provides a better development environment for minimizing the MUM effect. We used a quantitative approach with a sample size of 124 and independent-samples t-tests to explore the possible differences. Hereby we focused on the four contradictory factors of the development methods Wa-terfall and Scrum. Surprisingly, no significant difference be-tween the mean willingness to report of the two development environments was found.

In this study we also wanted to discover whether the aspects influenced the MUM effect individually in each development environment. For the Scrum development environment we found that the willingness to report score would not signifi-cantly change when the environment would be less flexible, when communication would be less frequent and more infor-mal, or when the team spirit would be lower or when respon-sibility would shift to the individual. However we found that the willingness to report bad news could maybe be increased by planning review moments on a less frequent basis.

For the Waterfall development environment on the other hand, we found that changing each individual factor would increase the willingness to report bad news. Thus the willing-ness to report bad news in an IS project could be increased by making the project more less rigid, communicating on a more frequent basis in an informal way, planning review moments on a more frequent basis, working on a higher team spirit and shifting the work responsibility from the individual towards the team.

Limitations

It is important to note that this study has its limitations when generalizing the results. First of all, in this study we were interested in the differences of the MUM effect in the devel-opment environments Waterfall and Scrum. Since it is not possible to control every aspect that influences the MUM ef-fect, we choose to focus only on four contradictory aspects of the development methods. The scenarios were built around these factors, which made the environment for testing the hy-potheses more controllable. But it is important to note that there might be other aspects that have something to do with the development environment that influence the MUM effect as well, that were not included in this study. Thus there is no reason to claim that these four factors are an exhaustive set of factors that matter in the development environment and could influence the MUM effect. Besides that we did not measure

(13)

the willingness to report when the four factors would be com-bined differently.

Secondly, although the scenarios were written in such a way that they were as realistic as possible, they were also a sim-plified version of the real world. In a real setting it might be harder to make decisions in situations like the one illustrated in the scenario and other aspects might be considered as well. The third limitation we should take into account is that we measured the intentions of the subjects instead of their actual behaviors when it comes to reporting bad news about an IS project. We can not assure that they would behave in their indicated way.

The fourth limitation here has to do with the way we mea-sured the influence of the willingness to report of each of the four factors. A new set of quite straightforward questions was constructed for this purpose that were derived directly from the hypotheses. Each of these measures were assessed through one single statement, instead of a couple of ques-tions, meaning that we cannot assure that the participant was not guided by the question, because no control questions were included in the survey.

Despite these limitations, we believe that the results of this study provided new insights on the subject matter and con-tributes to the overall understanding of the MUM effect in IS projects and therefore has implications for practice as well as research.

Implications For Practice

At this point, the contribution of this paper will be most valu-able for other researchers who want to further explore this domain. However, the ultimate goal lies in the improvement of the willingness to report in practice. For now, there are some insights based on the results of this study, that could already be taken into account in the practice of software de-velopment. We found that the Scrum method is quite a robust environment when it comes to the influence of the individual factors. Taking away for example the flexibility does not re-ally influence the MUM effect, since the other factors already provide a relatively good environment for sharing bad news. For the Waterfall environment on the other hand, we found that the change of each factor has a great effect on improving the willingness to report. We therefore have four recommen-dations concerning the Waterfall development environment. First of all, the willingness to report bad news in an IS project could be increased by making the project more less rigid. This might have something to do with the developer having the possibility to correct mistakes before it is too late. People might feel less obligated to address this issue in a more flex-ible project. Therefore organizations could look for ways to set up their IS projects in a more flexible manner. We believe that the Agile development methods provide a good example of this.

The second recommendation for improving the willingness to report in a Waterfall environment is chancing the way in which team members communicate with each other. Frequent and informal communication is implied to lower the MUM

effect. Communicating more often in a face-to-face setting, might break a culture of employee silence.

The third aspects that seems to influence the willingness to report is the planning of review moments. For the Waterfall development the results suggest that the frequency of review moments should be increased. Having one single test phase after all code is written might be too late. People might pro-crastinate the problem and just continue working and let the problem exist until then.

Lastly, a higher team spirit and shared work responsibility seem to affect the individuals decision on sharing bad news in a positive way when working in a Waterfall environment. Therefore we recommend to work on the group feeling. As said before, individuals then might be more likely to put the team’s interest above their own personal interest and decide to report bad news, because it can benefit the team. Also when the responsibility for work is not with the individual this person might be less afraid to share problems, because it will not necessarily affect them personally.

Recommendation for Future Research

This is the first study that examined the influence of the de-velopment method on the MUM effect. In this section we discuss three other areas that we recommend for further re-search.

First of all, we recommend to try to repeat the experiment conducted in this study with more participants. Further more, it would be valuable to do some pilot testing and refinements of the scenario and measurements used. It might be a good idea to add several measurements (statements) to measure the same thing and see how they relate to each other using Corn-bach’s Alpha.

Second, as noted before, this is not an exhaustive study. Therefore there is more to discover in this area. In this study we made a comparison between two software development methods Scrum and Waterfall and selected four aspects that might be interesting to look at. However when other devel-opment methods are compared there might be new additional factors that could also influence the MUM effect. Another approach to continue working on in this area is by the con-duction of a qualitative study on this phenomenon focused on finding the elements that people find to influence their will-ingness to share bad news in IS projects that are related to the development environment.

Lastly, it might be valuable to combine all factors that have proved to influence the willingness to report and see how they fit together in the basic Whistle Blowing Model. We believe that the modeling of this phenomenon could increase the un-derstanding of the MUM effect. It might be possible to create a model similar to the well known Technology Acceptance Model (TAM model). This could provide more insights in overcoming the MUM effect in IS project development envi-ronments, so that eventually the MUM effect might become less of a contributor to the failure of IS projects.

(14)

CONCLUSION

In short, the main research question in this study was: ’Does the development method Scrum provide a better environment for minimizing the MUM effect’. The answer to this question based on the results of this study would be no. There was a small difference found in willingness to report in those two settings, where the participants were more willing to report bad news in the Scrum Setting, however this was not signif-icant. The results also suggest that another study, or more sampling is required before any further conclusions should be drawn.

In this study we also wanted to discover whether the four as-pects influenced the MUM effect individually in each devel-opment environment. The results indicated that changing the following four aspects individually positively influences the willingness to report bad news in a Waterfall environment: more project flexibility, more frequent and informal commu-nication, more review moments throughout, a higher team spirit and shared work responsibility. For the Scrum devel-opment environment we found that taking the same factors away, would not influence the MUM effect.

ACKNOWLEDGEMENTS

I would like to thank Loek Stolwijk, for being my supervisor once again. His support, guidance and feedback was as valu-able as always. Further more, I thank my second assessor Arjan Vreeken for his time and interest in this thesis project. I also really appreciate everyone who participated in the ex-periment of this study.

REFERENCES

1. Abrahamsson, P., Salo, O., Ronkainen, J., Warsta, J., et al. Agile software development methods: Review and analysis, 2002.

2. Allen, I. E., and Seaman, C. A. Likert scales and data analyses. Quality progress 40, 7 (2007), 64.

3. Balaji, S., and Murugaiyan, M. S. Waterfall vs. v-model vs. agile: A comparative study on sdlc. International Journal of Information Technology and Business Management 2, 1 (2012), 26–30.

4. Beck, K., Beedle, M., Van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., et al. Manifesto for agile software development.

5. Bendelow, G. Using visual imagery to explore gendered notions of pain. SAGE FOCUS EDITIONS 152 (1993), 212–212.

6. Clancy, T. The standish group report. 1995.

7. Clason, D. L., and Dormody, T. J. Analyzing data measured by individual likert-type items. Journal of agricultural education 35 (1994), 4.

8. Cocco, L., Mannaro, K., Concas, G., and Marchesi, M. Simulating kanban and scrum vs. waterfall with system dynamics. In International Conference on Agile Software Development, Springer (2011), 117–131. 9. Cohen, J. Statistical power analysis. Current directions in

psychological science 1, 3 (1992), 98–101.

10. Corporation, N. I. Lifecycle models, 2006. http://zone.ni.com Accessed: 2017-05-28.

11. Costa, R. Who, where, what, when, why and how to use scrum - a simple guide, 2016. https://www.linkedin.com/pulse/

who-where-what-when-why-how-use-scrum-simple-guide-rui-costa

Accessed: 2017-07-02.

12. DeSanctis, G. Small group research in information systems: Theory and method. In The Information Systems Research Challenge: Experiment Research Methods., Cambridge, MA: Harvard Bus. (1989), 53–82.

13. Dozier, J. B., and Miceli, M. P. Potential predictors of whistle-blowing: A prosocial behavior perspective. Academy of Management Review 10, 4 (1985), 823–836.

14. Earley, P. C. East meets west meets mideast: Further explorations of collectivistic and individualistic work groups. Academy of management journal 36, 2 (1993), 319–348.

15. Google Forms. http://forms.google.com/.

16. Google Spreadsheets. http://spreadsheets.google.com/. 17. Hughes, D. L., Dwivedi, Y. K., Simintiras, A. C., and Rana, N. P.

Success and failure of IS/IT projects: A state of the art analysis and future directions. Springer, 2015.

18. Iacovou, C., and Dexter, A. Explanations offered by is managers to rationalize project failures. In Proc. Academy of Management Conf (1996), 11–14.

19. Jubb, P. B. Whistleblowing: A restrictive definition and interpretation. Journal of Business Ethics 21, 1 (1999), 77–94.

20. Karlm. Software lifecycle models. KTH, 2006.

21. Keil, M., and Robey, D. Turning around troubled software projects: An exploratory study of the deescalation of commitment to failing courses of action. Journal of Management Information Systems 15, 4 (1999), 63–87.

22. Keil, M., Smith, H. J., Pawlowski, S., and Jin, L. ’why didn’t somebody tell me?’: climate, information asymmetry, and bad news about troubled projects. ACM SIGMIS Database 35, 2 (2004), 65–84. 23. Miceli, M. P., and Near, J. P. Blowing the whistle: The organizational

and legal implications for companies and employees. Lexington Books, 1992.

24. Munassar, N. M. A., and Govardhan, A. A comparison between five models of software engineering. IJCSI 5 (2010), 95–101.

25. O’Neal, E., Levine, D., and Frank, J. Reluctance to transmit bad news when the recipient is unknown: Experiments in five nations. Social Behavior and Personality: an international journal 7, 1 (1979), 39–47. 26. Park, C., Im, G., and Keil, M. Overcoming the mum effect in it project reporting: Impacts of fault responsibility and time urgency. Journal of the Association for Information Systems 9, 7 (2008), 409.

27. Pool, L. M. Model voor planning en budgetcontrole bij de ontwikkeling van informatiesystemen.

28. Robey, D., and Keil, M. Blowing the whistle on troubled software projects. Communications of the ACM 44, 4 (2001), 87–93. 29. Robson, C., and McCartan, K. Real world research. John Wiley &

Sons, 2016.

30. Rosen, S., and Tesser, A. On reluctance to communicate undesirable information: The mum effect. Sociometry (1970), 253–263. 31. Schwaber, K. Scrum development process. In Business Object Design

and Implementation. Springer, 1997, 117–134.

32. Smith, H. J., and Keil, M. The reluctance to report bad news on troubled software projects: a theoretical model. Information Systems Journal 13, 1 (2003), 69–95.

33. Smith, H. J., and Mark Keil, G. D. Keeping mum as the project goes under: Toward an explanatory model. Journal of Management Information Systems 18, 2 (2001), 189–227.

34. Snoek, M. The use and methodology of scenario making. European Journal of Teacher Education 26, 1 (2003), 9–19.

35. IBM Corp. Released 2016. IBM SPSS Statistics for Macintosh, Version 24.0. http:

//www.ibm.com/analytics/us/en/technology/spss/.

36. Statistics, L. Independent-samples t-test using Stata. Statistical tutorials and software guides, 2016. https:

//statistics.laerd.com/Accessed:2017-07-02.

37. Tan, B. C., Smith, H. J., Keil, M., and Montealegre, R. Reporting bad news about software projects: Impact of organizational climate and information asymmetry in an individualistic and a collectivistic culture. IEEE Transactions on Engineering Management 50, 1 (2003), 64–77.