The influence of uncertainty on public procurement

(1)

Public Procurement

THE INFLUENCE OF UNCERTAINTY ON PUBLIC

PRO-CUREMENT

THESIS

Supervisor:

dr. A. Zerres

Second reviewer:

dr. K. Venetis

University of Amsterdam

Executive Programme in Management Studies – Marketing Track

by

Tamara van Vastenhoven

Student Number: 10317376

(2)

1

Preface

This thesis is written as part of the Executive Programme in Management Studies of the Uni-versity of Amsterdam.

I would like to thank Dr. A. Zerres for all of his help during the preparation of this thesis. He introduced me to the world of negotiation. He has definitely given me a different perspective on the public procurement process, in which the role of the evaluation committee has been ignored for too long. I also want to thank everyone who has contributed to the final version of this thesis. The authorities who provided the data, the people who participated in the pre-test of the experiment, the people who reviewed my final concept and finally my family, friends and colleagues who gave me the optimal support. A special thanks to my partner Gert-Jan Bonten. He created the conditions, allowing me to actually finish the thesis. I will definitely will have to get used to cooking a meal again.

(3)

2

Abstract

To improve objectivity, a commonly adopted procedure in the public procurement process is to judge price and non-price criteria separately. This means that members of an evaluation committee judge the performance on the non-price criteria, without knowing the price scores. This study assumes that the judgement of the non-price criteria is being influenced by condi-tions of uncertainty caused by not knowing the price scores. In doing so , the study compares the non-price criteria scores at different degree of uncertainty by research into existing data. Results of the study, with a sample of 145 completed European tenders, reveal no significant causal effect of uncertainty on the non-price score. In contrast with the hypotheses, the research found no evidence that the importance weight of the price criteria has a positive or negative effect on the difference between the highest achieved non-price score and the runner up. To ensure that all potential variation on the differences in price score is only caused by the variation of the importance weight of the price criteria, an experiment is designed. Another objective of the experimental design is to test the causal effect of the degree of uncertainty on the biases in the non-price score. Given the restricted time available for this research project, its focus lies on developing a new negotiation simulation paradigm with which to research public procure-ment processes. A pre-test of the simulation was run to adjust the materials, but the actual car-rying out of the experiment was left for follow-up research.

(5)

4

1. Introduction

It is difficult to imagine the public sector without outsourcing. Although a public authority may sometimes choose to insource, today more and more services, supplies and works are pro-cured from commercial firms (Harland, Telgen, Knight, Callender & Thai, 2007; Matthews, 2005; Waara, 2008). Such purchases are generally referred to as public procurement. In most countries public procurement accounts for 10% to 15% of the GDP (World Trade Organization, n.d.) and it therefore constitutes a significant market and is an important aspect of international trade. Moreover, because the public sector represents an extensive group of state authorities (i.e. national and local governments, together with their agencies and chartered bodies) and procurement spending is up to 50% of total government spending (uncitral, n.d.), the public sector has been recognized as one of the largest purchasers of goods and services in a country (Walker & Brammer, 2009). Many authorities also make use of public procurement in order to achieve policy objectives ((Harland, Telgen, Knight, Callender & Thai, 2007), such as stimu-lating competitive markets (Caldwell et al, 2005), promoting innovation (Edler & Georghiou, 2007) and achieving sustainable social aims and objectives (McCrudden, 2004; Preuss, 2009). Since public authorities become increasing dependent on their suppliers, the impact of se-lecting the ‘wrong’ supplier due to bad decision-making is becoming more evident (De Boer, Labor & Morlacchi, 2001). Products and services are frequently purchased for reasons of public policy in order to allow the authority to fulfil its function. Supplier selection has therefore been recognized as one of the most important activities in achieving an effective supply chain (Boran, Genc, Kurt & Akay, 2009; Cakravastia & Takahshi, 2004; Chou & Chang, 2008). An incorrect decision in the supplier selection process may cause serious problems for the authority and prevent it from adequately fulfilling its function (Piramuthu, 2005).

(6)

5

In order to ensure that the most suitable partner is found, there is a growing trend to award the contract to the bidder who provides the best value for money (Ohno & Harada, 2006; Berg-man & Lundberg, 2013). This means that it is not necessary to automatically award the contract the bidder offering the lowest price. The public authority normally uses a combination of price and non-price criteria to decide which supplier should be awarded.

Although adding both price and non-price criteria to the tender evaluation process improves efficiency, it also adds complexity to the procedure (Bergman & Lundberg, 2011; Lorentziadis, 2010). Non-price criteria are usually evaluated by a multi-headed evaluation committee con-sisting of different experts selected by the public authority (Bana e Costa, Corrêa, De Corte & Vansnick, 2002; Csáki & Adam, 2010; Tsai, Wang & Lin, 2007). As a result, the tender evalu-ation process is clearly a group process and decision making and negotievalu-ation among various (internal) stakeholders are important parts of this process (Csáki & Adam, 2010; Tsai, Wang & Lin, 2007).

It is almost certain that the people making the decision also affect the decision made. How-ever, since authorities today have become more dependent on their suppliers, it is in the author-ities’ best interests to ensure that the evaluation is as objective as possible and is not influenced by personal or group subjectivity. The contract should be awarded to the bidder who best meets the authorities’ specific objectives, rather than to the bidder who scored best in personal inter-ests. To avoid any subjective evaluations, the evaluation committee has to follow prescribed procedures and maintain transparency in the evaluation process (Falagario, Sciancalepore, Cos-tantino & Pietroforte, 2012).One commonly adopted procedure to stimulate objectivity and therefore also to improve the quality of the decision is to judge price and non-price criteria separately (Lengwiler & Wilfsetter, 2006). This means that members of the evaluation commit-tee judge performance on the non-price criteria of the several bidders, without knowing their price scores. This should protect members from being influenced by the financial proposals.

(7)

6

However, because the score of the price criteria is unknown to the evaluation committee, they do not know for sure how their evaluation of the non-price criteria will affect the final result. This means that, in the end, it is not the supplier who scored the best on the non-price criteria who will be awarded the contract, but rather the one with the highest overall score that combines both price and non-price criteria. The existing literature indicates that not knowing specific information that is important for the decision makes choosing between the options more complicated. Decision-makers who do not know the precise consequences of their decision try to change this to a more controllable situation (Huber, 2007). For example, according to the prospect theory (Kahneman & Tversky, 1979), decision-makers are very sensitive to losses and use different strategies in different uncertain situations in order to avoid them. Another well-known theory is the so-called ‘Ellsberg effect’ (Ellsberg, 1961), which suggest that decision-makers tend to select options for which the probability of a favorable outcome is known, over options for which the probability of a favorable outcome is unknown, despite the latter option possibly having a higher probability for gain.

These theories show that people do not act completely rationally when making a decision under conditions of uncertainty. Of course, this could possibly threaten the common practice of evaluation in the public procurement process. The entire standardized procurement process is arranged in order to promote objectivity, fairness and integrity in procurement (uncitral, n.d.). This study therefore focuses on the question how the evaluation committee’s judgment is influ-enced by conditions of uncertainty caused by not knowing the price scores. This is an area that, as far as the researcher is aware, has been ignored in public procurement research so far. To answer this question, the current study compares the evaluation of the non-price criteria on different levels of uncertainty by analyzing existing data and using a between subject experi-ment design.

(8)

7

It is interesting to investigate the effect of conditions of uncertainty on the judgment of an evaluation committee for both scientific and applied reasons. This study contributes to public procurement knowledge and practices. Public procurement practitioners face similar challenges in different countries (Harland, Telgen, Knight, Callender, & Thai, 2007) and providing a clear understanding of the effect of certain choices will help them to make more conscious choices and protect them from failing in the selection of suppliers. The entire public procurement pro-cess is organized with the aim of avoiding subjectivity (Falagario, Sciancalepore, Constantino & Pietroforte, 2011). Any possible threat to objectivity should therefore be recognized in order to optimize the public procurement process. Furthermore, this study contributes to the literature on public procurement. Most studies in this field initially focused on the similarity between private and public procurement (Harland, Telgen, Knight, Callender, & Thai, 2007). Research-ers assumed that results from private procurement research could be generalized to public pro-curement. It is now widely accepted that these disciplines are fundamental different (Arlbjørn & Freytag, 2012; Roodhooft & Van den Abbeele, 2006; Thai, 2001). However, the scientific analysis and knowledge of public procurement is lagging behind and more research is needed (Harland, Telgen, Knight, Callender, & Thai, 2007). In addition, there are fewer studies that focus on multi-person decision-making, i.e. group decision–making, under (Gong, Baron & Kunreuther, 2009). Moreover, when they do, these studies focus on group decision-making and uncertain linguistic environments (for example, Chen, Zhou & Han, 2011; Wei, 2009; Xu, 2004) or social uncertainty, i.e. uncertainty about other members’ choices (for example, Chen, Au & Komorita, 1996).

The following section introduces the theoretical background of the different research topics. The European tender process is discussed at greater length. This is followed by a continued discussion of the theoretical background of decision-making under uncertainty and group deci-sion-making. This chapter also include the hypotheses. In the fourth chapter the methodology ,

(9)

8

results and discussion of the research into existing data are described. The fifth chapter consist-ing information about the experimental design. Finally, the last chapter include the conclusion and the general discussion.

(10)

9

2. European tender process

European regulations require public authorities to publish a tender in a public environment for public contracts above a certain monetary amount, and to give all potential suppliers from the European Union the opportunity to submit a proposal (Eur-Lex, 2014). The procurement of supplies, services and works should follow mandatory procedures and directives that are de-fined at the European level. The awarding of contracts is based on objective criteria that corre-spond to the ground principles of European procurement, including transparency, non-discrim-ination and equal treatment (Eur-Lex, 2014). A public authority is therefore only allowed to use two different award criteria, namely, the lowest price or the most economically advantageous tender (MEAT).

For a lowest price procedure, only price criteria are evaluated and the bidder with the lowest price will be awarded the contract. By contrast, the MEAT procedure also takes other criteria into account. Important non-price criteria include factors such as quality, delivery time and environmental characteristics, and are evaluated together with the price. This study focuses only on tenders awarded according to the MEAT procedure.

The goal for each tender process is to award the contract to the bidder with the best solution for the authority (Mateus, Ferreira & Carreira, 2010). Including non-price criteria in the tender evaluation complicates the process (Lorentziadis, 2010). How does an authority evaluate multi-criteria proposals? Many studies in the literature focus on bidder ranking and combining both price and non-price criteria, with the aim of awarding the contract to the best suitable partner (De Boer, Labro & Morlacchi, 2001; Ho, Xu and Dey, 2010). Distinctions are commonly made between the absolute and the relative scoring methods (Bergman & Lundberg, 2013; Mateus, Ferreira & Carreira, 2010) and between compensatory and non-compensatory methods (De

(11)

10

Boer, van der Wegen & Telgen, 1998). Telgen and Schotanus (2010) discuss the various ad-vantages and disadad-vantages of several methods. Because the scope of this study is limited to the evaluation of the non-price criteria, the different scoring methods are not discussed further. However, because most tenders are evaluated by the compensatory model (De Boer, Labrò & Morlacchi, 2001; Ho, Xu & Dey, 2010), only those tenders in which a low score for one criteria can be compensated with a high score for another criteria are included in this study.

In order to identify which proposal provides the best value for money, the public authority specifies a relative importance weight between the several criteria. In a Swedish study by Waara and Bröchner (2006), a typical pattern of 70% price weight in combination with 30% for the non-price criteria was found. This means that the overall score for the proposals is calculated as follows: (Score Price Criteria * 0.7) + (Score Non-Price Criteria * 0.3) (Lengwiler & Wilfset-ter, 2006). The bidder with the highest overall score for its proposal will be awarded the con-tract. Thus, according to the MEAT procedure, the relative importance weight of the non-price can vary from 0% to 100%, but there should be a total of 100% when combined with the price criteria. In order to decide which criteria are important and what the relative importance weight between them should be, the public authority usually forms a project team (Waara & Bröchner, 2006) consisting of experts or other key players such as a purchaser (Bana e Costa, Corrêa, De Corte & Vansnick, 2002; Csáki & Adam, 2010; Tsai, Wang & Lin, 2007). The public authority is obliged to publish these criteria and their relative weight in its proposal request (Eur-Lex, 2014) and they are therefore generally known.

Price criteria are objectively assessed (Mateus, Ferreira & Carreira, 2010). For example, the bidder with the lowest price receives the maximum score. The scores of the other bidders are related to the lowest price. For example, a 20 percent higher price represents a 20 percent lower score. However, for most of the non-price criteria, it is impossible to define an objective prede-fined performance scale. Consider, for example, the evaluation of an implementation plan or an

(12)

11

architect’s design. It is impossible to evaluate criteria for these without knowing the content of the plan or the design. Therefore non-price criteria are evaluated by a multi-headed evaluation committee that usually comprises approximately five experts (Bana e Costa, Corrêa, De Corte & Vansnick, 2002; Csáki & Adam, 2010; Tsai, Wang & Lin, 2007). The judgment is done by assigning scores to the non-price criteria according a predefined performance scale (Mateus, Ferreira & Carreira, 2010). In practice, commonly used scale rangings are scales from 1 to 10 or 1 to 5 or else Excellent – Sufficient – Insufficient.

Members of the evaluation committee usual occupy various positions in different depart-ments and at various organizational levels. They have to provide a joint evaluation and this means that the non-price criteria are evaluated from different perspectives and by people with differing expertise and, consequently, different points of view. Tsai et al. (2007) discuss three representative methods for integrating the evaluation of the different individuals into a joint evaluation. First, in the Borda count method, the joint evaluation is calculated according to the sum of the individual evaluations (Goddard, 1983). Second, the consensus ranking method looks for the joint ranking that is nearest in average to the individual evaluations. (Cook & Seiford, 1978). Third, the multidimensional scaling analysis method (Kong, 2005 cited in Tsai, Wang & Lin, 2007) draws a perceptual map of the individual evaluations of all bidders and subsequently defines the ‘ideal fictitious bidder’. The bidder nearest to this ‘ideal fictitious bid-der’ gains first ranking and so on. However, all these methods ignore the possibility of major differences between the individual evaluations. Extreme evaluations by a few members will affect the joint evaluation and will result in a biased score of the non-price criteria. In extreme cases, this could mean that the ‘wrong’ supplier is selected and, as a result, the public authority does not award the contract to the most economically advantageous tender.

(13)

12

Therefore, with most tenders, the joint evaluation of the non-price criteria is achieved by the members of an evaluation committee reaching consensus on the score (Bana e Costa, Cor-rêa, De Corte & Vansnick, 2002; Tsai, Wang & Lin, 2007). Consensus was traditionally defined as the complete and unanimous agreement of all the individuals’ opinions (Herrera, Herrera-Viedema & Verdegay, 1996; Kacprzyk, & Fedrizzi, 1988), i.e. maximum consensus. Because this kind of unanimity is idealistic and difficult to achieve, a more common consensus is the maximum possible consensus (Chiclana, Mata, Martinez, Herrera-Viedema & Alonso, 2008; Herrera, Herrera-Viedema & Verdegay, 1996; Herrera-Viedema, Herrera & Chiclana, 2002; Kacprzyk, & Fedrizzi, 1988), which refers to the highest possible consensus of individuals. Both these systems are used in the public procurement process. Examples of the definition of consensus for public tenders are: “The evaluation committee has to reach consensus about the scores of the non-price criteria, which means that individual evaluations should not have a greater distance than two points”1 and “The evaluation committee has to reach a maximum consensus about the scores of the non-price criteria, which means that individual evaluations cannot differ. All members have to agree with the score given.”2

The entire standardized procurement process is arranged to promote objectivity, fairness and integrity in procurement and to minimize personal subjectivity (uncitral, n.d.), which is seen as one of the advantages of using group decision-making through consensus. Another commonly adopted procedure to stimulate objectivity, is judging price and non-price criteria separately (Lengwiler & Wilfsetter, 2006). The price criteria are scored by the purchaser and members of the evaluation committee are not informed about the price scores. This means that they do not know for sure how their evaluation will affect the final result and carry out their evaluation under conditions of uncertainty.

1_{As described in a catering tender of the municipality of Haarlemmermeer.}

(14)

13

3. Literature Review and Hypotheses

3.1. Decision-Making under Uncertainty

Research on behavior and decision-making under conditions of uncertainty dates back to the eighteenth century and the theory of Bernoulli (Lipshitz, Klein, Orasanu & Salas, 2001). A more recent theory is that of the expected utility of wealth of Von Neuman and Morgenstern (1944, as cited in Lipshitz, Klein, Orasanu, & Salas, 2001). The basic principle of this theory is that individuals try to maximize the expected utility of their choice between risky, uncertain options. They try to find a balance between the usefulness of a particular option and the proba-bility that this option will occur. Subsequently, the option is chosen where this balance is the most optimal. This means that the decision maker first calculates the expected usefulness of the outcomes associated with the various alternatives and then he chooses the alternative that max-imizes the expected utility. However, the theory of the expected utility of wealth has been crit-icized by several scholars (Rabin, 2000). Allais (1979, as cited in Epstein, 1999) questioned whether people are actually using the linear probability weights when they make a decision under conditions of uncertainty. Rabin (2000) proved mathematically that the theory of ex-pected utility of wealth cannot explain the dislike for losses. But the most “well-known” criti-cism is probably provided by Kahneman and Tversky (1979). They argued that the theory ig-nores environmental effects and responded to it by presenting a new theory, the prospect theory. It shows, among other things, that individuals make different decisions, even though they are exposed to exactly the same choices. It all depends on how the choices are presented. The pro-spect theory is based on the fact that people think in terms of gains and losses in relation to reference points rather than final outcomes.

Both the expected utility of wealth of Von Neuman and Morgenstern and the prospect the-ory, assume that decision-makers know the probabilities of the outcome (Camerer & Weber, 1992; Gollier, Hammitt & Treich, 2013). However, sometimes the probabilities are unknown

(15)

14

to decision-makers because of a lack of information. This type of uncertainty is better known as ambiguity (Camerer & Weber 1992; Yates & Zukowski, 1976). Daniel Ellsberg (1961) was the first to examined this type of uncertainty and his work is still used in much of today’s re-search. The so-called “Ellsberg effect” implies that decision-makers tend to select options for which the probability of a favorable outcome is known, over the option for which the probabil-ity of a favorable outcome is unknown.

Uncertainty may arise in different situations, and this is confirmed by the various definitions that researchers uses to explain the term. For example, Anderson (1981, as cited in Lipshitz & Strauss, 1997) describes uncertainty as “A situation in which one has no knowledge about, or knows only the probability of, which of several states of nature has occurred or will occur”. By contrast, Humphreys and Berkeley (1985, as cited in Lipshitz & Strauss, 1997) define uncer-tainty as: “The inability to assert with ceruncer-tainty one or more of the following: (a) act-event sequences; (b) event-event sequences; (c) value of consequences; (d) appropriate decision pro-cess; (e) future preferences and actions; (f) one’s ability to affect future events.” Lipshitz and Strauss (1997) introduced three aspects of uncertainty: “Inadequate understanding, incomplete information and undifferentiated alternatives”. This study understands the uncertainty compo-nent to include “the ability to affect future events” created by “incomplete information”.

Because the scores of the price criterion are unknown to the evaluation committee (i.e. they have incomplete information), its members do not know for sure how their evaluation will affect the final result (i.e. their ability to affect future events). In the end, it is not the supplier with the best score on the non-price criteria who will be awarded the contract, but the one who has the highest overall score that combines price and non-price criteria. This is because, according to the compensatory model (De Boer, Labro & Morlacchi, 2001; Ho, Xu & Dey, 2010), a low score for one criteria can be compensated with a high score for another criteria. This means that even if a bidder scored very low on their performance according to the non-price criteria, they

(16)

15

could still be awarded the contract because their proposal was low in terms of costs. From the point of view of the evaluation committee, the bidder that they considered to be the best suitable partner to do the job, may not be awarded the job because their costs are too high. In order to decrease this risk, one can expect the evaluation committee to protect itself against this un-wanted outcome (Gollier, Hammitt & Treich, 2013; Meyer & Meyer, 2011), and to try to change the uncertain situation to a more controllable one (Huber, 2007), by controlling the variable is can affect (Hubert, Wider, Huber, 1997). In this case the score of the non-price cri-teria. How much the price score has to be compensated for depends, among other things, on the relative importance weight between price and non-price criteria. The higher the importance weight of the non-price criteria, the more the evaluation committee can affect the overall out-come and vice versa.

This can be clarified with an example. As already mentioned, the overall score for the pro-posals is calculated as follows: (Score price criteria * Importance weight price criteria) + (Score non-price criteria * Importance weight non-price criteria) (Lengwiler & Wilfsetter, 2006). Since, the price/non-price ratio of services, products and works can differ, the importance weight of the price and the non-price criteria between tenders can differ. Consequently, the degree of uncertainty and the influence of the evaluation committee on the overall outcome can also differ. For example, Waara and Bröchner (2006) found a typical pattern of 70% price weight in combination with 30% non-price criteria. Because the price score is unknown to the evaluation committee, 70% of the overall score is uncertain. The weighted scores on price range from 0 to 70 points and are uncontrollable. However, the remaining 30% of the overall outcome, i.e. the importance weight of the non-price criteria, can be affected by the judgment of the evaluation committee. Of course, it could also be the other way around, with the price criteria

(17)

16

by the committee. In summary, the higher the importance weight of the price criteria, the more uncertainty appears and the less influence an evaluation committee has on the overall outcome. By contrast, the lower the importance weight of the price criteria, the less uncertainty appears and the more influence an evaluation committee has on the overall outcome (See Figure 1). The question, however, is how the degree of uncertainty affect the judgment of the evaluation com-mittee.

The potential positive effect of the importance weight of the price criteria on differences in the non-price score

Lipshitz and Straus (1997) studied how decision-makers deal with uncertainty and discov-ered that they use different strategies to cope with different kinds of uncertainty. Incomplete information was primarily managed by making assumptions based on the information they ac-tually had. Decision-makers make predictions about what might happen based on known infor-mation and they act accordingly. By following this argumentation, the evaluation committee use the relative importance weight between price and non-price criteria to predict the overall outcome. They know that the higher the importance weight of the price criteria, the less influ-ence they have and the greater the probability that the bidder they prefer will not be awarded

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 % P rice Cri teria & % Un ce rta in ty o f Ov era ll Score

% Non-Price criteria & % Influence on overall Score Figure 1. Uncertainty and Influence Evaluation Committee

(18)

17

the contract. Therefore, the higher the importance weight of the price criteria is, the more one can expect the evaluation committee to increase the difference of the non-price criteria score between the score of their favorite bidder and that of the runner up in order to protect themselves against this risk. Expressed formally this reads:

H1A The higher the importance weight of the price criteria, the greater the difference of the non-price criteria score between the highest score achieved and that of the runner up.

The potential negative effect of the importance weight of the price criteria on differences in the non-price score

The above prediction assumes that decision-makers act rationally under uncertainty. But it is also recognized that emotions have a significant impact on decision-making with conditions of uncertainty (Mellers, Schwartz & Ritov, 1999). People are not only influenced by probability and utility, but are also affected by psychological factors, such as ambiguity and familiarity (Linde & Sonnemans, 2012). They prefer options in which probabilities are known (Ellsberg, 1961) and they try to avoid feelings of pain that are caused by the outcome not meeting their expectations (Mellers, Schwartz & Ritov, 1999). The lower the importance weight of the price criteria, the more influence they have and the greater the probability is that the bidder they prefer will be awarded the contract and will represent their choice. Therefore, following this reasoning, one can expect that the lower the importance weight of the price criteria, the greater the evaluation committee’s tendency will be to increase the differences in the non-price criteria score between the score of their favorite bidder and that of the runner up. This leads to a hy-pothesis that is offered as an alternative to H1A:

(19)

18

H1B The lower the importance weight of the price criteria, the greater the difference of the non-price criteria score between the highest score achieved and that of the runner up.

This study offers H1A and H1B as co-equal, alternative hypotheses because arguments for both positive and negative effects can be found in the literature. The importance weight of the price criteria may have a positive effect on the differences of the non-price score between the highest score achieved and that of the runner up, because the evaluation committee want to increase the likelihood that their favorable bidder will be awarded the contract (H1A). In con-trast, the importance weight of the price criteria may have a negative effect on the differences of the non-price score between the highest score achieved and that of the runner up because the evaluation committee do not believe that they can influence the overall outcome if the im-portance weight of the price criteria is high. And, as a result, they will not invest in the non-price score (H1B).

3.2. Group Decision-Making and Uncertainty

Decision-making can generally be seen as finding the best solution to a problem among several options (Herrera, Herrera-Viedma, & Verdegay, 1996). Because of the involvement of multiple individuals, the evaluation of the non-price criteria involves a group process and should therefore be viewed in terms of group-decision making (Herrera, Herrera-Viedma, & Verdegay, 1996). Since there are studies that argue that the decision-making environment is more complex for group decisions than for dyads and that the decision-making rules differ be-tween these contexts (Beersma & De Dreu, 2002; Mannix, Thompson & Bazerman, 1989), this study defines group decision-making as an interaction between three or more interdependent

(20)

19

individuals (Ancona, Friedman & Kolb, 1991; Schei & Rognes, 2005). The fact is that an eval-uation committee usually consists of approximately five experts (Bana e Costa, Corrêa, De Corte & Vansnick, 2002; Csáki & Adam, 2010; Tsai, Wang & Lin, 2007).

Since the supplier selection process usually involves multiple persons, with complex inter-actions among both people and their individual and organizational goals, supplier selection has been recognized as a complex decision-making process (Webster & Wind, 1972). Although every member of the evaluation committee recognizes the same problem and intends to solve it (Herrera, Herrera-Viedema & Verdegay, 1997), its members may differ in their interests, preferences or opinions on the problem (Beersma & De Dreu, 2002). Members usually occupy various occupational positions, have different areas of expertise and even work for different departments on different organizational levels. Therefore the members of the committee may approach the problem from different perspectives.

This can be clarified by an example. A public authority wants to contract a supplier for the construction of a new highway bridge. In order to judge the non-price criteria in this tender, an evaluation committee has been formed consisting of a director, a project manager and a senior construction specialist. Each member has their own responsibility in the project. The director has the final responsibility, the project manager is responsible for the construction phase and the construction specialist is responsible for the maintenance of the bridge over the next 10 years. Because of this diversity, the bidders are evaluated from different points of view, but this also makes the evaluation more complex because each member has their own area of expertise and responsibility. The project manager, for example, is more interested in the bidder’s plan-ning than in the construction phase. By contrast, the senior construction specialist is more in-terested in the phase after the construction, namely, the kind of maintenance that the bridge will require over the next years. However, even though the group members have different interests, they eventually have to reach an overall evaluation.

(21)

20

In the group decision-making literature, the most common distinction that is made in order to come to an overall decision is the distinction between the unanimity rule and the majority rule (Schei & Rognes, 2005; Miller, Jackson, Mueller & Schersching, 1987; Beersma & De Dreu, 2002). According to the majority rule, an agreement among the majority of the group’s members is sufficient. According to the unanimity rule, in contrast, all group members have to agree with the decision that is eventually made and each individual is able to block this decision. In this study, the overall group decision is following the unanimity decision rule. Since, the evaluation committee’s judgment is based on consensus, all its members eventually have to agree with the overall non-price criteria score and every member have the power to block the decision made.

Given that the committee members’ opinions, preferences and interests differ widely, ne-gotiation is often part of the group decision-making process (Beersma & De Dreu, 1999). Thompson (2009) describes negotiation as interpersonal decision-making. Group members have to communicate about their different interests and opinions (Schei & Rognes, 2005), ex-plain and reevaluate their own preferences (Stassar & Titus, 1985) and probably eventually make some compromises (Beersma & De Dreu, 1999).

Because group members are confronted by other members’ interests, they feel an inner con-flict between their own interests and the joint interests of the group (Beersma & De Dreu, 1999); this is known as a mixed motive conflict (Komorita & Parks, 1995; Van Lange, Joireman, Park & Van Dijk, 2013). Group members seek good outcomes for themselves and have conflicting interests, which lead to competition between members. However, in order to come to an overall evaluation, group members also have to cooperate. They have to seek ways to maximize joint outcomes and actually reach a consensual agreement. (Deutsch & Krauss, 1960; Komorita & Parks, 1995; Mannix, Thompson & Bazerman, 1989). The degree of integration in the agree-ment represents the quality of the group-decision eventually made (Schei & Rognes, 2005). In

(22)

21

this study, an integrative agreement is seen as being in the authorities’ interests because it in-cludes the opinions and preferences of all group members. As a result, the judgment made draws on the various areas of expertise represented in the group.

In a negotiation process, people are confronted with uncertainty. For example, there is social uncertainty that refer to uncertainty regarding other members’ choices (Chen, Au & Komorita, 1996). In this study, the groups are also confronted with another type of uncertainty. The eval-uation committee do not know for sure how their judgment of the non-price criteria will affect the overall outcome because they miss crucial information, namely, the scores of the price cri-teria. The existing literature shows that negotiating in a context of uncertainty affects the like-lihood of cooperation (Chen, Au & Komorita, 1996; Gong, Baron & Kunreuther, 2009; Van Lange, Joireman, Parks & Van Dijk, 2013) and that this may threaten the quality of the decision. This is because a more integrated decision, as opposed to a decision that is biased by personal interest, increases the likelihood of awarding the contract to the bidder who provides the best value for money.

The potential positive effect of the importance weight of the price criteria on the degree of integration in the non-price score

The group members of the evaluation committee have to reach a joint agreement on the evaluation of the non-price criteria. Through a negotiation process, the group members have to achieve a balance between cooperation in order to reach an agreement and competition in order to fulfill their personal interests. Negotiation in the presence of uncertainty affects the likeli-hood of cooperation. Previous research has indicated that when environmental uncertainty in-creases, people become less cooperative because they are not sure how their judgment will affect the final outcome (Chen, Au, Komorita, 1996; Saywer, 1990). Therefore, this study ex-pects that the higher the importance weight of the price criteria, the less group members will

(23)

22

cooperate, causing the non-price score to be affected by personal interests. Consequently, the hypothesis of this study is that:

H2A The higher the importance weight of the price criteria, the more the non-price score is affected by the biases of individual interests.

The potential negative effect of the importance weight of the price criteria on the degree of integration in the non-price score

However, when other group members have a greater control over an outcome, people can feel that their personal interests are being threatened (Van Lange, Joireman, Park & Van Dijk, 2013). They become less cooperative because they have no faith in the honesty of the other group members (Evans & Krueger, 2011). This leads to a hypothesis that is offered as an alter-native to H2A:

H2B The lower the importance weight of the price criteria, the more the non-price score is affected by the biases of individual interests.

Like H1A and H1B, this study offers H2A and H2B also as co-equal, alternative hypotheses because arguments for both positive and negative effects can be found in the literature. The importance weight of the price criteria may have a positive effect on the biases in the non-price score, because in the presence of uncertainty group members will be less cooperative (H2A). However, the importance weight of the price criteria may have a negative effect on the biases in the non-price score, because the more influence members have, the more untrusting other members become and the more they feel that they have to defend their own interests (H2B).

(24)

23 3.3. Research Design

To test the above hypotheses, a multimethod quantitative study was conducted (Saunders, Lewis & Thornhill, 2009, p. 152). This study combined the analysis of existing data and an experiment. The purpose of the research on existing data was to study the administrative data generated in normal business conditions that are primary oriented to the causal relationship between the importance weight of the price criteria and the differences in the non-price criteria score between the highest score achieved and that of the runner up (Hypotheses 1A and 1B). Data stemming from business practice improves the external validity of the results (Bryman, 2012, p. 54). However, the experiment was specifically designed for this study. Its first purpose was to test whether the differences in the non-price criteria score between the highest score achieved and that of the runner up are affected by the importance weight of the price criteria (Hypotheses 1A and 1B) in a controlled environment (Saunders, Lewis & Thornhill, 2009, p. 591). The use of different data collection techniques within one study ensures that data indicate

what one thinks they indicate (Saunders, Lewis & Thornhill, 2009, p. 591). However, no psy-chological construct can be measured using existing data and variables to include information about the outcome rather than about the process that leads to the outcome (Shadish, Cook & Campbell, 2002, p. 203). Therefore, the second purpose of the experiment was to test whether different degrees of uncertainty affect the biases in the non-price score (Hypotheses 2A and 2B). Given the limited time available for this research project, the focus was on developing a new negotiation simulation paradigm with which to research the public procurement process. A pre-test of the simulation was run in order to adjust the materials, but the actual conduction of the experiment was left for follow-up research.

(25)

24

4. Research into Existing Data

4.4. Methods

4.1.5 Research Design and Sample

To test hypotheses 1A and 1B, existing data was gathered from five different contracting authorities in the Netherlands (three municipalities, one university and one Dutch Ministry). This makes it possible to analyze data that stems directly from doing business (Baarda & De Goede, 2006, p. 203). In keeping with the scope of this study, only European tenders with the following characteristics were included. 1) The tender has followed the open or the restricted procedure according to European directives 2004/14/EG (Eur-Lex, 2014). All the other proce-dures were excluded because the prices were known by the evaluation committee. 2) The award methodology followed the MEAT Procedure (Eur-Lex, 2014), and tender evaluation therefore included both price and non-price criteria. 3) A low score on one criteria can compensate for a high rating on the other criteria (De Boer, Labro & Morlacchi, 2001). 4) The non-price criteria score was based on consensus between the members of an evaluation committee and is therefore a result of negotiation

Data was collected by the researcher herself and was gathered in an Excel list. The re-searcher has extensive experience in the area of public procurement, which enabled her to un-derstand the data, thus decreasing the probability of observer biases (Saunders, Lewis & Thorn-hill, 2009, p. 157). The purchaser involved in the project team of the specific tender was pri-marily responsible for registering the data. The recorded data was initially not produced for the purposes of research, but rather to substantiate choices in supplier selection. To ensure the re-liability of the dataset (Saunders, Lewis & Thornhill, 2009, p. 276), only purchasers in the net-work of the researcher were approached in order to ensure that the quality of the data could be trusted. This means that 16 different purchasers were involved. The data were registered and

(26)

25

stored in different ways. The university used an application to support their procurement pro-cess. Consequently, the necessary data was summarized on one page in a software application, with a standard measurement scale through all the tenders. The data of the other authorities’ were dispersed in several documents in a digital archive. The layout of the documents and the measurement scales differ between authorities and even between purchasers. In order to avoid misinterpretation, all data were checked by a second person who also has experience in public procurement.

The sample population includes 145 completed European tenders in the Netherlands, with a time frame from 2007 to 2014. All tenders were subject to the European directives 2004/14/EG (Eur-Lex, 2014) and therefore the data can be compared and generalized, despite their different timeframes and the fact that they only include one country (Bryman, 2012, p. 54).

4.1.6 Dependent Variable

The central dependent variable in this part of the study was the score difference between the highest score achieved on the non-price criteria and that of the runner up. To measure this difference, the highest and the second highest scores achieved were gathered. This was done because it is in the interests of the evaluation committee to increase this difference in order to enhance the probability of awarding the contract to the bidder with the highest non-price score. Data values per subject differed as contracting authorities are free to choose their own meas-urement scale to score the criteria per tender (Matteus, Ferreira & Carreira, 2010). Conse-quently, non-price scores could range from 0 to 1000 points, while others reached no further than a maximum score of 10 or 100 points. To be able to compare the scores of the different tenders and consequently the score differences, measurement scales were standardized accord-ing to the index number mechanism (Saunders, Lewis & Thornhill, p. 449). The non-price score

(27)

26

of the highest and the second highest score achieved was calculated relative to the maximum score non-price criteria. Thus the highest score per case was calculated as follows: (highest score achieved/maximum score)*100. The second highest score per case was calculated as fol-lows: (second-highest score achieved/maximum score)*100. This means that the maximum scores of the non-price criteria was gathered too. Because the non-price score can never exceed the maximum score, the index numbers could never be above 100, which provided a double check on typing errors. Finally, the absolute score difference between the highest score achieved on the non-price criteria and that of the runner up per case was measured as follows: Highest score per case – Second highest score per case. The study opted to measure the absolute score rather than the relative score because the absolute judgment of the non-price criteria was used to calculate the score of the non-price criteria.

4.1.7 Independent Variable

Hypotheses 1A and 1B focus on one independent variable, namely, the importance weight of the price criteria. As noted above, according to the MEAT procedure, the contract will be awarded to the bidder with the highest overall score. The overall score for the received pro-posals was calculated as follows: (Score price criteria * Importance weight of price criteria) + (Score non-price Criteria * Importance weight of the non-price criteria) (Lengwiler & Wilfset-ter, 2006). Because the price criteria and non-price criteria are related to each other, the inde-pendent variable varied between 1 and 100, and, together with the importance weight of the non-price criteria, had a total sum of 100. Both the importance weight of the price criteria and that of the non-price criteria were gathered in order to check the data. Seven tenders did not mention the relative weight directly, but only communicated the maximum achievable points for both the criteria. To make the cases comparable, the importance weight of the price criteria

(28)

27

was calculated as follows: (Maximum achievable points price criteria/ (Maximum achievable points price criteria + Maximum achievable points non-price criteria)) * 100.

4.1.8 Control Variables

A number of control variables were used to ensure that possible effects were caused by the independent variable and could not be attributable to other variables.

Authority

Public Procurement is often discussed as if it referred to a homogenous grouping of author-ities dealing with the same problems and environments (Loader, 2010). However, the public sector consists of a range of different organizations, each with their own culture, needs and organizational structures (Thai, 2001) which could impact their procurement results. That is why the authorities are included as a control variable (1=Municipality A; 2=Municipality B; 3=Municipality C; 4= University; 5=Ministry).

Contract type

In their application of procedures and conditions, the European directives distinguish be-tween the procurement of work, supplies and services. For example, the thresholds for public work contracts are many times higher than the thresholds for the supply of service contracts (Eur-Lex, 2014). This means that in the sample used for this study, the work contracts have a higher contract value compared to service and supply contract. Furthermore, different procure-ment procedures are commonly applied to the different contract types (Ohno & Harada, 2006). Finally, the requirements for non-price criteria differ according to the various contracts. Be-cause these various contexts could have affected the results, this variable distinguishes between

(29)

28

contracts for supplies, services and work. The researcher assigned the various tenders to con-tract types using the description given in European directives 2004/14/EG (Eur-Lex, 2014) . Works includes all buildings and civil engineering works. This includes the construction of a bridge, an office building or a road. Maintenance and renovation are also works. Supplies in-cludes the purchase, rent or leasing of products. Supplies are tangible, but are not covered by work contracts. Services includes the remaining procurements. In order to avoid misinterpreta-tion, the classification was checked by a jurist who specializes in public procurement law. (1=Public work contracts; 2=Public service contracts; 3=Public supply contracts)

Number of bidders

Relations of power and dependence between the contracting authorities and their suppliers varies according to industry (Dubois & Pedersen, 2002) and this affects the number of bidders. Evaluating multiple bidders increases the number of options in the decision-making process and the number of options affects the decisions-making (Haynes, 2009). The number of bidders is included as a control variable. Only the bidders who submitted a valid proposal and were consequently evaluated by the evaluation committee were counted as bidders.

Number of non-price criteria

The number of non-price criteria varies per tender. Like the number of bidders, the number of price criteria increases the number of options in decision-making. The number of non-price criteria included the key criteria, and sub-criteria were excluded.

(30)

29

Evaluation performance scale

The contracting authority is free to choose a performance scale per tender, according to which the members of the evaluation committee have to evaluate the non-price criteria (Mateus, Ferreira & Carreira, 2010). However, as Dolnicar and Grün (2009) and Weng (2004) discuss, people response differently according to various construct scales and therefor the evaluation of the non-price criteria could be affected by the performance scale. These authors compare dif-ferent measurement levels and their approach was adopted in this study. The evaluation perfor-mance scales used were: 1) a 3-point scale; 2) a 4-point scale; 3) a 5-point scale; 4) a 6-point scale; and, finally, 5) a 10-point scale. At seven tenders the performance scales were unknown.

Number of members in the evaluation committee

The group size has an impact on group decisions because size impacts the range of factors such as expertise, interests and preferences (Mannix, Thompson & Bazerman, 1989). The num-ber of individuals participating in the evaluation committee was unfortunately untraceable for 49 cases. This meant that representative analyses could not be made and so it was decided to exclude this control variable.

4.5.Results

4.2.3. Descriptive Statistics

The sample used in this study includes 145 completed European tenders from five different Dutch authorities. Table 1 includes the sample characteristics, including the dependent, inde-pendent and control variables. The greater part of the sample consists of data from one munic-ipality and one university. Moreover, the samples are not evenly distributed between the con-tract types as the service concon-tracts make up the vast majority. Table 2 shows the correlations

(31)

30

Table 1 Sample Characteristics

between the continuous variables in this study. The independent and dependent variables are discussed at greater length below.

The independent variable:

The mean of the importance weight of the price criteria, and therefore the degree of uncer-tainty, in this sample was 42.85% (SD = 19.57). This means that an average of 42.85% of the overall score for the proposals was determined by the price criteria, and consequently an aver-age of 57.15% (SD = 19.57) was determined by the non-price criteria. We see from this that the evaluation committee has an important impact on the overall outcome.

The mean and the median of the importance weight are nearly the same, which indicates that the distribution of the importance weight of the price criteria in the sample is rather symmet-rical (Washington, Karlaftis, Mannering, 2003). The importance weight was not significantly

Variable Frequency (N) Percent (%)

Authority Municipality A 61 42.1 Municipality B 14 9.7 Municipality C 15 10.3 University 48 33.1 Ministry 7 4.8 Contract Type Work contracts 13 9.0 Service contracts 111 76.6 Supply contracts 21 14.5

Variable N Mean Median Std. Deviation Minimum Maximum

Absolute score difference 145 10.71 9.69 0.69 0.00 37.03

Importance Weight Price criteria

145 42.85 40.00 19.16 0.00 90.00

Number of bidders 145 6.15 5.00 5.74 1 38

Number of non-price crite-ria

138 4.48 4.00 2.02 1 11

Evaluation performance scale

(32)

31

Table 2 Correlations among the continuous variables

affected by the authorities, H(4) = 4.26, p > 0.05. This means that there is no difference in the price criteria between the different authorities.

However, the contract type had a significant influence on the importance weight, H(2) = 11.73, p < 0.05. At work contracts (M = 56.54), the price criteria were more important than at supply (M=46.84) or service (M=40.49) contracts. A Kruskal-Wallis Test was used for both analyses because the data did not meet the assumptions of a parametric test.

Furthermore, the number of the non-price criteria and the importance weight of the price criteria have a negative statistical relationship, r = -0.313, p < 0.01. In other words, if a tender had more non-price criteria, the importance weight of the price criteria was lower and vice versa. The same thing appears to occur with the number of bidders, i.e. the number of bidders increases when the price criteria becomes less important. The variables have a negative statis-tical relationship, r = -0.202, p < 0.005.

The dependent variable:

The dependent variable is the absolute difference between the highest score achieved on the non-price criteria and that of the runner up. In this sample, the mean of the highest score achieved was 86.01 (SD = 12.09), and the mean of the second highest score achieved was 75.30 (SD = 13.58). This means that there is an average difference of 10.71 between the two scores. This difference is significant, t(145) = 10.71, p < 0.01.

A Kruskal-Wallis Test was again used to test whether the data differed between different authorities and type of contracts. It was founded that the absolute difference was significantly

Variable 1 2 3 4 5

1. Absolute score difference -

2. Importance weight of the price criteria -0.007 -

3. Evaluation Performance Scale 0.096 -0.097 -

4. Number of bidders 0.004 -0.202* 0.278** -

(33)

32

affected by the authority, H(4) = 13.86, p < 0.01, Municipality A (M= 12.57); Municipality B (M= 4.87); Municipality C (M= 6.35); University (M= 11.23); Ministry (M= 12.07), but that the contract type did not influence the absolute difference, H(2) = 2.35, p > 0.05. Table 1 shows correlations between the continuous variables and indicates that none of the other variables have a significant direct relationship with the dependent variable, i.e. the absolute difference between the highest score achieved on the non-price criteria and that of the runner-up.

4.2.4. Testing of Hypotheses

The causal effect of the importance weight of the price criteria on the absolute difference between the highest non-price score achieved and the runner up is the subject of Hypotheses 1. Hypothesis 1A predicted a negative casual effect, i.e. that the higher the importance weight of the price criteria, the greater the difference between the highest score achieved of the non-price criteria and that of the runner up. By contrast, Hypothesis 1B predicted a positive causal effect, i.e. that the lower the importance weight of the price criteria, the greater the differ-ences between the highest score achieved of the non-price criteria and that of the runner up.

As mentioned previously, Pearson’s correlation analyses showed that there was no statis-tical relation between the absolute difference and the importance weight of the price criteria. (See Table 1). The result of this analysis is -0.007, which indicates that there is no relation among the variables, but rather a near perfect independence (Saunders, Lewis, Thornhill, 2009, p. 459). Because a positive or a negative relationship was expected, the hypotheses were tested in a two–tailed test (Field, 2009, p. 384). The p-value of the two-tailed test was 0.929 and was therefore not significant.

To test whether the importance weight of the price criteria (independent variable) has a positive or negative effect (dependent variable) on the absolute difference, a simple regression

(34)

33

was conducted. The results suggest that there was no significant proportion of the total variation in the absolute difference predicted by the importance weight. In other words, the importance weight of the non-price criteria is not a good predictor of the absolute difference, F(1, 143) = 0.008, p > 0.05.

Testing for interaction effects also failed to show any significant effects. The output can be seen in Table 3.

Table 3 Output for Importance weight Price Criteria * control variable on absolute score difference

Interaction effect (continuous control variables)

Variable B SE B β

Importance weight Price Criteria * Number of bidders

Constant 9.04 2.85

Importance weight Price Criteria 0.05 0.07 0.115 Numbers of Bidders 0.32 0.39 0.22 Importance weight Price Criteria *

Number of Bidders

-0.01 0.01 -0.24 Importance weight Price Criteria * Number of non-price criteria

Constant 11.15 4.73

Importance weight Price Criteria 0.00 0.01 0.01 Number of non-price criteria -0.03 0.97 -0.01 Importance weight Price Criteria *

Number of non-price criteria

0.00 0.02 -0.03 Importance weight Price Criteria * Evaluation performance scale

Constant 15.08 5.58

Importance weight Price Criteria -0.04 0.12 -0.09 Evaluation performance scale -0.441 0.69 -0.13 Importance weight Price Criteria *

Performance scale

0.00 0.01 0.05

Interaction effect (categorical control variables)

Variable F Df P

Importance weight Price Criteria * Authority

Importance weight Price Criteria 0.54 19 0.939

Authority 4.28 4 <0.001

Importance weight Price Criteria * Authority

0.65 17 0.844 Importance weight Price Criteria * Contract Type

Importance weight Price Criteria 0.45 19 0.975

Contract Type 1.37 2 0.258

Importance weight Price Criteria *Contract Type

0.55 17 0.835

(35)

34 4.6.Discussion

The research into existing data focused on the causal effect of the importance weight of the price criteria on the absolute difference of the non-price criteria between the highest score achieved and that of the runner up. In contrast to both assumptions, the research found no evi-dence that the importance weight of the price criteria has a positive or negative effect on the non-price score. In other words, no evidence was found that uncertainty had influenced the decision of the evaluation committee. This finding is not in line with existing theories such as the expected utility theory (Von Neuman & Morgenstern, 1944 as cited in Lipshitz, Klein, Orasanu, & Salas, 2001), the prospect theory (Tversky & Kahneman, 1979) and the “Ellsberg effect” (Daniel Ellsberg , 1961).

However, using existing data in research has some limitations and disadvantages. A previ-ously mentioned disadvantage is that the variables include information about the outcome rather than the process that led to the outcome (Shadish, Cook & Campbell, 2002, p. 203). The eval-uation committee may have been influenced by other factors that were not observed in the available data. The environment of the public procurement process is complicated and is influ-enced by many forces (Thai, 2001). In addition to internal forces, the environment of the public procurement process is also influenced by market forces, political forces, legal forces and social and economic forces. Therefore buying decisions are influenced by individual, social, organi-zational and environmental variables (Webster & Wind, 1972).

Consequently, a critical question that arises is whether all tenders can be treated as a homo-geneous group. Hypotheses H1A and H1B are based on preferences for the bidder who scores best on non-price criteria. The research assumed that the evaluation committee would protect itself against the risk that the contract would not be awarded to this bidder because their costs were too high. But can this assumption still be made in a complex environment such as the public procurement environment? There are differences between the procurement markets of

(36)

35

different contexts (Kraljic, 1983) In some purchasing areas, buying markets are saturated and the quality between different suppliers is almost the same. In such cases, the evaluation com-mittee does not have any preferences and group members do not care which bidder the contract is awarded to. Alternatively, after judging the non-price criteria, they may have concluded that one bidder is by far the worst and that their only goal is to prevent the contract being awarded to this supplier. In order to do this, they may increase the differences of the non-price criteria score between the score of their least favorable bidder and that of the penultimate. None of these effects can be observed in the available data because only the non-price score of the high-est score achieved and that of the runner up were gathered.

Another factor that could have influenced the judgment of the evaluation committee is the composition of the evaluation committee. The existing literature shows that several compo-nents, such as power dispersion (Greer & van Kleef, 2010; Van Lange, Joireman, Parks & Van Dijk, 2013), social factors (Beersma & Dreu, 1999) or the forming of coalitions (Komorita & Parks, 1995; Schei & Rognes, 2005), influence group decision-making. Factors that may have been included in the environment of the public procurement process and could consequently have affected the judgement of the evaluation committee, could not be observed in the available data.

In summary, there could be different explanations as to why no evidence was found that the importance weight of the price criteria has a positive or negative effect on the non-price score. However, since there is a growing trend to award the contract to the bidder who provides the best value for money (Ohno & Harada, 2006; Bergman & Lundberg, 2013), the quality of the decision of the committee that is responsible for the evaluation of the non-price criteria becomes more important. This scenario was confirmed by the data observed in this study. In this sample, a typical pattern was that of a 40% price weight combined with 60% non-price weight. Contrary to the findings of Waara and Bröchner (2006), the non-price criteria are more important in the

(37)

36

overall score than the price criteria. It is therefore important to gain a clear understanding of the effect of uncertainty on the judgement of the evaluation committee, and to exclude the possi-bility that a lack of information could influence the evaluation of the non-price criteria.

The following chapter therefore discusses the experiment that was designed. The advantage of an experiment is that a casual relation can be simulated in a highly controlled environment in which both the cause and the conditions can be manipulated. As result, group decisions are made under the same circumstances and the conditions in question, such as preferences and group composition, no longer play a role.

(38)

37

5. Experimental Research

As discussed previously, the experimental research has two objectives. The first is to test whether the differences in the non-price criteria score between the highest score achieved and that of the runner up are affected by the importance weight of the price criteria (Hypotheses 1A and 1B) in a controlled environment (Saunders, Lewis & Thornhill, 2009, p. 591). The exami-nation of existing data showed no evidence that the importance weight of the price criteria has a positive or negative effect on the non-price score. However, the environment of the public procurement process is complex and evaluators may be influenced by other factors that are not observed in the available data. One of the advantages of an experiment is that it provides an opportunity to control other variables (Saunders, Lewis & Thornhill, 2009, p. 144). The exper-imental design therefore needs to ensure that all potential variation on the dependent variable can only be caused by the factors of interest (i.e., variation of the price factor weight), but not by any other factors that are not controlled for. The second objective of the experiment is to test whether different degrees of uncertainty affect biases in the non-price score. Existing data only contain information about the outcome rather that the process leading to the outcome (Shadish, Cook & Campbell, 2002, p. 203). Therefore, in order to test hypotheses 2A and 2B, the causal effect of the degree of uncertainty on the biases in the non-price score is the second central point in the experiment.

Given the restricted time available for this research project, its focus lies on developing a new negotiation simulation paradigm with which to research public procurement processes. A pre-test of the simulation was run to adjust the materials, but the actual carrying out of the experiment was left for follow-up research. In the paradigm developed, three parties negotiate about the judgement of the non-price criteria in a tender evaluation, based on mixed motive aspects. The three negotiators have opposing individual interests, which create the basis for a task conflict and the necessity for negotiation given that all parties have to agree on the final

(39)

38

solution. In contract, although unknown to the participants, the negotiators have similar group interests. This provides them with opportunities for negotiating in their groups’ interest and increases the overall joint profit rather than individual profits.

The following section discusses the negotiation paradigm further and demonstrates why particular choices were made. The conducted pre-test is then discussed, followed by a discus-sion of the lessons that were learned from it.

5.1 Proposed Research Design and Stated Goals

The proposed research design was based on a one factorial design with two between subject conditions, the low uncertainty condition (importance weight price criteria of 30%) versus the high uncertainty condition (importance weight price criteria of 70%). In a between group de-sign, there is no possibility that performance in one condition can affect performance in the other, because of factors such as memory or learning effects (Field & Hole, 2003, p. 75). Once a participant has participated in an evaluation process, they have become more familiar with it, and this could affect the negotiation process and the outcome.

Since this study focuses on group decision-making, the negotiation needs to be carried out in groups with a minimum of three members (Schei & Rognes, 2005). An evaluation committee consists of five experts (Bana e Costa, Corrêa, De Corte & Vansnick, 2002; Csáki & Adam, 2010; Tsai, Wang & Lin, 2007), but including more members in a group complicates the tiation process (Mannix, Thompson & Bazerman, 1989). This study therefore limited the nego-tiation task to a group consisting of a municipal director, a municipal project manager and a municipal senior construction specialist. Since the literature defines small groups as having 20 or less members, the researcher assumes, that the results of this study can be generalized to an evaluation committee with 20 or less members (Mannix, Thompson & Bazerman, 1989).

The influence of uncertainty on public procurement

Public Procurement