Unpacking the spillover effect : the investigation of cognitive distortion, the halo effect and basic anchoring as possible causes

(1)

Amsterdam Business School

Unpacking the spillover effect:

The investigation of cognitive distortion, the halo effect and basic anchoring as possible causes

Name: Razvan-Stefan Ghita Student number: 10636633

Thesis supervisor: dhr. prof. dr. V.S. (Victor) Maas Date: 10 January, 2016

Word count: 12,126

(2)

Statement of Originality

This document is written by student Razvan-Stefan Ghita who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

Unpacking the spillover effect:

The investigation of cognitive distortion, the halo effect and basic anchoring as possible causes

Abstract

This paper examines which psychological concepts explain the spillover effect of unrelated information into subjective evaluations. Prior research has shown that cognitive distortion causes the spillover effect. However, cognitive distortion does not perfectly explain some empirical findings regarding the spillover effect. I hypothesize that the spillover effect can appear in a variety of situations and can be caused by cognitive distortion, the halo effect or basic anchoring. The experimental data supports cognitive distortion and basic anchoring as a potential cause for the spillover effect. Partial support was found for the hypothesis that the halo effect can cause the spillover effect.

Keywords:

(4)

This thesis examines which psychological concepts explain the spillover effect of unrelated information into subjective evaluations. Consistent with cognitive distortion, I investigate whether the knowledge of a performance level on another dimension biases subjective judgements through the internal tendency toward consistency of the supervisor. I also study if, consistent with the halo effect, the performance level on another dimension can be seen as indicative of the overall performance, thus biasing subjective evaluations. Finally, I examine if, consistent with basic anchoring, the spillover effect exists when the supervisor is primed with a measure that is not indicative of performance on any dimension.

Subjectivity permits the incorporation of noncontractible information into the evaluation of employees (Bol 2008), and therefore reduces the risk of misalignment for the performance evaluation system (Hayes and Schaefer 2000).

As a result, performance measurement systems may include both objective and subjective measures. The extent to which noncontractible information is included in subjective evaluation is diminished by the spillover effect of an unrelated piece of information into the subjective evaluation. Thus, subjective performance evaluation becomes less effective and the benefits of introducing subjectivity into performance measurement systems can be diminished.

The research question is important because, by understanding the underlying psychological mechanisms responsible for the spillover effect, designers of performance evaluation system will be better equipped to prevent it.

Current literature has proven the existence of a spillover effect, defined as the tendency of supervisors to “be influenced by the observed level of performance on one task when subjectively evaluating an employee’s performance on a separate task” (Bol and Smith, 2008: 1217). The spillover effect was explained through cognitive distortion, the predisposition of supervisors to unintentionally bias subjective evaluations in such a way that it does not contradict existing knowledge extracted from an unrelated objective performance measure (Bond et al. 2007). However, the causal relationship between cognitive distortion and the spillover effect can be contested due to the single instance in which the phenomenon has been studied. I slightly modify the conditions of the experiment in order to test this causal relationship.

I used a scenario experiment to answer the research question. Participants in the experiment were experienced supervisors recruited using the Prolific.ac website, an alternative for Amazon.com’s Mechanical Turk (MTurk). The task of the experiment was to subjectively

(7)

evaluate one employee on a performance dimension based on evidence that was constant across all conditions and was designed to indicate a neutral performance. In the treatment conditions, three groups were designed to test the influence of each of the causing mechanisms and one group was designed to replicate the Bol and Smith (2011) experiment. The group of condition varied the nature of the biasing information: the scale (absolute or relative), timing (before or after the relevant evidence) and relevance (objective performance measure on a separate dimension or anchor).

The results indicate that cognitive distortion and basic anchoring can cause the incorporation of unrelated information into subjective evaluations and partial support was found for the hypothesis that the halo effect produces the spillover effect. The replication of the Bol and Smith (2011) experiment showed that the mechanisms behind the spillover effect do not interact in a straightforward manner.

The thesis contributes to accounting literature on performance evaluation. By exploring additional causes for the spillover effect (the halo effect and basic anchoring), the thesis helps to better understand variances in the spillover effect phenomenon. For example, basic anchoring is better fit to explain the results of Heneman (1986), stating that the correlation between subjective and objective performance measures was stronger (weaker) when a relative (absolute) rating format was used.

Another contribution is the answer to a call for future research made by Bol and Smith (2011). The authors suggested that the spillover effect can also appear if the biasing information is presented after the relevant information. I investigated this possibility and found limited support for it.

The thesis is structured as follows. In the first chapter I review background literature and develop the hypotheses. The following chapter presents the methodology of the experiment. The next chapter presents the results. The final chapter concludes the thesis and discusses implication of the results.

(8)

2 Literature

This study analyzes the underlying mechanisms behind the spillover effect. The spillover effect was originally defined as the unintentional tendency towards consistency between subjective evaluations and knowledge about performance on other dimensions than the one evaluated (Bol and Smith 2011).

Bol and Smith (2011) argue that the underlying mechanism responsible for the spillover effect is cognitive distortion, the interpretation of new information in a biased way due to previously held beliefs. However, their findings could potentially be explained by other short circuits in the decision-making process. The two other mechanisms I propose as additional and non-exclusive explanations for the spillover effect are:

1. the halo effect, a piece of information with favorable implication that leads evaluators to infer that the person is performing well on all dimensions (Sundar and Kardes 2015) and 2. basic anchoring, the presence in the environment of a number that is completely

unrelated to the decision being made that, nonetheless, influences the final evaluation (Critcher and Gilovich 2008).

Figure 1

Spillover possible causes

The spillover effect

Bol and Smith (2011) Study

Anchoring

Halo effect

(9)

The rest of the literature section is structured as follows: the first section contains a description of subjective performance measures and examines influences on the outcome of subjective evaluations. The next sections introduces the spillover effect and enumerates the proposed causes for this phenomenon. The final sections examine and compare the proposed underlying mechanism, cognitive distortion, the halo effect and basic anchoring. The hypotheses are also developed in the last sections.

2.1 Subjective performance evaluation

Objective performance measures are seldom perfect, their narrow focus and distort or ignore information regarding important aspects of the employee’s activities (Baker et al. 1993). Due to this imperfection, managerial incentive contracts become incomplete, leaving managers free to ignore important but unmeasured activities resulting in a suboptimal effort allocation (Feltham and Xie 1994).

To mitigate the lack of complete congruence of objective performance measures, subjective evaluation can be used to reduce the distortions present in quantitative measures (Gibbs et al. 2003). Subjective performance measures allow supervisors to exercise discretion and to include noncontractible information in the evaluation of employees’ efforts, thus creating a more compete representation of the performance (Bol 2008). An instance in which the benefits of subjective evaluations are derived appears in the article by Gibbs et al. (2002). The authors show that subjective bonuses are positively related to the extent of long-term investments in intangibles, a noncontractible dimensions that increases firm value.

It is however unclear exactly what factors do supervisors consider when asked to subjectively evaluate an employee. For example, Ittner et al. (2003) found that psychology-based explanations are more prominent than economic-based explanations when it comes to the subjective weighting in a Balance Scorecard approach. Bol et al. (2010) found that supervisors use discretion in target setting in order to reduce their potential confrontation costs by setting easier targets for employees with relatively higher hierarchical status.

It is important to note that subjective evaluation is a specific instantiation of decision making. In other words, apart from the specific nuances of making subjective evaluations, the process is also influenced by all the usual biases and heuristics present in everyday decision making.

The psychology research is prolific in studies that prove humans, including subjective evaluators, are not perfect decisions makers (Kahneman 2011, Banaji and Greenwald 2013). Biases and

(10)

load of giving verdicts (Tversky and Kahneman 1974). An instance of biases and heuristics in subjective evaluation is the spillover effect which will be examined in the following section.

2.2 The Spillover Effect

The spillover effect relates to the unintentional tendency towards consistency between a subjective evaluation and knowledge about another performance dimensions (Bol and Smith 2011). For the purposes of this study, the spillover effect is defined more broadly as the unduly influence of an unrelated piece of information on a subjective evaluation.

The spillover effect has been demonstrated by Bol and Smith (2011) in an experimental setting. In the study, participants were asked to subjectively rate managers on a dimension while having knowledge about the managers’ objective performance on a completely unrelated dimension. The results showed that participants who were assigned in the high (low) objective performance condition judged the performance on the unrelated dimension significantly higher (lower). This effect was attributed to cognitive distortion, a way of processing information in a manner that does not alter previously held beliefs.

Numerous studies have validated the correlation between subjective and objective performance measures (Bommer et al. 1995; Murphy and Cleveland, 1991; Heneman 1986; Heneman 1986). The premise of this studies was mostly different, examining if subjective and objective measures could be used interchangeably. However, the correlation can be interpreted as the spillover effect.

Heneman (1986) examined the correlation between subjective and objective performance measures by performing a meta-analysis and concluded that, although the correlation is high and significant, subjective and objective performance measures cannot be used interchangeably. The correlation was stronger (weaker) when a relative (absolute) rating format was used.

Merchant et al. (2010) studied the spillover effect in a real world setting and found a small correlation between objective and subjective measures. The authors argued that, in the case of higher level managers, the objective performance indicators are considered less informative of the overall performance, thus having a lower influence on the subjective evaluation.

Based on the literature review, I propose three non-exclusive mechanism that cause the spillover effect: cognitive distortion, basic anchoring and the halo effect.

(11)

1. Cognitive distortion involves interpreting new information in a biased way due to current beliefs (Bond et al. 2007).

2. The halo effect appears when a piece of information with favorable implication that leads evaluators to infer that the person is performing well on all dimensions (Sundar and Kardes 2015).

3. Basic anchoring relates to the influenced of a number, unrelated to the judgement being made, in the environment that nevertheless influenced the final evaluation (Critcher and Gilovich 2008).

The following sections explain why a causal relationship between the above mentioned biases and the spillover effect is plausible and presents the concepts more fully. Further, operational distinctions between the concepts are made in order to examine their relationships to the spillover effect.

2.2.1 Cognitive Distortion

Bol and Smith (2011) define cognitive information distortion as a way of processing information in a manner that does not alter previously held beliefs. This explanation does not focus on how the information is gathered or the relative weight it receives, rather it considers the way in which new information is interpreted in light of the previously held belief (Bond et al. 2007).

Cognitive distortion can influence judgements and preferences. In an experimental study, Carlson et al. (2006) manipulated the order in which participants received brand related information and found that they could alter the preferred brand by the position of the favorable information.

2.2.2 The Halo Effect

Merchant et al. (2010) found a small correlation between objective and subjective performance measures for the evaluation of high-level managers in an Australian corporation. The authors argued that the objective performance indicators are considered less informative of the overall performance in the case of higher level managers. They reached this hypothesis by employing the halo effect as the underlying mechanism behind the spillover effect and by examining the informativeness of the objective performance measure.

The halo effect can be defined as the overgeneralization of one piece of information that leads evaluators to conclude that the evaluatee has favorable characteristics on different dimensions (Sundar and Kardes 2015). The initial research body of literature that examined the halo effect

(12)

focused on physically attractive persons and how people infer other positive qualities when meeting them (Eagly et al. 1991).

In accounting research, the halo effect has been shown to influence the assessment of audit risk (O'Donnell and Schultz 2005). The authors examine the consequences the auditors’ holistic opinion about the strategic risk on their ability to detect misstatement risks in patterns of fluctuation in accounting. They find that auditors who develop a lower (higher) strategic risk assessment are less (more) sensitive to inconsistent fluctuations.

The following section will examine the relationship between the halo effect and cognitive distortion and will develop hypotheses about their link with the spillover effect.

2.2.3 Cognitive distortion vs the Halo Effect

Cognitive distortion relates to the biased way in which prior knowledge influences the interpretation of new information. The halo effect involves a piece of information that the evaluator considers indicative of the overall performance of the evaluatee.

For the purposes of this thesis, the concepts will ignore the nuances involved (eg, knowledge vs information) and focus on a specific distinction between the two definitions.

Cognitive distortion is operationalized as a piece of information that biases the manner in which other information is received and processed. The halo effect is defined as the exaggerated effect of a piece of information on the total evaluation. More explicitly, the distinction between the concepts focuses solely on the time of delivery of the biasing piece of information. In the case of cognitive distortion, the biasing piece of information is presented before the relevant evidence, while in the case of the halo effect the biasing piece of information is presented after the relevant evidence.

In other words, this thesis assumes that the cognitive distortion is a specific instance of the halo effect, one in which the exaggerated effect that one piece of information is intensified by the evaluator’s biased way of interpreting new information.

I attempt to separate the two effects and observe their direct relationship with the spillover effect. Thus, I predict that, consistent with cognitive distortion, prior knowledge about the employee’s performance on an unrelated objective measure will bias the way in which relevant information for the subjective evaluation is interpreted and consequently will influence the subjective performance evaluation.

H1: The subjective evaluation will be more favorable (unfavorable) when the supervisor has knowledge about a high (low) level of performance on an unrelated dimension, measured

(13)

objectively and on a different scale that the subjective evaluation, prior to interpreting relevant information regarding the subjective evaluation

Additionally, I predict that, consistent with the halo effect, the performance on a separate dimension will be considered indicative of the overall performance and will influence the subjective performance evaluation.

H2: The subjective evaluation will be more favorable (unfavorable) when the supervisor observes a high (low) level of performance on an unrelated dimension, measured objectively and on a different scale that the subjective evaluation, after interpreting relevant information regarding the subjective evaluation

2.2.4 Anchoring

Heneman (1986) examined the correlation between subjective and objective performance measures and concluded that the correlation was stronger (weaker) when a relative (absolute) rating format was used. If cognitive distortion is assumed to be the singular mechanism responsible for the spillover effect, the above mentioned results cannot be theoretically explained. Thus, I propose that the nature of the scale (absolute or relative) for the objective and subjective performance measures influences the spillover effect. The theoretical construct that can explain this phenomenon is anchoring.

Anchoring was first conceptualized by Tversky and Kahneman (1974). In their study, the authors asked participants to generate a random number by spinning a wheel and then estimate the number of nations from Africa that are part of NATO. They found that the answers were systematically influenced by the random number. The effect was labeled as the anchoring-and-adjustment heuristic and explained by the insufficient anchoring-and-adjustments made from that anchor. Following the initial study, the effect was replicated in both experimental and field settings (Epley and Gilovich 2006) and in that process the term was used for explaining a wide variety of phenomena. Due to the high variability of effects that were labeled as anchoring, the concept was explained by a number of conflicting mechanisms.

In an accounting context, van der Heijden (2013) studied the effect of varying the anchor on subjective evaluations of a dual performance measure. The author found that the choice of anchor influences subjective performance through the accentuation of relevant information.

(14)

It is important to note that the term "anchoring" currently describes an effect, the assimilation of an unrelated piece of information in a decision, rather than the responsible mental mechanism (Mussweiler and Strack 2000).

Relevant for this study is a later instantiation of this class of phenomenon, namely basic anchoring. Basic anchoring is present when ‘‘...people’s judgments of a target are inﬂuenced by an anchor that is completely uninformative and people are not asked to consider the anchor as a possible target value’’ (Wilson et al. 1996: 389). For example, Wilson et al. (1996) found that, when asked about the number of physicians in their city, participants’ estimates were significantly influenced by their highlighted ID number.

The above mentioned example specifically drew attention to an anchor. Testing the limits of this effect, Critcher and Gilovich (2008) conducted experiments in which the participants estimated an athlete’s performance based on a photo and found that a higher number on the jersey positively influenced the evaluation. The authors maintained the basic anchoring label for this effect and associated it with simple numeric priming (Wong and Kwong, 2000).

What is relevant, for the purposes of this study, is basic anchoring where attention is not drowned to the anchor. It relates to the presence in the environment of a number, unrelated to the judgement being made, that influences the final evaluation (Critcher and Gilovich 2008). The border between cognitive distortion and anchoring is fuzzy and in some cases the two terms are used interchangeably (Mumma, and Wilson 1995). Although the scope of this study is not to examine the conceptual ambiguity between cognitive distortion and anchoring, a clear dichotomy is needed in order to understand and operationalize the mechanism behind the spillover effect. The following section will operationalize anchoring in the context of this thesis and will develop a hypothesis about the causal relationship between basic anchoring and the spillover effect. 2.2.5 Cognitive Distortion vs Anchoring

As explained above, the distinction between the concepts of cognitive distortion and anchoring is unclear. The former explains a biased algorithm of interpreting information, while the latter describes, in a rather broad manner, the effect of a piece of information presented in the early stages of a decision.

In their article, Bol and Smith (2011) specifically mention that the cognitive distortion effect is conceptually different from anchoring. Their argument is based on a conceptually narrow approach to anchoring, one that involves biased accessibility of information from memory as a responsible mechanism.

(15)

Because anchoring has broaden as a concept and has come to encapsulate many diverse mechanism that result in a large array of effects, it is difficult to argue that a situation in which a piece of information is presented in the early stages of the decision making process cannot be identified as an instance of anchoring.

Theoretically similar studies to Bol and Smith’s (2011) article, exploring how a piece of information that should not be incorporated in the final decision ends up being significant, are abundant in the legal domain and the effect is constantly labeled as anchoring. For example, Englich et al. (2005) found that the prosecutor’s sentencing demand affect the final sentence in rape court cases due to the anchoring effect.

While it is worthwhile to note the conceptual ambiguity between anchoring and cognitive distortion, an operationalization that would clearly indicate which mechanism is responsible for the spillover effect becomes impossible in this conditions. Due to the different nature of the proposed explanations, the motives behind the spillover effect are in a false dichotomy situation. In order to solve this issue I will only focus on basic anchoring where attention is not drawn to a completely unrelated anchor. Thus framed, a dichotomy is created between basic anchoring and cognitive distortion that can be operationalized and tested. The distinction is based on the relevance of the information that biases the decision. In the case of basic anchoring, the biasing information is completely unrelated to the judgement being made.

With this dichotomy in place, it can be argued that the spillover effect of objective performance measures into subjective measures from the Bol and Smith (2011) article is influenced by both cognitive distortion and basic anchoring. The operationalization of the studied variables indicates a strong propensity towards basic anchoring.

The main argument for the presence of basic anchoring is the identical scale used for both the subjective and objective measures. Participants were informed about the performance on the objective performance measure on a 1 to 10 scale and asked to judge the performance on a subjective measure on the same 1 to 10 scale.

Heneman (1986) examined the correlation between subjective and objective performance measures and concluded that the correlation was stronger (weaker) when a relative (absolute) rating format was used. Chapman and Johnson (1994) conducted experiments to test the limits of anchoring and discovered that expressing both anchors and preference judgements on the same scale made anchoring possible.

(16)

The presented literature suggests two non-exclusive and intertwined mechanisms that cause the spillover effect: on the one hand, cognitive distortion, involves interpreting new information in a biased way. On the other hand, basic anchoring, emerges when, the presence of a number in the environment influences the final evaluation, although the number is unrelated to the judgement being made.

I attempt to separate the two effects and observe their direct relationship with the spillover effect. The direct effect of anchoring is examined. I predict that, consistent with basic anchoring, the presence of a number, unrelated to the judgement being made, in the environment will bias the subjective performance evaluation.

H3: Subjective evaluation will be more favorable (unfavorable) when the supervisor is presented with a high (low) anchor, measured on the same scale as the subjective evaluation.

(17)

3 Method

In order to test the hypotheses I conducted an experiment in which participants assumed the role of a regional director who oversees district managers. The district managers were responsible for two separate activities: sales and office administration. Participants were informed that the two attributions were completely independent of each other. The task of the experiment was to subjectively evaluate one district manager on the office administration dimension.

I examined how the subjective evaluation score varies based on the nature and the value of the biasing information. The relevant information for the subjective evaluation was designed to indicate a neutral performance and was constant across all conditions.

Participants can fall into one of nine conditions that are that are organized into four treatment groups and a control condition. The treatment groups are described in Table 1.

Table 1

Description of groups

Group Characteristics Tests Presented information

Distortion

 Presents an objective performance measure  on an absolute (not relative) scale

 prior to relevant information

H1

High ($ 476,553) or low ($ 115,758) values for

the individual sales dimension

Halo

 Presents an objective performance measure  on an absolute (not relative) scale

 after relevant information

H2

High ($ 476,553) or low ($ 115,758) values for

the individual sales dimension

Anchor

 Presents an irrelevant anchor  on an relative (not absolute) scale  prior to relevant information

H3

High (9) or low (2) values for the “number

of employees” variable

Original

 Presents an objective performance measure  on an relative (not absolute) scale

 prior to relevant information

-

High (9) or low (2) values for the individual

(18)

The experiment design used in this thesis is based the instrument developed by Bol and Smith (2011), which was provided by the authors.

Neither an anchor nor any information about the objective performance measure is provided in the control condition.

3.1 Procedure

The online experiment was conducted between 19/4/2016 and 16/5/2016 through the distribution on an URL. Participants were randomly assigned to one the 9 conditions at the beginning of the experiment. The last 10 participants were randomly assigned to the conditions that had the fewest respondents.

On average, the experiment took approximately 8 minutes for all participants to complete and 8.5 minutes for the participants who passed all the attention checks.

Participants were asked to read a hypothetical scenario in which they assumed the role of a regional director that is responsible for overseeing 10 district managers in a manufacturing company that sells pipes and fittings.

The case informed participants that the performance of district managers is judged on two criteria that are weighted equally: individual sales, an objective performance measure, and office administration, a subjective performance determined at the participants’ discretion. The case also states that the two criteria are independent of each other and that the task of the participants is to subjectively evaluate the performance of one particular manager on the office administration dimension.

At this point, participants in the Cognitive Distortion group the Original group were informed about the individual sales performance for all the supervised managers. The Anchoring Group was presented with an anchor (number of employees) for each of the ten managers they supervise. In the Cognitive Distortion group, the individual sales measure was presented in absolute terms together with a graph that indicated the performance of the manager as compared to his peers. In the Original group, the individual sales measure was presented on a 1 to 10 scale. Participants in the control group received none of this information.

In the next step, the case presented information that was relevant to determine the office administration performance. Participants were informed that the performance could not be directly observed so they would have to make their judgement based on personal notes and interviews with the members of the office, which were presented. This information was constant

(19)

for all conditions and was designed to indicate an average level of office administration performance.

Participants in the Halo group were informed about the individual sales performance for all the supervised managers after reading the relevant information regarding the subjective evaluation. The individual sales measure was presented in absolute terms together with a graph that indicated the performance of the manager as compared to his peers. Participants in the control group received none of this information.

In order to ensure comprehension of the case, participants had to correctly answer at least half of the case related questions randomly selected from a question bank. Participants who did not meet this criterion were not paid for their participation and were directed to the end of the experiment without influencing the results of the study.

After passing the comprehension test, participants completed the main task of the experiment, they subjectively evaluated the office administration performance. On this page, participants in the Original group were again presented with information about the individual sales performance on the same scale as the subjective evaluation. Participants in the Anchoring group were primed with the “number of employees” anchor during the evaluation.

Participants were asked to answer manipulation check and demographic questions at the end of the experiment.

3.2 Independent variables

The scenario experiment employs an independent groups approach with 9 conditions that are organized into four treatment groups and a control condition. The treatment conditions are divided into four groups each containing a high and low condition for the independent variable:

1. The Distortion Group, the performance on the objective performance measure is presented in an absolute value and before the relevant information regarding the subjective evaluation,

2. The Halo Group, the performance on the objective performance measure is presented in an absolute value and after the relevant information regarding the subjective evaluation, 3. The Anchor Group, an anchor is presented before and during the task,

4. The Original Group, the performance on the objective performance measure is presented in a relative value, before the relevant information and during the subjective

(20)

Neither an anchor nor any information about the objective performance measure is provided in the control condition.

3.2.1 The Distortion Group

The main attributes of this group are:

 An objective performance measure is presented

 The objective performance is measured on a different scale than the subjective performance

 Participants receive information about the level of the objective performance measure prior to reading relevant information regarding the subjective evaluation

The group was designed to test H1 by examining the causal relationship between cognitive distortion and the spillover effect.

Participants in this group were provided with high ($ 476,553) or low ($ 115,758) absolute values for the individual sales measure and a bar graph that plotted the sales performance of the manager against his peers. This information was received before participants read the personal notes and interviews with the members of the office that were designed to indicate a neutral performance in the office administration dimension.

3.2.2 The Halo Group

 The objective performance is measured on a different scale than the subjective performance

 Participants receive information about the level of the objective performance measure after reading relevant information regarding the subjective evaluation

The group was designed to test H2 by examining the causal relationship between the halo effect and the spillover effect.

Participants in this group were provided with high ($ 476,553) or low ($ 115,758) absolute values for the individual sales measure and a bar graph that plotted the sales performance of the manager against his peers. This information was received after participants read the personal notes and interviews with the members of the office that were designed to indicate a neutral performance in the office administration dimension.

(21)

3.2.3 The Anchoring Group

The main attributes of this group are:  An anchor is presented

 The values of the anchor are on the same scale as the subjective evaluation

 Participants were primed with the anchor prior to reading relevant information regarding the subjective evaluation and during the subjective evaluation

The group was designed to test H3 by examining the causal relationship between basic anchoring and the spillover effect.

Participants in this group were informed about the number of subordinates that each manager was responsible for. In the high (low) condition the manager was responsible for 9 (2) employees and attention was not specifically drawn to this value.

The “number of subordinates” measure was chosen as a piece of information that is completely unrelated to the subjective evaluation regarding office administration performance. The measure was designed to influence the subjective evaluation solely through its presence as a number of a 1 to 10 scale and not through any cognitive distortion mechanism: a participant who learns that a manager has a higher number of subordinates is less likely to create a belief about that manager’s general performance.

The number of subordinates is not a perfect anchor due to its potential connection to general competency (eg. a manager that has more subordinates is in this position because he is outperforming his peers). However, a completely unrelated measure (eg. pieces of furniture in the office) would have drawn attention to itself and the result would be less likely to be representative of basic anchoring where attention is not drawn to the anchor.

3.2.4 The Original Group

 The objective performance is measured on the same scale as the subjective performance  Participants receive information about the level of the objective performance measure prior to reading relevant information regarding the subjective evaluation and the same information was presented again during the subjective evaluation

(22)

This group is designed to replicate the original study of Bol and Smith (2011) without considering the influence of controllability. All of the mechanisms presented (cognitive distortion, the halo effect and anchoring) are expected to influence the spillover effect.

The purpose of the group is to observe how the causal effects interact and to examine if there are significant differences in sampling between this thesis and the Bol and Smith (2011) study. Participants in this group were either assigned to the high sales condition, score of 9, or it the low sales condition, score of 2. The following formula was used in order to transform the objective measure from an absolute scale (in dollars) to a relative one: 1 point for each $ 50,000 generated. This information was received before participants read the personal notes and interviews with the members of the office that were designed to indicate a neutral performance in the office administration dimension. The individual sales score was also presented during the subjective evaluation.

Table 2 presents what effect are expected to influence the results of each group.

Table 2

Condition groups and the predicted effects

Groups

Cognitive Distortion Halo Anchoring Original

E ff ec ts Cognitive distortion     Halo effect     Basic anchoring     3.3 Dependent variables

The main dependent variable was the subjective evaluation of the office administration performance for one district manager. Participants also answered manipulation checks and demographic question at the end of the survey.

3.4 Participants

I recruited participants through the Prolific platform which employs workers for an online task and allows researchers to prescreen candidates based on certain filters. Recruiting participants online via a designated platform for employing workers has been proven suitable for conducting accounting experiments. Farrell et al. (2014) found that workers exhibit honesty, even when it is costly for them to do so, and effort characteristics similar to practicing managers.

Participants had to meet specific criteria before participating in the study, namely: participants had to be in a leadership position and to be at least 20 years old. Two additional conditions were

(23)

used in order to further enforce the above mentioned restriction: participants had to be employed and they had to be in charge of at least one subordinate.

The initial participant pool contained 297 participants from which 257 (86.5%) correctly answered more than half of the questions in the attention test and were allowed to complete the experiment. 12 participants were eliminated because they attempted to complete the study after they have failed the attention question. Participants were also selected based an addition attention question: was the dependent variable (individual sales or “number of subordinates”) of the manager high or low as compared to his peers. 228 (93.06%) correctly answered this question. Finally, 21 participants that reported no supervisory experience were not included in the study due to the pre-screening requirements. The remaining participants were paid £ 0.80. The final participant pool was formed out of 207 individuals that reported, on average, a work experience of 15 years, a supervisory experience of 6 years and were performing performance evaluations for a number of 5 years. In their current position, participants supervised an average of 7 employees. 140 (68%) of participants performed performance evaluations in their current position on an average of 7 employees. 115 (56%) were responsible for subjectively evaluating an average of 8 employees.

Participants were 60% male and, on average, 35 years old. 45% of participants lived in the United Kingdom and 32% from the United States of America. The responses came from a total of 17 countries.

3.5 Manipulation checks and Information Sufficiency

Participants felt they had enough information to evaluate the manager. They were asked to rate, on a five-point scale, their agreement with the following statement: “The personal notes and interview responses about the target manager allowed me to make a reasonable judgment about the manager’s office administration performance”. The resulting mean of the measure was 1.76 (s.d. = 0.66), significantly lower than the scale midpoint of 3 (p<0.01).

An analysis was conducted in order to ensure that there were no significant differences between groups for the information sufficiency measure. I tested the independence with a one-way analysis of variance (ANOVA), comparing the “Reasonable Evidence” measures across all 5 groups of conditions. The results (untabulated) show no statistically significant difference among groups as determined by one-way ANOVA (F(4,202) = 1.906, p=0.11).

(24)

Participants were also asked a five-point scale question regarding the confidence in the subjective evaluation. The mean of this measure was 1.89 (s.d. = 0.73), significantly lower than the scale midpoint of 3 (p<0.01).

A five-point scale question was asked in order to ensure that participants did perceive the sales or “number of employees” variables as being independent of the office administration performance.

In the case of the sales performance variable, the mean of this measure was 1.84 (s.d. = 1.16), significantly lower than the scale midpoint of 3 (p<0.01). An additional analysis was conducted only with participants that did not disagree with the independence of the measures.

In the case of the “number of employees” variable, the mean of the measure was 2.83 (s.d. = 1.1), not significantly lower than the scale midpoint of 3 (p=0.24). The respondents that disagreed with the independence (20) were excluded from the hypothesis test in order to ensure the anchoring nature of the “number of employees” variable. After the removal, the new mean of the measure was 2.15 (s.d. = 0.63), significantly lower than the scale midpoint of 3 (p<0.01).

Table 3

Group manipulation checks

Group Independence Confident Reasonable evidence

Anchoring Mean 2.83 1.9 1.76 N 59 59 59 Std. Deviation 1.1 0.82 0.73 Control Mean -- 2.05 1.55 N -- 20 20 Std. Deviation -- 0.83 0.51 Distortion Mean 1.6 1.67 1.6 N 43 43 43 Std. Deviation 0.98 0.52 0.58 Halo Mean 2.25 1.83 1.85 N 40 40 40 Std. Deviation 1.37 0.68 0.77 Original Mean 1.71 2.09 1.91 N 45 45 45 Std. Deviation 1.04 0.73 0.56

(25)

4 Results

Table 4 presents the mean and standard deviation of the subjective evaluation for every condition. Figure 2 shows the means of the four groups and the control conditions.

The mean of the control condition (M=6.1, SD=1.62) is lower than the high distortion condition (M=6.35, SD=2.03), the high halo condition (M=6.65, SD=1.35), the high anchoring condition (M=6.61, SD=0.85) and the high original condition (M=6.48, SD=1.5). The mean of the control condition (M=6.1, SD=1.62) is higher than the low distortion condition (M=4.55, SD=1.23), the low halo condition (M=5.95, SD=1), the low anchoring condition (M=5.33, SD=2.03) and the low original condition (M=5.5, SD=1.54).

Table 5 and Table 6 present the results of independent-samples t-tests between the conditions in each group and against the control condition. Meaningful and statistically significant differences at a 5% level were found between the high and low distortion conditions, t (41)=3.44, p<0.01, the high and low anchoring conditions, t (27.63)=2.63, p=0.01, the high and low original conditions, t (43)=2.16, p=0.04 and between the control and low distortion conditions, t (38)=-3.41, p=0.02. Statistically significant differences at a 10% level were found between the high and low halo conditions, t (38)=1.87, p=0.07.

Table 4

Mean of conditions

Group Mean N Std. Deviation

Control 6.1 20 1.62 DistrortionHigh 6.35 23 2.04 DistortionLow 4.55 20 1.23 HalolHigh 6.65 20 1.35 HaloLow 5.95 20 1 AnchoringHigh 6.61 18 0.85 AnchoringLow 5.33 21 2.03 OriginalHigh 6.48 23 1.5 OriginalLow 5.5 22 1.54

(26)

Figure 2 Means of conditions 6.35 6.1 4.55 4 4.5 5 5.5 6 6.5 7

DistrortionHigh Control DistortionLow

Cognitive Distortion

6.65 6.1 5.95 4 4.5 5 5.5 6 6.5 7

HalolHigh Control HaloLow

Halo effect

6.61 6.1 5.33 4 4.5 5 5.5 6 6.5 7

AnchoringHigh Control AnchoringLow

Basic Anchoring

6.48 6.1 5.5 4 4.5 5 5.5 6 6.5 7

OriginalHigh Control OriginalLow

(27)

Table 5

Mean differences

Groups T Df P

Mean Difference

Original High OriginalLow 2.16 43 0.04 0.98

Original High Control 0.79 41 0.43 0.38

OriginalLow Control -1.23 40 0.23 -0.6 AnchoringHigh AnchoringLow1 2.63 27.63 0.01 1.28 AnchoringHigh Control 1.2 36 0.24 0.51 AnchoringLow Control -1.33 39 0.19 -0.77 DistrortionHigh DistortionLow 3.45 41 <0.01 1.8 DistrortionHigh Control 0.44 41 0.66 0.25 DistortionLow Control -3.41 38 0.02 -1.55 HalolHigh HalolLow 1.87 38 0.07 0.7 HalolHigh Control 1.17 38 0.25 0.55 HalolLow Control -0.35 38 0.73 -0.15 Table 6 Mean differences Group Mean Mean Differences 1 2 3 4 5 6 7 8 9 1.Control 6.10 -- 2.OriginalHigh 6.48 0.38 -- 3.OriginalLow 5.50 -0.6 0.98** -- 4.AnchoringHigh 6.61 0.51 -- 5.AnchoringLow 5.33 -0.77 1.28** -- 6.DistrortionHigh 6.35 0.25 -- 7.DistortionLow 4.55 -1.55** 1.8*** -- 8.HalolHigh 6.65 0.55 -- 9.HalolLow 5.95 -0.15 0.7* -- * p < .1 (two-tailed). ** p < .05 (two-tailed). *** p < .01 (two-tailed).

(28)

4.1 Hypotheses test

4.1.1 Distortion

H1 states that the subjective evaluation will be more favorable (unfavorable) when the supervisor has knowledge about a high (low) level of performance on an unrelated dimension, measured objectively and on a different scale that the subjective evaluation, prior to interpreting relevant information regarding the subjective evaluation. I tested this hypothesis with a one-way analysis of variance (ANOVA), comparing the subjective evaluations from the high and low distortion conditions and the control condition.

The results, presented in Table 10 (Panel A), show that there was a statistically significant difference among groups as determined by one-way ANOVA (F(2,60) = 6.94, p <0.01). When only considering respondent that did not disagree with the independence of the office administration and sales performances, untabulated results find a statistically significant difference among groups (F(2,57) = 5.86, p =0.05).

The results support H1, suggesting that, consistent with cognitive distortion, the subjective evaluation is affected by the knowledge of an unrelated objective performance measure. Table 7 and Figure 3 present the mean differences and their corresponding statistical significance (as calculated by independent-samples t-tests) among the conditions of interest regarding H1.

Table 7

Mean differences Distortion

Mean Differences Group Mean 1 2 1.Control 6.1 -- 2.DistrortionHigh 6.35 0.25 3.DistortionLow 4.55 -1.55** 1.8*** * p < .1 (two-tailed). ** p < .05 (two-tailed). *** p < .01 (two-tailed). Figure 3

Mean differences distortion

6.35 6.1 4.55 4 4.5 5 5.5 6 6.5 7

DistrortionHigh Control DistortionLow

(29)

4.1.2 The Halo Effect

H2 states that the subjective evaluation will be more favorable (unfavorable) when the supervisor observes a high (low) level of performance on an unrelated dimension, measured objectively and on a different scale that the subjective evaluation, after interpreting relevant information regarding the subjective evaluation. I tested this hypothesis with a one-way analysis of variance (ANOVA), comparing the subjective evaluations from the high and low halo conditions. The results, presented in Table 10 (Panel B), show that there the difference among groups as determined by one-way ANOVA (F(1,38) = 3.48, p=0.07) was significant at a 10% level.

The hypothesis is partially supported by the main test but additional investigation that explore the robustness of the conclusion find no statistically significant support for the hypothesis. However, I conclude that the results partially support H2, suggesting that, consistent with the halo effect, the subjective evaluation is affected by the level of an unrelated objective performance measure because that measure is considered indicative of the overall performance. Table 8 and Figure 4 present the mean differences and their corresponding statistical significance (as calculated by independent-samples t-tests) among the conditions of interest regarding H2.

Table 8

Mean differences halo effect

Mean Differences Group Mean 1 2 1.Control 6.1 -- 2 HalolHigh 6.65 0.55 3 HalolLow 5.95 -0.15 0.7* * p < .1 (two-tailed). ** p < .05 (two-tailed). *** p < .01 (two-tailed). Figure 4

Mean differences halo effect

4.1.3 Basic Anchoring

H3 states that subjective evaluation will be higher (lower) when the supervisor is presented with a high (low) anchor, measured on the same scale as the subjective evaluation. I tested this hypothesis with a one-way analysis of variance (ANOVA), comparing the subjective evaluations from the high and low anchoring conditions and the control condition.

6.65 6.1 _5.95 4 4.5 5 5.5 6 6.5 7

HalolHigh Control HaloLow

(30)

The Levene’s F test revealed that the homogeneity of variance assumption was not met (p < 0.01). As such, the Welch’s F test was used. The results, presented in Table 10 (Panel C), show that there was a statistically significant difference among groups as determined by one-way ANOVA (F(2,34.38) = 3.64, p=0.04).

The results support H3, suggesting that, through basic anchoring, the subjective evaluation is affected by the presence of a number that is completely unrelated to the judgement being made. Table 9 and Figure 5 present the mean differences and their corresponding statistical significance (as calculated by independent-samples t-tests) among the conditions of interest regarding H3.

Table 9

Mean differences basic anchoring

Mean Differences Group Mean 1 2 1.Control 6.1 -- 2.AnchoringHigh 6.61 0.51 3.AnchoringLow 5.33 -0.77 1.28** * p < .1 (two-tailed). ** p < .05 (two-tailed). *** p < .01 (two-tailed). Figure 5

Mean differences basic anchoring

Table 10

ANOVA result, test for H1, H2 and H3

Panel A: Analysis of Variance for the Distortion Group

Source df Mean Square F-statistic p-value

Sales level 2 19.65 6.94 <0.01

Panel B: Analysis of Variance for the Halo Group

Sales level 1 4.9 3.48 0.07

Panel C: Analysis of Variance for the Anchoring Group

Number of employees 2 8.12 3.64 0.04 6.61 6.1 5.33 4 4.5 5 5.5 6 6.5 7

AnchoringHigh Control AnchoringLow

(31)

4.1.4 Original Design

Prior literature (Bol and Smith 2011) suggests that the subjective evaluation will be higher (lower) when the supervisor knows about a higher (lower) level of performance on an unrelated objective dimension, measured on the same scale as the subjective evaluation. I retested this hypothesis with a one-way analysis of variance (ANOVA), comparing the subjective evaluations from the high and low original conditions. The results, presented in Table 12 (Panel A), show that there is a statistically significant difference between groups (F(1,43) = 4.66, p=0.04). If the one-way ANOVA test compares the high and low original conditions and the control condition (Table 12, Panel B), no statistically significant difference among groups can be found (F(2,62) = 2.27, p=0.11)

I retests the effect considering only considering respondents that did not disagree with the independence of the office administration and sales performances. Results from the one-way ANOVA, comparing the subjective evaluations from the high and low original conditions (Table 12, Panel C), reveal no statistically significant difference between groups (F(1,40) = 2.71, p =0.11).

In sum, this hypothesis is supported by the main test but additional investigation that explore the robustness of the conclusion find no statistically significant support for the hypothesis. However, I conclude that under certain conditions, no restriction on the independence between performance dimensions, the hypothesis is supported. Table 11 and Figure 6 present the mean differences and their corresponding statistical significance (as calculated by independent-samples t-tests) among the conditions of interest regarding the replicated experiment.

Table 11

Mean differences original

Mean Differences Group Mea n 1 2 1.Control 6.1 _-- 2.OriginalHigh 6.48 _0.38 _-- 3.OriginalLow 5.5 _-0.6 _0.98** * p < .1 (two-tailed). ** p < .05 (two-tailed). *** p < .01 (two-tailed). Figure 6

Mean differences original

6.48 6.1 5.5 4 4.5 5 5.5 6 6.5 7

OriginalHigh Control OriginalLow

(32)

Table 12

ANOVA results Original group

Panel A: Analysis of Variance for the Original Group, excluding Control Group

Sales level 1 10.76 4.66 0.04

Panel B: Analysis of Variance for the Original Group, including Control Group

Sales level 2 5.45 2.27 0.11

Panel C: Analysis of Variance for the Original Group, including Control Group and independence requirement

(33)

5 Conclusion

This paper examined which psychological concepts explain the spillover effect of unrelated information into subjective evaluations. I investigated three possible causes for the spillover effect: cognitive distortion, the halo effect and basic anchoring. The results indicate that cognitive distortion and basic anchoring can cause the incorporation of unrelated information into subjective evaluations. Partial support was found for the hypothesis that the halo effect produces a spillover effect.

The study emphasizes the complex reality behind the spillover effect of unrelated information into subjective evaluations. The answer provided here is that cognitive distortion, the halo effect and basic anchoring can cause the spillover effect. However, this explanation is not meant to be exhaustive, rather it aims to reinforce the idea that subjective evaluations are naturally affected by biases and heuristics in a complex manner.

An implicit assumption of this study is that psychological mechanisms can be examined individually and, when recombined, their effects on the final decision will collate. The results from the replication of the Bol and Smith (2011) experiment prove that this assumption is partially false.

Both cognitive distortion and basic anchoring are presumed to have an effect on the results of the replicated group. However, the influence of the combined effects is weaker and less significant than when each effect is examined individually.

One explanation may be that biases do not interact in a straightforward manner. In other words, the effects of biases do not simply stack up, rather they combine in unexpected patterns. It may also be the case that biases can only account for a limited portion of the final decision.

Another possibility concerns only the design of this specific experiment. In the conditions that replicated the Bol and Smith (2011) experiment, participants were asked to average the results of the two performance measures after they have concluded the subjective judgement. This was done in order to ensure that participants observe the performance on the separate dimension. Evaluators may have decided on scores that made the result of the average an integer.

The above mentioned effect would diminish the influence of the biases because participants would more reluctant to deviate from a number that made the average calculation easier. This hypothesis is supported in part by the parity of the subjective evaluation (0.35 for the high

(34)

condition and 0.68 for the low condition). However, the parity of both conditions was not statistically significantly different that the expected value of 0.52.

The research question of the thesis: “which psychological concepts explain the spillover effect of unrelated information into subjective evaluations” is important for the academic literature because the proposed explanation for the effect (cognitive distortion) was not fully fitted to explain variances in the empirical findings of other studies3 (Merchant et al. 2010; Heneman 1986). By further examining additional causes behind the spillover effect (basic anchoring and the halo effect), the study helps better understand what circumstances are susceptible to the spillover effect.

The thesis also answers a call for future research made in the article of Bol and Smith (2011). The authors suggested that the timing of the biasing information may not limit the spillover effect. I investigate this possibility by altering the presentation order of the sales performance. The results partially support a less significant spillover effect when the sales level is presented after the relevant information regarding the subjective evaluation.

The results of this study are also relevant to practice. By better understanding the causes behind the spillover effect, designers of performance evaluation systems are better equipped to prevent it. First, the results show that the spillover effect is only diminished if the evaluator receives relevant information before the information about the performance on a different dimension. Second, designers of performance evaluation systems should consider the effect of basic anchoring on subjective evaluations. The influence of basic anchoring can be partially avoided by giving more consideration to the anchors (numbers or words) that are presented to the evaluators when they make the subjective evaluation.

It is important to note that simply informing evaluators about the independence of performance dimension does not completely remove the spillover effect. In the current experiment, participants were expressly informed about the independence between the subjective and objective performance measures but the spillover effect was still present.

2_{In the case of the high condition, the parity mean was lower by 0.15 (95% CI, -0.36 to 0.06) than the normal parity} of 0.5, t(22) = -1.45, p = 0.15. In the case of the low condition, the parity mean was higher by 0.18 (95% CI, -0.03 to 0.39) than the normal parity of 0.5, t(21) = 1.79, p = 0.08

3_{Assuming one accepts that the correlation between objective and subjective performance measures can be labeled}

(35)

To the extent that the circumstances allow it, companies can create a setting in which evaluators subjectively judge performance before observing the level of objective performance on other dimensions.

The study has limitation regarding internal validity, namely the operationalization of basic anchoring. It may be reasoned that the anchor, the “number of employees” measure, was not completely unrelated to the judgement being made, the subjective evaluation of office administration. Indeed, participants did not fully agree4 with the independence between office administration performance and the “number of employees” measure. Although the limitation was acknowledged and diminished through manipulation check, doubts can still be raised regarding whether basic anchoring was truly the responsible effect in that group of conditions. A number of limitation regarding external validity can also be raised. In the experiment participants had to evaluate a single employee. The findings of the thesis may not be transferable to a situation in which managers evaluate multiple employees due to the comparative element introduced. Another concern is the limited information to which evaluators where exposed. In practice, supervisors are presented with complex and conflicting pieces of information (past performance and evaluations from different sources) regarding an employee’s performance. The limitations of the study present opportunities for future research. First, the effect of anchoring on subjective evaluation should be more thoroughly examined. To my knowledge, no studies have been published on the effects of anchoring on the design of performance evaluation systems. Second, the spillover effect should be studied in a group setting in which the subjective evaluation is also affected by concerns of fairness and motivation. Third, the nature of the objective performance that caused the spillover effect should be altered. While Bol and Smith (2011) examine the influence of the controllability of the measure, other characteristics can be altered. For example, future research can examine the spillover effect when the objective performance measure exhibits varying degrees of noisiness, congruence or sensitivity.

(36)

References

Baiman, S., & Rajan, M. V. (1995). The informational advantages of discretionary bonus schemes. Accounting Review, 557-579.

Baker, G., Gibbons, R., & Murphy, K. J. (1993).Subjective performance measures in optimal incentive

contracts (No. w4480). National Bureau of Economic Research.

Banaji, M. R., & Greenwald, A. G. (2013).Blindspot: Hidden biases of good people. Delacorte Press. Bol, J. C. (2008, December). Subjectivity in compensation contracting. In AAA Management

Accounting Section (MAS) 2006 Meeting Paper.

Bol, J. C., & Smith, S. D. (2011). Spillover effects in subjective performance evaluation: Bias and the asymmetric influence of controllability. The Accounting Review, 86(4), 1213-1230.

Bol, J. C., Keune, T. M., Matsumura, E. M., & Shin, J. Y. (2010). Supervisor discretion in target

setting: An empirical investigation.The Accounting Review, 85(6),1861-1886.

Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S. B. (1995). On the interchangeability of objective and subjective measures of employee performance: A meta‐ analysis. Personnel Psychology, 48(3), 587-605.

Bond, S. D., Carlson, K. A., Meloy, M. G., Russo, J. E., & Tanner, R. J. (2007). Information distortion

in the evaluation of a single option.Organizational Behavior and Human Decision Processes,102(2),

240-254.

Carlson, K. A., Meloy, M. G., & Russo, J. E. (2006). Leader-driven primacy: Using attribute order to affect consumer choice. Journal of Consumer Research, 32(4), 513-518.

Chapman, G. B., & Johnson, E. J. (1994). The limits of anchoring. Journal of Behavioral Decision

Making, 7(4), 223-242.

Critcher, C. R., & Gilovich, T. (2008). Incidental environmental anchors. Journal of Behavioral

Decision Making, 21(3), 241-251.

Eagly, A. H., Ashmore, R. D., Makhijani, M. G., & Longo, L. C. (1991). What is beautiful is good, but…: A meta-analytic review of research on the physical attractiveness stereotype. Psychological

bulletin, 110(1), 109.

Englich, B., Mussweiler, T., & Strack, F. (2005). The last word in court--a hidden disadvantage for the defense. Law and Human Behavior, 29(6), 705.

Englich, B., Mussweiler, T., & Strack, F. (2006). Playing dice with criminal sentences: The influence of irrelevant anchors on experts’ judicial decision making. Personality and Social Psychology

Bulletin, 32(2), 188-200.

Epley, N., & Gilovich, T. (2006). The anchoring-and-adjustment heuristic Why the adjustments are insufficient. Psychological science, 17(4), 311-318.

(37)

Farrell, A. M., Grenier, J. H., & Leiby, J. (2014). Scoundrels or stars? Theory and evidence on the quality of workers in online labor markets. Theory and Evidence on the Quality of Workers in Online

Labor Markets (October 29, 2014).

Feltham, G. A., & Xie, J. (1994). Performance measure congruity and diversity in multi-task principal/agent relations. Accounting review, 429-453.

Furnham, A., & Boo, H. C. (2011). A literature review of the anchoring effect. The Journal of

Socio-Economics, 40(1), 35-42.

Gibbs, M., Merchant, K. A., Stede, W. A. V. D., & Vargus, M. E. (2004). Determinants and effects of subjectivity in incentives. The Accounting Review,79(2), 409-436.

Hayes, R. M., & Schaefer, S. (2000). Implicit contracts and the explanatory power of top executive compensation for future performance.The RAND Journal of Economics, 273-293.

Heneman, R. L. (1986). The relationship between supervisory ratings and results‐oriented measures of performance: A meta‐analysis. Personnel Psychology, 39(4), 811-826.

Ittner, C. D., Larcker, D. F., & Meyer, M. W. (2003). Subjectivity and the weighting of performance measures: Evidence from a balanced scorecard. The Accounting Review, 78(3), 725-758.

Kahneman, D. (2011). Thinking, fast and slow. Macmillan.

Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard: translating strategy into action. Harvard Business Press.

MacLeod, W. B. (2001). On optimal contracting with subjective evaluation. USC CLEO Research

Paper, (C01-8), 01-11.

Merchant, K. A., Theivananthampillai, P., & Stringer, C. (2010). Relationships between objective and subjective performance ratings.

Mumma, G. H., & Wilson, S. B. (1995). Procedural debiasing of primacy/anchoring effects in clinical‐ like judgments. Journal of clinical psychology,51(6), 841-853.

Murphy, K. R., & Cleveland, J. N. (1991).Performance appraisal: An organizational perspective. Allyn & Bacon.

Mussweiler, T., & Strack, F. (2000). The use of category and exemplar knowledge in the solution of anchoring tasks. Journal of personality and social psychology, 78(6), 1038.

O'Donnell, E., & Schultz Jr, J. J. (2005). The halo effect in business risk audits: Can strategic risk assessment bias auditor judgment about accounting details?. The Accounting Review,80(3), 921-939. Sundar, A., & Kardes, F. R. (2015). The role of perceived variability and the health halo effect in nutritional inference and consumption. Psychology & Marketing, 32(5), 512-521.

(38)

Van Der Heijden, H. (2013). Evaluating dual performance measures on information dashboards: Effects of anchoring and presentation format.Journal of Information Systems, 27(2), 21-34.

Wilson, T. D., Houston, C. E., Etling, K. M., & Brekke, N. (1996). A new look at anchoring effects: basic anchoring and its antecedents.Journal of Experimental Psychology: General,125(4), 387. Wong, Kin Fai Ellick, and Jessica Yuk Yee Kwong. "Is 7300 m equal to 7.3 km? Same semantics but different anchoring effects."Organizational Behavior and Human Decision Processes 82.2 (2000): 314-333.

Unpacking the spillover effect : the investigation of cognitive distortion, the halo effect and basic anchoring as possible causes

Table of Contents

2 Literature

The spillover effect

Bol and Smith (2011) Study

Anchoring

Halo effect

3 Method

4 Results

Cognitive Distortion

Halo effect

Basic Anchoring

5 Conclusion

References