Lying or trying
On the effects of dishonesty on incentivizing student performance
Vincent Horstink 6078516
UvA, Faculty of Economics and Business
Master’s thesis Behavioural Economics and Game Theory (15 ECTS)
2
Statement of Originality
This document is written by Vincent Horstink, who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and no sources other than those mentioned in the text and its references have been used in creating it.
The faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
3 Abstract
One way to enhance motivation in high school students is through the use of extrinsic incentives. The general conclusion from the literature is that, in the context of education, such extrinsic incentives have a small but positive effect on performance. This literature however strongly hinges on the assumption that incentives induce effort exertion and thereby performance. It does however not consider the possibility that there are other means with which to achieve the incentive goal, such as dishonesty. This thesis theoretically and experimentally investigates the consequences of such alternative means on the effectiveness of extrinsic incentives. A field experiment was set up in which students were offered rewards for performance. The experiment had a 2x2 design, varying in the possibility to be dishonest and target size. Dishonesty was observed zero times. It was ultimately concluded that the current experimental set-up did not suffice to answer the question it set out to.
Vincent Horstink1
Amsterdam, July 26, 2015
1 Special thanks to: dr. Joël van der Weele for his supervision, inspiration, numerous replied e-mails and provision of large quantities of envelopes; Gijs Heuff for his support, cooperation and ideas with regard to the experiment; fellow homework tutors for their help in gathering the returned envelopes; prof. Joep Sonnemans for introducing the possibility of a mini-presentation for CREED and permission on behalf of the ethics committee; and Myrte Schrage, Leonard Treuren and Melin Walet for their support, suggestions and reading time.
4
Table of contents
Abstract 3 Table of contents 4 List of tables 6 List of figures 71.
Introduction 82.
Literature 112.1 Effort and motivation 11
2.1.1 Extrinsic incentives 11
2.1.2 Extrinsic incentives in education 12
2.1.3 Target size 13 2.2 Dishonesty 14 2.2.1 Origins of dishonesty 14 2.2.2 Research aim 16
3.
Model 18 3.1 Extrinsic incentives 18 3.1.1 Target size 20 3.2 Dishonesty 22 3.2.1 Target size 25 3.3 Comparison 28 3.3.1 Heterogeneity 29 3.3.2 Conclusion 294.
Experimental methodology 31 4.1 Participants 315 4.3 Procedure 34 4.4 Measurements 35 4.5 Hypotheses 36
5.
Results 39 5.1 Descriptive statistics 39 5.2 Analysis 40 5.3 Non-hypothesized findings 44 5.3.1 Explanatory analysis 445.3.2 Large sample analysis 47
6.
Discussion 49 6.1 Effort provision 50 6.2 Dishonesty 52 6.3 Sample size 537.
Conclusion 55 References 58 Appendices 616
List of tables
Table 1: Experimental treatments, 2x2 design ……….. 33 Table 2: Descriptive results by treatment ……….. 39 Table 3: Average Baseline Grades, Actual Grades and Grade Differences, by treatment …... 40 Table 4: Wilcoxon Signed-Rank tests of grade-differences, by treatment ………... 41 Table 5: Self-reports of effort exertion ……….. 45 Table 6: Wilcoxon Signed-Rank tests of grade-differences, by course-type ……… … 46
7
List of figures
8
1. Introduction
In a report recently published by the inspection of education in the Netherlands, the authors state that the motivation of high-school students in the Netherlands in particular, is lacking (the Netherlands. Inspectie van het Onderwijs, 2015). Indeed, high-school students are not known for being highly motivated towards either performance or attendance (Gneezy, Meier and Rey-Biel, 2011). However, given that a lack of motivation may lead to low achievement (Gottfried, 1985), and that a subsequent lack of proper education may have both negative social and individual consequences (Rumberger, 1987), properly motivating high-school students is an important policy matter that has to be considered with care. Consequently, it does not come as a surprise that the amount of literature on this subject is large (see e.g. Deci, Vallerand, Pelletier and Ryan, 1991).
One option used to induce motivation in high school students is offsetting the immediate costs of effort with immediate benefits through external incentives (Lavecchia, Liu and Oreopoulos, 2014). Though the evidence on the effects of extrinsic incentives in earlier research seemed to point towards it working adversely (Deci, Koestner and Ryan, 1999), in more recent literature the overall effect within the context of education seems to be small but positive, at least in the short term (Gneezy et al., 2011; Lavecchia, Liu and Oreopoulos, 2014). What is more, when incentives are conditional on achieving targets, performance is believed to relate positively to the size of such targets through effort provision (Miller and Weiss, 2015).
The effectiveness of extrinsic incentives however need not be that straightforward. When giving students extrinsic incentives for academic achievement and therefore basing it on performance, additional problems may arise when it becomes hard or impossible to verify a student’s work. If a parent promises a child a reward for attaining a certain grade on an exam, that child may lie about having attained that grade in order to get the reward. As a result, extrinsic incentives may not work as expected in enhancing student motivation.
As both effort and dishonesty come with costs, one may substitute effort for dishonesty if the net-benefits of being dishonest are higher than those of effort. Therefore, under the possibility of dishonesty the assumed positive relationship between target size and performance additionally has to be called to question, as target size relates positively to the benefits of being dishonest through averting higher necessary effort costs.
9
Costs of dishonesty are themselves apparent even in the absence of detection and punishment, and may stem from an intrinsic aversion to the act of dishonesty (Erat and Gneezy, 2012). Moreover, such costs are heterogeneous not only between but also within people given that they depend on the size of the lie itself (Gibson, Tanner and Wagner, 2013). Consequently, the tendency towards dishonesty may be highly conditional on context, as people balance unconditional honesty with a cost-benefit analysis of being dishonest to decide whether a certain lie is worth to be told (Rosenbaum, Billinger and Stieglitz, 2014).
As a result, target size may also directly affect intrinsic costs to dishonesty and thereby further affect the relationship between target size and performance. A higher target may signal unrealistic or unreasonable expectations (Miller and Weiss, 2015), and may thereby lower the intrinsic costs to dishonesty. Consequently, this may further deteriorate the positive relationship between target size and performance when dishonesty is possible.
If dishonesty indeed diminishes the effectiveness of extrinsic motivation, this is something that should be considered by principals in educational and professional settings alike. In educational settings, parents reward their children for achievements that they cannot verify. Knowing this, children may be tempted to lie rather than try. In a professional context companies similarly pay employees working from home for their performance while not always being able to check their work. Employees know this and might be tempted to shirk rather than work. Higher targets may enhance this effect.
The current literature does not address such situations where effort is not necessary to achieve the goals on which rewards are conditional. However, in educational settings, disregarding these situations may cause attempts to motivate students to be ineffective. Therefore, this thesis experimentally investigated the question: “Do higher targets lead to more effort or rather to more dishonesty?” To that extent, two sub-questions were further defined. First: “If lying is not possible, do higher targets lead to more effort?” Second: “If lying is possible, how does this affect the effectiveness of the aforementioned targets?”
A lab-in-field experiment was set up at a homework institute in Amsterdam, in which students were presented with an external incentive for achieving a target grade on a regular school test. The experiment had a 2x2 design which varied in target size and the possibility to be dishonest. Variation in target size made it possible to see whether target size indeed positively affected performance in the absence of dishonesty. Additional variation in the ability to be
10
dishonesty made it possible to show in what way this process was affected by the possibility to be dishonest. A possibility of dishonesty was presented by letting students self-report their grades, and was identified by comparison of such self-reported grades to those actually attained.
In the end it was shown that there was insufficient evidence that the incentives led to higher performance, irrespective of whether dishonesty was possible or not. As none of the students lied, and no differences in effort exertion were observed between treatments with and without the possibility to be dishonest, it most likely did not affect behavior in a significant way. Moreover, higher targets, both in situations with and without dishonesty, did not lead to higher performance in situations. It is ultimately concluded that these findings most likely resulted from an imperfect experimental methodology and the fact that the sample size was undesirably small.
In Chapter 2 of this thesis an extensive overview of the relevant literature will be presented. Chapter 3 outlines a simple theoretical model concerning incentives and dishonesty. Chapter 4 consists of a detailed description of the research methodology that was used to set up and perform the experiment. In addition, it includes the experimental hypotheses. In Chapter 5, the results of the experiment will be presented, along with part of its interpretation. In Chapter 6 any possible discussion points will be considered while in Chapter 7, the last Chapter, a conclusion will be formed with regard to the research question.
11
2. Literature
2.1 Effort and motivation
A major assumption in the literature on incentives is that reaching goals requires effort. More importantly, that effort requires motivation: one needs to be moved towards reaching a goal in order to put in the effort necessary to achieve it (Gneezy et al., 2011). While devoting time and effort to tedious tasks is a burden for most, an unwillingness to do so leads to an especially large problem for students. Gottfried (1985) for instance found that intrinsic motivation correlated strongly with academic achievement in younger students. According to Gneezy, Meier and Rey-Biel (2011), high-school students tend to discount the future more than most and may underestimate the returns on education. Similarly, Lavecchia, Liu and Oreopoulos (2014) name the fact that students focus too much on the present one of four major behavioral barriers they encounter. Given a highly positive - though hard to measure - value of education, and the fact that a lack of it may not only cause individual but also social problems (Rumberger, 1987), enhancing student motivation seems adamant.
One way to enhance motivation in students is through extrinsic incentives. Imposing an incentive for achievement of a certain goal may extrinsically enhance motivation, which is believed to lead to higher performance through effort (Deci, Koestner and Ryan, 2001). As mentioned, this assumption may however not always hold as there might be other means by which to reach the set goal, such as dishonesty. Disregarding the possibility that there are other means by which to achieve a set goal may have large consequences for the theorized effectiveness of extrinsic incentives, thereby inferring a need for its reconsideration in the literature. To understand how alternative means of attaining incentives may affect the workings of extrinsic incentives, we however first review the current state of the literature on extrinsic incentives and its general conclusions, following the current assumptions.
2.1.1 Extrinsic incentives
Though the idea of using extrinsic incentives stems from economics, views on the effectiveness of extrinsic incentives are nowadays mostly based on the psychological notion that motivation can roughly be divided into being either ex- or intrinsic along the lines of Deci and Ryan’s
12
(1985) Self-Determination theory. Intrinsic motivation is defined as being driven to do something for its inherent pleasure, while extrinsic motivation concerns the motivational drive related to achieving a separable goal or outcome (Ryan and Deci, 2000). Historically, under the assumption that motivation was solely extrinsic, economists assumed that incentives had purely positive consequences through the price-effect and therefore were a viable method in enforcing desired behavior (Gneezy et al., 2011). Some time ago however, this topic came up for debate as different strands of literature from psychology pointed at the negative effect extrinsic incentives may have on intrinsic motivation (Frey and Jegen, 2000). In a review, Frey and Jegen (2000) indeed found convincing empirical evidence for what is now most often called the ‘crowding-out effect’ by economists and psychologists alike. Deci, Koestner and Ryan (1999) find similar results in a large meta-analysis for when the incentives are rewards.
Though it need not necessarily be the case, crowding-out effects may negatively affect the way incentives work in two ways. In some cases, though the price-effect may be larger, a crowding-out of intrinsic motivation may lead to lower motivation in the long term when the extrinsic incentive is ultimately abolished (Deci and Ryan, 1985). In other cases crowding-out effects may overrule the inherent price-effect of extrinsic incentives, thereby lowering overall motivation even in the short term (Gneezy and Rustichini, 2000).
According to Gneezy et al. (2011), if and how crowding-out plays a role, and whether extrinsic incentives consequently have the desired effect however mainly depends on the way and form they are specified. An example of misspecification is presented in Gneezy and Rustichini (2000), who find that giving too low rewards has an adverse effect on performance.
2.1.2 Extrinsic incentives in education
In education, an effectively implemented extrinsic incentive may give students the short-term benefit they need to outweigh the immediate cost of effort. Some scholars however argue that, rather than using extrinsic incentives to motivate students, the duty of educators should be to enhance students’ intrinsic motivation (Gneezy et al., 2011). One often advocated option to do so is to enhance motivation intrinsically through inspiring teaching (Ford and Roby, 2013). Despite the fact that some thus strongly disagree with extrinsic incentivizing in education on grounds of both morality and efficiency (see e.g. Kohn, 1999), the evidence on the latter is not so straightforward.
13
In a meta-analysis aimed specifically at research concerning crowding-out due to the use of rewards in education, Deci, Koestner and Ryan (2001) find that there indeed exists an overall crowding-out of intrinsic motivation, thereby rejecting earlier contra-dictionary evidence on grounds of it being flawed (see also Cameron and Pierce, 1996). The authors however do not further investigate to what extent this crowding-out effect is outweighed by a possible price-effect. In Gneezy et al. (2011), the authors stipulate that though it is not always cost-effective, extrinsic incentives may overall have a moderate, positive effect on motivation and thereby performance.2 A further distinction with respect to effectiveness is based on the incentivized goal; incentives work well for increasing enrollment and attendance, but tend to have mixed results regarding academic achievement (Gneezy et al., 2011).3 In conclusion, though crowding-out is apparent, the effect of extrinsic incentives in the context of education seems to be small but positive.
2.1.3 Target size
One important factor that is further believed to affect the effectiveness of intrinsic incentives in educational and non-educational contexts alike is target size. Following the set assumption on the effects of motivation on performance through effort, a higher target is expected to lead to more motivation, therefore also to more effort and consequently to higher performance (Locke and Latham, 2002). Indeed, in Miller and Weiss (2015) the authors provide an overview on the key findings of the extensive goal-setting literature, and find that there in general is a “direct positive relationship between goal difficulty and performance” (p. 15). Similar results are found by long-term goal-setting researchers Locke and Latham (2002), in a review of 35 years of research. Moreover, the relationship between target size and performance is believed to be linear so long as the goal does not surpass a person’s ability, or a person stops committing to a high difficulty goal (Locke and Latham, 2002; Miller and Weiss, 2015). Erez and Zidon (1984) discuss this non-linearity in the relationship between target size and performance, and conclude that it is driven by what they call ‘goal rejection’. As long as goals are accepted, target size is positively and linearly related to performance. However, when goals are too high and are therefore rejected, this relationship becomes negative.
2
Similar results are found by Lavecchia, Liu and Oreopoulos (2014).
3 Deci, Koestner and Ryan (1999) specifically mention that especially performance-contingent rewards have high probability of crowding out intrinsic motivation.
14 2.2 Dishonesty
All of the above is however based on the notion that more motivation leads to more effort and thereby to higher performance (Gneezy et al., 2011). Under that assumption the only question we had to ask ourselves was in what way we can most effectively enhance motivation, taking into account possible crowding-out effects. This is what has extensively been done in past literature on incentives in both educational and non-educational settings alike. Moreover, the literature on goal setting showed that higher targets might function as such (Miller and Weiss, 2015; Locke and Latham, 2002).
There are however scenarios that can be envisioned in which effort is not a necessity as there are other means with which to achieve the goal, such as dishonesty in cases where it is hard or impossible to verify a set goal. Within the context of education, these situations frequently occur when high school students self-report their grades or homework completion to parents.
When facing these situations we must thus additionally question in what way the possibility of being dishonest affects the perceived effectiveness of extrinsic incentives. The current literature fails to address this problem, even though it is apparent in everyday life and may have large, unconsidered consequences for the effectiveness of extrinsic incentives. Particularly in educational settings, proper consideration of the - possibly adverse - effects on extrinsic incentives is highly important. This thesis therefore considers just that. To understand how dishonesty can affect the effectiveness of extrinsic incentivizing, we must however first determine what affects a person’s tendency towards dishonesty rather than truthfulness. Therefore we review the current state of the literature on dishonesty.
2.2.1 Origins of dishonesty
In the standard economics framework, dishonesty is assumed to affect people purely like it would a standard homo economicus: people are dishonest more frequently when the benefits are higher, but less so when the cost or probability of being caught increases. In addition, it is assumed that dishonesty does not carry any implicit costs (Gneezy, Rockenbach and Serra-Garcia, 2013; Rosenbaum et al., 2014). Experimental research has however shown that behavior is generally inconsistent with these predictions. People are often found to be lying averse, even in the absence of detection or punishment (Gneezy et al., 2013; Fischbacher and Föllmi-Heusi,
15
2013). Therefore it is shown that people experience some inherent cost or disutility to lying, the origins of which are widespread. In Dufwenberg and Gneezy (2000) the authors address the possibility that people feel guilty for lying due to it hurting others, while Vanberg (2008) states that people rather just have a preference for keeping promises. Erat and Gneezy (2012) corroborate the latter view by showing that lying aversion is apparent also when lies help oneself without hurting the other.
The fact that lies induce intrinsic costs without hurting another may stem from peoples’ internalizing of social norms (Abeler, Becker and Falk, 2012). In Mazar et al. (2008) the authors argue that these internalized social norms work through the internal reward system and thereby influence people’s self-concept - which the authors define as “the way people view and perceive themselves” (Mazar et al, 2008, p. 634). In order to avoid negative adaptation of this self-concept, people avoid breaking internal rules by being honest. According to Fischbacher and Föllmi-Heusi (2013), this internalization may lead to behavior that is comparable to situations where social preferences are apparent.
Despite the fact that we may assume people to generally be lying averse, people at times still tend to be dishonest. Earlier evidence had pointed towards a possible type-based model to explain this behavior. In such a model two types exist: the economic type, who weighs cost against benefits much like a homo economicus, and the ethical type, who is always truthful (Gneezy, 2005; Gibson, Tanner and Wagner, 2013). In a large meta-analysis by Rosenbaum et al. (2014) the authors however mention that the literature points at people’s tendency towards dishonesty to be highly conditional. That is to say, people are heterogeneous in their preference for truthfulness rather than being a fixed type (Gibson, Tanner and Wagner, 2013). Moreover, this heterogeneity may be the result of a within-person balancing of preference for absolute truthfulness on the one hand, and a cost-benefit analysis of dishonesty on the other (Rosenbaum et al., 2014; Gibson et al., 2013). Thereby, heterogeneity in truthfulness exists not only between, but also within people. Along these lines some scholars now argue for people’s tendency to lie incompletely by avoiding major and minor lies, thereby offering evidence in favor of the heterogeneity approach (Shalvi et al., 2011; Fischbacher and Föllmi-Heusi, 2013).
A context of education does not lead us to assume significant differences in behavior with regard to dishonesty. Though at lower ages it may be the case that some children have not fully internalized social norms surrounding dishonesty (Bucciol and Piovesan, 2010) or cannot fully
16
distinguish reality from fiction (Piazza, Bering and Ingram, 2011), ages at which children start high-school are on the end of the scale of self-control development (Bucciol and Piovesan, 2010). Therefore we may assume no significant differences in the tendency towards dishonesty between students in high school and college, the latter of which being the reference group as most research in dishonesty is done among student populations (Rosenbaum et al., 2014).
2.2.2 Research aim
In short, the current literature on extrinsic incentives shows that the evidence on its effectiveness is mixed, though less so in the context of education where the effect is generally small but positive (Lavecchia, Liu and Oreopoulos, 2014). Crowding-out effects, while apparent, do not always outweigh the price-effect stemming from the imposed incentive. In those cases that they do not, extrinsic incentives are believed to be effective motivators that, through target size, relate positively to performance.
As this all relies heavily on the necessity of effort provision to achieve goals on which incentives are conditional, it calls to question in what way alternative means of achieving incentive goals may affect the incentives’ effectiveness. As shown, dishonesty carries costs even in the absence of detection and punishment, which implies a need for consideration of the relationship between such costs and those of effort. Moreover, as dishonesty is most likely a highly conditional act, specification of the correct target size of extrinsic incentives may be important in its effective implementation.
This thesis therefore experimentally and theoretically investigates in what way dishonesty affects the effectiveness of extrinsic incentives in education under different targets. As effective implementation of incentives is an important policy question in both educational and professional contexts, this thesis provides an important insight. It adds to literature in the sense that it investigates an area of extrinsic incentives in education on which no research currently exists by questioning one of its main assumptions. Moreover, it calls to question the perceived effectiveness of higher targets and thereby adds to the goal-setting literature. By also considering the effects of incentives in the absence of dishonesty, it additionally attempts to replicate earlier findings in that literature. Lastly, this thesis adds to the literature on dishonesty by providing a field-example of its occurrence.
17
The next chapter outlines a model regarding the effects of extrinsic incentives with and without the possibility to be dishonest. Subsequently, the methodology of an accompanying experiment will be described.
18
3. Model
As no literature currently exists on the effects of dishonesty when implementing extrinsic incentives, hypotheses cannot fully be formed on the basis of prior research. Therefore, a mathematical model is presented out of which expectations may be formed. In the first part of the model, section 3.1, we form expectations solely with regard to the effects of extrinsic incentives under high and low targets by looking at the population shares that exert effort. In the second part of the model, section 3.2, dishonesty is introduced and the relevant differences in expectations are discussed. Firstly, we consider the effects of dishonesty on the tendency to exert effort. Secondly, we discuss how target size affects the tendency to be dishonest. Lastly, we consider how targets affect the tendency to exert effort under the possibility of dishonest. The last part of the model, section 3.3, describes the difference-in-difference effect of both the imposition of a high target and the possibility to be dishonest and discusses the assumptions that were made regarding heterogeneity.
3.1 Extrinsic incentives
Consider a simple model in which an agent (A) needs to make decisions with regard to the attainment of a certain high-performance target. As mentioned, in the literature on extrinsic incentives it is assumed that incentives lead to higher performance through effort provision (Gneezy, Meier and Rey-Biel, 2011). Though there may be alternative means by which to achieve the target, such as dishonesty, this possibility will be considered from section 3.2 onwards. For now, the agent must input effort to achieve the target, which is costly at e. Effort is assumed to be a binary decision. Though evidence has long shown that people are not in fact selfish, rational homo economici (Yamagishi et al., 2014), we assume people to seek personal gain to some extent. Moreover, we may assume people to have a similar preference for avoiding personal cost, such as those of effort. Let 𝛼𝑖 be an individual parameter pertaining to effort cost. This parameter 𝛼𝑖 accounts for the agents’ intrinsic motivation. It relates negatively to intrinsic motivation as we may assume a person who is highly intrinsically motivated to feel effort to be less of a burden. Below, we will assume a specific distribution in 𝛼𝑖.
19
Performance (𝑍) can be either high (𝑍𝐻) or low (𝑍𝐿), and while effort does not necessarily lead to high performance it does affect the probability of reaching it. To that extent, the following is defined:
𝑃(𝑍𝐻) = {𝑃𝑇 𝑖𝑓 𝑒 = 1 0 𝑖𝑓 𝑒 = 0
Where 𝑃(𝑍𝐻) is the probability of achieving high performance, and subscript 𝑇 indicates the height of the target, which may be either High (𝐻) or Low (𝐿). For now we will consider the expectations for a general target 𝑇. Section 3.1.1 subsequently compares the expectations under high and low targets.
To incentivize the agent to exert effort, an extrinsic incentive b is exogenously introduced, which is conditional on achieving high performance. Parameter b follows the assumption from section 2.1 that the imposed incentive affects the agent positively through a price-effect (Gneezy, Meier and Rey-Biel, 2011). Therefore, b is defined as:
𝑏 = {1 𝑖𝑓 𝑍0 𝑖𝑓 𝑍𝐻 𝐿
However, as mentioned in subsection 2.1.1, the imposition of an extrinsic incentive may also result in the crowding-out effect. Such a lowering of intrinsic motivation is incorporated into parameter 𝛼𝑖, as it accounts for differing levels of intrinsic motivation. Under the assumption that effort is the only mean by which to achieve the set goal, let an agent i’s preferences can be represented by a utility function, which is defined as follows:
𝑢𝑖𝐴(𝑒
𝑖, 𝛼𝑖) = 𝑏 − 𝛼𝑖𝑒𝑖, 𝑤ℎ𝑒𝑟𝑒 0 < 𝛼𝑖 ≤ 1 and 𝛼𝑖 ~ 𝑈(0,1)
For simplicity we assume heterogeneity only in 𝛼𝑖, and assume the distribution in 𝛼𝑖 to be uniform. In this way we are able to form concrete and calculable expectations. Subsection 3.3.1 further discusses the assumed heterogeneity in 𝛼𝑖.
20
An agent is expected to exert effort in those cases where the imposed benefit outweighs the personal effort costs associated with the increased probability of achieving the goal. In other words; when the utility of exerting effort outweighs that of not doing so:
𝐸[𝑢𝑖𝐴(1)] > 𝑢 𝑖𝐴(0) 𝑃𝑇− 𝛼𝑖 > 0
𝑃𝑇 > 𝛼𝑖 [Condition 3.1] An agent is thus expected to exert effort when the probability of high performance through effort provision outweighs the personal costs to effort imposed through 𝛼𝑖. A higher 𝛼𝑖, given a fixed probability of high performance 𝑃𝑇, therefore indicates that an agent is less likely to exert effort. Consequently, the population share that exerts effort is the part of the population whose effort costs fall below a certain threshold value. Let 𝐻 and 𝐿, like before, indicate high and low targets and let subscript d indicate the possibility to be dishonest, where 𝑑 ∈ {0,1}. As we for now assume the agent cannot be dishonest, it takes the value 0. Let the following thereby define the thresholds for effort exertion corresponding to condition 3.1.
𝛼̅0𝑇, 𝑤ℎ𝑒𝑟𝑒 𝑇 ∈ {𝐻, 𝐿}
As we assume a uniform distribution in 𝛼𝑖 we may assume the corresponding population shares that exert effort to be the following, given 𝑓(𝛼𝑖) to be the probability density function. This holds for both situations with and without dishonesty.
𝑓(𝛼𝑖) = 1 1 − 0 = 1 𝐹𝛼(𝛼̅𝑑𝑇) = 𝑃(𝛼 ≤ 𝛼̅ 𝑑𝑇) = ∫ 𝑓(𝛼𝑖)𝑑𝛼𝑖 𝛼̅𝑑𝑇 −∞ = 𝛼̅𝑑𝑇 3.1.1 Target size
According to Miller and Weiss (2015) the general conclusion from the goal-setting literature is that higher targets are positively related to performance. Under the assumption that targets lead to performance through increased effort, it is implied that higher targets should lead to higher
21
performance through higher effort. Within the context of this model, given that effort is a binary decision, this means that a higher target lowers the probability of achieving higher performance and thereby the target. We therefore assume the following:
𝑃𝐻 < 𝑃𝐿
In essence, downward adjustment of probabilities is similar to redefining 𝑍𝐻 to indicate higher performance, under the assumption that those agents who still exert effort generate higher performance. Moreover, as the imposed incentive has not changed we assume no difference in crowding-out effects relative to the price-effect and therefore, both b and 𝛼𝑖 are unchanged.
Under these assumptions and the fact that the probability of achieving high performance goes down for a higher target, the threshold of effort cost below which an agent exerts effort changes. As the benefit of exerting effort goes down through the decreased probability, effort cost must also decrease to keep the net-benefit of effort exertion positive. Moreover, the threshold for effort exertion decreases by exactly the same amount, as the threshold value is the value of 𝛼𝑖 where LHS and RHS are equal in condition 3.1. As a result we find the following:
𝛼̅0𝐻 < 𝛼̅ 0𝐿 𝛼̅0𝐻− 𝛼̅
0𝐿 = 𝑃𝐻− 𝑃𝐿
As population shares relate monotonically to the thresholds due to the assumed uniformity in 𝛼𝑖, the difference in population shares that exert effort is defined by the difference between thresholds 𝛼̅0𝐻 and 𝛼̅
0
𝐿. Therefore, we may conclude that the population share that exerts effort is lower under a high than under a low target:
𝛿 𝐹𝛼(𝛼̅0𝑇) = 𝐹𝛼(𝛼̅0𝐻) − 𝐹𝛼(𝛼̅0𝐿) = 𝛼̅0𝐻− 𝛼̅
0𝐿 = 𝑃𝐻− 𝑃𝐿 < 0
22 Conclusion 1: Higher targets lead to higher performance
We may interpret the lower share of effort exertion as the goal not being accepted by those whose effort costs are now too high (Erez and Zidon, 1984). However, those who do accept the goal and exert effort will yield higher performance relative to when a lower target was imposed, following the linear positive relationship between target size and performance observed by Miller and Weiss (2015) and Locke and Latham (2002). However, as the size of performance does not directly affect the agent it does not show in the utility function. Though these effects affect performance in a conflicting way, we assume that, given the general conclusion of the goal-setting literature (Miller and Weiss, 2015) on average the increased performance in those who do exert effort outweighs the loss in performance of those who do not and as a result, performance increases.
3.2 Dishonesty
In situations where it is hard or impossible to verify the achievement of a target, the assumption that effort is necessary to attain a certain incentive has to be reconsidered and as a result, so do the general conclusions. Alternative means by which to attain the incentive reward, such as dishonesty, may act as substitutions and thereby affect the perceived effectiveness of extrinsic incentives.
As mentioned, effort carries costs and as people to some extent seek to avoid cost, they also seek to avoid exerting effort. In the model of section 3.1, effort e was the sole mean with which to attain incentive b, which meant that for any sufficiently high probability of reaching the goal through effort, effort was exerted. Consider the model of section 3.1 in which the agent, in addition to exerting effort, has the option to be dishonest about his or her performance. In this extension to the model, the imposed incentive b is conditional on a self-report of performance by the agent, rather than actual performance. Therefore, b is redefined as follows:
𝑏 = {1 𝑖𝑓 𝑟0 𝑖𝑓 𝑟𝐻
𝐿 𝑤ℎ𝑒𝑟𝑒 𝑟 = {𝑟𝐻, 𝑟𝐿}
Dishonesty, like effort, is a binary decision and is conditional on performance report r. Though any report that is different from performance may be construed as being an act of dishonesty, it is
23
assumed that dishonesty within the model only occurs when an agent over-reports by reporting high performance when performance is low. This is assumed because any other act of dishonesty does not positively affect an agents’ outcome and is therefore an irrational act. Following this, dishonesty d is defined to be a function of r as follows:
𝑑𝑖 = {1 𝑖𝑓 𝑟 = 𝑟0 𝑖𝑓 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒𝐻 𝑎𝑛𝑑 𝑍 = 𝑍𝐿
As mentioned, dishonesty like effort carries costs also in the absence of possible detection and punishment. Such intrinsic costs of lying (see e.g. Erat and Gneezy, 2012) differ between people. Therefore, parameter 𝛾 indicates individual lying costs. We assume that lying costs are not directly affected by target size, despite the fact that a possible effect may exist.4 Thus, let an agent i’s utility function be redefined as follows:
𝑢𝑖𝐴(𝑒
𝑖, 𝑑𝑖; 𝛼𝑖, 𝛾) = 𝑏 − 𝛼𝑖𝑒𝑖 − 𝛾𝑑𝑖, 𝑤ℎ𝑒𝑟𝑒 0 < 𝛼𝑖 ≤ 1 𝑎𝑛𝑑 0 < 𝛾 ≤ 1
We do not assume a type-based model of dishonesty and consequently the decision to be dishonest is dependent on the costs and benefits of that specific act of dishonesty (Gibson et al., 2013). As a result, the benefits of dishonesty must at the very least outweigh the costs associated with it. Therefore, both costs of dishonesty 𝛾 and benefit b are 1 at the most. We assume only those agents whose lying costs exactly outweigh their benefit b, to prefer honesty.
As mentioned, dishonesty may act as a substitute to effort in attaining the imposed incentive. Therefore we may assume that more often than not, an agent initially chooses one or the other. Given that both come with costs, the decision to choose one over the other is conditional on their relative costs. Assuming that effort does however not necessarily lead to high performance, the choice is thereby also conditional on the probability of achieving high performance following the decision to put in effort. Though dishonesty may act as a substitute to effort, this does not exclude the possibility of doing both. If an agent puts in effort but does not achieve high performance, we assume that agent to be dishonest as long as doing so at that point
4 As mentioned, a direct effect of target size on the costs of dishonesty does not lead to contradictory expectations. Rather it amplifies the current expectations and therefore, for simplicity, we assume the costs of honesty to be constant over targets.
24
has positive net benefits. In short, an agent is expected to exert effort if the expected benefits of doing so outweigh those of not doing so, given the possibility to be dishonest:
𝐸[𝑢𝑖𝐴(1, 𝑑 𝑖 ; 𝛼𝑖, 𝛾)] > 𝑢ì𝐴(0, 𝑑𝑖 ; α𝑖, γ) 𝑃𝑇+ (1 − 𝑃𝑇)(1 − 𝛾) − 𝛼𝑖 > 1 − 𝛾 1 − 𝛾 + 𝛾𝑃𝑇− 𝛼𝑖 > 1 − 𝛾 𝛾𝑃𝑇− 𝛼𝑖 > 0 𝛾𝑃𝑇 > 𝛼𝑖 [Condition 3.2] As in section 3.1, we formally define the thresholds below which an agent exerts effort. However, as dishonesty is now possible the thresholds get subscript 1.
𝛼̅1𝑇, 𝑤ℎ𝑒𝑟𝑒 𝑇 ∈ {𝐻, 𝐿}
Following from condition 3.2, we thus expect the conditions to exert effort are more stringent under the possibility of dishonesty, as 𝛾𝑃𝑇 < 𝑃𝑇. Moreover, the measure to which the possibility of dishonesty affects this condition is highly conditional on an agent’s costs of lying. Looking at the extremes, a 𝛾 of 1 indicates complete honesty, corresponding to the model of section 3.1. On the other side, a 𝛾 of 0 means that dishonesty is free of charge, making any effort input an irrational decision. As agents are expected to have lying costs somewhere between those extremes, and effort costs follow the same distribution, effort is exerted less often. Therefore we assume the following with regard to the difference of thresholds with and without dishonesty:
𝛼̅1𝑇 < 𝛼̅ 0 𝑇
As in section 3.1, population shares that exert effort under the possibility to be dishonest are monotonically related to the corresponding thresholds due to the assumed uniform distribution in 𝛼𝑖. Therefore, the change in the population shares that exert effort corresponding to the threshold decrease through the possibility of dishonesty is:
25 𝛿 𝐹𝛼(𝛼̅𝑑𝑇) = 𝐹𝛼(𝛼̅1𝑇) − 𝐹𝛼(𝛼̅0𝑇) = 𝛼̅1𝑇− 𝛼̅ 0𝑇 = 𝛾𝑃𝑇− 𝑃𝑇 < 0
Conclusion 2: The possibility of dishonesty leads to lower performance
As shown, introducing the possibility to be dishonest lowers the amount of people exerting effort, regardless of the target size 𝑇. Unlike the difference between high and low targets, overall performance is however assumed to go down. Introducing dishonesty does lower the amount of effort that is exerted, but there is no reason to assume that it leads to higher performance in those that still exert effort.
3.2.1 Target size
As in section 3.1, target size is likely to influence the decision to exert effort. As a higher target lowered the probability or reaching it given a fixed amount of effort, the decision to exert effort was shown to negatively relate to target size. As in that section, a higher target similarly lowers the probability of achieving high performance under effort exertion, which subsequently lowers the thresholds for effort exertion. As opposed to section 3.1 the thresholds for effort exertion are however related differently to the probability of achieving high performance, as LHS in condition 3.2 includes lying-cost parameter 𝛾. As a result, the following defines the change in thresholds due to imposing a higher target.
𝛼̅1𝐻 < 𝛼̅ 1𝐿 𝛼̅1𝐻− 𝛼̅1𝐿 = 𝛾𝑃𝐻− 𝛾𝑃𝐿
= 𝛾(𝑃𝐻− 𝑃𝐿)
The share of the population that exerts effort thus similarly goes down following the imposition of a higher target. As we assumed a uniform distribution in 𝛼𝑖, such shares are still monotonically related to the thresholds for effort exertion:
26 𝛿 𝐹𝛼(𝛼̅1𝑇) = 𝐹𝛼(𝛼̅1𝐻) − 𝐹𝛼(𝛼̅1𝐿) = 𝛼̅1𝐻− 𝛼̅ 1𝐿 = 𝛾(𝑃𝐻− 𝑃𝐿) < 0
Conclusion 3: Higher targets lead to higher performance under dishonesty
Thereby it is shown that under dishonesty, as in a situation without it, a higher target leads to less effort exertion. In addition, though the amount of agents exerting effort is expected to be lower, performance of agents who exert effort is expected to be higher than under a low target, similarly to subsection 3.1.1. As in that section, we use the general conclusion of the goal-setting literature that higher targets lead to higher performance (Miller and Weiss, 2015) to establish that the higher performance in those that exert effort outweighs the loss of performance in those that stop doing so. Therefore, a higher target is again expected to lead to higher performance.
In section 3.2 an agent not only has the option to exert effort, but also to be dishonest. Therefore, in addition to knowing in what way target size affects the decision to exert effort, we are also interested in knowing in what way it affects the tendency to be dishonest. Let the following therefore define the expected utilities of dishonesty and honesty respectively, taking into account the fact that the possibility to be dishonest follows the decision to exert effort:
𝑢𝑖𝐴(𝑒 𝑖, 1 ; 𝛼𝑖, 𝛾) = {1 − 𝛾 − 𝛼1 − 𝛾 if 𝑒𝑖 = 0 𝑖 if 𝑒𝑖 = 1 𝐸[𝑢𝑖𝐴(𝑒 𝑖, 0 ; 𝛼𝑖, 𝛾]) = {𝑃0 if 𝑒𝑖 = 0 𝑇− 𝛼𝑖 if 𝑒𝑖 = 1
To find what determines the decision to be dishonest we compare the utility of being dishonest against that of not being dishonest, both for situations with and without effort exertion. Let the following first define the condition to be dishonest when not exerting effort:
𝑢𝑖𝐴(0, 1 ; 𝛼
𝑖, 𝛾) > 𝐸[𝑢𝑖𝐴(0, 0 ; 𝛼𝑖, 𝛾]) 1 − 𝛾 > 0
27
When not exerting effort, it is thus strictly better to be dishonest except for when costs of dishonesty perfectly outweigh its benefits. Furthermore, let the following define the condition to be dishonest with effort exertion:
𝑢𝑖𝐴(1, 1 ; 𝛼
𝑖, 𝛾) > 𝐸[𝑢𝑖𝐴(1, 0 ; 𝛼𝑖, 𝛾]) 1 − 𝛾 − 𝛼𝑖 > 𝑃𝑇− 𝛼𝑖
1 − 𝑃𝑇 > 𝛾 [Condition 3.3]
Conclusion 4: Higher targets lead to more dishonesty
As shown the decision to be dishonest when not exerting effort is not affected by target size. However, a decision to be dishonest following effort exertion is dependent on the expected benefits of effort exertion. Therefore, for a decrease of 𝑃𝑇 following the imposition of a higher target, acts of dishonesty are expected to occur more often, keeping constant their costs. In other words, an agent is on average more likely to be dishonest when the target is high.
In conclusion, we expect higher targets to lead to higher performance both when dishonesty is possible and when it is not, despite the fact that the amount of people who exert effort is expected to decrease in both situations. As performance does not directly affect the agent in this model, we could not draw this conclusion based on the model alone. Therefore, we use the assumption that the net-effect of higher targets on performance is positive, as follows from the goal-setting literature (Miller and Weiss, 2015). Moreover, dishonesty is expected to occur more frequently under high than under low targets.
As effort is in this model assumed to be a binary decision, this might have influenced the expectations that were formed. Alternatively, effort could be modeled continuously, thereby keeping the probability of achieving high performance constant over low and high targets. A higher target would then imply that more effort has to be exerted, keeping constant the probability. Under the same imposed benefit, this would lead to less people exerting effort. However, performance would be expected to go up similarly to this model as those who do exert effort perform better. For that we however need to make a similar assumption regarding the net-effect of a higher target, as performance still does not show in the agent’s utility function. With regard to dishonesty, assuming that the costs of dishonesty do not scale with target size, this
28
implies that higher targets would increase the cost of effort relative to that of dishonesty. As a result, dishonesty would occur more frequently at the expense of effort and performance. In conclusion, though the derivation would be slightly different, the results are expected to be similar.
3.3 Comparison
Though a higher target is, both under honesty and dishonesty, expected to lead to less effort exertion, we may expect the effects to differ as dishonesty may act both as a substitute and addition to effort and the benefits of not exerting effort are therefore nonzero.5 In order to compare the effect of a high target on effort exertion between situations with and without dishonesty, we define the difference in effort exertion that is assumed to exist between situations with and without dishonesty, of the difference between a high and low imposed target. We will subsequently refer to this as the ‘difference-in-difference’.
∆𝐹𝛼(𝛼̅𝑑𝑇) = 𝛿 𝐹𝛼(𝛼̅1𝑇) − 𝛿 𝐹𝛼(𝛼̅0𝑇) = 𝛾(𝑃𝐻− 𝑃𝐿) − (𝑃𝐻− 𝑃𝐿)
= (𝛾 − 1)(𝑃𝐻− 𝑃𝐿) < 0
𝑤ℎ𝑒𝑟𝑒 𝛾 ≤ 1 𝑎𝑛𝑑 𝑃𝐻 < 𝑃𝐿
Conclusion 5: The difference in performance between high and low targets is smaller when dishonesty is not possible than when it is
We conclude that for any lowering of probability due to a higher imposed target, the difference-in-difference has a positive sign. That is to say, the amount by which effort exertion is expected to go down following a higher target is expected to go down more when dishonesty is not possible than when it is. One way to interpret this is as follows: though dishonesty provides an alternative action to effort, it does not exclude it. Therefore, even though the expected benefits of exerting effort go down, the effect is mitigated by the possibility to be dishonest following the exertion of effort. In other words, the expected benefits of exerting effort relative to not doing so go down less.
5 Assuming 𝛾
29
In contrast to the expected effect of target size, this added effect is not expected to lead to higher performance among those who cannot be dishonest. When considering solely the effect of a higher target, we assumed those who do exert effort to generate higher performance. A similar assumption is not made with regard to the effects of unconditional honesty. Therefore, given the same level of performance that is attained through effort, less effort exertion causes the expected increase in performance to be diminished. This expectation holds for all initial thresholds and differences in probability between high and low targets.
3.3.1 Heterogeneity
In this model we assumed that parameter 𝛼𝑖 is uniformly distributed, and while this does not affect the thresholds for effort exertion, it does affect the share of the population that exerts effort for such thresholds. Though similar assumptions regarding heterogeneity could be made for lying costs or the probability of achieving high performance, we for simplicity assumed only 𝛼𝑖 to be heterogeneous as effort costs are known to be distinctly different between people and it, as shown, affects thresholds both under dishonesty and honesty. Appendix A briefly discusses the differences in expectations when heterogeneity is considered in other variables. Though heterogeneity in 𝛼𝑖 can follow different distributions, we assumed a uniform distribution as it leads to straightforward calculation. More importantly however, prior research or knowledge does not lead us to believe that this distribution is ill suited. Assuming a different distribution in 𝛼𝑖 does lead to different expectations with regard to section 3.3. Appendix A therefore in addition gives some conjectures with regard to the expectations when we consider a normal distribution in 𝛼𝑖.
3.3.2 Conclusion
In conclusion, Chapter 3 outlined a mathematical model from which we formed expectations with regard to effort exertion and performance with and without the possibility to be dishonest. In the end the following was shown. Firstly, a higher target on average leads to higher performance both with and without the possibility to be dishonest, despite the fact that less people exert effort. Secondly, the possibility to be dishonest negatively affects the amount of effort exertion without affecting performance in those who exert effort. As a result, it negatively affects performance. Thirdly, higher targets lead to more dishonesty. Lastly, the decrease in
30
effort exertion following from a higher target is larger when dishonesty is not an option than when it is. Consequently, performance increases more through higher targets when dishonesty is possible. In the next Chapter, the methodology of an accompanying experiment will be described, along with the hypotheses that stem from the model and literature.
31
4. Experimental methodology
4.1 Participants
Along the lines of the model of Chapter 3 and the literature on extrinsic incentives, an experiment was conducted in June and July of 2015. Participants for this experiment were all high-school students between the ages of 13 and 18, enrolled in a large homework institute in Amsterdam. Students were enrolled in a Gymnasium, Athenaeum, HAVO or VMBO (T/K) high-school level at 16 different high high-schools in Amsterdam. Selection of participating students was done on basis of eligibility, with participation only being possible following active parental consent.6 Parental consent was obtained via a letter sent to the home address of the child’s parents or guardians, containing a detailed information sheet and the informed consent form. The consent forms were subsequently sent back to the homework institute in a closed envelope, thereby determining children’s eligibility for participation. Appendix B contains the information sheet and informed consent form sent to the parents and guardians.
Letters of consent were sent to the parents and guardians of 151 students, of which 56 were returned for participation. Four of these 56 students unfortunately could not start the experiment as at the time of consent their school year had ended. Following consent, students were in addition personally asked to participate. All eligible students signed up, and of those 52 participating students, 42 ultimately finished the experiment. Of the 10 students who did not finish the experiment, 5 dropped out because they did not receive their grade and 5 dropped out as they were unwilling or unable to meet with the experimenter to report their grade and fill in the questionnaire.
4.2 Design and materials
The experiment was conducted at a large homework institute in Amsterdam, and consisted of normal daily study-related behavior with an added incentive. Following the taxonomy of experiments by Harrison and List (2004), the conducted experiment was a framed field experiment. The homework institute was selected for two main reasons. Firstly, as the
32
experimenter was at the time an employee of the homework institute, this provided the unique possibility to obtain a pool of student participators through cooperation of the institutes’ ownership. Due to the nature of the homework institute, this pool was diverse in both age, level of education and social background. Secondly, being an employee the experimenter had access to students’ official grades making it possible to identify a baseline grade and to distinguish dishonest behavior.
The experiment consisted of two parts: the experiment introduction (1) and conclusion (2). In part one of the experiment the participants were offered a reward for a higher grade on an upcoming test that was part of their normal high-school curriculum. The test was randomly selected on the basis of time considerations. As tests happen irregularly both across and within schools, the courses to which the selected tests belonged varied.7 The reward, a €10,- voucher for Pathé cinemas, was identical over all participants. The reward was specifically chosen to be non-monetary, though with monetary value. In Levitt, List, Neckermann and Sadoff (2012) the authors investigate the notion that non-monetary incentives work better to incentivize children and find that they indeed do for children in low age groups, but less so for those in higher age groups. Therefore, an incentive was selected that is non-monetary, but with a clear value. The higher grade that needed to be attained to receive the reward was an increase from their baseline grade, which was their average grade over the last year. This baseline average was obtained from the students’ online grade systems.8
In this part a first variation was introduced, in which some students were instructed to attain a 0.5-point increase, while others were instructed to attain a 1.5-point increase. This first division was made to answer the question whether higher targets would actually lead to higher performance. We assume that the relationship between target size and goal difficulty is non-linear. That is to say, a 1.5-point increase from an average grade of 6 is not the same as an identical increase from an average grade of 8. Therefore, in an attempt to divide baseline grades equally over the different target-size treatments, division of treatments was done at random by applying a fixed order to the random order in which students were signed up for participation.9
A second treatment variation was introduced by giving half the students the possibility to be dishonest when grades were reported in part two of the experiment. The variation was
7 Full list included in appendix E. 8 See also section 4.4.
33
communicated through details in the experimental script, which is included in appendix C. The experimental script was in addition specified to account for several factors. Firstly, by signifying that the experimenter was performing the research in name of the University of Amsterdam rather than himself, an attempt was made to minimize experimenter demand effects. Secondly, as the homework has strongly established social norms with regard to - among other things - dishonesty, it was similarly attempted to make clear that the research was not related to the homework institute.
Most importantly however, through the experimental script some students were told to self-report their grade to determine whether they received the voucher, while others were told that it would be checked together with the experimenter. In this way, the former group was indirectly informed that dishonesty would be possible, while the latter group was informed that it was not. We will from now on refer to this treatment division as respectively being the Report and Verify groups. Report indicates the possibility to be dishonest as students self-report their grade. Verify indicates honesty, as the grade is verified by the experimenter. Again, treatment division was done at random, by applying a fixed order to the random order in which students were signed up for participation. By introducing this second variation, the experiment had a 2x2 design through which it was possible to, through a difference-in-difference analysis, answer the question in what way the possibility of dishonesty affects the effectiveness of higher targets. Table 1 provides an overview of the different treatments.
After the experiment students were asked to fill in a short questionnaire via Qualtrics (Qualtrics, Provo, UT). The questionnaire questions are outlined in appendix D. The questionnaire was introduced for several reasons. Firstly, as it was not possible to measure effort
Table 1: Experimental treatments, 2x2 design
Dishonesty
(Report)
Honesty (Verify) High target (High) Report-High Verify-High
Low target (Low) Report-Low Verify-Low
Notes: table outlines the different experimental treatments. Report indicates a 0 percent probability of checking; the reward is conditional on the report of the student, not on the official grade. Verify indicates a 100 percent probability of checking; the reward is conditional on the official grade as it is verified by the experimenter.
34
input directly, the questionnaire served as an indirect measure. Secondly, questions were asked concerning communication with fellow students, and any effects this might have had on their behavior. Lastly, questions were asked pertaining to familiarity with the experimenter and the homework institute, which was done to control for any possible experimenter demand effects.
4.3 Procedure
Participants were asked to follow the experimenter to a separate room, where they were seated. They were then proposed to take part in the experiment under the guise of it being research of the University of Amsterdam, unrelated to the experimenter as an individual. This was done to minimize conditioning of behavior, as participants were often familiar with the experimenter. Participants were asked to participate and made aware of the fact that their parents or guardians were aware and willing to let them take part. In addition students were told they could stop participation at any point. For those participants that accepted participation, the guidelines were set out along the lines of the script in appendix C. As mentioned, depending on the treatment to which the respective student belonged, instructions in the script differed. In the dishonesty treatments, students were asked to self-report their grade, as it would take a long time for the experimenter to check the grades of all students. Consequently, these students were indirectly informed that dishonesty was possible. In the honesty treatments, students were instead told that they would be looking up their grade together with the experimenter, leaving them no possibility to be dishonest.
After the full explanation, student details were logged in a data file; among which the baseline grade, date of sign-up and test and the desired grade. Data of other students was hidden while signing up students to minimize possible direct target size effects.10 In conclusion, students were asked to refrain from communicating details of the experiment to their fellow students. All participating students were given a reminder sheet with the course, date and required grade. This sheet was stapled into a booklet they, without exception, receive upon entering the homework institute. Introduction of the experiment was done between 7 and 9 days prior to the selected test, as this would give them sufficient time to transfer any increases in motivation into effort and subsequent performance. Between one and two weeks after the test, students were asked to meet
10 Familiarity with fellow students targets may increase students’ tendency to be unhappy with their assigned target, relative to that of others.
35
with the experimenter, much like when the experiment was introduced. The amount of time varied, as teachers took different amounts of time to grade the tests. At the time of the second meeting, students were asked to either self-report their grade or look at their grade together with the experimenter, in the dishonesty and honesty treatments respectively. Those grades were subsequently logged in the aforementioned data file. When students reported or showed - at least - the agreed upon grade, they were presented with the €10,- Pathé movie voucher. All students were asked to fill in the post-experimental questionnaire. After filling in the questionnaire, the experiment was completed, and students were again asked to refrain from communicating experimental details.
4.4 Measurements
Rather than using achievement of the target grade as a binary measure of performance, actual grades were used as a measure of performance under the assumption that baseline grades were well distributed over the different treatments. By using a continuous scale of performance, clearer differences in performance were expected to be detected under the small sample size. Actual grades were measured by looking up actual achieved grades in the students’ online grade systems: Magister (15 schools) and SOMtoday (1 school).11 These grades were then subsequently compared to the students’ baseline grades to determine whether grades had increased due to imposition of the incentive. As teachers personally log grades and weights on these online systems to determine averages and the baseline grades were obtained from these same systems, this is a valid measure. Actual grades of students in honesty treatments Verify-High and Verify-Low were looked up together with the student. Actual grades of students in dishonesty treatments Report-High and Report-Low were clearly not looked up together with the student, but were recorded by the experimenter after the school year had ended.12
As not all students had the same amount of time between the experimental introduction and their test, the exact amount of days was additionally measured. To account for differences in treatments, two dummy variables were introduced: one for target size and one for dishonesty. Relating to the test, further measurements were done with respect to the level of the test and the
11 SOM today (somtoday.nl); Magister (schoolmaster.nl).
12 In this way it was made sure that students were not deceived, given that they were told their grades would not be checked until the end of the school year.
36
course to which it belonged. The level of test was either SE (“Schoolexamen”) or PW (“Proefwerk”). Both levels are used by almost all schools in the Netherlands and thereby are of a constant level within the type. SE’s are considered to be of a higher level than PW’s.13
Further measurements of control were obtained by measuring personal characteristics through access to the homework institute’s own internal online system. These characteristics included the level of education, age and gender.
Reported grades in the Report-Low and Report-High treatments were not used as a measure of performance, but were used to distinguish acts of dishonesty. By comparing the reported grades to the aforementioned actual grades, dishonesty was distinguished and defined as a binary variable. Corresponding to the model of Chapter 3, dishonesty was identified as over-reporting: reporting high performance while achieving low performance.
4.5 Hypotheses
Following the experimental methodology, and with respect to the research question and sub-questions, multiple hypotheses were defined.
Hypothesis 1
Actual grades will be higher than baseline grades in all treatments.
The general conclusion from the literature is that extrinsic incentives in education have a small, though non-negative effect on performance through effort (Gneezy, Meier and Rey-Biel, 2011). The current design provides the opportunity to replicate this result. Baseline grades provided an appropriate control against which to measure differences in the students’ performance, as they are students’ own average grades over the last year. Though there may be variability in a student’s grades, we may expect that variability between students is even higher as long as the sample is diverse. Therefore, a within-subject approach is deemed appropriate.
13 “Proefwerken” (PW) determine whether a student is eligible to advance to the next grade, “Schoolexamens” (SE) are only taken by students in higher years (4th , 5th and 6th year), and count towards graduation.
