Confidence miscalibration and risk attitude

CONFIDENCE

MISCALIBRATION

AND

RISK

ATTITUDE

ABSTRACT:Although confidence is favored in professional environments, overconfidence is linked with inefficient results and unfavorable situations in previous studies (Camerer & Lovallo, 1999; Johnson et al., 2006; Neale & Bazerman, 1985). Previous research demonstrates a relationship between confidence miscalibration and risk attitudes (Campbell, Goodie, & Foster, 2004; Goodie, 2003; Murad, Sefton, & Starmer, 2016; Nosi’c & Weber, 2010). The findings suggest that incorrect assessments of probabilities are likely to be another inefficiency caused by miscalibration. This paper aims to understand the relationship between overconfidence and risk attitudes. It investigates the impact of miscalibration on risk attitudes. The research question asks whether a change in the confidence level of an individual influence their risk attitude? If so, how does reducing overconfidence or underconfidence influence risk attitude? An online, within group experiment is conducted with 137 participants. A feedback treatment is used to calibrate the miscalibrated indiviudals. The change in the risk attitude is measured after the treatment. The findings revealed that the feedback treatment did not reduce the absolute value of miscalibration. A causal relationship and a correlation are not found in risk attitudes and miscalibration. The hypothesis that overconfidence influences risk attitude is rejected. Possible reasons for the contradictory results to previous research and the hypothesis are discussed.

Idil Demircan 11818646 MSc. Economics, Behavioral Economics and Game Theory Track Supervisor: Matthijs van Veelen

Statement of Originality

This document is written by Idil Demircan who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document are original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

T

C

1. INTRODUCTION ... 3

2. LITERATURE REVIEW ... 4

2.1. RISK ATTITUDE ... 4

2.2. RISK ATTITUDE AND CONFIDENCE MISCALIBRATION ... 8

3. METHODOLOGY ... 10

3.1 RISK ATTITUDE ASSESSMENT ... 10

3.2 CONFIDENCE LEVEL ASSESSMENT AND FEEDBACK ... 13

3.3 HYPOTHESIS ... 14

4. RESULTS AND DISCUSSION ... 15

4.1 SUMMARY STATISTICS ... 15

4.2 TREATMENT EFFECT AND MEAN COMPARISONS ... 17

4.3 REGRESSIONS AND HYPOTHESIS ... 18

4.4 DISCUSSION ... 20

5. CONCLUSION ... 21

6. APPENDIX:ONLINE EXPERIMENT ... 22

1. INTRODUCTION

In professional environments, a major qualification to succeed is considered to have high confidence, frequently emphasized by motivational speeches, personal development books, and the successful businessmen themselves, including Microsoft co-founder Bill Gates (Umoh, 2017). Though it is seen as a positive personality trait by a group of people, the miscalibration of confidence is likely to end up with misguided decisions and thus inefficient results. Camerer and Lovallo (1999) showed that the reason high numbers of businesses fail is overconfidence of the managers. Neale and Bazerman (1985) conducted a controlled experiment on negotiators and found that overconfident ones are less successful. Johnson et al (2006) argued that overconfidence is a major driver of wars, because of the “positive illusions” it creates, which make the governments and individuals believe that they would win. This paper investigates whether incorrect assessments of probabilities are another inefficiency caused by miscalibration.

Since decisions under uncertainties and risky situations require an assessment of the individual, an individual who is unable to assess their own capabilities might be having troubles in assessing risky situations. If so, an overconfident individual could be assigning a higher probability of success to a risky situation, just like they are estimating their own capabilities excessively. Likewise, an underconfident individual might be avoiding risks and foregoing opportunities because of miscalibration. The results for the overconfident could be risking too much and end up losing their endowments; and for the underconfident, not being able to step out of their comfort zone and getting stuck in a level that is below their capability. If the individual could be able to assess their own capability, then it might be possible that their assessment of the probabilities improves and change their risk preferences.

Being overconfident or underconfident is defined as having wrong beliefs about their abilities or assessing one’s probability of being successful mistakenly (Campbell et al., 2004). The same way of thinking can be attributed to risk preferences. A link between overconfidence and risk is found in previous research. Goodie (2003) and Murad, Sefton, & Starmer (2016) found a correlation between the individuals risk attitude and overconfidence. Campbell et al (2004) conducted a study on narcissists and their poor making abilities. The result revealed that poor decision-making abilities are caused by both overconfidence and high risk-taking behavior, which are

arguably caused by narcissistic behavior. Nosi’c & Weber (2010) investigated investor behavior and found that miscalibration has a relation with overconfidence. Previous literature connected the two concepts together; however, it is still unclear that what causes the correlation and whether overconfidence influences risk attitudes.

This paper aims to understand the relationship between overconfidence and risk attitudes. It investigates the impact of miscalibration on risk attitudes. The research question asks whether a change in the confidence level of an individual influence their risk attitude? If so, how does reducing overconfidence or underconfidence influence risk attitude?

To find out, an online experiment is conducted on 137 participants. In the first stage, their risk attitudes and confidence miscalibration levels are elicited. The second stage is designed to reduce the absolute value of miscalibration with a feedback treatment. In the third stage, the effect of the feedback treatment is measured. In the final stage, the risk attitude of the participants was elicited again to see the treatment effect.

The findings conclude that the feedback mechanism did not reduce the absolute value of miscalibration. For the underconfident individuals, miscalibration dropped; however, for the overconfident individuals, it increased. The hypothesis that a change in the confidence level of an individual has an influence on their risk attitude is rejected. The change in miscalibration after the treatment is found to not have a significant effect on the change in number of safe choices, which represents the risk attitude. The direction between the change in miscalibration and the change in the number of safe choices, although not in a significant way, is found to be in the opposite direction as predicted, that is number of safe choices, thus risk aversion seems to increase with higher absolute value of confidence miscalibration. Possible explanations of the results, mainly focusing on experimental design is discussed and suggestions are made for future research.

2. LITERATURE REVIEW

2.1. RISK ATTITUDE

To understand the relationship between risk attitude and overconfidence, the concept of risk attitude is investigated by looking at the previous papers on risk attitude.

Risk attitude definitions varies across the literature, both among economics and psychology; and among the economic theory itself. A very broad definition would be how enjoyable an individual finds risky situations, the state of either being risk loving, risk averse or risk neutral (Concina, 2014); yet what defines or determines an individual’s risk attitude is answered by various points of views.

The classical economic theory explains risk attitude with expected utility theory (Wakker, 2010). Expected utility theory, developed by Neumann and Morgenstern (1944), formulates the utility obtained from uncertain outcomes. The expected utility of a lottery with a probability p of winning X and probability 1-p of winning Y is stated as follows (Concina, 2014):

𝐸𝑈(𝑋, 𝑌; 𝑝, 1 − 𝑝) = 𝑝𝑈(𝑋) + (1 − 𝑝)𝑈(𝑌) (1) Expected utility is the weighted average of the utilities of the uncertain outcomes. According to expected utility theory, the risk attitude of the decision maker depends on the concavity of the utility function (Levin, 2006).

The figures above show the utility functions of different decision makers who have different risk attitudes. In economics, money and wealth considered to have diminishing marginal utility (Rabin, 2000), meaning that for a millionaire, having one euro more wouldn’t make him as happy as a person who only has one euro. For decision makers who have a diminishing marginal utility and thus a concave utility function, expected utility theory suggests that those individuals are risk averse (C. A. Holt & Laury, 2002; Levin, 2006). The expected utility of the lottery with probability

p of winning X and probability 1-p of winning Y is equal to the utility of Z, which is referred as the certainty equivalent (Concina, 2014; C. A. Holt & Laury, 2002; Murad et al., 2016). In the risk averse individual’s case, the certainty equivalent is a lower value compared to the expected value of the lottery, which is pX+(1-p)Y (C. a. Holt & Laury, 2014; C. A. Holt & Laury, 2002). For a risk loving individual, having one euro more would make them happier if they are already wealthy, compared to their poor state. In the case of a risk loving individual, they have increasing marginal utility and their certainty equivalent is greater than the expected value of the lottery. This means that if they would be offered the weighted average of the two possible gains as a sure gain instead of entering the lottery, they would select to enter the lottery. For the risk neutrals, their certainty equivalents equal the expected value of the lottery, meaning they have constant marginal utility and entering the lottery would mean the same for them if they were offered the expected value of the lottery for sure (Concina, 2014; Levin, 2006; Vollmer, Hermann, & Mußhoff, 2017; Wakker, 2010).

The expected utility theory defines risk attitude as the sensitivity towards money (or any other stakes that is involved) (Wakker, 2010). This definition of risk attitude has been challenged by various economists by the observation of human behavior.

The major and primary opposition comes from Maurice Allais (1953). He opposes the idea that expected utility theory suggests; which is for rational individuals, the expected utility should be maximized (Maurice Allais, 1991). He demonstrates his idea by creating lotteries for participants to choose between. The lotteries he presents are as follows (Heukelom, 2015):

Option A: 100% chance of winning 100 million francs Option B: 10% chance of winning 500 million francs

89% chance of winning 100 million francs 1% chance of winning 0 francs

It turns out that most people prefer option A over option B. The result supports the expected utility theory that the individuals select the option not with the highest expected value but the highest

expected utility according to their utility functions. The expected utilities of the lotteries are summarized by Kahneman and Tversky (1979) as follows:

Option A expected utility: 𝑈(100) × 1 (2)

Option B expected utility: 𝑈(500) × 0.10 + 𝑈(100) × 0.89 + 𝑈(0) × 0.01 (3)

Option A expected utility > Option B expected utility

𝑈(100) × 1 > 𝑈(500) × 0.10 + 𝑈(100) × 0.89 + 𝑈(0) × 0.01 (4) 𝑈(100) × 0.11 > 𝑈(500) × 0.10 + 𝑈(0) × 0.01 (5) The second set of lotteries designed by Allais is as follows:

Option C: 11% chance of winning 100 million francs 89% chance of winning 0 million francs Option D: 10% chance of winning 500 million francs

90% chance of winning 0 million francs

With those lottery options presented, the subjects mostly preferred Option D. This is the point that the empirical evidence does not match with the assumptions of the expected utility theory. The expected utilities of Option C and D are as follows (Kahneman & Tversky, 1979):

Option C expected utility: 𝑈(100) × 0.11 + 𝑈(0) × 0.89 (6)

Option D expected utility: 𝑈(500) × 0.10 + 𝑈(0) × 0.90 (7)

Option D expected utility > Option C expected utility

𝑈(500) × 0.10 + 𝑈(0) × 0.90 > 𝑈(100) × 0.11 + 𝑈(0) × 0.89 (8) 𝑈(500) × 0.10 + 𝑈(0) × 0.01 > 𝑈(100) × 0.11 (9) Equations 5 and 9 contradict with each other. Allais’s lotteries demonstrate that expected utility theory is not observed in individuals’ behavior (M. Allais, 1953; Maurice Allais, 1991). The phenomenon is named the Allais paradox.

Rank dependence theory (Quiggin, 1982) and prospect theory (Kahneman & Tversky, 1979) are developed as the most popular theories against the expected utility theorem (Von Neumann & Morgenstern, 1944), which introduce the probability weighting function, w(p). Probability weighting function helps to explain Allais paradox and other contradictions with the expected utility (Prelec, 1998). Although the probabilities are stated, individuals process them differently by assigning them psychological values (Kahneman & Tversky, 1979; Wu & Gonzalez, 1996). Probability weighting function replaces the idea that risk attitude is the curvature of the utility function as suggested by the expected utility theory. In the non-expected utility theories such as prospect theory and rank dependent theory, probability weighting function reflects the risk attitude (Wakker, 2010). With the probability weighting function, the expected value is represented as follows in prospect theory (Kahneman & Tversky, 1979), v(.) representing the value function, which replaces the utility function in the prospect theory:

𝐸𝑉(𝑋, 𝑌; 𝑝, 1 − 𝑝) = 𝑤(𝑝)𝑣(𝑋) + 𝑤(1 − 𝑝)𝑣(𝑌) (10) Goldstein & Einhorn (1987) and Lattimore, Baker, & Witte (1992) incorporated the optimism parameter in the probability weighting function. Fehr-Duda, Schuerer, & Schubert (2006) found that optimistic people judge probabilities higher than they are.

2.2. RISK ATTITUDE AND CONFIDENCE MISCALIBRATION

Kruger & Dunning (1999) found that people in general are imperfect at assessing their abilities accurately, and mostly the unskilled ones are overconfident. Overconfidence is defined as a miscalibration problem by Lichtenstein, Fischhoff, & Phillips (1977). Lichtenstein et. al also defines calibration as the correct estimation of the probabilities of future events. An individual is calibrated if the probability she assigns to a future event is, for example, 40% and the event happens 40 times out of the total 100 options. Fischhoff et. al (1977)’s finding on miscalibration and overconfidence can be explained with probability weighting function (Kahneman & Tversky, 1979; Quiggin, 1982). The optimism parameter in the probability weighting function overstates or understates the probabilities. The more optimist or pessimist a person is, the more they would deviate from the objective probability and this leads to miscalibration. de la Rosa (2011) defines overconfidence as overestimation of more favorable probabilities, also in the literature referred as

‘unrealistic optimism’ (Weinstein, 1980). Since the prospect theory (Kahneman & Tversky, 1979; Tversky & Kahneman, 1992; Wakker, 2010) suggests that risk attitudes are represented by the probability weighting function, the miscalibrated individual who is overconfident about future events is likely to have a probability weighting function that augments the probability and thus is more risk loving. Fischhoff et. al (1977)’s theory and probability weighting function combined suggests that risk attitude and confidence miscalibration are two concepts that are related.

Campbell et al. (2004) studied narcissists and their overconfidence and risk attitudes. They found narcissists to be more overconfident than the people who are not narcissists; and that the narcissistic people have greater willingness to bet, which suggests risk loving behavior. Campbell et al. (2004)’s work presents a linkage between risk loving behavior and overconfidence. Their finding could be interpreted that the narcissistic personality trait might explain the overestimation of actual situations which are in individual’s favor.

Broihanne, Merli, & Roger (2014) ran a research on finance professionals and found that their risk-taking behavior is correlated with overconfidence and optimism. Another field experiment on investors by Nosi’c & Weber (2010) found that overconfidence has an impact on risk taking and that one of the factors risk taking is determined by is subjective risk attitudes.

Risk attitudes and confidence miscalibration are linked together in laboratory experiments as well. Goodie (2003) conducted an experiment where he measured the participants confidence level with general knowledge questions and then asked them to bet on their own answers. The finding was that the overconfident individuals bet higher and the underconfident individuals bet lower. Murad et al. (2016) conducted a set of experiments where they also replicated Goodie (2003)’s experiment. They argued that Goodie (2003)’s experimental method doesn’t control for risk attitudes since it is incentivized based on the correctness of the answers and replicated the experiment by controlling for risk attitudes. They also used another experimental design where the confidence measurement is not incentivized. Their finding was that incentivized experiments, when controlled for risk attitude demonstrates lower miscalibration. In the experiment where they did not incentivize the confidence measurement, they found significant relation among overconfidence and risk loving behavior.

Previous literature demonstrates the linkage between risk attitude and confidence miscalibration, but the relationship between the two concepts are rather unclear. This paper aims to contribute to

the risk attitude and confidence miscalibration literature by investigating whether the miscalibration in confidence causes miscalibration in risk assessment. To do so, a within group experiment is conducted where a treatment is expected to change the confidence miscalibration. By the expectation of calibrating the miscalibrated, the risk attitudes are measured again to see if miscalibration causes overestimation/underestimation of probabilities and risk loving/risk averse attitude.

3. METHODOLOGY

To answer the question whether a change in the confidence level impacts risk preference; an online experiment has been conducted. The reason for the selection of the online experiment rather than a laboratory experiment was to keep the sample size quantitatively high and reach a more heterogenous subject pool. The experimental design consists of 4 stages as shown below:

Figure 2: Summary of the Experimental Design 3.1 RISK ATTITUDE ASSESSMENT

The first stage assesses the subjects’ risk attitudes. The method used is a task designed by Holt & Laury (2002), called Ten Paired Lottery-Choice Decisions. There are 10 lotteries with two options, Option A and Option B. The decision maker is asked to choose between either Option A or Option B. Option A is considered a safer choice since the two possible payoffs are close to each other, whereas Option B is the riskier option with possible gains are being on the extremes. The lotteries and the expected payoff differences are given below.

Figure 3: Ten Paired Lottery-Choice Decisions

The first paper that they used these lotteries (Holt & Laury, 2002), they modeled the risk aversion with Constant Relative Risk Aversion (CRRA) formula. The higher the number of safe option is chosen, the higher the risk aversion. The classification of the risk attitude profiles are as follows:

Figure 4: Risk attitude classifications based on number of safe choice made

However, since the CRRA formula is derived from expected utility theory and this paper aims to explain risk aversion with non-expected utility theories, a model with probability weighting function is used. Holt & Laury uses the same task to conduct an experiment on the reflection effect of prospect theory, in which they also turn to the probability weighting function (Laury & Holt, 2005). As explained in their paper, Option A is chosen over Option B in the condition as follows, with function w(.) being the probability weighting function and V(.) being the value function:

Option A > Option B

Option A Option B Expected Payoff Difference

1/10 of €2, 9/10 €1.60 1/10 of €3.85, 9/10 €0.10 €1.17 2/10 of €2, 8/10 €1.60 2/10 of €3.85, 8/10 €0.10 €0.83 3/10 of €2, 7/10 €1.60 3/10 of €3.85, 7/10 €0.10 €0.5 4/10 of €2, 6/10 €1.60 4/10 of €3.85, 6/10 €0.10 €0.16 5/10 of €2, 5/10 €1.60 5/10 of €3.85, 5/10 €0.10 €-0.18 6/10 of €2, 4/10 €1.60 6/10 of €3.85, 4/10 €0.10 €-0.51 7/10 of €2, 3/10 €1.60 7/10 of €3.85, 3/10 €0.10 €-0.85 8/10 of €2, 2/10 €1.60 8/10 of €3.85, 2/10 €0.10 €-1.18 9/10 of €2, 1/10 €1.60 9/10 of €3.85, 1/10 €0.10 €-1.52 10/10 of €2, 0/10 €1.60 10/10 of €3.85, 0/10 €0.10 €-1.85

Number of Safe Choices Range of Relative Risk Aversion for U(x)= Risk Preference Classifications

0-1 r < -0.95 highly risk loving

2 -0.95 < r < -0.49 very risk loving

3 -0.49 < r < -0.15 risk loving

4 -0.15 < r < 0.15 risk neutral

5 0.15 < r < 0.41 slightly risk averse

6 0.41 < r < 0.68 risk averse

7 0.68 < r < 0.97 very risk averse

8 0.97 < r < 1.37 highly risk averse

9-10 1.37 < r stay in bed

𝑤(𝑝)𝑉(2) + 𝑤(1 − 𝑝)𝑉(1.6) > 𝑤(𝑝)𝑉(3.85) + 𝑤(1 − 𝑝)𝑉(0.1) (11)

𝑤(1 𝑝) 𝑤(𝑝)

>

𝑉(3.85) 𝑉(2)

𝑉(1.6) 𝑉(0.1) (12)

To define the value function and the probability weighting function, Murad et. al’s (2016) choice of value and probability weightingfunctions are used, which were originally taken from Bruhin, Fehr-Duda, & Epper (2009).

𝑉(𝑥) = 𝑥𝛼

Kahneman and Tversky (1992)’s α measure is taken as 0.88, which was constructed from an experiment where participants choose between lotteries. The value function is as follows:

𝑉(𝑥) = 𝑥0.88

The probability weighting function is taken from Murad et al. (2016), originally developed by Goldstein & Einhorn (1987), defined as

𝑤(𝑝) = 𝛽𝑝𝛾

𝛽𝑝𝛾_{+(1 𝑝)}𝛾

𝛾 is considered 1 for the experiment, which makes the probability weighting functions linear. Although it is not linear in theory (Tversky & Kahneman, 1992), since the experiment aims to assess the risk attitude only and not designed to elicit 𝛾, only β is analyzed. β represents optimism, which in this paper we will refer as probability estimation parameter, which represents the risk attitude. Recalculating the chart below we get:

Figure 5: Risk attitude classifications based on number of safe choices made, estimated with probability weighting functions

While β between 1.44 and 0.96 defines risk neutrality, which is individuals who choose 5 safe choices, above 1.44 is risk loving and below 0.96 is risk neutral.

The final stage is the same as stage 1, only with different payoff values; which was the numbers used in stage 1 multiplied by 1.6. The reason for that is to keep the probability estimation parameters constant while giving the participants a different set of numbers, so that they wouldn’t select the same choices as they did in stage 1.

3.2 CONFIDENCE LEVEL ASSESSMENT AND FEEDBACK

In the second stage, participants’ miscalibration level is assessed. The method used to assess miscalibration is taken from Fischhoff et. al (1977), a method later on has been used widely in literature (Campbell et al., 2004; Goodie, 2003; Murad et al., 2016). 10 general knowledge questions are asked with two options, only one option being correct. The reason for selecting a general knowledge test to assess miscalibration is that it doesn’t require a specific skill and a heterogenous group of people could have similar chances of getting the answers right.

The questions are randomly selected from Tauber et al.(2013)’s general knowledge questions. After each question, the participants were asked to indicate their confidence level on their answer from 50% to 100%. This step is to reveal the miscalibration of an individual. Miscalibration level is defined as the difference between the average self-assessment and the test score. Overconfident

Number of Safe Choices Probability Estimation Parameter β Risk Preference Classifications

0-1 β>8.66 highly risk loving

2 8.66>β>3.85 very risk loving

3 3.85>β>2.25 risk loving

4 2.25>β>1.44 slightly risk loving

5 1.44>β>0.96 risk neutral

6 0.96>β>0.64 slightly risk averse

7 0.64>β>0.41 risk averse

8 0.41>β>0.24 very risk averse

9 0.24>β>0.11 highly risk averse

people are expected to overly assess a confident level which exceeds their correct answers; whereas underconfident people were expected to assess a lower confident level for themselves which is below their number of correct answers. After completion of the questions, the participants were given the answers and could see their results for the general knowledge test. Feedback used here is the treatment in the experiment. Feedback methods were used before to lower the overconfidence (Arkes, Christensen, Lai, & Blumer, 1987; Pulford & Colman, 1997). The aim is to make the participants re-evaluate their own capacities and reduce the absolute value of miscalibration.

The 3rd stage is the same as Stage 2, with only difference being the questions. The results were not provided after stage 3. The aim for stage 3 is to see if the feedback treatment had an impact on reduction of miscalibration. The question set selected is also from Tauber et al. (2013) and has the same difficulty level as the question set that is used in Stage 2.

3.3 HYPOTHESIS

This paper aims to find the impact confidence miscalibration has on risk attitudes. From the previous literature studies (Campbell et al., 2004; Goodie, 2003; Murad et al., 2016; Nosi’c & Weber, 2010), overconfidence was shown to correlate with risk loving behavior. Following the literature and the assumption that probability estimates and self-capability estimates are driven by accurate or inaccurate calibration, the hypothesis is as follows: A change in the confidence level

of an individual influence their risk attitude. Reducing overconfidence would result in a more risk averse behavior and reducing underconfidence would result in a more risk seeking behavior. Since the miscalibration of confidence and assessment of probabilities are both

misguided estimates, it is plausible that an individual with a misguided assessing mechanism would have trouble with assessing risk as they are assessing their own capabilities. After the realization of their misguided assessment mechanism, subjects would update their probability estimation parameters, and they would end up having a more realistic and efficient risk estimate.

4. RESULTS AND DISCUSSION

4.1 SUMMARY STATISTICS

In figure 6, the descriptive statistics of the variables are summarized. Gender is a dummy variable which is 1 for female, 0 for male. The subject pool consisted of 137 participants, of which 61 are female. The age range is between 15 and 70, while the mean is 37.8. The data set is diverse in a sense that the results won’t represent a specific demographic of people. The education level is classified into 7 levels, 1 being the lowest years of schooling (less than high school) and 7 being the highest (doctorate). The mean for the education level is 5.43, with university graduates and higher are making up 90% of the data. Since the experiment requires a certain level of English knowledge and the test takers are in majority non-native speakers, the education level of the participants is mostly over the population average.

Figure 6: Summary Statistics (gender=1 for female)

For the hypothesis to be tested, the treatment in stage 2 of the experiment is expected to reduce miscalibration. Stated in figure 6, stage 3 miscalibration mean is higher than stage 2 miscalibration; which contradicts with the expectation. This is to be discussed in the following parts.

The risk attitudes of the participants are distributed as below. In stage 1, 29% of the participants revealed risk neutral behavior, while 36% in total are risk averse and 35% are risk loving, most of them being in the slightly risk loving group. In stage 4, the stage after the treatment, the average risk attitude is expected to change with the update on their calibration levels. The direction of the change is to be analyzed in the following parts, depending on the change of the miscalibration. With a slight difference in the mean (-0.06) of the number of safe choices compared to stage 1,

Variable Observations Mean Median Std. Deviation Min Max

#of safe choicess1 137 5.29 5 1.99 0 10

#of safe choicess4 137 5.23 5 2.01 0 10

miscalibrations2 137 0.07 0.071 0.12 -0.206 0.361

miscalibrations3 137 0.14 0.126 0.15 -0.178 0.53

gender 137 0.45 0 0.5 0 1

age 137 37.8 36 13.27 15 70

26% of the participants revealed risk neutral behavior. 42% of them are risk averse (mostly slightly) and 32% are risk loving (mostly slightly).

Figure 7: Frequency of number of safe choices selected (gender=1 for female)

Overconfident behavior is observed in most of the subjects in both stages. 74% of the participants are overconfident in Stage 2, where 63% are over 0.05 point miscalibration level. 23% of the participants are between -0.05 and 0.05 point miscalibration interval, which can be interpreted as almost calibrated. In stage 3, the miscalibration is expected to get closer to 0. The overconfident individuals make up 85% of the data, where 71% of them are above 0.05 point miscalibration level. 20% of the participants are between the interval of -0.05 and 0.05 point miscalibration level, which contradicts with the expectation.

Figure 8: Miscalibration of participants in Stage 2 and Stage 3

#of Participants Frequency #of Participants Frequency

0-1 highly risk loving 4 0.03 6 0.04

2 very risk loving 1 0.01 3 0.02

3 risk loving 9 0.07 9 0.07

4 slightly risk loving 34 0.25 26 0.19

5 risk neutral 40 0.29 35 0.26

6 slightly risk averse 21 0.15 33 0.24

7 risk averse 13 0.09 12 0.09

8 very risk averse 3 0.02 3 0.02

9 highly risk averse 2 0.01 3 0.02

10 stay in bed 10 0.07 7 0.05

Stage 1 Stage 4

Risk Preference Classifications Number of Safe Choices

4.2 TREATMENT EFFECT AND MEAN COMPARISONS

In stage 2, providing the subjects with the actual results of the general knowledge test is expected to make them reevaluate their calibration level. In stage 3, the miscalibration is measured again to see if the treatment is successful in reducing the absolute value of miscalibration. Paired sample t-test is performed to see if the change in the mean is significant, and since the mean of the miscalibration in stage 2 is positive (see figure 6), a one tailed (lower tailed in this case) t- test is conducted. While the expectation was that the mean difference between miscalibration in stage 2 and stage 3 will be greater than 0, the p value for the lower tailed t- test is 1.00, which is significantly higher than 0.05. The null hypothesis can’t be rejected, and the treatment seems to increase the miscalibration effect while it should have decreased it.

Figure 9: t-tests on variables for mean comparison

To understand the drivers, the mean difference between the correct answers in stage 2 and stage 3 and the reported confidences of the answers for stage 2 and 3 are analyzed, since miscalibration is the difference between those two variables.

The difficulty level of the questions in stage 2 and stage 3 are selected in a way that they are the same difficulty level, thus a mean difference of 0 in correct answers are expected between the stages. A two tailed t-test is performed, and the null hypothesis that they are equal is rejected (p=0.00<0.05). The correct answers in stage 2 are 81.5% among subjects, while in stage 3, it drops down to 73.7%. Although the questions are solved by a subject pool of 671 students before and a very close amount of errors is reported (Tauber et al., 2013), the participants in this experiment performed worse in the second set. This could be explained by the number of participants, the age and education level distribution since Tauber et. al (2013)’s study was only consisted of university students.

- t df Pr (|T |>| t | ) Pr (T>t) Mean Difference Std. Error Difference

Miscalibration -5.35 136 1.0000 -0.068 0.013

Test Score 5.98 136 0.0000 7.81% 0.013

Reported Confidence Level 1.70 136 0.0461 1.05% 0.006

Even if the second set of the questions were harder for the participants, they would still be expected to be more aware of their capabilities and report a calibrated confidence level. The mean difference in self reported confidence between stage 2 and 3 is 1.05%. The p value for a lower tailed t test is 0.046, which is smaller than 0.05 and a decrease in self reported confidence in stage 3 is significant. Even if the self reported confidence goes down, it doesn’t decrease as much as the correct answers given, thus resulting in a higher miscalibration.

As a conclusion of the mean differences, the treatment does not work as expected.

To see if there is a change in the number of safe options chosen between stage 1 and stage 4, a two tailed t-test has been performed and the p value is 0.72, thus there is not a significant difference between the number of safe options in stage 1 and 4. This is also against the expected result. With the treatment, the number of safe choices selected by participants were expected to change in accordance with their updated miscalibration.

4.3 REGRESSIONS AND HYPOTHESIS

Before testing for the hypothesis, the main assumptions that are founded by other papers (Campbell et al., 2004; Goodie, 2003; Murad et al., 2016; Nosi’c & Weber, 2010) that risk loving behavior and overconfidence are correlated is tested. For the miscalibration, which is positive for overconfidence and negative for underconfidence, stage 2 data is used since it is the original miscalibration of the participants before treatment. For the risk loving/averse behavior, the number of safe choices made in the first stage were used, since they represent the participants original risk profile before the treatment. Number of safe choices are regressed on miscalibration, while being controlled for gender, age and education level. In figure 10, in the first column, the coefficient for miscalibration is 0.81 and it isn’t statistically significant. The hypothesis of this paper is built upon the expectation, inferred from previous studies (Campbell et al., 2004; Goodie, 2003; Murad et al., 2016; Nosi’c & Weber, 2010), that the number of safe choices would decrease as the miscalibration increases. However, the result of the regression, although not significant, gives a result that is opposite of the direction of the expectation, which is an increase in miscalibration would result in an increase on the number of safe choices.

To see if the control variables are correlated with the explanatory variable, a regression is run on miscalibration and gender, age and education level. Being female, as backed up with previous

research (Rosen, Tsai, & Downs, 2003), is significantly (p<0.05) correlated with miscalibration and has a reverse relationship with overconfidence, suggesting that males are more overconfident than females. The age is also significantly (p<0.01) correlated with miscalibration, and according to the regression results, the older an individual is, the more overconfident they will be.

Figure 10: Regression results with dependent variables number of safe choices and miscalibration, (gender is 1 for female)

To test for the hypothesis, two additional variables are created. The change in number of safe choices indicates the difference between the safe choices between stage 4 and stage 1. The change in miscalibration is the difference in miscalibrations between stage 3 and stage 2. The hypothesis is that a change in miscalibration would significantly influence a change in number of safe choices and a decrease in miscalibration should result in an increase in number of safe choices. To see if the hypothesis holds, change in number of safe choices are regressed on change in miscalibration. The results are shown in figure 11. According to the regression, there is not a significant correlation between the changes in miscalibration and number of safe choices. Also, even in an insignificant level, the direction of the relationship contradicts with the hypothesis. With the regression results, the hypothesis is rejected.

Explanatory vairables #of safe choicess1 miscalibrations2

miscalibrations2 0.811 (1.417) gender -0.020 -0.045** (0.361) (0.021) age -0.016 0.002*** (0.014) (0.001) education level 0.018 -0.008 (0.152) (0.009) constant 5.766*** 0.048 (0.810) (0.056) Observations 137 137 R-squared 0.010 0.109

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1

Figure 11: Regression with dependent variable change in number of safe choices

4.4 DISCUSSION

The experimental design is set in a way that for a valid hypothesis testing, the treatment effect in stage 2 should work. According to the mean comparisons and paired sample t-tests, the treatment doesn’t calibrate the participant’s judgment on their actual abilities. A possible explanation for that is because the correct answer percentage is relatively high for stage 2, the participants might have found the questions easy and they had high results. For the underconfident, the feedback mechanism works, from an average miscalibration of -0.08, in stage 3, the underconfident participants of stage 2 report on average 0.06 miscalibration, which is closer to the perfectly calibrated value 0. However, the treatment doesn’t work for the overconfident participants, as their miscalibration and overconfidence increases in stage 3. Pulford & Colman (1997) showed that feedback for hard questions helps to reduce overconfidence, whereas for easy and medium difficulty questions, it doesn’t. The reason for the overconfident staying overconfident could be explained by the difficulty level of the questions, and as an improvement to experimental design for future research, stage numbers can be increased to test for the confidence miscalibration and response to feedback for different level of difficulty of questions.

The correlation between overconfidence and risk loving behavior is not found. Since this is established in previous studies (Campbell et al., 2004; Nosi’c & Weber, 2010; Murad et al., 2016), one explanation of the failure of the expectation could be that because of the non-incentivized experimental design, the participants did not reveal their real risk attitudes on stage 1 and stage 4.

Explanatory variables Change in #of safe choices

Change in miscalibration 0.877 (0.804) Constant -0.125 (0.179) Observations 137 R-squared 0.004

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1

Holt & Laury (2002) showed that paying the subjects with hypothetical high awards vs real high awards would change their risk attitudes in the lottery task.

The hypothesis is rejected with the final regression results in Figure 11. Since the treatment effect did not work as expected, the hypothesis testing is expected to not give the hypothesized results. However; even when analyzed only for the underconfident, for which the treatment effect relatively worked and reduced miscalibration, the regression results still show that the number of safe choices increases with increased miscalibration, which is a major contradiction to the hypothesis. The failed hypothesis could again be explained with the non-incentivized risk attitude test and the small number of lotteries and confidence assessment questions.

5. CONCLUSION

The hypothesis that a change in the confidence level of an individual does influence their risk attitude is rejected. The correlation findings are not statistically significant. The paper also hypothesized that an increase in miscalibration would decrease risk aversion. The relationship couldn’t be found, as the increase in miscalibration, even if not in a significant level, result in an increase in number of safe choices in the lotteries. The expectation that the treatment would decrease miscalibration’s absolute value is not supported with the mean comparison of the miscalibration levels of two stages. The absolute value of miscalibration level significantly increased after the feedback treatment.

In the overconfidence/underconfidence literature, it is founded that feedback mechanism works for changing the confidence level (Arkes et al., 1987; Pulford & Colman, 1997). For the following studies, a more detailed feedback system could be experimented, including stages with different difficulty levels.

As discussed in the discussion part, for the future study, the experimental design could be re-evaluated and instead of an online experiment, an experiment in a behavioral lab could be conducted, with more variations of risk attitude elicitation tasks and with a greater number of questions to increase the sample size and increase the variation in difficulty. It would also be possible to conduct the experiment with incentives this time since the subjects wouldn’t have access to the Internet and other sources which enables them to cheat, which would increase the

external validity of the study, and make the subjects reveal their risk attitude and confidence level more accurately.

6. APPENDIX:ONLINE EXPERIMENT

Below is the screenshot of the experiment including the instructions, results page with randomized answers and the demographic questionnaire in the end.

7. REFERENCES

Allais, M. (1953). Le Comportement de l’Homme Rationnel devant le Risque: Critique des Postulats et Axiomes de l’Ecole Americaine. Econometrica.

https://doi.org/10.2307/1907921

Allais, M. (1991). An outline of my main contributions to economic science. Theory and Decision. https://doi.org/10.1007/BF00134634

Arkes, H. R., Christensen, C., Lai, C., & Blumer, C. (1987). Two methods of reducing overconfidence. Organizational Behavior and Human Decision Processes. https://doi.org/10.1016/0749-5978(87)90049-5

Broihanne, M. H., Merli, M., & Roger, P. (2014). Overconfidence, risk perception and the risk-taking behavior of finance professionals. Finance Research Letters.

https://doi.org/10.1016/j.frl.2013.11.002

Bruhin, A., Fehr-Duda, H., & Epper, T. (2009). Risk and Rationality: Uncovering Heterogeneity in Probability Distortion. SSRN. https://doi.org/10.2139/ssrn.1415975

Camerer, C., & Lovallo, D. (1999). Overconfidence and excess entry: An experimental approach. American Economic Review. https://doi.org/10.1257/aer.89.1.306

Campbell, W. K., Goodie, A. S., & Foster, J. D. (2004). Narcissism, confidence, and risk attitude. Journal of Behavioral Decision Making. https://doi.org/10.1002/bdm.475 Concina, L. (2014). Risk attitude & economics.

de la Rosa, L. E. (2011). Overconfidence and moral hazard. Games and Economic Behavior. https://doi.org/10.1016/j.geb.2011.04.001

Fehr-Duda, H., Schuerer, M., & Schubert, R. (2006). What Determines the Shape of the Probability Weighting Function? SSRN. https://doi.org/10.2139/ssrn.928376

Fischhoff, B., Slovic, P., & Lichtenstein, S. (1977). Knowing with certainty: The appropriateness of extreme confidence. Journal of Experimental Psychology: Human Perception and

Performance, 3(4), 552–564.

Goldstein, W. M., & Einhorn, H. J. (1987). Expression Theory and the Preference Reversal Phenomena. Psychological Review. https://doi.org/10.1037/0033-295X.94.2.236

Goodie, A. S. (2003). The Effects of Control on Betting: Paradoxical Betting on Items of High Confidence with Low Value. Journal of Experimental Psychology: Learning Memory and Cognition. https://doi.org/10.1037/0278-7393.29.4.598

Heukelom, F. (2015). A history of the Allais paradox. British Journal for the History of Science. https://doi.org/10.1017/S0007087414000570

Holt, C. a., & Laury, S. K. (2014). Chapter 4 – Assessment and Estimation of Risk Preferences. Handbook of the Economics of Risk and Uncertainty. https://doi.org/10.1016/B978-0-444-53685-3.00004-0

Holt, C. A., & Laury, S. K. (2002). Risk aversion and incentive effects. The American Economic Review. https://doi.org/10.1257/000282802762024700

Johnson, D. D. P., McDermott, R., Barrett, E. S., Cowden, J., Wrangham, R., McIntyre, M. H., & Rosen, S. P. (2006). Overconfidence in wargames: Experimental evidence on expectations, aggression, gender and testosterone. Proceedings of the Royal Society B: Biological

Sciences. https://doi.org/10.1098/rspb.2006.3606

Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica. https://doi.org/10.2307/1914185

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology. https://doi.org/10.1037/0022-3514.77.6.1121

Lattimore, P. K., Baker, J. R., & Witte, A. D. (1992). The influence of probability on risky choice. A parametric examination. Journal of Economic Behavior and Organization. https://doi.org/10.1016/S0167-2681(95)90015-2

Laury, S. K., & Holt, C. A. (2005). Further Reflections on Prospect Theory. Andrew Young School of Policy Studies Research Paper Series No. 06-11.

https://doi.org/10.2139/ssrn.893614

Levin, J. (2006). Choice under Uncertainty. Lecture Notes. https://doi.org/10.1016/0377-2217(89)90259-2

Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1977). Calibration of Probabilities: The State of the Art. In Decision Making and Change in Human Affairs. https://doi.org/10.1007/978-94-010-1276-8_19

Murad, Z., Sefton, M., & Starmer, C. (2016). How do risk attitudes affect measured confidence? Journal of Risk and Uncertainty. https://doi.org/10.1007/s11166-016-9231-1

Neale, M. A., & Bazerman, M. H. (1985). THE EFFECTS OF FRAMING AND NEGOTIATOR OVERCONFIDENCE ON BARGAINING BEHAVIORS AND OUTCOMES. Academy of Management Journal. https://doi.org/10.2307/256060

Nosi’c, A., & Weber, M. (2010). How riskily do i invest? The role of risk attitudes, risk

perceptions, and overconfidence. Decision Analysis. https://doi.org/10.1287/deca.1100.0178 Prelec, D. (1998). The Probability Weighting Function. Econometrica.

https://doi.org/10.2307/2998573

Pulford, B. D., & Colman, A. M. (1997). Overconfidence: Feedback and Item Difficulty Effects. Personality and Individual Differences. https://doi.org/10.1016/S0191-8869(97)00028-7 Quiggin, J. (1982). A theory of anticipated utility. Journal of Economic Behavior and

Organization. https://doi.org/10.1016/0167-2681(82)90008-7

Rabin, M. (2000). Diminishing Marginal Utility of Wealth Cannot Explain Risk Aversion. Choices, Values, and Frames.

and education. Medical Decision Making : An International Journal of the Society for Medical Decision Making. https://doi.org/10.1177/0272989X03258431

Tauber, S. K., Dunlosky, J., Rawson, K. A., Rhodes, M. G., & Sitzman, D. M. (2013). General knowledge norms: Updated and expanded from the Nelson and Narens (1980) norms. Behavior Research Methods. https://doi.org/10.3758/s13428-012-0307-9

Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty. https://doi.org/10.1007/BF00122574 Umoh, R. (2017). Why Bill Gates says you need confidence at a young age to achieve success. Vollmer, E., Hermann, D., & Mußhoff, O. (2017). Is the risk attitude measured with the Holt and

Laury task reflected in farmers’ production risk? European Review of Agricultural Economics. https://doi.org/10.1093/erae/jbx004

Von Neumann, J., & Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press. https://doi.org/10.1177/1468795X06065810

Wakker, P. P. (2010). Prospect Theory. Management. https://doi.org/10.1017/CBO9780511779329

Weinstein, N. D. (1980). Unrealistic optimism about future life events. Journal of Personality and Social Psychology. https://doi.org/10.1037/0022-3514.39.5.806

Wu, G., & Gonzalez, R. (1996). Curvature of the probability weighting function. Management Science. https://doi.org/10.1287/mnsc.42.12.1676