Effort-Based Decision-Making in Virtual Reality: What is Effort Worth?

1

Effort-Based Decision-Making in Virtual Reality: What is Effort Worth?

Bachelor Thesis

A method that combines experimental control and ecological validity could contribute to research intro effort-based decision-making and our knowledge of decision-making. Virtual Reality (VR) could possibly be such a method. In this study, 39 participants were subjected to an effort-based decision-making task in three conditions: 2D, 3D and VR. They were given a choice between two options: high and low effort, for corresponding monetary rewards that changed as a function of their previous decisions. Because monetary rewards were involved, participants’ financial situation was taken into account. Their cost-benefit analyses were studied by means of a quantified value, using a staircasing procedure. We expected subjects to perceive the effort as heavier in VR, and that this would be reflected in their cost-benefit analyses. Participants reported feeling more present in VR, and experiencing the effort as heavier as well. However, their cost-benefit analyses did not indicate this, nor when financial situation was taken into account. The extent to which participants felt present throughout the three conditions, did not correlate with their decisions.

3 Table of content Introduction……… P. 3 Method……… P. 6 - Participants……… P. 6 - Materials……… P. 7 - Procedure……… P. 8 o The task……… P. 8 o Instruction……… P. 11

o Point of indifference (poi) ……… P. 11

Results……… P. 13

- Participants………. P. 13

- Presence………. P. 13

- Perceived effort……….. P. 14

- Effect of condition and present financial situation on poi………. P. 14 - Correlations between poi, IPQ and perceived effort……….. P. 16

Discussion……… P. 18 Appendices……… P. 22 - Appendix 1……… P. 22 - Appendix 2……… P. 22 - Appendix 3……… P. 23 - Appendix 4……… P. 23 - Appendix 5……… P. 24 References……… P. 25

4

Effort-Based Decision-making in Virtual Reality: What is effort worth?

“Every nation in every region now has a decision to make. Either you are with us, or you are with the terrorists.” – George W. Bush. The former US president posed this statement as if it was an easy decision to make and two options was more than enough. Decision-making however, is not as easy as George W. Bush made it out to be. It is a phenomenon that has been studied by several disciplines such as psychology, neurobiology, biology and economics. Decision-making is supported by a variety of brain regions such as the striatum, the dorsolateral prefrontal cortex and the anterior cingulate cortex which are associated with cognitive control, the orbitofrontal, medial and ventromedial cortex which are associated with value based decision-making (Blakemore & Robins, 2012; Deco, Rolls, Albantakis & Remo, 2013; Gläscher, 2012). Decisions are the result of cooperation, or the exchange of information, between these regions of the brain (Heekeren, Marrett & Ungerleider, 2008) and often depend on cost-benefit analyses about risk and reward (Floresco, Onge, Ghods-Sharifi & Winstanley, 2008).

Someone who may have put more thought into the concept of decision-making is Plato, who stated that "a good decision is based on knowledge and not on numbers”. Plato was right in that we use our knowledge to make decisions. In certain situations, we use information about past outcomes in evaluating our options, and together with our motivation in that particular situation, we make a decision. Damasio and Saver (1991) conducted a case-study from which they concluded that the integration of social knowledge is required for social decision-making. On the basis of this study and further research, Damasio developed the Somatic Marker Hypothesis according to which the changes in physiological arousal, or somatic markers, that arise from our emotions are needed to guide our decisions (Damasio, Everitt & Bishop, 1996). Another study of Damasio’s and Bechara’s contributed much to our knowledge of decision-making too. Bechara, Damasio, Damasio, and Anderson (1994) measured physiological arousal of participants while they were performing a task; the Iowa Gambling Task (IGT). The goal of this task was to optimize winnings. Participants were given the choice out of four decks of cards, two of which yielded high monetary rewards and the other two low rewards. But the decks also held cards that led to penalties. The high reward decks held cards that led to high penalties and the low reward decks held cards that led to low penalties. Over the long run, the low decks led to gains (good decks) whereas the high decks led to losses (bad decks). In this particular experiment, the cost-benefit analysis would rationally eventually result in the participants choosing the good decks. The bad decks yield high rewards but also great risks and are therefore less profitable . Bechara and colleagues (1994) found that participants’ physiological arousal changed when they hovered over the bad decks, indicating a form of stress. The participants did not yet consciously experience what the good or bad decks were, but unconsciously they did. This was a confirmatory result for Damasio’s somatic marker hypothesis. If this is indeed the case, Plato was not completely right because one’s tendency to choose the good decks is in the first place determined by the emotions

5 that are expressed in physiological changes, not by knowledge. But more importantly, knowledge and numbers cannot really be approximated separately; numbers are in fact a form of knowledge with regard to past outcomes that are incorporated in a decision. It is important to apprehend that decision-making comes down to a trade-off between the costs and the benefits.

Different studies into decision-making focus on different cost-benefit analyses. A study may focus on how probabilities of obtaining a reward influence people’s decision whereas another study focuses on how certain risks influence decisions. Research methods have been developed that try to make decision-making tasks more realistic. One of those methods is effort-based decision-making, which takes the effort that has to be carried out in order to obtain a particular reward into account. One frequently used effort-based decision making task is the T-maze where rats are placed in a maze, shaped like a ‘T’. Once they have been habituated to the maze, the rats start at the bottom of the T, and are usually given a choice between two options: 1) The low effort option, in which the rats have to climb a low barrier (10 cm) to obtain a reward (food pallet). 2) The high effort option, here the rats have to climb a higher barrier (30 cm) and by doing so, they can obtain a higher reward (Deacon & Rawlins, 2006). The rats have to determine whether they are willing to exert more effort for a higher reward. A research method that is often used when studying humans is called ‘effort expenditure for reward task’, or EEfRT (Treadway, Buckholtz, Schwartzman, Lambert & Zald, 2009), and is used to study the willingness to exert effort for monetary rewards (Wardle, Treadway, Mayo, Zald & de Wit, 2011). In this method, participants are given a choice between two options and just as in the T-maze task, there is one low effort option that corresponds to a low reward and one high effort option that yields a higher reward. In this case the low effort options consists of 30 button presses with the dominant hand in seven seconds whereas the high effort options consists of 100 button presses in 21 seconds with the non-dominant hand (Treadway et al., 2012). Again, the participants have to analyze the costs with reference to the benefits. In this case the cost is effort, which is usually considered to be a burden, but we exert effort everyday to achieve goals (Kurniawan, Guitart-Masip, and Dolan, 2011). Research into effort-based decision-making is relevant because it gives us a more realistic idea of how decisions are made in everyday life. But although it is more realistic than gambling tasks, the research method still faces problems. Two limitations of effort-based decision-making have regard to external validity.

As is in much psychological research, there is a lack of ecological validity in effort-based decision-making. That is, the effort may not be representative for the effort one has to carry out in everyday life because the experiments are executed in an experimental environment. This is necessary because of confounding factors in the natural environment that may influence one’s decisions, but comes at the expense of ecological validity. A research method that allows researchers to make tasks more realistic but keep the experimental control, would therefore be a considerable asset to psychological experimentation. In decision-making, effort needs to be realistic, or representative for

6 the natural environment, to draw sound conclusions. The controlled environment complicates this. The use of Virtual Reality (VR) could possibly be a solution to this problem. It is important that someone feels present in the virtual environment. Then, one will experience the virtual environment as real, and will behave accordingly. ‘Presence’ is the ‘psychological sensation of being in the virtual environment instead of in the physical environment’ (Bohil, Alicea & Biocca, 2011). Riva and colleagues (2007) showed that the extent to which someone feels present in a virtual reality, influences one’s emotional state. This indicates that the level of presence may also influence one’s motivation which is important for decision-making. Veling, Moritz and Van der Gaag (2014) stated that physiological and psychological responses are very similar to those in the real world, based on research into patients suffering from schizophrenia. Slater, Spanlang, Sanchez-Vives and Blanke (2010) drew similar conclusions from the experiment they conducted in which the comparison was made between first person and third person in VR. Participants were in a virtual house with two people, one girl that sat on a chair and a woman that stood in front of her. The male participants were either bystanders when a series of events occurred, or they experienced those events from the girl’s perspective. Afterwards, the participants filled out questionnaires. The experiment showed that the participants that had been in the first person condition, that is, they witnessed the series of events from the girl’s perspective, had had significantly greater physiological responses than the participants that had been in the third person condition when they witnessed these events. In addition to the result that the participants felt present in the virtual environment, Slater and colleagues (2010) concluded that the participants in the first person condition felt like the virtual body was theirs. This property of VR has great potential for effort-based decision-making because the extent to which an effortful task is perceived as hard work, could be influenced by this. That is, the effort one has to carry out could be experienced as harder work compared to a representation on a screen where the participant can see the physical environment of the laboratory. Wilson and Soranzo (2015) reviewed studies from the last 20 years and concluded that VR has many advantages for psychological research such as presentation of stimuli in 3D, creation of complex scenario’s, allowing participants to respond in a more ecologically valid way et cetera. For these reasons and the previously mentioned study results, we may conclude that VR can contribute a great deal to research into effort-based decision-making and therefore to our knowledge of how decisions are made. VR is not only gaining popularity in psychological research. Six major technology companies are building or improving VR devices. When companies such as Sony, Samsung, Apple, Facebook, Microsoft and Google see the potential of VR, it will in all probability play a part in our lives. It is important for psychology to keep up with such developments because we may very well be exposed to it in the near future.

The second problem of external validity is the possibility to generalize the results from a sample to the population. This too, is an issue that can impair psychological research. In general, many studies make use of psychology students as participants, while psychology students, or university

7 students, are not representative for the population. Using EEfRT, Treadway and colleagues (2012) showed that individual differences can be found in the cost-benefit analysis due to differences in dopamine regulation. This is reflected in the differences in willingness to exert a certain amount of effort to obtain a monetary reward (Treadway et al., 2012). Westbrook, Kester and Braver (2013) found similar results concerning cognitive effort. With an effort-discounting task they showed that the cognitive effort someone is willing to perform, differs per person (Westbrook et al., 2013). Much like the differences in willingness to exert cognitive or physical effort, in economics, the perceived value of a reward differs per person as well (Read, 2007). The utility, or perceived value of a desired good, could also be different for monetary rewards throughout the sample or the population. That is to say, different people may perceive the value of a monetary reward in a psychology task differently. Doya (2008) claims that the effort someone is willing to do, depends on their physical and economic needs. One could induce that, in line with this statement, participants who generally have more need for money in everyday life, perceive the value of the abstract reward in a psychology task as more valuable. From this perspective it would be wise to take one’s financial situation into account, for it may influence one’s decisions in an effort-based decision-making task.

With the two limitation concerning external validity in mind, this study shall try to answer the question what the effect of a virtual reality representation of an effort-based decision-making task is on one’s perceived effort and whether one’s present financial situation is of influence on the decisions made. We expect the extent to which participants feel present to be Higher in VR than when the task is presented on a computer screen. As a result of this, we hypothesize people to perceive the effort as more realistic and heavier. We expect this to be reflected in people’s cost-benefit analysis. Furthermore, we expect less fortunate people to be willing to exert more effort than well fixed participants.

Method Participants

39 Dutch-speaking participants aged 19-28 took part is this experiment of which 27 were female and 12 were male (M = 23.18; SD = 2.97). As mentioned before, we regarded it as relevant to obtain a sample that consists not only of psychology students to ensure representativeness for the population. To acquire a more representative sample, we recruited participants on the street as well as through the university’s website through which participants could enroll themselves for our study. In total, 14 participants were not enrolled in courses at a university. Participants received a € 20 compensation or two research credits needed for psychology bachelors, or they volunteered. People younger than 18, older than 28, people suffering from Epilepsy or from other neurological conditions and people that quickly experience dizziness in rollercoasters or carousels, were not allowed to participate.

8 Materials

The task, which will be explained in more detail in the procedure section, was run in Unreal Engine 4 (for reference see Appendix 1). All three conditions were built in this same program. Participants were subjected to three conditions: a 2D representation of the task on the computer screen, a 3D representation of the task on a computer screen and a virtual reality representation of the task, using the Oculus Rift DK2 (for reference see Appendix 1). Participants were subjected to all three conditions, the order in which they were subjected, was counterbalanced. A head tracker was placed on top of the computer screen to keep track of where the participant was looking towards when in the virtual reality condition. An Xbox-one controller was used in all three conditions (for reference see Appendix 1).

Another computer was used in the other room. A number of questionnaires were conducted using Google-forms. The questionnaires, along with references, are shown in Box 1. The survey relevant for the present paper was a self designed survey on present financial situation. Questions such as ‘How much money do you have left every month after fixed expenses have been paid?’ and ‘Do you experience stress about your monthly expenses with regard to your monthly income?’ were scaled to determine each participant’s present financial situation. The options were: less than € 0 /€ 150/ € 300/€ 500/ € 800/€ 1200/€ 1800/€ 2400, more than € 2400’. The amount of stress participants experienced was determined using a four point Likert scale from ‘not at all’ to ‘very much’. Furthermore, questions concerning participant’s present and finished educational programs were asked. In total, the survey consisted of 11 questions. Participants also filled out the IPQ with which the extent to which they felt present was determined. Another survey was conducted that determined the extent to which participants perceived the effort as heavy.

Box 1. Questionnaires that the participants filled out.

Questionnaires References

Gaming experience Self designed questionnaire

Igroup presence questionnaire (IPQ) Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience of presence: Factor analytic insights. Presence: Teleoperators and Virtual Environments. Presence, 10, 266–281.

Immersive tendencies questionnaire Witmer, B. G., & Singer, M. J. (1998) Measuring presence in virtual enviroments: a presence questionnaire. Presence, 7, 225-240.

Listening span Vos, S. H., Gunter, T. C., Kolk, H. H. J., & Mulder, G. (2001). Working memory constraints on syntactic processing: An electrophysiological investigation.

9 Locus of Control Nowicki, S., & Strickland, B. R. (1973). A locus of

control scale for children.Journal of consulting and

clinical psychology, 40(1), 148.

Perceived effort Borg G. A. V. (1998) Borg’s rating of perceived exertion and pain scales. Champaign, IL: Human

Kinetics, viii-104.

Present financial situation Self designed questionnaire

SQUASH Wendel-Vos, G. W., Schuit, A. J., Saris, W. H., &

Kromhout, D. (2003). Reproducibility and relative validity of the short questionnaire to assess health-enhancing physical activity. Journal of clinical

epidemiology, 56(12), 1163-1169.

Temporal experience of pleasure Gard, D. E., Gard, M.G., Kring, A.M., & John, O.P. (2006) Anticipatory and consummatory components of the experience of pleasure: a scale development study. J. Res. Pers., 40, 1086–1102

Note: all questionnaires were either translated to Dutch or designed in Dutch.

Procedure

The task. Participants found themselves in a railway cart, in the 3D and VR condition, subjects sat in this cart and saw two levers in front of them. In the 2D condition, the participants did not see the railway cart. In every condition though, participants could see an abstract representation of the track that consisted of different bars with different colors. The colors represented the effort that had to be exerted to pass that particular part of the track. Red bars represented high effort, orange bars represented medium effort and green bars low effort. Above the representation of the two tracks, an abstract reward was shown that they would earn if they would complete the trial. The decisions were made on the basis of the effort they would have to exert and the reward they could obtain by completing the chosen track. Both tracks consumed the same amount of time, making the decision a trade-off solely on the basis effort and reward without confounding factors. Each condition consisted of 20 trials. The 2D representation is shown in Figure 1 with an explanation in the figure caption.

10 Figure 1. The 2D representation of the task before the choice has been made. On both sides of the screen,

the representations of the tracks are shown. The left tracks consists of two green bars and three red bars whereas the right consists of two red bars and three green ones. The high effort track is therefore the left option. This options also corresponds to a higher reward: 1087; whereas the low effort option corresponds to a lower reward: 913. The participant has to determine whether he or she feels like the reward is high enough for the amount of effort he or she has to exert to obtain that reward.

Participants were to put the second (or middle) knuckles of both their little fingers on the Xbox-one analog sticks with their fists pointing towards the screen, as though they were holding levers. By moving both joysticks to one side, participants could choose a track. When a participant had chosen a track, that track was presented in the middle of the screen and the progression was shown by a blue bar moving up as the participant got further along the track (see Appendix 2). In order to move forward, participants had to move the analog sticks in opposite directions at a certain speed. The higher the effort, the faster they had to move the analog sticks. At all times, the speed at which the participant moved the analog sticks, had to be enough to keep the power-bar (shown in the middle of the screen in Figure 1), above a certain threshold. When the power-bar got below this threshold, the railway card would slow down and eventually stop.

In the 3D condition (Figure 2), the task is the same with regard to the participants having to choose between two railways that hold different amounts of effort, corresponding to different rewards. But in the 3D representation, the railway cart could be seen and after the choice was made, the two railways could also be seen (see Appendix 3 for an image during the task). After the choice had been made (on the basis of abstract representations of effort and reward), the doors opened and the cart would move on the chosen track. The high effort parts of the track consisted of bushes that were on the

11 railway, the medium effort consisted of grass on the railway and low effort was empty railway (shown in Appendix 3).

Figure 2. The 3D representation of the task before the choice has been made. Here the power-bar is shown

in the railway cart. The representations of effort and reward are shown on the beams left and right.

In the virtual reality condition, the Oculus Rift DK2 was put on and participants were in the same environment as in the 3D condition. Now, however, they could look around by moving their head. In Figure 3, the VR representation is shown before the choice was made. An image of the track in VR in progress is shown in Appendix 4.

Figure 3. The VR representation of the task before the choice has been made. Note that this image could be

12 Instruction. Participants were first explained what the task would entail. When they had made their choice, participants had to exert effort by moving the analog sticks on the controller back and forth with the second knuckles of their little fingers placed on top of the analog sticks. Furthermore, they were told that when performing effort, the palms of their hands should be in the air, the second knuckles of their little fingers still on the analog sticks. When however, little effort had to be exerted and the power-bar was full, they were allowed to rest their palms on the X-box controller. This ought to have made the low effort parts of the track easier and the high effort parts more exhausting. To see whether they had understood the instructions correctly, the participants performed two trials in the 3D condition under supervision of the experimenter. After these two trials, the experimenter started the first condition, which had been determined randomly, and left the room. The participants were told that if they felt dizzy or nauseous in VR, they were to quit immediately.

Point of indifference (poi). To approximate the cost-benefit analysis, a staircasing procedure was used (Tversky & Kahneman, 1992). On each trial, the difference in reward between the two conditions was determined based on the difference in effort, multiplied by a valuation function. This function was updated dynamically as the task proceeded. It increased when the low effort option was selected, making the next high effort option more lucrative per unit effort and vice versa. That is, should the participant have chosen to exert little effort in the first 4 trials, the reward per unit effort would have been high. When the participant would however have been satisfied with difference in reward between low and high effort options on a trial, he or she would then have chosen the high effort railway as a result of which, the reward per unit effort would have decreased again. This method was used in order to establish an equilibrium that would provide an estimate of how valuable participants deem their exerted effort per experimental condition; point of indifference. The poi in each condition was determined by averaging the last five trials of that condition. (note that of the 20 trials, the last five were used for another experiment, the five trials that were used to calculate the poi were trials 11 to 15) The staircasing procedure and poi are explained in more detail in Figure 4.

13 Figure 4. In this figure, the reward per unit effort is displayed as a function of the number of trials. Two

scenario’s can be distinguished: scenario A and scenario B. In scenario A, the participant felt that the reward in the high effort option on the first trial, was not high enough with regard to the amount of effort she had to exert; she therefore chose the low effort option. As a result of the first choice, the reward per unit effort increased. This participant continued to choose for the low effort option until she felt that the reward was high enough for the effort that had to be exerted to obtain the reward. Now this participant chose the high effort options as a result of which, the reward per unit effort decreased. This participant now was no longer willing to exert the required amount of effort for that reward and chose the low effort option, after which, the reward per unit effort increased again. This participant had reached a point where she thought the amount of effort she would have to carry out to obtain a reward was good, and she therefore choose the high and low effort option in turns. The reward per unit effort then balanced around that particular value; point of indifference. In scenario A, the participant reached a high point of indifference which means that she wanted to exert little effort for a high reward. In scenario B however, the participant did not mind exerting a lot of effort in the first couple of trials. This participant reached a low point of indifference which means that this subject was prepared to exert much effort for a low reward.

14 Results

Participants. In total, 17 out of 39 participants were excluded. Two of which were not able to complete the VR condition due to nausea and quit immediately. The other 15 were non responsive. That is, they did not respond to the experimental manipulation. By not taking the effort they would have had to exert into account, their poi values balanced around zero , meaning that these participants only paid attention to the reward. These 17 subjects were excluded, and further analyses were done for the remaining 22 subjects. The mean scores on presence (IPQ), perceived exertion, cost-benefit trade-off (poi) and the standard deviation are shown in Table 1. The order in which participants were subjected to the conditions, were assigned the numbers ‘1‘ tot ‘6’ (the different orders are shown in Appendix 5).

Table 1.

Mean values of IPQ, perceived exertion and poi.

Condition Mean IPQ (SD) Mean perceived

exertion (SD) Mean poi (SD) ___________________ ___________________ ___________________ ___________________ 2D 21.910 (1.046) 1.523 (.264) 18.318 (3.538) 3D 34.680 (.944) 1.455 (.136) 17.000 (2.723) VR 49.500 (.660) 2.523 (.255) 19.182 (2.861)

Note: the standard deviations are computed over within subject normalized data.

Presence. To measure the extent to which participants felt present in the 2D, 3D and VR environments, subjects filled out the IPQ. To find out whether subjects felt more present in VR than in the other two conditions, a repeated measures analysis was conducted with the IPQ values for the three different conditions as within subjects factors and the order in which participants were subjected to the conditions as between subjects factor. Only one order was not normally distributed in de 2D condition, when participants were first subjected to VR, then 2D and then 3D; D(5) = .756, p = .034. The results should therefore be treated with caution. The other conditions and orders in which participants were subjected to these conditions were normally distributed. A significant main effect of condition on presence (by means of the IPQ) was found F (2,32) = 139.082, p < .001. The contrasts showed that the extent to which subjects felt present in the different conditions, differed between 2D and VR F (1,16) = 264.240, p < .001, as well as between 3D and VR F (1,16) = 153.855, p < .001, as is shown in Figure 5.1. Overall, participants felt more present in the VR condition than in the 2D and the 3D conditions, but subjects felt more present in 3D than in 2D as well. No significant interaction effect was found between the IPQ values in the different conditions and the order in which participants were

15 subjected to these conditions F(10,32) = .954, p = .500. This suggests that order did not influence the extent to which participants felt present in the different representations of the task.

Perceived effort. The other manipulation was the amount of effort participants had to exert in order to complete trials. Again, the order in which participants had gone through the conditions, was taken into account and the within subjects factor was the perceived effort over the three conditions; to what extent the participants reported to experience the effort as heavy. In this case, six of the 18 orders throughout the conditions (each condition consisted of six orders) were not normally distributed, (a non-parametric analysis should be conducted but that goes beyond the scope of this paper), hence we should interpret the results and the graph cautiously. A significant main effect of condition on perceived effort was found F(2,32) = 8.257, p = .001. The contrast showed that there were significant differences between 2D and VR F (1,16) = 8.286, p = .011, as well as between 3D and VR F (1,16) = 11.855, p = .003. As can be seen in Figure 5.2, the perceived effort was higher in virtual reality and the perceived effort in the 2D and 3D conditions did not differ much. No significant interaction effect was found F(10,32) = 1.005, p = .461, suggesting that the extent to which subjects experienced the effort as heavy, was independent of the order in which they were subjected to the conditions. Because there is no difference between the 2D and 3D conditions, it seems then that a physically more realistic environment does not necessarily contribute to the way effort is perceived. VR appears to have some characteristic that makes participants perceive the effort as heavier.

Effect of condition and present financial situation on poi. The main variables of interest in the present study were the poi and the present financial situation. The latter based on how much money participants had left after their fixed expenses had been paid and the amount of stress they experienced with regard to their current financial situation. Order was also taken into account. Originally, it was planned to take age into account as well, but due to the small sample size, age would decrease the power and is therefore not used in this analysis. We hypothesized that poi’s would differ within subjects and would be dependent on one’s present financial situation. Only for order ‘5’ in the 3D condition, were the values not normally distributed W(5) = .716, p = .014. But in the 3D and VR conditions, the poi values of females were not normally distributed. The results should thus be interpreted carefully. No significant effect of the condition was found on poi F(2,26) = 1.662, p = .209. No significant interaction effect of either condition and financial situation, or condition and stress about current financial situation was found; F(2,26) = 3.240, p = .055, F(2,26) = 2,311, p = .119, respectively. Although no interaction effect was found, there seems to be some sort of trend concerning the present financial situation (p = .055). It is however difficult to conclude that this can account for some of the variance since the error bars are so large, as is shown in Figure 5.3.

16 Figure 5. The error bars in these three graphs represent the standard error of the mean, as computed over within-subject normalized data. These error bars are therefore informative for within subject effects.

0 10 20 30 40 50 60 2D 3D VR IP Q va lu es Condition

Presence

perceived exertion

Cost-Benefit analysis

Mean IPQ values as indication of the sense of presence throughout the different conditions where higher values mean that participants felt more present in the particular condition.

Figure 5.2

Mean perceived effort values throughout the conditions on a scale from zero to three where zero entails that the effort is not perceived as heavy at all and three entails that the effort is perceived as very heavy.

Figure 5.3

Mean poi values throughout the conditions as an indication of cost-benefit analysis. High poi values indicate that participants wanted to carry out little effort for a high reward, suggesting that they perceived the effort as heavy whereas low poi values indicate that participants had no problem exerting a lot of effort for little reward.

17 Correlations between poi, IPQ and perceived effort. It is still interesting to study whether the differences in poi between conditions correlate with the differences in IPQ between the same conditions. It was expected that the more someone would feel present (IPQ), the more this person would experience the effort as heavy or real, leading to more frequent choosing of the low effort track (poi). The differences were calculated by subtracting the poi of the 3D condition from that of the VR condition, the poi of the 2D condition from that of the 3D condition and that of the 2D condition from that of the VR condition. The same was done for the IPQ values. A correlation was then run to compare the differences of the VR-poi minus the 3D-poi with the differences of the VR-IPQ minus the 3D-IPQ and the same for the other comparisons of conditions. These correlations were however not significant (see Table 2.1). Two exploratory analyses were done comparing the differences between conditions on reported perceived exertion to differences on reported sense of presence between these conditions (Table 2.2) and differences between reported perceived effort and poi between these conditions (Table 2.3); none of these correlations were significant.

18 Table 2.1. The differences in poi values between conditions compared to the differences on IPQ between the same conditions.

IPQ VR – IPQ 3D IPQ 3D – IPQ 2D IPQ VR – IPQ 2D _______________ ______________ _______________ _______________ _______________ Poi VR – poi 3D Correlation

Significance

.103 .638 Poi 3D – poi 2D Correlation

Significance

.205 .349 Poi VR – poi 2D Correlation

Significance

.282 .193

Table 2.2. The differences in perceived exertion between conditions compared to the differences on IPQ between the same conditions.

IPQ VR – IPQ 3D IPQ 3D – IPQ 2D IPQ VR – IPQ 2D _______________ ______________ _______________ _______________ _______________ Pe VR – pe 3D Correlation Significance -.235 .291 Pe 3D – pe 2D Correlation Significance .139 .537 Pe VR – pe 2D Correlation Significance .377 .084

Table 2.3. The differences in poi values between conditions compared to the differences in perceived exertion between the same conditions.

poi VR – IPQ 3D poi 3D – IPQ 2D poi VR – IPQ 2D _______________ ______________ _______________ _______________ _______________ Pe VR – pe 3D Correlation Significance .012 .959 Pe 3D – pe 2D Correlation Significance .174 .439 Pe VR – pe 2D Correlation Significance .211 .346

19 Discussion

“After a certain point, money is meaningless. It ceases to be the goal. The game is what counts.” Aristotle Onassis

In the present study, one’s perceived effort on an effort-based decision-making task as a result of 2D, 3D and VR representations and one’s present financial situation was studied. Although participants indicated that they perceived the effort as heavier in the VR condition and felt more present in this condition as well, no results were found that supported the hypothesis that participants would choose the high-effort track less often due to higher perceived effort in VR. There were no correlations found that would suggest that differences in cost-benefit trade-offs between two conditions correspond to differences in presence between the same conditions.

Before speculating what these results could mean for virtual reality and effort-based decision-making, let’s examine the limitations of the present study. Participants reported that they experienced the effort as heavier in VR and felt more present in VR as well, but that could not be derived from their poi values. As discussed in the results section, many participants were excluded because they did not choose low effort when there was no difference in reward, from which we can deduce that they did not find it a problem to exert much effort for a low reward. Because many participants did not base their decisions on the effort, according to the poi values, the credibility of the poi, could be questioned. But in order for the poi to serve as a credible indication of one’s cost-benefit analysis, a couple of factors need to be well designed. Because poi is a measure of cost-benefit analysis, the effort and the reward should be designed so that people will actually compare the high and low effort options with regard to the corresponding rewards. First, the high effort parts of the track should, to some extent, tire the participants out. Should the high effort parts of the track be heavier than the medium and low effort parts, but still not heavy enough to tire someone out, the manipulation will not work. If this is accomplished, people will take the extent to which they are tired into account when making their decision. Second, the reward should be high enough for people to be willing to exert more effort for a higher reward, but not too high, making the decision only dependent of reward and not of effort. Let’s examine these two factors in the present study. The poi values suggest that 15 participants did not take effort into account at all. Hence, the effort has to be made heavier in future research. As for the reward, in the present study, participants were told that they could earn up to 50 eurocents, while the rewards on the screen were represented in numbers with three or four digits. Doya (2008) stated that decisions are dependent on one’s psychological and economic needs. This should not be different for abstract rewards but maybe rewards represented in the local currency come more natural to wanting than abstract monetary rewards. To make the reward more realistic, it could be represented as Eurocents instead of these abstract numbers. The reward could for example differ between 3 eurocents

20 and 5 eurocents. This way, the realistic monetary reward could appeal more to one’s desire to obtain money as they have in their everyday life. So realistic rewards may appeal more to the economic needs that Doya (2008) spoke of.

When studying the influence of one’s financial situation on their decisions in an effort-based decision-making task, it is important that the rewards used are proportional. That is, the reward should be high enough to tempt less wealthy participants to exert more effort for a reward that is a little higher than the low effort option. Even for someone who does not have a lot to spend, 50 eurocents still is not much. In future research the reward could be higher so that participants may be more inclined to exert more effort for a higher reward. The reward should however not be too high, making it appealing to always choose the option of higher reward, independent of one’s financial situation and the effort that has to be carried out to obtain that reward. It is important to realize that the reward should be high enough for someone to want to exert effort to obtain it, but not too high so that participants will not pay attention to the effort that has to be carried out. This brings us back to the previous paragraph; in order to draw conclusions from poi values, the effort and reward have to be in balance.

Another limitation is the number of participants with regard to the cost-benefit trade-off. Should the poi be much greater in de VR condition than in the other conditions, within-subject differences should be observable, even when only 22 subjects participated. In a small sample, five subjects with contradictory poi’s, could already change the outcome. Of course, assumptions are more easily violated when using small samples as well, leading to difficulty in interpretation results or nonparametric testing. Should the differences in poi’s between VR condition and the 2D and 3D conditions be subtle, more participants are required to demonstrate whether there is an effect or not.

A factor that could also be of importance, is the extent to which the task is, or the conditions are, mentally exhausting. Some participants reported that they found the 2D condition rather boring. This could affect their decisions in two ways. 1) Due to the exhaustiveness of the task in 2D, they experience the effort as heavier and would therefore choose the low effort option more frequently. This can be derived from the mean poi values, for the mean poi in the 2D condition is higher than in the 3D condition. 2) On the other hand, some participants reported that they had chosen the high effort option simply because it was a little more fun, which would lead to poi’s around zero. This is not consistent with the results, but because the results consist of a lot of noise and many subjects were excluded, we cannot reject this hypothesis on the basis of these results. According to this line of reasoning, the VR representation would be easier because it is more fun. This leaves us with a dilemma; should the task be more realistic to have a higher ecological validity or should it not be more realistic because it would be more fun, and would therefore be less mentally exhausting than the other conditions? Slater and colleagues (2010) concluded that by experiencing a series of events in first

21 person in VR, not only the environment but also the body is experienced as more real, compared to experiencing these events as a bystander in VR. Hence, the addition of a body and moving arms, corresponding to the participant’s movement, could contribute to the sense of presence. The arms seen in VR, could feel as though they are the participant’s own arms, much like in Slater’s experiment (Slater et al., 2010). But when the sense of presence should increase, emotions and maybe motivation could increase as well (Riva et al., 2007). This could affect the participant so that he or she is willing to exert more effort. This leaves us at the same dilemma.

Now what does this study mean for future research into effort-based decision-making, VR and applications of VR? There was no effect of condition on the perceived effort when using poi as a measure. However, the participants declared feeling more present in VR and experienced effort as heavier as well. Although this shows prospect, we need another measure than introspection. Ideally, we would use some sort of physiological measurement that, according to Damasio’s Somatic Marker Hypothesis (Damasio & Saver, 1996), would indicate the physiological arousal that leads to a decision. It is however difficult to obtain knowledge about motivation through measurements of physiological arousal because physical effort is being carried out in this experiment. As mentioned before, Doya (2008) stated that the effort that someone is willing to exert, is dependent on that person’s economic and physical needs. Using EEG, distinct oscillations can be observed, associated with motivation (Knyazev, 2007). These oscillations positively correlate with motivation; when motivation increases, the amplitudes of the waves increase as well, or vice versa (Knyazev, 2007). Riva and colleagues (2007) stated that presence could influence one’s emotions which could suggest that presence influences motivation as well. Using EEG, the motivation can be measured throughout the conditions. Having knowledge about motivation is valuable because it may explain certain (missing) effects. For example, the effort could be perceived as heavier in VR, but when motivation is higher as well, people may still be willing to exert much effort. This measure would also be a valuable objective addition to reports about present financial situation and stress about that situation. This could contribute to knowledge about financial motivation in decision-making which is relevant for many academic fields, from psychology tot economics.

Before concluding that perceived effort does not increase as a result of a VR representation of a task, we ought to adjust reward and effort well, so that poi can really suffice as a measure of cost-benefit analysis. Only then, can we really conclude whether there is an effect of VR on the perceived effort, not measured by introspection. This acquired insight ought to be implemented in future research, along with other factors that could improve future studies. We should also consider that there may not be an effect at all. Maybe effort is simply not perceived as heavier when the environment is experienced as more realistic. This however does not mean that VR could not be a valuable asset to

22 psychological research. Some studies have already shown that VR could have clinical applications. It has already been used in psychiatric disorder, pain management and neurorehabilitation (Bohil et al., 2011). But VR has shown to contribute to experimental psychology as well, for example in perceptual research (Wilson & Soranzo, 2015). Furthermore, Six major technology companies all are involved in VR. If these companies see the potential of VR, it is bound to become a part of our everyday lives, and everything that contributes to the environment that people live in, psychology is interested in.

You can design and create, and build the most wonderful place in the world. But it takes people to make the dream a reality – Walt Disney.

23 Appendices

Appendix 1. materials that were used.

Material reference

Unreal Engine 4 Epic Games. (2012). Unreal Engine 4. Retrieved from https://www.unrealengine.com/unreal-engine-4 (software).

Oculus Rift DK2 Oculus VR. (2014). Oculus DK2. Retrieved

from https://www.oculus.com/dk2/.

X-Box one controller Microsoft. (2013). Xbox One Wireless Controller. Retrieved from www.xbox.com/en-US/xbox- one/accessories/controllers/wireless-controller.

Appendix 2. The 2D representation of the task once the choice has been made. The green bar at the bottom of the screen is the power-bar (which has to be above the threshold in order for the cart to keep moving). The Blue bar represents the progression; The parts that the participants has covered so far.

24 Appendix 3. The 3D condition after the choice has been made. The participant in this figure currently moves through grass, which is a medium effort part of the track. Further along the track, bushes can be seen that represent a high effort part of the track.

Appendix 4. The VR condition after the choice has been made. The participant in this image moves through bushes, which represent the high effort part of the track. Note that this image is the representation on the monitor, in this condition, the participants did see one image through the Oculus Rift DK2.

25 Appendix 5. The different orders in which the participants were subjected to the three conditions. The orders were counterbalanced.

Number assigned to order Order (first, second, third) _______________________________________ 1 ________________________________________ 2D, 3D, VR 2 2D, VR, 3D 3 3D, 2D, VR 4 3D, VR, 2D 5 VR, 2D, 3D 6 VR, 3D, 2D

26 References

Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007). The role of the dorsal striatum in reward and decision-making. The Journal of Neuroscience, 27(31), 8161-8165.

Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1), 7-15.

Blakemore, S. J., & Robbins, T. W. (2012). Decision-making in the adolescent brain. Nature

neuroscience, 15(9), 1184-1191.

Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature

reviews neuroscience, 12(12), 752-762.

Damasio, A. R., Everitt, B. J., & Bishop, D. (1996). The somatic marker hypothesis and the possible functions of the prefrontal cortex [and discussion].Philosophical Transactions of the Royal Society B: Biological Sciences,

351(1346), 1413-1420.

Deacon, R. M., & Rawlins, J. N. P. (2006). T-maze alternation in the rodent. Nature Protocols, 1, 7-12. Deco, G., Rolls, E. T., Albantakis, L., & Romo, R. (2013). Brain mechanisms for perceptual and reward-related decision-making. Progress in Neurobiology, 103, 194-213.

Doya, K. (2008). Modulators of decision making. Nature neuroscience, 11(4), 410-416.

Floresco, S. B., Maric, T. L., & Ghods-Sharifi, S. (2008). Dopaminergic and glutamatergic regulation of effort-and delay-based decision making. Neuropsychopharmacology, 33(8), 1966-1979.

Floresco, S. B., Onge, J. R. S., Ghods-Sharifi, S., & Winstanley, C. A. (2008). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cognitive, Affective, & Behavioral

Neuroscience, 8(4), 375-389.

Gläscher, J., Adolphs, R., Damasio, H., Bechara, A., Rudrauf, D., Calamia, M., ... & Tranel, D. (2012). Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex. Proceedings of the National Academy of Sciences, 109(36), 14681-14686.

Heekeren, H. R., Marrett, S., & Ungerleider, L. G. (2008). The neural systems that mediate human perceptual decision making. Nature reviews neuroscience, 9(6), 467-479.

Knyazev, G. G. (2007). Motivation, emotion, and their inhibitory control mirrored in brain oscillations. Neuroscience & Biobehavioral Reviews, 31(3), 377-395.

Kurniawan, I. T., Guitart-Masip, M., & Dolan, R. J. (2011). Dopamine and effort-based decision making. Frontiers in neuroscience, 5.

Pizzagalli, D. A., Holmes, A. J., Dillon, D. G., Goetz, E. L., Birk, J. L., Ryan Bogdan, A. M., ... & Fava, M. (2009). Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. The American journal of psychiatry, 166(6), 702-710.

Read, D. (2007). Experienced utility: utility theory from Jeremy Bentham to Daniel Kahneman. Thinking &

27

Riva, G., Mantovani, F., Capideville, C. S., Preziosa, A., Morganti, F., Villani, D., ... & Alcañiz, M. (2007). Affective interactions using virtual reality: the link between presence and emotions. CyberPsychology &

Behavior, 10(1), 45-56.

Rolls, E. T., & Grabenhorst, F. (2008). The orbitofrontal cortex and beyond: from affect to decision-making. Progress in neurobiology, 86(3), 216-244.

Rushworth, M. F., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011). Frontal cortex and reward-guided learning and decision-making. Neuron, 70(6), 1054-1069.

Saver, J. L., & Damasio, A. R. (1991). Preserved access and processing of social knowledge in a patient with acquired sociopathy due to ventromedial frontal damage. Neuropsychologia, 29(12), 1241-1249.

Slater, M., Spanlang, B., Sanchez-Vives, M. V., & Blanke, O. (2010). First person experience of body transfer in virtual reality. PloS one, 5(5), e10564.

Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315(5811), 515-518.

Treadway, M. T., Buckholtz, J. W., Cowan, R. L., Woodward, N. D., Li, R., Ansari, M. S., ... & Zald, D. H. (2012). Dopaminergic mechanisms of individual differences in human effort-based decision-making. The

Journal of Neuroscience, 32(18), 6170-6176.

Treadway, M. T., Buckholtz, J. W., Schwartzman, A. N., Lambert, W. E., & Zald, D. H. (2009). Worth the ‘EEfRT’? The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS

One, 4(8), e6598.

Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5(4), 297-323.

Veling, W., Moritz, S., & Van der Gaag, M. (2014). Brave New Worlds—Review and Update on Virtual Reality Assessment and Treatment in Psychosis. Schizophrenia bulletin, sbu125.

Wardle, M. C., Treadway, M. T., Mayo, L. M., Zald, D. H., & De Wit, H. (2011). Amping up effort: effects of d-amphetamine on human effort-based decision-making. The Journal of Neuroscience, 31(46), 16597-16602. Westbrook, A., Kester, D., & Braver, T. S. (2013). What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference. PLoS One, 8(7), e68210.

Wilson, C. J., & Soranzo, A. (2015). The Use of Virtual Reality in Psychology: A Case Study in Visual Perception. Computational and Mathematical Methods in Medicine.