BACHELORPROJECT
Effort-based Decision Making, Heart Rate
Variability & Reward Sensitivity in Virtual Reality
Author: Maria Vasileiadi Supervised by: Jasper Winkel
Date: 31-05-2016 Student number: 10313729
Abstract
Decision making processes can be studied using an effort-based decision making (EBDM) paradigm. Individual differences in reward sensitivity might influence the choices made in EBDM. Dopaminergic systems are thought to be related to reward sensitivity as well as to heart rate variability (HRV). Therefore, it is hypothesized that reward sensitivity in an EBDM paradigm could be related to HRV. In this study, a virtual reality (VR) paradigm is used. This method is thought to have a higher ecological validity than standard psychological tools. In a within-subjects design 51 participants performed a novel EBDM task in three different conditions. The degree of realism of reward and effort was manipulated in these conditions. The results indicate no effects of the different degrees of realism on EBDM. There was no relationship found between HRV and reward sensitivity. The sense of presence was higher in realistic effort condition. This study shows that it is possible to manipulate levels of presence and realism in VR.
Introduction
Decision making is an everyday activity. In the rich complex social life people have, they are constantly forced to choose between options. These can range from minor to very significant decisions, such as what to wear on a night out and to order food in or go out, to more far-reaching, such as whether to continue your education or to drop out of school. Irrespectively of the nature of the decision, it is a cognitive process that can be thought of as a problem-solving activity. This activity is ended by a solution that is supposed to be satisfactory. A major part of this process is the evaluation of the different options. The alternatives differ in terms of evaluative criteria. Depending on these criteria some of them might look more appealing while others might seem less tempting.
The behavioural research on decision-making is concerned with how people make these choices. People can have several motives for choosing one or another option. In general, there is an agreement in the field of psychology on the importance of two fundamental motives for decision making. Typically, people want to reduce uncertainty or pain and are motivated to attain pleasure (Bentham, 1948). Besides these motives there are additional aspects that can influence the process of decision making. For example, previous research has found that individuals differ in how much they are willing to risk, work or wait for a certain reward (Holt, Green, & Myerson, 2003). Another aspect, that can cause variability between individuals in decision making, is the differences in decision processing and confidence judgments (Blais, Thompson, & Baranski, 2005). The presence of these different aspects that affect decision-making gives the opportunity to consider the process in detail.
Decision-making processes can be studied using different paradigms. An example of a framework that can be used is called Effort-Based-Decision Making (EBDM). Effort is usually experienced as a strain and a burden, but to achieve a desired goal people are willing
to go through the effort. The EBDM framework takes into account the amount of effort people are willing to perform in order to obtain a certain goal or reward that is associated with it (Kurniawan, Guitart-Masip, & Dolan, 2011). This paradigm is frequently studied in animals, especially in rats.
It is possible to study EBDM in rats by using a T-maze task (Salamone, Cousin, & Bucher, 1994). In these studies, the rats were firstly deprived of food and were then exposed to barriers in a T-maze. The barriers were placed in both arms. Behind the barriers the rats would find food. The arms differed in the amount of barriers they contained. The high density (HD) arm contained four barriers, and the low density (LD) arm had two barriers placed. The HD arm required high effort and the LD arm required low effort. Behind the barriers in the HD arm there was more food placed than in the LD arm. The rats were placed in the start box and could choose only one arm on each trial. This set up created two conditions: high effort/ high reward and low effort/low reward. The rats in this experiment tended to go for the high effort/high reward option more often.
An adapted version of this paradigm is needed to study EBDM in humans. In a fMRI study participants were shown a cost-benefit cue on the screen (Croxson et. al., 2009). This indicated the trial type they were experiencing. They were then instructed to use a trackball to click on square shapes that showed up on the screen to make them disappear. The amount of effort was defined by the number of shapes needed to remove and the reward value was based on the points they earned each trial. In this study the participants were assigned to the conditions, they did not voluntarily decide the amount of effort. As the decision making process is a crucial part in studying EBDM, based on this study, no conclusions can be made about EBDM in humans.
To study how effort affects actual choice other stimuli were used in different fMRI study (Kurniawan et. al., 2010). The participants made a series of consecutive choices
between high and low effort options that were indicated by visual stimuli on the screen. The low effort option implied holding a grip device, while the high effort option was an effortful grip. The factors of force and reward were varied (low and high). For the holding option participants were rewarded the minimum. The results showed that the level of force (amount of effort) influenced the likelihood of choosing an effortful grip, with participants being less likely to choose the grip option when level of force is high.
Along with behavioural studies that involve EBDM paradigms there is a growing body of research that considers the biological mechanism behind the process. Recent studies have considered brain mechanisms and neurotransmitters and their relation to decision making. In particular, the role of dopamine is reviewed. Previous research has shown that dopamine in the midbrain is involved in decisions about effort and delay in rats. Depletion of dopamine discourages rats to choose effortful alternatives (Denk et. al. 2005; Phillips et. al., 2007). Furthermore, evidence is found that dopamine and adenosine in the nucleus accumbens interact to regulate effort-related functions (Salamone et. al., 2009). Recently, studies in humans reported an effect of dopamine on EBDM as well. There is evidence that dopamine is needed to overcome the costs of high effort to obtain a desired goal in humans (Kurniawan et. al., 2011). Considering dopamine activity as a key role in decision making, research has found individual differences in these mechanisms. These studies highlight the role of dopamine as underlying individual differences in EBDM paradigms (Treadway et. al., 2012).
Aside from effects on effort, previous research has also concentrated on the role of dopamine in reward. In research with rats, dopamine treatment was found to bias in choosing large rewards in contrast to treatment with a receptor antagonist (Bardgett et. al., 2009). Further, results from drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. From a review of the literature on the role of
dopamine in reward it appears that dopamine contributes to incentive salience and is necessary for a feeling of ‘wanting’ (Berridge, 2007). Taking all these finding into account, it could be implied that dopamine systems play a key role in EBDM because of their effects on reward and effort.
Individual differences in dopaminergic systems may reflect differences in sensitivity to reward. This involves motives to approach goals. The psychological system believed to underlie approaching motivations is called the behavioural activation system (BAS; Gray, 1981). This system is thought to be responsive to signals of reward. When it becomes active people are motivated to move towards the goal. Greater BAS sensitivity is thought to reflect more motivation to perform effort that is goal-directed and more positive emotions when confronted with the prospective reward. It is possible that these differences can be reflected by changes in the activity of the autonomous nervous system (ANS), by means of the effects of dopamine on the parasympathetic and sympathetic systems. These individual differences in sensitivity to reward may be reflected by changes in heart rate. It could be expected that the heart rate will increase as a result of the expectation of a high reward and thus the effect of dopamine.
Heart rate variability (HRV) can be stated as the variation over time of the period between heartbeats. HRV decreases as heart rate intensity increases. The signal of HRV can be used to get an understanding of the state of the ANS. The frequency and amplitude variations in HR are caused by the regulation of the autonomic control systems underlying the response (Saul, 1990). Increased activity of the sympathetic nervous system causes cardio-acceleration, which means that the consecutive heart beats will follow each other more rapidly (Acharya et. al., 2006). Dopamine activity is associated with increased activity of the sympathetic nervous system. This can be seen in the HRV signal, as the heart rate will be more intense.
Considering individual differences in dopaminergic systems and reward sensitivity it is conceivable that these differences could affect HRV. The activation of the sympathetic nervous system will increase the heart rate and decrease the HRV. In addition, it can be thought that differences in reward sensitivity might affect decision making behavior in EBDM paradigms. As all these described elements can influence human behavior, the goal is to take as many of them into account while doing research. The methods used in psychological research to study processes like decision making are meant to produce results that can be generalized to real-life situations. However, most tasks that are used in research are performed in labs and are therefore not representative for real-life situations. Computer tasks make use of computer-based abstractions of real-world objects or situations. These tasks generally have a high internal validity but the ecological validity is highly debatable. To make inferences about real-life situations in regard to psychological decision making the tasks and environment should match real-life situations as much as possible. An option to meet this criterion is to study the phenomena in the field. One of the drawbacks of field studies is the lack of standardization. The upcoming technological development of virtual reality might provide a solution for this problem.
Virtual reality (VR) was promised to be the next big thing in the 1990s, but the technology back then did not have enough to offer. With the advances in technology the rise of VR is a fact. Although there is high interest from in VR from the gaming industry, the applications of the technology are not limited to gaming. VR might also be a useful tool in psychological research. It could solve the enduring tension between experimental control and ecological validity (Bohil, Alicea, & Biocca, 2011). In VR the environment is simulated by a computer and it gives the user the experience of being present in that environment. The computer generates a 3-dimensional environment. The method of VR gives experimenters the ability to control key variables in a contextually rich scenario. The goal of VR systems is to
create the illusion of a believable simulation of reality (Biocca & Levy, 1995). This is possible because of the ability to simulate multiple sensory channels at once. For example, it is possible to provide auditory stimuli along with visual stimuli. The combination of the controllable variables and the highly realistic experience provide a high level of ecological validity and experimental control. VR already has proven to be useful in clinical studies as a tool for treatment of different phobias, for example arachnophobia (Parsons & Rizzo, 2008). The results are promising for further research.
A concept that plays an important role in the effectiveness of VR is immersion. To achieve full immersion one needs to take into account the breadth of information and the depth of information. This includes the resolution, quality and effectiveness of the audio visuals and the number of sensory present at a time (Desai, Desai, Ajmera, & Mehta, 2014). The virtual environment is presented on a head-mounted display (HMD) that is closed off around the eye region. Stereoscopic vision provides depth perception and the position of the head is constantly tracked. In addition, the position of the user in the virtual environment is tracked. As a result, the user can, for example, lean forward to have a closer look. These aspects of VR add to a full immersive experience.
Besides immersion another concept that is of influence on the experience in the virtual environment is ‘presence’. This concept refers to the feeling of ‘being there’. When one feels and behaves as if present in the environment created, there is a ‘feeling of presence’ (Sanchez-Vives & Slater, 2005). Although this concept is of importance in VR, it is not limited to a simulated virtual environment. An experience of presence could also be accomplished by reading a book, or watching a movie. In most research sense of presence is measured by subjective questionnaires that the users fill in after the VR experience. To accomplish a feeling of presence in VR there are different factors that need to be considered. Previous research has shown that when head tracking and stereoscopic cues were provided,
the reported level of presence was higher (Ijsselstijn et. al., 2001). In addition, a lower latency between movements of the head and the updating of the display was also found to influence reported sense of presence positively (Meehan, Razzaque, Whittion, & Brooks, 2003).
Overall it could be concluded that a more realistic experience in VR will add to immersion and feeling of presence. The disparity between the VR experience and the real world will be minimized.
In the present study an EBDM paradigm is examined in a virtual environment. Since immersion and the sense of presence in VR make the experience more realistic, this effect may expand to the degree of realism of the effort and reward. Thus in this experiment participants complete the EBDM task in a VR environment. In one condition the effort representation is realistic while the reward is represented in 2D. In the other two conditions there is a 2D representation of effort, combined with respectively a realistic representation and a 2D representation of reward.
This study focusses on two main objectives. One of them is to examine if the degree of realism has an effect on the perceived effort. When the degree of immersion has an effect, this would suggest that there is more perceived effort when the representation of effort is realistic. This, in turn will result in higher levels of reward needed for participants to be willing to perform the same amount of effort compared to the 2D effort representation.
Another goal is to examine the impact of the different representations of rewards on choice behavior. When rewards are represented realistically, they are expected to be valued higher in comparison to the 2D representation. It is hypothesized that, as a result, the HRV will be higher during valuation of the rewards when they are more realistic. Simultaneously, as a result of the effects of dopamine on the heart rate and relation to reward sensitivity, the value of HRV is thought to be affected by the reward sensitivity of individuals.
Method Participants
In total 51 people participated in the experiment. Of them, 37.3 % were female, 82.4% were students and the average age was 23 (SD = 3.2). Participants were not paid for their attendance, but could win payment of the mean amount of coins collected over conditions, with each virtual coin representing 10 eurocents. One participant was randomly chosen afterwards and awarded the money. Most participants were recruited by members of the project group out of their group of friends and family. In addition, recruitment was done through flyers in the lab of the university.
Material Input Device
For this experiment a custom-made input device was created. The goal was to make participants expand considerable effort and to enhance one’s immersion by mimicking the virtual environment (VE). In the VE, participants were driving a mine cart, similar to a human powered handcar. To mimic the motion made operating such a cart, a bicycle pump was selected as the base of the input device. The ‘pumping’ motion resembled the motion of operating a handcar. In addition, the (air) resistance felt while operating the pump, makes it an effortful activity.
In order to make this bicycle pump an appropriate input device for a computer, a strip of aluminum was attached to the handle of the pump. The strip covered the entire length of the pump. The strip moved along with the motion. Over the aluminum strip, a computer mouse (Logitech G300) was fixed. The participants were able to reach the mouse with their index fingers to make a choice during the task. When moving the bicycle pump-handle up and down, the aluminum strips moved similarly along the fixed computer mouse. In this manner, the computer mouse registered the motions of the pump.
Questionnaires:
To assess individual differences in reward sensitivity the BIS/BAS scales were used. These scales are believed to reflect the behavioral inhibition system and the appetitive aversive system (BIS and BAS, Carver and White’s, 1994). The self-report questionnaire consists of 20 items using four-point scales (1 = strongly disagree to 4 = strongly agree) that can be divided into the two main scales: BIS (7 items) and BAS (13 items). The BAS scale can be further divided into three subscales: BAS-Reward (5 items), BAS-Fun (4 items) and BAS-Drive (4 items). An example of an item in the BAS-Reward scale: ‘When I get something I want, I feel excited and energized’. For this study the Dutch version of the BIS/BAS scales was used. The internal consistency of the scales was previously found to be modest to good. Cronbach’s alpha ranged from .59 to .79 for the various scales (Franken et. al., 2005).
To measure the sense of presence for the different conditions the Igroup Presence Questionnaire (IPQ) was used (Schubert, Friedmann, & Regenbrecht, 2001). This self-report questionnaire consisted of 13 items using five-point scales (1 = completely disagree tot 5 = completely agree). All 13 items were evaluated for each condition, resulting scores to range from 13 tot 65 for each condition. A higher total score represented a higher sense of presence and feeling of immersion. An example of an item: ‘I had the feeling the virtual world was ‘real’ during the task’.
An additional number of questionnaires with wide ranging subjects were obtained but were not used in this study.
Physiological Recording
While participants performed the task their heart rate was measured by electrocardiography (ECG). Three Ag/AgCl monitoring electrodes were attached via the lead-2 placement technique. The signal was digitized at 1000Hz. The ECG recording from
the time between the stimulus onset and the decision making for every trial was used for analysis. The data was processed using VSRRP98 v 7.29 (developed by the Technical Support Group of the psychology department from the UvA). To deal with the main interference a filter of 50 Hz was applied to remove the noise. The automatic beat-detection of VSRRP is highly robust and accurate in the presence of ECG artifacts. The inter-beat intervals were taken from the ECG signal and the HRV was derived from the variance of these intervals.
Task12
Participants completed 3 conditions in counterbalanced order, with conditions consisting of 13 trials each, all implemented in virtual reality. Participants were instructed to power a mine cart over a track by making pumping motions with a bicycle pump. At the beginning of each trial, participants were given a choice between a high effort/ high reward (HE/HR) route or a low effort/ low reward (LE/LR) route. Color-coding in presentation of the tracks informed participants of the amount of effort a route would require; green sections of the track required no pumping input, orange sections of the track required medium effort and red sections of the track required high effort pumping. In the baseline condition, the different route options and coin rewards were displayed on two screens, one on each side of the virtual room. The coins were displayed abstractly on these screens as stacked orange bars, see Image 1. After choosing a route with a mouse click, a third display in the middle of the room showed a power bar and the progress within the chosen track. Participants were only able to track their progress on this screen: no visible cart was moving.
1 For a video of a participant performing the task see:
https://www.youtube.com/watch?v=B87NGwa5jlU
2
Image 1. Visual representation of reward and effort in the baseline condition.
In the realistic reward condition, participants started in the same virtual environment. Choices were again represented on two screens, but now rewards were realistically represented as stacks of golden coins on each side of a desk in front of the participant instead of on the screens, see Image 2. After choosing, the coins of the chosen route would fly into a chest in front of the participant. Then participants drove the cart the same way as in the baseline condition, and saw their progress in the middle screen.
In the realistic effort condition, participants found themselves in a mine cart inside a room with two screens displaying the different route options and abstract coins as stacked orange bars, see Image 3. After selecting a route, a large door would open and they drove themselves into an outside natural environment, using the same bicycle pump that they now saw integrated into their cart, see Image 4. Some sections of the outside tracks were overgrown with either grass (medium effort) or shrubs (large effort).
Image 3. Visual representation of reward and effort in the realistic effort condition.
Before the start of the experiment, participants completed at least one test trial from every condition, to make sure they understood the instructions.
Reward modifier
For each HE choice the difference between track rewards reduces and for each LE choice the difference between track rewards grows by using the reward modifier. For each trial the reward for both tracks is calculated by subtracting the total effort values from both tracks (zero for green, two for orange and four for red) and multiplying this value with the reward modifier. The outcome is then added to ten for the high effort track and subtracted from ten for the low effort track.
The value of the reward modifier ranges between 0, which means no difference between rewards, and 1.25, which is the maximum possible reward (20 coins) divided by the maximum possible difference between tracks (16). Each trials starts with a reward modifier value of 0.625, which is the maximum reward modifier value divided by two. For each HE choice a value is subtracted from the reward modifier value and for each LE choice a value is added to the reward modifier. The added or subtracted values grow for each consecutive choice of the same effort type. This value is 0.02 for the first, 0.05 for the second, 0.1 for the third and 0.2 for the fourth or a higher consecutive choice of the same type of effort. With each switch in effort type this value drops back to 0.02. The minimum value of the reward modifier has been set to 0 to avoid that the LE choice pertains a higher reward than the HE choice. The maximum value of the reward modifier has been set to 1.25 to avoid scores higher than 20 coins.
POI
As a measure of the relationship between reward and perceived effort, the point of indifference (POI) for each subject was determined per condition. Assigning POI values has
demonstrated to be a reliable method for measuring individual differences in subjective effort (Westbrook, Kester, & Braver, 2013). The POI was reached when the subject no longer expressed a preference for the options. At this point the subject chooses the HE/HR just as much as the LE/LR.
To determine the POI value for each condition, the average of the values of the reward per unit effort modifier of the last four trials was calculated. When the POI is low, the subject needed less reward to choose the HE/HR option. A higher POI means that the subject needed a higher reward to choose the HE/HR option. When the POI is equal to zero the subject chooses the HE/HR option, without consideration of the effort required.
The differences in POI values represent the differences in perceived effort. The representation of the effort and reward could account for these differences. When reward representation is constant, differences in the POI values are due to perceived effort. When effort representation is held constant, differences in the POI values can be attributed to perceived reward.
Box 1
Computer system:
The system specifications of the computer system that was used were: Intel Core i7-3770 CPU, 3.4Ghz, 4 cores
Zotac GeForce GTX 970 Extreme Core Edition 16GB of DDR3 memory
256GB Solid state hard drive Logitech Gaming Mouse G300 Oculus Rift DK2
Results
Of the original 51 participants, 15 were excluded from the analysis due to ceiling effects in measurements. A ceiling effect was thought to occur when in at least two of the last four trials the reward for the chosen option was 10 coins. This indicated that the required amount of effort did not affect these participants in their choices. When a ceiling effect occurred in at least two out of three blocks, participants were excluded from the analysis regarding the POI values.
These ceiling effects were not thought to have an effect on the reward sensitivity or the HRV of the participants. Thus, for the second analysis, including reward sensitivity and HRV, these participants were not excluded based on these criteria.
One participant did not partake in all the conditions due to nausea, and was thus excluded. Another participant performed only in one condition due to a mistake of the experimenter.
Analysis of POI and IPQ scores:
To analyze the sense of presence in the different conditions a one-way repeated
measures ANOVA was conducted on the mean IPQ scores. A total of 5 participants did not complete the questionnaire and thus the data of 46 respondents was used for the analysis. The participants that performed on ceiling level during the task were not excluded from this analysis since it is was not believed to have influence on the sense of presence. The mean scores on the IPQ questionnaire are displayed in Table 1 and Figure 1:
Table 1
Average scores on the IPQ and standard deviations for the baseline, realistic reward and realistic effort conditions
Mean Std. Deviation
IPQ Baseline 35.37 4.78
IPQ Realistic Reward 36.43 5.74
IPQ Realistic Effort 38.54 4.64
Figure 1. Average IPQ scores for the three conditions,
Baseline (Baseline), realistic reward (Reward) and realistic effort (Effort). Error bars represent the within subject normalized errors of the mean N = 46 participants.
Mauchly’s test indicated that the assumption of sphericity had not been violated, χ2(2) = 4.67, p = .097. The results show that the IPQ values were significantly affected by the condition, F(2, 90) = 11.19, p < 0.001. A test of within-subject contrasts revealed that the
30 32 34 36 38 40 42 44
Baseline Reward Effort
Mea n I P Q Condition
IPQ values were significantly higher in the realistic effort condition compared to the baseline condition, F(1, 45) = 22.55, p < 0.001. Post hoc tests using the Bonferroni correction revealed that the IPQ values for the realistic reward condition were significantly lower in comparison to the IPQ values for the realistic effort condition, p = 0.028. The IPQ values for the realistic reward condition did not differ significantly from the baseline condition. These results suggest that the participants felt more present during the realistic effort condition than in both the realistic reward condition and the baseline condition. This suggests that the manipulation of presence in the realistic effort condition was successful.
For the 34 participants that remained after the exclusion based on ceiling effects, a one-way repeated-measures ANOVA was conducted on the mean POI values.
The average POI values are represented in Table 2 and Figure 2:
Table 2
Average Point of Indifference (POI) values and standard deviations for the baseline, realistic reward and realistic effort conditions
Mean Std. Deviation
POI Baseline .456 .282
POI Realistic Reward .482 .371
Figure 2. Average Point of Indifference (POI) values for the three conditions,
Baseline (Baseline), realistic reward (Reward) and realistic effort (Effort). Error bars represent the within-subject normalized standard errors of the mean. N = 34 participants.
Mauchly’s test indicated that the assumption of sphericity had not been violated, χ2(2) = 3.19, p = .203. The results show that the POI values were not significantly affected by the different conditions, F(2, 66) = 1.205, p = .306. This goes against the hypothesis that differences in form of representation would have an effect on the perceived effort. It was expected that higher POI values would have been found in the realistic effort condition compared to the baseline condition. Post hoc contrasts did not reveal any significant contrasts between the different conditions.
In addition, the relation between reward sensitivity and POI values in the different conditions was explored. To assess the sensitivity to reward, the total scores on the BAS scale
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
Baseline Reward Effort
Mea n P O I Condition
from the BIS/BAS scales were calculated. A Pearson correlation analysis was performed on the total BAS scores and the three different conditions of the task. There was no significant relationship found between the total BAS scores and the baseline condition (R = 0.1, p = 0.59) or the reward condition ( R = 0.21, p = 0.24) nor with the effort condition (R = -0.11, p = 0.54). In addition, there was no significant relationship between the BAS-Reward subscale and the different conditions. This suggests that differences in sensitivity to reward did not predict individual differences in the relationship between perceived reward and effort.
Analysis of HRV and Reward Sensitivity:
The ECG data from 17 participants was not useful for analysis due to issues with the ECG recording. Issues with these recordings included but were not limited to, missing data from different blocks and loose electrodes. Of the remaining data an additional 3 participants were considered to be outliers. Data points were considered outliers when they were 1.5 times the interquartile range above the third quartile or 1.5 times below the first quartile. This was calculated for all three conditions. When a subject was considered to be an outlier in at least one condition, the subject was removed from analysis for all conditions.
For the remaining 31 participants the inter-beat interval (IBI) values during the trials were obtained for each participant in each condition. The IBI’s of interest were the values between stimulus onset and moment of choice during the trial. Time markers were placed after data collection to acquire these ranges of IBI’s. The IBI’s from the first trial of every condition were excluded from analysis, due to too much noise. For the 12 remaining trials the HRV was calculated from the root mean square of successive differences in RR intervals. The HRV was obtained for each participant for each trial in every condition. The average HRV was calculated for each participant for each condition.
For the remaining 31 participants, a one-way repeated-measures ANOVA was conducted on the mean HRV values. The average HRV values are represented in Table 3 and Figure 3:
Table 3
Average Heart Rate Variability (HRV) values in milliseconds and standard deviations for the baseline, realistic reward and realistic effort conditions
Mean Std. Deviation
HRV Baseline 62.11 58.68
HRV Realistic Reward 70.32 66.41
HRV Realistic Effort 59.43 52.70
Figure 3. The average Heart Rate Variability (HRV) values in milliseconds for the three conditions, Baseline (Baseline), realistic reward (Reward) and realistic effort 40 45 50 55 60 65 70 75 80 85 90
Baseline Reward Effort
Mea n H R V Condition
(Effort). Error bars represent the within-subject normalized standard errors of the mean. N = 31 participants.
Mauchly’s test indicated that the assumption of sphericity had not been violated, χ2(2) = .677, p = .713. The results show that mean HRV values did not significantly differ for the different conditions, F(2, 60) = .729, p = .486. This suggests that amount of realism in reward and effort representations does not affect the HRV values. These results go against the hypothesis that more realistic reward representation would affect HRV values. Further analyses include sensitivity to reward.
To analyze the relation between sensitivity to reward and HRV values, a Pearson correlation analysis of the total BAS scores and the mean HRV values was performed. There was no significant relationship found between the BAS scores and the baseline condition (r = -.14, p = 0.45) or the reward condition ( r = -0.08, p = 0.68) nor with the effort condition (r = .15, p = 0.42). This suggests that total BAS scores did not predict differences in HRV in the different conditions.
To further explore the relation between HRV values and reward sensitivity, an analysis of covariance was performed with total BAS scores as the covariate. Mauchly’s test indicated that the assumption of sphericity had not been violated, χ2(2) = .1.341, p = .511. The covariate, BAS score, was not significantly related to the mean HRV values F(2, 58) = 1.658, p = .199. This confirms the previous results that sensitivity to reward did not affect differences in HRV. These results go against the hypothesis that HRV values would be related to the total BAS scores.
Discussion
The present study had two main objectives. One of them was to inspect if the degree of realism of the effort representation had an effect on the perceived effort. It was expected
that a realistic representation would result in more perceived effort. The results of this study suggest, based on the POI values, that there was no difference in the perceived effort between the different conditions. This is in line with the results from the self-report questionnaire. Despite the lack of an effect of realism on perceived effort, the participants reported to feel more present in the realistic effort condition. This suggests that differences in degree of immersion were created by the different conditions.
In the interest of the second objective the impact of representations of reward on HRV was examined. It was expected that when rewards were represented realistically, the HRV would be higher during the valuation period. This was thought to be affected by the reward sensitivity of individuals. The results of this study suggest that the differences in representation of rewards did not affect the HRV. In addition, there was no relation found between HRV and individual differences in reward sensitivity.
Since there was no effect found of the different conditions on the POI, it is not surprising that the analysis of the HRV in these different conditions did not show an effect either. While the POI was not thought to directly influence HRV, it does reflect the relationship between the perceived effort and the perceived reward. In this manner the lack of effect on the POI, also reflects a lack of effect on perceived reward. This suggests that the different representations of reward did not cause participants to perceive the rewards differently. This also follows from the results of the self-report questionnaire. Participants did not perceive the realistic reward as more appealing than the 2D representation of reward, as was hypothesized. Therefore, there would not have been a difference in the activity of the dopaminergic system, which in turn could have resulted in no detectable differences in HRV between the different conditions.
Considering these results it could be concluded that the amount of realism does not affect the perceived effort in effort-based decision making. Some caution must be taken when
making concluding statements based on the present results. During this experiment there were some methodological flaws that could have affected the results. Some of the problems will now be discussed. In addition, suggestions for improvement will be coupled with these problems.
First of all, some participants reported that the required effort was not experienced as burdensome. Even in the high-effort condition most participants reported the effort not to withhold them from choosing the high effort option. This could be problematic for the results of this study. When the effort is not considered burdensome, the participants can choose the high-effort/high-reward without taken into account the amount of effort. This would then result in low POI values. Since participants with ceiling effects were excluded, the average POI values were reasonably above zero. It is therefore possible that the lack of burdensome effort could have affected the results. To increase the required effort, the input device needs to be modified. The pumping motion is not considered as effortful and should be additionally weighted. One consideration could be to apply air pressure to the pump making the pumping motion heavier. This should then be controllable for the different levels of effort.
Secondly, although some participants did not consider the effort burdensome, there were multiple that reported to be severely fatigued after completion of the task. This could possibly be ascribed to their position and individual difference in pumping movement. Observing the participants while they performed the task, it occurred that there were individual differences in the efficiency of movements. Some participants used their whole body to pump while others only made a slight movement with their arms. These differences would not be expected to influence the results drastically because of the within-subject design, but should be considered in further development of this task. It is highly favorable to standardize the required movements for the task. For this, detailed instructions should be provided.
In the same way, the conclusions based on the results of the HRV data in combination with the reward sensitivity values should be interpreted with caution. It is known from previous research that HRV is affected by activity of the ANS. Therefore, the lack of significant results of the HRV analyses in this study could be ascribed to the lack of differences in the activity of the ANS for the different conditions. It could be that the realism of the rewards was not sufficiently different and more exciting than the 2D representation to provoke a measurable response in the ANS. If this seems to be the case, it is advisable to increase the level of realism of the rewards in the realistic reward condition. For example, the coins used to represent rewards could be of a real currency that is used by the participants.
Furthermore, some methodological flaws could have affected the results presented in this study. For example, the ECG data was not scanned for artifacts. Before analyzing the ECG data only a 50Hz filter was applied to filter out the noise. Therefore, the ECG data used for the analysis of the HRV probably contained artifacts that acted as noise in the data. This could have masked the effect of different conditions on the HRV. In addition to possibly noisy data, there were no markers implemented for the intervals of IBI’s that were used for the analysis. These markers were added after the data was collected. The time points of the markers were calculated by converting the real time of stimulus onset and time of decision making to milliseconds from midnight, then the IBI’s were added to a point after this start marker, when this point was reached, the IBI’s were written down until they reached the stop marker. This method is not as precise as the placing of markers in the ECG data at chosen time points while recording. Furthermore, for analyses of the ECG data it is more convenient to be able to cut out the epochs with markers in the data. Due to the lack of precision with the ECG data it is possible that any effect on HRV was masked. It is then advised to use more precise methods of ECG recordings in further research.
Even though the results of this study do not reveal any significant effects of level of realism in an EBDM paradigm nor an effect of reward sensitivity and level of realism on HRV, it is possible that these effects exist but were masked by methodological flaws and could be revealed when taken care of these flaws. This study has, at least, shown that it is possible to manipulate levels of presence and realism in a virtual reality. An EBDM paradigm was successfully presented in a VR environment. This gives rise to possibilities for future psychological research using virtual reality as a research tool.
Literature
Acharya, U. R., Joseph, K. P., Kannathal, N., Lim, C. M., & Suri, J. S. (2006). Heart rate variability: a review. Medical and biological engineering and computing, 44(12), 1031-1051.
Bardgett, M. E., Depenbrock, M., Downs, N., Points, M., & Green, L. (2009). Dopamine modulates effort-based decision making in rats. Behavioral neuroscience, 123(2), 242-251.
Bentham, J. (1948). An introduction to the principles of morals and legislation (Original work: 1789 ed.). Oxford: Blackwell.
Berridge, K. C. (2007). The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology, 191(3), 391-431.
Biocca, F., & Levy, M. R. (1995). Virtual reality as a communication system. Communication in the age of virtual reality, 15-31.
Blais, A. R., Thompson, M. M., & Baranski, J. V. (2005). Individual differences in decision processing and confidence judgments in comparative judgment tasks: The role of cognitive styles. Personality and Individual Differences, 38(7), 1701-1713.
Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature reviews neuroscience, 12(12), 752-762.
Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. Journal of personality and social psychology, 67(2), 319.
Croxson, P. L., Walton, M. E., O'Reilly, J. X., Behrens, T. E., & Rushworth, M. F. (2009). Effort-based cost–benefit valuation and the human brain. The Journal of Neuroscience, 29(14), 4531-4541.
Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F. S., & Bannerman, D. M. (2005). Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology, 179(3), 587-596.
Desai, P. R., Desai, P. N., Ajmera, K. D., & Mehta, K. (2014). A review paper on oculus rift-A virtual reality headset. arXiv preprint arXiv:1408.1173.
Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature neuroscience, 8(11), 1481-1489.
Franken, I. H., Muris, P., & Rassin, E. (2005). Psychometric properties of the Dutch BIS/BAS scales. Journal of Psychopathology and Behavioral Assessment, 27(1), 25-30.
Gray, J. A. (1981). A critique of Eysenck’s theory of personality. In A model for personality (pp. 246-276). Springer Berlin Heidelberg.
Holt, D. D., Green, L., & Myerson, J. (2003). Is discounting impulsive?: Evidence from temporal and probability discounting in gambling and non-gambling college students. Behavioural Processes, 64(3), 355-367.
IJsselsteijn, W. A., Ridder, H. D., Freeman, J., Avons, S. E., & Bouwhuis, D. (2001). Effects of stereoscopic presentation, image motion, and screen size on subjective and objective corroborative measures of presence. Presence, 10(3), 298-311.
Kurniawan, I. T., Seymour, B., Talmi, D., Yoshida, W., Chater, N., & Dolan, R. J. (2010). Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. Journal of Neurophysiology, 104(1), 313-321.
Kurniawan, I., Guitart-Masip, M., & Dolan, R. (2011). Dopamine and effort-based decision making. Frontiers in neuroscience, 5.
Meehan, M., Razzaque, S., Whitton, M. C., & Brooks Jr, F. P. (2003, March). Effect of latency on presence in stressful virtual environments. In virtual reality, 2003. Proceedings. IEEE (pp. 141-148). IEEE.
Parsons, T. D., & Rizzo, A. A. (2008). Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: A meta-analysis. Journal of behavior therapy and experimental psychiatry, 39(3), 250-261.
Phillips, P. E. M., Walton, M. E., & Jhou, T. C. (2007). Calculating utility: preclinical evidence for cost-benefit analysis by mesolimbic dopamine. Psychopharmacology (Berl.) 191, 483–495.
Salamone, J. D., Cousins, M. S., & Bucher, S. (1994). Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behavioural brain research, 65(2), 221-229.
Salamone, J. D., Correa, M., Farrar, A. M., Nunes, E. J., & Pardo, M. (2009). Dopamine, behavioral economics, and effort. Neuroeconomics, 60.
Sanchez-Vives, M. V., & Slater, M. (2005). From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6(4), 332-339.
Saul, J. P. (1990). Beat-to-beat variations of heart rate reflect modulation of cardiac autonomic outflow. Physiology, 5(1), 32-37.
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience of presence: Factor analytic insights. Presence, 10(3), 266-281.
Treadway, M. T., Buckholtz, J. W., Cowan, R. L., Woodward, N. D., Li, R., Ansari, M. S., ... & Zald, D. H. (2012). Dopaminergic mechanisms of individual differences in human effort-based decision-making. The Journal of Neuroscience, 32(18), 6170-6176.
Westbrook, A., Kester, D., & Braver, T. S. (2013). What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference. PLoS One, 8(7), e68210.
Appendix A Personal Process, Development of the Task and HRV Data Processing The experiment and paradigm described in this bachelor thesis are not entirely unique. In the academic year of 2014/2015 a bachelor thesis project group already focused on EBDM in virtual reality. Since the results, derived from that project, might have been distorted by methodological flaws, another attempt at studying EBDM in virtual reality was done in this year’s bachelor thesis project.
This year’s project group was divided into two smaller groups. Both groups had a different focal point during the project, they respectively had a psychological focus and a technical focus. During this project I was a member of the technical group. Different tasks were assigned to the groups. The general task that was assigned to the technical group concerned the improvement of last year’s experimental paradigm. Since last year’s project was used as a foundation for this year’s project, it was necessary for all members to acquire an understanding on a technical level of this project.
The task was developed using the program Unreal Engine 4 (UE4). This program makes use of a visual scripting system called blueprints. This system can be used to build games and all sorts of applications. In addition to blueprints it is possible to program more hard coded material in a programming language called C++. The task was programmed mainly using blueprints, this made it more approachable and user friendly for a beginner programmer. Prior to this project I had little, to no, experience with programming. However, I was very eager to learn.
To be able to work on this project I had to learn how blueprints operated. To accomplish this understanding I watched YouTube video’s and read a lot of information on the UE4 website. In addition to passively reading and watching video’s, I also did some tutorials for working with blueprints provided by UE4. As a result of my efforts, I can now
In this project I used my acquired knowledge on blueprints to improve the already existing blueprints of the ‘Reward Modifier’. This year’s project group had some remarks on the system that was used last year to modify the reward values. This meant that changes in the blueprints of this system had to be implemented. Firstly, the maximum reward values were set from 2000 to 20. We argued that this was needed to make the values more manageable. Along with the change in maximum values the value of the modifier had to be changed as well. This value was set to 0.625 at the beginning of each trial.
The updating function of the value of the modifier had to be changed in respect to last year’s version, because the first two trials had too much impact on the end value of the modifier. Together with a member of my group I constructed a blueprint that matched our desires. In last year’s version the added and subtracted values were even for each choice made. The main difference is that the present version takes into account the amount of consecutive choices for the same type of effort. The added or subtracted values grow for each consecutive choice of the same effort type. The detailed outcome of our modifications of this blueprint can be read in the Method section of this thesis under ‘Reward modifier’.
In addition to working with blueprints I also exposed myself to the world of static meshes. Prior to this project I had no knowledge of the existence of static meshes, let alone an understanding of how to work with them. In this year’s project the representation of the rewards was supposed to be realistic in one of the conditions. In last year’s project the visual representation of rewards were just numbers on a screen. This had to be changed into realistic coins in one condition. This was a task I was happy to take on. I started reading about static meshes on the UE4 website and watching several video’s and tutorials.
As all beginnings are difficult, and you need to make a lot of mistakes to get it right, I started importing models and images in Blender to try and create some sort of coin. Blender is a professional software product used for creating, amongst other things, visual effects, 3D
models, video games. I searched a long time for 3D models of golden coins. As these models were not all useful at once, I had to modify some aspects to get the result I wanted. That meant that I had to learn how UV unwrapping, texturing and 3D modeling worked. For this I again watched video’s and read instructions online. After some time, I found the right coin model and then imported this into UE4, after changing the lights and texturing, so it could be used in the application. Further development on the coins has not been a result of my own work. Later physics were applied to them and the coins were given a shining element that made them seem more realistic and appealing. I did not partake in this further development of the coins as this went beyond my knowledge.
Another aspect that had a lower priority, but also needed to be addressed, was the fact that the bushes on the track penetrated the cart. This was a huge immersion breaker. Although it was considered low priority, I tried to solve this problem. After some research on collision boxes and, again, static meshes and actor blueprints, I tried to modify the existing blueprint of the bushes. Despite the time and effort I put in this task, this part of the project was with no success for me. I ended up passing this task along to another group member who had more experience with programming and eventually solved the problem. Nonetheless, I learned a lot about collision boxes and working with actor blueprints.
In addition to the technical part of developing this project, I took on another challenge involving the analysis of the HRV data. This process turned out to be very time consuming and challenging. Not having dealt with such large data sets before, at first I had no idea how to analyze this. Together with one of my group members we dived into the literature of HRV analyses and mainly used the manual of VSRRP to get a better understanding of the data that we had acquired. We soon came to realize that the best option to sort all these data was to use MATLAB. Luckily enough we both took the course ‘Introduction to Programming’ and therefore had a basic understanding of MATLAB.
The output that VSRRP provided us with, contained the series of IBI’s for each condition, for each participant. This was our starting point. We extracted these IBI’s per person for each condition in an Excel file, by hand. These matrices were then imported into MATLAB creating a 3D matrix with the three different conditions on the x-axis, IBI’s on the y-axis and participants on the z-axis.
In addition to the matrix contain the IBI’s we made a start-matrix and a stop-matrix, containing respectively the start and stop markers for each trial in each condition for each participants. This was again a 3D matrix with markers for the trials on the y-axis, the three conditions on the x-axis and participants on the z-axis.
We then wrote a function in MATLAB that took the three matrices that we created as input arguments, and outputted the epochs of interest. Then we wrote another function that took these epochs as an input argument and outputted a matrix with the HRV values for each trial, for each condition for each participant. From this matrix the average HRV values over 12 trials for each condition were then calculated for each participant. These were used for statistical analysis in SPSS.
All taken together a lot of time and effort was put in obtaining these final HRV values that were used in the statistical analysis. Since we both had only a basic understanding of programming it took us a lot of time to debug our functions. Although we quickly had a conceptual idea of how we could analyze this data, the ‘language barrier’ made it difficult to translate this into MATLAB. I’m very proud of both us that we made it work in the end.
In summary I can state that as a member of the technical group, with no technical background, I have learned a lot from this project. Looking back, I can say that I have developed my problem solving skills and my ability to think in abstract terms and ideas. I don’t think this would have happened to this extent if I had chosen the more ‘safe’ option of the psychological group. I am glad that I challenged myself.
