Confidence as an Integration of Choice and Post-Decision Evidence

(1)

Confidence as an Integration of Choice and

Post-Decision Evidence

E.J. van der Putten

University of Amsterdam

First Supervisor: Steve Fleming

Second Supervisor: Richard Ridderinkhof University of Amsterdam/New York University

Brain and Cognitive Sciences – Cognitive Neuroscience track Date: 05/08/15

(2)

Several perceptual decision studies imply that the same information that is used to make a decision is also used to determine confidence in that decision. However, these findings do not appear to apply to all situations, suggesting that there is more to confidence than has currently been found. Behavioral and computational studies show that the same information does not predict both the decision and the confidence rating when additional evidence is presented after a decision has been made. Human brain activity during the period in which additional evidence is processed, between the initial decision and a confidence judgment, previously has been unstudied in neuroscience. The present study aims to explore how presenting additional evidence influences brain activity when confidence regarding decision performance has to be judged. We used a mediation analysis to put three variables together, in order to find which brain activations mediate the relationship between post-decision evidence and confidence. We found a network of brain activity involved in mediating this relationship, amongst others areas in the posterior parietal cortex and the frontal gyri, as well as the anterior cingulate gyrus. Our findings suggest that these areas could be involved in an evaluation mechanism, to integrate evidence with expectations in order to estimate the likelihood of a successful decision as well as to judge decision confidence. Future studies are needed to investigate how confidence is computed by further examining the specific roles and functions of different parts of the brain activation network.

(4)

Introduction

Humans are very capable of monitoring and evaluating their everyday actions and decisions.

Currently there is considerable interest in understanding the neural mechanisms of decision certainty, or confidence in perceptual decision making. Several studies investigated how brain activity is related to perceptual evidence and decision confidence in humans, primates and rodents. Studies have, for example, shown that activity in single neurons in the OrbitoFrontal Cortex in rats is modulated by correct and incorrect choices as well as evidence strength (Kepecs et al., 2008), and that single neurons in the Lateral IntraParietal cortex in monkeys might represent decision certainty as well as choice (Kiani and Shadlen, 2009). Additionally, microstimulation to the visual motion related brain area (V5) affected confidence in monkeys (Fetsch et al., 2014). The provided explanation for such findings is that the same neural circuits that support an initial decision also support the confidence judgment regarding that decision. Concluding a single decision variable determines both choice and confidence is consistent with signal detection based models, which derived both decisions and confidence ratings from the same decision variable (Smith & Vickers, 1988; Galvin et al., 2003). Together, these findings present a coherent view on how the brain processes initial perceptual evidence into a decision and a subsequent confidence judgment, as well as how this computation might take place.

Studies with an altered design, however, implied that there may be more to the neural computation of confidence judgments. Several such studies showed dissociation between

performance in perceptual decision-making, and confidence regarding these decisions. Documented factors that disrupted confidence judgments but not first order decisions are masking stimuli (Lau & Passingham, 2006; Bona & Silvanto, 2014; Vlassova et al., 2014), brain lesions (Fleming et al., 2014), and temporary inactivation of brain regions (Rounis et al., 2010). In contrast, the modulation of spatial attention (Wilimzig et al., 2008) improved first order decisions without affecting confidence judgments. Additionally, individual differences were found in the capacity to accurately reflect on performance with a confidence judgment (Fleming et al., 2010; Song et al., 2011; Barttfeld et al., 2013; McCurdy et al., 2013). The implication of a dissociation between decision and confidence is that confidence computations in the brain are likely also dissociated from the computation of an initial decision variable. This is consistent with the finding that rostrolateral PreFrontal Cortex (rlPFC) activity corresponded to confidence judgments but not to non-metacognitive judgments, and was also modulated by individual metacognitive accuracy (Fleming et al., 2012). Although the level of reported confidence sometimes appears to only be dependent on evidence strength, specific experimental designs, as mentioned above, decouple the two decisions. By showing that there is a

(5)

dissociation between first order decision and second order confidence, these studies show the necessity of other brain areas and processes being involved in the computation of confidence compared to first order decisions.

Several studies investigated which computations might be involved in addition to the processing of pre-decisional evidence. A possibility to account for confidence judgments on top of decisions and reaction times was the introduction of post-decisional processing as an extension to random walk/diffusion models (Pleskac and Busemeyer, 2010). In order to justify post-decision processing, the hypothesis that confidence is computed from the same decision variable as the first order choice had to be dropped. Another study found that a classifier trained to differentiate between errors and correct responses could be generalized to differentiate between different levels of graded confidence (Boldt and Yeung, 2015). This finding implies a possible shared neural marker for error awareness and confidence, which might be part of the post-decision processing as modeled by Pleskac and Busemeyer (2010). To date, the temporal extension of evidence for decisions over time, even after the decision has been made, has received little attention in neuroscience (Yeung and Summerfield, 2012). Most previous studies are unable to account for how post-decision evidence changes decision confidence over time, because they did not involve such temporal extensions. For example, providing extra perceptual evidence after the first order perceptual discrimination decision influenced confidence judgments related to the amount of post-decision evidence provided (Moran et al., 2015). This implied again that confidence judgments are not solely based on the initial decision variable, but under the specific circumstances integrate post-decision evidence as well. A pipeline model suggested some post-decision processing could occur in first-order circuits, as a consequence of the delays involved in making a response (Resulaj et al., 2009). However, this pipeline model cannot account for cases in which new evidence is presented after the response, as will be the case in the present work.

The integration of post-decision evidence as part of a confidence computation has so far not been studied in the human brain. Due to the general use of a study design in which confidence immediately follows or even replaces the first order decision, previous studies have not been able to look at the separate effects of evidence processing for a first and second order decision. An altered study design is needed in order to dissociate between evidence processing and the integration of evidence into a confidence judgment. The design that will be used in this study consists of pre-decision evidence followed by a first order perceptual discrimination choice, after which participants view extra evidence before making a confidence judgment regarding their previous decision. By using different levels of evidence strength before and after the decision, this design explicitly differentiates between pre and post decision evidence.

(6)

The aim of the present study is to investigate how the brain integrates new perceptual evidence with decision evidence that has been processed previously and the decision response. In contrast to prior work that focused on pre-decision evidence and confidence, we focus on the integration of post-decision evidence with a first order decision on a whole-brain level. We will test the hypothesis that post-decision evidence strength increases confidence in correct choices, while lowering confidence in incorrect choices. We expect increased confidence-related brain activity in response to higher post-decision evidence after a correct response, but decreased confidence-related activity after an incorrect response. Therefore, we combined post-decision evidence with

performance to capture this effect in the fMRI analysis. An exploratory analysis on the whole-brain level will provide first insight into which brain areas mediate the relationship between post-decision evidence strength as a function of performance and confidence. Based on previous studies on confidence, metacognition and error awareness, we are interested in the involvement of particular candidate brain regions. These regions are the rlPFC and Anterior Cingulate Cortex (ACC), based on findings by Fleming et al. (2012) and Baird et al. (2013) who showed that these areas are involved in perceptual metacognition and metacognitive accuracy. The posterior parietal cortex is also expected to be involved based on the finding that error related positivity in the parietal cortex corresponds to both confidence and error awareness (Boldt & Yeung, 2015). Individual parietal cortex cells were also involved in both evidence accumulation and decision confidence (Kiani & Shadlen, 2009).

(7)

Materials and Methods

Participants:

Twenty-six participants (15 females, mean age = 24.9, SD = 5.9) gave written informed consent to take part in this study. Two participants only participated in the first session of the study, two additional participants were excluded from further analysis due to excessive head motion in the fMRI scanner. Twenty-two participants were included in the remaining analysis (13 females, mean age = 23.7, SD = 3.6). The procedures in this study were approved by the NYU’s University Committee on Activities Involving Human Subjects.

Stimuli:

The visual stimuli and task were generated on a personal computer using Psychtoolbox (version 3.0.12) for Matlab (version 2014b). All out-of-scanner trials were presented on an iMac desktop computer running MATLAB, at a viewing distance of approximately 45 cm. The monitor refresh rate was 60Hz. All in-scanner trials were presented on a projector, at a viewing distance of approximately 58 cm.

Participants performed a random dot motion direction estimation task. They were instructed to look at a white fixation dot (0.2° diameter) in the middle of the screen and decide the motion direction of a field of moving white dots (0.12° diameter) on a uniform grey background. The dots were contained in a 7° diameter circular white aperture. Each set of dots lasted for 1 video frame and was replotted 3 frames later (Roitman & Shadlen, 2002). For each dot, its new location was either replotted randomly or, for a subset of dots determined by the coherence percentage, from their original location in the direction of the motion. Motion direction was either to the left (- 90 degrees) or to the right (+ 90 degrees). The coherently moving dots moved at a speed of 5°/s and the number of dots in each frame was specified to create a density of 30 dots/deg2_{/s. After the pre-decision motion the fixation dot} turned red for the post-decision motion stimuli. Post-decision motion stimuli were generated and plotted in an identical manner to the pre-decision stimuli. Motion responses were made by pressing the 1 and 2 keys on a standard computer keyboard with the left hand, for a leftward and rightward decision respectively.

Task and procedure:

Participants attended two sessions on separate days, the first session only tested behavior out of the scanner and the second took place in the fMRI scanner.

Behavioral session:

Calibration phase: To match the difficulty level of the motion stimulus across participants, we carried out a calibration session to obtain each participant’s psychometric function for left/right motion

(8)

discrimination (see Appendix A). Each participant performed 240 trials of motion direction estimation, without post-decision motion and confidence judgments. These trials were equally distributed over 6 levels of coherence: 3%, 8%, 12%, 24%, 48% and 100%. Only during this calibration phase feedback on performance was provided using a high-pitched tone for correct judgments and a low-pitched tone for incorrect judgments. The three coherence levels to be used in the main

experiment were individually determined for each subject from online fits to the calibration data using probit regression (for report, see Fleming et al., 2013). For each participant, the estimated coherence level that corresponded to 60%, 75% and 90% correct choices were stored for use in the main experiment.

Experimental phase: The main experiment consisted of 9 blocks, each containing 100 trials. Each trial consisted of the following events in sequence (as depicted in figure 1). First a fixation dot and circle were presented for 500 ms, followed by a motion stimulus for 300 ms. Participants were asked to decide whether the motion stimulus was moving to the left or right. After their response was a 100 ms gap, after which they were again shown a motion stimulus for 300 ms, of which the direction was the same but the coherence could vary. For both the pre- and post-decision motion the same 3 coherence levels were used that were obtained in the calibration phase. We used a full factorial design with 9 conditions, that were formed from 3 pre-decision coherence levels x 3 post-decision coherence levels, and each consisted of 100 trials. The instructions participants received regarding the post-decision motion was that it was bonus evidence they could use to inform their confidence, but not to change their initial decision. After a 200 ms gap participants were asked to indicate their confidence on a horizontal scale (length = 14°) from 0% to 100%. Confidence responses were made with the right hand, with a mouse click anywhere on the scale. A red vertical cursor appeared for 500 ms at the location they indicated as their confidence rating before the next trial began. There was no time limit for the confidence rating or the first response, and no feedback on performance was provided.

The confidence scale had the numbers 0, 20%, 40%, 60%, 80 and 100% above the scale marking confidence in ones probability of being correct, as well as (following (Boldt & Yeung, 2015)) verbal indicators of confidence, “certainly wrong”, “probably wrong”, “maybe wrong”, “maybe correct”, “probably correct” and “certainly correct” below the scale. The scale had ticks at the different levels, as well as an additional tick in the midpoint for a 50% or neutral confidence rating. Half of the participants saw the scale with 0% on the left and 100% on the right and the other half saw the scale reversed. Participants were instructed they could earn between 0 and 100 points per trial, and were then asked which confidence ratings they should provide in certain scenarios. They received the following feedback after answering those questions.

(9)

“The correct answers were: If you are sure you responded correctly, you should select 100% confidence/certainly correct. If you are sure you picked the wrong direction, you should respond 0% confidence/certainly wrong. If you are not 100% sure about being correct or incorrect you should select a location in between according to the following descriptions on the confidence scale: probably incorrect = 20% confidence; maybe incorrect = 40% confidence; maybe correct = 60% confidence; probably correct = 80% confidence. You can also click anywhere in between these percentages.”

Participants were incentivized to provide correct confidence judgments with the Quadratic Scoring Rule (QSR) (von Holstein, 1970). They were given the formula of the QSR with the following instructions: points=100∗[1−

(

accuracy−confidence

)

2] “This formula may appear complicated, but what it means for you is very simple: You will get paid the most if you honestly report your best guess about the likelihood of being correct. You can earn between 0 and 100 points for each trial.”

Accuracy is 1 on correct trials and 0 on incorrect trials, confidence is entered as a probability between 0 and 1 as indicated by the participant. In this way the QSR ensures that maximizing the fit between the accuracy of choices and the degree of confidence allows a participant to obtain maximum earnings. Participants received $1 extra for every 5000 points they earned.

fMRI session:

Calibration phase: During the structural scan at the start of the experiment, participants carried out 120 trials of left/right motion judgments without confidence ratings. The purpose of these motion judgments was to recalibrate the participants coherence levels that corresponded to 60, 75 and 90%

(10)

accuracy. For this session the QUEST adaptive staircase procedure was used (Watson & Pelli, 1983). Three randomly interleaved QUEST procedures received the previous coherence level as a prior and updated the coherence levels according to performance over 40 trials per QUEST. These coherence levels were then saved and stored for use in the main experiment phase.

Experimental phase: Prior to entering the scanner, participants were familiarized with the task and confidence rating scale. Changes compared to the behavioral session included a time limit of 1.5 s for the first response and a time limit of 3 s for the confidence rating, as well as a slider scale rather than a mouse-click confidence scale. The confidence scale itself still looked the same as in the behavioral session (as depicted in figure 1). Both motion judgments and confidence judgments were made with an fMRI button box in their right hand. To rate their confidence, participants used the 1 and 2 buttons to move a cursor in steps of 10% to the left or right side of the confidence scale respectively. The initial location of the cursor was in a random position on the scale for each individual trial, to discourage advance motor preparation of the confidence response. They then confirmed the confidence rating by pressing 3, after which the cursor changed from white to red and remained red in that location for 500 ms to confirm the confidence rating.

The main experiment consisted of 4 blocks, each containing 90 trials. Each trial consisted of the same events in sequence as the behavioral session (see figure 1). After the main experiment, participants passively viewed 20 alternating displays of both moving dots (50% coherence; 50/50 randomized left/right direction) and stationary dots, each lasting 12 s, as a localizer for V1 and V5 activity. These data will be used for future ROI analyses and will thus not be discussed in the current paper.

fMRI acquisition:

Whole-brain fMRI images were acquired using a 3T Allegra scanner (Siemens) with an NM011 Head transmit coil, Nova Medical (Wakefield, MA). BOLD-sensitive functional images were acquired using a Siemens epi2d BOLD sequence (42 transverse slices; repetition time (TR) = 2.34 s; echo time = 30 ms; 3 x 3 mm resolution; flip angle = 90 degrees; 64 x 64 Matrix; slice tilt, -30° T > C; interleaved

acquisition). The main experiment consisted of 4 runs of 315 volumes, and the localizer task consisted of a single run of 211 volumes. For each participant a T1-weighted anatomical scan (T1 MPRAGE; 1x1x1 mm resolution; 176 slices) and local field maps were also collected.

Behavioral analyses:

The effects of condition on confidence ratings – for both correct and incorrect responses – and accuracy were assessed using hierarchical mixed-effects regression using the lme4 package in R (Version 3.2.0; (Bates et al., 2014). We obtained p values for regression coefficients using the car package (Fox & Weisberg, 2010). Mixed-effects logistic regression was used to quantify the effect of

(11)

pre- and post-decision coherence on response accuracy. All effects were taken as random at the participant level and condition estimates and statistics are reported at the population level.

fMRI preprocessing and analysis:

All imaging processing and analysis were carried out using SPM12 (Statistical Parametric Mapping; www.fil.ion.ucl.ac.uk/spm). The first five volumes of each run were discarded to allow for T1 equilibration. Functional images were first slice-time corrected and then realigned and unwarped using collected field maps (Andersson et al., 2001). Normalization of the structural scan was to the Montreal Neurological Institute (MNI) template brain, using SPM 12’s normalization routine. The resulting warp was applied to all the functional images. Normalized images were spatially smoothed using a Gaussian kernel with full-width at half-maximum of 8 mm.

Whole-brain multilevel mediation analysis:

A standard fMRI model often looks at the effect of a certain manipulation on brain activation, or the relation between brain activation and behavior. A mediation analysis provides an extension on univariate GLM models by adding a third outcome variable. This allows testing how the brain mediates the relationship between an experimental manipulation and an outcome. The mediation analyses test three effects to investigate whether certain brain regions link the type of post-decision evidence a participants receives to their reported confidence. It simultaneously tests the effect of post-decision evidence type on brain activity, what brain activity is related to confidence ratings and the formal mediation effect. Each of these effects will be described below in more detail.

As per Atlas et al. (2010), a path model was set up in this study to relate the different effects and variables to each other. The initial variable (X) is the experimentally manipulated amount of post-decision evidence a participant receives (which is 1 for low coherence, 2 for medium coherence and 3 for high coherence levels). However, because our hypothesis predicts that post-decision evidence levels will have a different effect for correct and incorrect trials, we specified the X variable by interacting it with performance (i.e. the opposite values will be used for error trials; -1 for low coherence, -2 for medium coherence and -3 for high coherence levels). The reason for the negative coding of error trials is due to our expectation of decreased confidence-related brain activity if post-decision evidence with a higher coherence is shown on an error trial. Using this way to code the X variable, the coherence x performance interaction provides a linear predictor for the amount of post-decision evidence a participant sees in favor of their post-decision. The outcome variable (Y) is the participants’ series of confidence ratings (ranging from 0 to 1). The mediating variable (M) is a single voxel’s series of beta estimates, one per trial. The direct effect of X on Y, or post-decision evidence on confidence is referred to as path c.

(12)

The first effect of interest will be the effect of post-decision evidence on brain activity, which will be referred to as the path a, or post-decision effects. Path a is similar to a standard GLM,

performance x post-decision evidence interaction analysis, in which performance has 2 levels and post-decision evidence has 3 levels, and no other factors are being controlled for. The second effect of interest is the correlation between brain activity (M) and confidence judgments (Y) and will be referred to as path b, or confidence-related activity. The mediation analysis assesses path b while controlling for the X variable, or the post-decision evidence type. This means that paths a and b are assumed to be uncorrelated under the null hypothesis of no mediation, and identify two separable processes. The b path is thus expected to identify regions that predict trial-by-trial variations in confidence judgments. Because neither brain activity nor confidence response are experimentally manipulated, this component of the path model is correlational in essence. In general the third effect tests whether the mediator explains a significant amount of the effect between the experimental manipulation and the measured outcome. In this case, the test infers whether including brain activation (M) in the path model explains a significant amount of the covariance between post-decision evidence and confidence.

A multilevel formulation of the path model is used, treating the participant as a random effect when assessing the relation between post-decision evidence, brain activity and confidence (Wager et al., 2009). The multilevel model thus allows for modeling the relations between behavior and brain activation at the population level, while also allowing for individual differences at the subject level. Additionally, like in the standard mediation model, the relationship between post-decision evidence, brain activity and confidence judgment is studied over time. The voxelwise analysis framework, as developed by Wager et al. (2008, 2009), was applied in this study to address a multilevel case. These analyses were carried out using SPM 8 due to its compatibility with the mediation toolbox. For the results of the mediation analysis we reported all clusters meeting a threshold of p < 0.05 cluster-level FWE-corrected, with the exception of a threshold of p < 0.001 uncorrected for a priori regions of interest in rlPFC, the ACC and the posterior parietal cortex. Activations were displayed at the cluster-defining threshold of p < 0.001, uncorrected, using SPM12 software. Clusters of brain activation were labelled using the descriptions of the peak activation voxels as well as the cluster extent. Description labels were supplied by XJView software (http://www.alivelearn.net/xjview), from both the

(13)

Results

Mediation Analysis:

Behavior/ Relation between condition, performance and confidence, path c:

Analysis of perceptual performance (percent correct), separated by condition is shown in figure 2A for the behavioral session and 2C for the fMRI session. As expected, both figures show that

participants’ performance increased as the quality of pre-decision evidence increased (β = 7.14, SE = 0.06, p < 0.001), but by design remained the same across the three different levels of post-decision evidence (β = -0.27, SE = 0.16, p = 0.09. Participants performed on average slightly better than the calibration thresholds of 60%, 75% and 90% during the behavioral session. This effect mostly disappeared in the fMRI session, after the recalibration. Figures 2B and 2D show the confidence ratings, split out for both pre- and post-decision evidence strength. Consistent with the hypothesis, confidence increase on correct trials and decreases on incorrect trials during both experiment sessions (also, see Appendix B). A regression of confidence, controlling for reaction time on both the initial decision and the confidence judgement, shows the effects of pre- and post-decision evidence in more detail (Figure 3 and Table 1).

During both sessions, increased quality of pre-decision evidence led to increased confidence ratings on correct trials (Behavioral: β = 0.35, SE = 0.07, p < 0.001; fMRI: β = 0.35, SE = 0.07, p < 0.001). On incorrect trials however, only in the behavioral session decreased pre-decision evidence was

significantly related to decreased confidence ratings, although the fMRI session showed a trend in the same direction (Behavioral: β = -0.37, SE = 0.16, p = 0.03; fMRI: β = -0.27, SE = 0.16, p = 0.09). The

(14)

main effect of post-decision evidence on confidence ratings was polarizing as a function of accuracy, with increased post-decision evidence leading to increased confidence ratings on correct trials (Behavioral: β = 0.37, SE = 0.08, p < 0.001; fMRI: β = 0.46, SE = 0.07, p < 0.001) and decreased confidence ratings on incorrect trials (Behavioral: β = -0.93, SE = 0.11, p < 0.001; fMRI: β = -1.07, SE = 0.11, p < 0.001, see Figure 3 and Table 1). These findings are consistent with participants integrating their choice as well as post-decision evidence when determining their confidence judgment. Additionally, on both correct and incorrect trials there was an interaction between pre- and post-decision coherence, with an increase in post-post-decision coherence leading to a decrease in the pre-decision effect on confidence (Behavioral: β = -0.68, SE = 0.19, p < 0.001 (correct); β = -0.11, SE = 0.02, p = 0.02 (incorrect), fMRI: β = -0.81, SE = 0.16, p < 0.001 (correct); β = 0.82, SE = 0.29, p = 0.004 (incorrect)), although in the case of correct trials clustering of confidence ratings at the bounded upper limit may account for this finding.

Table 1. Regression coefficients for effects of performance and condition on confidence ratings.

Behavioral

fMRI

Beta SE p-value Beta SE p-value

Correct –

pre

0.35 0.07 < 0.001 *** 0.35 0.07 < 0.001 ***

Correct –

post

0.37 0.08 < 0.001 *** 0.46 0.07 < 0.001 ***

C

**-pre*post**

-0.68 0.19 < 0.001 *** -0.81 0.16 < 0.001 ***

Error – pre

- 0.37 0.16 0.03 * -0.27 0.16 0.09

Error – post

- 0.93 0.11 < 0.001 *** - 1.07 0.11 < 0.001 ***

(15)

E –

**pre*post**

-0.11 0.02 0.02 * 0.82 0.29 0.004 **

fMRI

The modulation of post-decision evidence x performance and brain activation, path a.

This analysis identifies brain voxels in which activity corresponds to the fluctuation in post-decision evidence, for correct and error trials separately. The coherence x performance interaction provides a linear predictor for the amount of post-decision evidence a participant sees in favor of their decision. Because the XM mediation path closely resembles a GLM univariate model in which no other

variables are being controlled for, it would be interesting to compare these findings to a GLM model at a later point, in order to control for possibly confounding variables. This XM path mediation analysis revealed a number of regions in which activity was greater if evidence was more supportive of the preceding decision (Figure 4 and Table 2), specifically the posterior and anterior cingulate gyrus, the middle temporal gyrus, the hippocampus and cuneus. The analysis additionally found a network of regions that showed greater activity if the level of evidence in support of the decision decreased (Figure 4). The areas in which brain activity is inversely related to supporting evidence for the preceding decision are the inferior and middle frontal gyrus, the inferior and superior parietal lobules, the supplementary motor area, BA 6 and the insula (see Figure 4 and Table 2).

(16)

Table 2: Summary of significant activations for the XM mediation path, positive or negative contrast, as reported in the main text (cluster-defining threshold is p < 0.001, uncorrected; all reported

activations are corrected for multiple comparisons at p < 0.05).

Contr

ast

Label

Voxels at p < 0.001 Pea k z scor e P (cluster FWE corrected ) Peak voxel MNI coordinate s

Later

ality

Positiv

e

Posterior

cingulate gyrus & precuneus

421 4.85 < 0.001 0, -46, 32 L / R

cingulate cortex & medial frontal gyrus 835 5.13 < 0.001 -3, 26, -19 L / R Middle temporal gyrus 238 61 5.10 4.16 < 0.001 0.023 -57, -19, -13 54, -10, -7 L & R Parahippocampal gyrus 164 4.87 < 0.001 -21, -25, -13 L Cuneus 79 4.77 0.007 3, -85, 26 L / R Precentral gyrus 149 4.35 < 0.001 27, -28, 62 R Middle frontal gyrus 62 4.01 0.022 -18, 32, 41 L

negati

ve

Inferior & middle frontal gyrus 468 5.35 < 0.001 -42, -1, 32* 42, 5, 29 L & R Inferior and superior parietal lobule 481 390 5.20 4.70 < 0.001 -39, -46, 44 30, -55, 41 L & R Supplementary motor area & cingulate gyrus

312 5.30 < 0.001 -3, 14, 47 L / R

BA 6 177 4.90 < 0.001 24, -1, 59 R

Insula 392 5.19 < 0.001 -30, 14, 5 L

*Part of the cluster that is reported under the label ‘insula ’, left

MNI = Montreal Neurological Institute, L = left, R = right, BA 6 = Brodmann area 6

Brain activity related to reported confidence judgments, path b:

This next effect identifies brain voxels in which trial-by-trial activity predicts confidence judgments, controlling for post-decision evidence. This effect is the path b, or confidence-related activity. Activity in brain regions that are identified in this analysis predicts variation over trials in reported confidence,

(17)

for a post-decision evidence type that is being held constant. Activation in the parahippocampal gyri was positively related to confidence judgments (see Figure 5 and Table 3). Several regions were inversely related to confidence judgment, showing greater activity for lower confidence. These regions are the inferior frontal gyrus, middle temporal gyrus, inferior temporal cortex, supplementary motor cortex and thalamus, as shown in Figure 5 and Table 3.

Table 3: Summary of significant activations for the MY mediation path, as reported in the main text (cluster-defining threshold is p < 0.001, uncorrected; all reported activations are corrected for multiple comparisons at p < 0.05).

Contr

ast

Label

Voxels at p < 0.001 Peak z score P (cluster FWE corrected) Peak voxel MNI coordinate s

Later

ality

Positiv

e

Parahippo-campal gyrus 42 29 3.90 4.61 0.009 0.048 -21, -13, -16 24, -19, -16 L & R

negati

ve

Inferior frontal gyrus, middle frontal gyrus & cingulate gyrus 1809 5.77 < 0.001 -39, 5, 32 L / R Inferior parietal lobule 1454 5.74 < 0.001 -39, -43, 35* 36, -40, 41 L & R

(18)

& precuneus Middle temporal gyrus 55 3.80 0.002 36, -76, 17 R Inferior temp-oral cortex 75 4.26 < 0.001 -42, -61, -10 L Middle frontal gyrus & dlPFC 53 3.86 < 0.001 36, 32, 17 L thalamus 110 4.46 < 0.001 -9, -16, 11 L Lingual gyrus 30 3.75 0.042 -21, -82, -13 L Cerebellum anterior lobe 36 45 4.07 4.15 0.019 0.006 -24, -58, -28 33, -49, -34 L & R

* peak voxels in the same cluster, reported separately for left and right brain activation dlPFC = dorsolateral prefrontal cortex

Brain mediators of performance x post-decision evidence on reported confidence

This formal mediation analysis tests which regions contribute to the relationship between post-decision evidence and confidence. We used whole-brain multilevel mediation to identify regions that mediated the behavioral relationship between the post-decision evidence x performance interaction and trial-by-trial confidence reports. As hypothesized, the anterior cingulate cortex and parietal cortex mediate this relationship. Additional brain regions that significantly mediated the relationship according to our exploratory analysis were the inferior and middle frontal gyrus, as well as the precuneus (see Figure 6 and Table 4).

(19)

Table 4: Summary of significant activations for the XMY mediation path, as reported in the main text (cluster-defining threshold is p < 0.001, uncorrected; all reported activations are corrected for multiple comparisons at p < 0.05).

Cont

rast

Label

Voxels at p < 0.001 Peak z score P (cluster FEW corrected) Peak voxel MNI coordinate s

Later

ality

positi

ve

Anterior Cingulate gyrus & SMA 212 4.57 < 0.001 6, 17, 44 L / R Inferior parietal lobule 191 4.00 < 0.001 -33, -55, 41 L Superior and inferior parietal lobule 430 4.47 < 0.001 21, -67, 53 R Inferior frontal gyrus 251 81 4.22 2.42 < 0.001 -45, 2, 32 45, 5, 26 L & R Precuneus 40 4.24 0.003 -6, -67, 47* L & R Middle frontal gyrus 56 4.00 < 0.001 27, -10, 56** R & L Superior temporal gyrus 19 4.10 0.094 51, -49, 11 R

* the right precuneus is part of the cluster that is reported under ‘superior and inferior parietal lobule’, right **the left meddle frontal gyrus is part of the cluster that is reported under ‘inferior frontal gyrus (precentral)’, left

(20)

Discussion

In this study brain activity was explored in relation to post-decision evidence, and in relation to reported confidence. In addition, we studied which brain activation mediates the relationship between post-decision evidence and reported confidence. The aim of the present study was to investigate how the brain integrates new perceptual evidence after a decision in order to determine decision confidence. For this purpose, prior to rating their confidence participants in an fMRI scanner were able to view post-decision evidence after making a motion judgment on a random dot motion task. We tested the hypothesis that candidate brain regions, such as the anterior cingulate cortex and the posterior parietal cortex, would significantly mediate the relationship between post-decision evidence and confidence. Here, we will discuss the results of the mediation analysis and their

possible implications, how these findings relate to current developments in the research field, as well as which future analyses and studies are required based on the current findings.

The behavioral results of the direct effect of post-decision evidence and accuracy on confidence were in accordance with computational predictions. The behavioral results from path c showed that post-decision evidence increases confidence for correct decisions, and decreases confidence for incorrect decisions. We found not only that higher pre-decision evidence increased confidence, but also that confidence depended on an interaction between pre- and post-decision evidence. This interaction means that when post-decision evidence was stronger, stronger pre-decision evidence increased confidence to a lesser degree. According to our hypothesis post-pre-decision evidence polarizes confidence towards certainty of correctness or incorrectness. For correct decisions confidence leveled off at 100%, although it does not level off at 0% for incorrect decisions. A possible explanation for this finding is that after a preliminary decision participants are less influenced by post-decision evidence, suggesting a so-called post-decision commitment effect (Bronfman et al., 2015).

In response to post-decision evidence as a function of correct or incorrect responses, named the XM path, diverse brain activation was found, amongst which were the inferior and superior parietal lobules, anterior cingulate cortex and inferior and middle frontal gyrus (for a full list of activations, see Table 3). Activity has previously been reported in the inferior and superior parietal lobules, as well as in the inferior and middle frontal gyrus, in relation to the amount of evidence in favor of a choice (Hebart et al., 2014). This diverse pattern of activation could be caused by neural processes related to motion perception and the expectation of reward, for which knowing the motion direction is essential. However, we cannot be sure whether activity is related to these two processes only, because the mediation analysis does not account for possible confounding factors. For example, this analysis might also capture the effect of pre-decision evidence, or the effects of reaction times on

(21)

the decision and confidence response on brain activity. These factors need to be taken into account when interpreting the activity of the XM path.

The present study revealed brain activation related to confidence reports, named the MY path, in the parahippocampal gyri, the inferior parietal lobules, the dlPFC/middle frontal gyrus, and in the ACC. Activity in the dlPFC and ACC is consistent with an earlier fMRI study looking at post-decision confidence judgments (Hilgenstock et al., 2014), and parts of the parietal cortex have also been implied to be involved in decision confidence (Kiani & Shadlen, 2009; Boldt & Yeung, 2015). As far as we are aware parahippocampal gyrus activation is a novel finding in relation to perceptual decision confidence, although it previously has been reported in relation to high memory confidence (Moritz et al., 2006). Possibly the mediation analysis was able to reveal brain activity related to increased confidence, that has not been found before due to confounding effects of stimulus factors.

Furthermore, MY path activations might be caused by neural processes related to expecting a certain amount of reward, keeping in mind how or where to move the cursor and remembering the direction of the initial motion judgment for which a confidence response had to be made. In general, in the MY path brain activation is correlated to the subsequent confidence reports. However, even though the brain activation precedes the confidence judgment, it is not possible to draw definite causal

conclusions. In order to make causal statements about the effect of MY path brain activation more invasive manipulations are required, such as the use of transcranial magnetic stimulation (TMS) (e.g. Rounis et al., 2010; Fleming et al., 2015).

The formal mediation analysis revealed brain activity in the inferior parietal lobe, anterior cingulate gyrus, inferior and medial frontal gyrus. These findings are consistent with earlier studies reporting ACC and medial frontal gyrus activity to be related to confidence (Hilgenstock et al., 2014). The findings are also consistent with a study showing that inferior parietal lobe, inferior frontal gyrus and medial frontal gyrus activity are related to the amount of evidence in favor of a choice (Hebart et al., 2014). The advantage of our mediation approach is that we can suggest these areas to mediate the relationship between post-decision evidence and subjective confidence, rather than just being involved in either one or the other. Furthermore, the activation in the posterior parietal cortex and anterior cingulate cortex was consistent with our expectations. The involvement of other brain areas implies that more than one decision variable or one brain area is needed for confidence computation. This raises the question what roles the different areas fulfill in mediating the relationship between post-decision evidence and confidence.

To our knowledge, our study is the first to analyze how the brain mediates the relationship between post-decision evidence and confidence. Our study therefore extends on recent findings showing that certain circumstances can lead to dissociated performance and confidence responses (Lau & Passingham, 2006; Wilimzig et al., 2008; Rounis et al., 2010; Bona & Silvanto, 2014; Fleming et

(22)

al., 2014; Vlassova et al., 2014). The possibility of dissociation between performance and confidence implies that confidence is not always computed based on the same brain activation that led to the initial decision. The current study supported the finding by Moran et al. (2015) of additional

integration of evidence under circumstances where post-decision evidence is presented. The effect of post-decisional evidence processing on brain activity was additionally investigated. Although

behavioral and computational studies have shown that post-decisional processing (Pleskac &

Busemeyer, 2010) and sometimes also integration of post-decision evidence is likely to take place, the neural underpinnings of these processes remained unknown.

The present study also extends on understanding of the functions of the brain areas that were found to be involved in mediating between post-decision evidence processing and confidence reporting. Several other studies have pointed at the involvement of the posterior parietal cortex in confidence judgments. Activation in this area has been found to represent a sense of error awareness and confidence (Boldt & Yeung, 2015), as well as planned response in combination with decision certainty (Kiani & Shadlen, 2009). We show that after viewing post-decision evidence on error trials, participants often rate their confidence under 50%, which effectively implies the induction of a change of mind. This effect of post-decision evidence on confidence demonstrates that awareness of errors is tightly linked to decreased confidence. Activity in the parietal cortex mediating the

relationship between post-decision evidence and confidence is consistent with a single shared neural marker supporting both error detection and confidence in the parietal cortex (Boldt & Yeung, 2015).

Our mediation analysis also revealed anterior cingulate activity, which generally has been associated with increased task difficulty and error detection (Barch et al., 1997; Carter et al., 1998; Botvinick et al., 2004; Baird et al., 2013). In our study, anterior cingulate gyrus activity was negatively correlated to a combination of post-decision evidence and performance, as well as to confidence. In the formal mediation analysis the anterior cingulate gyrus mediated the relationship between the two variables. A message of error awareness from the anterior cingulate gyrus possibly influences brain activity elsewhere related to increased or decreased confidence. It would be interesting to determine which of the areas that were found in the current study are influenced by the anterior cingulate gyrus. Functional and structural connectivity analyses combined could shed more light on this matter.

The findings discussed here represent the current progress in our fMRI analysis, which will be extended in the next few months. An additional analysis that is planned for the near future is a univariate GLM analysis to find the brain activation in response to levels of post-decision evidence. A univariate analysis will enable us to control for factors such as pre-decision evidence, reaction time for the decision and the confidence response, while taking into account whether a response was correct or incorrect. The goal of such an analysis would be to provide a more specific account of the

(23)

effect of post-decision evidence as a function of performance, by eliminating a number of possible confounding variables. We will additionally perform ROI analyses in V1, V5, ACC, rlPFC and posterior parietal cortex in order to investigate how activation in these areas loads on different factors. More specifically we will attempt to determine whether the activity in these ROIs is mostly in response to correct or incorrect choices, and/or to pre-decision or post-decision evidence. The goal of such an analysis would be to obtain more insight into the role of these ROIs in the processing of post-decisional evidence.

Follow-ups to the present study could alter the study design to allow for causal statements about brain activity mediating post-decision evidence and confidence. In order to make inferences regarding causality, both the task input and brain activity have to be manipulated (Holland, 1988; Sobel, 2008). That would improve the findings of the present study, as the current conclusions are all based on correlations. Future studies could also perform ROI analyses on brain areas that mediate the relationship between post-decision evidence and confidence, in order to show which factors (e.g. increased or decreased post-decision evidence, correct or incorrect responses, increased or

decreased pre-decision evidence) drive their involvement. This would be a next step in uncovering what neural mechanism is being used to compute decision confidence based on a decision and on post-decision evidence.

In summary, this study shows that post-decision evidence polarizes confidence judgments, with increased post-decision evidence making participants more certain of being right or wrong. Furthermore, we show this effect to be mediated by brain activity in posterior parietal areas, in the anterior cingulate gyrus and in the middle and inferior frontal gyrus. Our findings suggest that these brain regions might be essential for the integration of post-decision evidence and decision to modulate confidence, and therefore for the computation of metacognitive judgments.

(24)

Acknowledgements

I would like to thank Nathaniel Daw for providing me with all the hard- and software that was needed to make, conduct and analyze the experiments in his lab at NYU. He has been most welcoming to me, and always made the time to help out with any problems or questions that came up. I thank Steve Fleming for his enthusiastic and extensive supervision during all stages of my research. From obtaining a VISA for the United States to creating a final research report there have been many obstacles, which he handled with the utmost patience and willingness to help. I also wish to thank Richard Ridderinkhof for supervising me, even though I asked him to during his education sabbatical.

I would also like to thank Astrid Schomaeker and Wim van der Putten for their feedback on various stages of this manuscript. A final thanks goes to Ruud de Joode, Valentina Perdomo, Corine Quarles, Evan Russek and Oliver Vikbladt. They provided me with mental support to keep going, and occasionally with a helping hand or some constructive advice.

(25)

References

Andersson, J. L., Hutton, C., Ashburner, J., Turner, R., & Friston, K. (2001). Modeling geometric deformations in EPI time series. Neuroimage, 13(5), 903-919.

Atlas, L. Y., Bolger, N., Lindquist, M. A., & Wager, T. D. (2010). Brain mediators of predictive cue effects on perceived pain. The Journal of neuroscience, 30(39), 12964-12977.

Baird, B., Smallwood, J., Gorgolewski, K. J., & Margulies, D. S. (2013). Medial and lateral networks in anterior prefrontal cortex support metacognitive ability for memory and perception. The

Journal of Neuroscience, 33(42), 16657-16665.

Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., Noll, D. C., & Cohen, J. D. (1997). Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia,

35(10), 1373-1380.

Barttfeld, P., Wicker, B., McAleer, P., Belin, P., Cojan, Y., Graziano, M., ... & Sigman, M. (2013). Distinct patterns of functional brain connectivity correlate with objective performance and subjective beliefs. Proceedings of the National Academy of Sciences, 110(28), 11577-11582.

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2014). Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823.

Boldt, A., & Yeung, N. (2015). Shared Neural Markers of Decision Confidence and Error Detection. The Journal of Neuroscience, 35(8), 3478-3484.

Bona, S., & Silvanto, J. (2014). Accuracy and confidence of visual short-term memory do not go hand-in-hand: behavioral and neural dissociations. PloS one, 9(3), e90808.

Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: an update. Trends in cognitive sciences, 8(12), 539-546.

Bronfman, Z. Z., Brezis, N., Moran, R., Tsetsos, K., Donner, T., & Usher, M. (2015, July). Decisions reduce sensitivity to subsequent information. In Proc. R. Soc. B (Vol. 282, No. 1810, p. 20150228). The Royal Society.

Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D. (1998). Anterior cingulate cortex, error detection, and the online monitoring of performance. Science, 280(5364), 747-749.

Fetsch, C. R., Kiani, R., Newsome, W. T., & Shadlen, M. N. (2014). Effects of cortical microstimulation on confidence in a perceptual decision. Neuron, 83(4), 797-804.

Fleming, S. M., Huijgen, J., & Dolan, R. J. (2012). Prefrontal contributions to metacognition in perceptual decision making. The Journal of Neuroscience, 32(18), 6117-6125.

Fleming, S. M., Maloney, L. T., & Daw, N. D. (2013). The irrationality of categorical perception. The Journal of Neuroscience, 33(49), 19060-19070.

(26)

Fleming, S. M., Ryu, J., Golfinos, J. G., & Blackmon, K. E. (2014). Domain-specific impairment in metacognitive accuracy following anterior prefrontal lesions. Brain, 137(10), 2811-2822.

Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010). Relating introspective accuracy to individual differences in brain structure. Science, 329(5998), 1541-1543.

Fox, J., & Weisberg, S. (2010). An R companion to applied regression. Sage.

Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003). Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions. Psychonomic Bulletin & Review, 10(4), 843-876.

Hebart, M. N., Schriever, Y., Donner, T. H., & Haynes, J. D. (2014). The relationship between perceptual decision variables and confidence in the human brain. Cerebral Cortex, bhu181.

Hilgenstock, R., Weiss, T., & Witte, O. W. (2014). You’d Better Think Twice: Post-Decision Perceptual Confidence. NeuroImage, 99, 323-331.

Holland, P. W. (1988). Causal Inference, Path Analysis and Recursive Structural Equations Models. Program Statistics Research, Technical Report No. 88-81.

von Holstein, C. A. S. S. (1970). Measurement of subjective probability. Acta Psychologica, 34, 146-159.

Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227-231.

Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. science, 324(5928), 759-764.

Lau, H. C., & Passingham, R. E. (2006). Relative blindsight in normal observers and the neural correlate of visual consciousness. Proceedings of the National Academy of Sciences, 103(49), 18763-18768.

McCurdy, L. Y., Maniscalco, B., Metcalfe, J., Liu, K. Y., de Lange, F. P., & Lau, H. (2013).

Anatomical coupling between distinct metacognitive systems for memory and visual perception. The Journal of Neuroscience, 33(5), 1897-1906.

Moran, R., Teodorescu, A. R., & Usher, M. (2015). Post choice information integration as a causal determinant of confidence: Novel data and a computational account. Cognitive psychology, 78, 99-147.

Moritz, S., Gläscher, J., Sommer, T., Büchel, C., & Braus, D. F. (2006). Neural correlates of memory confidence. Neuroimage, 33(4), 1188-1193.

Pleskac, T. J., & Busemeyer, J. R. (2010). Two-stage dynamic signal detection: a theory of choice, decision time, and confidence. Psychological review, 117(3), 864.

Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009). Changes of mind in decision-making. Nature, 461(7261), 263-266.

(27)

Roitman, J. D., & Shadlen, M. N. (2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. The Journal of neuroscience, 22(21), 9475-9489.

Rounis, E., Maniscalco, B., Rothwell, J. C., Passingham, R. E., & Lau, H. (2010). Theta-burst transcranial magnetic stimulation to the prefrontal cortex impairs metacognitive visual awareness. Cognitive Neuroscience, 1(3), 165-175.

Smith, P. L., & Vickers, D. (1988). The accumulator model of two-choice discrimination. Journal of Mathematical Psychology, 32(2), 135-168.

Sobel, M. E. (2008). Identification of causal parameters in randomized studies with mediating variables. Journal of Educational and Behavioral Statistics, 33(2), 230-251.

Song, C., Kanai, R., Fleming, S. M., Weil, R. S., Schwarzkopf, D. S., & Rees, G. (2011). Relating inter-individual differences in metacognitive performance on different perceptual tasks.

Consciousness and cognition, 20(4), 1787-1792.

Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., ... & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic

anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15(1), 273-289. Vlassova, A., Donkin, C., & Pearson, J. (2014). Unconscious information changes decision accuracy but not confidence. Proceedings of the National Academy of Sciences, 111(45), 16214-16218.

Wager, T. D., Davidson, M. L., Hughes, B. L., Lindquist, M. A., & Ochsner, K. N. (2008).

Prefrontal-subcortical pathways mediating successful emotion regulation. Neuron, 59(6), 1037-1050. Wager, T. D., Waugh, C. E., Lindquist, M., Noll, D. C., Fredrickson, B. L., & Taylor, S. F. (2009). Brain mediators of cardiovascular responses to social threat: part I: reciprocal dorsal and ventral sub-regions of the medial prefrontal cortex and heart-rate reactivity. Neuroimage, 47(3), 821-835.

Watson, A. B., & Pelli, D. G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception & psychophysics, 33(2), 113-120.

Wilimzig, C., Tsuchiya, N., Fahle, M., Einhäuser, W., & Koch, C. (2008). Spatial attention increases performance but not subjective confidence in a discrimination task. Journal of Vision, 8(5), 7.

Yeung, N., & Summerfield, C. (2012). Metacognition in human decision-making: confidence and error monitoring. Philosophical Transactions of the Royal Society B: Biological Sciences,

(28)

Appendix A:

(29)

Appendix B

Effect of pre-decision and post-decision coherence on confidence responses

Behavioral session, displayed per condition

Confidence as an Integration of Choice and Post-Decision Evidence