Exploring the relationship between readiness potential and
pupillary response prior to voluntary action in a new framework for
self-initiated movement
By Anne Zonneveld (11645571), intern at the Conscious Brain Lab Mentor: Stijn Nuiten
19-06-2020
Keywords: readiness potential, pupil, arousal, free-will, accumulator model Abstract
Classically, volition has been studied under the assumption that the RP seen before voluntary action is a causal marker of preparation to act. Recently a new movement is studying the origin of the RP in light of fluctuations of cortical state, proposing the RP is rather a manifestation of continuous spontaneous fluctuations in cortical activity that brings the activity level closer to or further from an internal decision threshold (accumulator
framework). Several studies suggest that cortical state is regulated by arousal, i.e. activity of neuromodulators, and can be indexed by pupil diameter. Thus, one would also expect a particular pupillary response prior to voluntary action, although conclusive evidence concerning the role of pupil-linked arousal and cortical state in the initiation of voluntary actions remains elusive. This is thus the focus of the current study, while also exploring the proposed accumulator framework. This was done by empirical research with a voluntary keypress task and by modelling, using a leaky stochastic accumulator model. The current study found evidence for a particular pupillary response in line with previous research, showing an increase in pupil dilation prior to voluntary action. However, there was no clear support for the theory of the RP being modulated by this pupillary response, except for the fact that models for high arousal were associated with relatively lower noise parameter values than models for low arousal. We also explored the accumulator framework, but did not find any supporting evidence. This new framework is very much in its early stages and requires much more research to be done in the future.
Introduction
Volition is very complex and nuanced concept. Just now you have made the voluntary choice to read this thesis. Or at least intuitively it feels like it was you who consciously chose to perform this action, which caused that action to happen. We use this type of dualistic language in our daily lives, but this view of endogenous causation is actually problematic, since it implies that there is a mental ‘you’ distinct from your body and brain that causes bodily movement (Ryle, 2002). Therefore, in the research area of decision making and volition a voluntary action is defined as being free from immediacy, meaning being
independent from any direct external stimuli (Shadlen & Gold, 2004). This makes empirical research on voluntary actions very complicated and therefore mostly highly controversial. E.g., the results of the classic experiment by Libet et al. (1983) are still up for debate. This study showed that the conscious experience of intention in a voluntary action is preceded by a so-called readiness potential (RP), a negative potential measured over the motor cortex that starts ramping up in negativity ~500 ms before the actual action. Participants were asked to report the time on a clock when they first felt the intention (or urge) to move. This experience of intention preceded the actual movement by ~200ms. A more recent study, using the same paradigm, shows a similar pre-urge RP build up at single neuron level (Fried et al., 2012). These studies suggest that our actions are actually shaped by (unconscious) neural processes and that our conscious experience of intention is just a byproduct that only emerges afterwards. This implies a very restricted role for conscious control in voluntary action, which could have great implications for our conceptions of freedom and moral responsibility in society (Haggard et al., 2008). However, the validity of the original Libet paradigm and the assumption of the RP being some sort of causal marker of preparation of voluntary actions has been challenged by several studies (Haggard et al., 2008; Guggisberg and Mottaz, 2013; Miller et al., 2011; Trevena and Miller, 2010; Schurger et al., 2012). Schurger et al. (2012) propose that this increasing negative potential over the motor cortex seen before action is rather a manifestation of continuously ongoing spontaneous
fluctuations in cortical activity that brings the activity level closer to or further from an internal decision threshold. Surpassing this threshold will lead to the neural decision to move now (which is conceptually different from the experience of intention to move). In the study by Schurger et al. (2012) this process of threshold crossing was modelled with a leaky stochastic accumulator (LSA) with four parameters: drift, leak, bound and noise. This was empirically sufficient to explain the typical shape of the RP and the distribution of waiting times (WT; interval between start trial and issuance of action) as seen in the results of the study by Libet el al. (1983) (Fig. 1).
Following the theory of Schurger et al. (2012), if interrupted by a compulsory response cue, it would be more likely that fast responses primarily happen when the spontaneous
fluctuations were already close to threshold, possibly due to processes reflected by any of earlier mentioned parameters (drift, leak, bound or noise). Indeed, their model and empirical data showed that fast responses were preceded by significantly larger-amplitude negative going voltage deflections than slow responses. This seems to show that voluntary actions originate from an accumulation of internal physiological noise that sometimes exceeds the neural threshold to move now based on chance and that sensory cued responses can also be preceded by RP-like activity.
Fig 1. From article Schurger et al. (2012). The stochastic-decision model reproduces the distribution of waiting times and the characteristic shape and time course of the RP. (A) Visualization of model. After a stochastic exponential transition period, the leaky accumulator generates noisy trajectories whose threshold crossings determine movement times. An epoch centered on the threshold crossing time is extracted and the accumulator is then reset to zero for the start of the next trial. The shaded thick line in foreground shows the mean trajectory over 1000 simulated trials. (B) Mean waiting time (time from trial onset to threshold crossing) distribution from empirical data (gray line) and output of the simulation (black line). Inset shows distribution of al data from all subjects pooled together. (C) Mean RP (average over all simulated epochs, time locked to threshold crossing) from empirical data (gray line) and output of the simulation (black line).
Thus, the origin of the RP is now being studied in the light of ongoing fluctuations in cortical activity. Classically, fluctuations of cortical state have been related to wakefulness and the sleep cycle. The cortex would be synchronized during slow-wave sleep, characterized by strong low-frequency oscillations and correlation of activity between neurons, and
desynchronized during rapid eye movement sleep and waking, characterized by suppression of strong low-frequency oscillation and decorrelation of neuronal activity (Steriade &
McCarley, 2005). However, many recent rodent studies demonstrate that cortical state is not bimodal but actually a very complex continuum that also differs during wakefulness and that ideally should be determined by multiple neuronal and behavioral variables (Poulet & Petersen, 2008; Greenberg et al., 2008; Okun et al., 2010; McGinley et al., 2015). Though, generally speaking, desynchronized cortical state is often associated with active behavior, like locomotion and whisking in the case of rodents (Poulet & Petersen, 2008), but also with enhanced behavioral performance (McGinley et al., 2015; Beaman et al., 2017) and sensory perception (Supèr et al., 2003). These changes in cortical state and behavior are suggested to be regulated by arousal, i.e. the global activity of neuromodulators like acetylcholine and noradrenaline, often measured by pupil diameter (Janisse, 1977). Recent studies focus on the effects of these neuromodulators on perception by using pupil diameter as an index for cortical state (De Gee et al., 2017; Gilzenrat et al., 2010). Pupil diameter seems to be a good proxy to assess cortical state, since there is an evident relationship between pupil diameter, low frequency oscillations in membrane potential/local field potential and exploratory
behavior (McGinley et al., 2015; Reimer et al., 2014, Vinck et al., 2015, Gilzenrat et al., 2010). Active behavior was found to be associated with suppression of low-frequency oscillations and large pupil size. Even in absence of this active exploratory behavior, the presence of a large pupil size was associated with a reduction of low frequency rhythmic activity. This relation between pupil diameter and cortical activity has been observed across different sensory cortical areas and its generality is underlined by the relationship between pupil diameter and sharp-wave ripple rate in the hippocampus (McGinley et al., 2015) (Fig. 2).
Fig. 2. From article McGinley et al. (2015). Pupil diameter is an accurate predictor of variations in different parameters related to brain state. (A) Experimental set up. (B) Simultaneous recording of pupil diameter and membrane potential of cortical neuron in layer 2/3 of primary visual cortex. Exhibits a strong relationship between slow rhythmic activity (2-10 Hz) and pupil constriction and between suppression of this activity and pupil dilation (‘desync’). (C) Simultaneous recording of pupil diameter and membrane potential of cortical neuron in layer 5 of auditory cortex. Exhibits tight anticorrelation: increase of pupil diameter is being associated with suppression of low-frequency activity. (D) Relationship between pupil diameter and rate of occurrence of ripples in CA1 of the hippocampus exhibits a tight anticorrelation: increase of pupil diameter is being associated with suppression of ripples.
Combining the results of these studies, if voluntary movements indeed originate through a manifestation of accumulation of spontaneous fluctuations of cortical activity, specifically neuromodulator-induced, then one would also expect to find a particular pupillary response prior to action. A study by Einhauser et al. (2010) shows such particular response in pupil diameter prior to voluntary action, namely an increase, and a significant relation between timing of this response and time of action. One would specifically expect this particular response in pupil diameter to be linked to the amount of synchrony of cortical state during the process to threshold crossing. The amount of synchrony could be reflected by the standard deviation of cortical activity during this process or by the noise parameter in the model of Schurger et al. (2012). However, there seems to be not much literature about this topic, especially literature linking the two phenomena of the cortical RP and the pupillary dilation prior to voluntary action. This study therefore focuses on the relationship between pupil diameter and cortical activity prior and during the act of voluntary movement, while also exploring the RP framework focused on accumulation of spontaneous neural activity
proposed by Schurger et al. (2012). This was performed by looking at different properties of cortical activity and pupil diameter in relation to reaction times. For the task in this study, an adapted paradigm was used in which participants were presented with a stream of letters at a constant frequency and were instructed to perform the ‘voluntary’ action of pressing the spacebar, whenever they felt like doing so (adapted from Parés-Pujolràs et al. (2019)). After execution of this action, participants were sometimes asked to recall the letter that was presented when they first felt the intention to act. An interruption condition (the occurrence of a loud noise) was included, for which participants were instructed to react as fast as possible by pressing the spacebar, based on the ‘Libetus Interruptus’ paradigm from Schurger et al.
(2012). During the task, pupil diameter was measured with an eye tracker and the cortical activity was recorded with EEG.
Based on previous research, we expected a gradual build up in pupil diameter and negative potential over the cortex prior to movement, which we thought to be linked by the amount of synchronization during the process to threshold crossing (possibly being reflected by the standard deviation of EEG signal or the noise parameter in the model of Schurger et al. (2012)). Furthermore, in the case of interruption trials, we expected fast reaction times to be associated with relatively larger amplitude negative cortical potential and also with relatively larger pupil diameter.
This study was set up to give new insight on how cortical activity and arousal interact during the emergence of voluntary actions and could possibly be used to support the model
proposed by Schurger et al. (2012). This then could have greater implications for our understanding of the fundamentals of free will.
Materials and methods
Participants
We recruited 2 participants through an online advertisement on the website of LAB (lab.uva.nl), a section of the Faculty of Social and Behavioral Science of the UvA (FMG-UvA). All participants had normal or corrected to normal sight. The study was approved by the Ethics Review Board (FMG-UvA). All participants gave written informed consent before starting the experiment and were rewarded for their participation with €15 / 1.5 research credits per hour. Participants were invited for one session of three hours.
Stimuli and task
Participants were seated in a quiet room ~ 50 cm away from a computer screen. Instructions were handed out beforehand on paper, verbally explained and then later also displayed on the computer screen. All tasks were programmed in Python 2.7, with the use of Psychopy (Version 1.84.1, Peirce et al., 2019), Pygaze (Version 0.6.0a25, Dalmaijer et al., 2014) and with custom made software.
The behavioral task was preceded by a so called ‘flicker-fusion’ measurement to determine subjective equiluminance and minimize visually evoked pupil responses (Raphael &
MacLeod, 2011). Participants had to manipulate the flickering intensity by slightly changing the hue of a red/green/blue colored disk until this appeared somewhat stable to them. The according RGB color values of the most stable color, according to the participant, were then used for the color of the stimuli in the behavioral task.
The behavioral task was divided into blocks of 100 trials. A trial was defined as every time the participant pressed spacebar. There was no predetermined quantity of blocks, only a predetermined duration of the whole session (3 hours, including setting up). The pseudo-random letter stream that was shown consisted of only consonants and any repetitions were separated by 8 other letters. Letters were presented at a frequency of 6 Hz (stimulus
duration time ~166,7 ms). The task could be categorized into three conditions (Fig 3): 1. Self-paced condition
Participants were instructed to watch the letter stream on the screen and press space whenever they felt like it. The difference between the start of the trial and the
instructed not to use any tactics or strategies (e.g. pressing every time the letter ‘k’ appears) and to try not to fall into a rhythm of automatic motor responses. They were also told that there was no need to rush their responses, since the duration of their participation was predetermined. Furthermore, participants were instructed to keep track of the moment they first felt the intention to move and to remember the letter that was presented at that moment, since they sometimes would have to report it. 2. Urge report condition
Occasionally (0.2 chance), after freely having pressed space, the letter stream would be terminated and the participant would be asked which letter was on the screen when they first felt the urge to move. Participants then had to press the according key on the keyboard and confirm their answer by pressing space. The difference between the moment of presentation of this assigned ‘urge letter’ and the actual keypress was defined as the Libet time (LT).
3. Interrupted condition
Sometimes, an interruption (a loud noise) would occur during the letter stream. Participants were instructed to press space as fast as possible within 2 seconds after the occurrence of an interruption. The letter stream would continue and within these 2 seconds after the occurrence of this interruption no urge report could be asked. The difference between the occurrence of the interruption and the keypress was defined as the reaction time (RT).
The first block, the ‘classic’ block, was a separate block, consisting of only 50 trials without interruptions, and was used to determine a personal WT distribution. From that block on, every block was a so called ‘interruption block’. Interruptions could occur any time during a trial within the limits of the mean WT divided by two and the mean WT multiplied by two, if this trial was predetermined to be an interruption trial (0.75 chance). However, if the current trial was to be an interruption trial but the participant performed a voluntary action before the actual occurrence of the interruption, this trial would be classified as a self-paced trial.
Eye tracking
During the whole behavioral task the pupil diameter was measured with an EyeLink 1000 Plus eye tracker (SR Research, Mississauga, Ontario, Canada). All data was recorded with a 1000 Hz sampling frequency. The eye tracker was calibrated with 9-point calibration at the beginning of every block.
EEG recording
For the EEG recording a Biosemi ActiveTwo system with 64 Active electrodes, arranged according to the international 10/20 system, was used. The data was recorded using the Biosemi AD-box and USB2-receiver with a 1024Hz sampling frequency. All electrodes were referenced to two external electrodes on the right and left earlobe. Four ocular electrodes, one above and one below the right eye and two on the outer canthi of both eyes, were used to record electro ocular activity.
Data analysis
All analyses were performed in Python 3.7. Most statistical analyses were only conducted on subject level, because of the restricted quantity of available participants due to the COVID-19 outbreak. All statistical analyses were performed two-sided and with a significance-level of = 0.05. For all analyses concerning interruption trials, trials of which the RT exceeded the limits of the subject mean RT ± 3*SD were excluded from the dataset (absolute [3 ± 0] ; percental [0.46 ± 0.04]).
Behavioral data analysis
All statistical behavioral analyses were conducted on group level.
First, the WT probability density distribution for the classic block and interruption block were calculated. A two sample Kolmogorov-Smirnov test was used to test goodness of fit between the two distributions.
Additionally, the mean LT was calculated for self-paced trials that contained an urge report for the classic block and interruption block separately. The assumption of normality was tested with a Shapiro test. If met, a two-sample t-test would be performed (otherwise a Mann-Whitney U test).
Furthermore, in case of interruption trials, the mean pre-interruption time, being the time between the start of the trial and the occurrence of the interruption, was calculated separately for fast, medium and slow response trials (based on a RT third split).
Assumptions of normality and equal variance were tested with a Shapiro and Levene’s test. If met, an ANOVA would be conducted (otherwise a Kruskal Wallis test).
EEG data analysis
For all EEG analyses, data was down sampled from 1024 Hz to 64Hz. Analyses were conducted with the MNE toolbox (Gramfort et al., 2013) for Python. A special trigger channel was used to insert temporal markers in the data for trial onset, keypresses, stimuli (letters in letter stream) and occurrence of interruptions. The electrodes of interest were Cz, CPz, FCz, C1 and C2. Every channel was checked individually for noise and interpolated if noisy (absolute [1.5 ± 0.5] ; percental [30 ± 10]). If one of the following conditions was true, epochs/trials were excluded from the EEG and pupil dataset (absolute [15 ± 8] ; percental [2.23 ± 1.06]):
- If a self-paced action was performed before presentation of the first letter of a trial (too fast trials).
- If a self-paced action occurred right after interruption presentation, but is still registered as voluntary (coincidence trial).
- If there was no reaction within 2s of interruption presentation (miss trial).
- If there was any other form of incoherency between the classification of epochs/trials in EEG and behavior.
To create a more robust and reliable signal, data from all channels of interest was averaged. Then, different baselines were applied for different epoch types (Table 1), after which independent component analysis (ICA) was used to remove ocular artifacts from the EEG signal. No detrending was performed.
Epoch type Baseline
Action locked -3.5s to -1s
Stimulus locked -2.5s to 0s
Stimulus locked stimulus presentation control
-0.5s tot 0s
Table 1. Baselines used for different epoch types for EEG data.
Subject level
Analyses on subject level were only conducted for the interruption block. Epochs were extracted for self-paced trials and interruption trials separately with a time range of -3.5s to 1s time locked to action and time-locked to stimulus. If a self-paced trial contained an urge report, an intention-locked epoch would be extracted, on top of the action-locked and
stimulus-locked epochs. These epochs were then averaged and the accompanying standard error mean (SEM) for every timepoint was calculated to create ERP signals for the different conditions. A cluster-corrected permutation t-test with 1024 permutations was performed for self-paced trials, interruption trials and difference scores between self-paced and interruption trials for both action-locked and stimulus-locked epochs.
Another analysis was performed for the interruption trials, focusing on the relationship between RT and cortical activity. Trials were divided into fast response trials and slow response trials (based on a RT median split). Action-locked and stimulus-locked epochs with a time range of -3.5s to 1s were extracted for both conditions separately. Epochs for all different conditions were averaged and the accompanying SEMs calculated to create an ERP signal. A cluster corrected permutation t-test with 1024 permutations was performed for fast response trials, slow response trials and difference scores between fast and slow response trials for both action-locked and stimulus-locked epochs.
To control for the effect of stimulus presentation on cortical activity for different RTs, an ERP was calculated by baselining stimulus-locked interruption epochs with a baseline from -0.5s to 0s for fast and slow response trials.
Additionally, based on the mean EEG amplitude of stimulus-locked interruption trials over the time course of -2.5s to 0, the mean reaction time was calculated for lower and higher median EEG conditions. The assumption of normality was tested with a Shapiro test. If met, a two-sample t-test would be performed (otherwise a Mann-Whitney U test).
Group level
The same procedure, but with subject ERP data, was performed on group level to create grand average ERP data. However, no statistical analyses were conducted on group level, since we were restricted to data of only two subjects. On top of that, a few other analyses
were conducted.
To control for the use of different paradigms, epochs with a time range of -3.5s to 1s time locked to action were extracted separately for self-paced trials in the classic block and the interruption block for all participants. These self-paced epochs were then averaged and the accompanying SEM was calculated to create grand average ERP signals for the two different conditions.
Furthermore, self-paced epochs and interruption epochs with a time range of -3.5s to 1s time-locked to action were extracted from the interruption block for all participants. If a self-paced trial were to contain a urge-report, another intention-locked epoch (with time range of -3.5 to 1s) would be extracted. These intention-epochs were averaged and the
accompanying SEMs calculated to create a grand average ERP.
Pupil data analysis
For all pupil analyses z-scored data (over the whole session) of the left eye was used. Generally no baseline correction was performed, except to create the ERP signal to control for the effect of stimulus presentation. The same procedure as for the EEG data analysis was used to extract and analyze the pupil diameter on both subject and group level.
Model data analysis
Similar to Schurger et al. (2012), a stochastic-decision model that implements a leaky stochastic accumulator process was used to model the neural decision to act now. However, our model included four free parameters (drift, leak, bound and noise) instead of three (drift, leak, and bound). These four parameters were estimated separately for self-paced trials for the classic block and the interruption block. Parameters were fit using an iterative differential evolution process, where on every iteration 1000 trials were simulated under a particular set of parameter values. Waiting times for these trials were defined as the first threshold
crossing time from the start of the trial. The log-likelihood of this resulting simulated WT distribution given the empirical WT distribution was calculated via kernel density estimation (KDE). The differential evolution algorithm ultimately produced the set of parameters resulting in the highest log-likelihood.
Subsequently, all self-paced trials were divided separately for the classic block and
interruption block in high and low arousal trials based on the according mean pupil diameter (measured over a timeframe form -3.5s to -1s to action). Since we were mainly interested in the relationship between pupil diameter and cortical state, all parameters were fixed at the previous found values for the according block except for the noise parameter. The SD of the EEG signal and thus the amount of synchrony could namely possibly be approximated by noisiness. The same procedure as before was used to find the model and corresponding noise parameters with the highest log-likelihood for the different arousal conditions. Within these models, the influence of the noise parameter on the WT distribution was assessed by fixing all parameters at the previously found values and implementing different pre-set noise parameter values ranging from 0.1 to 0.5.
Intercategorical analysis
An intercategorical analysis was done by performing correlation tests between different properties of pupil, cortical activity and behavior. Properties of interest were calculated separately for self-paced action- locked epochs and stimulus-locked interruption epochs with
a time frame of -3.5s to -1s and -2.5s to 0s respectively. The properties of interest were the following:
- Mean pupil diameter
- Mean EEG amplitude of channel averaged signal
- SD EEG amplitude of channel averaged signal
- Slope of EEG amplitude of channel averaged signal
For both self-paced and interruption epochs separately it was evaluated if any of the mentioned properties of an epoch exceeded the limits its subject mean ± 3*SD. If so, the concerned epoch would be excluded from intercategorical analysis (absolute [14 ± 1] ; percental [2.18 ± 0.32] .
First, correlation tests between mean pupil diameter and the different EEG signal properties during self-paced trials were conducted. The assumption of bivariate normality was tested with a Shapiro test for both the dependent and independent variable. If met, a Pearson correlation test would be performed (otherwise a Spearman correlation test).
Second, correlation tests between RT and all different properties of interest during
interruption trials were conducted. The assumption of bivariate normality was tested and the according correlation test was used.
Results
Behavioral data
The following WT distributions were found for the classic block and interruption block (Fig 4A). A two sample Kolmogorov-Smirnov test showed that the distributions are significantly different [D = 0.165, p = 0.013]. Subjects were more likely to have a shorter WT in the classic block than in the interruption block. However, no significant difference in LT was found between the two blocks [U = 1221, p = 0.304] (Fig 4B).
Furthermore, no significant difference in pre-stimulus time was found for different RT trials (fast, medium and slow) [K = 0.826, p = 0.663], showing that different RTs were equally distributed throughout the timespan of trials during the interruption block (Fig 4C).
EEG data
Reviewing the group level data, the EEG signals for the classic block and the interruption block followed the same temporal pattern, but showed different magnitudes (Fig 5A). First
the signals moved from a positive potential to a negative potential until -0.2s to action. Then both signals increased and peaked around 0.2s after action. A similar temporal pattern in
EEG signal was found for interruption trials (Fig 5B), however the peak in amplitude was much larger for the interruption signal compared to the voluntary signal and seemed to have a slightly larger slope. These two characteristics for the interruption signal were consistent among subjects and both showed a significant difference between the interruption condition and self-paced condition around timing of the peak (see Supplementary, Fig S1). The temporal pattern of both self-paced and interruption trials was however not consistent between subjects. The signals for subject 6 followed the previously mentioned pattern (Fig S1B), while signals for subject 5 first increased from -3.5s to -2.0s, then decreased until roughly -0.5s and then again increased and peaked (Fig S1A). This resulted in both subjects showing an ongoing window of significant difference between the two conditions from -3.5s to -2.3s. However, for subject 5 this was due to the fact that interruptions trials had relatively more negative potential than self-paced trials and for subject 6 because of the fact that interruption trials had a relatively less negative potential than self-paced trials. Also, for subject 5 the maximum of the peak for interruption trials seemed to be located relatively earlier compared to maximum of the peak for self-paced trials than for subject 6.
The general temporal pattern could also be seen for self-paced trials that contained an urge-report (Fig 5C). The mean timing of intention was -559ms to action, slightly before EEG signal starts to increase.
To investigate the relationship between cortical activity and RT, interruption trials were divided in fast and slow RT trials through a median split. It seemed like the signal for slow RT trials moved from a positive to a negative potential from -3.5s to -0.3s to action (Fig 6A) (or to 0.1s after stimulus presentation (Fig 6C)), while the signal for fast RT trials seemed to fluctuate close to a potential of 0 µV in the same time window. After this, both signals
increased and peaked. However, this development in activity for slow and fast RT trials over time did not seem to be consistent among subjects. The signals for subject 5 moved from a negative to a positive potential (Fig S2A and S2B), while the signals for subject 6 moved from a positive to a negative potential (Fig S2D and S2E). Data from both subjects did show an increase and peaked -0.3s to action (Fig S2A and S2D)(or 0.1s after stimulus
presentation (Fig S2B and S2D)).
Over time the relationship between the two signals shifted, resulting in the fast RT signal having a relativily more positive potential than the slow RT signal shortly prior to action and
stimulus presentation (Fig 6B and 6C). This difference in potential was especially large around the timing of the peak, from -0.3 to action (Fig 6B) or 0.1s after stimulus presentation (Fig 6D). Both subjects also showed a significant difference between the fast and slow RT signals around the timing of the peak (Fig S2A, S2B, S2D, S2E). However, no significant difference in RT was found between lower and higher median EEG activity for either of the participants [U = 9058, p = 0.310; t = 1.567, p = 0.121] (Fig S3A, S3B) and results showed opposite directions.
Pupil data
Reviewing the group level data, pupil diameter followed the same pattern for the classic block as the interruption block (Fig 7A). Pupil diameter decreased until -0.7s from action, after which it started to increase. Around the performance of action this increase attenuated. This same development could also be seen for the different trial types (during the
interruption block) (Fig 7B). During this preamble to action, self-paced trials had an overall-larger pupil diameter than interruption trials. However, when looking at individual subject data (Fig S4A and S4B), this was not a consistent phenomenon. For subject 5 interruption trials had an overall larger pupil diameter (even significantly from -1.4s to 1.0s ) (Fig S4A), but the opposite is true for subject 6, however there was no significance between the two conditions (Fig S4B).
For intention-locked self-paced epochs pupil diameter decreased right until -0.2s to the experience of intention, after which pupil diameter increased again (Fig 7C).
To investigate the relationship between pupil diameter and RT, interruption trials were divided in fast and slow RT trials through a median split. Again, the same general
development of the pupil diameter over time for both fast and slow RT trials was seen: first pupil diameter decreased and then it increased (Fig 8A and 8C). Stimulus-locked data showed that this increase started just before t=0, the time of stimulus presentation (Fig 8C). To control for the effect of stimulus presentation, an analysis on subject level was done for
baseline corrected data (baseline from -0.5s to 0s) for stimulus locked interruption epochs (Fig S5C and S5F).The baseline corrected stimulus evoked data showed an increase of pupil diameter after stimulus presentation, for which the effect was significant for both fast and slow RT trials for subject 5 around 0.3s to 1.5s after stimulus presentation (Fig S5F). Additionally, there was a significant difference between the two conditions from 1.3s to 1.7s after stimulus presentation. Also, slow RT trials predominantly had a larger pupil diameter than fast RT trials (fig 8B and 8C). However, when looking at subject level data there was no constancy in relativity of pupil diameter signals among subjects over time. For subject 5, fast RT pupil diameter was first larger than slow RT pupil diameter, but then from -2.3s to action precedes to be smaller than slow RT pupil diameter. For subject 6, fast RT pupil diameter was first smaller than slow RT pupil diameter, but then from -2.3s to -1.0s was larger than slow RT pupil diameter, after which again it was smaller than slow RT pupil diameter. However, for subject 5 there was a window of significant difference between fast and slow RT trials from -0.7s to -0.3s to action (Fig S5A). Nonetheless, this window of significant difference was not seen for the stimulus locked signal and neither for subject 6. Additionally, no significant difference in RT was found between small and large pupil trials for both
Model data
For both the classic block and interruption block independently a 1000 trials were simulated with all parameters being free. The following fits with corresponding parameters values were found for the different models (Table 2 and Fig 9).
Block Drift Leak Bound Noise
Log-likelihood
Classic 0.461 0.885 0.516 0.581 173.105
Interruption 0.304 0.673 0.433 0.344 1445.537
Then, all self-paced trials were divided in high and low arousal, based on a median split of mean pupil baseline. All parameters, except for noise, were fixed at the previously found values and the different models were run separately for high and low arousal. This resulted in the following fits with corresponding noise values (Table 3 and Fig 10A, 10B, 10D and 10E).
Type Noise Log-likelihood
Classic low arousal 0.092 103.919
Classic high arousal 0.091 60.467
Interruption low arousal 0.109 698.201
Interruption high arousal 0.103 752.714
Table 3. Noise parameter values for different models with corresponding highest likelihood. Individually determined for different types.
Overall, it was evident that all models for the interruption block had a higher log-likelihood than the models for the classic block. For both the classic block and the interruption block, it could be seen that high arousal trials resulted in a relatively lower noise parameter value than low arousal trials.
Within each model, the influence of the noise parameter on the WT distribution was
assessed. Both models showed a relatively higher probability of shorter WTs for larger noise parameter values (Fig 10C and 10D).
Intercategorical data
Several correlation tests were performed on subject level. First, we reviewed the relationship between mean pupil and several EEG properties: mean amplitude, SD amplitude and slope (Fig 11). None of the relationships were significant. Also, the relationship between mean pupil diameter and mean EEG signal [r = -0.018, p = 0.729; r = 0.003, p = 0.953] (Fig 11A and 11D) and the relationship between mean pupil diameter and SD EEG signal [r = 0.062,
p = 0.218; r = -0.071, p = 0.120] (Fig 11B and 11E) did not show the same direction between
subjects. This was only the case for the relatonship between mean pupil and EEG slope (Fig 9C and fig 9F), which were both best portrayed by a negative correlation coefficient [r = -0.049, p = 0.336; r = -0.027, p = 0.557]. More correlation tests were conducted to asses the relationship between RT and mean pupil diameter, mean EEG amplitude, SD EEG
amplitude and EEG signal slope (Fig 12). Only the relationship between RT and SD EEG signal for subject 6 was siginifcant, showing a positive correlation coeffcient [r = 0.265, p =
0.010] (Fig 12G). However, this relationship showed an opposite direction for subject 5,
being close to significance [r = -0.111, p = 0.068] (Fig 12C). The relationship between RT and mean EEG signal was also of opposite directions among subjects [r = -0.013, p = 0.830; r = 0.197, p = 0.059] (Fig 12B and Fig 12F). The relationship between RT and mean pupil (Fig 12A and 12E) and the relationship between RT and EEG signal slope (Fig 12D and 12H) were consistent among subjects and were best portrayed by a positive correlation coeffcient [r = 0.033, p = 0.591; r = 0.047, p = 0.653] and a negative correlation coeffcient [r = -0.012, p = 0.844; r = -0.163, p = 0.120] respectively.
Discussion
Similarly to the results of Schurger et al. (2012), subjects behaved differently in regards to WT for the classic block and interruption block. Therefore, it would be sensible to analyze the data for the two blocks separately, as we did. Our EEG data shows similar negative going RP-like activity prior to voluntary action as seen in Schurger et al. (2012). However, one noticeable difference is that for Schurger et al. (2012) the potential was at its lowest at the time of action, while in our data this lowest potential seemed to occur earlier (e.g. Fig 5A). A possible explanation could be that in our case the neural decision to move was made relativity early, after which preparation signals from the motor cortex were already being registered, causing a relatively early increase in positivity of the signal. Furthermore, there was no significant difference in pre-stimulus time for different RT trials. Thus, different RTs were equally distributed throughout the timespan of trials. This cancels out the contingent negative variation (CNV) theory (Birbaumer et al., 1990), stating that the RP would reflect the preparation to act that slowly builds up over the course of a trial, as a possible
explanation of the RP, since, if true, one would expect fast responses to happen relatively late in a trial compared to slow responses. These results are in line with Schurger et al. (2012) and enforce the theory that the RP is rather a manifestation of spontaneous fluctuations.
Reviewing the EEG data, RP-like activity similar to the data of Schurger et al. (2012) was seen for self-paced trials. It was also evident that the signal for the classic block had a larger magnitude than the signal for the interruption block. This is probably because of the large difference in sample size used to create the two signals (classic block N = 50; interruption block N = 496), which causes the interruption signal to be more robust and reliable. The same temporal pattern was seen for self-paced trials and interruption trials on group level (Fig 5B), but was not consistent between subjects (Fig S1). However, for both group level and subject level data, the positive peak in amplitude around 0s to action was much larger for interruption trials than for the voluntary trials. This is due to the stimulus evoked response caused by presentation of the interruption (Fig S2C and S2F). The positive increase of the EEG signal before action in interruption trials is thus partially due to presentation of the stimulus.
Our pupil data roughly showed a similar consistent pattern for all different possible
conditions for all different subjects. First, there was a decrease in pupil diameter and then a short period of relatively constant small pupil diameter, after which around -1s to action the pupil diameter started to increase and then peaked around 0.5s after action. Thus, pupil diameter seemed to increase shortly before anticipation of voluntary action. These results are in line with the study by Einhauser et al. (2010). For intention locked epochs, this increase started around -0.2s before intention. However, timing of intention was calculated with the reported ‘urge letter’, which was presented for 166.7 ms. Therefore, the actual intention could be anywhere within the range of this time period and thus it could be possible that pupil diameter actually started increasing right at the experience of intention. No
constancy in relativity concerning pupil amplitude was found among subjects between interruption trials and self-paced trials. The significant difference between the interruption signal and self-paced signal from -1.4s to 1.0s for subject 5 (fig S4A), can be explained by the found stimulus onset effect (fig S5C), showing that stimulus presentation provokes an increase in pupil diameter, which is present for interruption trials but absent for self-paced trials. This found stimulus onset effect suggests that part of the preliminary increase in pupil diameter before action seen in interruption trials is purely due to presentation of the stimulus.
The relationship between cortical activity and pupil diameter prior to voluntary action was investigated by performing several correlation tests. None of the relationships between mean pupil diameter and different EEG signal properties showed a significant correlation. Also, only the correlation coefficients for the relationship between mean pupil diameter and EEG slope showed the same direction across subjects (fig 11C and 11F), suggesting that a more negative EEG slope is associated with a larger pupil. There was no clear evidence
suggesting a negative relationship between mean pupil diameter and the SD of the EEG signal, even though this was expected based on previous research (McGinley et al., 2015; Reimer et al., 2014, Vinck et al., 2015, Gilzenrat et al., 2010). However, one could question the validity of the use of SD of the EEG signal as proxy for the amount of synchrony (or noise). E.g., a very steep negative deflection is not necessarily synchronized (or noisy), but probably does have a high SD. An alternative would be to analyze the relationship between pupil diameter and cortical state with a time-frequency analysis or to use a different measure such directional variance (Schurger et al., 2015) to capture synchrony of the EEG signal. Another factor that could have influenced the intercategorical analysis, is a possible lag between alterations in cortical state and pupil diameter due to the slow time course of pupillary movement to changes in (para)sympathetic activity (Loewenfield, 1999). McGinley et al (2015) e.g. found a lag of 1.4s between transient membrane depolarization and pupil dilation. The intercatergorical analysis used the same time frame for EEG and pupil data to extract information about different properties. Future research should take this lag into account and it would also be interesting to use a sliding time window for this type of analysis to see how the relationship between different properties change over time.
The relationship between cortical activity and pupil diameter prior to voluntary action was further investigated by modelling. Overall, all models for the interruption block had a higher log-likelihood than the models for the classic block (Table 2 and 3). This is probably due to the fact that the empirical WT distribution for the interruption block is much more robust, because of the bigger sample size (N = 923 vs N = 100), and thus easier to approximate. Models for high arousal trials (large mean pupil diameter) resulted in relatively lower noise parameter values than models for low arousal trials (small mean pupil diameter) (Table 3). The noise parameter value could possibly approximate the SD of the EEG signal and thus the amount of synchrony. In this case, the modeling results are in line with previous research (McGinley et al., 2015; Reimer et al., 2014, Vinck et al., 2015, Gilzenrat et al., 2010),
showing that large pupil diameter is associated with desynchronization (small SD EEG signal). When assessing the influence of the noise parameter on the WT distribution within the different models for the classic block and interruption block, it was evident that a larger noise parameter generally leads to higher probability of short WTs (fig 10C and 10F). The found association between large pupil diameter and a lower noise parameter value would then suggest that large pupil is related to a relatively lower probability of short WTs. This means that high arousal would be associated with relatively longer WTs, which feels counterintuitive. A pupillometry study on decision making by Kempen et al. (2019) that distinguished between tonic and phasic pupil response, i.e. pupil baseline and pupil response, did find that larger pupil baseline was predictive of short RTs. However, RT to a stimulus may not be comparable with WT of a self-initated movement. For future research it thus would be interesting to do a similar analysis also focused on the influence of the noise parameter, but then distinguishing between tonic pupil baseline and phasic pupil response and also modelling interruption trials and their corresponding RT distributions.
The current study also explored the newly proposed accumulation framework by Schurger et al. (2012). This was done by assessing the relationship between RTs of interruption trials and cortical activity/pupil diameter.
Reviewing the EEG group level data for the RT split interruption trials (Fig 6), the data looks similar to the data found by Schurger et al. (2012) at first glance. However, when looking closer, there are a few differences, namely the absence of a clear initial negative deflection after stimulus presentation and the fact that slow and fast RT signals switch in polar relativity mid time frame, resulting in slow RT trials having a relatively more negative potential shortly before stimulus presentation (Fig 6C). This switch was actually only seen for subject 6, however, since the signals of subject 6 were of a larger amplitude than those of subject 5, the data of subject 6 was more reflected in the group level grand average. To remove this between-subject variance, it would be sensible to use (over whole session) z-scored EEG data in future research, similarly to the pupil analysis. The fact that EEG amplitude of slow RT trials was relatively more negative than for fast RT trials shortly before stimulus
presentation was consistent among subject (Fig S5B and S5E). Also, both subjects showed a similar window of significant difference between fast and slow RT trials during peak increase (Fig S2A and S2C). This could mostly be explained by the significant difference found in the stimulus onset effect (Fig S2C and S2F), suggesting that cortical activity during fast RT trials was more affected by stimulus presentation and that fast RTs are associated with a relatively more positive evoked potential. These results are not in in line with the results found by Schurger et al. (2012).
Reviewing the pupil group level data for the RT split interruption trials (Fig 8), pupil diameter seemed predominantly larger (or say: has a larger baseline) for slow RT trials compared to fast RT trials. Similar results were found by van Kempen et al. (2019), but they also showed that large phasic pupil responses actually were predictive of fast RTs. Our study, however, did not include analysis of the relationship between phasic pupil response and RT. For future research it would be informing to distinguish between the tonic and phasic elements of pupil diameter response and relate these to RT, e.g. with correlation tests. Furthermore, no constancy in relativity concerning pupil diamterer was seen among subjects between fast and slow RT trials. However, there was an evident significant difference in pupil diameter between fast and slow RT trials seen for subject 5 from -0.7s to -0.3s (fig S5A). This significant difference could not possibly be the same effect as seen from 1.2s to 1.7s in the stimulus evoked pupil diameter (Fig S5C), since timing of these effects do not add up. Both effects are in line with the results found by van Kempen et al. (2019) that a higher pupil diameter baseline is predictive of short RTs.
Performing several correlation test between RT and different variables, only the relationship between RT and SD of EEG signal for subject 6 was significant (fig 12G), suggesting relatively small SD of EEG signal would be associated with fast RTs. This is not in line with the results of the model data, which suggest that a low noise parameter would be associated with a lower probability of short WTs. However, the model only concerns WTs and not RTs and, again, SD of EEG signal may not be the best proxy to capture synchrony (or noise). Additionally, this relationship between RT and SD of EEG signal showed an opposite direction for subject 5 (fig 12C). The relationship between RT and mean pupil diameter was consistent among subjects, suggesting large pupil diameter would be associated with larger RTs (fig 12A and 12E). This is in line with results of Kempen et al. (2019), showing that large baseline pupil predicts larger RTs. The relationship between RT and EEG signal slope also was consistent among subjects, suggesting a relatively negative slope would be associated
with a larger reaction time. This is however not in line with the theory by Schurger et al. (2012) that fast RT trials would be preceded by a relatively larger amplitude deflection. The very restricted amount of participants due to the COVID-19 outbreak limits the validity and reach of the current study. We were required to perform fixed-effect within subject analyses instead of group level population analyses. Thus, most conclusions should be critically received.
The current study found evidence for a particular pupillary response prior to voluntary action, similar to the phenomenon of the RP. The results were in line with previous research by Einhauser et al. (2010). Investigating the relationship between the RP in the EEG data and the preliminary pupillary response, we did not find any clear support for the theory of the RP being modulated by the pupillary response prior to voluntary action (arousal) through
affecting synchrony of cortical activity, except for the fact that models for high arousal were associated with relatively lower noise parameter values than models for low arousal. For future research it would be interesting to also analyze the relationship between cortical RP and pupillary response prior to voluntary action in a similar way to the study done by
Kempen et al. (2019) by looking at amplitude and build-up rate of the RP for different sorted pupil bins.
We also explored the framework by Schurger et al. (2012), proposing that the RP might be a manifestation of spontaneous activity. However, we did not find any supporting evidence for this framework. This new view on the classic experiment by Libet et al. (1983), focusing on the origin of the emergence of voluntary actions and experience of intention, is very much in its early stages and requires much more research to be done in the future.
Acknowledgements
I would like to give a special thanks to my mentor Stijn Nuiten, who assisted me during my internship and in writing this thesis. I would also like to thank dr. Simon van Gaal and the Conscious Brain Lab for providing me this opportunity. I thoroughly enjoyed the atmosphere of the research group and all the interesting weekly meetings and discussions about current research on consciousness.
References
Beaman, C.B., Eagleman, S.L., Dragoi, V. (2017). Sensory coding accuracy and perceptual performance are improved during the desynchronized cortical state. Nat Commun 8:1308. Birbaumer, N., Elbert, T., Canavan, A.G., & Rockstroh, B. (1990). Slow potentials of the cerebral cortex and behavior. Physiological reviews, 70(1), 1-41.
Dalmaijer, E.S., Mathôt, S., & Van der Stigchel, S. (2014). PyGaze: an open-source, cross-platform toolbox for minimal-effort programming of eye tracking experiments. Behavior
Research Methods, 46, 913-921. doi:10.3758/s13428-013-0422-2
de Gee, J.W., Colizoli, O., Kloosterman, N.A., Knapen, T., Nieuwenhuis, S., & Donner, T. H. (2017). Dynamic modulation of decision biases by brainstem arousal systems. ELife, 6, e23232. https://doi.org/10.7554/eLife.23232
Einhauser, W., Koch, C., & Carter, O. (2010). Pupil dilation betrays the timing of decisions. Frontiers in human neuroscience, 4, 18.
Fried, I., Mukamel, R., Kreiman, G. (2011) .Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron 69:548–562.
Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M., & Cohen, J.D. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function.
Cognitive, Affective, & Behavioral Neuroscience,10(2), 252–269.
https://doi.org/10.3758/CABN.10.2.252
Gramfort, A., Luessi, M., Larson, E., Engemann, D.A., Strohmeier, D., Brodbeck, C., ... & Hämäläinen, M. (2013). MEG and EEG data analysis with MNE-Python. Frontiers in
neuroscience, 7, 267.
Greenberg, D.S., Houweling, A.R. & Kerr, J.N. (2008). Population imaging of ongoing neuronal activity in the visual cortex of awake rats. Nature Neurosci. 11, 749–751
Guggisberg, A.G., & Mottaz, A. (2013). Timing and awareness of movement decisions: does consciousness really come too late?. Frontiers in human neuroscience, 7, 385.
Haggard, P. (2008). Human volition: towards a neuroscience of will. Nature Reviews
Neuroscience, 9(12), 934-946.
Janisse, M.P. (1977). Pupillometry. Washington, DC: Hemishpere.
Knapen, T.H.J. (2018). Exptools. Retrieved from https://github.com/VU-Cog-Sci/exptools. Libet, B., Gleason, C., Wright, E.W., Pearl, D.K., (1983). Time of conscious intention to act in relation to onset of cerebral activity (Readiness-Potential). Brain 106, 623–642.
Loewenfeld, I.E. (1999). The Pupil. Anatomy, Physiology, and Clinical Applications, Volume
1 (Boston: Butterworth Heinemann).
McGinley, M.J., David, S.V., and McCormick, D.A. (2015). Cortical membrane potential signature of optimal states for sensory signal detection. Neuron 87, 179–192.
Miller, J., Shepherdson, P., & Trevena, J. (2011). Effects of clock monitoring on electroencephalographic activity: Is unconscious movement initiation an artifact of the clock?. Psychological Science, 22(1), 103-109.
Okun, M., Naim, A. & Lampl, I.(2010). The subthreshold relation between cortical local field potential and neuronal firing unveiled by intracellular recordings in awake rats. J. Neurosci. 30, 4440–4448 .
Parés-Pujolràs, E., Kim, Y.W., Im, C.H., & Haggard, P. (2019). Latent awareness: Early conscious access to motor preparation processes is linked to the readiness potential.
Neuroimage, 202, 116140.
Peirce, J.W., Gray, J.R., Simpson, S., MacAskill, M.R., Höchenberger, R., Sogo, H.,
Kastman, E., Lindeløv, J. (2019). PsychoPy2: experiments in behavior made easy. Behavior
Poulet, J.F. & Petersen, C.C. (2008). Internal brain state regulates membrane potential synchrony in barrel cortex of behaving mice. Nature 454, 881–885.
Raphael, S., MacLeod D.I.A. (2011). Mesopic luminance assessed with minimum motion photometry. Journal of Vision, 11(9):14, 1–21
Reimer, J., Froudarakis, E., Cadwell, C.R., Yatsenko, D., Denfield, G.H., and Tolias, A.S. (2014). Pupil fluctuations track fast switching of cortical states during quiet wakefulness. Neuron 84, 355–362.
Ryle, G. The Concept of Mind (Univ. Chicago Press, 2000).
Schurger, A., Sarigiannidis, I., Naccache, L., Sitt, J.D., & Dehaene, S. (2015). Cortical activity is more stable when sensory stimuli are consciously perceived. Proceedings of the
National Academy of Sciences, 112(16), E2083-E2092.
Schurger, A., Sitt, J.D., & Dehaene, S. (2012). An accumulator model for spontaneous neural activity prior to self-initiated movement. Proceedings of the National Academy of
Sciences, 109(42), E2904-E2913.
Shadlen, M. N., & Gold, J. I. (2004). The neurophysiology of decision-making as a window on cognition. The cognitive neurosciences, 3, 1229-1441.
Steriade, M., & McCarley, R. W. (2005). Neurotransmitter-Modulated Currents of Brainstem Neurons and Some of Their Forebrain Targets. Brain Control of Wakefulness and Sleep, 211-254.
Supèr, H., van der Togt, C., Spekreijse, H., & Lamme, V.A.F. (2003). Internal state of monkey primaryvisual cortex (V1) predicts figure ground perception. The Journal of
Neuroscience : The Official Journal of the Society for Neuroscience, 23(8), 3407–3414.
Trevena, J., & Miller, J. (2010). Brain preparation before a voluntary action: Evidence against unconscious movement initiation. Consciousness and cognition, 19(1), 447-456.
van Kempen, J., Loughnane, G.M., Newman, D.P., Kelly, S.P., Thiele, A., O'Connell, R. G., & Bellgrove, M.A. (2019). Behavioural and neural signatures of perceptual decision-making are modulated by pupil-linked arousal. Elife, 8, e42541.
Vinck, M., Batista-Brito, R., Knoblich, U., and Cardin, J.A. (2015). Arousal and locomotion make distinct contributions to cortical activity patterns and visual encoding. Neuron 86, 740– 754.
