Single Trial Analyses in Perceptual Decision Making

Single Trial Analyses in Perceptual Decision Making

Student: Wouter Boekel - 5834023 - MBCS University of Amsterdam, track: Cognitive Neuroscience Supervisor: Leendert van Maanen, PhD

Co-assessor: Max Keuken, Msc

Abstract

When analyzing data from cognitive neuroscience experiments, researchers often compute averages over trials which reflect individual scores on a certain psychological task. However, The investigation of trial-to-trial variability in behavior and brain processes has recently become more prominent within the field of perceptual decision-making. In this paper, I will discuss research aimed at developing methods for single-trial analyses of RT, EEG and fMRI data-sets. Furthermore, I will provide an overview of research into trial-to-trial variation within perceptual decision-making and related fields. Moreover, throughout this paper I will discuss the advantages of single-trial analyses methods, as well as the challenges for future research aimed at investigating trial-to-trial variability in neurocognitive mechanisms underlying perceptual decision-making.

Introduction

Throughout our daily life we are constantly required to use our cognitive abilities to solve certain problems or reach certain goals. We constantly make decisions, inhibit certain prepotent responses, or switch our attention to different stimuli. The brain is of critical importance in executing these behaviors efficiently, allowing us to come closer to our goals, while simultaneously keeping us alive and healthy.

Research within Cognitive Neuroscience is mainly aimed at understanding the cognitive processes that underlie our daily behavior, and how neural mechanisms give rise to these cognitive processes. In general, research within cognitive neuroscience is done by having human or animal participants perform a certain task. These experimental tasks consist of

trials in which participants are required to respond to a certain stimulus. Seeing as there is often large variability (e.g. in response times (RT)) on these paradigms within individuals, a large number of trials are gathered to gain an accurate estimate of a participantʼs average performance on a certain paradigm, and thus inform researchers about a certain cognitive process. Neurophysiological or neuroimaging measurements can be acquired during these paradigms in order to acquire information about the neural mechanisms underlying these cognitive processes.

This kind of research often aims at investigating differences between groups, experimental conditions, or single participants. Between-trial fluctuations are assumed to be a source of noise in these experiments, and are thus ignored by computing, for example, mean RT. This approach can help us understand cognitive processes in terms of differences between groups, conditions or individuals. However, it is difficult to investigate the processes themselves (more specifically, between-trial adjustments in the processes) by using this traditional approach. Single-trial analysis methods provide a solution to this problem, allowing researchers to investigate the between-trial variability or time-course of certain cognitive processes. Furthermore, applying the single-trial analysis approach to neuroimaging data allows researchers to investigate variability in neural mechanisms underlying variability in cognitive processes.

One of the fields within cognitive neuroscience which has seen a recent increase in the use of single-trial analysis methods is perceptual decision-making. Research in this field employs a range of behavioral paradigms in order to investigate cognitive processes underlying simple perceptual choices (Gold and Shadlen, 2007). One prevalent paradigm within perceptual decision-making is the random-dot-motion paradigm. In this paradigm, participants see a cloud of dots on a computer screen. Some of these dots are moving

coherently to the left or to the right, and some of these dots are moving randomly. Participants are asked to respond to the general direction in which the cloud of dots is moving. The difficulty in this paradigm can be adjusted by specifying the ratio of coherently versus randomly moving dots (Britten et al., 1992).

Below, I will describe an example experiment using the RDM paradigm. Subsequently I will provide an overview of recent research in which single-trial analysis methods are used to investigate (aspects of) perceptual decision-making and its underlying neurocognitive mechanisms. I will use the example experiment to demonstrate how the different single-trial analysis techniques that are described can be applied on a data set acquired from running such an experiment. Finally, I will discuss advantages as well as shortcomings of single-trial analysis methods.

One can easily imagine running an experiment using the RDM paradigm to investigate how people make hard and easy decisions. In the particular design that you are using, there are two conditions: A difficult condition, in which 5% of the dots are moving coherently, and an easy condition, in which 15% of the dots are moving coherently. Participants are asked to respond to the general direction in which the cloud of dots is moving. Their accuracy scores and reaction times are gathered on every trial.

The data set that is acquired by running such a straightforward and simple design, is already rich enough to allow multiple kinds of analyses. Accuracy scores will differ between conditions and participants. The results will probably show us that participants make more errors in the hard condition, and that some participants perform better on this task than others (Palmer et al., 2005). Similarly, extracting mean or median RT per condition (easy vs. hard), will tell us something about whether participants, overall, need more time to reach a decision when the stimulus contains less usable information and

more noise (i.e. lower coherence). Computing RT per participant then supplies us with a measure of inter-individual difference in general performance on the task. Extracting these individual RT values separately per condition provides us with an individual measure of sensitivity to the stimulus (e.g. a participant who is more sensitive to the stimulus will show a lower increase in RT on the difficult condition). Finally, one might try to combine both accuracy and RT data by fitting a computational model, in order to extract measures of certain latent cognitive processes underlying the generation of RT distributions. One of the most validated computational models for the analysis of RT data is the drift-diffusion model (DDM; Ratcliff, 1978). This model is based on the idea that when faced with a decision, sensory evidence starts accumulating until a certain threshold is reached, evoking a choice. By fitting this model to behavioral data from an RDM paradigm, one can derive specific information about the decision process that is reflected in the parameters of the model. This model-based approach allows researchers to investigate differences in latent cognitive processes such as drift rate between conditions (easy vs. hard), or between individuals.

The abovementioned types of analyses all have in common that trial-to-trial variability in RT is thought of as noise. Many trials are acquired to decrease the impact of this variability in RT, and thus ʻfilter out the noiseʼ. This approach allows researchers to gain insight into differences in cognitive processes between certain conditions, individuals, or populations. However, one might be interested in the time-course of the decision making process over trials, or the occurrence of attentional lapses over trials. Computing the mean RT per condition or participant will effectively remove the information necessary to investigate these processes. It might still be possible to adjust experimental conditions in such a way that one is able to extract a measure of variability in behavior by, for example, extracting and comparing a single RT-variability score per condition. However, I will argue that it is

much more effective to exploit trial-to-trial fluctuations to investigate variability in behavior. I will do so by showing examples from a range of different subfields related to perceptual decision making that all use a form of single trial analyses that leads to new insights into perceptual decision making and its underlying neural mechanisms. In order to first provide an overview of the different dependent measures that can be analyzed by single-trial analysis methods, below I will describe methodological research aimed at developing these single-trial methods for three different types of data: RT, electro- and magnetoencephalography (EEG/MEG), and functional magnetic resonance imaging (fMRI).

Dependent measures

In order to investigate neurocognitive mechanisms underlying trial-to-trial fluctuations in behavior, we require measurements that reflect these mechanisms. RT is one of the most commonly used measures in psychology, and thus many methods of analyzing RT data have been developed. EEG and MEG are used to measure electrical and magnetic activity non-invasively from the scalp. fMRI is a more recently developed neuroimaging technique, which measures changes in blood flow non-invasively, and thus indirectly informs researchers about activity in a certain brain region. These neuroimaging data-sets are often combined with RT data to allow investigation of neural mechanisms underlying cognitive processes. Below I will provide an overview of research aimed at developing single-trial analysis methods which can be applied to data-sets acquired from these measurements.

RT

RT is one of the most used measures in experimental psychology. RT experiments gather RT data from participants performing a large amount of trials in which, for example, a choice has to be made between two alternatives. Mean RT values can be used to investigate differences in RT between individuals or groups. Computational models such as the DDM have been developed to investigate latent cognitive processes thought to underlie a decision (Ratcliff, 1978). This model assumes that sensory evidence starts accumulating when faced with a perceptual decision. This evidence will reach a certain threshold, evoking a choice. By fitting this model to behavioral data from, for example, an RDM paradigm, one can derive specific information about the decision process that is reflected in the parameters of the model. For example, drift rate reflects the rate or speed of accumulation, and is often thought to represent the quality of sensory information (Ratcliff, 1978), and subsequently the difficulty of the decision. The starting point of accumulation tells us something about the subjectʼs bias towards one of the options (Mulder et al., 2012), and threshold values (or more accurately, the difference in between the starting point and threshold) contain information about the trade-off between speed and accuracy (Bogacz et al., 2010). The non-decision time reflects processes not related to the decision, such as visual or motor processes. In addition, the model contains three parameters for across-trial variability (variability in starting point, variability in non-decision time, and variability in drift rate).

The abovementioned across-trial variability parameters are possible candidates for the investigation of trial-by-trial variability. However, these are single values of variability over the entire time-course of an experiment, and are thus averaged measures of trial-by-trial variability. Recently, van Maanen et al. (2011) proposed a new method for estimating single-trial parameters of drift rate and starting point, allowing researchers to investigate

the trial-to-trial variation of these latent cognitive processes, and subsequently investigate the underlying neural mechanisms of these processes. In a different (but related) field, reinforcement learning models have been developed which are aimed at investigating the time-course of a learning process. I will describe these models in more detail in the reward learning section below.

In short, the large usage of RT data in traditional experimental psychology has led to answers to many questions. These answers have led to new questions which could not be answered by traditional mean-RT analyses. In order to answer these new questions, researchers have developed analysis techniques aimed at investigating latent cognitive processes based solely on the trial-to-trial variability in RT. The challenge of future research utilizing RT data-sets is to investigate latent cognitive processes on a single-trial basis, and moreover, to combine this with neuroimaging techniques to inform researchers about the neural mechanisms underlying the trial-to-trial variability in cognitive processes.

EEG/MEG

Electroencephalography (EEG) and magnetoencephalography (MEG; although to a lesser extent) are two of the oldest and most used non-invasive techniques to investigate neural processes. Traditionally, EEG is measured at a high sampling rate while participants perform a task. EEG data are then segmented based on a single event within each trial. When considering the example experiment described above, you might create segments, or epochs, centered around the onset of the stimulus or feedback events within a single trial of the RDM paradigm. These epochs are averaged over trials, providing researchers with an average of the event-related potential (ERP). Average ERPʼs can then be linked to individual differences in behavior, or differences between groups can be investigated (Luck, 2005). However, variability in the ERP is removed by this averaging. Moreover,

differences in phases of single-trial ERPʼs could result in the average ERP being cancelled out completely.

One way of solving this problem is by simply computing single trial ERPs. However, this method still suffers from between-trial phase-shifts. Time-frequency analyses solve this problem in a more sophisticated way, by extracting power values of specific frequencies per trial. Seeing as the power of a certain frequency is independent of the phase of the oscillation at stimulus onset, single-trial oscillatory power values can easily be averaged across trials without the problem of canceling out oscillations due to across-trial phase differences (Cohen, 2011). However, trial-to-trial variation in oscillatory power may still be present, and in fact be informative about a certain cognitive process. MEG data-sets are similar to EEG data-sets, in that they measure signal from the scalp at a high sampling rate, and have thus been analyzed in a similar way.

One of the aims of methodological EEG research, is to increase reliability and signal-to-noise ratios of single-trial analysis methods. Grandchamp and Delorme (2011) describe a single-trial normalization method for spectral analyses of EEG data, which reduces sensitivity to outliers. Traditionally, post-stimulus time-frequency data are baseline-corrected by, for example, subtracting the trial-average pre-stimulus power, per frequency, from the time-frequency signal at every time-point within the post-stimulus time window of interest. Grandchamp and Delorme (2011) show that this method is susceptible to pre-stimulus outliers. However, when using a single-trial baseline correction method, they show that the influence of these outliers is reduced. This work shows that even in a pre-processing step such as baseline-correction, averaging over trials can affect the results in a way that is not desirable. Instead, single-trial methods should be applied to properly account for the trial-to-trial variability.

Gaspar et al. (2011) modeled single trial ERP amplitudes using a general linear model (GLM) approach. This provided the researchers with an R2 _{(explained variance)}

timecourse, representing a measure of to what extent the ERP signal at every time point can be predicted by a specific regressor, set by the researcher. This approach is often used in fMRI, and thus provides a way of integrating fMRI and EEG analyses (e.g. by using two simultaneously acquired datasets to estimate parameters of a general linear model (Kiebel and Friston, 2004)). Moreover, Gaspar et al. (2011) performed an intraclass correlation analysis (ICC), which shows how strongly observations from the same participants resemble each other. Using these methods, the authors showed that the within subject correlations were much higher than the between-subject correlations in the R2 _{timecourse, and concluded that this method of analyzing EEG provides reliable}

test-retest results in a face processing paradigm.

One of the recurring topics of research into single-trial methods of analyzing EEG data is single-trial classification. Kostelecki et al. (2011) describe a method of classification of MEG data, using Granger causality. This method is based on testing to what extent a certain time-series data-set (e.g. EEG or MEG) can be used to predict future time-series data of another location (in the case of encephalography, another electrode or pool of electrodes). Kostelecki et al. (2011) showed that using a single-trial classification method based on concepts of Granger causality, it is possible to reliably classify whether participants had to respond with a specific hand, or whether they could freely choose to respond with either hand.

Obermaier et al. (2001) proposes an alternative method for online classification of EEG data, and compares its performance to more traditional linear discrimination techniques.

Obermaier and colleagues discuss prior research on classification techniques used for online discrimination within EEG-based brain-computer-interface (BCI). One of the problems they discuss is that “The spatio-temporal EEG patterns associated with motor imagery are not always stable, but often demonstrate a dynamic behaviour”. Obermaier et al. (2001) use a Hidden Markov Model, which takes into account this dynamic behavior of brain oscillations, to more accurately perform online classification of EEG data compared to a more traditional linear discrimination approach.

From the abovementioned research, we can conclude that the application of single-trial analysis methods on EEG data-sets can have several useful functions, such as increasing data quality and reliability (Gaspar et al., 2011; Grandchamp and Delorme, 2011), and single-trial classification (Obermaier et al., 2001; Kostelecki et al., 2011). Furthermore, these methods allow for the analysis of trial-to-trial fluctuations in the EEG signal, which are otherwise removed by averaging over trials. The development of additional single-trial analysis techniques for EEG data will increase the variety of methods, and thus allow researchers to choose the most appropriate methods for the analysis of their EEG data-sets.

fMRI

Functional Magnetic Resonance Imaging (fMRI) is a relatively new brain imaging technique compared to EEG and MEG. One of the advantages of fMRI over encephalography is the increased spatial resolution, while one of the disadvantages is the decreased temporal resolution due to the sluggishness of the (blood-oxygen-level-dependent) BOLD response. Early research into brain activity using fMRI employed blocked designs, in which two different stimuli were repeatedly shown to a subject in separate blocks (Huettel et al., 2009). Differences in average BOLD activation between

blocks would then inform researchers about where in the brain these stimuli were being processed. More recently, researchers have employed an event-related design, in which different stimuli, or different conditions of a certain paradigm are alternated, often pseudorandomly and including variable inter-stimulus and inter-trial intervals (Huettel et al., 2009). GLMs are then used to model the BOLD response by convolving regressors with for instance a hemodynamic response function (HRF; represents an assumed shape of the BOLD response). This approach allows researchers to present participants with more stimuli and/or conditions within a single scan-session, and thus increases the power of data acquired from performing psychological paradigms during an fMRI scanning session.

Moreover, parametric regressors may be used, which take into account trial-to-trial variation in behavior. A simple RT-regressor can inform researchers about the brain areas of which the activity is significantly predicted by trial-to-trial variations in RT. In a study by Yarkoni et al. (2009), a finite impulse response (FIR) approach is used to investigate correlations between brain activations and trial-to-trial fluctuations in RT. While they use a standard GLM to analyze the fMRI signal, the traditional HRF is replaced by the FIR. This method has slightly reduced power due to the larger amount of degrees of freedom, but in return provides the researcher with an estimation of additional parameters of the BOLD response. Yarkoni et al. (2009) propose several potential effects of single-trial RT on BOLD responses, which can be investigated using their method. Using the FIR approach, they uncover an effect whereby fluctuations in RT modulate fluctuations in the onset for the BOLD response. This work shows that a single-trial RT regressor, in combination with a non-traditional way of analyzing the BOLD response (i.e. using FIR instead of HRF), can provide researchers with an account of the neural mechanisms underlying variation in RT.

Gaudes et al. (2011) describe a method of analyzing fMRI that detects BOLD-responses on a single-trial level. This provides researchers with the possibility of investigating processes that operate between trials and manifest themselves as trial-to-trial variation in the BOLD response. The method that Gaudes et al. (2011) describes can be thought of as a univariate version of an independent-component analysis (ICA; Beckmann and Smith 2004). This method is data-driven, and does not require the researcher to define the onset and duration of events, which is traditionally done in GLM analyses of fMRI. The method was able to detect finger tapping events within an fMRI time course, without prior knowledge about the timing or duration of the events. Thus, this method provides researchers with an unbiased and data-driven approach of analyzing a BOLD time course. One example in which this method could be useful is when we are interested in neural processes of which we are not certain when they occur, or which are not directly detectable using behavioral measurements. For instance, task-unrelated-thoughts can be hazardous in situations where one needs to be alert, and yet we lack tools to investigate the occurrence of these events. Using the method described by Gaudes et al. (2011), it could be possible to detect these events in a healthy individual performing a cognitive task, while occasionally lapsing in attention due to a task-related thought.

One of the advantages of fMRI over other neuroimaging techniques such as EEG is its high spatial resolution. In order to optimally exploit this advantage, multi-voxel pattern analyses have been developed. MVPA can potentially detect fMRI activity patterns which traditional univariate approaches cannot (Jimura and Poldrack, 2012). MVPA requires the extraction of single-trial BOLD responses from fMRI time courses. A recent study by Mumford et al. (2012) investigated eight methods for extracting the single-trial BOLD response and found that the best approach entails including a regressor for the trial of which the single-trial BOLD response is extracted, and a nuisance regressor for all other

trials. This approach is very easy to implement, and conceptually simple, as it expands on traditional GLM analyses of fMRI data.

The field of fMRI is still very young, and much effort is being put into the development of new analysis techniques. Single-trial analysis methods for BOLD-data will allow researchers to investigate trial-to-trial fluctuations in brain activity with a high spatial resolution.

Simultaneous EEG/fMRI

EEG has a high temporal but low spatial resolution, while fMRI has a high spatial but a low temporal resolution. Combining data-sets acquired from EEG and fMRI can potentially increase our temporal as well as spatial understanding of a certain process. The most direct way of combining these different methods is to acquire EEG and fMRI data simultaneously. This allows the investigation of neurocognitive processes in the spatial as well as temporal domain. It is not surprising that in many simultaneous EEG/fMRI studies, single-trial analyses are employed. After all, if one werenʼt interested in the trial-to-trial dynamics of a certain neurocognitive process, one wouldnʼt have to measure EEG and fMRI simultaneously (i.e. separately acquired EEG and fMRI data sets are sufficient in order to investigate individual differences or group-differences, while still informing researchers of the spatial as well as temporal nature of these differences). Below I will provide an overview of the single-trial analysis methods applied on simultaneous data-sets. For a more general review on simultaneous EEG/fMRI, see Huster et al. (2012).

A much used method for the analysis of simultaneous EEG/fMRI data, is to predict BOLD activations from EEG data. Using this method, one is able to identify the structures underlying a certain neural process with high spatial accuracy. Eichele et al. (2005) used

an auditory oddball paradigm, in which participants are asked to respond when an infrequent ʻoddballʼ stimulus is presented among more frequent stimuli. Eichele and colleagues introduced predictability in their design by alternating between random and fixed target-to-target intervals (TTI). Amplitude modulations (AM) of ERPʼs were found as a function of the number of times an interval was repeated. These amplitude modulations were then correlated on a single-trial level to BOLD activations. By using a single-trial analysis method on simultaneous EEG/fMRI data, Eichele et al. (2005) were able to identify the brain structures underlying amplitude modulations in response to the repeated presentation of auditory sequences. Note that because of each methodʼs specific limitations, this result would not have been found by acquiring only EEG or fMRI data, or even by acquiring both data-sets non-simultaneously.

Fuglø et al. (2011) describe an experiment in which a basic visual stimulation paradigm was used in order to assess the between-subject reliability of single-trial EEG-fMRI correlations. The authors found that when combining the amplitudes of two commonly reported visual evoked potentials (VEP; more specifically the P1 and N1) in the EEG domain, they found high correlations with BOLD activation in visual areas, which were consistent across all participants. Several other studies have utilized simple experimental paradigms in conjunction with single-trial simultaneous EEG/fMRI methods in order to gain insight into the spatial locations of previously investigated electrophysiological phenomena (Esposito et al., 2009; Goldman et al., 2009; Mulert et al., 2010; Scheibe et al. 2010; Becker et al., 2011; Yuan et al., 2011)

Debener et al. (2005) used a Flanker task while recording simultaneous EEG/fMRI. Participants had to report the direction of an arrow displayed in the center of a screen, surrounded by either congruent (same direction) or incongruent (opposite direction)

arrows. Debener and colleagues found that single-trial error-related negativity (ERN) amplitudes were predictive of the magnitude of the increase in RT on the subsequent trial (post-error slowing; PES). Moreover, trial-to-trial coupling of EEG and fMRI showed that the ERN was predictive of BOLD activation in the rostral cingulate zone (RCZ), which is often associated with performance-monitoring (Ridderinkhof et al., 2004). By combining a single-trial EEG-behavior method with a single-trial EEG-fMRI method, the authors were able to identify both neural mechanisms underlying performance monitoring with high temporal as well as spatial precision.

More sophisticated methods for the analysis of simultaneous EEG and fMRI data-sets are being developed. Correa et al. (2010) describe a method for the fusion of two data-sets using multi-set canonical correlation analysis (M-CCA). This multivariate method is data-driven and allows the researcher to investigate connectivity across functional networks. To discuss the specifics of this method is unfortunately beyond the scope of this review.

The research described above shows that much effort is being put into developing new methods for the analysis of RT, EEG and fMRI data. Furthermore, combining techniques to acquire simultaneous RT/EEG/fMRI data will allow researchers to more precisely investigate (latent) cognitive processes, and their underlying neural mechanisms, both in the temporal as well as in the spatial domain.

Application of single-trial analyses in p

erceptual decision making

The research described above provides us with a wide array of methods to apply to different types of data-sets. From this section onward, I will discuss the application of these single-trial analysis techniques in perceptual decision making. I will start by generally discussing perceptual decision making, followed by three sections on aspects related to

perceptual decision making.

General

As described in the dependent measures section, methods from machine learning have been adjusted to provide researchers with analysis techniques which can be applied to M/ EEG and fMRI datasets. Furthermore, computational models are being applied to investigate latent cognitive processes underlying perceptual decision making. Below, I will discuss perceptual decision making research using some of these single-trial analysis methods.

Kamitani and Tong (2006) show that by using MVPA on data acquired from participants performing a RDM paradigm during fMRI scanning, they were able to train a classifier to reliably distinguish trials with different directions of motion, from BOLD activations in early visual areas V1 through V5. In addition, they showed that it is possible to classify the attended motion direction when two opposite motion directions were displayed simultaneously. Using this method, Kamitani and Tong (2006) provide an alternative method for investigating visual direction selectivity in humans, with a higher spatial resolution (through the use of MVPA) than conventional fMRI analyses.

Philiastides and Sajda (2005) used a face-car discrimination task with multiple levels of coherence (much like the example experiment described above, but with images of faces and cars instead of dots moving to the left or right), and showed that they were able to train a classifier to distinguish between face and car trials using single-trial EEG data from an early and a late ERP component. The accuracy of this classifier was highest in the high-coherence condition, and lowest in the low-coherence condition. Moreover, they found that the optimal time-window for classification in the late (but not in the early) EEG

component was shifted for more difficult conditions in which stimuli were more ambiguous. The authors suggest that as opposed to the early face-selective ERP component, this late ERP component is representative of the accumulation of evidence, given that it shifts temporally based on the difficulty of the stimulus (and thus the increase in RT). Using this single-trial method, the authors were able to more specifically investigate ERP components, and draw more sophisticated conclusions about two ERP components, which would, in a more traditional analysis, otherwise simply be interpreted as being correlated to the process of distinguishing cars from faces. In a later study, Philiastides et al. (2006) expand on their earlier work by employing a diffusion model, and correlating the strength of discrimination of the late ERP component to the mean drift rate per condition.

While diffusion models do take into account entire RT distributions, they lack the ability to provide single-trial estimates of parameters such as drift rate and threshold. Some efforts have been made to solve, or circumvent this drawback. For instance, Ratcliff et al. (2009) use the early and late ERP components acquired on a face-car discrimination task to divide the data into more face-like and more car-like groups, and then fit a diffusion model to the separate groups. They find that differences in drift rate were found between groups when the data were separated based on the late ERP component, while this was not the case for when the data were separated based on the early ERP component. This result reinforces the notion that the late component is associated with evidence-accumulation.

To more accurately investigate the underlying neural mechanisms of latent cognitive processes such as evidence accumulation, a single-trial implementation of drift-diffusion type analyses is required. Van Maanen et al. (2011) describe a single-trial implementation of the linear ballistic accumulator model (STLBA). The LBA (Brown and Heathcote, 2008) differs from the DDM in that it contains accumulators for each option separately, meaning

that it is possible for the LBA to fit multiple-choice data-sets. In addition, the LBA achieves RT variability by including variability in drift rate and starting point. While the drift-rate distribution is gaussian, the starting point distribution is uniform. Van Maanen et al. (2011) Use the gaussian nature of the rate distribution to estimate single-trial values of drift-rate and starting-point. More specifically, a default LBA model is fit to provide the researcher with average drift rate and starting point values and decision time. The mean drift rate is then set as the most likely drift rate for every trial. Corresponding starting point values are then selected, based on the decision time. If the starting point value required for the decision-time on a single trial, using the mean drift rate, is an invalid one (i.e. falls outside of the distribution of starting points), the drift rate for that trial is adjusted to provide the most likely combination of drift rate and starting point, given the single trial decision-time. Van Maanen et al. (2011) use an RDM paradigm with a speed-accuracy-tradeoff manipulation, and show that when participants were instructed to switch between speed and accuracy conditions, single-trial starting point values were correlated with single-trial BOLD estimates in medial frontal gyrus (MFG), anterior cingulate cortex (ACC) and Striatum. In addition, when participants were specifically instructed to speed up their responses, a correlation between these measures in pre-supplementary motor area (preSMA) was found. In a control analysis, van Maanen et al. (2011) show that performing similar analyses using single-trial RT instead of model parameters results in a large amount of unspecific brain areas associated with RT. Using this single-trial model-based approach is more specific and allows investigation of latent cognitive processes such as evidence accumulation or response caution on a single trial level.

Reward learning

Reward learning is an important aspect of perceptual decision-making. When we make perceptual decisions, we might be influenced by the previous outcomes of similar

decisions. In order to benefit from previous experience, we need to be able to learn stimulus-reward associations (OʼDoherty et al., 2004). Research into reward learning naturally invites methods of analyses which take into account the time-course of certain learning processes. The learning of stimulus-reward associations can be modeled by a model-based framework known as reinforcement learning theory (Sutton and Barto, 1998). These models provide a computational account of how certain stimulus-reward associations are formed over time, and how reward-rate is subsequently optimized. A recent study by Cavanagh et al. (2010) utilized a specific model derived from reinforcement learning theory: Q-learning. This approach estimates action values and prediction errors per trial, providing the researcher with a single-trial measure of the subjectʼs reward expectation and subsequent prediction error. Cavanagh et al. (2010) showed that following negative prediction errors, RT slowed down, while it speeded up after positive prediction errors. Furthermore, they showed that there was a relation, especially in incorrect trials, between single-trial prediction error and single-trial theta (4-8hz) power over the medial prefrontal cortex (mPFC). mPFC theta power was also predictive of the next-trial change in RT. Additional single-trial analyses showed that theta power in the lateral prefrontal cortex (lPFC) was related to prediction error, as well as to the RT change on the subsequent occurrence of the same stimulus as opposed to the current one. Finally, theta phase synchrony between mPFC and lPFC was related to the single-trial prediction error estimates. Using these single-trial analyses, Cavanagh et al. (2010) were able to investigate the role of theta oscillations during the time-course of the reinforcement learning process in much more detail than simple trial-averaging would have allowed them.

Several other studies have employed single-trial analyses to investigate reinforcement learning processes. For example, Cavanagh et al. (2011) investigate the exploitation/

exploration trade-off and find that frontal theta power is predictive of single-trial parameters reflecting uncertainty and unexpectedness. Cohen and Ranganath (2005) used a reinforcement-learning paradigm while measuring functional MRI, and found that individual differences in value functions can be linked to individual differences in BOLD activations, and that future decisions can be predicted by neural activity on previous trials. In a later study, Cohen (2006) estimated single-trial prediction errors and found a correlation with BOLD activity in a network of areas including left Substantia nigra, dorsal cingulate cortex, bilateral PFC, and right cuneus. Wunderlich et al. (2009) used reinforcement learning theory in conjunction with functional MRI to investigate neural correlates of trial-to-trial fluctuations in action values, and found that the BOLD signal in SMA predicted single-trial action value estimates irrespective of which action was chosen, while BOLD signal in the ventromedial prefrontal cortex (vmPFC) and intraparietal sulcus were specifically related to the action values for the chosen actions.

In short, reinforcement learning models have been shown to provide researchers with an excellent tool with which they can investigate the time-course of reward-learning processes.

Other types of single-trial analyses have also been applied in the field of learning and memory (Behrens et al., 2007; Nieuwenhuis et al., 2011). Behrens et al. (2007) use a Bayesian account of learning to estimate single-trial values of volatility, which is a measure of to what extent recent experiences are more predictive of future events than distant experiences. They showed that this parameter was correlated on a single-trial level to BOLD activity within the SMA, and volatility related activity in the ACC was related to individual differences in learning rates. Nieuwenhuis et al. (2011) measured functional connectivity by correlating single-trial oscillatory power between areas involved in

associative memory, and found that functional connectivity between the left anterior temporal lobe (ATL) and the posterior parietal cortex (PPC) and fusiform face area (FFA) increases over time when face-location associations are learned. This experiment provided evidence for the emergence of memory representations within neocortical areas, and suggested that the development of single-trial gamma-power correlations over time could be an underlying process.

The abovementioned methods can be used and adapted to investigate the reward-learning aspects of perceptual decision-making. Imagine adapting the example experiment described above to only reward participants on a subset of correct trials. You might reinforce correct “left” decisions on only 50% of all trials, while reinforcing “right” decisions 75% of all trials. over time, participants might develop a bias towards choosing the “right” decision. As described in the previous paragraph, you could apply a reinforcement learning model to your data in order to estimate single-trial prediction errors or action values, and subsequently correlate them to single trial EEG or BOLD measurements.

Based on the studies discussed one can argue that single-trial analysis methods are broadly being used in the field of reward-learning. Within this field, neuroimaging methods such as EEG/MEG and fMRI are being combined with computational models to inform researchers about the underlying neural mechanisms of reward-learning.

Attention

The research field of attention is one that easily links to perceptual decision making. A decision-maker needs to attend to different options in order to properly choose between them. One of the questions within attention is whether it is possible to predict lapses in attention from brain activity. Weissman et al. (2006) investigated whether single-trial

performance on a global/local selective task could be predicted by single-trial BOLD activity. In this task, participants are shown a letter that consists of smaller letters. Participants are instructed to report either the global or the smaller local letters. Trials can be incongruent or congruent, depending on whether the global and local letters are different or the same. The authors specifically looked at trial-to-trial variation in reaction times, and predicted that larger reaction times would be linked to smaller BOLD activity in frontal control regions. The results indeed showed this effect, and the authors interpreted the reduction in BOLD activity in these areas as reflecting attentional lapses, which resulted in longer response times. In addition, longer response times were found to correlate to increased activity within regions of the mode network. The default-mode network is often found to be activated when participants are instructed to do nothing, meaning that participants are not actively engage in a task., and deactivated when participants engage in a task (although this is not a clear dichotomy; see Spreng (2012)). This effect again suggests the increased response times are linked to a lapse in attention.

Eichele et al. (2008) followed up on this by investigating whether specific patterns of activation can predict the occurrence of an error. In this study, a Flanker task was used in which participants were asked to report on a target stimulus, which is surrounded by distractor stimuli, which can be either congruent or incongruent to the target stimulus. Eichele et al. (2008) used independent component analyses (ICA) of fMRI data to specifically extract components of activity consisting of areas associated with the default mode network, as well as components which represented a network of areas associated with engagement in a task. It was found that activity in the default mode network gradually increases (or rather, deactivation gradually decreases), preceding an error. Activation in task-relevant brain areas gradually decreased preceding errors. In addition, just shortly before errors occurred activity in sensorimotor areas abruptly declined. Shortly after the

error occurred, these activity patterns reversed. These effects show that errors can be predicted by fluctuations in brain activity more than one trial preceding the erroneous behavior. This suggests that averaging over pre-error trials might not be the most powerful method to investigate brain activity leading to errors. Instead, investigating multiple trials preceding errors (and thus, employing a single-trial analysis method) might provide a better insight into what causes an error to occur. Moreover, it might even be beneficial to combine neuroimaging techniques, by simultaneously measuring fMRI and EEG (the latter of which has a much greater temporal resolution) in order to more closely investigate trial-to-trial variation, and even within-trial variation in brain activity.

More recently, Prado and Weissman (2011) performed an fMRI study using a simple detection task containing both visual and auditory cues. Participants had to report on either the visual or auditory cue, while suppressing the distractor cue. Similar to the studies described above, the paradigm contained congruent and incongruent trials. The authors found that an increase in functional connectivity between posterior cingulate cortex (PCC) and dorsolateral prefrontal cortex (DLPFC) was associated with slower responses in current trials, but faster responses in the next trial. These results are supportive of the mode interference hypothesis, which states that activity in default-mode areas interferes with task-related processing, thereby decreasing performance or producing attentional lapses. However, the authors acknowledge that the positive influence that they found on future performance is less clear. Given that this effect inherently entails fluctuations in behavior and brain activity between trials, future studies should continue to use single-trial analyses to investigate fluctuations in attention.

To illustrate the effectiveness of single-trial analyses in this field, imagine adapting our example experiment to induce attentional lapses. You might increase the difficulty and

length of the paradigm, while simultaneously depriving participants of sleep and/or coffee. Participants will surely become fatigued and start making errors (Ratcliff and van Dongen, 2009, 2011), possibly because they werenʼt paying attention when the stimulus was presented. The single-trial analyses described above can easily be adjusted to accommodate analyses of RT data from the adjusted RDM paradigm. You might investigate functional connectivity or increased activity within the default-mode network, to investigate whether longer RTʼs are associated with default-mode network activation. Alternatively, you might employ a single-trial model such as the STLBA method (van Maanen et al., 2011). Attentional lapses might result in a temporary decrease in sensitivity to the stimulus, which would be reflected in a decrease in single-trial drift rate parameter values. These single-trial values could then be linked to BOLD activity, which could potentially provide a more specific view into the underlying neural mechanisms of attentional lapses, than one would get when using mean RTʼs.

Spontaneous brain fluctuations

The attentional and reward-learning aspects of perceptual decision making have in common that the processes involved in these aspects are not random, and contain a certain functionality. However, random processes in the brain might also have an influence on perceptual decision making.

Research into spontaneous brain fluctuations directly investigates the question why there is variability in the brain, and how this variability manifests itself in terms of behavior (For a review on spontaneous brain fluctuations, see Fox et al. (2007a)). It is obvious that researchers in this field would use single-trial analyses, seeing as averaging over trials effectively removes the temporal variability that is being investigated when looking at intrinsic brain fluctuations.

Fox et al. (2006) investigated intrinsic spontaneous fluctuations in BOLD activity by exploiting the correlation of intrinsic activity between left and right somatomotor cortex (SMC). Participants were first instructed to passively view a movie, during which fMRI was measured. During a second movie viewing, participants were instructed to press a button with their right index finger on specific occasions. The left SMC was functionally defined by the button press condition (i.e. the area in which activity corresponded to a button press was defined as the SMC). Then, the right SMC was defined by using a functional connectivity analysis. Lastly, the BOLD activity in the right SMC was subtracted from activity in the left SMC. This resulted in a decrease in noise of the task-related BOLD response in left SMC, while still retaining the average response that was found in the initial analysis. Furthermore, they showed that the magnitude of the event-related BOLD response could be predicted by whether the time of the event occurred in a trough or peak of the intrinsic BOLD response. By doing this, they showed that event-related activity can be influenced by event-unrelated intrinsic activity. Fox et al. (2007b) expanded on their earlier experiment by linking the variability in BOLD activity to variability in button press force. While they didnʼt employ a purely single-trial analysis, they did show that the shape of the BOLD response could predict whether the associated button press was attributed to the “hard” or “soft” bin. Furthermore, this effect was extinguished by regressing out the BOLD time-course of the right SMC. This result suggests that intrinsic fluctuations can influence subsequent event-related BOLD activity, which in turn is associated with variability in behavior.

The research described above shows us that intrinsic fluctuations in brain activity can be a source of trial-by-trial variability in behavior. In our example experiment, intrinsic brain fluctuations might underlie trial-to-trial variation in RT. One of the challenges of future

research into spontaneous brain fluctuations is to develop models that can take into account these fluctuations in a data set acquired from healthy participants performing a psychological task, like the RDM.

Conclusion

In this review, I have provided an overview of research into trial-to-trial variation in perceptual decision-making and related fields. I have shown that much work is being done to develop single-trial analysis methods which can be applied to RT, EEG/MEG, and fMRI data-sets. In addition, I have shown many examples of experiments employing single-trial analysis methods to investigate perceptual decision-making and related cognitive and neural processes such as reward learning, spontaneous brain fluctuations, and attention.

Despite the many promising single-trial methods that have been developed and applied, many studies do not explicitly show that by using a single-trial analysis method, results are found which could not have been obtained by using traditional averaging methods. For future research, it is important to emphasize the advantage of these single-trial analysis methods by reporting both results from traditional averaging and single-trial analysis methods. This will provide a more clear picture of when single-trial analyses can, and when they cannot, provide researchers with a better insight into trial-to-trial variability in perceptual decision making.

In conclusion, the application of single-trial analysis techniques as opposed to trial-averaging potentially provides researchers with the possibility to investigate cognitive processes and their underlying neural mechanisms in more detail. Whereas more traditional research has looked into the difference of a specific process between groups or individuals, single-trial methods allow researchers to investigate the time course of the

process itself, informing the researcher of how the process develops over time. The challenges of future research aimed at investigating trial-to-trial variation in perceptual decision making is two-fold: First, future methodological research will have to integrate simultaneously acquired data-sets from different neuroimaging techniques, and subsequently develop analysis methods which take into account trial-to-trial variability. Second, future research will have to develop new psychological paradigms in which trial-to-trial variability in cognitive and neural processes can be investigated, using these single-trial analysis techniques.

References

Becker R, Reinacher M, Freyer F, Villringer A, Ritter P (2011) How ongoing neuronal oscillations account for evoked fMRI variability. J Neurosci 31:11016-11027.

Beckmann CF, Smith SM (2004) Probabilistic independent component analysis for functional magnetic resonance imaging. IEEE Trans Med Imag 23:137–152.

Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214-1221.

Bogacz R, Wagenmakers EJ, Forstmann BU, Nieuwenhuis S (2010) The neural basis of the speed-accuracy tradeoff. Trends Neurosci 33:10-16.

Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci 12:4745-4765. Brown A, Heathcote A (2008) The simplest complete model of choice response time: Linear ballistic accumulation. Cogn Psychol 57:153-178.

Cavanagh JF, Figueroa CM, Cohen MX, Frank MJ (2011) Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cereb Cortex 22:2575-2586. Cavanagh JF, Frank MJ, Klein TJ, Allen JJB (2010) Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. Neuroimage 49:3198-3209.

Cohen MX (2006) Individual differences and the neural representations of reward expectation and reward prediction error. Soc Cogn Affect Neur 2:20-30.

Cohen MX (2011) Itʼs about time. Front Hum Neurosci 5:2.

Cohen MX, Ranganath C (2005) Behavioral and neural predictors of upcoming decisions. Cogn Affect Behav Ne 5:117-126.

Correa NM, Eichele T, Adali T, Li Y, Calhoun VD (2010) Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. Neuroimage 50:1438-1445.

Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK (2005) Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 25:11730-11737. Eichele T, Debener S, Calhoun VD, Specht K, Engel AK, Hugdahl K, von Cramon DY, Ullsperger M (2008) Prediction of human errors by maladaptive changes in event-related brain networks. Proc Natl Acad Sci U S A 105:6173-6178.

Eichele T, Specht K, Moosmann M, Jongsma MLA, Quiroga RQ, Nordby H, Hugdahl K (2005) Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A 102:17798-17803. Esposito F, Mulert C, Goebel R (2009) Combined distributed source and single-trial EEG-fMRI modeling: Application to effortful decision making processes. Neuroimage 47:112-121.

Fox MD, Raichle ME (2007a) Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci 8:700-711.

Fox MD, Snyder AZ, Vincent JL, Raichle ME (2007b) Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior. Neuron 56:171-184.

Fox MD, Snyder AZ, Zacks JM, Raichle ME (2006) Coherent spontaneous activity accounts for trial-to-trial variability in human evoked brain responses. Nat Neurosci 9:23-25.

Fuglø D, Pedersen H, Rostrup E, Hansen AE, Larsson HBW (2011) Correlation between single-trial visual evoked potentials and the blood oxygenation level dependent response in simultaneously recorded electroencephalography-functional magnetic resonance imaging. Magn Reson Med 000:000-000.

Gaspar CM, Rousselet GA, Pernet CR (2011) Reliability of ERP and single-trial analyses. Neuroimage 58:620-629.

Gaudes CC, Petridou N, Francis ST, Dryden IL, Gowland PA (2011) Paradigm free mapping with sparse regression automatically detects single-trial functional magnetic resonance imaging blood oxygenation level dependent responses. Hum Brain Mapp 000:000-000.

Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535-574.

Goldman RI, Wei CY, Philiastides MG, Gerson AD, Friedman D, Brown TR, Sajda P (2009) Single-trial discrimination for integrating simultaneous EEG and fMRI: Identifying cortical areas contributing to trial-to-trial variability in the auditory oddball task. Neuroimage 47:136-147.

Grandchamp R, Delorme A (2011) Single-trial normalization for event-related spectral decomposition reduces sensitivity to noisy trials. Front Psychol 2:236.

Huettel SA, Song AW, McCarthy G (2009) Functional magnetic resonance imaging. Sunderland, MA: Sinauer.

Huster RJ, Debener S, Eichele T, Herrmann CS (2012) Methods for simultaneous EEG-fMRI: An introductory review. J Neurosci 32:6053-6060.

Jimura K, Poldrack RA (2012) Analyses of regional-average activation and multivoxel pattern information tell complementary stories. Neuropsychologia 50:544-552.

Kamitani Y, Tong F (2006) Decoding seen and attended motion directions from activity in the human visual cortex. Curr Biol 16:1096-1102.

Kiebel SJ, Friston KJ (2004) Statistical parametric mapping for event-related potentials: I. Generic considerations. Neuroimage 22:492-502.

Kostelecki W, Dominguez LG, Velázquez JLP (2011) Single trial classification of magnetoencephalographic recordings using Granger causality. J Neurosci Methods 199:183-191.

Luck SJ (2005) An introduction to the event-related potential technique. Cambridge, MA: MIT Press.

Mulder MJ, Wagenmakers EJ, Ratcliff R, Boekel W, Forstmann BU (2012) Bias in the brain: A diffusion model analysis of prior probability and potential payoff. J Neurosci 32:2335-2343.

Mulert C, Leicht G, Hepp P, Kirsch V, Karch S, Pogarell O, Reiser M, Hegerl U, Jäger L, Moller HJ, McCarly RW (2010) Single-trial coupling of the gamma-band response and the corresponding BOLD signal. Neuroimage 49:2238-2247.

Mumford JA, Turner BO, Ashby FG, Poldrack RA (2012) Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses. Neuroimage 59:2636-2643.

Nieuwenhuis ILC, Takashima A, Oostenveld R, McNaughton BL, Fernández G, Jensen O (2011) The neocortical network representing associative memory reorganizes with time in a process engaging the anterior temporal lobe. Cereb Cortex 22:2622-2633.

Obermaier B, Cuger C, Neuper C, Pfurtscheller G (2001) Hidden Markov models for online classification of single trial EEG data. Pattern Recogn Lett 22:1299-1309.

OʼDoherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. Palmer J, Huk AC, Shadlen MN (2005) The effect of stimulus strength on the speed and accuracy of a perceptual decision, J Vis 5:376-404.

Philiastides MG, Ratcliff R, Sajda P (2006) Neural representation of task difficulty and decision making during perceptual categorization: A timing diagram. J Neurosci 26:8965-8975.

Philiastides MG, Sajda P (2005) Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cereb Cortex 16:509-518.

Prado J, Weissman DH (2011) Heightened interactions between a key default-mode region and a key task-positive region are linked to suboptimal current performance but to enhanced future performance. Neuroimage 56:2276-2282.

Ratcliff R (1978) A theory of memory retrieval. Psychol Rev 85:59-108.

Ratcliff R, Philiastides MG, Sajda P (2009) Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proc Natl Acad Sci U S A 106:6539-6544.

Ratcliff R, Van Dongen HPA (2009) Sleep deprivation affects multiple distinct cognitive processes. Psychon B Rev 16:742-751.

Ratcliff R, Van Dongen HPA (2011) Diffusion model for once-choice reaction-time tasks and the cognitive effects of sleep deprivation. Proc Natl Acad Sci U S A 108:11285-11290. Ridderinkhof KR, Ullsperger M, Crone E, Nieuwenhuis S (2004) The role of the medial frontal cortex in cognitive control. Science 306:443-447.

Scheibe C, Ullsperger M, Sommer W, Heekeren HR (2010) Effects of parametrical and trial-to-trial variation in prior probability processing revealed by simultaneous electroencephalogram/functional magnetic resonance imaging. J Neurosci 30:16709-16717.

Spreng RN (2012) The fallacy of a “task-negative” network. Front Psychol 3:145.

Sutton RS, Barto AG (1998) Reinforcement learning: An introduction. Cambridge, MA: MIT Press.

Van Maanen L, Brown SD, Eichele T, Wagenmakers EJ, Ho T, Serences J, Forstmann BU (2011) Neural correlates of trial-to-trial fluctuations in response caution. J Neurosci 31:17488-17495.

Weissman DH, Roberts KC, Visscher KM, Woldorff MG (2006) The neural bases of momentary lapses in attention. Nat Neurosci 9:971-978.

Wunderlich K, Rangel A, OʼDoherty JP (2009) Neural computations underlying action-based decision making in the human brain. Proc Natl Acad Sci U S A 106:17199-17204. Yarkoni T, Barch DM, Gray JR, Conturo TE, Braver TS (2009) BOLD Correlates of trial-by-trial reaction time variability in gray and white matter: A multi-study fMRI analysis. PLoS one 4(1):e4257.

Yuan H, Perdoni C, Yang L, He B (2011) Differential Electrophysiological Coupling for Positive and Negative BOLD responses during Unilateral Hand Movements. J Neurosci 31:9585-9593.