Dimitris Katsimpokis Research Program in Psychological Methods Department of Psychology University of Amsterdam 24/7/2016 Supervisor: Dr. Leendert van Maanen Co-assesor: Dr. Guy Hawkins
Experimental Evidence on Collapsing Thresholds and their Relation to
Skewness and Conditional Accuracy Functions
*Abstract. The present paper presents experimental evidence for collapsing
thresholds in models of perceptual decision-making. It tests the predictions of Frazier & Yu (2008) regarding monotonically collapsing thresholds that apply when the decision-maker is faced with deadlines. These predictions concern skewness and Conditional Accuracy Functions of the Reaction time distributions, and their interrelation as well. Furthermore, it challenges some established interpretations of the speed-accuracy tradeoff manipulation, as well as making a number of methodological remarks on the analysis of Conditional Accuracy Functions and skewness of the Reaction Time distributions. The paper concludes by discussing potential further research in related areas such as experimental economics.
1. Introduction
The study of perceptual decision-making has been advanced by the mathematical modeling of cognitive processes, flourishing particularly in the field of psychophysics (Ratcliff, 1978; Ratcliff & McKoon, 2008) and recently reinvigorated by cognitive neuroscience (Bogacz et al., 2010; Bogacz & Gurney, 2007; Lo & Wang, 2006). These mathematical models are characterized by an ability to provide detailed mathematical accounts of the behavioral data, including Reaction Time (RT) distributions of correct and erroneous responses. These mathematical accounts boil down to a set of parameters, each of which describe a particular sub-process that decision-making consists of. Phenomena such as the speed-accuracy tradeoff (SAT) (Heitz, 2014; Wickelgren, 1977) or biased decision-making (Mulder et al., 2013) have been effectively modeled and attributed to single parametrical changes. The formal nature of the models and the attribution of behavioral changes to a single parameter have the desirable effect of providing a rigorous explanation of cognition at a higher
* I would like to thank the audience of the COMA meeting for their constructive comments on a presentation of
some earlier results of the paper. I am indebted to Guy Hawkins and especially Leendert van Mannen for their insightful comments and helpful suggestions as well as their overall support during the time I was conducting this research. I would also like to thank Georgia Odantzi and Marloes Warnar for helping me with style and proofreading.
level, while at the same time allowing for parametrical (not only behavioral) correlations with activity at lower neural levels.
These models assume a one-stage cognitive process, where the decision-maker accumulates noisy information in favor of specific choices at hand. The process of accumulation stops when the latter reaches a threshold (Forstmann, et al., 2016). Usually, thresholds are modelled as parametrical values, which are allowed to vary from trial-to-trial. However, the shape of the thresholds is not assumed to be the same across models. In fact, the treatment of it defines two classes of models: models such as the Drift Diffusion Model (DDM) treat thresholds as an unaltered variable within a given trial (Ratcliff & McKoon, 2008). Other researchers have made use of collapsing or other kind of thresholds to optimize the predictions of their model and thus to make it explain a larger set of behavioral phenomena (Cicek et al. 2009; Hawkins et al, 2015; Thura et al., 2012). In particular, Frazier and Yu (2008) suggested a particular shape of thresholds for cases where the decision-maker is faced with deadlines, that is cut-offs during the decision process, after which no answer can be given.
In this paper, we formulate and test two predictions derived from the mathematical analysis and simulations of Frazier and Yu, which concern the skewness of the RT distributions and the Conditional Accuracy Functions (CAFs), namely accuracy as a function of time, in conditions involving deadlines. We tested these predictions experimentally, making an explicit use of deadline manipulations. Our goal is to show (i) that these predictions are validated, (ii) that the shape of the CAFs and the respective skewness of the RT distributions are interconnected, (iii) that behavioral signatures of deadlines can effectively be distinguished from those of speed-accuracy tradeoff (SAT) through conditional manipulation and (iv) that the study of skewness of the RT distributions and CAFs requires a new set of methodological approaches
2. Collapsing bounds under deadlines
Frazier & Yu (2008) provided a proof about the collapsing shape of thresholds under deadlines in the decision-making process. They proved that when the decision-maker is confronted with deadlines, thresholds should be of collapsing shape, monotonically converging
to !", where α is the value of the threshold for a given trial. In contrast, when no deadlines are
imposed, thresholds should be fixed, following the standard design of the DDM.
The proof of the collapsing shape of thresholds regards the optimal strategy of the decision-maker, that they should follow within a given trial. Thus, Frazier & Yu (2008) base their proof on a Bayesian learning framework, where the participant is assumed to update
beliefs according to a version of Bayes’ rule, particularly designed for two-alternative forced choice tasks:
(1) 𝑝$%& = 𝑝 𝜃 = 1 𝒙$%&) = ,-./ 012/
,-.
/ 012/ % & 4,- .5 6-2/
where 𝒙 is the vector of observations x
&
x"
⋮
drawn from two underlying density probability functions 𝑓&and 𝑓:, 𝜃 is their index and 𝑝$ is the probability of 𝜃 = 1 at time t. The probability
p in (1) gets iteratively updated at every time step, so that it incorporates the prior beliefs of
the participant, as well as the likelihood of the present probability of the data being drawn from f1 or f2. There is a cost coupled with each unit time of delay and also cost for not responding within the deadline. Finding the stopping time and the correct hypothesis at that time which minimizes the overall cost, constitutes the optimal decision-making policy, given the deadline.
The cost of each unit time can be divided into stopping cost at time t or the cost of continuing at time t. The proofs of Frazier and Yu (2008) determine the shape of cost curves as a function of 𝑝$ (Figure 1). All cost curves should be concave down and generally speaking,
stopping at time t is preferable than continuing one step more, which in turn is preferable than continuing two steps more. Only when 𝑝$ is around 0.5, continuing becomes less costly than
stopping. The shape of cost curves implies a “window of continuation”, which necessarily gets narrower over time, as the cost generally accumulates, so that optimal behavior is reached (Frazier and Yu, 2008). The shape of thresholds follows this window of continuation, and thus they should be monotonically collapsing, converging at !", (Figure 1).
Figure 1: Cost of continuing one unit time (blue), two units (black) and stopping cost (red) as a function of the probability p of θ = 1, in conditions involving deadlines (figure adopted from Frazier and Yu (2008)).
The collapsing shape of thresholds makes two fundamental predictions with respect to accuracy and time of the decision-process. In particular, RT distributions should be less skewed when the decision-maker faces deadlines, as compared to cases where the space of continuation is unlimited (as in the case of DDM for example). Some models express this prediction strongly, by formulating the collapsing shape of the thresholds in terms of a linear urgency function, namely a signal of the urge of giving a response which grows over time (Cisek et al., 2009; Thura et al., 2012). These models predict that RT distributions should be Gaussian (Hawkins et al., 2015). However, human RT distributions are generally positively skewed and not Gaussian (Hawkins et al., 2015; Luce, 1986). Instead, a more gradual measure of the departure from normality would suffice as a tool to studying the change of RT distributions. For that reason, we take skewness of the distributions as the measure of deviation from normality.
The second prediction has to do with the shape of CAFs, defined as accuracy over unit time. As shown by Frazier & Yu’s own simulations, CAFs follow the thresholds: if thresholds are fixed then CAFs remain fixed, while if thresholds are collapsing, then CAFs follow too. Therefore, the signature of collapsing thresholds should be found in collapsing CAFs.
3. Methods of Analysis
The analysis of CAFs followed the suggestions of Jaeger (2008) namely that ANOVA is not the appropriate tool for binomial data. The basic argument concerns the violation of the assumption of homogeneity: the variance and the mean of a binomial distribution depend on the probability p that the answer is correct in every trial. As a result, a change in p induces a change in both mean and variance, contrary to the assumption of ANOVA regarding equal variance in distributions across experimental conditions (Cochran, 1940). More generally, p determines that the variance in sample proportions follows the shape of an inverted parabola with a global maximum at p = 0.5. If p is around 0.5, it has a minor effect on the variance, but as p reaches either {0,1}, the effect enlarges.
Jaeger (2008) instead advocates the use of logit mixed models (LMM), which are generalized linear models in ln-odd space (see Winter, 2013 for an introduction). Generalized linear models extend the scope of applicability of the general linear model to cases where the response variable does not follow a normal distribution. Specifically for LMMs, probabilities of outcomes are transformed into odds of which the natural logarithm follows the equation:
(2) ln&4,, = ln 𝛽: + ln 𝛽& 𝑥&+ ln 𝛽" 𝑥"+ ⋯ + ln 𝛽A 𝑥A
where β are the coefficients and x are the predictor values. Given ln 𝛽A 𝑥A are constants and
given that the right part of equation (2) is a linear combination of predictors and coefficients (scalars), equation (2) can be further reduced to:
(3) ln&4,, = 𝒙𝑻𝜷
where 𝒙𝑻 is the transposed vector of predictor values and 𝜷 the vector of coefficients.
Furthermore, as every mixed-effect model, LLMs incorporate a random-effects component that can be specified, thus accounting for additional variance that cannot be explained by predictors as well as correcting for violations of the assumption of independence across experimental distributions, which in our case stems from the within-subjects experimental design. The random-effects component minimally assumes random intercepts for each participant and the model’s intercept is the mean of them.
Following the suggestion of Barr et al. (2013), we kept the random-effects component as maximal as possible by assuming also random slopes for every predictor, something which
prevents the model from making too many type I errors. In addition, we kept the models full, that is we always included all the interaction terms in the independent variables, so that to prevent the model from overcontributing significant effects to the main effects only. To verify whether predictors have a main effect, we proceeded with model comparisons between the full LMM and a “null” version of it, which did not include the predictor of interest from the fixed and the random effects component.
The visualization of CAFs followed the standard method of bin separation (Heitz, 2014). Each RT distribution of each condition was separated into 20 bins of 5% quantile interval length and the mean accuracy of every interval was computed. This method is a less complicated instance of kernel regression, which computes an expected value 𝐸(𝑌|𝑋) by locally computing an arbitrary function over X, which in many cases involves a process of local kernel averaging (Simonoff, 1996). The main merit of this method lies in its simplicity. It should be noted however, that on the one hand, there is loss of information as a result of averaging and on the other hand, bin-based CAFs assume that the response variable is non-categorical. For these reasons, the predictions of the LLMs were plotted against CAFs, in order to verify the robustness of our visualization procedure.
The analysis of skewness was based on Kelly’s coefficient and 𝐺& which is a moment-based measure; an extension of Pearson’s moment coefficient for samples (Ekstrom & Jammalamadaka, 2013; Goreneveld & Meeden, 1984). Kelly’s coefficient is a percentile-based measure that encompasses 80% of the distribution and is defined as follows:
(4) JK5 % J/54 "JL5
JK5 4 J/5
where 𝑃A is the nth percentile of the distribution and 𝑃N: is the median. Given a symmetric distribution, the distance from the median to the 𝑃O: should be equal to the distance from the median to the 𝑃&:. Kelly’s coefficient calculates the deviation from symmetry as difference between this distances of the 90th and 10th percentile from the median.1 Kelly’s coefficient is a measure of absolute skewness, and for that reason its range of values spans the interval (-1, 1), where -1 indicates the most skewed distribution with right tail, 1 with left tail, and 0 indicates a Gaussian distribution.
Pearson’s moment coefficient is a more frequently used measure of skewness and is defined as the third standardized moment:
(5) Ε[ R 4 ST U] =SW
TW
where
𝑋
is a random variable,𝜇
is the population mean,𝜎
is the standard deviation and𝜇
Uisthe third central moment. For sample skewness the 𝐺& statistic is used among others, which is
defined below: (6) 𝐺& = A Z (A4&)(A4") [W \W
where 𝑛 is the sample size and 𝑚 and 𝑠 are the third central moment and standard deviation respectively. As it was the case with Kelly’s coefficient, the sign 𝐺& indicates the direction of
the tail of distribution (negative sign, negatively skewed distribution, positive sign, positively skewed). However, in 𝐺& the values are not bounded within (-1, 1) interval.
Joanes and Gill (1998) showed that 𝐺& along with other moment-based measures of skewness are unbiased estimators when the sample is drawn from a normal distribution. For the 𝜒" distribution that they tested however, the statistic turned out to be biased, but it was the
less biased statistic (along with 𝐺") out of these calculating skewness through the method of
moments. For these reasons both Kelly’s coefficient and 𝐺& were used to verify whether any
discrepancy occurred between the two as well as to assess which of them captures our experimental data better.
A final point on methods of analysis concerns our research question about the non-linear shape of CAFs under deadlines. Typically, they exhibit an initial time interval under which accuracy is at chance levels. As time increases, CAFs increase as well to reach their maximum value, resembling a logistic function. We are interested in the interval of the CAF curve after this maximum value. No model has been proposed in the literature that captures non-linearity, to our knowledge. As a result, trimming operations to isolate and obtain the intervals of interest become crucial. In addition to it, the LMM, which we previously proposed to apply is a specification of the linear model, meaning that major deviations from linearity would induce artifacts to its coefficients.
A rule of thumb we used in our analysis was to trim CAFs after 90% percentile and before the median of the respective RT distribution. Trimming the last 10% would secure homoscedasticity for the LMM, namely the homogeneity of variance of the predicted value across time. The major motivation for trimming pertains to human RT distributions, which are not normal but positively skewed, as their long right tail contains only a small amount of observations; an amount which decreases as time increases. For heavily skewed distributions, such as the ones from conditions not involving deadlines in the decision-making process, it is important that homoscedasticity be protected.
For the critical mind however, even if the last 10% is to be cut out justifiably, trimming values below the median seems arbitrary. Cutting at 45% or 55% would sound similar. To rebut such a doubt, we will give a theory-driven motivation about the optimal cutting point. Given that the shape of CAFs for some arbitrary initial period is known (resembling a logistic function) as well as given that CAFs reach their maximum after that period, there are three scenarios to consider: the rest of the CAFs either increasing, decreasing or staying fixed. If the function increases, trimming a little above or below the median would not affect the coefficients of the LMM, because the relationship is linearly increasing anyway. If the function stays fixed, trimming it before it comes to its maximum (we suppose it corresponds around its median) would lead to a positive coefficient, which is not predicted by Frazier and Yu (2008), nor by the literature. In this case, we have a theory-derived motivation to consider the trimming point as being set too early. If we trimmed after the median, the coefficients would remain close to zero and thus it would not constitute a problem for the LMM. In case of a collapsing bound, if we trimmed too early, the coefficients would turn out to be close to zero, because the collapsing shape of the CAFs at the second half of it would counter-act the first increasing half. This would violate the assumption of linearity of the linear model and thus would be dismissed. If we trimmed after their maximum value, that would not have any effect on the LLM’s coefficients. Thus it turns out that the only successful trimming operation is the one which yields a linear negative relation or no relation at all, because of theory-derived reasons. If the relation is not linear, this means the trimming point (the median in this case) has to move to the right. Finding the optimal trimming point would secure homoscedasticity and linearity for the LMM. Until a new method that would presuppose a non-linear method of analysis is
proposed, the aforementioned method of analysis seems the optimal way of studying the shape of CAFs.2
4. An interlude on experimental manipulation
Before we report on our experimental design and results, it is important to cast a critical gaze on a set of established interpretations of the SAT manipulation in perceptual decision-making. It turns out that the SAT manipulation induced showed signs of deadlines, as the ones suggested by Frazier and Yu (2008), in a way that undermines the original goal of these studies. The result of such a misinterpretation underscores the importance of an experimental manipulation targeted at distinguishing deadline effects from SAT; a step that would also help research on the neural mechanisms, underlying such effects. We will first discuss Forstmann et al. (2008) and Mulder et al. (2013), where the signs of deadlines are strong and we will conclude with Wagenmakers et al. (2008), where no strong sign of deadlines was found.
Forstmann et al. (2008): experimental design. The experimental design of Forstmann
et al. had three conditions: one where participants were cued to be fast, another one to be neutral and a third one to be speedy. The design can be thought of as involving an implicit deadline in the speed condition. In this condition, participants received feedback every time their RT exceed 450ms stating “too late response”, and additionally, stimuli disappeared from the screen after 1500ms. Negative feedback worked as a reminder of a late response, while the disappearance of the stimuli from the screen acted as an indication of a cut-off in the decision-making process, after which a response should be given. If the speed condition was interpreted by the participants as implying a deadline, then we should be able to find signatures in the data like the ones I discussed before.
Forstmann et al. (2008): results. To test the two predictions on Forstmann et al. (2008)
dataset, a LMM was constructed and fitted to the CAFs of speed and accuracy conditions, with two predictors, namely RTs and SAT (the first numeric the second categorical with two levels)3
. Both RTs (𝜒"(4) = 15.29, p = 0.004) and SAT had a main effect on accuracy (𝜒"(4) = 67.78, p < 0.001). Specifically, the coefficient of the LMM for the accuracy condition was significantly negative (β = -1.103, SE = 0.473, Z = -2.333, p = 0.019), while the coefficient for the speed condition showed a trend (β = -1.011, SE = 0.608, Z = -1.661, p = 0.096). SAT had
2 A word of caution needs to be delivered here: the trimming method which we proposed holds if and only if the
shape of CAFs is linear for the interval after their maximum value.
3 There was a third level in the original study, the “neutral” condition, where the participant was not instructed
no effect on the skewness (t(19) = 0.405, p = 0.689), when the latter was calculated by the method of moments. However, it showed a main effect when calculated by the method of quantiles (t(19) = 5.58, p < 0.001). Finally, no correlation among the coefficients of the LMM and skewness was found (𝜒"(1) = 1.99, p = 0.157), but a correlation of SAT and coefficients was found, as expected (𝜒"(1) = 21.44, p < 0.001).
Mulder et al. (2013): experimental design. Mulder et al. (2013) represents an extension
of the standard random-dot motion task for two alternative-forced choices into a different modality, namely auditory perception. In this new task, participants had to indicate whether the frequency of a sound increases or decreases, while the sound itself was embedded into a cloud of other sounds, whose frequencies were moving to random directions. The experimental design of Mulder et al. (2013) was similar to Forstmann et al. (2008). In Mulder et al (2013), a personalized timing of the “too slow” message was set, after fitting a version of the diffusion model to data of the accuracy condition. Also, Mulder et al. (2013) make no mention of disappearance of the visual or auditory stimuli from the screen or computer speakers. The following reanalysis concerns only the auditory data of the dataset.
Mulder et al. (2013): results. The same LMM design was followed as before. Both RTs
(𝜒"(4) = 112.27, p < 0.001) and SAT had a main effect on accuracy (𝜒"(4) = 80.80, p < 0.001).
As in Forstmann et al. (2008), the coefficient of the LMM for the accuracy condition (β = -2.105, SE = 0.877, Z = -2.400, p = 0.016) and the coefficient for the speed condition were significantly negative (β = -2.105, SE = 0.878, Z = -2.396, p = 0.016). Unlike Forstmann et al. (2008), SAT had no effect on skewness of the distributions, either calculated through the method of moments (t(3) = -0.105, p = 0.923) or based on quantiles (t(3) = 0.130, p = 0.904). Furthermore, no correlation among the coefficients of the LMM and skewness was found
(𝜒"(1) = 0.016, p = 0.899), neither among skewness and the coefficients of the LMM, as
expected (𝜒"(1) = 0.687, p = 0.407).
Wagenmakers et al. (2008): experimental design. Wagenmakers et al. (2008) studied
behavioral performance on the lexical decision task with respect to decision criteria. As part of their experiments, they ran a SAT manipulation task, controlling for high frequency, low frequency, very low frequency words and non-words derived from the aforementioned three frequency categories (Experiment 1; Wagenmakers et al. (2008)). The experimental design of the task departed from the one in Forstmann et al. (2008) and Mulder et al. (2013). On the one hand, it included accuracy feedback in the accuracy conditions and speed feedback in speed
conditions. However, speed feedback was not given during the trial but only after it, if that trial exceeded 800ms.
Wagenmakers et al. (2008): results. With respect to CAFs, the coefficients of RTs in
all accuracy conditions were not found significantly different from zero (all p values were found larger than 0.4). Furthermore, the coefficients of RTs in all speed conditions (all p values were again found larger than 0.4) were also not found significantly different from zero. With respect to skewness of RT distributions, SAT (F(1, 16) = 37.31, p < 0.001) and frequency of words (F(5, 80) = 6.405, p < 0.001) had a main effect on skewness, when the latter was calculated based on quantiles. Furthermore, only SAT had a main effect on skewness (F(1, 16) = 15.64, p = 0.001) but not word frequency (F(5, 80) = 1.915, p = 0.101), when skewness was calculated by the method of moments. Eventually, no correlation among the coefficients of the LMM and skewness of the distributions was found (𝜒"(9) = 9.178, p = 0.420).
The analysis of Forstmann et al. (2008), Mulder et al. (2013), and Wagenmakers et al. (2008) underscores the importance of manipulation through specific experimental conditions. The design options of Forstmann et al. and Mulder et al. showed the main signatures of a deadline being involved, despite the authors aiming at a SAT manipulation. However, no certain conclusion can be drawn, because of particular difficulties in each data set. In Mulder et al. for example, no correlation among the LMM coefficients and skewness and no stable calculation of skewness could be established because of the very small sample size (the study contained only 4 participants). In Forstmann et al., the evidence for a deadline manipulation is more apparent given the negative shape of CAFs and the less skewed distribution found. However, it is not clear why a correlation among CAFs and skewness was not obtained. The data of Wagenamekers et al. shows the fewest signatures of deadlines, and this could be attributed to the different feedback configuration that the authors opted for. However, in this case too, it remains unclear why speed and accuracy had a main effect on skewness, given that no negative coefficients were found. All in all, it seems that deadline signatures depend on the particular experimental manipulation, but none of these previously published studies was capable to clearly show it. It is assumed that a direct deadline manipulation would be possible to induce deadline effects while separating the effects of SAT from deadline.
To overcome the problems observed in the previous three studies, we focus on setting manipulations capable of inducing deadline effects, and also of distinguishing them from SAT effects. The most straight-forward way seemed to be the imposition of deadlines in some conditions, while the participants being aware of the existence of a deadline during the
deicion-making process. In other conditions, no deadline applied, which would not limit the continuation horizon, in a DDM-like process.
5. Experiment 1
5.1 Materials and Methods
Participants. Twenty-four participants affiliated with the University of Amsterdam
were chosen through the UVA LAB platform (mean age = 23.45, SD age = 7.86). Fourteen of them were female (58%). All participants signed an informed consent as a requirement for participation, and chose between a monetary or research credit reward at will.
General procedure. The experiment took place at the University of Amsterdam, in a
closed cubical. The experiment was conducted on a desktop computer, and the stimuli were presented on a 21-inch monitor set at 60Hz frame rate. After the completion of the first task, the experimenter entered the cubical to switch the tasks.
Visual stimuli and tasks. The experiment was set up using Psychopy (Peirce, 2007)
implemented in Python programming language, and it included two tasks. The first was the Random-dot motion task (RDMT; Ball and Sekuler, 1982) for which the default Random Dot Kinematic component was used following the configuration of Scase et al. (1996). The default settings were mostly used: dot life-time of 5 frames, dot size of 4 pixels, cloud-size of 400 dots and speed of dots of 0.01 (movement per frame in the default window units). As for the movement algorithm, we followed one from Scase et al. (1996, p. 2580) in whih “noise dots have been plotted in new locations, randomly selected within the display area, on successive frame of the sequence. Pairing of noise dots in successive frames are therefore random; They should show a statistically isotropic distribution of directions and wide range of speeds”.
The task had a 2X2 design reflecting the conditions under investigation (speed vs accuracy and deadline vs no-deadline). It consisted of two parts. One was the training session comprised of 240 randomly shuffled trials, in which the level of difficulty (coherence) of the task was manipulated. In the RDMT, the level of difficulty changes as a function of the percentage of dots moving to left or right in each frame. There were 6 levels of 3%, 7%, 11%, 15%, 19% and 23%, each of which was presented for 40 trials. The second part of the experiment consisted of the four conditions, each of which presented for 200 trials. The level of difficulty was kept fixed across conditions, depending on the training session of each participant. This was chosen so that we exclude the possibility of any effect by a varying degree of difficulty. Specifically, the training level at which participants scored between 80% to 90% was selected, giving priority to the most difficult one over the others (if a participant scored
between 80% to 90% for 3% and 7% levels, for instance, 3% would have been selected). With respect to the timing of trials, in the training session and in the no deadline conditions, each trial finished with the participant’s response.
In the deadline conditions, decision periods (after the presentation of the stimulus) were cut off at 1.35sec after initiating the presentation of the stimulus, irrespectively of whether participants gave a response. The pre-stimulus period (fixation cross) lasted 250ms, while the post-stimulus period duration depended on the negative or positive feedback. In case where feedback was positive, it lasted 250ms, and in case where it was negative, it lasted 1sec, in order to help participants not to make mistakes.
The task was counterbalanced and at the end all the participants had seen all different permutations of the conditions (4! = 24 possible permutations for 24 participants). Moreover, trials of same condition were presented together during the Experiment (for instance, 200 trials of speed-deadline, then other 200 of accuracy-deadline and so on).
The second task used was the Flash Τask (FT), where participants are presented with two flashing dots and they are asked which of them has the higher flashing frequency rate. The design and timing of the task remained the same compared to the RDMT. Due to the nature of the task, difficulty levels were manipulated differently. There were 6 levels of 55% - 45%, 60% - 40%, 65% - 35%, 70% - 30%, 75% - 25% and 80% - 20% frequency rate, where the first percentage represents the rate of one dot and the second of the other. As in the RDMT, the training level at which participants scored between 80% to 90% was selected, giving priority to the most difficult one. The task was counterbalanced and all participants saw all different permutations of the conditions. For the rest, the same configuration was followed as in the RDMT.
Instructions. All the instructions were given on screen. In the beginning, participants
were briefly introduced to the relevant task and were given instructions for the keyboard keys they had to press during the trials. They were notified that there would be a training period, after which the original experiment would run. Participants were asked to focus on being accurate during training. Furthermore, before each condition was introduced, participants were instructed to either focus on being speedy or accurate, and they were notified about whether the trials included deadlines; all of which depending on the condition.
Feedback. After each trial and conditional to each one, participants were given
feedback on their performance. In training sessions, participants received feedback on their accuracy. In the deadline conditions, participants received feedback on whether they passed the deadline or not and, in the accuracy conditions, the feedback concerned the participants’
correct or erroneous response. In the speed conditions, feedback regarded whether participants’ responses were speedy or not. In order to assess the speed of the responses, the RT distribution of the training session was taken as a reference.
Specifically, each participant’s RT was compared to their training RT distribution in each respective trial. We fitted a gamma distribution to the training RT distribution and we executed parameter recovery, for the two parameters of the gamma distribution (rate and shape). After obtaining the parameters, the probability of this particular RTs being drawn by the training distribution was calculated as well as and its inverse probability, through the cumulative distribution function. Consequently, the two probabilities were assigned to two messages, “be faster” and “good time” respectively, one of which was randomly selected, based on the aforementioned probabilities. This feedback procedure ensured within-subjects personalization and within-trial timing stochasticity, thus avoiding uniform cut-offs like the ones found in Forstmann et al. (2008).
5.2 Results
The results of both tasks were analyzed through the R programming language for statistical programming (R Development Core Team, 2008).
RDMT. The overall CAFs showed a pattern frequently noted in the literature (Figure
1A, left panel): up to the first 40% of the RT distribution, accuracy is at chance levels, or at a “fast guessing” stage, where participants randomly choose an answer. After this period of time, CAFs increase and reach their maximum value. The interval of interest pertaining to one of our research question lies after this maximum: do CAF curves decrease monotonically or do they remain linearly fixed?
An LMM was fitted to the data. The three predictors were RTs, SAT and deadline, where the first is a continuous variable and the last two are categorical with two-levels each (speed-accuracy and deadline-no-deadline respectively). An overall effect of RT (𝜒"(8) = 40.509, p < 0.001), of SAT (𝜒"(8) = 124.48, p < 0.001) and of deadline (𝜒"(8) = 65.043, p < 0.001) were found. The coefficients of RT were almost all found to be significantly negative (Table1). Deadline had an effect on accuracy only in the accuracy conditions. SAT alone had no effect on accuracy.
RDMT
Predictors Estimate SE Z p RTaccuracy-deadline -4.1207 0.8911 -4.624 3.76e-06 *** RTspeed-deadline -2.4541 0.7971 -3.079 0.00208 ** RTaccuracy-nodeadline -0.0401 0.6185 -0.065 0.94827 RTspeed-nodeadline -2.2201 0.7541 -2.944 0.00324 **FT
Predictors Estimate SE Z p RTaccuracy-deadline -0.1958 0.5252 -0.373 0.7093 RTspeed-deadline 0.1992 0.6109 -0.326 0.7444 RTaccuracy-nodeadline -0.5215 0.1833 -2.845 0.0044 ** RTspeed-nodeadline 0.6768 0.4248 1.593 0.1111Table 1: Beta estimates, standard errors, z and p values of LMM's fixed effects for the RDMT and FT in logit space. The sign of the coefficients represents the predictor’s slope. Each predictor’s reference category is shown in subscript. There are three significance categories: ‘*’ for p<0.05, ‘**’ for p<0.01 and ‘***’ for p<0.001.
Moving on to the analysis of skewness, each participant’s skewness of the RT distributions of the four conditions was calculated with G1 and with Kelly’s coefficient (Figure
2B). For the data obtained through the method of moments, a 2X2 ANOVA was designed with two predictors, namely SAT and deadline, each of which having two levels (speed-accuracy and deadline-no-deadline respectively). A main effect of deadline was found (F(1, 23) = 24.81, p < 0.001. No main effect of SAT (F(1, 23) = 2.955, p = 0.099) or interaction effect were found (F(1, 23) = 2.099, p = 0.161). For the data generated by applying Kelly’s coefficient, the same ANOVA design was followed. A main effect of deadline of deadline (F(1, 23) = 11.65, p = 0.00238) and SAT (F(1, 23) = 6.223, p = 0.0202) was found. No interaction effect was found (F(1, 23) = 3.783, p = 0.0641). A 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Quantiles Accur acy Conditions deadline accuracy deadline speed nodeadline accuracy nodeadline speed 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Quantiles Accur acy Conditions deadline accuracy deadline speed nodeadline accuracy nodeadline speed
B C D
Figure 2: Results of Experiment 1. Row A: Untrimmed CAFs for the four conditions, with bin length of 5% percentile interval (left column RDMT, right column FT. Row B: RDMT, skewness calculated with the method of moments (left side) and based on quantiles (right side). Row C: FT, skewness calculated with the method of moments (left side) and based on quantiles (right side). Row D: Correlations of coefficients and quantile-based skewness (left column RDMT, right column FT). 'Dacc' stands for 'Deadline accuracy', 'Ndacc' for 'Nodeadline accuracy', 'Dsp' for 'Deadline speed' and 'Ndp' for 'Nodeadline speed'.
FT. The overall CAFs are shown in Figure 2A, right panel. The same statistical analysis
was used as the one for the RDMT. An overall effect of RT (𝜒"(8) = 24.046, p = 0.002), of SAT (𝜒"(8) = 63.159, p < 0.001) and of deadline (𝜒"(8) = 50.002, p < 0.001) were found
All coefficients of RT in the LMM were not found to be different from zero, except for the accuracy-no-deadline condition, where the coefficient was found negative (Table1).
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 Dacc Ndacc Dsp Ndsp Conditions Sk e wness −1.0 −0.5 0.0 0.5 1.0 Dacc Ndacc Dsp Ndsp Conditions Sk e wness −1.0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Dacc Ndacc Dsp Ndsp Conditions Sk e wness −1.0 −0.5 0.0 0.5 1.0 Dacc Ndacc Dsp Ndsp Conditions Sk e wness 0.00 0.25 0.50 0.75 −4 −2 0 Coefficients Sk e wness Conditions Deadline.acc NoDeadline.acc Deadline.sp NoDeadline.sp 0.0 0.2 0.4 −1.0 −0.5 0.0 0.5 1.0 Coefficients Sk e wness Conditions Deadline.acc NoDeadline.acc Deadline.sp NoDeadline.sp
Deadline had an effect on accuracy conditions, though not in speed ones and SAT had an effect on no-deadline conditions.
Moving on to skewness, each participant’s skewness of the RT distributions of the four conditions was calculated with the method of moments and Kelly’s coefficient (Figure 2C). The same statistical analysis was applied as in RDMT. Regarding the data obtained with the method of moments, a main effect of deadline was found (F(1, 23) = 22.96, p < 0.001). No main effect of SAT (F(1, 23) = 4.064, p = 0.055) or interaction effect were found (F(1, 23) = 0.162, p = 0.691). For the quantile-based data, the same ANOVA design was followed. A main effect of deadline of SAT (F(1, 23) = 4.46, p = 0.045) and an interaction effect between SAT and deadline (F(1, 23) = 7.872, p = 0.01) were found. No main effect of deadline was found (F(1, 23) = 0.431, p = 0.518).
To verify and quantify the relationship between skewness and the shape of CAFs the coefficients of the LMM, as well as exclude the possibility of solely the conditions having an effect of skewness, we constructed a linear mixed-effects model with three predictors, coefficients of the LMM, SAT and deadline (Figure 2D: left RDMT, right FT). In the RDMT, for the method of moments skewness, coefficients did not have a main effect on skewness
(𝜒"(1) = 1.969, p = 0.160) either SAT (𝜒"(1) = 2.953, p = 0.085). Only deadline induced a
main effect (𝜒"(1) = 10.7, p = 0.001). For quantile-based skewness, coefficients showed a main effect (𝜒"(1) = 3.854, p = 0.049), along with deadline (𝜒"(1) = 4.822, p = 0.028) and SAT
(𝜒"(1)= 12.057, p < 0.001). In FT, on the other hand, coefficients did not have a main effect
for quantile-based skewness (𝜒"(1) = 0.732, p = 0.392). Neither did deadline (𝜒"(1) = 0.894, p = 0.344) nor SAT (𝜒"(1) = 1.088, p = 0.296). For moments-generated skewness, coefficients and SAT did not have a main effect as well (𝜒"(1) < 0.001, p = 0.9898 and 𝜒"(1) = 2.92, p = 0.087). Only deadline showed a main effect (𝜒"(1) = 23.39, p < 0.001).
5.3 Discussion
Almost all predictions on CAFs and skewness were supported in the RDMT and partly in the FT. Furthermore, the correlation between skewness and collapsing CAFs (the coefficients of the LMM) was also partly supported. Specifically, the RDMT exhibited collapsing CAFs for deadline conditions and RT distributions were less skewed. The opposite found in the accuracy no-deadline condition, namely fixed CAF and more skewed RT distribution. However, the speed no-deadline condition diverged from that pattern by resembling a deadline condition.
To understand this divergence, we have to consider how often participants received positive feedback for their speed (Figure 3). Speed feedback was given only in the speed conditions, and only if the deadline was not exceeded. Each trial’s RT was compared to the training RT distribution, selected for every participant. The training distribution was fitted a gamma distribution and through the cumulative function, the probability of trial was obtained. Figure 5 shows the results. The probability that participants received positive feedback for their speed is generally high (higher for FT than for the RDMT), which means that trials were generally shorter in speed conditions, compared to the training session. As a result, the speed feedback acted as a deadline, which on the one hand succeeded in inducing a speed-accuracy trade-off, on the other hand failed to induce conditions of no-deadline (cf. also that historically SAT manipulation was achieved by imposing deadlines in the decision-making process).
In the FT, the relationship between collapsing bounds and less positive skewness was again supported, but not in the expected conditions. Specifically, the CAF of accuracy no-deadline showed a collapsing shape, while the rest being fixed. A potential explanation concerns mean accuracy between the tasks. The FT was perceived as more difficult with respect to RDMT. Because accuracy levels were a bit higher than chance levels in the FT, with the exception of no-deadline accuracy, this prevented CAFs from collapsing in the respective conditions.
For these reasons, a new experiment was designed that firstly would increase the accuracy over RT and, secondly would block the speed feedback as a potential deadline.
A 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
deadline speed nodeadline speed Conditions
B
Figure 3: probability of RTs of each trial being drawn by the selected training distribution. A: RDMT, B: FT. Probabilities were calculated through the cumulative distribution function of the fitted gamma distribution of every selected training distribution. In the FT, the probabilities are closer to zero as opposed to the RDMT. Thus, participants received a message with positive feedback on their RT, much more frequently in FT than in RDMT.
6. Experiment 2
6.1 Materials and Methods
Participants. Twenty-four participants affiliated with the University of Amsterdam
were randomly chosen through the UVA LAB platform (mean age = 22.75, SD age = 3.40). Fifteen of them were female (62%). All participants signed an informed consent as a requirement for participation, and chose between a monetary or research credit reward at will.
General procedure. The same procedure was followed as with the previous experiment. Visual stimuli and tasks. In Experiment 2, only the FT was employed because it was
the one that showed the less confirmatory results. The experiment had a 2x3 design, namely two levels for SAT and 3 levels for deadline (early, late and no deadline). The late deadline was set as the time where the deadline in the first experiment was set, namely 1.35s, while the early deadline was set at 683ms. The latter was selected as the 75% quartile of the aggregate RT distribution in deadline conditions of the data from the previous experiment. The goal was to set a deadline that would be early enough to induce the expected behavioral signature, but not too early to cause chance-level accuracy.
Most of the presentation settings were retained as in Experiment 1. The training trials reduced to 30. Unlike the previous experiment, a lower level of difficulty was selected in order to assist participants boost their accuracy over time, so that to exclude the possibility of not finding collapsing CAFs because of low aggregate accuracy levels, as in Experiment 1. The training level at which participants scored between 90% to 100% was picked up, giving priority to the most difficult one. The task was counterbalanced and all participants saw different
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
deadline speed nodeadline speed Conditions
combinations of the conditions. The combinatorial method used was Latin-Graeco square, because of the difficulty of presenting all possible permutations to each participant. Since the experiment had 6 conditions, this would require 6! = 720 unique permutations and thus 720 participants of each of them. Conditions involving deadlines were presented together in order to assist participants learn the deadline.
Instructions. The instructions settings were kept the same as in Experiment 1. The only
addition had to do with the new conditions introduced.
Feedback. Unlike Experiment 1, we generalized accuracy feedback for every trial
including speed conditions and training. To ensure that speed conditions would show a difference with respect to RT distributions and accuracy, speed feedback was given after each trial of the respective condition. The same RT comparing procedures along with the same feedback text were used as in Experiment 1.
6.2 Results
The results can be seen in Figure 4. For the study of CAFs, a LMM was constructed with three predictors to reflect the 2X3 experimental design, namely SAT, deadline and RT, the first having two levels (“speed” and “accuracy”), the second having three levels (“no-deadline”, “early deadline” and “late deadline”), and the third being a contiuous variable (Figure 4A). RT was found to have a main effect on accuracy 𝜒"(11) = 22.56, p = 0.020), as
well as SAT (𝜒"(11) = 43.27, p < 0.001) and deadline (𝜒"(17) = 88.58, p < 0.001). The coefficients of RT for all conditions are given in Table 2. CAFs remain fixed for no-deadline conditions, as expected, but only two out of four showed negative estimates.
Regarding the quantile-based skewness (Figure 4B upper panel), only deadline had a main effect on skewness (F(2, 46) = 23.59, p < 0.001), unlike SAT (F(1, 23) = 0.251, p = 0.621). Furthermore, no interaction effect was found (F(2, 46) = 0.026, p = 0.974). When skewness was calculated through the method of moments (Figure 4B lower panel), a similar pattern was found: deadline showed a main effect (F(2, 46) = 83.78, p < 0.001) but not did SAT (F(1, 23) = 0.534, p = 0.472). As for interaction, a trend was found (F(2, 46) = 3.122, p = 0.053).
FT
Predictors Estimate SE Z p RTaccuracy-nodeadline -0.648 0.451 -1.439 0.150 RTspeed-nodeadline 0.086 0.898 0.096 0.923 RTaccuracy-early deadline -2.853 2.034 -1.402 0.160 RTspeed-early deadline -8.215 3.932 -2.089 0.036 * RTaccuracy-late deadline -1.608 0.652 -2.468 0.013 * RTspeed-late deadline -0.137 0.713 -0.192 0.847Table 2: Beta estimates, standard errors, z and p values of LMM's fixed effects for Experiment 2 in logit space. The sign of the coefficients represents the predictor’s slope. Each predictor’s reference category is shown in subscript. There are three significance categories: ‘*’ for p<0.05, ‘**’ for p<0.01 and ‘***’ for p<0.001.
The correlation between CAFs and skewness was established (Figure 4C). With respect to the method of moments, coefficients did not have a main effect on skewness (𝜒"(11) = 7.413,
p = 0.764) either SAT (𝜒"(11) = 14.20, p = 0.222). Only deadline induced a main effect (𝜒"(17) = 42.67, p < 0.001). For quantile-based skewness, coefficients showed a main effect (𝜒"(11) = 22.46, p = 0.021), along with SAT (𝜒"(11) = 29.548, p = 0.001) and deadline (𝜒"(17 )= 34.059, p = 0.008).
A
B
C
Figure 4: Results of Experiment 2. A: Untrimmed CAFs for the six conditions, with bin length of 5% percentile interval. Column B: skewness calculated with the method of moments (upper panel) and based on quantiles (lower panel). C: Correlations of coefficients and quantile-based skewness.
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Quantiles Accur acy Conditions earlyd.acc earlyd.sp lated.acc lated.sp nod.acc nod.sp −3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
Earlyd.acc Lated.acc Nod.acc Earlyd.sp Lated.sp Nod.sp Conditions Sk e wness −0.2 0.0 0.2 0.4 0.6 −10.0 −7.5 −5.0 −2.5 0.0 Coefficients Sk e wness Conditions EarlyD.acc LateD.acc NoDead.acc EarlyD.sp LateD.sp NoDead.sp −1.0 −0.5 0.0 0.5 1.0
Earlyd.acc Lated.acc Nod.acc Earlyd.sp Lated.sp Nod.sp Conditions
Sk
e
6.3 Discussion
Experiment 2 strengthened the behavioral evidence for deadline signatures as compared to FT in Experiment 1. Firstly, it replicated the findings of Experiment 1, thus further strengthening the initial observations made based on Experiment. It established the linear positive relationship between the shape of CAFs and the skewness of the RT distributions. It went further on to show the gradual increase of skewness of the distributions from early to late and to no-deadline conditions, while at the same time retaining same skewness levels of the RT distributions between speed and accuracy conditions. Moreover, Experiment 2 improved CAF fitting from undershooting, as only two out of the six conditions exhibited unexpected results. All in all, Experiment 2 constitutes a straight-forward experimental support for the mathematical model of collapsing bound along the lines of Frazier and Yu (2008).
Furthermore, unlike Experiment 1, where interactions effects of SAT and deadline were found, Experiment 2 clearly separates deadline effects from SAT effects, thus providing experimental support for the independence of SAT and deadline manipulations. As we saw in Wagenmakers et al. (2008) for instance, SAT had an effect on skewness, without being supported by collapsing CAFs. This casted doubt on whether that was a true deadline signature or an effect of a different sort. With Experiment 1 and 2, SAT and deadline effects are clearly separated through specific experimental manipulations.
Eventually, the results of Experiment 2 underscore the importance of selecting the proper method for calculating skewness. Quantile-calculated skewness was able to capture fine-grained, gradual differences among conditions and also to correlate with the shape of CAFs, showing a good fit of the data. Moment-based skewness however was found unable to correlate with CAFs, also producing biased values, as skewness turned out to take a lot of negative values in early deadline conditions; a result which contradicts main insights about human RTs. In general, the inadequacy of moment-based skewness to capture the important features of the data indicates the crucial significance of methodological considerations, which the paper started off with. Furthermore, the optimal way of trimming CAFs arose as an additional important methodological aspect. Because no non-linear method of studying CAFs has been proposed so far to your knowledge, trimming became an essential tool for obtaining the specific interval of interest. Trimming in turn emphasizes the importance of methodological considerations or implications about the particular methodological strategy that we are or are not incline to.
7. Conclusions and Future Research
The present paper provided behavioral evidence in favor of collapsing thresholds in line of Frazier and Yu’s (2008) mathematical predictions and simulations. Our data shows that a controlled and targeted manipulation can separate deadline from SAT effects, contrary to some established SAT experiments, in which deadline and SAT signatures had been blended together. The paper also sheds light on the relation between shape of CAFs and the skewness of the RT distributions, by demonstrating a positive correlation between the two. Furthermore, it makes a set of methodological remarks with respect to the optimal analysis of CAFs, by making use of LMMs, and skewness of the distributions, by comparing two distinct methods of calculation, the method of moments and the method through quantiles.
The paper did not touch upon a number of issues which are of crucial importance for future research. Firstly, a DDM-like model with collapsing thresholds could fit to the data, as a further means of verifying the separation between a SAT and a deadline effects. Secondly, a link towards experimental economics would advance the explanatory power of such a model, because decision-making under deadlines can be viewed as a type of risky behavior; a topic which experimental economics extensively studies (see Engelmann & Tamir, 2009 for instance). Thirdly, expanding the formulae of Frazier and Yu (2008) towards a full mathematical description of a DDM-like model with collapsing thresholds, by providing a set of differential equations that would relate the change of thresholds (fixed or collapsing) and accuracy with the change or existence of deadlines, would provide a unifying formal account of the separation between SAT and deadline effects.
8. References
Barr, D.J., Levy, R., Scheepers, C. and Tily, H.J., 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language,
68(3), pp. 255-278.
Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D., 2006. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychological review, 113(4), pp. 700-765.
Bogacz, R., & Gurney, K., 2007. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural computation, 19(2), pp. 442-477.
Cisek, P., Puskas, G. A. & El-Murr, S., 2009. Decisions in changing conditions: the urgency-gating model. Journal of Neuroscience, 29(37), pp. 11560-11571.
Cochran, W.G., 1940. The analysis of variance when experimental errors follow the Poisson or binomial laws. The Annals of Mathematical Statistics, 11(3), pp. 335-347.
Ekström, M. and Jammalamadaka, S.R., 2012. A general measure of skewness. Statistics &
Probability Letters, 82(8), pp. 1559-1568.
Engelmann, J.B. and Tamir, D., 2009. Individual differences in risk preference predict neural responses during financial decision-making. Brain research, 1290, pp. 28-51.
Frazier, P. & Yu, A. J., 2008. Sequential hypothesis testing under stochastic deadlines.
Advances in Neural Information Processing Systems, 20, pp. 465-472.
Forstmann, B. et al., 2008. Striatum and pre-SMA facilitate dicision-making under time pressure. PNAS, 105(45), pp. 17538-17542.
Forstmann, B., Ratcliff, R. & Wagenmakers, E.-J., 2016. Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Annual Review of
Psychology, Volume 67, pp. 641-666.
Groeneveld, R.A. and Meeden, G., 1984. Measuring skewness and kurtosis. The Statistician, pp. 391-399.
Hawkins, G. E. et al., 2015. Revisiting the evidence for collapsing boundaries and urgency signals in perceptual decision-making. The Journal of Neuroscience, 35(6), pp. 2476-2484.
Heitz, R. P., 2014. The speed-accuracy tradeoff: history, physiology, methodology, and behavior. Frontiers in neuroscience, 8, pp. 1-15.
Hinkley, D.V., 1975. On power transformations to symmetry. Biometrika, 62(1), pp. 101-111. Jaeger, T. F., 2008. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of memory and language, 59(4), pp. 434-446. Joanes, D.N. and Gill, C.A., 1998. Comparing measures of sample skewness and kurtosis.
Journal of the Royal Statistical Society: Series D (The Statistician), 47(1), pp. 183-189.
Luce, R. D., 1986. Response times: Their role in inferring elementary mental organization. Oxford: Oxford University Press.
Mulder, M. J., Wagenmakers, E. J., Ratcliff, R., Boekel, W., & Forstmann, B. U., 2013. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. The
Journal of Neuroscience, 32(7), pp. 2335-2343.
Peirce, J.W., 2007. PsychoPy—psychophysics software in Python. Journal of neuroscience
methods, 162(1), pp. 8-13.
Scase, M.O., Braddick, O.J. and Raymond, J.E., 1996. What is noise for the motion system?.
Simonoff, J.S., 1996. Smoothing Methods in Statistics. NYC: Springer.
Smith, P.-L. & Ratcliff, R., 2015. An Introduction to the diffusion model of decision making. In: B. Forstmann & E. Wagenmakers, eds. An introduction to model-based cognitive
neuroscience. Amsterdam: Springer.
Ratcliff, R., 1978. A theory of memory retrieval. Psychological review 85(2), pp. 59-108. Ratcliff, R. & McKoon, G., 2008. The diffusion decision model: theory and data for two-choice
decision tasks. Neural Computation, Volume 20, pp. 873-922.
Team, R. D. C., 2008. R: A language and environment for statistical computing. Vienna: R Foundation for statistical computing.
Thura, D., Beauregard-Racine, J., Fradet, C.-W. & Cisek, P., 2012. Decision-making by urgecy-gating: theoretical and experimental support. Journal of Neurophysiology, 108(11), pp. 2912-2930.
Wagenmakers, E.J., Ratcliff, R., Gomez, P. and McKoon, G., 2008. A diffusion model account of criterion shifts in the lexical decision task. Journal of Memory and
Language, 58(1), pp. 140-159.
Wickelgren, W.-A., 1977. Speed-accuracy tradeoff and information processing dynamics. Acta
Psychologica, Volume 41, pp. 67-85.
Winter, B., 2013. Linear models and linear mixed effects models in R with linguistic applications. arXiv:1308.5499. [http://arxiv.org/pdf/1308.5499.pdf]
