Testing the reliability of two different versions of the visual search task.
J.S.H. de Leeuw
University of Amsterdam
Student number: 10335978
Bachelor project Psychology
First reader: Dr. B. Van Bockstaele
Second reader: Mw. T.J. den Uyl
Under the supervision of: Prof. Dr. R. Wiers, Dr. B. Van Bockstaele, & Dr. K. Nikolaou
Words: 3405
Abstract
Anxiety disorders are the most prevalent psychological problem in many Western countries, with social anxiety being the most prevalent. In the etiology and maintenance of anxiety disorders, biases in processing threat-related information have been assigned a prominent role, especially attentional bias. Different paradigms such as the dot-probe task and the visual search task (VST) have been used to measure attentional bias but with poor or even unknown psychometrics. The purpose of this study was to measure the split-half reliability for two different versions of the VST: a feature-relevant VST (FR-VST) and a feature-irrelevant VST (FI-VST). In a VST, participants are asked to localize and identify a target stimulus among a varying number of distractors, meaning for a FR-VST finding the target, either a happy or an angry face, in an angry or happy crowd, respectively. In a FI-VST the participants are searching for a neutral target superimposed on pictures with emotional facial expressions. Results indicated that the reliability of the FR-VST was good and the FI-VST was unreliable. Future research on this topic should focus on making the tasks more reliable by for instance engaging participants more through gamification, comparing the scores with social anxiety questionnaires or using eye tracking as supporting evidence for attentional bias.
Table of contents Introduction 4 Method 8 Results 11 Discussion 13 References 17
Testing the reliability of two different versions of the visual search task.
Social anxiety refers to persistent fears of situations involving social interaction or social performance, or of situations in which there is the potential for scrutiny by others, which in the end can lead to the avoidance of social encounters (American Psychiatric Association, 2013). Social anxiety is an anxiety disorder, and according to the World Health Organization anxiety disorders are the most prevalent psychological problem in many Western countries (Kessler et al., 2007). In The Netherlands, 19.6% of the population has ever had one or more anxiety problems, with social anxiety (9.3%) being the most prevalent (Depla, Margreet, van Balkom & de Graaf, 2008). Social anxiety disorder is also a risk factor for subsequent depressive symptoms and substance abuse (Stein & Stein, 2008). Especially during adolescence, important cognitive and social-emotional developments take place, marked by increased social pressure, more intense emotional experiences, and risk-taking behaviors (Casey et al., 2010). These developments can lead to disorders like anxiety disorders, and have a harmful influence on social and academic functioning during the vulnerable period of adolescence, increasing the risk of psychiatric disorders in adulthood (Woodward & Ferguson, 2001). Given these pervasive numbers of anxiety disorders, and social anxiety in particular, comprehensive understanding and measurement of social anxiety is necessary to track down this disorder early on and increase the chances of successful treatment.
In the etiology and maintenance of anxiety disorders, biases in processing threat-related information have been assigned a prominent role. Anxious individuals, in comparison to nonanxious individuals, are assumed to be more likely to interpret a neutral or ambiguous stimulus as being threatening (interpretation bias), to more easily recall threatening events from memory (memory bias), and to show a preference to attend to threatening stimuli over
nonthreatening stimuli in their environment (attentional bias) (Van Bockstaele et al., 2014). Specifically, the attentional system of anxious individuals may be distinctively sensitive to threat related stimuli in the environment and therefore be biased in favor of those threat related stimuli (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van Ijzendoorn, 2007). Socially anxious individuals tend to show enhanced selective attention to threat cues, such as signals of social disapproval or criticism (Rapee & Heimberg, 1997). Some cognitive models of anxiety even argue that there is a mutually reinforcing relation between attentional bias toward threat and anxiety (Eysenck, 1997; Williams, Watts, MacLeod & Matthews, 1997; for a review, see Van Bockstaele et al., 2014). As such, attentional bias can be regarded as one of the key factors in social anxiety, and the valid and reliable measurement of
attentional bias is imperative.
Attentional bias for threat has been measured using different paradigms, such as the emotional Stroop task, the dot-probe task, the emotional spatial cuing task, and visual search paradigms. Evidence suggests that all these paradigms reflect the operation of attentional processes (Driver, 2001), but also that they do not all tap into the same aspects of attention (Shalev & Algom, 2000). An often used measure for social anxiety is the dot-probe task, which was developed by MacLeod, Mathews and Tata (1986). In the dot-probe task, two cues are simultaneously presented on a computer screen for a brief period of time, after which a dot is presented at the location of one of the cues. Participants are asked to respond to the location of the dot. Studies that have employed threatening and neutral cue pictures found that anxious participants have faster reaction times on congruent trials(where the dot is presented at the location of the threatening picture) than on incongruent trials (where the dot is presented at the location of a neutral picture). Often, these results are seen as evidence that support a rapid attentional capture of threat (Mathews, Mackintosh, & Fulcher, 1997). Attention allocation to threat is measured indirectly by the reaction times to the dot: an
attentional bias to threat is indicated by fast reactions to dots that replace threat words and slow reactions to dots that replace neutral words (Schmukle, 2005).
This interpretation however does seem to have some limitations (Notebaert, Crombez, Van Damme, De Houwer, & Theeuwes, 2011). First, most studies using the dot-probe task fail to differentiate between rapid capture of attention by threat and a difficulty to disengage attention from threat once detected (Theeuwes & Van der Stigchel, 2006). Second, a possible better investigation in the capture of attention by threatening information is in experimental paradigms with varying number of competing stimuli (Frischen, Eastwood, & Smilek, 2008). Another notable limitation of the dot probe task is its poor psychometric qualities. Schmukle (2005) systematically assessed the reliability of two versions of the dot probe task, a version with words as stimuli and another version with pictorial stimuli. Neither version of the task in his research turned out to be either internally consistent, nor was the test-retest reliability of the dot-probe task sufficient. These results indicate that the dot-probe task is not suitable to investigate the relationship between anxiety and attention allocation to threat in non-clinical samples. The dot-probe task was found to only measure error variance, without leading to further substantial and replicable effects. If a test measures only error variance,
interindividual differences are not substantial, because these differences are only due to measurement error therefore effects are only observed by chance (Staugaard, 2009). In a continuation on Schmukle’s research, Staugaard (2009) tested the reliability of the dot-probe task using photographs of human faces, basing his research on the fact that more recent dot-probe tasks have favored the use of faces over words, given their presumed higher ecological validity (Gilboa-Schechtman, Foa, & Amir, 1999). His findings were in line with Schmukle’s research, showing the dot-probe task with photographic faces was not found to be reliable.
Because the reliability of the dot-probe task is poor, it is imperative that other measures and even new measures of attentional bias are designed and tested for their psychometric properties. One such possible alternative for the dot probe task is the visual search task (VST). In a VST, participants are asked to localize and identify a target stimulus among a varying number of distractors. In the feature-relevant visual search task (FR-VST) the stimuli are nonconfounded standard photographs of emotional facial expressions, with the target being either a happy or an angry face in an angry or happy crowd, respectively (Juth, Lundqvist, Karlsson & Öhman, 2005). In the feature-irrelevant visual search task (FI-VST) the participants are searching for a neutral target superimposed on pictures with emotional facial expressions. In this VST, the emotional pictures are task-unrelated, background
distractors (Theeuwes, 1995).Attention to threatening information is not instrumental for this task, which makes it helpful ruling out whether participants are intentionally looking for threatening stimuli (Notebaert et al., 2011). Attention to threatening information is often considered to be an unintentional process, making the FI-VST theoretically relevant (McNally, 1995).
With the VSTs using varying numbers of competing stimuli, it is an often used
measurement for attentional bias. Both the dot-probe task and the VST tap the same aspect of attention, it reflects visual-spatial attention (Van Bockstaele et al., 2014). However,
psychometric properties of the VST have not yet been examined. Therefore, in this study we tested two different versions of a VST: the FR-VST, with a grid of 16 faces and the search for either a happy or an angry face within a grid of angry or happy faces, respectively. And the FI-VST with a grid of either happy or angry faces but the search is for an emotionally neutral target. We assessed the split half reliability of these two different versions of the VST. If reliability for the two tasks is high, we expected high correlations between attentional bias scores in a split-half reliability analysis
.
Method
Participants. A total of 61 participants (36 female, 25 male) took part in this study. Their average age was 22.2, with a range from 18 to 29 years. All participants gave informed consent to take part in the study, and the study was approved by the University of Amsterdam ethics committee. Fifty-one participants were students of the University of Amsterdam in different tracks and gained a research credit by conducting this research, while the remaining ten participants each received 10€.
Materials.
Visual Search Tasks.
Both VSTs had the same build up. Each trial in both versions began with a fixation cross presented at the centre of the computer screen. This was followed by a centrally presented 4x4 square grid that in total consisted of 16 images. For the trial to begin, and to make sure that the participant was fixated on the centre of the screen at the beginning of each trial, participants were required to click on the fixation cross. The presentation duration of the grid was indefinite for each trial, a response from the participant was needed to terminate the trial. For each trial the time to click on an image location since the start of the trial was recorded as the response time, and whether the response was a correct one.
Feature Relevant Visual Search Task (FR-VST).
In the FR-VST, participants completed 2 blocks of trials presented in a counterbalanced order. In one block, participants were instructed search for the image depicting an angry face within a grid of happy faces (negative block), while in the other, the target image was a happy face in a grid of angry faces (positive block). Both the negative block and the positive
block contained 32 trials. Attentional bias in this task was calculated by subtracting the reaction times on the negative block from the reaction times on the positive block.
Feature Irrelevant Visual Search Task (FI-VST).
In the FI-VST, participants were instructed to search for the letter “b” whilst ignoring the letters “p”, “d” and “q”, with the letters being the emotionally neutral distractor targets. These letters were presented randomly in the center of the 16 images that comprised the grid. Two conditions will be discussed here: In the zero-negative condition the grid of images
comprised of 16 happy faces; in the sixteen-negative condition the grid of images comprised of 16 angry faces (the eight-negative condition and the one-negative condition will not be discussed here). Each condition was presented in a random order in one continuous block, which in total contained 128 trials. Attentional bias in this task was calculated by subtracting the reaction times on the zero-negative block from the reaction times on the sixteen-negative block.
Photographic faces. The faces used in the FR-VST and the FI-VST were selected from the Karolinska Directed Emotional Faces (KDEF) database. This database contains 4900 pictures of human facial expressions, with particular attention paid to for instance use of uniform T-shirt colors and the use of a grid to center participants’ face during shooting (Lundqvist, Flykt & Öhman, 1998). In total, 64 pictures with centrally focused faces were selected for this study with half happy expressions and the other half angry expressions. These pictures were randomly separated into two sets, with each set containing 8 angry females faces, 8 happy female faces, 8 angry male faces and 8 happy male faces. Set 1 was used in the FR-VST and set 2 was used in the FI-VST. Each picture used was 80x80 pixels in size.
Design/procedure. The study was conducted in a computer room of the laboratory of the University of Amsterdam. All the participants were tested individually. The experimenter
explained the procedure to the participant and remained available for any possible questions during the entire procedure. First, the participants had to fill in the SPS and the FNE, the two questionnaires regarding social anxiety, followed by two questionnaires regarding alcohol consumption (which will not be discussed here), after which they completed both VSTs. The order of the two VSTs was counterbalanced across participants and stratified by gender. After the VSTs were completed, an IAT for social anxiety was conducted (which will not be
discussed here). The complete procedure took about 55 minutes.
Data preparation. Reaction times from trials with errors were excluded, and those reaction times more than three standard deviations above the mean were considered as outliers and removed. One participant was completely removed due to a procedural error during testing, leaving 60 participants to be studied. The primary variable of interest was the attentional bias score, which was calculated separately for the FR-VST and the FI-VST. For the FR-VST, the attentional bias score for each participant was calculated by calculating their mean reaction time in the negative block and the mean reaction time in the positive block. Subtracting the mean reaction time on the negative block from the mean reaction time on the positive block gave the attentional bias score. On the FI-VST , the attentional bias score for each participant was calculated by calculating their mean reaction time in the zero-negative condition and their mean reaction time in the sixteen-negative condition. The attentional bias was calculated by subtracting the mean reaction time on the zero-negative condition from the mean reaction time on the sixteen-negative condition. On both VSTs, a positive score represented an attentional bias for angry faces, a negative score meaning an avoidance for angry faces and scores around zero indicate having no attentional bias towards angry faces. For each task and participant, we created a new dummy variable which numbered the remaining trials. Based on this new variable, another new dummy variable was created to differentiate between the odd and even trials. Per task, for all the odd trials and all the even trials of each participant,
the attentional bias score was then calculated. The scores on the odd trials and even trials were then correlated for each task.
Results
For the FR-VST the correlation between the odd and even trials was high, r = .594, with a Spearman-Brown corrected rho = .75. For the FI-VST, the correlation between the two odd and the even trials was low, r = -.157. Table 1 shows the mean and standard deviation of the attentional bias score for each task and for the odd and even trials.
Table 1
Mean and Standard Deviation of the Attentional Bias Scores on the FRVST and the FIVST for the Odd and Even Trials and the Total Score
FRVST (N = 60) FIVST (N = 60)
Mean Standard Deviation Mean Standard Deviation
Total Task -181.67 673.81 4.79 478.39
Odd -196.03 731.34 -92.09 746.40
Figure 1 illustrates the correlation of the odd and even trials in the FR-VST. Each dot corresponds with a participant and as Figure 1 shows, the dots cumulate around the
regression line, indicating a high correlation.
Figure 1. The Correlation for the FR-VST
Whereas in Figure 2 the correlation of the odd and even trials in the FI-VST are shown, and the dots do not cumulate around the regression line, therefore indicating a low correlation.
Figure 2. The Correlation for the FI-VST
Discussion
The aim of this study was to assess the split-half reliability of two different versions of the VST. We expected high correlations between attentional bias scores in a split-half reliability analysis for both tasks, if the reliability was high. The split-half reliability was calculated by correlating the odd trials and the even trials (Staugaard, 2009). For the FR-VST the odd and even trials correlated significantly, so we can say with 95% certainty that the trials measured the same construct. The correlation was high, giving the FR-VST a very good internal consistency. The correlation between the odd and even trials on the FI-VST was low. The internal consistency for the FI-VST was thus poor. Therefore it can be concluded that the FR-VST could be used as an appropriate measurement of attentional bias whereas the FI-VST is as yet an inappropriate measurement.
Low (or even negative) reliability will likely compromise the task’s capacity to sensitively assess attentional bias at the individual participant level (MacLeod & Clarke, 2015). One limitation that may have caused this low internal reliability of the FI-VST could be the monotonous nature of the tasks. The total of 128 trials in the FI-VST can lead to boredom, fatigue and loss of concentration, which could lead to the participant rushing
through the trials, which in turn could lead to a low internal consistency in the attentional bias at the individual participant level. Either, the total amount of trials for the FI-VST should be brought back to 64 (as is the case with the FR-VST), or future research should try to maybe work towards “gamification” of a task, making the task more interactive and thus engaging the participants more in ways that can improve measurement for attentional bias towards threatening stimuli (MacLeod & Clarke, 2015). Future research should confirm the underlying mechanisms and whether these are applicable to the VST.
One possible way of improving the FI-VST is by using a different ratio of happy versus angry faces in the grid. The grid now consisted of either 16 happy or 16 negative faces. More research should be conducted with different ratios, for instance with a grid of eight angry and eight happy faces. If the grid of distractors contains faces that all have the same facial expression, maybe the attention is captured by all those faces at once and the grid can be seen as one major distractor. Research showed that people tend to look longer at arousing pictures than neutral pictures (Lang, Greenwald, Bradley, & Hamm, 1993) and their attention was captured by these arousing pictures (Van Bockstaele et al., 2012). This could give an explanation for the low reliability in the FI-VST, because the participant may have been distracted by the full grid of angry faces whereas the full grid of happy faces did not capture their attention. With the different ratios in the grid of distractors, the attentional bias score should indicate that when the emotionally neutral target stimulus is placed on an angry face in a grid of happy and angry faces the participant is faster in localizing that target than
when the emotionally neutral target stimulus is placed on a happy face in a grid of happy and angry faces. This should improve the FI-VST in ways that these scores reflect more on attentional bias towards threatening stimuli.
Another way of improving the FI-VST is by using different emotionally neutral target stimuli. The use of big sized letters on a picture could mean that the face is hard to recognize in the first place. By using bigger pictures and a smaller sized emotionally neutral target stimuli, the faces are more obvious. In that way, there is more certainty that the attention of the participant is actually captured by the faces .
Another point for discussion is the measuring of attentional bias via reaction time, which may be problematic. The problem lies within the fact that reaction time based
measurement has a weakness to inappropriate factors such as time to select a response or the delay in registration of a response due to inhibition in pressing the mouse button. An
alternative to measurement of reaction time is mentioned by Miloff, Savva and Carlbring (2015) by using visual and cognitive processing of threatening stimuli and the allocation of attention instead of speed of reacting. Eye tracking for instance could be used as supportive evidence for attentional bias. The movement and pattern of gaze away, from, and towards stimuli could provide extra attentional bias data, since emotional information reduces
processing efficiency in anxiety (Derakshan & Koster, 2010). Reaction times could be slower when measured by pressing the mouse button because the emotional information reduced the participant’s efficiency. When using eye tracking, it could demonstrate the difference in reaction time since the first localization of the stimuli and the actual pressing of the mouse button. If this difference is bigger for threat-related stimuli, this adds to the attentional bias data. Future studies concerning the VST may want to use eye tracking as supporting evidence for standard reaction time measurement.
Given the previous research on the psychometrics of other paradigms used to measure attentional bias, it became imperative that current measures and even new measures for attentional bias needed to be tested on their psychometrics. The reliability for the FR-VST was good, making it a reliable, robust and stable paradigm to assess attentional bias. The FI-VST was unreliable, leaving room for more research to be conducted on this topic especially given the pervasive numbers of anxiety disorders and social anxiety in specific. The
limitations of this research give way to new research and new directions with some ideas to support the reliability of the VST.
References
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC; Author. doi:
10.1176/appi.books.9780890425596.744053
Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & Van Ijzendoorn, M. H. (2007). Threat-related attentional bias in anxious and nonanxious individuals: A meta-analytic study. Psychological Bulletin, 133(1), 1-24. doi: 10.1037/0033-2909.133.1.1
Bradley, B. P., Mogg, K., & Millar, N. H. (2000). Covert and overt orienting of attention to emotional faces in anxiety. Cognition and Emotion, 14, 789–808. doi:
10.1080/02699930050156636
Bradley, B. P., Mogg, K., Millar, N., Bonham-Carter, C., Fergusson, E., Jenkins, J., & Parr, M. (1997). Attentional bias for emotional faces. Cognition and Emotion, 11, 25–42. doi: 10.1080/026999397380014
Casey, B. J., Jones, R. M., Levita, L., Libby, V., Pattwell, S. S., Rubbery, E. J.,…Somerville, L.H. (2010). The storm and stress of adolescence: insights from human imaging and mouse genetics. Developmental Psychobiology, 52, 225-235. doi: 10.1002/dev.20447
Cisler, J. M., & Koster, E. H. (2010). Mechanisms of attentional biases towards threat in anxiety disorders: An integrative review. Clinical Psychology Review, 30(2), 203-216. doi: 10.1016/j.cpr.2009.11.003
Clark, D. M., & Wells, A. (1995). A cognitive model of social phobia. In R.G. Heimberg, M.R. Liebowitz, D.A. Hope, & F.R. Schneier (Eds.), Social phobia: Diagnosis, Assessment, and Treatment (pp. 69-93). New York, NY; The Guilford Press
Collins, K. A., Westra, H. A., Dozois, D. J., & Stewart, S. H. (2005). The validity of the brief version of the Fear of Negative Evaluation Scale. Journal of Anxiety Disorders, 19(3), 345-359. doi:10.1016/j.janxdis.2004.02.003
Cox, B. J., Swinson, R. P., & Direnfeld, D. M. (1998). A comparison of social phobia
outcome measures in cognitive-behavioral group therapy. Behavior Modification, 22, 285–297. doi: 10.1177/01454455980223004
Depla, M. F., Margreet, L., van Balkom, A. J., & de Graaf, R. (2008). Specific fears and phobias in the general population: results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Social Psychiatry and Psychiatric
Epidemiology, 43(3), 200-208. doi: 10.1007/s00127-007-0291-z
Derakshan, N., & Koster, E. H. (2010). Processing efficiency in anxiety: Evidence from eye-movements during visual search. Behaviour Research and Therapy, 48(12),
1180-1185. doi: 10.1016/j.brat.2010.08.009
Driver, J. (2001). A selective review of selective attention research from the past
century. British Journal of Psychology, 92, 53–78. doi: 10.1348/000712601162103
Eysenck, M. W. (1997). Anxiety and cognition: A unified theory. Hove, England: Psychology Press.
Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007). Anxiety and cognitive performance: attentional control theory. Emotion, 7(2), 336-353. doi: 10.1037/1528-3542.7.2.336
Frischen, A., Eastwood, J. D., & Smilek, D. (2008). Visual search for faces with emotional expressions. Psychological Bulletin, 134, 662–676. doi:
Gamer, J., Schmukle, S. C., Luka-Krausgrill, U., & Egloff, B. (2008). Examining the
dynamics of the implicit and the explicit self-concept in social anxiety: changes in the implicit association test–anxiety and the social phobia anxiety inventory following treatment. Journal of Personality Assessment, 90(5), 476-480. doi:
10.1080/00223890802248786
Gilboa-Schechtman, E., Foa, E. B., & Amir, N. (1999). Attentional biases for facial expressions in social phobia: The Face-in-the-Crowd paradigm. Cognition & Emotion, 13(3), 305-318. doi: 10.1080/026999399379294
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual
differences in implicit cognitions: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464-1480. doi: 10.1037/0022-3514.74.6.1464
Heimberg, R. G., Mueller, G. P., Holt, C. S., Hope, D. A., & Liebowitz, M. R. (1993). Assessment of anxiety in social interaction and being observed by others: The Social Interaction Anxiety Scale and the Social Phobia Scale. Behavior Therapy, 23(1), 53-73. doi: 10.1016/S0005-7894(05)80308-9
Juth, P., Lundqvist, D., Karlsson, A., & Öhman, A. (2005). Looking for foes and friends: Perceptual and emotional factors when finding a face in the crowd. Emotion, 5(4), 379. doi: 10.1037/1528-3542.5.4.379
Kessler, R. C., Angermeyer, M., Anthony, J. C., De Graaf, R., Demyttenaere, K., Gasquet, I.,... Uestuen, T. B. (2007). Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry, 6(3), 168-176.
Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions.Psychophysiology, 30(3), 261-273. doi: 10.1111/j.1469-8986.1993.tb03352.x
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces - KDEF, CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, ISBN 91-630-7164-9.
Macleod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal of Abnormal Psychology, 95, 15–20. doi: 10.1037/0021-843X.95.1.15
MacLeod, C., & Clarke, P. J. (2015). The Attentional Bias Modification Approach to Anxiety Intervention. Clinical Psychological Science, 3(1), 58-78. doi:
10.1177/2167702614560749
Mathews, A., Mackintosh, B., & Fulcher, E. P. (1997). Cognitive biases in anxiety and attention to threat. Trends in Cognitive Sciences, 1(9), 340-345. doi: :10.1016/S1364-6613(97)01092-9
Mattick, R. P., & Peters, L. (1988). Treatment of severe social phobia: effects of guided exposure with and without cognitive restructuring. Journal of Consulting and Clinical Psychology, 56(2), 251-260. doi: 10.1037/0022-006X.56.2.251
Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour Research and Therapy, 36(4), 455-470. doi:10.1016/S0005-7967(97)10031-6
McNally, R. J. (1995). Automaticity and the anxiety disorders. Behaviour Research and Therapy, 33, 747–754. doi: 10.1016/0005-7967(95)00015-P
Miloff, A., Savva, A., & Carlbring, P. (2015). Cognitive Bias Measurement and Social Anxiety Disorder: Correlating Self-Report Data and Attentional Bias. Internet
Interventions. doi: 10.1016/j.invent.2015.03.006
Mogg, K., McNamara, J., Powys, M., Rawlinson, H., Seiffer, A., & Bradley, B. P. (2000b). Selective attention to threat: A test of two cognitive models of anxiety. Cognition & Emotion, 14, 375–399. doi: 10.1080/026999300378888
Notebaert, L., Crombez, G., Van Damme, S., De Houwer, J., & Theeuwes, J. (2011). Signals of threat do not capture, but prioritize, attention: A conditioning
approach. Emotion, 11(1), 81-89. doi: 10.1037/a0021286
Öhman, A., Flykt, A., & Esteves, F. (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology: General, 130(3), 466-478. doi: 10.1037/0096-3445.130.3.466
Pergamin-Hight, L., Naim, R., Bakermans-Kranenburg, M. J., van IJzendoorn, M. H., & Bar-Haim, Y. (2015). Content specificity of attention bias to threat in anxiety disorders: A meta-analysis. Clinical Psychology Review, 35, 10-18. doi: 10.1016/j.cpr.2014.10.005
Price, R. B., Kuckertz, J. M., Siegle, G. J., Ladouceur, C. D., Silk, J. S., Ryan, N. D., ... & Amir, N. (2014). Empirical recommendations for improving the stability of the dot-probe task in clinical research. Psychological Assessment, 27(1). Published ahead of
Rapee, R. M., & Heimberg, R. G. (1997). A cognitive-behavioral model of anxiety in social phobia. Behaviour Research and Therapy, 35(8), 741-756. doi: 10.1016/S0005-7967(97)00022-3
Schmukle, S. C. (2005). Unreliability of the dot probe task. European Journal of Personality, 19(7), 595-605. doi: 10.1002/per.554
Shalev, L., & Algom, D. (2000). Stroop and Garner effects in and out of Posner's beam: Reconciling two conceptions of selective attention. Journal of Experimental Psychology: Human Perception and Performance, 26(3), 997–1017. doi: 10.1037/0096-1523.26.3.997
Staugaard, S. R. (2009). Reliability of two versions of the dot-probe task using photographic faces. Psychology Science Quarterly, 51(3), 339-350.
Stein, M. B., & Stein, D. J. (2008). Social anxiety disorder. The Lancet, 371(9618), 1115-1125. doi: 10.1016/S0140-6736(08)60488-2
Theeuwes, J. (1995). Abrupt luminance change pops out - Abrupt color-change does not. Perception & Psychophysics, 57, 637–644. doi: 10.3758/BF03213269
Theeuwes, J., & Van der Stigchel, S. (2006). Faces capture attention: Evidence from inhibition of return. Visual Cognition, 13, 657–665. doi:
10.1080/13506280500410949
Van Bockstaele, B., Koster, E. H., Verschuere, B., Crombez, G., & De Houwer, J. (2012). Limited transfer of threat bias following attentional retraining.Journal of Behavior
Therapy and Experimental Psychiatry, 43(2), 794-800. doi:
Van Bockstaele, B., Verschuere, B., Tibboel, H., De Houwer, J., Crombez, G., & Koster, E. H. W. (2014). A review of current evidence for the causal impact of attentional bias on fear and anxiety. Psychological Bulletin, 140(3), 682-721. doi: 10.1037/a0034834
Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33(4), 448-457.doi: 10.1037/h0027806
Williams, J. M. G., Watts, F. N., MacLeod, C., & Mathews, A. (1997).Cognitive Psychology and Emotional Disorders (2nd ed.). Chichester,England: Wiley.
Woodward, L. J., & Fergusson, D. M. (2001). Life course outcomes of young people with anxiety disorders in adolescence. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 1086-1093. doi: 10.1097/00004583-200109000-00018
