Unconscious Interference on Visual Short-Term Memory
Probing the influence of invisible distracting information on a delayed orientation discrimination task
Abstract
Visual Short-Term Memory (VSTM) or Working Memory (WM) is considered to be closely related to consciousness. However, recent empirical evidence coming from delayed orientation discrimination tasks with invisible distractor interference suggests a new view on their relationship: WM can be modulated without conscious awareness. This study was aimed to replicate original findings and to add an objective definition of unconscious interference: an objectively invisible distractor, using signal detection theory. Results show that the distractor could be rendered objectively invisible, but no evidence for any influence of the distractor, visible or invisible, on VSTM performance was found. Although the result might suggest otherwise, no clear evidence against original findings can be presented because even participants who indicated they could see the distractor, performed as well as without distractor. However, if distractor interference effects do exist, they might be much smaller than previously assumed.
Name Student Number Course Supervisor Word Count Date Fried Schölvinck 10729267 Bachelor Project Brain & Cognition dhr. dr. T. (Timo) Stein 2819 30-05-2018
Introduction
The problem of consciousness is widely researched in psychology and cognitive neuroscience. An often closely related concept to consciousness is working memory (WM). WM is the store for cognitive information that is relevant for a few seconds. It operates as a control system; maintaining, updating and manipulating information that is used in the service of current tasks. Visual short-term memory (VSTM) stores visual information and is part of the overall WM.
Consciousness or conscious awareness is what we typically explain as the ability to experience and as something we can report. WM and especially VSTM have been considered closely related to consciousness. The relationship between WM and consciousness can be described in a few ways.
The first WM model, proposed by Atkinson and Shiffrin (1971, 1968) was a simple model, where everything in WM is implicitly thought of as conscious. There is no explicit role of consciousness, but every process or item in their model is conscious by assumption.
Another model holds the assumption that WM is equal to consciousness, the multicomponent WM model from Baddeley (2003). In that model, all information in WM is conscious and WM only operates on conscious information. In the global workspace theory (Baars, 2003; Dehaene, 2006) WM provides a workspace for consciousness. The information in the global workspace is accurately reportable and therefor conscious.
Lamme (2003) and Block (1995) researched visual awareness up to a point where consciousness was divided into attended and unattended information, dependent on recurrent interactions (Lamme, 2003). The unattended information is ‘phenomenally conscious’, meaning that it is fragile visual short-term memory, which is not held in WM. It is not controllable or reportable, but it is conscious. Therefore, conscious experience overflows conscious access, meaning that there can be consciousness outside of WM.
The other way around, consciousness can be thought of as a part of WM. The action-based model of Cowan (1999) and later Oberaurer (2002) states that something should be in the focus of attention in order to be conscious. This means that not all information in WM is conscious, but has to be attended to ‘use’ it.
Unconscious encoding
This leads to the suggestion that consciousness is actually not that related to WM as previously assumed. If consciousness is not equal to WM, could it then also exist apart from WM or could WM even operate outside of visual awareness? Recently, empirical evidence
has been found, challenging the way consciousness and WM were previously linked. This evidence comes from studies with unconscious stimuli that are somehow being maintained in WM (Bergström & Eriksson, 2014, 2015; Dutta et al., 2014; Soto et al., 2011). It showed that participants had above chance performance on an orientation discrimination task, despite not being aware of the memory cue. For example, participants were able to tell whether a memory probe was tilted clockwise or counter wise relative to the memory cue, even though they indicated they hadn’t seen the cue (Soto et al., 2011). Dutta and colleagues (2014) also studied the neuronal correlates of this unconscious information and found that the left dorso-lateral PFC was activated when participants were not aware of the stimulus. This activation correlated with the task performance, suggesting that the left-dlPFC plays a role in unconscious storage. The left-dlPFC is often associated with WM (Barbey, Koenigs & Grafman, 2013; Goldman-Rakic, 1995), so WM might play a part in unconscious storage. Unconscious interference
More evidence comes from studies where unconscious stimuli can actually interfere with WM (Silvanto & Soto, 2012; Bona et al., 2013), a phenomenon which will be discussed in detail over the next paragraphs. These are studies about the relationship between the subjective experience of visual WM representation and its objective accuracy.
The research of Silvanto & Soto and Bona et al. forms the basis for our study. There, participants were asked to maintain the visible orientation of a memory cue, after which a memory probe was shown and a delayed orientation discrimination task followed. An invisible (13 – 17 ms) distractor was shown during the delay period. The orientation of the distractor was either congruent or incongruent with the memory cue, causing interference with WM in the task. The participant had to indicate whether the memory probe was tilted to the left or the right in comparison to the original cue. After this, the participant reported the visibility of the distractor using a subjective awareness measure (perceptual awareness scale (PAS): Overgaard et al, 2010; Sandberg et al, 2010). The results of these studies show a significant decrease in WM performance when an incongruent distractor was presented for a short 13 – 17 ms, even when participants indicated they didn’t see anything.
To summarize, a review paper (Soto & Silvanto, 2014) on this topic provides an overview on the overarching research question about the relationship between WM and consciousness. It states that the content in WM and what we are aware of are two different things. Not all WM content is open to introspection.
Subjective awareness
The subjective awareness measure (PAS), used by Silvanto & Soto and Bona et al. is sensitive to the influence of individual decisional criteria. This is a response bias, meaning that participants could systematically indicate less visible distractors as invisible (Schmidt, 2015). This results in some cases where answer ‘1’, meaning ‘I didn’t see the distractor’, would imply that weakly conscious information could have been maintained in WM and could have affected performance. In other words, subjective threshold effects can always alternatively be explained by weak conscious perception (Snodgrass, 2004).
In our study, we check whether those effects can indeed be modulated by participants’ individual criteria for reporting distractor presence. So, on top of trying to detect the same effect as previous studies mentioned above, this study aims to measure the objective sensitivity to the distractor, meaning the participants’ ability to distinguish between distractor absence vs. presence. In order to do so, a new condition is created following an extra analysis on the distractor visibility ratings, such as in signal detection theory where the proportion of hits and misses is taken into account, resulting in a sensitivity index or d’ (Stanislaw & Todorov, 2014)
Here, we directly compare interference from distractors rendered invisible according
to both the subjective and objective definitions. This is important because it might suggest that visual awareness is not necessary to assume in preciously discussed WM models. The mask used in Silvanto & Soto (2012) is probably not effective enough to render the distractor completely invisible. Therefore, this study ads a second, more effective mask to make the distractor less visible.
To summarise
To collect more empirical evidence against the one-to-one mapping of WM and consciousness, this study aims to replicate the results of previous studies which showed that subjectively invisible distractors can interfere with visual short-term memory (VSTM). However, because subjective measures can be influenced by decisional criteria, distractors might have been weakly conscious rather than unconscious. Thus, this study also tests whether interference is limited to distractors that are presented below this subjective threshold, or whether similar interference can be found even when distractors are rendered objectively invisible.
If unconscious interference in WM truly exists, results in both the subjective and objective conditions must show a significant decrease in VSTM performance on the
discrimination task, when an incongruent distractor was presented. If results from previous studies might have shown weakly conscious perception instead of real unconscious WM interference, the results in the objective threshold condition will show no significant decrease in VSTM performance with an objectively invisible distractor.
Methods Participants
33 participants took part in the experiment, of whom 2 were left out due to poor performance. Data of 31 participants (7 male, Mage = 21) was analysed eventually. For a
medium effect size of 0.5 and 80% power, 34 participants were required. 31 participants leads to a power of 76.9%. Participants were recruited via the UvA Online Lab Environment and received participation credits.
Materials
This experiment was taken in a psychology lab (dimmed light) at the University of Amsterdam. Stimuli were presented on a 1920 x 1080 monitor with a 60 Hz refresh rate. All tasks run on Psychtoolbox in MatLab. The dependent variable in this study is the performance accuracy on the delayed orientation discrimination task, reflecting the ability to tell whether the memory probe was tilted to the right or to the left relative to the memory cue. There are two independent variables; the subjective vs. objective threshold and the distractor type (absent, congruent, incongruent).
Experimental Procedure
Participants were clearly instructed in this study. There was no cover-up story and they were given some practice trials before start in order to get used to the experiment and the different masks.
The experiment itself entailed 480 trials with a pause of at least 20 seconds after every 120 trials. This took about just over an hour per participant. The memory cue had 10 different possible orientations: -10/+10, -20/+20, -30/+30, -40/+40, -50/+50 degrees. The memory probe was tilted either 10 degrees to the left or to the right, compared to the cue. This was randomized. The orientation was a circular Gabor stimulus of 166 pixels width and height with 4 cycles with maximum contrast at 10%. In 2/3 of the trials a distractor was present, which was either congruent (50%) or incongruent (50%) with the memory cue (160 trials
each). To render the distractor completely invisible, a circular mask (as seen in Fig 1) was used on 50% of the trials.
Each trial began with a black fixation point in the middle of the screen for 500 ms, after which the trial followed as seen in Fig 1. VSTM performance was measured using the ‘left’ and ‘right’ arrows. Distractor visibility was measured using a 4-point perceptual awareness scale (PAS), adapted from Overgaard et al (2010) and Sandberg et al (2010) (1 = did not see the distractor, 2 = maybe saw something, 3 = saw the distractor but not its orientation, 4 = saw the distractor and its orientation). There was no time limit.
Fig 1. Trial structure: example trials with black (left) and circular (right) mask. Before the memory cue, a blank screen with a fixation point
was presented for 500 ms. Cues were presented for 300 ms, the distractor for 17 ms. The questions had no time limit.
Results
The data of 31 participants (7 male, Mage = 21, SD = 1.73) was analysed in two steps.
First, a manipulation check was done to check if the distractor was indeed rendered invisible in the objective threshold (circular mask) condition. Second, performance on the main task was analysed using repeated measures ANOVA’s.
Distractor visibility
To check whether the mask manipulation was effective, visibility ratings were analysed. Response to the Perceptual Awareness Scale (PAS) is shown in Fig 2. Proportions of PAS = 1 presses (“Did not see the distractor”) differed between mask conditions within both distractor absent, t(30) = -2.869, p = .007, as in distractor present, t(30) = -3.869, p < .001.
Fig 2. Proportion of responses to the perceptual awareness scale (PAS) in the subjective threshold condition (left) and objective threshold
condition (right).
To measure objective sensitivity, a sensitivity index or d’ is used, derived from signal detection theory. A hit was defined as PAS = 2, 3 or 4 when the distractor was actually present. A false alarm was defined as PAS = 2, 3 or 4 when the distractor absent. A d’ of zero would mean that the participant could not differentiate between distractor present or absent. In the black mask condition participants could discriminate between distractor present or absent (d’ = 0.51, p < .001). In the circular mask condition participants could not differentiate (d’ = -0,02, p = .702). There was also a significant difference between both sensitivity indexes (t(30) = 4.424, p < .001). This all means that the manipulation of distractor visibility was effective, as the circular mask rendered the distractor invisible. The black uniform mask made the distractor more visible than the circular mask, meaning that sometimes the distractor was subjectively invisible, but objectively detectable. With the circular grating mask, the distractor was not objectively detectable with a d’ of 0.
VSTM Performance
VSTM performance was analysed, looking at the proportion of accurate performance across all trials (independent on PAS-presses), depending on two mask and three distractor conditions (Fig 3). Analysis of variance showed no congruency effect within the mask condition, F(1,30) = 0.352, p = .557, ηp2 = .012, as within the distractor condition, F(2,30) =
1.192, p = .311, ηp2 = .038. No interaction effect was found, F(2,30) = .207, p = .814, ηp2 =
Fig 3. VSTM performance on all trials for the subjective (black) and objective (circular) threshold conditions. No effects were found within
or between conditions. No interaction effect was found.
To follow up on the studies of Silvanto & Soto (2012) and Bona et al. (2013), data from only the subjective threshold condition was analysed. A pairwise comparison between the congruent and incongruent distractor types for just the invisible trials showed no significant decrease in VSTM performance, t(30) = -0.147, p = .884. See Fig 4 (left).
Fig 4. VSTM performance in the subjective threshold condition. Left: accuracy on trials where participants pressed 1: “Did not see the
distractor”, showing no distractor interference. Right: accuracy on both visible and invisible trials, showing a slight but not significant decrease in memory performance.
Fig 4 (right) also shows visible trials in the subjective threshold condition. No significant decrease in VSTM accuracy was found, t(24) = .789, p = .438. Some participants were left out of this analysis because they had no visible trials.
A power analysis was done to calculate the likelihood of finding an effect. To detect the effect Silvanto & Soto (Cohen’s d = 1.61) and Bona et al. (Cohen’s d = 1.11) found, we had 99.9% power.
Discussion
Results from the main analysis in both the subjective and objective threshold condition showed no significant decrease in VSTM performance. Depending on any kind of distractor, congruent, incongruent, visible or invisible or absent, VSTM accuracy did not change. The only significant difference in this study was found in the comparison of d’ between conditions and relative to 0, which showed that the manipulation was successful and the circular mask rendered the distractor invisible to an objective definition.
This study failed to find similar results as Silvanto and Soto (2012) and Bona et al. (2013) did. The goal of this study, trying to detect those findings in an objective threshold condition, was also not achieved. However, this does not necessarily have to be evidence against invisible distractor interference, because no interference of any kind was found. This contradicts previous findings on participants’ impaired ability to maintain a memory cue when a distractor is shown in a delay period (Magnussen et al., 1991, 2003, 2009). In our case, even when participants indicated they could ‘see’ the distractor, it had no effect on memory performance. This might be due to the amount of data we had in the ‘visible’ condition, as most participants only pressed 2, 3 or 4 a few times or not at all. Thus, there is not much to say on whether the effects of Silvanto & Soto and Bona et al. are modulated by participants’ individual criteria for reporting distractor presence. Future studies need to collect more data on participants’ discrimination abilities, keeping d’ as significantly close to 0 as possible. On top of the decrease in memory performance, one could aspect that accuracy might rise when a congruent distractor is presented. Although this effect had not been found in this study or previous studies, it would make sense to say that distractor interference could also have a positive effect on VSTM performance if the distractor is congruent with the memory cue. It would be interesting to see what future research finds on this question.
Besides statistical issues, one could argue that methodological problems also play a role in the outcome of this study. For example, within the delayed orientation discrimination
task it is difficult to focus on both the main task and detecting the distractor. It could be that participants differed in behaviour, leading to focus on either the main discrimination task or detection of the distractor. Although participants were instructed to do both, they could easily lose their attention to the bright screen in a dark room for over an hour. Furthermore, it might also be motivationally beneficial to include any feedback or reward in the task. During the 480 trials and even after, participants could not track their performance. Of course, this might also influence performance, but it could also reduce guessing.
With the data of this study there is unfortunately not much to say on relationship between consciousness and working memory. Further exploration on unconscious interference effects from objectively invisible distractors is necessary to tell whether previous results were due to participants’ individual decisional criteria.
References
Baars, B. J. (1997). Some Essential Differences between Consciousness and Attention, Perception, and Working Memory. Consciousness and Cognition, 6(2–3), 363–371. https://doi.org/10.1006/ccog.1997.0307
Baars, B. J., & Franklin, S. (2003). How conscious experience and working memory interact.
Trends in Cognitive Sciences, 7(4), 166–172.
https://doi.org/10.1016/S1364-6613(03)00056-1
Baddeley, A. (2003). Working memory: Looking back and looking forward. Nature Reviews
Neuroscience, 4(10), 829–839. https://doi.org/10.1038/nrn1201
Barbey, A. K., Koenigs, M., & Grafman, J. (2013). Dorsolateral prefrontal contributions to
human working memory. Cortex, 49(5), 1195–1205.
https://doi.org/10.1016/j.cortex.2012.05.022
Bergström, F., & Eriksson, J. (2014). Maintenance of non-consciously presented information engages the prefrontal cortex. Frontiers in Human Neuroscience,
8(November), 1–10. https://doi.org/10.3389/fnhum.2014.00938
Bergström, F., & Eriksson, J. (2015). The conjunction of non-consciously perceived object identity and spatial position can be retained during a visual short-term memory task.
Frontiers in Psychology, 6(September), 1–9. https://doi.org/10.3389/fpsyg.2015.01470
Bona, S., Cattaneo, Z., Vecchi, T., Soto, D., & Silvanto, J. (2013). Metacognition of visual short-term memory: Dissociation between objective and subjective components of
VSTM. Frontiers in Psychology, 4(FEB), 1–6.
https://doi.org/10.3389/fpsyg.2013.00062
Dehaene, S., & Changeux, J. P. (2011). Experimental and Theoretical Approaches to
Conscious Processing. Neuron, 70(2), 200–227.
https://doi.org/10.1016/j.neuron.2011.03.018
Dehaene, S., Changeux, J. P., Naccache, L., Sackur, J., & Sergent, C. (2006). Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends in Cognitive
Sciences, 10(5), 204–211. https://doi.org/10.1016/j.tics.2006.03.007
Dutta, A., Shah, K., Silvanto, J., & Soto, D. (2014). Neural basis of non-conscious visual
working memory. NeuroImage, 91, 336–343.
https://doi.org/10.1016/j.neuroimage.2014.01.016
Goldman-Rakic, P. S. (1995). Cellular basis of working memory. Neuron, 14(3), 477–485. https://doi.org/10.1016/0896-6273(95)90304-6
Holender, D., & Duscherer, K. (2004). Unconscious perception: The need for a paradigm
shift. Perception and Psychophysics, 66(5), 872–881.
https://doi.org/10.3758/BF03194980
Kanai, R., Walsh, V., & Tseng, C. H. (2010). Subjective discriminability of invisibility: A framework for distinguishing perceptual and attentional failures of awareness.
Consciousness and Cognition, 19(4), 1045–1057. https://doi.org/10.1016/j.concog.2010.06.003
Kouider, S., & Dehaene, S. (2007). Levels of processing during non-conscious perception: a critical review of visual masking. Philosophical Transactions of the Royal Society B:
Biological Sciences, 362(1481), 857–875. https://doi.org/10.1098/rstb.2007.2093
Kunimoto, C., Miller, J., & Pashler, H. (2001). Confidence and accuracy of near-threshold discrimination responses. Consciousness and Cognition, 10(3), 294–340. https://doi.org/10.1006/ccog.2000.0494
Lamme, V. A. F. (2003). Why visual attention and awareness are different. Trends in
Cognitive Sciences, 7(1), 12–18. https://doi.org/10.1016/S1364-6613(02)00013-X
Magnussen, S. (2009). Implicit visual working memory. Scandinavian Journal of Psychology,
50(6), 535–542. https://doi.org/10.1111/j.1467-9450.2009.00783.x
Magnussen, S., Greenlee, M. W., Aslaksen, P. M., & Kildebo, O. Ø. (2003). High-fidelity perceptual long-term memory revisited - And confirmed. Psychological Science,
14(1), 74–76. https://doi.org/10.1111/1467-9280.01421
Magnussen, S., Greenlee, M. W., Asplund, R., & Dyrnes, S. (1991). Stimulus-specific mechanisms of visual short-term memory. Vision Research, 31(7–8), 1213–1219. https://doi.org/10.1016/0042-6989(91)90046-8
Overgaard, M., Timmermans, B., Sandberg, K., & Cleeremans, A. (2010). Optimizing subjective measures of consciousness. Consciousness and Cognition, 19(2), 682–684. https://doi.org/10.1016/j.concog.2009.12.018
Rademaker, R. L., Weerd, P. De, & Sack, A. T. (2015). The Impact of Interference on Short-Term Memory for Visual Orientation, 41(6), 1650–1665.
Schmidt, T. (2007). Measuring unconscious cognition: Beyond the zero-awareness criterion.
Advances in Cognitive Psychology, 3(1–2), 275–287.
https://doi.org/10.2478/v10053-008-0030-3
Schmidt, T. (2015). Invisible stimuli, implicit thresholds: Why invisibility judgments cannot be interpreted in isolation. Advances in Cognitive Psychology, 11(2), 31–41. https://doi.org/10.5709/acp-0169-3
Shanks, D. R. (2017). Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processes. Psychonomic Bulletin and Review, 24(3), 752–775. https://doi.org/10.3758/s13423-016-1170-y
Silvanto, J., & Soto, D. (2012). Causal evidence for subliminal percept-to-memory
interference in early visual cortex. NeuroImage, 59(1), 840–845.
https://doi.org/10.1016/j.neuroimage.2011.07.062
Snodgrass, M., Bernat, E., & Shevrin, H. (2004). Unconscious perception: A model-based approach to method and evidence. Perception and Psychophysics, 66(5), 846–867. https://doi.org/10.3758/BF03194978
Soto, D., & Silvanto, J. (2014). Reappraising the relationship between working memory and conscious awareness. Trends in Cognitive Sciences, 18(10), 520–525. https://doi.org/10.1016/j.tics.2014.06.005
Soto, D., Mäntylä, T., & Silvanto, J. (2011). Working memory without consciousness.
Current Biology, 21(22), R912–R913. https://doi.org/10.1016/j.cub.2011.09.049
Stanislaw, H., & Todorov, N. (1999). Calculating of signal detection theory measures.
Behavior Research Methods, Instruments, & Computers, 31(1), 137–149.
https://doi.org/10.3758/BF03207704
Stein, T., Kaiser, D., & Hesselmann, G. (2016). Can working memory be non-conscious?
Neuroscience of Consciousness, 2016(1), niv011. https://doi.org/10.1093/nc/niv011
Velichkovsky, B. B. (2017). Consciousness and working memory: Current trends and research perspectives. Consciousness and Cognition, 55(April), 35–45. https://doi.org/10.1016/j.concog.2017.07.005
