Unconscious Processing and Neural Dynamics of Subjectively and Objectively Invisible Images Christina Bruckmann
University of Amsterdam
Student Number: 12282162
Project: Research Project 1 (Research Master Brain and Cognitive Sciences) Supervisor: Dr. Timo Stein
Second Assessor: Dr. Simon van Gaal Year: 2019/2020
Abstract
Remarkable findings regarding the unconscious processing of information such as blindsight or the effects of priming have led to extensive research into the processing mechanisms of
unconscious compared to conscious information in the brain. However, previous efforts to determine whether unseen images are subject to higher-level processing have been inconclusive. The aim of the current study was to address the discrepancies found across previous studies regarding the neural processing of unconscious information and to determine whether a
subjective or objective measure of conscious perception is preferable. To this end, we conducted two separate experiments; an EEG study to investigate the neural dynamics of house and face stimuli categorized as invisible through different measures, and a behavioral experiment.
Category information of invisible images was decodable from EEG data only when a subjective measure of conscious perception was used; when performance was at chance-levels, category information could not be decoded. The present results suggest that the inconclusive results of previous studies might be related to methodological differences in the assessment of conscious perception and that invisible images do not reach higher processing stages when a performance- based measure of visibility is employed. However, the results of an additional behavioral
experiment testing the role of context manipulation on the different measures did not allow us to draw any clear conclusions about whether a subjective or an objective threshold of awareness is preferable.
Unconscious Processing and Neural Dynamics of Subjectively and Objectively Invisible Images
Remarkable findings regarding the unconscious processing of information such as blindsight (see Stoerig & Cowey, 2007) or the effects of priming (see Elgendi et al., 2018) have led to extensive research into the processing mechanisms of unconscious compared to conscious information in the brain. One of the main questions of interest is to which extent information which does not reach consciousness undergoes neural processing. While it appears to be the case that unseen images enter low-level processing stages and basic features can be extracted (e.g. Salti et al., 2015), it is not clear as of yet to which degree unconscious information is processed at higher stages (Dubois & Faivre, 2014; Hesselmann, 2013; Lamme, 2020; Sterzer et al., 2014; van Gaal & Lamme, 2012). Although clearly defining what constitutes a higher-level process is not straightforward (Groen et al., 2017), some of the perceptual processes that have been the focus include the neural processing of category membership, emotions, or faces.
A commonly used methodological approach to uncover the extent of unconscious processing has been based on findings from conscious processing; it has been demonstrated repeatedly that visual stimuli elicit distinct neural activity based on their conceptual category when consciously perceived (e.g. fMRI: Downing et al., 2005; Kaiser et al., 2016; Spiridon et al., 2005; EEG: Coggan et al., 2016; Rousselet et al., 2007; Simanova et al., 2010). For example, neural activity associated with the conscious visual perception of a face is uniquely different from that elicited by conscious perception of other objects such as houses or tools (e.g. fMRI: Kanwisher & Yovel, 2006; EEG: Rossion et al., 2003). Based on these findings, several studies have attempted to assess whether visual stimuli which are not consciously perceived exhibit similar neural distinctiveness depending on their conceptual category and can thus be considered subject to category-specific, higher-level processing.
Looking at spatial patterns of neural activity evoked by unseen images, results from fMRI studies have overall been conflicting. Some studies reported category-specific activity for
unconsciously perceived images of faces and houses respectively (Fang & He, 2005, Hesselmann et al., 2011, Schurger et al., 2009), whereas others have not found such distinct activity
(Rodríguez et al. 2012). Sterzer, Haynes, and Rees (2008) reported no significant category-specific activity for indivisible images with standard fMRI analysis but could discriminate above chance-levels between categories when using multivariate pattern analysis (MVPA).
Similar studies have been conducted on the temporal signature of category information using EEG, although those are few. Jiang and colleagues (2009) investigated the neural response to visible and invisible emotional faces compared to scrambled faces. They reported that
invisible fearful faces were associated with a larger negative deflection in the event-related potential around 220ms post stimulus onset than invisible neutral faces, suggesting unconscious affective processing. On the other hand, Rodríguez and colleagues (2012) did not report any decodable category-specific activity for invisible faces in their EEG study. Moreover, Kaunitz and colleagues (2011) found no category-specific EEG-signature for objectively invisible animal and tool images, whereas rapid neural categorization was detected for visible images. Notably, they reported no category processing for invisible stimuli even when using the more sensitive analysis method of MVPA. Using MEG however, Sterzer and colleagues (2009) detected attenuated, but still category-specific neural signatures for faces rendered invisible through continuous flash suppression compared to invisible houses. Similarly, Suzuki and Nogushi (2013) used continuous flash suppression and luminance manipulation to present invisible upright and inverted face stimuli to participants. Using a performance-based measure of consciousness, they reported a larger N170 response to invisible upright faces compared to inverted faces, suggesting that even when objectively invisible, face stimuli can elicit
category-specific neural activity. Thus, as illustrated above, current results from unconscious processing studies have been inconclusive.
Methodological Inconsistencies
Complicating the evaluation of conflicting results across both fMRI and EEG studies is the methodological diversity in assessments of conscious perception, as well as in the neural decoding method used. While some studies sort trials post-hoc into visible and invisible trials based on the subjective report of participants (e.g. Rodríguez et al., 2012), other studies employ an objective, performance-based measure (e.g. Fang & He, 2005; Jiang et al., 2009, Sterzer et al., 2009). Objective visibility was often determined in a separate experiment in these studies (e.g. Jiang et al., 2009; Kaunitz et al., 2013), sometimes with only a subset of participants.
Assessing the latent threshold of conscious perception in a rigorous scientific manner has been notoriously challenging (Balsdon & Clifford, 2018; Lamme, 2020; Seth et al., 2008; Sterzer et al., 2014).Two approaches have been commonly used, although neither of them is without caveats. Subjective measures of conscious perception are based on participants’ introspective reports of whether they have perceived a stimulus, occasionally including post-decision wagering or confidence ratings (Persaud et al., 2007; Peters et al., 2015). Objective measures are
performance-based; a stimulus is considered ‘visible’ if the participant can classify its category with above-chance accuracy. That subjective and objective measures of awareness can lead to different results has been most strikingly illustrated in cases of blindsight patients, who report being completely blind through subjective measures, but perform above chance-level in a variety of visual tasks (Stoerig & Cowey, 2007).
Whether an objective or a subjective measure of conscious perception provides greater validity however, is a question of much debate (Peters et al., 2017). Whereas the objective
method runs the risk of overestimating the extent of conscious perception, the subjective report can be impacted by biases and criterion-shifts unrelated to stimulus perception (Persuh, 2018; Schmidt, 2015). Specifically, correct classification of a stimulus which is reported as unseen through subjective measures could either reflect true unconscious higher-level processing, or reflect the participant’s conservative bias, who reports not having seen a stimulus despite having a very faint perceptual experience (Schmidt, 2015). One method to potentially determine whether an objective or a subjective threshold of conscious perception more accurately reflects the latent, ‘true’ threshold is the employment of signal detection theory in a context-manipulation
paradigm. Based on the assumption that the latent threshold remains stable as long as stimulus properties remain the same (Pylyshyn, 1999), a valid measure of conscious perception should be immune to context manipulations. As a signal detection approach allows for disentangling performance (sensitivity, d’) and bias (criterion, c) during behavioral classification and detection tasks, the influence of context manipulations on each of those measures can be assessed
simultaneously in a single task. With the bias-free sensitivity representing an objective,
performance-based threshold and the criterion the subjective, reported threshold, should one of these measures be robust against context manipulations, this measure could be considered a more accurate reflection of the ‘true’ threshold.
Beyond the measure of conscious perception, another methodological inconsistency found across unconscious decoding studies is the method which is used to decode information from neural data. Some studies focus on standard fMRI and EEG analyses, such as the general linear model and event related potentials respectively (Jiang et al., 2009; Suzuki et al., 2013), whereas other studies (e.g. Kaunitz et al., 2013) use the more sensitive decoding method of MVPA. MVPA for fMRI data allows for the detection of more fine-grained differences and consistencies in neural data by examining patterns of voxel activity, rather than activity of
individual voxels (Norman et al., 2006). When applied to EEG data, MVPA permits the characterization of neural dynamics through temporal generalization and provides a more sensitive measure of neural activity than classical analysis of event-related potentials, since it does not rely on neural activity during specific points in time. Instead, it can detect distinct patterns of dynamic changes across time periods (King & Dehaene, 2014). Some studies have directly contrasted the more ‘traditional’ analyses with MVPA; for example, while Kaunitz and colleagues (2011) found no differences based on the analysis method used, Sterzer and
colleagues (2008) obtained significant results only with MVPA, underlining the importance of distinguishing between different methodologies when comparing results.
This problem concerning the varying sensitivity of the different analysis methods, combined with the use of different measures of conscious perception across studies, could be responsible for the conflicting results obtained in the decoding studies discussed above (Peters et al., 2017; Rothkirch & Hesselmann, 2017). Therefore, to systematically investigate the extent to which invisible stimuli are processed in the brain, a study design that makes use of the sensitive measure of MVPA and allows for a direct comparison between decoding of category information from objectively and subjectively invisible images is needed. Determining not only whether the conflicting decoding results could be explained by the different measures used, but also
establishing whether a subjective or an objective assessment is preferable would have far-reaching implications for consciousness research and the subjective nature of consciousness.
Current Study
In the current study, we address the methodological inconsistencies outlined above. Most unconscious processing studies draw conclusions based on one experimental paradigm and analysis method only, thus making it challenging to directly compare the effects different
methodologies have on findings. Additionally, as discussed above, it is not clear as of yet,
whether a subjective or objective measure of conscious perception is preferable. To resolve these issues and shine light on the sources of inconsistencies in unconscious processing studies, we designed two experiments that together allow for a direct contrast between neural processing associated with subjective and objective measures.
First, to determine whether a subjective or objective threshold provides a more valid measure of conscious perception, we conducted a behavioral experiment that allowed us to investigate the susceptibility of each measure to context manipulations. Based on the premise that the latent ‘true’ threshold of conscious perception depends on the physical properties of the stimulus only and is not subject to context manipulations (Pylyshyn, 1999)1, we expected the objective threshold to be robust against context manipulations whereas the subjective threshold would not be. In line with a signal detection approach, we thus predicted that the criterion, representing the subjective measure, would shift with context manipulations, whereas the sensitivity (d’), representing the objective, performance-based measure, was expected to remain the same regardless of context. Should this be the case, this finding would indicate that an objective measure represents indeed a more robust and accurate assessment tool of conscious perception. Here, it is important to emphasize that while several signal detection approaches have focused on the evaluation of confidence ratings in forced-choice paradigms (e.g. Jachs et al., 2015; Ko & Lau, 2012), in the present study no confidence ratings were obtained, and the signal detection approach was applied to the detection and discrimination performance itself.
In a second step, after demonstrating behaviorally that objective and subjective measures can lead to different outcomes, we conducted an EEG study to determine the influence of
1 This premise is not uncontested and has been criticized heavily in light of predictive processing accounts of perception.
different awareness measures on the results of unconscious decoding studies. Specifically, participants viewed images of houses and faces, the masking contrast of which was varied across trials to result in four visibility categories: objectively visible, subjectively visible, subjectively invisible and objectively invisible. Then, a classifier was trained to determine the decodability of category information for each of these visibility conditions. We expected that category
information of unseen images would be decodable for subjectively invisible, but not objectively invisible trials. This finding would further illustrate that decodability of unseen images depends on the awareness assessment method and that no higher-level processing is taking place for images during performance at chance-levels. We specifically chose to employ MVPA to make use of optimal sensitivity while attempting to unravel category-specific patterns of neural activity across time (King & Dehaene, 2014).
Overall, investigating the extent to which unconscious information is subject to higher-level processing and in how far this is dependent on the measure of consciousness used, not only has implications for evaluating the conflicting results found in previous studies but has also far reaching implications for phenomena of unconscious processing such as blindsight and priming.
Experiment 1 – Behavioral Computer-Task
To determine whether an objective or a subjective measure provides a more valid assessment of conscious perception, we designed a behavioral experiment during which
participants were required to detect masked images of houses and faces. During the different task conditions, the context was manipulated without changing the perceptual properties of the target stimuli. We applied a signal detection approach to the analysis, disentangling the effects of context manipulation on the objective measure and the subjective measure of conscious perception. The primary measures from signal detection theory, d-prime (d’) and criterion (c), can be considered to represent the objective and subjective threshold of perception respectively;
d’ is a measure of bias free, objective performance, whereas c represents the subjective threshold applied by the participant determining how conservatively or liberally detection decisions are made under perceptually noisy conditions.
Methods Experiment 1 Participants
28 healthy participants (6 males) with a mean age of 21.6 years (SD=4.66) were
recruited to take part in a behavioral experiment. This study was approved by the Ethics Review Board of the University of Amsterdam (2019-BC-10091).
Procedure
Each participant completed two task blocks consisting of 360 trials each, both of which were preceded by a short training block of eight trials. The training blocks were not included in the analysis. During the task, participants viewed forward and backward masked images on a computer screen with a frame rate of 120hz. In each trial, either a target (face or house) or a scrambled image was presented. After each presented image, the participant indicated through a button press whether the shown image was visible or not and made a forced-choice decision whether the image was a face or a house. The participants were instructed to guess the identity of the stimulus image even for trials during which they reported not having seen an intact image. Each trial started with a fixation cross being displayed for 1000ms, followed by a black screen for 500ms. Then, after a forward mask of 100ms, the target stimulus was shown for 17ms and subsequently backward masked for 183ms. Following the stimulus presentation,
participants had 1800ms to indicate through a button-press whether they had seen the stimulus or not, and report or guess whether the stimulus was a face or a house. The response period was kept constant by showing a fixation cross after a given response until the 1800ms were up, or
automatically continuing with the next trial in case no response was given before the response period elapsed. Inter-trial intervals were jittered (100-900ms) with a mean duration of 500ms. Within-Subject Manipulation
The task of the two main blocks was identical, except for the percentage of target trials compared to the percentage of scrambled trials. In one block, a target image was presented 33% of the time, with the rest being scrambled images. In the other block, 66% of the trials consisted of target images. The order of the blocks was counterbalanced across participants and
participants were explicitly told how many unscrambled target trials they could expect in each block (33% or 66%). We hypothesized that explicitly including a higher percentage of
unscrambled target images in the task would lead participants to adopt a more liberal response criterion, whereas their bias-free performance (d’) would remain unaffected.
Between-Subjects Manipulation
Half of the participants (Group 1, N=14, 3 males, mean age: 21.2, SD: 4.77) were shown trials that consisted of the following masking contrasts: 8.7%, 23.1%, and 100%. The other half (Group 2, N=14, 3 males, mean age: 21.9, SD: 4.70) saw stimuli that were masked by contrasts of 3.3%, 8.7%, and 23.1%. The dependent variables were thus the two masking strengths that were equal across both groups. Both participant groups received identical instructions with no mention of the masking strength. We hypothesized that changing the reference contrast (100% versus 3.3%) would lead participants to change their response bias during the other two
contrasts (8.7% and 23.1%), whereas their bias-free performance (d’) would remain unaffected. Materials
Target stimuli consisted of 10 face photographs with a neutral expression from the FACES database (Ebner, Riediger, & Lindenberger, 2010) and 10 house photographs found
online. All images were converted to grayscale and cut into oval shapes. Scrambled images were created by Fourier transforming the house and face images, then shuffling the values of the frequency image. This way, the contrast of scrambled images was kept the same as the contrast of the target images.Forward and backward masking was achieved with grayscale Mondrian-like masks, generated using a code by Marin Hebart (
www.martin-hebart.de/webpages/code/stimuli.html). For example-stimuli of each category, see Appendix A. Analysis
All data were analyzed using MATLAB R2017b (MathWorks Inc., Natick, MA, USA) and IBM SPSS Statistics 23 (IBM Corp., Armonk, NY, USA), based on principles of signal-detection theory (Stanislaw & Todorov, 1999). Detection rates (scrambled vs. unscrambled images) and discrimination rates (houses vs. faces) were analyzed separately. Detection rates were calculated based on the visible-invisible responses given by the participants in relation to the presence or absence of a stimulus in each trial. Discrimination rates were obtained from participant responses reporting or guessing whether they had seen a face or a house, compared to the category of the stimulus shown in each trial (when stimuli were present). To determine whether the between-subject context manipulation had an effect on visibility reports and
performance, the sensitivity (d’) and criterion (c) were assessed for detection rates and compared between the two participant groups with an independent-samples t-test. Further, to investigate whether the within-subject context manipulation of the percentage of scrambled vs. unscrambled images affected the performance and visibility reports, a paired-samples t-test was used to compare d’ and c respectively for blocks of 33% unscrambled image presence and 66%
unscrambled image presence. Lastly, to assess whether any of the context manipulations had an influence on the sensitivity of face vs. house discrimination, an independent samples t-test and a
paired-samples t-test were conducted for the differences in d’ for the between- and within-subject manipulations respectively.
Results Experiment 1 Between-Subjects Detection (Present vs. Scrambled)
An independent-samples t-test was conducted to determine the influence different reference masking-contrasts had on the objective and subjective visibility of trials masked with stable target contrasts. The reference masking-contrast for Group 1 was stronger than the target contrasts, whereas the reference contrast for Group 2 was weaker than the masking contrast during target trials. In line with our hypothesis, Group 1 applied a significantly more liberal criterion in both presence conditions (33% and 66%) and both contrast conditions (8.7% and 23.1%) compared to Group 2: 33%: t(26)=-2.715, p=.012 (masking 8.7%); 23.1%: t(26)=-4.019, p<.000 (masking 23.1%); 66%: t(26)=-2.619, p=.015 (masking 8.7%); t(26)=-3.365, p=.002 (masking 23.1%). While this occurred without a significant difference in sensitivity (d’) for trials with a masking strength of 23.1% (33%: t(26)=1.562, p = .130; 66%: t(26)=1.255, p=.221), the sensitivity of both groups was significantly different in the condition with a contrast of 8.7% (33%: t(26)=-3.987, p <.000; 66%: t(26)= -2.493, p=.019).
Between-Subjects Discrimination (Face vs. House)
Determining the effect that the manipulation of the reference contrast had on discrimination performance during target trials, an independent-samples t-test revealed no significant change in sensitivity between groups; neither for a mask contrast of 8.7% with an intact stimulus presence of either 33% (t (26) = -1.172, p= .252) or 66% (t (26)=-1.926, p = .065), nor for a contrast of 23.1% with an intact stimulus presence of either 33% (t (26)=.478, p
= .636) or 66% (t (26)=.642, p = .527). This finding was in line with our hypothesis that the bias free sensitivity is immune to context manipulations.
Within-Subject Detection (Present vs. Scrambled)
Investigating whether context manipulations can induce within-subject changes in objective and subjective measures of awareness, a paired-samples t-test indicated that a higher percentage (66%) of present targets led the participants to apply a significantly more liberal criterion in both masking strength conditions of 8.7 % (M= -.27, SD=.67) and 23.1% (M=.73, SD=.67), compared to a lower percentage of present targets (33%) in both masking conditions (8.7%: M=-.04, SD=.57; 23.1%: M=.9, SD=.59); 8.7%: t(27)=2.392, p=.024; 23.1%:
t(27)=2.072, p=.048. As we had predicted, the sensitivity (d’) was not significantly affected by changes in the percentage of unscrambled vs. scrambled images; 8.7%: t(27)=.404, p=.689; 23.1%: t(27)=-.038, p=.970.
Within-Subject Discrimination (Face vs. House)
While a paired-samples t-test indicated no significant change in sensitivity based on manipulations of stimulus presence for trials with a mask contrast of 23.1% (t (27) =-1.409, p=.170), contrary to our predictions, sensitivity did change significantly as a function of stimulus presence during trials with a masking contrast of 8.7% (t(27) =-5.312, p<.000).
Discussion Experiment 1
To determine whether an objective or a subjective measure of conscious perception provides a more accurate measure of the latent threshold, we examined the extent to which each of these two measures is susceptible to context manipulations in a behavioral experiment. As predicted, we showed that different context manipulations do indeed consistently influence the subjective threshold, indicating that participants adjust their response bias even when stimulus
properties are kept the same. While these findings could suggest that the subjective threshold is not a robust and thus not an accurate measure of conscious perception, contrary to our
expectations, we also found significant context-dependent influences on the sensitivity, i.e. the objective threshold. Specifically, participants were significantly better at discriminating between houses and faces as well as presence or absence of stimuli in certain context manipulations, even though the properties of the target stimuli were kept the same. Whereas the subjective threshold was always significantly influenced by context manipulations, we did not find significant shifts in the objective threshold for all context manipulations, indicating that although both are influenced by the context, the subjective threshold and the objective threshold do not vary consistently together. This further illustrates the need for a clear distinction between these two thresholds in studies of conscious perception.
Experiment 2 – EEG
After demonstrating that subjective and objective measures of conscious perception can be affected differentially by context manipulations, we wanted to determine whether this difference would translate into divergent findings regarding unconscious processing in
neuroimaging studies. To this end, participants carried out a similar category detection task as in the behavioral experiment, this time while their neural activity was measured with EEG.
Subsequently, we analyzed their neural responses separately for trials of different visibility conditions (subjectively visible/invisible, objectively visible/invisible) to establish whether category information can be decoded from the EEG signal for each of these conditions.
Methods Experiment 2 Participants
To determine whether category information of invisible stimuli is decodable from EEG data, 27 healthy participants were recruited at the University of Amsterdam. One participant was
excluded due to missing data files. Analysis was done on the data of the remaining 26
participants (5 males, mean age: 21.2 years, SD = 4.1). This study was approved by the Ethics Review Board of the University of Amsterdam (2019-BC-11387).
Procedure
The EEG experiment consisted of two tasks; an experiment task and a localizer task. During the experiment task, consisting of a practice block and five identical task blocks with 160 trials each, participants viewed forward- and backward-masked images on a computer screen with a frame rate of 120hz. Presentation time, inter-trial interval timing and response time was identical to the task used in the behavioral experiment described above.
In each trial, a target image (face or house) was shown under different masking conditions, some stronger than others. The masking contrasts were chosen to result in four different stimulus conditions; subjectively visible, subjectively invisible, objectively visible, and objectively invisible. A quarter of the trials was objectively visible (2% masking contrast), another quarter was objectively invisible (100% masking contrast), with these masking contrasts being chosen based on the results of a prior behavioral experiment not reported here. During the other half of the trials, the contrast was dynamically altered based on performance to lead to 25% subjectively visible and 25% subjectively invisible trials in total. The training block at the beginning of the task was not included in the analysis.
Figure 1. Example of one objectively visible trial (left) and one objectively invisible trial (right).
Visibility was manipulated by varying the strength of the masking contrast.
The second task, which was a localizer task, required participants to watch a rapid stream of unmasked, visible face and house images and to press the space bar whenever the exact same image was shown twice in a row. The localizer consisted of one block only, with 960 trials.
Materials
Target stimuli consisted of the same 10 face (Ebner, Riediger, & Lindenberger, 2010) and 10 house photographs found online as in the behavioral experiment, converted to grayscale and cut into oval shapes. Forward and backward masking was again achieved with grayscale Mondrian-like masks, generated using a code by Marin Hebart (
www.martin-hebart.de/webpages/code/stimuli.html). For example-stimuli of each category, see Appendix A. Brain activity was measured and collected with a Biosemi ActiveTwo EEG amplifier and 64-electrode system (BioSemi B.V., Amsterdam, Netherlands) at a sampling rate of 1000hz.
The tasks were programmed and ran on MATLAB R2017b (MathWorks Inc., Natick, MA, USA) with Psychtoolbox (Brainard, 1997; Kleiner et al., 2007; Pelli, 1997;).
Analysis
The EEG data were pre-processed and analyzed using MATLAB R2017b (MathWorks Inc., Natick, MA, USA) including the additional toolboxes Fieldtrip (Oostenveld et al., 2011) and CoSMoMVPA (Oosterhof et al., 2016), and the function boundedline.m (Kearney, 2020). Preprocessing
Preprocessing steps included defining events, correcting trigger timing, resampling data to 200hz and removing excessively noisy channels and trials based on visual inspection. Further, an independent component analysis was run for each participant to determine and eliminate components representing blinking and saccade movements whenever visually identifiable.
Multivariate Pattern Analysis
Linear Discriminant Analysis (LDA) classifiers were trained on the data of the localizer task to discriminate between house and face trials and were subsequently tested on the various visibility conditions of the main task. Training was done on the localizer task to isolate the neural response of the percept and to thus exclude the possibility of decoding the manual button-press response, which was only present in the main task and not the localizer. Classification accuracy was based on the percentage of correct classifications made by the classifier.
Diagonal Decoding. First, a diagonal decoding procedure was performed for each condition, determining for each time point post stimulus-presentation how well a classifier trained on the same time point of the localizer task can correctly classify the image. This
analysis resulted in a multi-comparison corrected z-value for each of the 200 time points, indicating the decodability of the presented image against chance-levels at each of these points.
In a second analysis step, some of the different visibility conditions were directly compared to investigate whether decoding accuracy of one condition was significantly higher than of another. Specifically, the two conditions objectively invisible and subjectively invisible were contrasted with each other across all diagonal time points. The same was done for
subjectively visible and subjectively invisible conditions.
Temporal Generalization. To determine the stability of neural representations across time, an exploratory time-by-time decoding analysis was run on the data of each visibility condition separately. This analysis consisted of comparing all time points from the localizer task with each time point from the main task to obtain a decoding matrix. Z-values corrected for multiple comparisons were obtained for each decoding accuracy at the different time points. For further details of this procedure, see Kaiser et al., 2016.
Results Experiment 2 Diagonal Decoding
Investigating the decoding accuracy of a classifier trained on time-points of the localizer task against the exact same time-points of the main task during different visibility conditions revealed significant decoding accuracies (against chance-levels) for objectively visible and subjectively visible images as expected. Further, decoding accuracies were also significantly above chance levels at several time points for invisible stimuli, but only for subjectively invisible images. Objectively invisible stimuli could not be decoded significantly above chance at any point in time. For a time-course of diagonal decoding results against chance levels, see Figure 2.
Figure 2. Diagonal decoding accuracy against chance-levels over time. Stimulus onset at 0ms. Bold lines on top indicate significance corrected for multi-comparisons. Red: objectively visible, blue: subjectively visible, green: subjectively invisible, purple: objectively invisible.
Comparing different visibility conditions directly with each other revealed that the difference in decoding accuracy between objectively invisible and subjectively invisible images was significant (p<0.05) 425ms to 500ms post stimulus onset (Figure 3). Surprisingly, decoding accuracy for subjectively visible trials was only slightly higher (at 155ms to 165ms post stimulus onset) than for subjectively invisible trials, suggesting that the distinction between what is reported to be perceived and not perceived is rather small (Figure 4).
Figure 3. Difference in diagonal decoding accuracy. Red line indicates time points of significant differences in decoding accuracy. (A) subjectively invisible – objectively invisible, (B) subjectively visible – subjectively invisible.
Temporal Generalization
To investigate the neural time course of category information, we conducted an
exploratory temporal generalization analysis during which each time point of the main task was decoded based on each time point of the localizer task. This created a matrix of decoding accuracy which allows for determining the stability of category representation across time.
The results indicate that for visible stimuli, the temporal generalization of category information increases with time; at around 250ms post stimulus-onset (time point 90) decoding accuracy appears to lose temporal dependency and neural representations become stable. Subjectively visible stimuli did not show drastically different generalization patterns compared to objectively visible stimuli, while overall decodability appears to be attenuated. Further, the neural representation of category membership seems to become somewhat stable at around 145-170ms post stimulus-onset (time points 70-75); a classifier trained on localizer data at around
this time can discriminate between categories above chance at all following time points in the main task.
Based on visual inspection, the temporal generalization analysis of subjectively invisible trials revealed a decoding matrix strikingly similar to those of visible trials, though in
substantially weaker form. In contrast, trials that were objectively invisible could, as already indicated by the diagonal analysis, not be decoded successfully. No category-specific
information seems to be present when performance is at chance levels. Temporal generalization matrices for all visibility conditions can be seen in Figure 4.
Figure 4. Temporal Generalization of Category Information During Different Visibility Conditions.
Significance matrix (z-values; corrected for multiple comparisons) of temporal generalization analysis. Stimulus onset at time point 40. Classifiers were trained on the localizer and tested on trials of the main task.
Behavioural Analysis. To confirm the validity of the different visibility conditions, mean accuracy of house/face discrimination was calculated for each visibility condition. Mean accuracy for discriminating objectively invisible images was slightly above chance (t(25)=2.127 p=.043). A summary of behavioral findings can be found in Appendix B.
Discussion Experiment 2
As predicted, decoding accuracy of neural data was significantly above chance for invisible images, but only if a subjective threshold was used. Objectively invisible images, i.e. trials in which participants performed at chance-levels, could not be decoded accurately above chance. Directly comparing the decoding accuracies between the different visibility conditions revealed that the decoding accuracy of subjectively invisible images was significantly higher than that of objectively invisible images at about 425ms to 500ms after stimulus onset.
Surprisingly, subjectively visible trials could only be decoded with slightly higher accuracy than subjectively invisible trials, suggesting that the difference in neural activity between seen and unseen images with a subjective threshold is rather small.
Temporal generalization analysis of objectively visible trials revealed that the neural stability of category information increases with time; at around 225ms post-stimulus onset
decoding accuracy appears to lose temporal-dependency. This ‘square pattern’ is characteristic of neural representations of consciously perceived stimuli; conscious perception appears to be associated with stable neural patterns at later time points, whereas unconsciously perceived stimuli evoke diagonal, non-generalizable signals only (Dehaene & King, 2016). Training a classifier at 145ms-170ms post stimulus onset of the localizer task allowed for time-independent
decoding at all following time points of the main task for objectively visible trials. This finding is in line with the commonly established N170 event-related potential in classical EEG analysis; negative deflections at around 170ms post stimulus onset have been shown to consistently reflect neural processing of faces and familiar objects (e.g. Rossion et al., 2003).
Most strikingly, subjectively invisible images elicited temporal generalization patterns similar to those of visible images, though attenuated. Compared to the general finding that ‘square patterns’ are characteristic of consciously perceived information, these results suggest that subjectively invisible stimuli might be consciously perceived, despite the conflicting subjective report. As in the diagonal analysis, objectively invisible images were not decodable above chance levels. Compared to Dehaene and King (2016), we did not find the diagonal pattern of temporal generalization characteristic for unseen images, likely because their findings are based on low-level stimulus properties, which have been kept constant in the current study. Thus, the lack of category-specific neural activity during objectively invisible trials in the current study suggests that higher-level processing is not present when stimuli are objectively invisible.
General Discussion
The aim of the current study was to address the discrepancies found in previous studies regarding the neural processing of unconscious information and to determine which measure of conscious perception presents a more accurate assessment of perception. To this end, we
conducted two separate experiments; a behavioral experiment to shine light on which assessment method more closely reflects the ‘true’ latent consciousness threshold and an EEG study to investigate the neural dynamics of stimuli categorized as invisible through different methods.
The results of the current behavioral experiment do not allow for a clear-cut judgement as to which measure of conscious perception is to be preferred; while changes in criterion and not sensitivity would have suggested that the subjective threshold might not be a reliable indicator of awareness, changes in both criterion and sensitivity would have implied that the adjustment of the subjective threshold likely represents a true change in perception and would thus be accurate. Neither of those cases was found here exclusively, with the first option being observed for a masking contrast of 23.1% and the second option for a masking contrast of 8.7%, rendering the interpretation of current findings less straightforward.
Assuming that in line with our initial hypothesis sensitivity does not change since the perceptual properties of the stimuli have remained the same, we would have to question what could have led to shifts in sensitivity. While we can likely exclude potential training effects on performance, due to the counter-balanced design of the study, another explanation for increased sensitivity during certain context-manipulations could be heightened motivation and by proxy heightened attention during blocks with many intact stimuli present. It appears possible that participants were more motivated and attentive in blocks with a higher percentage of target stimuli. Both motivation and attention have been shown to affect sensitivity and performance during perceptual detection tasks (Abbott & Sherratt, 2013; Engelmann et al., 2009; Engelmann et al., 2014; Martinez-Trujillo & Gulli, 2018; Seli et al., 2015), although this relation is not always straightforward, and motivation can vary without affecting performance (Hafenbrack & Vohs, 2018). Motivation could also explain why this shift in sensitivity was only seen for trials with a masking strength of 8.7% and not for those with a masking strength of 23.1%. Assuming that being presented more often with target stimuli keeps attention and motivation high, this would presumably only be the case in masking conditions that nonetheless do not obscure perception of the images too much. If target images are too strongly masked so that many are
still missed, these conditions would be less likely to increase attention and motivation. Although this could be the case, considering the trials with different masking strengths were intermixed in the task design, it is unclear how motivation could apply to one masking condition only.
Another possibility worth considering in light of recent discoveries within a predictive processing framework, is that context manipulations might in reality shift the latent threshold of conscious perception. While there seems to be evidence that correct expectations can indeed increase the chances that a stimulus is reported as ‘seen’ (Lupyan & War, 2013; Meijs et al., 2019; Melloni et al., 2011), the report of a stimulus as ‘seen’ is a subjective measure of consciousness and thus these results can only explain the shifts in response bias found in this study. Yet, expectation-induced changes in performance have also been found, with several studies demonstrating improved stimulus discrimination based on correct expectations regarding stimulus category (Pinto et al., 2015; Puri & Wojciulik, 2008; Stein & Peelen, 2015). This could explain the significant changes in sensitivity for detection of stimulus absence versus stimulus presence. Yet, it is unclear whether expectations can explain the shift in sensitivity for house/face discrimination; whereas participants can be considered to be expecting more intact than
scrambled images during certain context manipulations, the current study design did not allow for a formation of valid expectations regarding stimulus-identity. Therefore, without correct expectations regarding stimulus-identity, increased discrimination performance based on
expectations appears unlikely. It is conceivable, however, that expectation-based accelerations of conscious perception (Melloni et al., 2011; Panichello et al., 2013; Pinto et al., 2015) aided stimulus identification, even in absence of an identity-specific cue; faster access to consciousness based on the expectation of an intact stimulus being presented could have promoted successful stimulus-identification by allowing the stimulus to enter conscious perception during the short presentation-period, before the backward-mask was shown.
A further issue with the expectation-based interpretation of improved performance is that the sensitivity was affected by expectations only in trials with a masking contrast of 8.7% but not in those with a masking contrast of 23.1%. This might be explained by findings that expectations seem to influence perception predominantly in situations of uncertainty (Clark, 2013; Kok et al., 2012; Pinto et al., 2015); potentially, a masking contrast of 8.7% was ambiguous enough to be affected by expectations, whereas the contrast of 23.1% was already too strong to be sufficiently uncertain. This, however, is only speculation and the current results do not allow for a clear-cut explanation for shifts in conscious perception thresholds.
Category Processing of Subjectively and Objectively Invisible Images
First, we replicated the prevalent finding that visible house and face images evoke category-specific neural activity that can be distinguished by MVPA. House and face stimuli could be decoded significantly above chance for both objectively and subjectively visible trials. Regarding unconscious processing, the current results indicate that while subjectively invisible stimuli do reach higher processing-stages, this seems not to be the case for objectively invisible stimuli; decoding accuracy of category-information was significantly above chance only for subjectively invisible trials. Further, the difference in decodability between objectively and subjectively invisible images was significant, suggesting that the choice of awareness measure can indeed play a critical role in discrepancies across decoding studies. Beyond suggesting that inconsistent previous findings might be a result of differences in methodology, these findings support the notion that unconscious processing is indeed limited; with a conservative measure of conscious perception, invisible stimuli do not seem to enter higher-level processing regions. While the current EEG findings leave open the possibility that category-specific information could be represented in the spatial, rather than the temporal neural signature, findings from a
prior fMRI study (Stein et al., submitted) are largely in line with current results. Thus,
objectively invisible stimuli do not seem to be subject to unconscious higher-level processing. An unexpected finding was the small difference between the decodability of subjectively visible and subjectively invisible information. One possible explanation for this is that the difference between stimuli being reported as seen or unseen is not as striking as intuitively expected and the neural signatures between these two conditions are not inherently dissimilar. Or rather, the fine line between a stimulus being subjectively visible or invisible, appears to be not dependent on differences in the neural dynamics of category-processing. However, as this difference between subjectively visible and invisible stimuli was more pronounced in a similar experiment carried out with fMRI (Stein et al., submitted), the current findings are more likely a byproduct of the neuroimaging method used. As a prevalent suggestion about the nature of conscious perception emphasizes the importance of neural feedback activity for conscious perception (e.g. recurrent processing theory, Lamme, 2010), it could be speculated that this feedback activity is more easily picked up by fMRI, driving the differences between decoding of seen and unseen information.
Nevertheless, despite this unexpected finding, the current results rather clearly illustrate that subjective and objective visibility thresholds are not equivalent and can lead to different results in decoding procedures, possibly accounting for the discrepancies found in unconscious decoding studies illustrated above. As discussed previously, it is not clear as of yet which measurement technique most closely assesses the line between conscious or unconscious perception, with both objective and subjective thresholds having their caveats (Persuh, 2018; Peters et al., 2017; Schmidt, 2015; Seth et al., 2008). However, results from the exploratory temporal generalization analysis reveal neural signatures of subjectively invisible stimuli more similar to those of consciously perceived images than those expected for unconsciously
processed images (Dehaene, & King, 2016). Thus, subjectively invisible images might still be consciously perceived, despite subjects reporting no conscious visual percept. This finding supports the notion that subjective measures of conscious perception are not an ideal measure of awareness.
Implications and Limitations
Beyond determining whether a subjective or objective threshold is a better measure of conscious perception, one has to consider the working definition of consciousness used in this context. For example, within the common distinction of access and phenomenal consciousness, a subjectively reported stimulus has by definition entered access consciousness, but whether a non-reported stimulus reaches phenomenal consciousness or remains unconscious is more difficult to entangle (Dehaene et al., 2006). The distinction between an objective and subjective threshold might allow for a discrimination between phenomenal and access consciousness, as well as unconscious perception (Irvine, 2009). In this case, asking which of the two measures is more accurate might be misguided, as they might simply assess different aspects of consciousness; access and phenomenal consciousness respectively. Regardless of the interpretation of the thresholds however, the current study clearly illustrates that objective and subjective thresholds cannot be used interchangeably in research settings and that a distinction must always be made when examining evidence regarding unconscious perceptual processing. Further, should unconscious higher-level processing not occur, as results obtained with an objective threshold suggest, this would call into question the extent to which phenomena such as blindsight or priming can be considered evidence for unconscious perception and might indicate that
for unconscious processing of category information, it appears that conscious perception might be a necessary prerequisite for certain higher-level functions.
There are some limitations to this study worth mentioning. The main limitation of the current study is the premise of the behavioral experiment. The experimental set-up as well as the interpretation of results rest on the cognitive impenetrability hypothesis (Pylyshyn, 1999); the assumption that the ‘true’ threshold of conscious perception remains stable as long as the
stimulus properties remain the same. However, as discussed above, this assumption is not proven as of now and there appears to be some evidence that certain contextual factors, such as
expectations, can influence perception-based task performance. Assuming that the premise of our interpretation might be incorrect, a measure that is not robust to context manipulations might still accurately reflect the threshold of conscious perception, if that threshold itself is not robust to context manipulations.
Further, different analysis methods and awareness thresholds cannot fully account for the contradictory results seen in previous studies, as even studies with similar experimental designs obtained strikingly different results (see for example Kaunlitz et al., 2011; but Sterzer et al., 2009). Thus, additional investigation of these discrepancies and other methodological differences is needed. For example, a methodological variable that was not addressed in the current study are the differential effects of different visual masking/suppression paradigms. While the current two experiments both made use of forward and backward masking, there is some evidence that different paradigms used to render visual stimuli invisible can lead to divergent results in
unconscious processing studies (Almeida et al., 2018; Dubois & Faivre, 2014; Faivre et al., 2012, Fogelson et al., 2014; Peremen & Lamy, 2014; but see Stein et al., 2020). Further, it seems to be the case that familiarity (Jiang, Costello, & He, 2007) as well as affective salience (Schmack et al., 2015) of selected stimuli can influence the degree to which they enter awareness during
suppression paradigms. This methodological diversity adds another dimension to the
interpretation of decoding results and should be taken into account in future studies. Moreover, the method of post-decision wagering or confidence ratings for subjective visibility judgements, which has been argued to provide a more accurate measure than subjective visibility judgements alone (e.g. Peters et al., 2015; but Fleming & Dolan, 2010), has not been included as a measure of conscious perception in the current study and could be explored in further experiments.
Additionally, the use of a localizer during the decoding paradigm has both advantages as well as disadvantages; the decision to train the classifier on a separate localizer task was
motivated by the aim to eliminate confounding signals from manual responses, eye gaze, and working memory (Grootswagers et al., 2017; Mosterd et al., 2018) - all aspects present in the main task and absent in the localizer task. Thus, by training the classifier on a passive viewing task allowed us to discern the perceptual response to the stimuli, eliminating other components of the main task. On the flip-side, training the classifiers on a different task might introduce additional noise and thus lower the obtained decoding accuracy. However, we believe that this drawback does not outweigh the benefits gained from a localizer approach and thus chose the more conservative method of localizer-based decoding.
A potential limitation of the EEG task design is the masking strength of the objectively invisible trials; the masking strength required to render stimuli objectively invisible was
established in a separate behavioral pilot experiment but might have been too strong to allow for any neural processing. Assessing whether objective invisibility can be achieved with a weaker masking contrast which allows for more neural processing would be important before excluding any potential higher-level unconscious processing with an objective threshold of conscious perception. However, as discrimination performance was slightly above chance-levels even for
objectively invisible images, it appears unlikely that a weaker masking contrast could sufficiently render images objectively invisible.
Lastly, due to circumstances beyond our control, the EEG study had to be cut short, leading to an overall smaller sample size (N=26) and in turn to lower statistical power. Ideally, this study should be extended upon in the future with an appropriate number of participants.
Conclusion
While the current study does not allow for conclusive judgements about which measure of conscious perception is to be preferred, the EEG results clearly illustrate that objective and subjective measures can lead to different findings regarding higher-level unconscious processing. Further, the behavioral experiment demonstrated that the influence of context manipulations on target visibility is not equal for objective and subjective measures. Thus, conclusions regarding the extent of unconscious processing which are drawn based on studies using one of the two thresholds should be interpreted with caution and individual results should not be used to guide general statements about unconscious perception. Determining to which extent unconscious information does get processed in the brain requires a consistent definition of the threshold of conscious perception as well as a consistent measure. Considering at the moment there is no clear consensus as to whether the threshold of conscious perception is a gradual or an all-or-nothing threshold, whether a subjective or objective measure is more suited for assessment, or whether conscious perception is stable or can be influenced by expectations, investigating the neural processing of unconscious information systematically represents a challenging endeavor.
After demonstrating the need for a clear distinction between results obtained with
different thresholds, the current findings should be followed-up by an in-depth investigation as to what underlying quality each threshold measures and which method most closely captures the
‘real’ latent threshold of conscious perception. Standardizing and clarifying the various approaches to research into unconscious visual processing will be an important step toward understanding the neural characteristics of visual consciousness.
References
Abbott, K. R., & Sherratt, T. N. (2013). Optimal sampling and signal detection: unifying models of attention and speed–accuracy trade-offs. Behavioral Ecology, 24(3), 605–616.
Almeida, J., Mahon, B. Z., Nakayama, K., & Caramazza, A. (2008). Unconscious processing dissociates along categorical lines. Proceedings of the National Academy of Sciences, 105(39), 15214-15218.
Balsdon, T., & Clifford, C. W. (2018). Visual processing: Conscious until proven otherwise. Royal Society Open Science, 5(1), 171783.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10(4), 433–436.
Coggan, D., Andrews, T., & Baker, D. (2016). Investigating the temporal properties of visual object processing using a multivariate analysis of EEG data. Journal of Vision, 16(12), 1311.
Dehaene, S., Changeux, J., Naccache, L., Sackur, J., & Sergent, C. (2006). Conscious, preconscious, and subliminal processing: A testable taxonomy. Trends in Cognitive Sciences, 10(5), 204-211.
Dehaene, S., & King, J. (2016). Decoding the dynamics of conscious perception: The temporal generalization method. Research and Perspectives in Neurosciences Micro-, Meso- and Macro-Dynamics of the Brain, 85-97.
Downing, P. E., Chan, A. W., Peelen, M. V., Dodds, C. M., & Kanwisher, N. (2005). Domain specificity in visual cortex. Cerebral Cortex, 16(10), 1453-1461.
Dubois, J., & Faivre, N. (2014). Invisible, but how? The depth of unconscious processing as inferred from different suppression techniques. Frontiers in Psychology, 5.
in young, middle-aged, and older women and men: Development and validation. Behavioural Research Methods 42, 351–362.
Elgendi, M., Kumar, P., Barbic, S., Howard, N., Abbott, D., & Cichocki, A. (2018). Subliminal priming state of the srt and future perspectives. Behavioral Sciences, 8(6), 54.
Engelmann, J. B., Damaraju, E. B., Padmala, S. B., & Pessoa, L. B. (2009). Combined effects of attention and motivation on visual task performance: Transient and sustained
motivational effects. Frontiers in Human Neuroscience, 3.
Engelmann, J. B., & Pessoa, L. (2014). Motivation sharpens exogenous spatial attention. Motivation Science, 1(S), 64–72.
Faivre, N., Berthet, V., & Kouider, S. (2012). Nonconscious influences from emotional faces: A comparison of visual crowding, masking, and continuous flash suppression. Frontiers in Psychology, 3.
Fang, F., & He, S. (2005). Cortical responses to invisible objects in the human dorsal and ventral pathways. Nature Neuroscience, 8(10), 1380–1385.
Fleming, S. M., & Dolan, R. J. (2010). Effects of loss aversion on post-decision wagering: Implications for measures of awareness. Consciousness and Cognition, 19(1), 352-363. Fogelson, S. V., Kohler, P. J., Miller, K. J., Granger, R., & Tse, P. U. (2014). Unconscious
neural processing differs with method used to render stimuli invisible. Frontiers in Psychology, 5. doi:10.3389/fpsyg.2014.00601
Groen, I. I., Silson, E. H., & Baker, C. I. (2017). Contributions of low- and high-level properties to neural processing of visual scenes in the human brain. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1714), 20160102.
Grootswagers, T., Wardle, S. G., & Carlson, T. A. (2017). Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience, 29(4), 677-697.
Hafenbrack, A. C., & Vohs, K. D. (2018). Mindfulness meditation impairs task motivation but not performance. Organizational Behavior and Human Decision Processes, 147, 1–15. Hesselmann, G. (2013). Dissecting visual awareness with fMRI. The Neuroscientist, 19(5),
495–508.
Hesselmann, G., Hebart, M., & Malach, R. (2011). Differential BOLD activity associated with subjective and objective reports during "blindsight" in normal observers. Journal of Neuroscience, 31(36), 12936-12944.
Irvine, E. (2009). Signal detection theory, the exclusion failure paradigm and weak
consciousness—Evidence for the access/phenomenal distinction? Consciousness and Cognition, 18(2), 551–560.
Jachs, B., Blanco, M. J., Grantham-Hill, S., & Soto, D. (2015). On the independence of visual awareness and metacognition: A signal detection theoretic analysis. Journal of
Experimental Psychology: Human Perception and Performance, 41(2), 269-276. Jiang, Y., Costello, P., & He, S. (2007). Processing of invisible stimuli: Advantage of upright
faces and recognizable words in overcoming interocular suppression. Psychological Science, 18(4), 349-355.
Jiang, Y., Shannon, R. W., Vizueta, N., Bernat, E. M., Patrick, C. J., & He, S. (2009). Dynamics of processing invisible faces in the brain: Automatic neural encoding of facial expression
information. NeuroImage, 44(3), 1171–1177.
Kaiser, D., Azzalini, D. C., & Peelen, M. V. (2016). Shape-independent object category responses revealed by MEG and fMRI decoding. Journal of Neurophysiology, 115(4), 2246-2250.
Kaiser, D., Oosterhof, N. N., & Peelen, M. V. (2016). The neural dynamics of attentional selection in natural scenes. Journal of Neuroscience, 36(41), 10522-10528.
Kanwisher, N., & Yovel, G. (2006). The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society B: Biological
Sciences, 361(1476), 2109-2128.
Kaunitz, L. N., Kamienkowski, J. E., Olivetti, E., Murphy, B., Avesani, P., & Melcher, D. P. (2011). Intercepting the first pass: Rapid categorization is suppressed for unseen stimuli. Frontiers in Psychology, 2.
Kelly Kearney (2020). boundedline.m (https://www.github.com/kakearney/boundedline-pkg), GitHub. Retrieved April 14, 2020.
King, J.-R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: the temporal generalization method. Trends in Cognitive Sciences, 18(4), 203–210.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What's new in psychtoolbox-3. Perception, 36(14), 1-16.
Ko, Y., & Lau, H. (2012). A detection theoretic explanation of blindsight suggests a link between conscious perception and metacognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1594), 1401-1411.
Lamme, V. A. (2010). How neuroscience will change our view on consciousness. Cognitive Neuroscience, 1(3), 204-220.
11.
Lupyan, G., & Ward, E. J. (2013). Language can boost otherwise unseen objects into visual awareness. Proceedings of the National Academy of Sciences, 110(35), 14196–14201. Martinez-Trujillo, J., & Gulli, R. A. (2018). Dissecting modulatory effects of visual attention
in primate lateral prefrontal cortex using signal detection theory. Neuron, 97(6), 1208–1210.
Meijs, E. L., Mostert, P., Slagter, H. A., Lange, F. P. D., & Gaal, S. V. (2019). Exploring the role of expectations and stimulus relevance on stimulus-specific neural representations and conscious report. Neuroscience of Consciousness, 2019(1).
Melloni, L., Schwiedrzik, C. M., M¨uller, N., Rodri- guez, E., & Singer, W. (2011). Expectations change the signatures and timing of electrophysiological correlates of perceptual
awareness. Journal of Neuroscience, 31(4), 1386–1396
Mostert, P., Albers, A. M., Brinkman, L., Todorova, L., Kok, P., & Lange, F. P. (2018). Eye movement-related confounds in neural decoding of visual working memory
representations. Eneuro, 5(4).
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006). Beyond mind-reading: Multi voxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10(9), 424-430. Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011). FieldTrip: Open source
software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011, 1–9.
Oosterhof, N. N., Connolly, A. C., & Haxby, J. V. (2016). CoSMoMVPA: Multi-modal
multivariate pattern analysis of neuroimaging data in Matlab/GNU Octave. Frontiers in Neuroinformatics, 10.
experience. Frontiers in Psychology, 3.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10(4), 437–442.
Peremen, Z., & Lamy, D. (2014). Comparing unconscious processing during continuous flash suppression and meta-contrast masking just under the limen of consciousness. Frontiers in Psychology, 5.
Persaud, N., Mcleod, P., & Cowey, A. (2007). Post-decision wagering objectively measures awareness. Nature Neuroscience, 10(2), 257–261.
Persuh, M. (2018). Measuring Perceptual Consciousness. Frontiers in Psychology, 8. Peters, M. A. K., Kentridge, R. W., Phillips, I., & Block, N. (2017). Does unconscious
perception really exist? Continuing the ASSC20 debate. Neuroscience of Consciousness, 2017(1).
Peters, M. A., & Lau, H. (2015). Human observers have optimal introspective access to perceptual processes even for visually masked stimuli. ELife, 4.
Pinto, Y., Gaal, S. V., Lange, F. P. D., Lamme, V. A. F., & Seth, A. K. (2015). Expectations accelerate entry of visual stimuli into awareness. Journal of Vision, 15(8), 13. Puri, A. M., & Wojciulik, E. (2008). Expectation both helps and hinders object perception.
Vision Research, 48(4), 589-597.
Pylyshyn Z. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. The Behavioral and brain sciences, 22(3), 341–423.
(2012). Absence of face-specific cortical activity in the complete absence of awareness: Converging evidence from functional magnetic resonance imaging and event-related potentials. Journal of Cognitive Neuroscience, 24(2), 396–415.
Rossion, B., Joyce, C. A., Cottrell, G. W., & Tarr, M. J. (2003). Early lateralization and orientation tuning for face, word, and object processing in the visual cortex. NeuroImage, 20(3), 1609-1624.
Rothkirch, M., & Hesselmann, G. (2017). What we talk about when we talk about unconscious processing – A Plea for Best Practices. Frontiers in Psychology, 8.
Rousselet, G. A., Husk, J. S., Bennett, P. J., & Sekuler, A. B. (2007). Single-trial EEG dynamics of object and face visual processing. NeuroImage, 36(3), 843-862.
Salti, M., Monto, S., Charles, L., King, J.-R., Parkkonen, L., & Dehaene, S. (2015). Distinct cortical codes and temporal dynamics for conscious and unconscious percepts. ELife, 4. Schmack, K., Burk, J., Haynes, J., & Sterzer, P. (2015). Predicting subjective affective
salience from cortical responses to invisible object stimuli. Cerebral Cortex, 26(8), 3453-3460.
Schmidt, T. (2015). Invisible stimuli, implicit thresholds: Why invisibility judgments cannot be interpreted in isolation. Advances in Cognitive Psychology, 11(2), 31–41.
Schurger, A., Pereira, F., Treisman, A., & Cohen, J. D. (2009). Reproducibility distinguishes conscious from nonconscious neural representations. Science, 327(5961), 97–99.
Seli, P., Cheyne, J. A., Xu, M., Purdon, C., & Smilek, D. (2015). Motivation, intentionality, and mind wandering: Implications for assessments of task-unrelated thought. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(5), 1417–1425.
Seth, A. K., Dienes, Z., Cleeremans, A., Overgaard, M., & Pessoa, L. (2008). Measuring consciousness: Relating behavioural and neurophysiological approaches. Trends in Cognitive Sciences, 12(8), 314-321.
Simanova, I., Gerven, M. V., Oostenveld, R., & Hagoort, P. (2010). Identifying object categories from event-related EEG: Toward decoding of conceptual representations. PLoS ONE, 5(12).
Spiridon, M., Fischl, B., & Kanwisher, N. (2005). Location and spatial profile of category specific regions in human extrastriate cortex. Human Brain Mapping, 27(1), 77-89. Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory
measures. Behavior Research Methods, Instruments, & Computers, 31(1), 137–149. Stein, T., et al. (2020). Differential processing of subjectively and objectively invisible stimuli in
human visual cortex. Manuscript submitted for publication.
Stein, T., & Peelen, M. V. (2015). Content-specific expectations enhance stimulus detectability by increasing perceptual sensitivity. Journal of Experimental Psychology:
General, 144(6), 1089–1104.
Stein, T., Utz, V., & Opstal, F. V. (2020). Unconscious semantic priming from pictures under backward masking and continuous flash suppression. Consciousness and Cognition, 78, 102864.
Sterzer, P., Haynes, J. D., & Rees, G. (2008). Fine-scale activity patterns in high-level visual areas encode the category of invisible objects. Journal of Vision, 8(15), 10-10.
Sterzer, P., Stein, T., Ludwig, K., Rothkirch, M., & Hesselmann, G. (2014). Neural processing of visual information under interocular suppression: A critical review. Frontiers in
Psychology, 5.
during binocular suppression. NeuroImage, 46(3), 803-808. Stoerig, P., & Cowey, A. (2007). Blindsight. Current Biology, 17(19).
Van Gaal, S., & Lamme, V.A. (2012). Unconscious high-level information processing. The Neuroscientist, 18, 287 - 301.
APPENDIX A
