The link between categorization and consciousness:
Do unconscious stimuli still evoke a category specific response in IT?
Student: Werner de Valk, 5927145
Supervisor: Yair Pinto
UvA Representative: Birte Frostmann
Co-assessor: Birte Frostmann
Research institute: Cognitive Neuroscience Group Amsterdam, University of Amsterdam
Master: MSc in Brain and Cognitive Sciences, University of Amsterdam, Cognitive
Science Track
EC: 19
Abstract
Categorization of visual stimuli is thought to take place in inferior temporal cortex (Kriegeskorte et al. 2008). Research using binocular rivalry and continuous flash suppression, in which stimuli are presented to both eyes independently, rendering one of them unconscious for a period of time, has
not pointed out whether categorization still occurs during unconscious perception. The question remains whether categorization in IT is tightly linked to consciousness. Here, the experiment of Kriegeskorte et al. (2008), was partly replicated while the stimuli were rendered unconscious using
binocular rivalry. Subjects were presented with images of two categories (houses and faces) while their brain activity was measured in inferior temporal and occipital cortex, with functional MRI.
Categorization activity was measured by comparing the correlation of the activity patterns corresponding to same-category images with the correlations of patterns corresponding to images of
different categories. During seen conditions (in which subjects were consciously aware of the category-stimuli) no classification accuracy was found. During unseen conditions (in which the subjects did not consciously see one of the six pictures) classification activity was measured, but only
in lateral occipital cortex, not in IT.
Introduction
In order to get a better understanding of (the mechanisms behind) consciousness, the link between cognitive processes and consciousness has to be understood thoroughly. When the number of cognitive processes that do not require consciousness is increased, this helps narrow down the cognitive bases of consciousness. When on the other hand strong links between cognitive processes and consciousness are found, this provides us with important insights about the prerequisites for consciousness. Of these, the link between consciousness and categorization of visual stimuli has received predictions from multiple theories of consciousness, which can effectively be tested.
An influential theory of consciousness that has made predictions about this connection is the global workspace theory. Dehaene and Naccache (2001) isolate three empirical observations concerning consciousness, which on first sight all allow categorization to occur unconsciously. First of all, a large amount of processing takes place without consciousness: categorization could be part of this. Second, attention is needed for consciousness: so in case categorization happens without attention it could happen unconsciously. However, whether categorization could happen without attention is based on this not yet clear. Third, some cognitive tasks need consciousness (tasks that require information to be maintained for a longer period; novel combinations of actions; spontaneous behavior): categorization does not seem to be included in these tasks.
Whereas automatic/unconscious processes are mostly modular (both on a functional as well as neurobiological level), consciousness seems to be non-modular, according to Dehaene and Naccache. They postulate that non-modular conscious processes take place in a distributed neural system (the global workspace), that interconnects different brain regions, which would allow the modular systems to access the content of other systems. In order for consciousness to take place, temporary mobilization of active (unconscious) processing neurons into a connection loop with workspace neurons is needed. In other words: complex integration in the global workspace produces consciousness. When categorization is thought of as complex integration of visual features, the global workspace would predict that categorization would not be able to take place during unconscious perception.
However, some predictions made by Deheane and Naccache are more nuanced. They postulate that the modular, unconscious processes do not necessarily have to be low-level processes. In fact, high-level processes that are based on neural pathways that were created during development, learning or evolution could proceed unconsciously as well. Subsequently, the global workspace model makes the prediction that face perception (which, when happening consciously, correlates with activity in the fusiform face area, or FFA), would break down when rendered unconsciousness, however, there still could be significant FFA activation present without consciousness. In this case,
categorization is thought of as a modular process that reaches the workspace during conscious perception, but can function unconsciously (albeit with activity of a lower degree) as well.
Whereas according to the global workspace theory the unconscious-consciousness dichotomy relies on the complex integration of different modular processes, according Lamme and Roelfsema (2000), recurrent processing of information is the most important prerequisite. In the feedforward sweep of information processing, the sequential hierarchical levels of the visual cortex are activated after the presentation of an image, by means of cascading feedforward connections. A side from these feedforward connections (providing input from low-level cells), feedback connections (providing input from higher-level cells) exist as well. According to Lamme and Roelfsema the distinction between feedforward and recurrent processing relates to the difference between conscious and unconscious processes: whereas the feedforward sweep is mainly involved in unconscious vision, recurrent processing is a prerequisite for conscious vision.
When categorization takes place during the feedforward sweep (so without any recurrent processing), this leads to the prediction that categorization takes place during unconscious perception. This is supported by the finding that rapid saccades towards visual stimuli of different categories can occur within 100 ms after stimulus presentation. Importantly, the estimated amount of time of one feedforward sweep, from lowest-level visual cells to the highest-level (i.e. temporal cortex), is about 100 ms; recurrent connections therefore have to take place during tasks in which longer delays are measured. A forced-choice saccade task was used in which two natural scenes where shown to the left and right hemifields for 20 ms (Kirchner & Thorpe, 2006). The participants made reliable saccades to the side were an animal was depicted, within as little as 120 ms. In another study, stimuli were presented for a longer period (400 ms) and consisted of three categories: vehicles, animals and faces (Crouzet, Kirchner, & Thorpe, 2010). Saccades were most rapid for the face category, with the fastest reliable saccades just 100-110 ms after stimuli onset. As most neurophysiologists would agree on 20 ms for motor preparation, in both experiments the left over visual processing time is so short that little time is left for anything other than a feed-forward processes. Although faces are known to have special computational status (which allows them to be processed faster and more efficiently than other categories) these findings give an indication that categorization could possible take place even without recurrent processing, and therefore without consciousness.
Two other theories of consciousness that make predictions about its link with categorization are the ‘dynamic core’ of consciousness, and Neural Darwinism (incorporating this core). Tononi and Edelman (1998), describe two important phenomenal features of conscious experience that have to be accounted for when describing the neural processes underlying consciousness. First of all, these processes should be highly integrated: every conscious scene is unified and cannot be decomposed into different components. Although the thalamocortical system is thought to play a role in consciousness, conscious activity could not rely on neural activity in distributed neural populations within this system, unless this activity is integrated quickly. Integration can be measured using functional clustering, which is indicated by synchronous firing within cortical areas or between cortical and thalamic areas. As a second feature, processes have to be highly diffentiated (a great number of different conscious states can be experienced in a short period of time). On top of functional clustering, a rich and diverse range of differentiated neural activity patterns is required.
Based on this the dynamic core of consciousness is introduced: a group of neurons that is part of a functional cluster (integration), which show high diversity of activity patterns (differentiation). This includes posterioir corticothalamic regions that play a role in perceptual categorization, which interact reentrantly with anterioir regions involved in concept formation, value-related memory and planning. Therefore, according to this theory, perceptual categorization is a prerequisite for consciousness. However, when the reentrant interaction between posterior and anterior areas would be lacking, consciousness would not arise, whereas categorization could in fact take place (in posterioir corticothalamic regions).
Neural Darwinism (Seth & Baars, 2005), having roots in evolutionary theory, stresses the importance of value (reflected by neural systems mediating pleasure, pain and emotional salience) as
well as reentry (especially in extending the general Neural Darwinism theory to account for consciousness). Neural Darwinism integrates the dynamic core: the connection between posterior regions responsible for perceptual categorization, with anterior regions responsible for value-based memory, which would allow for the same conclusion based on dynamic core. Linking current perceptual events with past value-related memory would create selective advantages. Although Neural Darwnism restricts itself to stating that categorization is plausibly associated with consciousness, by incorporating the dynamic core mechanism, the same conclusion has to be made here: categorization is a prerequisite for consciousness but can occur outside of it (i.e. consciousness is no prerequisite for categorization).
However, when testing the Neural Darwinism framework in biologically inspired automata with simulated brains (called the ‘Darwin’ series, or ‘brain-based devices’) opposite predictions arise. In these automata, embodied sensors and motors were included, as well as neural simulation (including plasticity), and a value system. Demonstrating simple perceptual categorization, these models showed the importance of reentry in binding the features of a visual object, as well as cross modal and sensorimotor integration. Importantly, removing these reentrant connections abolished the perceptual categorization. I.e. reentry could be a prerequisite for categorization. In this case categorization does only appear during processes that according to Neural Darwinism account for consciousness, which would predict the opposite of the previous statement: categorization can only occur during conscious perception.
In order to test these predictions, visual stimuli have to be rendered unconscious, after which neural activity corresponding to categorization can be investigated. During conscious perception, this categorization categorization is thought to take place in inferior temporal cortex (IT). In an fMRI study, Kriegeskorte et al. (2008) presented ninety-two images of real-world objects to their subjects, while measuring their IT pattern activity. This allowed for the creation of response-pattern dissimilarity matrices, which were based on the correlations between the response patterns corresponding to different objects. Because this resulted in clear clusters corresponding to animate and inanimate objects, with subcategories of faces and bodies in the animate objects, it was concluded that categorization of visual stimuli during conscious perception takes place in IT.
Replicating the previous study, while rendering the stimuli unconscious, would give an insight in the categorization activity during unconscious perception. In order to render stimuli unconscious, different paradigms can be used. In motion induced blindness (Bonneh, Cooperman, & Sagi, 2001), binocular rivalry (Blake & Logothetis, 2002; Ooi & He, 1999; Sheinberg & Logothetis, 1997; Tong, Nakayama, Vaughan, & Kanwisher, 1998; Tononi, Srinivasan, Russell, & Edelman, 1998), (continuous) flash suppression (Almeida, Mahon, Nakayama, & Caramazza, 2008; Sterzer, Haynes, & Rees, 2008; Sterzer, Jalkanen, & Rees, 2009; Tsuchiya & Koch, 2005) binocular switch suppression (Arnold, Law, & Wallis, 2008), and backward masking (Almeida, Mahon, Nakayama & Caramazza 2008), a salient image is presented to one or both eyes but, at least occasionally, not experienced consciously. Although it is not possible to form conclusions about causal relationships (i.e. whether categorization of a stimulus is a precursor of the conscious awareness of this stimulus, or vice versa), looking at categorization activity in IT while the stimuli are rendered unconscious using these paradigms could help investigate whether there is a tight link between categorization and conscious perception. When categorization activity is much lower during unconscious than conscious perception, this provides evidence supporting that categorization is strongly modulated by consciousness.
Of these paradigms, continuous flash suppression (CFS) is often used to render stimuli unconscious in order to investigate whether categorization occurs during unconscious perception of visual stimuli. In CFS target stimul are presented to one eye while Mondrian-like patterns are flashed at a high frequency (around 10 Hz) to the other eye, suppressing conscious awareness of the target stimuli, with a duration of at least ten times greater than produced by binocular rivalry (Tsuchiya & Koch, 2005). CFS in combination with MEG measurements showed category specific activity in IT, during conscious perception as well unconscious perception (Sterzer et al., 2009). This indicates that (rudimentary) visual categorization takes place even when the subject is not consciously aware of the
visual stimuli. FMRI-measurements support this finding: activity patterns in FFA and PPA turned out to differentiate between face and house stimuli, even when these stimuli were rendered invisible (Sterzer, et al., 2008). This differentiation could only be found using multivoxel pattern analysis (MVPA), whereas no difference in FFA and PPA activity during unconscious perception could be found when standard general linear model (GLM) was used.
However, some remarks have to be made. In order to perform MVPA, a classifier has to be ‘trained’ on parts of the MRI-data, to subsequently be used to predict the condition the rest of the MRI-data corresponds to (Mur, Bandettini, & Kriegeskorte, 2009). In Sterzer et al. (2008) the classifiers were only able to successfully predict the kind of stimuli that were rendered unconscious for 58.8% in FFA, and 62.5 % in PPA. This, albeit significant, is not very impressive compared to chance level (50%). More importantly, CFS introduces low-level confounds. Whereas the stimuli in binocular rivalry are both as visually salient and constant as each other, in CFS the alternative stimulus flashes. Therefore one cannot conclude whether less categorization activity occurs in IT during unconscious versus conscious perception: was categorization reduced because consciousness was reduced as well, or because of these low-level confounds? Importantly, these flashes introduce a lot of noise as well, reducing decoding capacity.
A paradigm that might potentially overcome these disadvantage is binocular rivalry (BR). In BR, two different stimuli (often a target image and a Mondrian-like distractor image) are presented to both eyes independently, which results in stochastic alternations of conscious perception of only one of the two images. It is widely believed that this phenomenon is the automatic outcome of viewing stimuli without a unique solution to their interpretation (Blake & Logothetis, 2002). The main advantage of using BR over CFS is that no low-level confounds (flashes) are introduced. Originally, using BR, it was found that activity in IT reflects conscious awareness, rather than the direct effects of retinal stimulation. This was found using single neuron recording in monkeys (Sheinberg & Logothetis, 1997), as well as fMRI recording of humans (Tong et al., 1998). In both studies, perceiving stimuli unconsciously resulted in low, or almost completely suppressed IT activity.
However, because of its stochastic nature BR is characterized by unpredictable changes of dominance, which makes it difficult to probe a period of suppression with confidence. Furthermore, at the beginning of (and also at moments during) the trial false fusion occurs: a mixture of both stimuli is perceived (Arnold et al., 2008). During CFS this happens as well, and because in both CFS experiments the stimuli were presented for only short periods (either 800 ms or 700 ms), this effect might have negatively influenced the data.
Using longer trials would provide a solution to the issue of false fusion, because it enhances the chance of longer ‘break troughs’. The relative low amount of false fusion in long trial BR would provide a better measurement of what someone is experiencing, reducing the chance that conscious experience is present during an unconscious report. Furthermore, because of the lack of low-level confounds on top of this, conscious and unconscious perception can be compared more directly. Although there is no use in claiming that there is no consciousness left during unconscious report at all, because subjects still can make mistakes in their report, long trial BR does offer the chance to investigate whether consciousness modulates categorization.
The disadvantage however, of long trial BR is the slow hemodynamic response during the long trials hampering straight forward connection with activity corresponding to categorization. To compensate for this hemodynamic lag, responses can be shifted in time for about 4 seconds, after which a restricted (small) time range around this point can be selected to be linked with the behavioral responses (Tong et al., 1998).
It is still unclear whether categorization in IT is tightly linked to consciousness. Here we used fMRI to look at the activity in IT, to investigate whether categorization is involved in unconscious processing or not. Long trials of BR will be used to overcome the issues of CFS, which also bypasses the negative effects of the stochastic nature of BR, because it enhances the chance of longer break-through periods. Healthy human participants were presented with Mondrian-like patterns to the one eye and house and face stimuli to the other, evoking binocular rivalry. MVPA was used to perform data-analysis. Of the different classification methods that exist for applying MVPA, the
minimum-distance classifier as introduced by Haxby et al. (2001) was used. Specifically, this classifier was applied by means of pattern correlation, corresponding to the method of Kriegeskorte et al. (2008). The activity patterns corresponding to the presentation of the different categories of stimuli were correlated, after which these correlations were compared within and between categories. It was predicted that during conscious perception, the pattern activities corresponding to stimuli of the same category would correlate stronger than those corresponding to stimuli of different categories. This correlation difference will be much smaller during unconscious perception (but not necessarily at chance level), providing evidence that categorization is strongly modulated by consciousness.
Materials and Methods
Participants
Ten naïve subjects (5 female; age range 21-26 years, mean age = 24.1, SD = 1.9) participated as paid volunteers. They provided written informed consent to participate in this study, which was approved by the local ethics committee of the University of Amsterdam. All subjects had normal or corrected-to-normal vision and were naïve to the experimental hypothesis.
Stimuli
In both the computer task and the normal trials of the fMRI task the backgrounds consisted of grayscale Mondrian-like patterns. The screen was split vertically through the middle: in the centers of the two halves of the screen a gray sphere with superimposed stimuli was presented. These two halves could be adjusted horizontally and vertically to achieve perfect perceptual balance. A white fixation cross was presented at the middle of the stimuli. For the mask stimulus a red-blue-and-gray Mondrian-like pattern was used. The six target stimuli were color photographs, depicting three faces three houses, see figure 1.
In both the computerized test and during fMRI, stimuli were presented and key presses were recorded using MATLAB (MathWorks) with Psychophysics Toolbox (www.psychtoolbox.org). During the computer test, stimuli were presented on an LCD projector (Asus VG236H, 60 Hz refresh rate) which the subjects viewed through a stereoscope (NVP3d) which was mounted on the desk, on a distance of 60 cm to the screen. The size of the stimuli on the screen was 23.8 x 23.8 mm.
During fMRI, stimuli were presented with a digital light processing (DLP) beamer, using MATLAB with Psychophysics Toolbox. Through prism goggles, the subjects looked at the projection screen (61 x 36 cm) via an angled mirror (17 x 10 cm) which was located 8 cm from their eyes. The size of the stimuli on the screen was 28.4 x 28.4 mm.
Figure 1. The six stimuli and mask (Mondrian-like pattern)
Binocular Fusion
In both tasks, the backgrounds were identical for the two halves of the screen, assisting binocular fusion. Before onset the halves of the screen were reoriented vertically and/or horizontally until
subjects reported that the visual stimuli were perceived as one. During the task behind the computer dichoptic perception was achieved using a stereoscope. The mirrors in the stereoscope were reoriented until the subjects reported that the stimuli merged into one.
During the fMRI task, a black cardboard was placed perpendicular to the projection screen, facing the mirror. In combination with converging prims goggles this ensured that the stimuli on the left was visible only to the left eye and the right stimuli to the right eye, which resulted in dichoptic perception.
Procedure
Both the computer task and the fMRI tasks consisted of 7 blocks of 24 trials. Every trial lasted 26 seconds. First, during the computer task, subjects placed their chin on a plastic bar which was part of the construction that supported the stereoscope, allowing them to look through the device from a static position as comfortable as possible. Subjects were asked to report during every trial what they perceived by holding down keys on the computer keyboard corresponding to either a Mondrian, a picture (of a face or house) or, when a combination of both was experienced, to hold down both.
Computer tasks were performed to let the subject get used to the experimental set-up, but more importantly to adjust the contrasts of the pictures. Specifically, the contrast of the stimuli were corrected in every new block according to the responses of the participants during the last block, in order to balance the amount of the time the mask and target stimuli were consciously perceived. When one of the two was perceived longer than the other, the next block the contrast of this stimulus would be lower, which results in a lower amount of time this pattern is perceived by then. This allowed for a better response balance at the start of the fMRI-task.
During fMRI subjects were asked to indicate their perception by responding the same way as they did during the computer task, however instead of holding down keys on a keyboard they responded by pressing the two middle buttons on the response box. Whereas the computer task only contained normal trials, half of the fMRI task consisted of ‘playback’ trials: here stimuli were presented according to what the participant reported to have seen in a random preceding trial from the same block, see figure 2. The presentation simulation of their perception was corrected with -300 milliseconds, to correct for reaction times.
Figure 2. Normal trial and playback trial: (A) In normal trials the stimuli presentation was static; perception alternated
between experiencing a face, a Mondrian or a combination of both. Response (M=Mondrian; P=picture, in this case a face; M+P=combination of both) was recorded. (B) Playback trials consisted of a simulation of the perception during another trial (in this case the one depicted in A), corrected with -300 milliseconds.
* The experience of perceiving both a face and a Mondrian was simulated by displaying parts of both. fMRI data acquisition and equipment
Functional MRI was performed on a Philips Achieva 3 Tesla scanner, using the standard 32 channel SENSE head coil. As the main focus of this study was on responses in IT, the volume was positioned to fit those areas as completely as possible. Functional scans were acquired using two echo-planar imaging (EPI) sequences. For four of the subjects the following parameters were used: repetition time (TR) = 2000 ms, echo time (TE) 27.63 ms, flip angle (FA) = 76.1, slices = 37, with 3.0 mm thickness and 0.3 mm inter-slice gap, field of view (FOV) = 240 mm x 121.8 mm, 80x80 reconstruction matrix, slice acquisition = ascending. For the other six subjects, the parameters were: repetition time (TR) = 2384 ms, echo time (TE) 39.14 ms, flip angle (FA) = 79, slices = 37, with 3.0 mm thickness and 0.3 mm inter-slice gap, field of view (FOV) = 240 mm x 121.8 mm, 80x80 reconstruction matrix, slice acquisition = ascending.
Pre-processing
MRI data were pre-processed and analysed using MATLAB (MathWorks) and FEAT (FMRI Expert Analysis Tool), part of FSL (FMRIB’s Software Library). Registration to high resolution structural was carried with the aid of FLIRT (Jenkinson & Smith, 2001; Jenkinson, Bannister, Brady, & Smith, 2002).
The data was motion and slice time corrected, and low pass filtering was applied. During the time that was available for this project, pre-processing of MRI data from only four of the ten subjects was finished and used for later analysis.
Per subject the MRI data were first split over a maximum of 168 divisions (7 blocks * 6 images * 4 responses). Because some participants did not complete all seven blocks and some events might not occur (e.g. one of the images was never seen during normal trials of one of the blocks) this number was a maximum. Corresponding voxel activity was averaged over the time periods corresponding to the +/- 168 events.
Voxel selection
Initial voxel selection was based on the following regions of interest (ROI’s): early occipital (OC), lateral occipital (LOC) and inferior temporal gyrus (IT), see figure 3. Additional to IT, which accounts to the research question directly, occipital regions (where feature and object detection takes places) were included in analysis because category-specific correlations could be found there as well. When activity patterns evoked by the stimuli of the same categories would correlated strongly in these occipital regions, this might suggest that the category-specific pattern differences in IT are based on overall differences in visual features of the stimuli, instead of on category information solely.
During later data-analyses, GLM could be applied on the brain data corresponding to the report of seeing a picture for the normal trials, which could be compared with activity corresponding to the presentation of the same picture only for the playback trials. Their activity is expected to be similar. However, when comparing the activity corresponding to not seeing a face in normal trials (i.e. reporting to see a Mondrian) with not seeing a face in playback trials (presenting a Mondrian solely), difference would be expected. This difference would be based on the unconscious perception of the face during the normal trials, which is absent during the playback trials. Voxels where difference in activity between these conditions could be found would be an interesting place to look at categorization activity during unconscious processing.
Figure 3. From left to right: OC, LOC and IT.
Decoding analysis
The average activity per category (house or face) per response was calculated over every block. Then, the data was split in half: mean activities were averaged over the odd blocks and again over the even blocks as well. This made it possible to calculate the accuracy of the identification of the category the subject was seeing, by comparing within-category and between-category correlations. Four comparisons were possible between the two within-category correlations (odd house with even house and odd face with even face) and the two between-category correlations (odd house with even face and even house with odd face), see figure 4.
Figure 4. Comparing within-category classification with between-category classification.
Correct identification of a category was achieved when the within-category correlation was higher than the between-category correlation, which resulted in a maximum accuracy of four correct classifications, and chance level of two correct classifications. To determine the probability of this accuracy per category per ROI, one-sample t-tests were used to test whether the mean proportion of the comparisons that achieved correct identifications, averaged over all subjects, exceeded chance level (50%).
Further notes
In the generation of the previous 168 divisions, two important variables were left out. First of all, the distinction between normal and playback trials was not included. Therefore the term ‘unconscious’ cannot be used, because in half of the trials (i.e. in the playback trials) the unconscious condition only consisted of Mondrians, lacking the target picture that in the normal trials was only unconsciously perceived. Conditions will therefore be named ‘seen’ and ‘unseen’.
Furthermore, the events should have been split into two halves. Now the blocks were compared, which this is not advisable, for these consisted of separate scanning runs. Additionally, the contrasts of the images were corrected over on the beginning of each block, making the stimuli slightly different over the different blocks. Correct would be to split the events into two halves, after which the blocks could be averaged over both halves. The total amount of the new divisions would thus be (7 blocks * 6 images * 4 responses * 2 halves *2 trial types) 672.
Results
Because the time that was available for this research project was limited, the results that were gathered were based on the initial 168 divisions. When analyzing all three ROI’s independently, only for lateral occipital, only during the unseen condition significant correct classification had occurred (see table 1). When taking the three ROI’s together, only for the unseen condition the accuracy of identification differed from chance level (50%). Strangely, when comparing the percentages between seen and unseen, for every ROI the accuracy of identification was higher in the unseen condition, whereas the opposite was predicted.
Table 1. Accuracy of identification, over three different ROI’s. The accuracies are the percentages of
successful comparisons (i.e. higher between-category correlation than within-category correlation).
Region Accuracy of identification Seen Unseen % p % p LOC 81,25 0,0796 100 0 OC 62,5 0,6376 87,5 0,0577 IT 50 1 81,25 0,0796 All ROI’s 64,5833 0,2064 89,58 0,000025
Significant correct categorizations (differing from 50% with p<0.05) in bold.
However, these results were based on the MRI-data from only four subjects, with big variance in their mean accuracy of identification, see table 2. Still, all of the average accuracy’s were higher than 50%, indicating a trend towards correct classification.
Table 2. Average accuracy of identification per subject
Subject Average % p-value
pp01 pp02 pp03 pp04 75 83,333 91,6667 58,333 0,0409 0,025 0,0041 0,6952
Discussion
In the ‘seen’ condition, correlation accuracy did not differ from chance level for every ROI. In the ‘unseen’ condition, only correlation accuracy based on activity in lateral occipital cortex differed from chance. These results do not match the predictions based on the main hypothesis but in fact would support the idea that consciously perceived stimuli do not evoke categorization in inferior temporal or occipital cortex; and that unconsciously perceived stimuli only evoke categorization in lateral occipital cortex. On top of contradicting literature, this is not in line with common sense: that categorization activity only happens when the subject is not aware of the stimuli it is seeing. In any case, less visual object categorization activity would be predicted when no object was being perceived consciously, compared to consciously perceiving it.
However, these results were based on analysis that was not yet finished. As already mentioned, there are three important shortcomings of the data. First of all the difference between normal and playback trials was not taken into account; second, instead of splitting the events in half, and comparing these halves, odd and even blocks were averaged and compared; and lastly only the data of four of the ten subjects could be used, decreasing power. The result of taking normal and playback trials together is that in the unseen condition only halve of the time the picture (of a face or house) was visually present. Therefore it would be expected that this averaged unseen condition would have even lower classification accuracy than the unseen condition during normal trials (i.e. ‘unconscious’ perception).
The argument made by (Sheinberg & Logothetis, 1997) holds here as well: a small percentage of neurons in IT is not modulated by perception and might be responsible for categorization
information in itself. On top of that, inhibitory interneurons could also carry information that is specific to the suppressed stimulus, and could interact with the neurons that were coding for the stimuli that was dominant at that moment. The possibility exists that the activity pattern correlation that was found during unconscious perception could have relied on a number of voxels containing these two types of neurons.
Although the stimuli were quite diverse, low-level features of the pictures could overlap within the two categories. When this overlap would be stronger within categories than in between, difference in category information would be present on feature level. In case within category correlation (i.e. significant classification accuracy) would be found in the lower-level ROI’s (OC and LOC), the category-specific activity patterns found in IT could be based on these low-level feature differences instead of stemming from category information per se. However, because these correlations could also be influenced by recurrent processing stemming from IT (see below), only investigating classification accuracy during unconscious perception (not during conscious) would give an indication.
Since no reliable results were gathered during the period of this research project, a discussion of the possible outcomes of the fully completed data-analysis is at place. In case classification accuracy in IT would not differ strongly between conscious and unconscious perception, this would disconfirm the hypothesis. This would provide evidence that categorization and consciousness are not tightly linked: categorization can take place even without consciousness. On the other hand, when classification accuracy in IT would collapse during unconscious perception, the hypothesis would be confirmed: categorization is modulated by consciousness. Small but significant classification accuracy during unconscious report would not necessarily indicate that categorization is still taking place during unconscious perception, because this could be based on small mistakes subjects make in their report.
In case evidence for categorization to be modulated by consciousness will be found, this results in important implications with respect to the consciousness theories as discussed in the introduction. The global workspace theory (Dehaene & Naccache, 2001) would be supported, which predicts a strong link between categorization and consciousness. No strong statements can be made however about the prediction made by Dehaene and Naccache that low but significant categorization activity would be present during unconscious perception, because possible low but significant classification activity could be based on small mistakes in the response of the subjects.
According to Lamme and Roelfsema (2000) unconscious perception takes place during the feedforward sweep, subsequently leading to conscious perception during recurrent processing. In case categorization can take place during the unconscious feedforward sweep, a strong link between consciousness and categorization would provide counterevidence for this theory. As an interesting follow-up analysis, the link between recurrent processing and consciousness could be investigated. When during conscious perception stronger correlation between IT and lower visual regions (LOC and OC) would be found than during unconscious perception, this would give an indication of recurrent processing during conscious perception, supporting the recurrent processing theory of consciousness. (To clarify the influence of recurrent processing during the earlier discussed checkup of low-level feature similarity: this recurrent processing could result in category-specific information in (L)OC, which would disrupt the identification of low-level feature overlap in these regions. In order to correct for this possibility, correlation during unconscious perception should be calculated, since this could solely be based on low-level features, independent of recurrent processing.)
When evidence for categorization to be modulated by consciousness would be found, this would challenge the predictions made by the the ‘dynamic core’ of consciousness (Tononi & Edelman, 1998). Namely, when consciousness arises because posterioir corticothalamic regions that play a role in perceptual categorization, interact reentrantly with anterioir regions involved in value-related memory, the opposite is predicted: that categorization is a prerequisite for consciousness, but can take place without. The same accounts for Neural Darwinism (Seth & Baars, 2005), for it incorporates
this dynamic core. Support would on the other hand be provided for the Darwin automata that result from this theory, which do predict a strong link between categorization and consciousness.
As said, these implications follow from possible outcomes of the current study whereas the real results are not yet gathered. Either way, when no strong link between consciousness and categorization would be found, this would narrow down the cognitive bases of consciousness: other cognitive mechanisms could play a role in it. When however categorization is in fact modulated by consciousness, this strong link between the two would provide an interesting insight into the mechanisms behind consciousness, supporting theories incorporating categorization (the global workspace, Darwin automata) but challenging others. Alternative explanations have to be found in order for the dynamic core, Neural Darwinism and recurrent processing theory of consciousness to stand their ground.
References
Almeida, J., Mahon, B. Z., Nakayama, K., & Caramazza, A. (2008). Unconscious processing dissociates along categorical lines. Proceedings of the National Academy of Sciences of the United States of
America, 105(39), 15214–15218.
Arnold, D. H., Law, P., & Wallis, T. S. A. (2008). Binocular switch suppression: a new method for persistently rendering the visible “invisible”. Vision research, 48(8), 994–1001.
Blake, R., & Logothetis, N. K. (2002). Visual competition. Nature reviews. Neuroscience, 3(1), 13–21. Bonneh, Y. S., Cooperman, A., & Sagi, D. (2001). Motion-induced blindness in normal observers.
Nature, 411(6839), 798–801.
Crouzet, S. M., Kirchner, H., & Thorpe, S. J. (2010). Fast saccades toward faces: Face detection in just 100 ms, 10, 1–17.
Dehaene, S., & Naccache, L. (2001). Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework, 79, 1–37.
Haxby, J. V, Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539), 2425–2430.
Jenkinson, M., & Smith, S. (2001). A global optimisation method for robust affine registration of brain images. Medical image analysis, 5(2), 143–156.
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002). Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images. NeuroImage, 17(2), 825– 841.
Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Research, 46, 1762–1776.
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., et al. (2008). Matching categorical object representations in inferior temporal cortex of man and monkey.
Neuron, 60(6), 1126–1141.
Lamme, V. A., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 23(11), 571–579.
Mur, M., Bandettini, P. A., & Kriegeskorte, N. (2009). Revealing representational content with pattern-information fMRI - an introductory guide. Social cognitive and affective neuroscience,
4(1), 101–109.
Ooi, T. L., & He, Z. J. (1999). Binocular rivalry and visual awareness: the role of attention. Perception,
28(5), 551–574.
Seth, A. K., & Baars, B. J. (2005). Neural Darwinism and consciousness. Consciousness and cognition,
14(1), 140–168.
Sheinberg, D. L., & Logothetis, N. K. (1997). The role of temporal cortical areas in perceptual
organization. Proceedings of the National Academy of Sciences of the United States of America,
Sterzer, P., Haynes, J., & Rees, G. (2008). Fine-scale activity patterns in high-level visual areas encode the category of invisible objects. Journal of Vision, 8, 1–12.
Sterzer, P., Jalkanen, L., & Rees, G. (2009). Electromagnetic responses to invisible face stimuli during binocular suppression. NeuroImage, 46(3), 803–808.
Tong, F., Nakayama, K., Vaughan, J. T., & Kanwisher, N. (1998). Binocular rivalry and visual awareness in human extrastriate cortex. Neuron, 21(4), 753–759.
Tononi, G., & Edelman, G. M. (1998). Consciousness and Complexity. Science, 282(5395), 1846–1851. Tononi, G., Srinivasan, R., Russell, D. P., & Edelman, G. M. (1998). Investigating neural correlates of
conscious perception by frequency-tagged neuromagnetic responses. Proceedings of the
National Academy of Sciences of the United States of America, 95(6), 3198–3203.
Tsuchiya, N., & Koch, C. (2005). Continuous flash suppression reduces negative afterimages. Nature
