Privileged unconscious face processing of
conspecifics: Evidence from a novel approach
to studying unconscious visual perception
BSc Psychobiology Bachelorproject Supervised by Timo Stein
University of Amsterdam, Department Brain & Cognition
Roos van Oeveren
11671866
2
ABSTRACT
The limits of unconscious processing are highly debated. Previous studies suggested that subliminal processing can activate high-level cortical regions, whereas others promote a rather limited scope of unconscious processing. Previously, evidence for a category-specific inversion effect has been found in the breaking Continuous Flash Suppression paradigm. As a consequence, this has been interpreted as category-specific high-level unconscious processing of human faces. However, as the adopted paradigm lacks an awareness measure, the effect could have reflected conscious processing differences. Here, we studied the limits of high-level unconscious processing of human and chimpanzee faces. To test whether the effects occurred with no awareness of the manipulation, a novel approach was adopted in which subjects had to determine whether stimuli were presented upright or inverted. To summarize the results, unconscious face inversion effects were found for human faces. These results demonstrate high-level unconscious processing of faces, specific to conspecifics. Furthermore, the localization-discrimination dissociation paradigm is demonstrated to be a promising novel approach to studying unconscious visual perception. The current results are relevant for defining the limits of unconscious processing, as well as influencing the approach of future studies.
KEYWORDS
Unconscious Processing – Visual Awareness – Face Detection – Face Inversion Effect – Detection-Discrimination Dissociation – Localization-Detection-Discrimination Dissociation
INTRODUCTION
Cognitive psychology and neuroscience have cooperated on studying the capacity of unconscious perception. Clinical research suggests that unconscious processing is possible. In blindsight, for example, patients present above chance discrimination of stimulus properties presented in their blind field, i.e. without a conscious visual experience. These properties include, not exclusively, localization, reaching, size and direction of motion, up to affective or semantic content (Danckert, Tamietto & Rossetti, 2019). Multiple clinical studies have described this phenomenon (Brogaard, Marlow & Rice, 2014; Solcà, Guggisberg, Schnider & Leemann, 2015; Weiskrantz, Barbur & Sahraie, 1995). Furthermore, neuroimaging data of unconscious and conscious processing has already resulted in a better understanding of the physiopathology of disorders of consciousness (DoC) (Bourdillon, Hermann, Sitt & Naccache, 2019). DoC are reduced states of consciousness including coma, vegetative state and minimally conscious state (Eapen, Georgekutty, Subbarao, Bavishi & Cifu, 2017). For example, differences in brain activity have already been found for unconscious and conscious processing of auditory regularities (Bekinschtein et al., 2009). This demonstrates how clinical cases assisted research into unconsciousness.
Aside of clinical applications, scientists created a broadened interest in unconscious perception regarding the healthy population. There is evidence that functions such as task switching, response inhibition and conflict and error detection can occur unconsciously (van Gaal & Lamme, 2012). It has been suggested that unconscious information can affect conscious experiences such as perceptual, emotional, and motivational processes, motor response, and goal pursuit (van Gaal & Lamme, 2012). Uttermost, it is proposed that unconscious processes can operate the same high-level functions as conscious processes (Hassin, 2013). On the contrary, others have entitled this hypothesis farfetched and premature (Hesselman & Moors, 2015). Furthermore, recent research likewise promoted a rather limited extension of unconscious processing (Stein, Utz & van Opstal, 2020). One major reason for this disagreement is the diversity in defining unconsciousness. Stafford (2014) recognized a shift from defining unconscious as “without awareness of the stimuli” to “without awareness of the influence of the stimuli”. The unsettled definition provides contrasting interpretations on subliminal processing. For example, Erdelyi (1986) defended to be the first to find evidence for semantic discrimination of subliminal
3
perception. However, his classical dissociation paradigm has been criticized for overestimating the extent of unconscious processing (Stein, 2019). Participants might report no awareness for a stimulus that is, in fact, briefly visible. Some features of invisible rendered stimuli were thought to be processed unconsciously, however, not on a conceptual or semantic level (high-level processing) (Gayet, van der Stigchel & Paffen, 2014).
Recently, many studies on unconscious visual processing did succeed to demonstrate various high-level properties that are processed subliminally. For instance, emotional content (e.g. Yang & Yeh, 2018), visual complexity (e.g. Ionescu, 2016; Song & Yao, 2016), social relevance (e.g. Nakamura & Kawabata, 2018), language (Lupyan & Ward, 2013) and familiarity (e.g. Jiang Costello and He, 2007; Stein, Sterzer & Peelen, 2012). Research suggests that subliminal perception can be processed through the same feature response network as consciously perceived stimuli (Emmanouil, Burton, & Ro, 2013). Faces can be used to examine broad properties of cognitive systems, such as social cueing and evaluation, multisensory integration and emotion (Axelrod, Bar & Rees, 2015). Therefore, faces are regularly adopted as target stimuli to determine such high-level unconscious processing. Unconscious processing has been demonstrated for multiple face properties, including for faces making eye contact, faces facing the participant, trustworthy, non-dominant and attractive faces, and faces of friends (for a review see Gayet et al., 2014; Stein, 2019).
Many recent studies on unconscious visual processing adopted the breaking Continues Flash Suppression (b-CFS) paradigm, firstly applied by Jiang and colleagues (2007). This method is expected to demonstrate high-level processing of unconscious stimuli. b-CFS is currently the most adopted paradigm for determining unconscious perception (Gayet et al., 2014; Stein, 2019). b-CFS consists of one task only. A target stimulus is presented in one eye and rendered invisible through a high-contrast mask in the other eye. This is called interocular suppression (Jiang et al., 2007). It generally takes a few seconds for the target stimulus to ‘break through’ the mask and appear visible to the participant. The time it takes for a stimulus to break through is proposed to reflect unconscious processing. A shorter suppression time is related to accelerated and stronger unconscious processing (Stein, 2019).
In b-CFS, image inversion is often adopted as a control condition. Inversion disrupts semantic and contextual properties, however, it preserves low-level features. Various low-level visual properties, such as contrast or luminance, have been found to influence suppression time (Stein, Awad, Gayet & Peelen, 2018). Therefore, image inversion rules out low-level confounds that might explain previous results. Furthermore, the category-specific Face Inversion Effect (FIE), is one of the most robust b-CFS findings associated with high-level unconscious processing (Jiang et al., 2007; Stein et al., 2012; Zhou, Zhang, Liu, Yang & Qu, 2010; for a review see Axelrod et al., 2015; Gayet et al., 2014; Stein, 2019). The FIE measures that an upright face gains dominance over the suppression mask more rapidly than an inverted (180˚ rotated) face (Jiang et al., 2007). A significant FIE has been interpreted as reflecting unconscious processing of face orientation. It suggests that some information of the suppressed target is still able to reach visual areas of high hierarchy such as the Fusiform Face Area (FFA) (Jiang et al., 2007). Additionally, the FIE is in line with the selective sensitivity of the human visual system for upright depictions of conspecifics, as found by Stein and colleagues (2012). Significant inversion effects for human faces, headless human bodies and person silhouettes were established. In addition, the FIE was also found for chimpanzee faces, however, the effect was significantly larger for human faces (Stein et al., 2012). These results were proposed to show privileged processing of human faces, assuming that a face inversion effect reflects tuning of the visual system to stimuli in their normal upright orientation. A larger inversion effect could be seen as stronger cell ‘tuning’, leading to prioritized processing of matching stimuli. These results suggest that detection mechanisms can operate with a predisposition to conspecifics.
4
Nonetheless, the validity of the b-CFS paradigm is heavily debated. b-CFS requires participants to press a key immediately once a target stimulus appears visible. This reaction time (RT) based measure maximally reduces conscious influences in comparison with the classical paradigm (Jiang et al., 2007). However, the b-CFS paradigm assumes that the progress of unconscious to conscious processing is temporal. Furthermore, it assumes that it is dichotomous, including a transition point from unconscious to conscious perception. Accepting consciousness as dichotomous instead of gradual, it is still debatable whether reaction time is an accurate calculation for the transition point (Stein, 2019). RT measures are influenceable by multiple biases, decreasing the validity of the results. For instance, it is impossible to control for correct interpretation of the instructions. Most importantly, in b-CFS, it is unknown whether a detection difference is caused by conscious or unconscious processes. Therefore, it is uncertain whether subjects perform their response for a partly or fully recognizable target stimulus. Here, we adopted a novel approach to distinguish between conscious and unconscious influences on visual detection. To do so, an awareness measure was added. Stimulus processing was measured through stimulus detection or localization performance. Both approaches were combined with a discrimination task to determine awareness. As discussed above, inversion could function as a reliable manipulation. In summary, two novel approaches were developed: a detection-discrimination dissociation and a localization-discrimination dissociation. In the detection paradigm, subjects had to determine whether a face was present or absent. In the localization paradigm, subjects had to determine whether the face was presented left or right. We tested whether face inversion effects can be obtained at zero awareness. The introduced forced-choice tasks are expected to cancel all post-perceptual biases (Stein, 2019). It would represent blindsight-like phenomena that do not rely on stimulus reportability, but rather on unconscious processes that occur before stimulus awareness. In addition, this paradigm does not require consciousness to be either dichotomous or gradual as it applies to both theories. Therefore, this paradigm may be beneficial compared to current paradigms such as b-CFS.
However, a considerable complication of the novel approach is the difficulty to prevent floor or ceiling effects. This paradigm requires a very precise stimulus presentation time to which stimulus awareness is absent but unconscious processing is present. Finding this threshold may be difficult in b-CFS, therefore, adopting other psychophysical techniques may be preferable. Stein and Peelen (2020) aimed to test whether contrast and inversion effects as found with b-CFS could also be established with techniques other than interocular suppression. After normalizing the effects for different temporal scales and calculating relative differences, results showed that both contrast and inversion effects were numerically smaller in CFS than in masking paradigms. Furthermore, variation within effects was larger in CFS. Firstly, this reveals that the contrast and inversion effects as found in CFS are not related to CFS specific unconscious processing. Secondly, backward masking generated bigger and more stable effect sizes than CFS, implying that this technique may be preferred over b-CFS. As for the detection-discrimination and localization-detection-discrimination paradigms, a psychophysical technique such as backward masking is therefore beneficial as it is more reliable and it conjointly decreases the difficulty of creating an optimal threshold.
As discussed above, previous studies into unconscious processing might have based conclusions on misleading results due to insufficient control conditions, theoretical misconceptions and (post)perceptual biases. Therefore, the aim of this study is both to determine whether face orientation can be processed unconsciously, as to test a novel paradigm for studying unconscious visual perception. It is hypothesized that an unconscious face inversion effect will be demonstrated (Stein et al., 2012). Additionally, a predisposition to the detection of human faces would imply a potential mechanism for the detection of conspecifics that can operate unconsciously. To test this hypothesis, both a detection-discrimination as a localization-discrimination paradigm will be tested in use of backward masking (Stein, 2019; Stein & Peelen, 2020). The category-specific inversion will be tested through various neutral human faces and neutral chimpanzee faces. If participants cannot consciously report face orientation, but differences in detection or localization of upright faces are found, it is assumed that face orientation can be processed unconsciously. The FIE will be tested for various
5
presentation times, to decrease the chance of floor- or ceiling effects. The detection-discrimination dissociation experiment will consist of four different presentation times; 8.3 ms, 16.7 ms, 25.0 ms, and 33.3 ms. For the localization-discrimination dissociation, a fifth presentation time of 8.3[8.3] ms will be added. It is expected that for a specific presentation time, 1) discrimination properties of human faces are not significantly above chance (zero-awareness criterion), whilst 2) detection or localization of human faces is significantly above chance, indicating unconscious processing. Further expected is that 3) upright human faces are detected or localized more accurately than inverted human faces (i.e. face inversion effect), 4) human faces are better detected or localized than chimpanzee faces, and 5) the FIE for human faces is significantly larger than for chimpanzee faces, indicating that unconscious high-level processing of faces is specific to conspecifics. The expectations are illustrated in Figure 1.
Fig. 1: Expectations human faces and chimpanzee faces
(a) Expectations of the result for human faces, generalized for both the detection as well as the localization experiment. Detection/localization of upright faces is significantly higher than for inverted faces, thus showing an inversion effect. If this inversion effect is found for a presentation time for which discrimination is not significantly different from zero, unconscious processing is demonstrated. (b) Expectations of the results for chimpanzee faces, generalized for both the detection as well as the localization experiment. Detection/localization and discrimination is generally lower than for human faces. Upright faces are better detected/localized than inverted, but the effect is significantly smaller than for human faces.
METHOD Participants
For this study, a total of 75 participants were recruited. All participants were undergraduate students at the University of Amsterdam (UvA) and received research credit for their participation. All participants reported normal or corrected-to-normal vision. In this study, 42 participants (7 male), aged between 18 and 25 [mean ± SD: 19.98 ± 1.87], participated in the detection-discrimination task. The localization-discrimination task was performed by 30 participants (5 male) aged between 17 and 24 [mean ± SD: 19.83 ± 1.76]. The other three participants were excluded due to a coding error. Shanks (2017) reported that, when studying unconsciousness, non-random exclusion processes may result in regression to the mean. Therefore, no participants were excluded based on their task performance. All participants received identical written and verbal instructions, and were given examples of the stimuli as used in this study. The ethics committee of the Faculty of Social and Behavioural Sciences at the UvA approved for this study. All participants signed an informed consent and were naïve to the aims of this study.
6
Stimuli
The experiment was programmed in MatLab/Psychtoolbox R2018b. All participants were seated in a darkened room at approximately 60 cm from the 24-inch (1920 x 1080 pixels resolution) monitor with a 120Hz refresh rate. The background screen was black. Stimuli were ten different neutral human and chimpanzee faces collected from the Ekman and Friesen (1976) data set and the internet (Stein et al., 2012). Hair and outer facial features were removed. All stimuli adjustments (luminance, contrast, etc.) were similar to the research of Stein and colleagues (2012). Inverted stimuli were generated by rotation of 180˚ (Figure 2). Stimuli were presented in a grey box at the centre of the screen (7.5˚ x 7.5˚). Greyscale masks were generated by randomly arranging white, grey and black circles as shown in Figure 3. Masks covered the entire grey box.
Fig. 2: Face stimuli
Three examples of different neutral
chimpanzee faces (left), as well as neutral human faces (right), as used in this experiment. The upper row
demonstrates upright faces and the lower row shows inverted (180˚ rotated) faces.
Procedure
Prior to the start of the experiment, participants had to complete six practice trials in the presence of the experimenter. Practice trials were included to control whether all participants could perform the discrimination task correctly for a longer presentation time of 67.0 ms. The experiment itself consisted of a total of 960 trials and was split up in two halves of each 480 trials. Both halves consisted of either only human faces or only chimpanzee faces. Whether participants started with human or chimpanzee faces varied in the following sequence: chimps – chimps – human – human – chimps – chimps – human - etc. Both halves included three short breaks (10 sec.), and participants were allowed to take an additional break up to 10 minutes halfway through the experiment. The entire experiment took up one hour to one hour and a half.
Experiment 1: Testing a detection-discrimination dissociation
Firstly, a black fixation cross appeared in the centre of the grey box for 1000 ms, followed by a blank screen of 500 ms. Thereafter, it was randomly selected whether the face stimulus was absent, or presented upright or inverted. Similarly, the stimulus presentation time was randomly selected out of four options, either 8.3 ms, 16.7 ms, 25.0 ms or 33.3 ms. Figure 3 illustrates the procedure. At the end of the experiment, every condition (upright, inverted or absent) was equally presented, i.e. a face was displayed in two-third of the trials (640 trials). Similarly, each presentation time was equally presented, respectively. If the stimulus was absent (320 trials), it was replaced by a blank box that likewise appeared in the centre of the screen. After a Stimulus Onset Asynchrony (SOA) of 8.3 ms, three masks of each 100 ms were presented. Thereafter, two questions appeared one after the other. “Was the face upright or inverted?” to measure discrimination properties, and “was the face present or absent?” to indicate awareness. Both questions were untimed. Participants had to express their answers through the arrow keys of a QWERTY keyboard, indicating either ‘upright’ or ‘inverted’ for discrimination or ‘yes’ or ‘no’ for detection (Figure 3b). No feedback was given throughout the entire experiment. Whether participants started with the awareness or discrimination task alternated every participant. After responding, the task directly continued with the next trial. After completing the first half of the experiment (480 trials) with
7
either human faces or chimpanzee faces, the experiment was exactly replicated with the other set of stimuli (human or chimpanzee).
Fig. 3: Procedure of experiment 1
This figure shows an example of one trial belonging to experiment 1.
(a) This figure illustrates a chimpanzee face that is presented upright. The face stimulus is presented for either 8.3 ms, 16.7 ms, 25.0 ms or 33.3 ms. The detection and discrimination tasks were displayed after the masks. (b) This figure demonstrates the tasks for a present, upright chimpanzee face. These tasks followed the 300 ms masks. The question was presented as shown above. Participants were required to respond through the arrow keys on a QWERTY keyboard. The red dot illustrates the key that would have been calculated as a correct response.
Experiment 2: Testing a localization-discrimination dissociation
Similar to experiment 1, a trial started with a black fixation cross (1 second), followed by a blank screen for 500 ms. Different to experiment 1, face stimuli were presented for either one of five presentation times (8.3 ms, 8.3[8.3] ms, 16.7 ms, 25.0 ms or 33.3 ms). By that, it was randomly selected whether the stimuli were presented left or right in relation to the fixation cross (with an offset of 52 pixels; Figure 4). At the end of the experiment, stimuli were presented equally left or right (both 480 trials). Thus, contrary to experiment 1, no face-absent trials were included. The presentation time of 8.3[8.3] ms indicates an additional blank screen between stimuli and mask, such that the SOA becomes 16.6 ms. After presenting the masks, both discrimination (upright or inverted) as localization (left or right)
a.
8
were questioned in the same manner as experiment 1. Again, participants answered through arrow keys on a QWERTY keyboard and the order of the questions varied between subjects. For localization, the left arrow key indicated a ‘left’ response and vice versa. The remaining procedure was similar to experiment 1.
Fig. 4: Procedure of experiment 2
This figure shows an example of one trial belonging to experiment 2.
(a) This figure illustrates an inverted human face at the left side of the box. The face stimulus is presented for either 8.3 ms, 8.3[8.3] ms, 16.7 ms, 25.0 ms or 33.3 ms. The localization and discrimination tasks were displayed after the masks. (b) This figure demonstrates the tasks for an inverted human face at the left. These tasks followed the 300 ms masks. The question was presented as shown above. Participants were required to respond through the arrow keys on a QWERTY keyboard. The red dot illustrates the key that would have been calculated as a correct response.
a.
9
Data analyses
This study applied a within-group design. The dependent variable was either detection or localization. Orientation (upright or inverted), presentation time and species (human or chimpanzees) were independent variables.
Firstly, data was analyzed in MatLab R2019b. The goal was to compare discrimination properties to detection or localization of the stimuli. All three measures were converted to the Signal Detection Theory (SDT) measure d’ (d prime). This sensitivity d’ is calculated through the means of hit rates and false alarm rates. For discrimination, an ‘upright’ response was counted as a hit for upright faces, and as a false alarm for inverted faces. Similarly, for detection, a ‘yes’ response was counted as a hit for present faces, and as a false alarm for absent faces. Likewise, a ‘right’ response in the localization task was counted as a hit for faces that were presented at the right side of the fixation cross, and as a false alarm for faces presented at the left side. Subsequently, the hit and false alarm rates of 0 and 1 were converted to 1/(2N) and 1-1/(2N), with N representing the number of trials. For the calculation of d’, the hit rates and false alarm rates were z-transformed and thereafter the false alarm rate was subtracted from the hit rate. To compensate for the difference between the localization and detection task, the d’ for localization was divided by the square root of 2 (Macmillan & Creelman, 2005). Statistical analyses were performed in JASP (2019). For both human as well as chimpanzee faces, discrimination-d’ was analyzed with a one-way repeated measures ANOVA with the factor presentation time. This analyzed whether discrimination sensitivity increased with longer presentation times. Both detection-d’ and localization-d’ were likewise analyzed with a factorial repeated measures ANOVA with the factors orientation (upright or inverted) and presentation time. This ANOVA tested whether performance increased with presentation time, whether there was a face inversion effect, and whether the inversion effect differed as function of presentation time. To determine the longest presentation time for which discrimination-d’ was not significantly different from zero (zero-awareness criterion), a one sample students t-test was executed. For this presentation time, detection or localization of both face orientations were analyzed in a paired samples t-test. The latter was to investigate whether there is an unconscious inversion effect, which was the primary goal of this study.
Finally, to analyze whether detection mechanisms can operate with predisposition to conspecifics, the data of both experiment 1 as well as experiment 2 were analyzed with a three-way repeated-measures ANOVA. The factors were species, orientation and presentation time. This ANOVA tested the hypothesis that human faces were better detected than chimpanzee faces and furthermore, that the FIE was significantly larger for human faces than for chimpanzee faces.
RESULTS
The assumption of sphericity for a repeated measures ANOVA was tested with a Mauchly’s W test. If the assumption of sphericity was violated, the Greenhouse-Geisser correction was applied and corrected degrees of freedom are reported.
Experiment 1: detection-discrimination paradigm Human faces
As shown in Figure 5a, discrimination and detection sensitivity significantly increased with presentation time [F(1.64, 67.48) = 141.5, p < 0.001, ƞp² = 0.78; F(1.97, 80.66) = 139.64, p < 0.001, ƞp² = 0.77]. The sensitivity for detection of upright faces was significantly larger than for inverted faces, i.e. a significant inversion effect was found [F(1, 41) = 71.75, p > 0.001, ƞp² = 0.64]. Furthermore, a significant interaction effect between orientation and presentation time was shown, i.e. the inversion effect increased with presentation time [F(2.52, 103.42) = 22.39, p < 0.001, ƞp² = 0.35] (Figure 5a).
10
To test the hypothetical unconscious inversion effect, it was critical to analyze the longest presentation time for which discrimination-d’ was considered zero. Discrimination sensitivity did not significantly differ from zero for the presentation time of 8.3 ms [mean ± SD: 0.03 ± 0.32; t(41) = 0.5, p = 0.62, d = 0.08], but it was significantly above zero for 16.7 ms [0.31 ± 0.62; t(41) = 3.26, p = 0.002, d = 0.50], 25.0 ms [0.80 ± 0.83; t(41) = 6.23, p < 0.001, d = 0.96], and 33.3 ms [2.23 ± 1.08; t(41) = 14.04, p < 0.001, d = 2.17]. Therefore, to meet the zero-awareness criterion, the paired samples t-test was executed for the detection sensitivity of upright and inverted faces at 8.3 ms. Detection-d’ of upright and inverted faces was not significantly different for this presentation time [t(41) = 0.56, p = 0.58, d = 0.09], indicating no significant face inversion effect at 8.3 ms.
Chimpanzee faces
Similarly to the human faces, discrimination and detection sensitivity increased with presentation time [F(1.40, 57.38) = 43.40, p < 0.001, ƞp² = 0.51; F(1.56, 63.94) = 88.76, p < 0.001, ƞp² = 0.68] (Figure 5b). In addition, a significant inversion effect was found [F(1, 41) = 10.82, p = 0.002, ƞp² = 0.21]. Furthermore, the inversion effect increased with presentation time [F(3, 123) = 8.17, p < 0.001, ƞp² = 0.17] (Figure 5b).
For both the presentation times of 8.3 ms and 16.7 ms, discrimination-d’ was not significantly different from zero [mean ± SD: 0.04 ± 0.23; t(41) = 1.10, p = 0.28, d = 0.17; -0.003 ± 0.34; t(41) = -0.06, p = 0.95, d = -0.01]. However, beginning at 25.0 ms, it exceeded zero [0.27 ± 0.48; t(41) = 3.66, p < 0.001, d = 0.57]. For both the presentation times of 8.3 ms and 16.7 ms, no significant effect of face orientation was found, i.e. no inversion effect was demonstrated [t(41) = -1.14, p = 0.26, d = -0.18; t(41) = 1.54, p = 0.13, d = 0.24].
Fig. 5: Results of experiment 1
(a) This graph displays the group means of discrimination-d’, upright d’ and inverted detection-’d for human faces. These tasks were performed for four different presentation times. A significant face inversion effect was found. However, not for the critical presentation time of 8.3 ms. Error bars show 95% Confidence Intervals. (b) This graph displays the group means of discrimination-d’, upright detection-d’ and inverted detection-d’ for chimpanzee faces. It is represented for all four presentation times. A significant face inversion effect was found. However, not for the critical presentation time of 8.3 ms. Error bars show 95% Confidence Intervals.
No difference in inversion effects was found for the critical presentation time of 8.3 ms.
To test whether detection mechanisms can operate with predisposition to conspecifics, data was analyzed with a three-way repeated-measures ANOVA with the additional factor species. Overall, a significant effect of species was found [F(1, 41) = 18.81, p < 0.001, ƞp² =0.32]. This indicates that human faces were generally better detected than chimpanzee faces (Figure 5). The interaction effect
11
between species and orientation was also significant, indicating a larger face inversion effect for human faces than for chimpanzee faces [F(1, 41) = 34.19, p < 0.001, ƞp² = 0.46]. At the critical presentation time of 8.3 ms, a significant inversion effect was absent for both human as well as chimpanzee faces. Indeed, no significant interaction between species and orientation was found for this presentation time [F(1, 41) = 1.49, p = 0.23, ƞp² = 0.04]. This shows no significant difference between the FIE of human and chimpanzee faces at 8.3 ms.
Experiment 2: the localization-discrimination paradigm Human faces
As shown in Figure 6a, both discrimination-d’ as localization-d’ increased with presentation time [F(1.69, 49.10) = 88.28, p <0.001, ƞp² = 0.75; F(2.68, 77.79) = 188.57, p < 0.001, ƞp² = 0.87]. Localization-d’ was higher for upright faces than for inverted faces [F(1, 29) = 15.84, p < 0.001, ƞp² = 0.35]. Specifically, a significant face inversion effect was found. Additionally, the interaction between orientation and presentation time was significant [F(4, 116) = 6.39, p < 0.001, ƞp² = 0.18]. Figure 6a shows that the inversion effect is largest in the middle and disappears for the shortest and longest presentation times.
Discrimination properties did not differ significantly from zero for both 8.3 ms [mean ± SD: 0.06 ± 0.33; t(29) = 1.00, p = 0.33, d = 0.18], as well as 8.3[8.3] ms [0.12 ± 0.44; t(29) = 1.47, p = 0.15, d = 0.27]. At 16.7 ms, discrimination-d’ exceeded zero [0.53 ± 0.58; t(29) = 4.99, p < 0.001, d = 0.91]. To increase the probability of finding an unconscious inversion effect, it is critical to focus on the longest presentation time that meets the zero-awareness criterion. Therefore, the paired samples t-test was run for localization at 8.3[8.3] ms. For this presentation time, a significant face inversion effect was found [t(29) = 2.47, p = 0.02, d = 0.45]. Moreover, this effect was absent for the shortest presentation time [t(29) = -1.82, p = 0.08, d = -0.33].
Chimpanzee faces
Discrimination sensitivity and localization sensitivity increased with presentation time [F(1.60, 46.33) = 44.55, p < 0.001, ƞp² = 0.61; F(2.46, 71.39) = 228.59, p < 0.001, ƞp² = 0.89] (Figure 6b). No significant difference between the localization of upright and inverted faces was found, i.e. the face inversion effect was not presented [F(1, 29) = 6.97E-7, p = 0.99, ƞp² = 0.00]. Likewise, no significant effect of interaction was found for orientation and presentation time, indicating that the FIE was absent for all presentation times [F(4, 116) = 0.47, p = 0.76, ƞp² = 0.02].
Discrimination-d’ was not significantly above zero for the presentation times of 8.3 ms [mean ± SD: -0.06 ± 0.27; t(29) = -1.28, p = 0.21, d = -0.23], 8.3[8.3] ms [-0.01 ± 0.27; t(29) = -0.19, p = 0.85, d = -0.04], and 16.7 ms [0.10 ± 0.35; t(29) = 1.55, p = 0.13, d = 0.28]. The inversion effect was absent for all individual presentation times [p > 0.14].
12
Fig. 6: Results of experiment 2
(a) This graph displays the group means of discrimination-d’, upright localization-d’ and inverted localization-’d for human faces. These tasks were performed for five different presentation times. A significant face inversion effect found for the critical presentation time of 8.3[8.3] ms. Error bars represent 95% Confidence Intervals. (b) This graph displays the group means of discrimination-d’, upright localization-d’ and inverted localization-’d for chimpanzee faces. Results are represented for all five presentation times. The face inversion effect was absent for all individual presentation times. Error bars show 95% Confidence Intervals.
The face inversion effect was significantly larger for human faces at the critical presentation time of 8.3[8.3] ms.
To test whether detection mechanisms can operate with predisposition for conspecifics, data was analyzed with a three-way repeated-measures ANOVA with the additional factor species. Generally, human faces were better localized than chimpanzee faces, i.e. a significant effect of species was found [F(1, 29) = 11.18, p = 0.002, ƞp² = 0.28]. In addition, the interaction effect between species and orientation was also significant, indicating a larger face inversion effect for human faces than for chimpanzee faces [F(1, 29) = 9.44, p = 0.005, ƞp² = 0.0.25]. For the critical presentation time of 8.3[8.3] ms, a significant face inversion effect was only presented for human faces. Indeed, a repeated-measures ANOVA with the factors species x orientation for 8.3[8.3] ms, showed a significant interaction effect [F(1, 29) = 39.43, p < 0.001, ƞp² = 0.58]. Thus, the inversion effect was significantly larger for human faces than for chimpanzee faces at 8.3[8.3] ms (Figure 6).
DISCUSSION
In this paper, we studied unconscious processing through the category-specific face inversion effect for both neutral human and neutral chimpanzee faces. To do so, a novel dissociation paradigm for determining unconscious processing was developed. The novel paradigm can distinguish between unconscious and conscious contributions to stimuli detection, of which the widely adopted b-CFS paradigm is not capable. Overall, an unconscious inversion effect was found specific for human faces. More specifically, in experiment 1, a significant inversion effect was found for both human as well as chimpanzee faces. However, the inversion effect was significantly larger for human faces. As for the critical presentation time, the zero-awareness criterion was established at the presentation time of 8.3 ms. Nonetheless, at 8.3 ms, the inversion effect was absent for human as well as for chimpanzee faces. Furthermore, in experiment 2, a significant inversion effect was present only for human faces. Moreover, at the presentation time of 8.3[8.3] ms, this effect was presented unconsciously as well. In summary, upright human faces are significantly better localized than inverted human faces, also in absence of awareness.
a.
b.
13
In this study, a new approach to studying unconscious perception was developed. The current findings show that the new localization-discrimination dissociation approach can be applied successfully. Our approach can be adopted to test whether differences in stimulus detectability, as measured with paradigms such as b-CFS, reflect conscious or unconscious processing differences. In other words, the novel approach can truly isolate unconscious processes. Furthermore, this paradigm allows the comparison of detection and awareness on the same scale, which is essential for evident conclusions on unconscious processing (Franz, & von Luxburg, 2015). Finally, as discussed in the introduction, this paradigm applies to both dichotomous as gradual theories on consciousness, therefore avoiding theoretical complications.
Nonetheless, evidence for unconscious processing of faces was only found through the localization-discrimination dissociation paradigm. The detection-discrimination dissociation did not reveal evidence, for various potential reasons. First of all, the critical presentation time of 8.3[8.3] ms as found in experiment 2, was not included in experiment 1. This could imply that the presentation times were not optimally set, such that 8.3 ms was too confined to establish unconscious processing, and 16.7 ms was too extended to meet the zero-awareness criterion. Secondly, the detection-discrimination dissociation is more sensitive to implicit and explicit biases in subject responses. More specifically, sensitivity scores were calculated by subtracting the false alarm rates from the hit rates to control for participants that would constantly report that, for example, faces were presented upright (see method section). However, in experiment 1, faces were present in two third of the trials and therefore not equal to the face absent trials. Subsequently, if participants would constantly report that faces were present, the sensitivity calculation would be biased as the hit rates would be larger than the false alarm rates. Contrary, in experiment 2, faces were equally presented at the right as at the left side, which prevents such bias. Finally, the evidence for an overall dissociation between discrimination and detection may be less explicit. Specifically, the visual resolution of the retina is highest in the fovea (the centre), due to higher photoreceptor density (Rossi & Roorda, 2010). Therefore, images presented in the centre of the visual field are generally easier discriminated (Shah, Dakin & Anderson, 2012). As a consequence, the dissociation between detection and discrimination might therefore decrease. Experiment 2 may yield larger dissociations, as stimuli are presented in the periphery instead of the centre of the visual field. In the periphery, discrimination thresholds are higher than detection thresholds (Shah et al., 2012). The localization-discrimination dissociation may therefore present larger effects.
Nonetheless, experiment 2 likewise has limitations. A significant inversion effect was found for human faces at 8.3[8.3] ms. This led to the conclusion that unconscious processing was established at this presentation time. However, the evidence for the zero-awareness criterion was anecdotal. Specifically, when we replaced the students one sample t-test with a Bayesian one sample t-test for the presentation time of 8.3[8.3] ms, an inversed Bayes Factor of 1.96 was given [BF01 = 1.96]. This BF01 indicates that the null hypothesis is only 1.96 times more likely than the alternative hypothesis. In other words, that the likeliness of the stimuli to have met the zero-awareness criterion was only 1.96 times larger than the likeliness that the stimuli were perceived consciously. Moreover, a Bayesian paired samples t-test for the critical presentation time of 8.3[8.3] ms, resulted in a Bayes Factor of 2.58 [BF10 = 2.58]. This BF10 factor indicates that the alternative hypothesis is 2.58 times more likely than the null hypothesis. Essentially, the presence of a significant face inversion effect was 2.58 times more likely than no present face inversion effect. Bayesian statistics can be very useful to analyze how evident the finding is (Woertman, Groenewoud & van der Wilt, 2014). Bayes Factors between 1 and 3 are considered anecdotal evidence, which indicates that these values are not conclusive. To increase the evidence, a larger sample size is needed. Indeed, experiment 2 had a small sample size of 30 subjects. Future research should replicate this study with bigger sample sizes for more evident effects.
The results of this study provide essential information for the debate on the mechanism of visual integration. As inversion is expected to disrupt image integration, the face inversion effect is thought to function as an indicator of visual integration. Due to variance caused by the b-CFS paradigm, it remained uncertain whether consciousness was required for visual integration (Sklar, Douell & Hassin, 2018).
14
This study provides evidence for unconscious high-level processing on a perception-integration level. Therefore presenting evidence for visual integration independent of awareness. Moreover, the current results only show unconscious face inversion effects for human faces, which proposes that the unconscious integration may be specific to conspecifics. This may reveal a prioritized pathway for the integration of, for example, social relevant stimuli. Future research should determine whether unconscious integration is limited to faces, whether it extends to other familiar and social relevant images, or even to other object categories (Stein & Peelen, 2020).
Furthermore, the current results can contribute to models of face detection. For example, a potential neurophysiology of face detection is based on the notion that some information of unconscious images is sufficient enough to reach high-hierarchy brain areas, such as the fusiform face area (FFA) (Jiang et al., 2007). Earlier, the right FFA, right superior temporal sulcus (STS) and amygdala responded to fearful and neutral faces that were rendered invisible through interocular suppression (Jiang & He, 2006). Furthermore, the cortical responses to invisible neutral and invisible fearful faces were dissociable. This was hypothesized to reflect primarily feed-forward processing, which was specialized for dissociating facial identity and expression information (Jiang & He, 2006). The results of this study are comprehensive with the idea that some subliminal stimulus information is preserved in high-level cortical regions. Moreover, our results confirm the notion that this unconscious processing can be very specific, suggesting that facial identity might be specialized for conspecifics. However, further neuroimaging research is needed to confirm this hypothesis.
Additionally, unconscious face processing is proposed to be generated by rapid cell firing in a feedforward sweep (Lamme & Roelfsema, 2000). It is hypothesized that the cell firing is not disrupted by mask suppression, which facilitates unconscious high-level processing. The results of this study are consistent with this suggested model of face detection. Moreover, they add to the discussion that the feedforward sweep might be very selective for conspecifics. Further research is needed to determine the tuning properties of these cells. For example, they may possibly be tuned to ecological or social relevant stimuli.
Furthermore, rapid and unconscious processing of faces is thought to reflect a face-specific template in the brain (Lewis & Ellis, 2003). Generally, the idea is that faces are detected by matching the visual input to a specific face template. The template is thought to mismatch inverted faces, which causes the FIE. Stein, Peelen and Sterzer (2011) found evidence for a FIE for complex face images as well as very schematic representations (3 dark blobs). Therefore, they concluded that face templates might also respond to very schematic and unspecific face properties, such as basic contrast relations. Similarly, Stein and colleagues (2012) found face inversion effects for both human as chimpanzee faces. However, the effects were significantly larger for human faces. Thus, face templates might generalize to other species, however, are optimally tuned for conspecifics. Inconsistently, Stein, End and Sterzer (2014) found significant larger face inversion effects for faces of own race and age, indicating rather specific face templates. The current study exclusively found an unconscious face inversion effect for human faces, likewise promoting very specific face templates with privileged detection of conspecifics. For example, the template might respond solely to specific skin texture or to eyes being darker compared to the rest of the face, which is characteristic for human and not for chimpanzee faces. Moreover, these face templates might be linked to neural processing in the feedforward sweep. For example, the cell firing of the feedforward sweep might be tuned specifically to the face template. However, as the findings on category-specificity of different studies are not consistent, replicating studies are needed to further study the category-specificity of unconscious processing.
In addition, the results of this study could translate to a broader, clinical level as well. For disorders such as blindsight or DoC, research into unconsciousness may reveal essential neuronal correlates useful for novel research and treatments. For example, research into subliminal perception can be used to map the neuronal correlates of unconscious and conscious processing. Subsequently, the neural correlates of unconscious and conscious processing can be useful to detect consciousness
15
in DoC patients. Defining conscious neuronal responses in DoC patients could be essential for the diagnosis of minimally conscious state (MCS) or vegetative state (VS) (Bekinschtein et al, 2009). An accurate diagnoses is essential for the treatment and therefore the life expectancy of patients (Naci et al., 2012). Since the novel dissociation paradigm successfully isolates unconscious processing, a combination with neuroimaging studies may be effective to map neuronal correlates of unconscious and conscious processing.
Some suggestions for future research have already been briefly described above. The main goal of future research is to investigate the category-specificity of unconscious processing on a neurological level. These studies might provide essential information on how selective unconscious processing is, i.e. the specificity of the feedforward sweep, the face template and visual integration. More specifically, future research should predominantly focus on neuroimaging studies to map the differences between unconscious processing of human and chimpanzee faces. Previous neuroimaging studies have mostly adopted interocular suppression, which resulted in high heterogeneous findings (Sterzer, Stein, Ludwig, Rothkirch & Hesselmann, 2014). With the novel backward masking paradigm, more robust findings could be provided. For example, human as well as chimpanzee faces could be rendered invisible through backward masking. The quantity of FFA activation could be determined through fMRI. The FFA response should be compared between human and chimpanzee faces. If the FFA activity only increases for subliminal human faces, unconscious processing is specific to conspecifics. Furthermore, such category-specific unconscious processing implies that subliminal information can be processed very specifically, i.e. in very high-hierarchy brain areas.
In summary, the current findings demonstrate that the novel localization-discrimination dissociation paradigm is a promising paradigm which can successfully distinguish between conscious and unconscious perception. Furthermore, this paradigm presented high-level unconscious processing of face orientation. Moreover, this finding was specific to human faces. However, according to Bayesian statistics, the current findings are categorized as anecdotal evidence. Therefore, future studies should firstly try to replicate these findings with larger sample sizes. Thereafter, future research could focus on revealing the neuronal mechanisms of unconscious face processing through neuroimaging studies. These results may have great implications for the understanding of unconscious and conscious processes such as visual integration.
16
REFERENCES
1. Axelrod, V., Bar, M., & Rees, G. (2015). Exploring the unconscious using faces. Trends in Cognitive Sciences, 19(1), 35-45. doi:10.1016/j.tics.2014.11.003.
2. Bekinschtein, T. A., Dehaene, S., Rohaut, B., Tadel, F., Cohen, L., & Naccache, L. (2009). Neural signature of the conscious processing of auditory regularities. Proceedings of the National Academy of Sciences of the United States of America, 106(5), 1672–1677. doi:r 3. Brogaard, B., Marlow, K., & Rice, K. (2014). Unconscious influences on decision making in
blindsight. Behavioral and Brain Sciences, 37(1), 22-23. doi:10.1017/S0140525X13000654 4. Bourdillon, P., Hermann, B., Sitt, J. D., & Naccache, L. (2019). Electromagnetic Brain
Stimulation in Patients With Disorders of Consciousness. Frontiers in neuroscience, 13, 223. doi:10.3389/fnins.2019.00223
5. Danckert, J., Tamietto, M., & Rossetti, Y. (2019). Definition: Blindsight. Cortex, 119, 569-570. doi: 10.1016/j.cortex.2019.01.027
6. Eapen, B.C., Georgekutty, J., Subbarao, B., Bavishi, S., & Cifu, D.X. (2017). Disorders of Consciousness. Physical Medicine and Rehabilitation Clinics of North America, 28(2), 245-258. doi: 10.1016/j.pmr.2016.12.003
7. Ekman, P. & Friesen, W.V. (1976). Pictures of facial affect. Palo Alsto, CA: onsulting Psychologists Press
8. Emmanouil, T. A., Burton, P., & Ro, T. (2013). Unconscious processing of unattended features in human visual cortex. Journal of cognitive neuroscience, 25(3), 329–337. doi:10.1162/jocn_a_00320
9. Erdelyi, M. (1986). Experimental indeterminacies in the dissociation paradigm of subliminal perception. Behavioral and Brain Sciences, 9(1), 30-31. doi:10.1017/S0140525X00021348
10. Franz, V. H., & von Luxburg, U. (2015). No Evidence for Unconscious Lie Detection: A Significant Difference Does Not Imply Accurate Classification. Psychological science, 26(10), 1646–1648. https://doi.org/10.1177/0956797615597333
11. van Gaal, S., & Lamme, V. A. F. (2012). Unconscious High-Level Information Processing: Implication for Neurobiological Theories of Consciousness. The Neuroscientist, 18(3), 287–301. doi:10.1177/1073858411404079
12. Gayet, S., Van der Stigchel, S., & Paffen, C.L.E. (2014). Breaking continuous flash suppression: competing for consciousness on the pre-semantic battlefield. Frontiers in Psychology, 5, 460. doi:10.3389/fpsyg.2014.00460
13. Hassin, R. R. (2013). Yes It Can: On the Functional Abilities of the Human Unconscious. Perspectives on Psychological Science, 8(2), 195–207. doi:10.1177/1745691612460684 14. Hesselmann, G., & Moors, P. (2015). Definitely maybe: can unconscious processes perform the
same functions as conscious processes? Frontiers in Psychology. doi:10.3389/fpsyg.2015.00584
17
15. Ionescu M. R. (2016). Subliminal perception of complex visual stimuli. Romanian journal of ophthalmology, 60(4), 226–230.
16. Jiang, Y., & He, S. (2006). Cortical responses to invisible faces: dissociating subsystems for facial-information processing. Current biology, 16(20), 2023–2029. doi:10.1016/j.cub.2006.08.084
17. Jiang, Y., Costello, P., & He, S. (2007). Processing of Invisible Stimuli: Advantage of Upright Faces and Recognizable Words in Overcoming Interocular Suppression. Psychological Science, 18(4), 349–355. doi:10.1111/j.1467-9280.2007.01902.x
18. Lamme, V. A., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 23(11), 571–579. doi:10.1016/s0166-2236(00)01657-x
19. Lewis, M.B. and Ellis, H.D. (2003), How we detect a face: A survey of psychological evidence. Int. J. Imaging Syst. Technol., 13, 3-7. doi:10.1002/ima.10040
20. Lupyan, G., & Ward, E. J. (2013). Language can boost otherwise unseen objects into visual awareness. Proceedings of the National Academy of Sciences of the United States of America, 110(35), 14196–14201. doi:10.1073/pnas.1303312110
21. Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide (2nd ed.). Lawrence Erlbaum Associates Publishers
22. Nakamura, K., & Kawabata, H. (2018). Preferential access to awareness of attractive faces in a breaking continuous flash suppression paradigm. Consciousness and Cognition, 65, 71-82. doi:10.1016/j.concog.2018.07.010.
23. Naci, L., Monti, M.M., Cruse, D., Kübler, A., Sorger, B., Goebel, R., Kotchoubey, B., & Owen, A.M. (2012), Brain–computer interfaces for communication with nonresponsive patients. Ann Neurol., 72, 312-323. doi:10.1002/ana.23656
24. Rossi, E. A., & Roorda, A. (2010). The relationship between visual resolution and cone spacing in the human fovea. Nature neuroscience, 13(2), 156–157. Doi:10.1038/nn.2465
25. Shah, N., Dakin, S. C., & Anderson, R. S. (2012). Effect of optical defocus on detection and recognition of vanishing optotype letters in the fovea and periphery. Investigative ophthalmology & visual science, 53(11), 7063–7070. doi:10.1167/iovs.12-9864
26. Shanks, D.R. (2017). Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processes. Psychon Bull Rev 24, 752–775. doi:10.3758/s13423-016-1170-y
18
27. Sklar, A.Y., Deouell, L.Y., & Hassin, R.R. (2018). Integration despite Fractionation: Continuous Flash Suppression. Trends in Cognitive Sciences, 22(11), 956-957. doi:10.1016/j.tics.2018.07.003.
28. Solcà, M., Guggisberg, A. G., Schnider, A., & Leemann, B. (2015). Facial blindsight. Frontiers in human neuroscience, 9, 522. doi:10.3389/fnhum.2015.00522
29. Song, C., & Yao, H. (2016). Unconscious processing of invisible visual stimuli. Scientific Reports, 6, 38917. doi:10.1038/srep38917
30. Stafford T. (2014). The perspectival shift: how experiments on unconscious processing don't justify the claims made for them. Frontiers in Psychology, 5, 1067. doi:10.3389/fpsyg.2014.01067
31. Stein, T. (2019). The breaking continuous flash suppression paradigm: Review, evaluation, and outlook. In G. Hesselmann (Ed.), Transitions between Consciousness and Unconsciousness. Routledge. http://www.timostein.net/uploads/9/9/8/0/99806454/stein_b-cfs_review_hesselmann_book_2019.pdf
32. Stein, T., Awad, D., Gayet, S., & Peelen, M. V. (2018). Unconscious processing of facial dominance: The role of low-level factors in access to awareness. Journal of Experimental Psychology: General, 147(11), e1–e13. doi:10.1037/xge0000521
33. Stein, T., End, A., & Sterzer, P. (2014). Own-race and own-age biases facilitate visual awareness of faces under interocular suppression. Frontiers in human neuroscience, 8, 582. doi:10.3389/fnhum.2014.00582
34. Stein, T., & Peelen, M.V. (2020) Dissociating conscious and unconscious influences on visual detectoin. Manuscript submitted for publication.
35. Stein, T., Peelen, M. V., & Sterzer, P. (2011). Adults' awareness of faces follows newborns' looking preferences. PloS one, 6(12), e29361. doi:10.1371/journal.pone.0029361
36. Stein, T., Sterzer, P., & Peelen, M.V. (2012). Privileged detection of conspecifics: Evidence from inversion effects during continuous flash suppression. Cognition, 125(1), 64-79. doi: 10.1016/j.cognition.2012.06.005
37. Stein, T., Utz, V., & van Opstal, F. (2020). Unconscious semantic priming from pictures under backward masking and continuous flash suppression. Consciousness and Cognition, 78. doi:10.1016/j.concog.2019.102864.
38. Sterzer, P., Stein, T., Ludwig, K., Rothkirch, M., & Hesselmann, G. (2014). Neural processing of visual information under interocular suppression: a critical review. Frontiers in psychology, 5, 453. doi:10.3389/fpsyg.2014.00453
19
39. Weiskrantz, L., Barbur, J. L., & Sahraie, A. (1995). Parameters affecting conscious versus unconscious visual discrimination with damage to the visual cortex (V1). Proceedings of the National Academy of Sciences of the United States of America, 92(13), 6122–6126. doi:10.1073/pnas.92.13.6122
40. Woertman, W. H., Groenewoud, H. M., & van der Wilt, G. J. (2014). Bayesiaanse statistiek: wat, hoe en waarom? [Bayesian statistics: what, how and why?]. Nederlands tijdschrift voor geneeskunde, 158, A7485.
41. Yang, Y. H., & Yeh, S. L. (2018). Can emotional content be extracted under interocular suppression?. PloS one, 13(11), e0206799. doi:10.1371/journal.pone.0206799
42. Zhou, G., Zhang, L., Liu, J., Yang, J., & Qu, Z. (2010). Specificity of face processing without awareness. Consciousness and Cognition: An International Journal, 19(1), 408–412. doi:10.1016/j.concog.2009.12.009
