1
Thesis Article
Format: Journal of Neuroscience First Draft
Reseach Master Psychology UvA By: Irene Graafsma
Student number: 10169865 Supervisor UvA: Simon van Gaal Second corrector UvA: Heleen Slagter
Daily supervisor Monash University: Naotsogu Tsuchiya
2 The effect of perceptual fading in Troxler fading on SSVEP.
Abstract (234 words/250 max)
To resolve the disagreement on whether attention and consciousness are identical (De Brigard & Prinz, 2010; Mole 2008; Posner, 1994; Merikle & Joordens, 1997; O’Regan. & Noe, 2001) or distinct processes (Koch & Tsuchiya, 2007; Lamme, 2003; van Boxtel et al., 2010), it is necessary to study both processes in dissociation. Previous research has shown that Troxler fading might allow for a dissociation because focused attention seems to lead to a decrease in conscious perception (Lou, 1999). The current research has investigated the possibility to use the Troxler fading illusion in relation to Electroencephalography (EEG) measurements. During this experiment participants were presented with a Troxler fading report task with four flickering peripheral stimuli and a dynamic moving background. Results showed clear frequency tagging for the activity of the dynamic background. When studying this background activity in relation to perceptual disappearances expectations were for background activity to increase during disappearances, as the targets are perceived to be replaced by the background, thereby increasing background perception across the screen. Results suggested that this was indeed the case, and that background related activity increased with more disappearing targets. Localization in the occipital region of these effects suggests that these are visual effects rather than motor or cognition related effects. The fact that perceptual disappearances can be recognized in brain activity offers possibilities to use this task to test possible dissociations between attention and consciousness in the future.
Significance statement (109 word/120 max)
Peripheral fading might be the way to study the difference between the attention and consciousness , as previous research has shown that it might allow for a dissociation between these two factors, with focused attention leading to a decrease in conscious perception. However, peripheral fading has only been tested behaviorally. We show that effects of conscious perception can be recognized in EEG brain signal through their effects on activity elicited by a dynamic background. Additionally, we show that EEG activity elicited by the dynamic background increases with the disappearance of more peripheral targets. This allows us to make quantitative rather than just qualitative descriptions of perception in EEG .
3 Introduction (624 words/650max)
There is still disagreement on the relationship between attention and consciousness, with some researchers arguing that they are identical processes (De Brigard & Prinz, 2010; Mole 2008; Posner, 1994; Merikle & Joordens, 1997; O’Regan. & Noe, 2001), while others argue that they are distinct processes (Koch & Tsuchiya, 2007; Lamme, 2003; van Boxtel, Tsuchiya and Koch, 2010). In order to resolve this, it is important to look at the effects of attention and consciousness separately. As van et al (2010) explain, previous studies have mostly tried this by manipulating either attention or consciousness separately. In those studies they could not control for attention while looking at the effects of consciousness and vice versa, meaning that the influence of one factor could not be excluded when concluding effects of the other factor. This is why van Boxtel et al. (2010) set up a first behavioral experiment that could control for one factor while manipulating the other through the use of a task where the processes can be dissociated. The results suggested that attention and consciousness are likely to be distinct processes.
Dissociation as shown by van Boxtel et al (2010) have not yet been studied extensively, and much still remains unclear on the characteristics of such a dissociation. The current study expands our understanding of such dissociation is two ways. Firstly, it studies a similar dissociation in a new paradigm, namely that of Troxler fading. This is the effect where fixating on a central point leads to the temporary disappearance of stimuli in the periphery (Martinez-Conde, Macknik, Troncoso & Dyar, 2006). Lou (1999) showed that increased attentional focus lead to more disappearances in Troxler fading. In other words, an increase in attention leads to a decrease in consciousness. This provides conditions where attention is focused while conscious perception is absent and vise versa, allowing to study one process while controlling for the other process.
Secondly, the current study relates perceptual disappearances to activity in Electroencephalography (EEG). Previous research has shown that both attentional focus (Morgan, Hansen, and Hillyard, 1996) and conscious perception (Brown & Norcia 1997; Zhang, Jamison, Engel, He, & He, 2011) can influence target related steady-state visual evoked potential (SSVEP). SSVEP can be obtained by presenting flickering stimuli and relating their flicker frequency to the oscillatory response in the EEG (Regan, 1989, Morgan, Sutoyo & Srinivasan, 2009). The amplitude of the SSVEP then reflects the extent to which the stimulus is being processed (Proverbio, 2003). The current study attempted to obtain an SSVEP during a Troxler fading task by presenting flickering peripheral stimuli, as several studies have shown that flickering stimuli still disappear during Troxler fading (Anstis, 1996; Schieting & Spillmann, 1987). The pilot process preceding the current experiment showed that better SSVEP signals were obtained for the dynamic background of a Troxler fading task than for flickering peripheral targets. Therefore the final experiment focused on the SSVEP activity of the dynamic background in the analysis.
During target disappearances with a dynamic background, observers perceive the target area to be filled in by the background pattern. This is known as perceptual filling in (Ramachandran, Gregory, & Aiken, 1993). Before the current study no research had looked at the SSVEP activity of a dynamic background during perceptual filling in. We hypothesize that background related SSVEP increases when targets disappear and decreases when targets reappear, as background perception increases to fill in the area where the target was previously perceived. We designed a Troxler fading report task with four peripheral flickering targets in order to study the effects of different numbers of disappearing targets. With regard to the number of disappearing targets we hypothesize dynamic background SSVEP to be higher when more targets seem to disappear, as the background then fills in a larger area.
4 Materials and Methods
Participants
Twenty-nine healthy volunteers (eleven male, eighteen female, 18-39 years of age, mean ± SD age, 24.6 ± 5.4) took part in the study. Participants had normal or corrected to normal vision and were ensured not to have any medical sensitivity to flickering stimuli. Participants gave written informed consent and received a monetary compensation (30 AUD) for their participation. The ethics committee of Monash University approved the study.
Pilot procedure
The current study attempted to use stimuli that would elicit SSVEP activity, while still disappearing during Troxler fading. Previous research has shown that smaller targets in the periphery with a high flicker rate and low contrast tend to disappear better during Troxler fading (Anstis, 1996). However, large targets in the centre of eye focus with low flicker rates and high contrast tend to elicit the clearest SSVEP signals (Müller-Putz, Eder, Wriessnegger, & Pfurtscheller, 2008; Zhu, Bieger, Molina, & Aarts, 2010). During the pilot phase different stimuli were tested but no stimulus was found to elicit a clear SSVEP signal.
Therefore the last pilot phase increased target size, contrast, eccentricity, and checkerboard pattern. To ensure that such a target would still disappear a dynamic background was added, as this is known to facilitate target fading (Spillmann & Kurtenbach, 1992). Pilot results showed that targets did still not elicit a clear SSVEP signal. However, a strong SSVEP signal was found for the background activity. These findings have formed the basis of the current experimental and analysis setup.
Stimuli and procedure
The experiment was conducted in a dark room, with the participant seated at 50 cm distance from the computer screen (29 by 51 cm, 1080 by 1920 pixels, refresh rate of 60Hz). The stimuli for this experiment were presented with Matlab using psychtoolbox, and were based on scripts from a previous pilot study. A still image of the experiment can be seen in Figure 1.
The experiment used a dynamic background. Prior to the start of the experiment 100 random
backgrounds were computed. Each background was made up by squares of two by two pixels where the luminance of each square was set randomly in the range from black to white. During the experiment a new background was randomly selected from this set of 100 backgrounds at a rate of 20Hz. This resulted in a noisy random background, as can be seen in Figure 1.
Four circular stimuli were presented, one in every quadrant of the screen at 0.23 degree of visual angle (dva) diagonally from the centre of the screen (216 pixels up or down and 384 pixels left or right, 11.7 cm from the centre). Targets had a two by two checkerboard layout and a diameter of 150 pixels. They had transparent Gaussian edges (Sd=35 pixels) so that they blended into the dynamic background. During each trial the targets flickered at four different flicker rates: 8, 13, 15 and 18 Hz. The luminance profile of each target was modulated with a sine-wave profile at one of the four frequencies. The positions of the targets with the different flicker rates was randomized over trials. Results showed no frequency tagging for the target flicker rates, therefore further analysis in this study focused on activity related to the dynamic background.
Participants were presented with 24 trials, each with the duration of one minute. Each one minute trial contained one catch event meant to check for the accuracy of the participant’s report during that trial. During this catch event one or several stimuli physically disappeared simultaneously for a duration of 3.5 to 6.5 seconds. Catch events never appeared within the first or last ten seconds of a trial. The moment and duration of disappearance and the number and position of disappearing stimuli during the catch was
5 randomized over trials. A catch was defined as missed if the participant failed to press one or several of the buttons corresponding to a disappearing target. Participants were not informed about the presence of these catch events.
During each trial participants were instructed to maintain eye focus on a fixation cross in the middle of the screen (15 pixels long and wide, line width of 4 pixels). They were instructed to keep the middle finger and the index finger of each hand on the four keyboard buttons corresponding to the four targets on the screen: ‘A’ for the top left target, ‘Z’ for the bottom left target, ‘K’ for the top right target and ‘M’ for the bottom right target. They were instructed that if the eyes remained still enough on the fixation cross, they would sometimes experience a visual illusion where one or several targets would disappear from
consciousness. They were instructed to press the button corresponding to a target when that target disappeared, and to hold the button until the target reappeared.
Figure 1: The layout of the experiment screen. It contains a fixation cross in the middle, a dynamic moving background that changes at a rate of 20Hz and four flickering checkerboard stimuli, each flickering with a different frequency (8, 13, 15 or 20 Hz).
Analysis
Behavioral data analysis
Behaviorally, previous research suggests that stimuli with a higher flicker frequency should disappear more than stimuli with a lower flicker frequency (Anstis, 1996). The behavioral analysis looked at the number of disappearances per trial, the total duration of disappearance per trial, and the average length of
disappearances per trial as dependent variables. Flicker rate and number of disappearing targets were taken as the independent variables. These effects were analyzed with repeated measure ANOVA’s.
The observed response patterns suggested that the disappearances of different targets interacted within trials. To further investigate whether the observed response patterns could also occur if target
disappearances were independent, an explorative permutation analysis was performed. For this analysis 200 shuffled trials were created for each participant. These trials were obtained by selecting each button response randomly from one of the 24 available trials within the participant. This meant that, for example, the top left button response could be obtained from trial 1 while the bottom right button response was obtained from trial 9. These random trials for all participants were used to compute figures for the number of disappearing targets.
Electrophysiological recordings and analysis
The EEG data was recorded with 64 active electrodes arranged across an elastic scalp cap (BrainVision Acticap). The impedances of the electrodes were kept below 10 KΩ. The Brainvision recorder software was used to record the activity. For this a sampling rate of 1000Hz was used and data were filtered with a bandpass of 0.531-70Hz. Data analysis was performed in Matlab with use of the EEGLab (Delorme &
6 Makeig, 2004), chronux (Mitra & Bokil, 2008) and custom-written scripts from previous experiments that were adjusted for analysis of the current study. During the initial recording the default reference and ground electrodes were used to record data. During later analysis the data was re-referenced per time bin with the mean activity of all electrodes.
To obtain the overall frequency spectrum, a multi-taper spectrum analysis (Mitra & Bokil, 2008) was performed with the time-bandwidth product set to 5 and the number of tapers to 9. This power spectrum over all time points was used to compute the frequency spectrum and topographical plots to check for overall frequency tagging and localization of the tagged signal.
The spectrogram (Mitra & Bokil, 2008) was computed with a moving window of 1 taper and a step size of 0.15 seconds. Both the time-bandwidth product and the number of tapers to be used were set to 1, resulting in a half bandwidth of 1. The power spectra were converted to a logarithmic scale
(10*log10(spectrogram)).
All further analysis looked at the signal to noise ratio (SNR). The SNR was calculated by using the signal at the target frequency bin as the signal, and the signal in neighboring frequency bins as noise. Choosing nearby frequency bins to compute the noise allowed to control for general trends in that frequency area. The size of the frequency bins used to compute the SNR was 0.9766. The SNR was computed by
averaging the activity from the second, third and fourth frequency bins above and below the frequency bin of the target frequency. This corresponded to the frequencies from 16.63 to 17.58 Hz as the noise below the signal, and 21.48 to 23.44 Hz as the noise above the signal. The mean of these two samples was then subtracted from the activity at the target frequency on the logarithmic scale. The signal to noise ratio does not have corresponding units in the figures.
To study the effects around the button presses, SNR values were adjusted by aligning the average SNR from three to two seconds before button press or the two to three seconds after button release with zero and subtracting that activity from the other time points. This was done for each disappearance before averaging within and over participants. This corrected for the different overall levels of SNR during different disappearances, and therefore would show an effect more clearly than without this correction. For the computation of error bars over time, the error was corrected for between participant differences in the way as described by Cousineau (2007). This enhanced the visual effects by eliminating overall differences in SNR between participants.
Lastly, whenever multiple t-tests or ANOVAs were performed, a threshold of the false discovery rate was computed with for a false discover rate of 0.05 (Benjamini, Krieger, & Yekutieli, 2006).
Results Behavioral results
To visualize the nature of the behavioral data obtained during this study, Figure 2 shows an illustration of the behavioral results as obtained per trial. For this figure, six exemplar trials from six participants were selected. The images show the recorded button press activity during the trial. Button presses are recorded per position, and the sum of the buttons pressed at each time is represented in the top row. These images do not show the information on the flicker frequencies of the target. The positions of the different flicker rates differed with every trial.
7 Figure 2: Each panel in this figure shows the button press response during a trial. For this six random trials from six random participants were selected. It illustrates the button presses per button position: Top Left (TL), Top Right (TR), Bottom Left (BL) and Bottom Right (BR). The top row (All) of each panel shows the total number of buttons pressed at every time point. The red lines in the figures indicate the catch moments, where the target physically disappeared. The moment of disappearance and the number and position of disappearing targets differed with each trial. These figures suggest that targets often disappeared at the same time.
Analyzing report of the catch trials showed that participants were fairly accurate in their report of the disappearances, as the average percentage of accurately reported catches was 87.1 percent (on average participants missed 3.10 out of 24 catches, Sd=2.68). From previous research we expect stimuli with higher frequencies to disappear more frequently and longer (Anstis, 1996). Three repeated measures ANOVAs were used to investigate effects of frequency on the number of disappearances, total
disappearance duration and duration per disappearance. For all three tests Mauchly’s test indicated that the assumption of sphericity was violated. Therefore a Greenhouse –Geisser correction was used to correct the degrees of freedom. The analysis showed a significant increase in number of disappearances per trial (F(1.17, 32.70)=12.41, p<0.01) total duration of disappearance per trial (F(1.12, 31.37)= 24.10, p<0.01) and average duration per disappearance per trial (F(1.14,31.81) = 11.38, p=0.001) for increased flicker rates, confirming expectations (Figure 3 a-c).
Further analysis on the number of disappearing targets showed that disappearances with a larger number of disappearing targets happened less often than disappearances with fewer disappearing targets
(F(3,84)=36.1, p<0.01). A repeated measure ANOVA showed that disappearances lasted longer in total (F(2,56)=35.73, p<0.01) and longer per disappearance if more targets disappeared (F(2,56)=7.92, p=0.004).
8 Figure 3. Button press analysis in the top row show the effects of the different flicker frequencies of the targets on target
disappearance. In each of these panels the horizontal axes indicate the frequencies. The vertical axis indicate the average number of disappearances during a one minute trial, the average total disappearance duration of a target during a one minute trial and the average duration of one disappearance respectively. All these variables show an increase for higher flicker frequencies. The panels in the second row show the relationship between the number of targets disappearing and the total duration and length of disappearances respectively. Panel d shows that disappearances with fewer simultaneously disappearing targets happened more often than disappearances with more simultaneous disappearing targets. Panel e shows that the total disappearance duration increased for the number of simultaneously disappearing targets. Panel f confirms that disappearances with more disappearing targets last longer than disappearances with fewer targets. The error bars in all panels indicate the standard error for N=29 subjects.
The behavioral results are in line with expectations from previous research, with more and longer disappearances for targets with a higher flicker frequency (Anstis, 1996). Additionally, the analysis shows that disappearances with more targets occur less often than disappearances with fewer targets. However, when disappearances with more targets occur they tend to last longer.
To investigate whether the effects of simultaneous disappearances indicate a true response pattern, and not a pattern as expected by chance, a permutation analysis was performed (Figure 4). This analysis aimed to compare the previously observed patterns with patterns obtained when randomly shuffling button responses from different trials within participants. The results of this analysis, as shown in Figure 4, show a clear difference between the shuffled and the non-shuffled, actual behavioral analysis. The number of disappearances are still lower for several simultaneously disappearing targets. However, the effect of total disappearance duration is reversed, with now shorter disappearances for disappearances with more simultaneous targets. The effect of disappearance duration is also reversed and shows no difference between disappearances with higher numbers of disappearing targets anymore. These results are to be expected when randomly selecting button responses from different trial, as chances of several buttons pressed at the same time by coincidence would be expected to be smaller than chances of fewer buttons being pressed at the same time. The fact that the non-shuffled results show the opposite effect suggests that target disappearances interact within trials, and that the disappearance of one target increases the probability of disappearance for other targets. Implications are discussed further in the discussion.
9 Figure 4. For this figure a permutation analysis was performed as described in the method section. All panels show the number of targets that were simultaneously invisible on the vertical axis and the number of disappearances, the average total disappearance duration during a one minute trial on the vertical axis and the duration per disappearance on the vertical axis respectively. The bar plots show the actual behavioral results, and the orange lines show the results obtained with the permutation analysis. Panel a shows a decrease in number of disappearances for more disappearing targets for both the behavioral data and the permutation data. Panels b and c show opposing effects for the behavioral data and the permutation data, with the behavioral data showing an increase in disappearance duration for an increasing number of disappearing targets, and the permutation data showing a decrease in disappearance duration for an increasing number of targets. The error bars in all panels represent the standard error for N=29 subjects.
The behavioral results with regard to the flicker frequencies follow the expected patters, and therefore suggest that participants were able to report most disappearances accurately. This is confirmed by the fact that most catch trials were reported accurately, and suggests a reliable basis for the EEG analysis.
Additionally the behavioral analysis compared to the permutation analysis suggests that target
disappearances interact during trials. This is interesting, as it might mean that the neurological state of a participant at a certain moment might correspond to an increase in number of disappearing targets. To study this further we must look at the electrophysiological results.
Electrophysiological results
In order to interpret the EEG results it is important to check whether target and background tagging frequencies are detected in the EEG pattern. An analysis of the spectrum for individual participants and for participants combined showed no frequency tagging of the peripheral stimuli (Figure 5). However, clear tagging for the background was observed at 20Hz and at its harmonics, in this case 40Hz (Figure 5). This is why the remainder of the analysis focused on the effects of target disappearances on the activity related to the dynamic background. The 20Hz dynamic background activity showed clear localization in the occipital region for the corresponding SNR (Figure 5).
10 Figure 5: Panel a shows the log transformed frequency spectrum across all trials and all participants for EEG channel POz (circled in the topoplot in panel b). It shows the frequencies on the horizontal axes and power on the vertical axis. The graph shows a clear peek for background activity at 20Hz and at its first harmonic of 40Hz. It does not show any peeks at the target frequencies of 8, 13, 15 and 18 Hz. Panel b shows a topoplot with the head seen from above, with the nose at the top and the ears on the sides. Signal to noise ratio (SNR) values on a logarithmic scale corresponding to the 20Hz dynamic background related activity are plotted per electrode. The colorbar indicates the level of 20Hz SNR for the different colors as displayed across the scalp. These SNR values were calculated across all trials and all participants. It shows clear localization of the SSVEP in the occipital region. Channel POz was observed to be the most responsive channel and was chosen as a focus for further analysis. This channel is circled in black in panel b.
The localization of the SNR activity in the occipital region suggests that we might be able to recognize effects of the perceptual disappearances in this region. It was hypothesized that background activity would increase with target disappearance, and decrease with target reappearance because of perceptual filling in (Ramachandran, Gregory, & Aiken, 1993). To study those effects, channel POz (circled in Figure 5 panel b) was selected to observe activity over time. Time frames of two seconds were selected around both button press (perceptual disappearance) and button release (perceptual reappearance). For each time point activity was corrected for baseline activity by subtracting the mean SNR activity for the duration of three to two seconds before the button press. Additionally, activity was corrected for individual fluctuations in the way described by Cousineau (2007). Multiple t-tests were performed to compare activity at each time point with the baseline activity of zero. For multiple comparisons the threshold of the false discovery rate was computed for a false discovery rate of 0. 05 (Benjamini, Krieger, & Yekutieli, 2006). This analysis showed a p-value threshold of 0.001 for the time course around the button press, and a p-value threshold of 0.03 for the time course around button release. These time courses with corresponding p-values are plotted in Figure 6. The p-values that met their corresponding threshold are marked in blue and can be considered significant. The plots show a significant increase in the 20 Hz SNR before the button press (perceptual disappearance), which is in line with expectations. However, it also shows a decrease during and after the button press, which was not hypothesized. The time course around the button release shows a decrease in activity before the button release and remains low after button release, which is in line with hypothesized expectations.
11 Figure 6. Panel a shows the average time course across trials and participants with two seconds before and two seconds after the button press (perceptual disappearance). Panel b shows the average time course across trials and participants with two seconds before and two seconds after the button release (perceptual reappearance). Baseline was defined as the average activity from two to three seconds before the button press or after button release Both panels show time on the horizontal axis and 20Hz SNR activity on the vertical axis. Both plots show green error-bars, representing the standard error after correcting for individual differences. Both plots show the p-values for each time point, indicating whether that time point differs from baseline activity. For multiple comparisons the threshold of the false discovery rate was computed for a false discover rate of 0.05. This analysis showed a p-value threshold of 0.001 for the time course around the button press, and a p-value threshold of 0.03 for the time course around button release. The p-values that met their corresponding threshold are marked in blue and can be considered significant.
So far the results show a clear localization of the 20Hz activity and effects on that 20Hz SNR activity in the expected directions, with an increase in SNR for background stimulation shortly before the button press (perceptual disappearance) and a decrease in SNR shortly before button release (perceptual
reappearance). One unexpected aspect, however, is the quick drop in activity during the button press. We would expect background SNR to increase and stay high during the disappearance, but instead it rises and then quickly drops again.
One possible explanation for the unexpected quick dip in SNR after the button press could be the short duration of some disappearances. For some disappearances this could mean that the target was already reappearing by the time the participant pressed the button, leading to a drop in activity. To control for this, the duration of each disappearance was computed. This was computed for each participant individually. Then for each participant the 20 Hz SNR activity in channel POz corresponding to the disappearances was grouped in quadrants according to disappearance duration. These quadrants were averaged over participants and are plotted in figure 7a. The category with the longest disappearances was plotted a line in figure 7b. The figure shows that SNR levels stayed higher for longer disappearances. The activity for the longest disappearances shows the same effect as observed in figure 6a, namely an increase in activity before the button press, but also shows that activity remains high for more than three seconds before dropping again for these longer disappearances. This suggests that the quick dip as observed around the button press in Figure 6 might be due to short disappearance durations.
12 Figure 7. This figure shows the 20 Hz SNR activity corresponding to the four quadrants of disappearance duration in panel a. The top row shows the activity for the shortest disappearance durations, the second and third row show the middle durations and the bottom row shows the longest durations. The figure shows a prolonged increase in activity for longer disappearances. The quadrant with the longest disappearances is plotted as a line with standard error bars (N=29 subjects) in panel b. The plot still shows an increase in SNR before the button press, but shows that the quick dip no longer occurs for these long disappearances. The analysis around target disappearance and reappearance suggests effects for target disappearance on the SNR activity elicited by the background. A second step in the analysis is to investigate whether the level of 20Hz SNR activity differed during disappearances with different numbers of invisible targets. In order to investigate this, 20Hz SNR activity was averaged within periods with zero, one, two or three invisible targets. Catch moments were excluded from this analysis. This was done for each trial and was then averaged over trials and participants. This average activity was plotted in Figure 8. Results from a repeated measures ANOVA showed that background SNR increased significantly with the disappearance of a higher number of targets (F(3,63)=39.07, p<0.01). This is in line with what would be expected, as more disappearing targets would require more perceptual filling in from the background.
Figure 8. This plot shows the average 20Hz SNR activity during the disappearance of different numbers of targets. This was computed by selecting the timeframes for the different numbers of disappearing targets and averaging SNR activity over those timeframes for all trials and all participants. The horizontal axes show the number of invisible targets and the vertical axis show 20Hz SNR activity. The plot suggests an increase in background SNR activity as more targets disappear. The error bars indicate the standard error for N=29 subjects.
13 This analysis focused on the average activity during the entire disappearance. One might wonder whether these differences can be recognized over the time course as well. To investigate this, the time courses as observed in Figure 6 were computed for the different types of disappearances: 1, 2 or 3 disappearing targets. This figure focuses on the difference between the disappearances with different numbers of targets over time. It used a repeated measures ANOVA with a p-value thresholds at a false discovery rate of 0.05 to test for significant differences between the types of disappearances at each time point. This gave a p-value threshold of 0.0287 for the time course around the button press and a p-value threshold of 0.0246 for the time course around button release. This figure did not test for time course effects within the different disappearances. The time courses around the button press (perceptual disappearance) and button release (perceptual reappearance) can be observed in Figure 9, and show general higher activity for three disappearing targets, suggesting that these differences can indeed be recognized in the time course. Disappearances with one and two disappearing targets however show the reversed pattern around the button press, with higher SNR for one disappearing target, and do not seem to differ around button
release.
Figure 9. 20Hz SNR activity over the time course from two seconds before the button press (panel a) or button release (panel b) to two seconds after button press (panel a) or button release (panel b), separated for disappearances with 1, 2 or 3 disappearing targets. Activity has been corrected for baseline activity, defines as 2 to 3 seconds before button press or after button release. P-values show the results of a repeated measures ANOVA comparing the different types of disappearances. P-P-values thresholds were computed at a false discovery rate of 0.05 and were 0.0287 for panel a and 0.0246 for panel b. P-values that met the threshold are displayed in blue and indicate a significant difference. The figure shows higher activity over the time course for disappearances with three disappearing targets. Error bars were computed after correcting for individual differences. Overall these results suggest a relationship between the number of disappearing targets and the SNR activity related to the 20Hz dynamic background, with higher activity when more targets disappear, supporting the hypothesis.
14 So far, we find effects that run mostly in the expected direction around target disappearance and
reappearance, suggesting that perceptual disappearances can be recognized in the activity elicited by the dynamic background. However, one could argue that the changes in activity might not be perceptual, but rather motor-related or related to cognitive demand, as both of these could be higher when a subject is required to press an increased number of buttons (Tsuchiya, Wilke, Frässle, & Lamme, 2015). A clear localization of the effect in the occipital region would support the visual nature of the effect (Wang, Wang, Cheng, & Jung, 2012).
To study this, several topoplots were computed over different conditions and time points. Firstly, the effects as observed in figure 6 were computed in topoplots over time. The effects for the button press (perceptual disappearance) can be seen in Figure 10 and the effects for button release can be seen in Figure 11.
Figure 10. The top row shows the 20Hz SNR activity over time across the scalp, after correcting for baseline activity, defined as 3 to 2 seconds before the button press. The bottom row shows the corresponding p-values when cutting the value off at its threshold of 0.0053, as given by the false discovery rate analysis with a false discovery rate of 0.05. Black areas show significant change compared to baseline. The titles indicate the time in seconds before or after the button press, with 0 indicating the moment of button press. These panels show strong fluctuations in the occipital region, suggesting a localization of the effect.
15 Figure 11. The top row shows the 20Hz SNR activity over time across the scalp, after correcting for baseline activity, defined as 3 to 2 seconds after the button release. The bottom row shows the corresponding p-values when cutting the value off at its threshold of 0.0023, as given by the false discovery rate analysis with a false discovery rate of 0.05. Black areas show significant change compared to baseline. The titles indicate the time in seconds before or after the button release, with 0 indicating the moment of button release. These panels show strong fluctuations in the occipital region, suggesting a localization of the effect. The results for both time courses around perceptual disappearance and reappearance suggest localization of the effect in the occipital region. This supports the idea that at least part of the change in activity is related to perception. However, other factors such as motor activity or cognitive demand cannot be excluded based on these results.
Lastly, the occipital nature of the effect would be supported if the difference in SNR activity between the number of disappearing targets was also localized in the occipital region. Figure 12 suggests that this might be the case. To test this statistically a repeated measures ANOVA was performed for each channel to compare disappearances between 0, 1, 2 or 3 disappearing targets. These effects were again controlled for multiple comparisons by computing the p-value threshold of 0.0340 for a false discovery rate of 0.05 (Figure 12 e).
16 Figure 12: Panels a-d show 20Hz SNR activity, not baseline corrected, for the different numbers of disappearing targets. 20Hz SNR activity was averaged over time within the disappearances and across trials and participants. The plots show an increase in activity in the motor areas and in the occipital region, suggesting a visual effect. Panel e shows the p-values obtained by an ANOVA analysis comparing the four different disappearance conditions as seen in panels a-d. P-values have been cut off at their threshold of 0.340. Black areas in this plot indicate significant differences across conditions. Therefore this panel b suggests that the most significant differences do in fact not occur in the occipital region.
The results when investigating localization over the time courses indicate occipital localization of significant change in the expected directions before button press and release. This supports the presence of a visual effect. However, the time courses also indicate significant fluctuations in the frontal and motor regions. Localization of 20Hz activity during the different types of disappearances suggests change in the occipital region. However, when comparing significance of these changes across brain regions the most significant effects seem to occur in the mid-frontal area of the scalp. Suggesting that the most significant change in activity is motor-related. These results suggest that there is an occipital effect, but that
significant changes also occur in other brain regions. This means that a pure perceptual effect cannot be concluded from the current results.
Discussion (1490 words/1500 max)
The current research shows that perceptual disappearances in Troxler fading can be recognized in EEG activity related to a dynamic background, and that the number of disappearing targets is reflected in this activity. Strong tagging for a changing random background pattern has not been reported in previous research, and provides a new way to optimize SSVEP measures in future research. The current study also shows that participants were able to report on the disappearances of four targets simultaneously and that disappearing targets seem to interact within trials. This study thereby provides important new insights, both for paradigms of perceptual fading and for the role of dynamic backgrounds in EEG studies. Previous research has mostly used report paradigms where participants reported on a maximum of two targets at once, while the current study required participants to report on four targets simultaneously. Disappearances were shown to behave as expected, with higher frequencies disappearing more often than lower frequencies (Anstis, 1996). Participants were found to be accurate on most catch trials. These results show that participants were able to report on target disappearance accurately.
17 Behavioral results also show that disappearances with fewer targets occurred more often than
disappearances with more targets. However, when they occurred, disappearances with more targets lasted longer. Results showed that these effects are unlikely to occur if target disappearance would be
independent. This suggests that targets disappear according to a pattern, with targets being more likely to disappear together. Previous research suggests that target disappearances might depend on a specific brain state, as TMS disruption of the Intraparietal Sulcus enhanced target disappearance (Kanai, Muggleton, & Walsh, 2008). The data of the current study could be used for future analysis to investigate these
interactions between target disappearances further by investigating whether the simultaneous removal of more catch targets lead to more perceptual disappearances co-occurring with the catch.
EEG results show the expected results of target disappearance on dynamic background activity, with an increase in activity before reported target disappearance and a decrease before reported target
reappearance. Analyses also show that background activity was higher during disappearances with more invisible targets. These results suggest that target disappearance can be recognized in the activity of a dynamic background. The unexpected quick decrease at the button press seems to be explained by short disappearance durations.
One might argue that the dynamic background effects could be due to motor activity or frontal activity related to the cognitive demand of reporting on several stimuli (Tsuchiya et al, 2015). Analysis shows a localization of the effects in the occipital region, suggesting that a visual effect is present. However, it also shows fluctuations in other brain areas, meaning that effects of motor activity or cognitive demand cannot be excluded. To investigate this further, future research could look at the effect of target size for
disappearing targets: a larger disappearing target would be expected to cause a larger increase in background activity, as a larger area would be filled in. However, a larger target would not increase cognitive or motor demand related to report, as it would still require pressing the same number of buttons.
The current research shows a lack of frequency tagging for the peripheral targets, which could be due to three reasons. A first explanation could be the small size and peripheral nature of the targets. A second explanation could be that the frequencies of the targets and the background interacted, leading to intermediate frequencies that could not be recognized in the EEG signal (Chen, Chen, Gao, & Gao, 2013). A last explanation is the layout of the target, which currently had Gaussian blurred edges and a sine-wave modulated flicker. A follow-up pilot study by a colleague suggests that the first explanation is unlikely. In this pilot the participant was asked to look directly at the targets while SSVEP signals were analyzed online. Results showed that even at the centre of fixation the target did not elicit an SSVEP, suggesting that the lack of tagging is not due to size or eccentricity. The other two explanations could be tested further in future research, firstly by tagging of the targets as was done in the follow-up pilot, but without use of the dynamic background. This would eliminate possible interactions between the frequencies. Secondly, by testing other settings for the targets such as a direct black to white flicker and sharp rather than Gaussian edges (Ng, Bradley & Cunnington, 2012).
The current research shows a relationship between conscious perception and dynamic background activity. However, it remains unclear whether an increase in background activity causes target
disappearance, or whether target disappearance results in an increase in background activity. The current research started from the idea that the filling in of the target area by the background would cause an increase in activity related to this dynamic background. However, two issues raise the question of whether this could not be the other way around. Firstly, the fact that targets were not salient enough to elicit activity in the EEG raises the question of whether the filling in of that same area could be expected to lead to strong visual effects. Secondly, the effects as perceived on the background activity occur about a second before disappearance or reappearance is reported. This exceeds what is generally found as reaction
18 time (Poulton, 1950), suggesting that the change in dynamic background activity might precede the visual effect. Both these ideas suggest that the changes in dynamic background activity might be the cause rather than the effect of target fading.
This causal relationship has also been suggested in previous research with Motion Induced Blindness (MIB) by Donner, Sagi, Bonneh, and Heeger (2008). This FMRI study found that activity increased around perceptual disappearance in dorsal visual cortex areas corresponding retinotopically to the mask. This effect happened prior to report and could not be considered the effect of target disappearance. This increase only occurred when report was required and did not occur for physical removal, suggesting a causal role. As the dynamic background in the current study might fulfill a similar role to the mask in the MIB study, the effects in the current study might be similar to the early causal effect as observed in the dorsal visual cortex. With the current data, future analysis could investigate this by analyzing the dynamic background activity during catches. If physical disappearances show the same effects as perceptual disappearances, the changes in dynamic background activity are likely to be the effect of target
disappearance. However, different effects for physical disappearances would exclude a purely visual effect, and would thus suggest a more causal role for the changes in dynamic background activity.
Future experiments could investigate the nature of the dynamic background effects further by using three variations of the current design, based on a study by Kloosterman, Meindertsma, Hillebrand, van Dijk, Lamme, and Donner (2015). Firstly, a trial-replay condition could be added, where a previous trial is presented, but targets are physically removed at times where the participant previously reported perceptual disappearances. This allows to study physical disappearances that approach the patterns of perceptual disappearances as close as possible. Secondly, conditions could be added where participants are not required to report disappearances. This could separate the effects of report from the effects of attention and conscious perception. Lastly, the instructions for button presses could be reversed, with button release indicating target disappearance and button press indicating target reappearance. This would allow to control for motor effects.
A last aim for future research is to investigate the effects of attention in the current paradigm. Lou (1999) suggested a dissociation between attention and consciousness in Troxler fading, with focused attention leading to more perceptual disappearances. The current Troxler fading task has been modified to contain flickering targets and a dynamic background, and thus has changed compared to the experiment by Lou (1999). A first step would therefore be to investigate whether the behavioral effects of attention remain the same in the current task. Secondly, it would be interesting to see whether attention influences dynamic background activity. This would again relate to the question of causality, as increased attention might lead to an increase in background activity, which might lead to target disappearance. Adding an attention manipulation would allow to investigate whether attention increases background activity outside of the disappearances, which would suggest that the change in background activity depends on attention and precedes target disappearance.
The current research suggests a relationship between conscious perception and activity in the visual area. It contributes to future research in two important ways. Firstly, it shows that filling in of a fading target with a dynamic background can be related to EEG activity. Secondly, it shows that Troxler fading can potentially be used as a neuropsychological task to dissociate between attention and consciousness. It gives directions to further investigate the possibilities for target frequency tagging, but specifically shows that focusing on background tagging is a promising way to study the properties of conscious perception.
19 References
De Brigard, F., & Prinz, J. (2010). Attention and consciousness. Wiley Interdisciplinary Reviews: Cognitive Science, 1(1), 51-59.
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrika, 93(3), 491-507.
van Boxtel, J. J., Tsuchiya, N., & Koch, C. (2010). Opposing effects of attention and consciousness on afterimages. Proceedings of the National Academy of Sciences, 107(19), 8883-8888.
Brown, R.J., and Norcia, A.M. (1997). A method for investigating binocular rivalry in real-time with the steady-state VEP. Vision Res. 37, 2401–2408.
Chen, X., Chen, Z., Gao, S., & Gao, X. (2013). Brain–computer interface based on intermodulation frequency. Journal of neural engineering, 10(6), 066009.
Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in quantitative methods for psychology, 1(1), 42-45.
Delorme, A. & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics , Journal of Neuroscience Methods 134:9-21
Donner, T. H., Sagi, D., Bonneh, Y. S., & Heeger, D. J. (2008). Opposite neural signatures of motion- induced blindness in human dorsal and ventral visual cortex. The Journal of Neuroscience, 28(41),
10298-10310.
Koch, C., & Tsuchiya, N. (2007). Attention and consciousness: two distinct brain processes. Trends in cognitive sciences, 11(1), 16-22.
Kloosterman, N. A., Meindertsma, T., Hillebrand, A., van Dijk, B. W., Lamme, V. A., & Donner, T. H. (2015). Top-down modulation in human visual cortex predicts the stability of a perceptual illusion. Journal of neurophysiology,113(4), 1063-1076.
Lamme, V. A. (2003). Why visual attention and awareness are different.Trends in cognitive sciences, 7(1), 12-18.
Lou, L. (1999). Selective peripheral fading: evidence for inhibitory sensory effect of attention. Perception, 28(4), 519-526.
Martinez-Conde, S., Macknik, S. L., Troncoso, X. G., & Dyar, T. A. (2006). Microsaccades counteract visual fading during fixation. Neuron, 49(2), 297-305.
Merikle, P. M., & Joordens, S. (1997). Parallels between perception without attention and perception without awareness. Consciousness and cognition,6(2), 219-236.
Mitra ,P., Bokil, H. (2008), Observed brain dynamics. Oxford University press, New York.
Mole, C. (2008). Attention and consciousness. Journal of Consciousness Studies, 15(4), 86-104.
Morgan, S. T., Hansen, J. C., & Hillyard, S. A. (1996). Selective attention to stimulus location modulates the steady-state visual evoked potential.Proceedings of the National Academy of Sciences, 93(10), 4770-4774.
Müller-Putz, G. R., Eder, E., Wriessnegger, S. C., & Pfurtscheller, G. (2008). Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEP-based BCI. Journal of neuroscience methods,168(1), 174-181.
Ng, K. B., Bradley, A. P., & Cunnington, R. (2012). Stimulus specificity of a steady-state visual-evoked potential-based brain–computer interface. Journal of Neural engineering, 9(3), 036008.
O’Regan, J.K. & Noe, A. (2001) A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939–973
Poulton, E. C. (1950). Perceptual anticipation and reaction time. Quarterly Journal of Experimental Psychology, 2(3), 99-112.
Proverbio, A.M. (2003). The cognitive electrophysiology of mind and brain. Academic press.
Ramachandran, V. S., Gregory, R. L., & Aiken, W. (1993). Perceptual fading of visual texture borders. Vision research, 33(5-6), 717-721.
20
Spillmann, L., & Kurtenbach, A. (1992). Dynamic noise backgrounds facilitate target fading. Vision research, 32(10), 1941-1946.
Schieting, S., & Spillmann, L. (1987). Flicker adaptation in the peripheral retina. Vision research, 27(2), 277-284.
Sutoyo, D., & Srinivasan, R. (2009). Nonlinear SSVEP responses are sensitive to the perceptual binding of visual hemifields during conventional ‘eye’rivalry and interocular ‘percept’rivalry. Brain research, 1251, 245-255.
Tsuchiya, N., Wilke, M., Frässle, S., & Lamme, V. A. (2015). No-report paradigms: extracting the true neural correlates of consciousness. Trends in cognitive sciences, 19(12), 757-770.
Wang, Y. T., Wang, Y., Cheng, C. K., & Jung, T. P. (2012). Measuring steady-state visual evoked potentials from non-hair-bearing areas. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE (pp. 1806-1809). IEEE.
Zhang, P., Jamison, K., Engel, S., He, B., & He, S. (2011). Binocular rivalry requires visual attention. Neuron, 71(2), 362-369.
Zhu, D., Bieger, J., Molina, G. G., & Aarts, R. M. (2010). A survey of stimulation methods used in SSVEP-based BCIs. Computational intelligence and neuroscience, 2010, 1.
