Dorsal vs. ventral: Differential effects of feedback modulation in
figure-ground segregation.
A combined theta-burst stimulation and EEG study.
Research report
of
Lea K. Hildebrandt
6240917
For the 1
stresearch project within the masters’ program
Brain and Cognitive Sciences,
Cognitive Neuroscience Track,
University of Amsterdam, Netherlands
Conducted at the
Cognitive Neuroscience Group,
Department of Psychology,
University of Amsterdam,
Amsterdam, Netherlands
Supervisor:
Dr. Martijn Wokke
Cognitive Neuroscience Group,
Department of Psychology,
University of Amsterdam,
Amsterdam, Netherlands.
Co-Assessor:
Dr. Steven Scholte,
Cognitive Neuroscience Group,
Department of Psychology,
University of Amsterdam,
Amsterdam, Netherlands.
Abstract
Segregating a scene into objects and background is a fundamental and process in visual perception, crucial for functioning in everyday life. It has been shown that figure-ground segregation depends not only on feed forward but also on and recurrent processing within the visual cortex. However, the origin of this feedback activation remains unclear. Moreover, one important cue for the distinction between objects and background is motion. Motion- and object-recognition are assumed to be carried out in distinct cortical pathways: the dorsal and the ventral stream. The aim of this study was to investigate the contributions of the two streams to the feedback to early visual cortex, which is necessary for the recognition of motion-defined objects. For this end, transcranial magnetic stimulation was used to interfere and inhibit the lateral occipital and the medial temporal cortices, located in the ventral and dorsal respectively, while concurrently measuring EEG. The participants were asked to discriminate between different stimuli that enabled the discrimination between activity related to general processing of objects and surface segregation necessary for complex figure-ground segregation. The results indicate significant differences between processing of figures and a homogeneous background. However, no clear differences between processing related to simple figures and more complex figure surface segregation could be found.
Introduction
Distinguishing objects from the background is a fundamental aspect of visual perception, which allows us to interpret and interact with the environment. Traditionally, it has been assumed that visual information is processed in a hierarchical feed forward fashion, in which low-level information such as figure edges is processed early. Subsequently, the low-level information would be integrated in higher cortical regions. This model was supported by the findings of Hubel and Wiese (1959), who suggested that neurons in the visual cortex process information from small parts of the visual scene, called classic receptive fields (cRF), which have been shown to increase in size the higher a region lies in the
hierarchical model of the visual cortex. Moreover, it has been shown that neurons in the early visual cortex are sensitive to basic features of a scene, such as contrast or line orientation, whereas receptive field properties of later visual areas become increasingly complex. Hence, it is assumed that the
increasing receptive field size allows for integration of low-level features, resulting in complex processes such as feature binding, eventually leading to a coherent percept of the world around us (Maunsell & Newsome, 1987).
Recurrent processing
However, more recently, evidence against a strictly linear feed forward model has been found, supporting models that incorporate recurrent processing in the visual cortex (Enns & Di Lollo, 2000; Lamme & Spekrijse, 1998; Lamme & Roelfsema, 2000). This theory is mainly based on findings showing that the activity of early visual cortex neurons is modulated by contextual information, which originates outside of the cRF. For example, both Lamme (1995) and Zipser, Lamme, and Schiller (1996) have found that neuronal responses of primary visual cortex (V1) neurons differed depending on whether the part of the visual scene corresponding to the cRFs was part of a figure or the background, even though the exact same physical information was presented in the cRFs of those neurons. Notably, the boundaries of the figures were not adjacent to the cRFs.
This modulated activity has been associated with feedback from higher regions in the visual cortex (Lamme, 1995; von der Heydt, Zhou, & Friedman, 2000). Importantly, the functional relevance of these feedback (and horizontal) connections has been suggested to lie in the integration of low-level information. In this sense, pure feed forward processing would only allow for the perception of low-level features, whereas recurrent (and horizontal) processing enables complex processing such as figure-ground segregation (V. a Lamme & Roelfsema, 2000).
In line with this interpretation are findings of Scholte, Jolij, Fahrenfort, and Lamme (2008), who used stimuli that differed in the amount of figure surface, but shared the same amount of figure
boundary, which allowed isolation of activity related to surface segregation from more low-level figure boundary detection. Using these stimuli, Scholte et al. (2008) demonstrated that activity related to boundary detection spreads in a feed forward manner, while figure surface segregation seems to follow
an opposite direction, starting out in the temporal cortex, spreading to parietal areas and subsequently feeding back to occipital areas.
More support for feedback to early visual cortex during figure-ground segregation comes from Wokke, Sligte, Scholte, and Lamme (in press) who found evidence for two phases of activation of early visual cortex necessary for figure-ground segregation. Using transcranial magnetic stimulation (TMS), they disrupted processing in early visual cortex at different points in time just after stimulus
presentation, while concurrently recording electroencephalographic (EEG) signals. General figure detection was impaired after early TMS stimulation (96-119 ms), whereas more complex figure-ground segregation (surface segregation) selectively decreased when stimulating at a later period (236-259 ms). Interestingly, a TMS pulse at an intermediate time (156-179 ms) did not interfere with either sort of processing. Hence, this finding indicates a causal relationship between a relatively late period of activity in early visual cortex and successful figure-ground segregation.
To summarize, these findings provide evidence for a relatively late (re-)activation of early visual cortex that most likely depends on feedback and is crucial for successful figure-ground segregation. However, the cortical origin of this feedback remains unclear beyond the rather vague suggestion of Scholte et al. (2008) pointing towards the temporal cortex.
Dorsal and ventral streams
Interestingly, another widely accepted theory of visual processing could be helpful in shedding light on the origin of feedback necessary for successful figure-ground segregation: The dual pathways model of vision (Goodale & Milner, 1992; Mishkin & Ungerleider, 1982).According to this theory visual information is processed via two parallel cortical pathways: the ventral and the dorsal stream. The ventral stream spreads from the early visual cortex to temporal regions and is thought to be responsible for object recognition, whereas the dorsal stream runs from early visual cortex to parietal cortex, processing spatial and motion information (Goodale & Milner, 1992; Mishkin & Ungerleider, 1982).
Often in everyday life, objects do not only differ from the background in color, contrast, or illumination, but also in motion (e.g. approaching cars or moving animals). In this case both motion (dorsal stream) and object (ventral stream) recognition can be assumed to play a role in successful identification, which strongly indicates the necessity of an integration of the object features processed in each stream. Having provided evidence for the importance of feedback in these integration processes, it could be fruitful to incorporate the dual pathways theory in the recurrent processing account in order to shed light on the contributions of these two streams to the feedback necessary for figure-ground segregation, especially of motion-defined objects.
Along these lines, Raudies and Neumann (2008) proposed that motion-defined boundaries are detected within the dorsal pathway, whereas the shapes defined by those boundaries are processed in
the ventral stream. They suggest that perception of the whole motion-defined figure would require feedback from both streams to early visual cortex, where this information is eventually integrated.
Very recently, Wokke, Scholte, and Lamme (submitted) demonstrated that dorsal and ventral stream could have opposing contributions during motion defined figure-ground segregation. They used repetitive TMS and concurrent EEG recordings to show that area LO and MT, located in the ventral and dorsal stream, respectively (Ferber, 2003; Grill-Spector & Malach, 2004), are differentially involved in figure-ground segregation of motion-defined stimuli.
The medial temporal cortex (MT) is assumed to be sensitive to the perception of motion(Ferber, 2003; Grill-Spector & Malach, 2004). It has been found that MT plays an important role in figure-ground segregation, possibly inhibiting the surrounding background in early visual cortex through feedback (Likova & Tyler, 2008). This implication is supported by the fact that anatomical feedback connections exist from MT to V1 (Cowey & Walsh, 2001; Lamme & Spekrijse, 1998).
The lateral occipital complex (LOC) and in particular the subregion LO is known to play an important role in object recognition ((Appelbaum, Wade, Vildavski, Pettet, & Norcia, 2006; Grill-Spector, Kourtzi, & Kanwisher, 2001; Reddy & Kanwisher, 2006)). Findings on the precise contribution of LO to figure-ground segregation are, however, controversial. It seems that LO is either responsive to shapes or surfaces (Grill-Spector et al., 2001) or involved in contour processing (Schira, Fahle, Donner, Kraft, & Brandt, 2004). In the study by Schira and colleagues (2004), LO was active whenstimuli were shown outside of awareness, which is assumed to reflect feed forward processing only (Lamme & Roelfsema, 2000).
In particular, Wokke et al. (submitted) found that interruption of LO with rTMS led to an increase in accuracy of detecting motion-defined stimuli, whereas rTMS over MT resulted in a decrease in performance compared to not rTMS administration.Wokke and colleagues interpreted these findings as evidence for a compensatory “push-pull” interaction between the two streams.
Current study
The aim of the current study is to investigate the differential contributions of the ventral and the dorsal stream, in particular LO and MT, to the feedback to early visual cortex, which is assumed to be necessary for the successful perception of motion defined stimuli. In contrast to Wokke et al.’s
(submitted) study, the current study is designed to investigate the effects of feedback on the frequency distribution in early visual cortex, rather than on behavioral effects.
For this end, we will use continuous theta burst stimulation (cTBS) to disrupt and hence inhibit signaling in either MT or LO (Huang, Edwards, Rounis, Bhatia, & Rothwell, 2005). Based on theta burst stimulation paradigms used to induce long-term potentiation in animals, up to 600 pulses are
administered in bursts of three pulses repeated at 5 Hz. Advantages of cTBS over traditional repetitive TMS are a shorter administration period (40s), a longer lasting inhibitory effect (40min), and a lower
intensity of the stimulation (Cárdenas-Morales, Nowak, Kammer, Wolf, & Schönfeldt-Lecuona, 2010). Moreover, motion-defined stimuli will be used similar to those of Wokke et al. (submitted), which enable differentiation between figure border detection and surface segregation. To detect the frequency changes in early visual cortex (V1/V2) activation, we will measure EEG during the task and subsequently carry out time-frequency analyses.
Based on the findings summarized above, we expect that both MT and LO play a role in figure-ground segregation of motion-defined stimuli. MT is assumed to play an important role in feeding boundary information back to early visual cortex, which is necessary for successfully perceiving motion-defined complex figures. Specifically, we expect that inhibiting MT with cTBS will lead to a decrease of feedback, which will be reflected in a lack of (re-)activation of V1/V2. This, in turn, will be conveyed in a higher inhibition (alpha band activity) in V1/V2, in response to feedback-dependent stimuli. In contrast, LO might be activated in the perception of more low-level, feed forward dependent features, such as boundary or shape recognition. Therefore, if LO is less a source of feedback, we expect that inhibiting LO would not as much affect a relatively late (re-)activation of early visual cortex. Moreover, we cautiously hypothesize that inhibiting LO could, based on the findings of Wokke et al. (submitted), lead to a higher activity of MT, which would be visible in V1/V2 as an increase in activity related figure-ground
segregation. The possibility that the two streams interact has been widely suggested (e.g. Farivar, 2009; Goodale & Milner, 1992; Raudies & Neumann, 2008; Schenk, 2012; de Haan & Cowey, 2011).
Methods Subjects
Eight students from the University of Amsterdam participated for financial reimbursement, of whom one participant was male and the average age was 23.4 years. All participants had normal or corrected-to-normal vision and no history of neurological diseases. The experiment was approved by the ethics committee of the department of Psychology of the University of Amsterdam and all participants had signed informed consent. One session lasted approximately 90 minutes. Data of one participant were excluded from analysis due to missing files.
Stimuli
Three categories of stimuli were used: frames,
stacks, and homogeneous patterns. In order to create
these motion-defined stimuli, randomly distributed black and white dots of 1 pixel in size were displayed on the whole screen and displaced. In particular, the display was divided into three screen regions: a square in the middle of the screen (2.42°, 24.8 cd/m²),
Figure 1. Schematic representation of stack, figure, and homogeneous stimuli
a frame around that square (3.23°, 24,8 cd/m²), and the background (17.99°, 24.8 cd/m²). The
displacement of all dots in one region was in the same direction, whereas the displacement of dots in different regions could be the same or be perpendicular (thus 45°, 135°, 225°, or 315°).During one screen refresh (16,6 ms), all the random dots of one region were displaced by one pixel. Subsequently, during another screen refresh, the dots were displaced back in the opposite direction.
The homogeneous stimuli consisted of only a uniformly moving display, thus all dots of all three regions were displaced in the same direction. The frame stimuli were composed of the inner square moving in the same direction as the background, while the dots belonging to the frame region displaced in a perpendicular direction. Stack stimuli consisted of the inner square, the frame, and the background regions all moving in different directions, which gave an impression of two stacked squares (see figure 1).
To avoid different amounts of flicker at the regions’ borders due to (dis)appearing dots, the pixels within each region did not cross their fixed borders. Importantly, stack and frame stimuli
consisted of the same amount of borders (regions where motion was in perpendicular directions). This does not only provide equal local motion contrast but resulted in stimuli that only differ in the amount of surfaces that can be perceived. Hence, to discriminate homogeneous stimuli from either stacks or frames, only figure border detection is needed; however, the discrimination between stacks and frames depends on further figure-ground (or surface) segregation.
Paradigm
After displaying a blank screen for 1500 ms (24.8 cd/m²), a fixation dot was displayed for 250 ms in the middle (0.15°) of the screen (1023*768 pixels, 17 inch DELL TFT monitor, refresh rate 60 Hz, approx. 90 cm distance to participant), which was filled with randomly distributed black and white dots. Subsequently, three of the same stimuli were presented, either on the left or right of a central fixation point for a duration of 67ms. More precisely, this period was made up out of four screen refreshes of 16,7ms each in which the random dots moved back and forth (see above). The trial ended when the participant gave a response by pushing one of three buttons, irrespective of side of presentation: Using the left index finger for homogeneous stimuli, the right index finger for frames, and the right middle finger for stacks. The order of stimuli was randomized. Data were collected in 12 sessions per subject (due to TMS-exposure restrictions of the ethics committee): four control sessions, four with TBS over MT, and four sessions with TBS over LO. Each session consisted of five blocks of 60 trials, and lasted, without preparation, approximately 30 min. The order of the sessions was randomized. All participants were trained in the task beforehand, and all have, in addition, participated in a similar study or pilot before.
Theta Burst Stimulation
Prior to the experimental sessions, continuous theta burst stimulation (cTBS) using a Magstim Rapid² (Magstim Company, UK) stimulator with a 70 mm figure-eight coilwas applied for a period of 40 seconds at a stimulation strength of approximately 80% of the individual phosphene threshold.
Locations targeted were either the right medial temporal (MT) or the right lateral occipital (LO) area using an MRI-guided navigation system (ANT- Visor system). The subject-specific locations of LO and MT were specified in advance using an object-recognition (Grill-Spector & Malach, 2004) and a motion-detection task (Dumoulin et al., 2000), respectively (for details see Wokke et al, submitted). During stimulation, the center of the coil was located approximately 1.5 cm above the targeted regions, and with a maximum sideway deviation of not more than 0.4 cm. Three pulses at 50Hz were repeatedly administered in a 5 Hz rhythm, resulting in 600 pulses in total. This stimulation was assumed to disrupt the targeted region for up to 40 min (Huang et al., 2005) . Furthermore, a control condition without any TBS stimulation was also implemented.
Data analysis
Behavioral analysis
Accuracy, the mean percentage of correct responses per participant, was calculated separately for the three different stimulus types, two sides of stimulus presentation, and three TBS condition. With these data, a 3 x 2 x 3 repeated-measures ANOVA was carried out. A similar 3 x 2 x 3 analysis was carried out on mean reaction times on the correct trials as dependent variable.
EEG measurement and analysis
EEG was recorded at 1048Hz using a 64 electrode set-up (ANT – ASA – LAB system of ASA). Furthermore, two horizontal and two vertical EOG (electrooculography) electrodes were used to record eye movements, each referenced to their counterpart.
After acquisition, EEG data was down-sampled to 512 Hz. Data were analyzed using Brain Vision Analyzer (BVA, Brainproducts) and the EEGlab toolbox (Delorme & Makeig, 2004) for Matlab
(Mathworks) In BVA, data from the different sessions belonging to the same condition were combined per subject and were preprocessed using a high-pass filter with a cut off of 1Hz and a low-pass filter of 50Hz. The signal was rereferenced to Cz.. After having imported the data to Matlab, independent component analyses (ICA) were carried out on every combined data set to eliminate artifacts, such as eye blinks. In a few cases, ICA did not work in Matlab and was carried out with BVA. The continuous data were divided into segments starting 500ms before stimulus onset and ending 750ms post-stimulus onset, and the data were baseline corrected by subtracting the average value of the trials’ -200 ms to 0 ms before stimulus onset from the data. Trials with remaining artifacts were manually rejected and, if necessary, data of noisy periods of single electrodes were removed and interpolated using inverse
distances. Subsequently, Current Source Density (CSD) transformation based on spherical spline surface Laplacian estimate was calculated (Perrin, Pernier, Bertrand, & Echallier, 1989) and the data were divided according to the different stimuli and side of presentation.
For the further analysis, poolings of left (electrodes O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital areas were created in order to decrease the number of individual comparisons and increase the signal-to-noise ratio. For the same end, we looked specifically at the time window between 80 and 350 ms post-stimulus. Based on earlier studies (Wokke et a., submitted), we expected most effects to occur in this period.
Event-related potentials (ERPs). The average of the time courses over all trials of the different conditions and stimuli was calculated. In order to isolate activity related to general processing of figures, we subtracted the average time course of homogeneous stimuli from the time course averaged over frame and stack stimuli. Furthermore, to isolate the activity related to figure-ground segregation, we subtracted the average data of frames from those of stacks. Using the pooled electrodes and the restricted time window, we performed random-effects analyses by applying sample-by-sample two-tailed paired t-tests to test at which time points the conditions differed significantly (p<0.05) from zero (FDR corrected).
Time-frequency analysis. The preprocessed data was transformed using complex Morlet wavelets with 80 steps between 2 and 40Hz and a width of 5 cycles. The data were baseline corrected with a baseline ranging from 500 to 200 ms prior to stimulus presentation. Subsequently, the extracted power was averaged over trials and the difference between stack- and frame-related activity was again calculated for both sides of stimulus presentation (S-F(L) & S-F(R)), as well as the difference in activity between figure (average of stacks and frames) and homogenous stimuli. Eventually, the resulting values were analyzed for significant differences between conditions using multiple t-tests, corrected for multiple comparisons with false discovery rate according to Benjamini, Krieger and Yekutieli (2006).
Results Behavioral analysis
Accuracy
Mauchly’s test revealed that the assumption of sphericity was met for all effects (all p’s > 0.1). The repeated-measures ANOVA with TBS condition, stimulus, and side of stimulus presentation as within-subjects factors resulted in no significant effects (see table 3), although the main effect of condition did reach significance in the multivariate analysis (F(2, 6) = 6.413, p = 0.032).
Reaction time
Sphericity can be assumed for all effects but the main effect of stimulus (W(2) = 0.324, p = 0.034) and therefore, Greenhouse-Geisser corrected degrees of freedom will be reported (ε = 0.597). The repeated-measures ANOVA revealed significant differences in reaction times for the different stimuli (F(1.19, 8.35) = 7.022, p = 0.025) as well as for the interaction between stimulus and side of presentation (F(2, 14) = 4,673, p = ,028). None of the other effects reached significance (see table 4).
Table 1 . Means and standard deviation of mean percentage of correct responses for the three conditions and the six stimuli.
Total Frame Stack Homogeneous Condition Left Right Left Right Left Right LO Mean 0.8251 0.7561 0.7305 0.8502 0.8701 0.8693 0.8654 St.Dev. 0.0427 0.1726 0.1786 0.1028 0.0874 0.1327 0.1555 MT Mean 0.8506 0.7960 0.7443 0.8916 0.9294 0.8608 0.8704 St.Dev. 0.0469 0.1640 0.1892 0.0628 0.0603 0.0928 0.1353 NO Mean 0.8254 0.7434 0.6818 0.8697 0.8948 0.8703 0.8780 St.Dev. 0.0588 0.1641 0.1702 0.0834 0.0756 0.0997 0.1266
Table 2. Means and standard deviation of reaction times for only correct trials and all trials, for all three conditions and all six stimuli.
Frame Stack Homogeneous
Condition Left Right Left Right Left Right Correct trials LO Mean 642.449 621.495 528.303 542.006 565.0126 557.834 St.Dev. 78.8145 82.3004 83.8576 67.1266 84.6695 91.8091 MT Mean 632.281 613.780 532.512 552.017 586.240 581.802 St.Dev. 80.6146 72.3194 97.8087 80.9001 100.0140 107.3122 NO Mean 628.063 601.254 517.841 534.380 563.494 555.948 St.Dev. 71.6096 56.5580 76.0140 68.4313 82.7617 88.46.35 All trials LO Mean 658.663 629.603 536.081 545.191 583.164 569.253 St.Dev. 101.600 93.214 102.209 91.433 96.749 101.291 MT Mean 649.026 623.560 537.970 566.866 595.589 598.933 St.Dev. 94.222 90.695 95.647 81.585 104.413 117.540 NO Mean 630.062 597.382 528.522 544.191 570.770 568.961 St.Dev. 88.464 72.533 77.142 76.736 93.763 101.165
Table 3. Test statistics of repeated-measures ANOVA on mean percent correct responses. Effect F Df p Condition 3.483 2, 14 0.059 Stimulus 2.619 2, 14 0.108 Side stimulus 0.103 1, 7 0.758 Condition * stimulus 1.001 4, 28 0.423 Condition * side 0.282 2, 14 0.758 Stimulus * side 1.199 2, 14 0.331 Condition * stimulus * side 1.266 4, 28 0.307
Table 4.Test statistics of repeated-measures ANOVA on mean reaction times. Note: * Greenhouse-Geisser corrected.
Effect F Df p Condition 0.639 2, 14 0.542 Stimulus 7.022 1.193, 8.354* 0.025 Side stimulus 0.667 1, 7 0.441 Condition * stimulus 0.964 4, 28 0.443 Condition * side 0.538 2, 14 0.595 Stimulus * side 4.673 2, 14 0.028 Condition * stimulus * side 0.197 4, 28 0.938
Pairwise comparisons of the three stimuli
(Bonferroni-corrected) revealed that the main effect is due to a significant difference between frames and stacks (p < 0.001), whereby the average reaction time is higher for trials where frames where presented (M = 623.22) than on stack-trials (M = 534.51).
Reaction times for homogeneous stimuli (M = 568.39) did not differ significantly from those on stack and frame trials (p’s > 0.3). Further paired-samples t-test comparing the two sides of
stimulus presentation for all three stimuli separately, however, did not reveal any significant differences in reaction times for the two sides of presentation (t(7)frames = 2.182, p = 0.065; t(7)stacks = -1.978, p =
0.088; t(7)homogeneous = 0.826, p = 0.436). As can be seen in figure 2, the interaction was due to more
extreme reaction time for the right stimuli: for frames and homogeneous stimuli, reaction times were higher but for stacks lower than left stimuli.
ERP’s
Figure – Homogeneous Stimuli
The sample-wise, FDR-corrected paired-samples t-tests of the averaged EEG signal over trials, electrodes within the pooling, and restricted to the period of 80 to 350 ms after stimulus presentation was used to compare ERP-related activity of figure stimuli to homogeneous stimuli. The test was carried out separately for left- and ride-sided stimuli, and calculations were divided into left and right occipital activity (TBS was administered to the right occipital cortex).
In the NO TBS condition, the t-test revealed that stimuli presented on the left led to a significant difference between figure- and homogeneous-related activity in the right occipital cortex (all t’s(6) ≥ │2.278│, 85 out of 139 comparisons reached significance). In contrast, none of the time points of this comparison with right-sided stimuli and left-sided electrodes reached significance, and neither did the figure and homogeneous ERPs differ significantly if the stimuli where presented on the same side as the electrodes analyzed (see figures 4 and 5, last rows).
Figure 3. Interaction between side of presentation and stimulus type on reaction times.
Figure 4. ERPs of the left-sided figure & homogeneous stimuli and the difference between those, for the three different conditions and left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
Figure 5. ERPs of the right-sided figure & homogeneous stimuli and the difference between those, for the three different conditions and left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
After administration of TBS over LO, periods of significant differences between activity related to figure perception and activity related to the homogeneous stimuli can be found in both sides of the early visual cortex after left-sided stimulus presentation (Left: all t’s(6) ≥ │3.478│, 36/139; Right: all t’s(6) ≥ │2.694│, 64/139) and in the left early visual cortex after having presented right-sided stimuli (all t’s(6) ≥ │2.428│, 83/139)). See the second row of figures 4 and 5 for these ERPs.
TBS over MT resulted in significant differences in left early visual cortex for both sides of stimulus presentation (Left: all t’s(6) ≥ │3.478│, 52/139; Right: all t’s(6) ≥ │2.428│, 58/139)). No differences could be found in the right occipital cortex (figures 4 & 5, row 1).
Stack – Frame stimuli
In the NO TBS condition, no significant differences between stack-, and frame-related activity could be found (figures 6 & 7, row 3). Furthermore, neither TBS over LO nor over MT resulted in significant differences in the right occipital cortex, regardless of side of stimulus presentation. Right-sided stimuli did to very few significant deviations in the right occipital cortex after administration of both MT (all t’s(6) ≥ │2.694│, 8/139), but not with TBS over LO. Therefore, TBS administration did not lead to many significant differences between stack- and frame- related activity in early visual cortex (see figures 6 & 7).
Figure 6. ERPs of the left-sided stack & frame stimuli and the difference between those, for the three different conditions and left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
Figure 7. ERPs of the right-sided stack & frame stimuli and the difference between those, for the three different conditions and left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
Between conditions comparisons
In order to compare the effects of TBS, the differences between figure- and homogeneous-related activity as well as the differences between stack- and frame-homogeneous-related activity was compared between all three TBS conditions. For this end, paired-samples t-tests were performed, comparing either LO-NO, MT-NO, or MT-LO conditions. Again, the comparisons were restricted to a limited time window and the left and right occipital electrodes.
These comparisons revealed that none of the differences between conditions were significant. In figures 8 and 9, the stack-frame and figure-homogeneous difference waves of all three conditions are presented.
Figure 8. ERPs for the three different conditions (MT, LO, NO) of the left-sided stack & frame, and figure & homogeneous stimuli for left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
Figure 9. ERPs for the three different conditions (MT, LO, NO) of the right-sided stack & frame, and figure & homogeneous stimuli for left (O1, PO3, PO5, PO7) and right (O2, PO4, PO6, PO8) occipital electrodes. Red parts denote significance at FDR-corrected α = 0.05.
Time-frequency analysis
The FDR-corrected paired-samples t-tests revealed that none of the time-frequency points differed significantly between stacks and frames or figure and homogeneous stimuli. Similarly, the t-tests comparing the different conditions also did not result in any significant rime-frequency points. However, clear differences in activity between figure- and homogeneous-related activity can be seen in figure 9. This activity spreads roughly from shortly after stimulus presentation to 500 ms later and from 5 to 20 Hz. Moreover, in the right occipital cortex and after right-stimulus presentation, a second, higher frequency difference in activity can be found around 500 ms and between approximately 20 and 35 Hz (See figures 10 & 11).
Figure 10. Time-frequency plots for figure-homogeneous related activity after left (top) and right (bottom) stimulus presentation, left pooling.
Figure11. Time-frequency plots for figure-homogeneous related activity after left (top) and right (bottom) stimulus presentation, right pooling.
In the stack-frame comparison, low frequency (~0 – 20 Hz) can be found after left stimulus presentation in LO and NO conditions in both hemispheres, whereas this activity can be found throughout the trial in the MT condition. With right stimulus presentation, this activity is less but another difference between stacks and frames can be found around 600 ms pos- stimulus and between 15 and 35 Hz. All the described differences are positive, which means that stack-related activity at these time-frequency points is higher than frame-related activity (see figures 12 & 13).
Figure 12.Time-frequency plots for stack-frame related activity after left (top) and right (bottom) stimulus presentation, left
pooling.
Figure 13.Time-frequency plots for stack-frame related activity after left (top) and right (bottom) stimulus presentation,
Discussion
The aim of this study was to investigate the effects of disrupting areas in the dorsal and ventral visual processing streams on figure-ground segregation-related activity in the early visual cortex. For this end, TMS according to a TBS protocol was administered over areas MT and LO, and the participants were asked to discriminate between different sorts of motion-defined stimuli while EEG was
concurrently measured. The different stimuli were designed to be able to differentiate activity related to general figure perception (figure-homogeneous) and to isolate more complex figure surface segregation (stacks-frames).
The behavioral results revealed no differences in accuracy in response to the stimuli between the three TBS conditions. However, visual inspection of figure 2a, which presents the mean percentage correct of all conditions and stimuli, shows that frames are generally worse recognized, and the figure stimuli (stacks and frames) hint towards an effect of TBS: With these stimuli, TBS over MT leads to a slight increase compared to NO TBS, whereas TBS over LO leads to increases with frame stimuli and possibly decreases with frame stimuli. Similar results are found when analyzing the reaction times: Visual inspection of figure 2b reveals an increase in reaction times for frame stimuli and a slight speed up for stack stimuli, both compared to homogeneous stimuli. In the case of reaction times, the difference between stacks and frames was significant, and more pronounced for stimuli presented on the left side of the screen.
The lack of significant findings with the behavioral measures is not necessarily against our expectations. Three stimuli of the same type were always presented together, which would lead to a facilitated recognition compared to presenting one stimulus at a time. However, having presented stimuli in the periphery of the screen could account for not finding a ceiling effect in the mean accuracy scores. Thus, correct stimulus identification was in a good range, and mistakes were evenly distributed over all sorts of stimuli. Furthermore, the trends found by visual perception are partly in line with earlier findings (Wokke et al, submitted), where the study was focused more on the behavioral effects.
The EEG analysis was subdivided into ERP- and time-frequency-analysis in order to research both the general time course of activity as well as the frequency distribution of activity within the time course. The ERP analysis revealed that the activity related to processing of figures does differ from the one related to homogeneous stimuli. Particularly, it is found that without TBS administration, stimuli presented on the left side do lead to differences in the right early visual cortex. This is in line with the prevalent idea that visual information is processed in the contralateral parts of the early visual cortex. Unfortunately, this result was not found in the left hemisphere when stimuli were presented on the right side. TBS administration over LO does not change this significant difference; however, after TBS over MT, the difference is diminished and not significant anymore.
Interestingly, TBS over MT as well as LO results in a difference between figure- and
homogeneous-related activities after left-sided stimulus presentation in the ipsilateral left early visual cortex. This difference is not found in the baseline, NO TBS condition. Moreover, this same left-sided activity in MT, but not LO, is only slightly found after right-sided stimulus presentation.
Visual inspection of figure 5, representing the activity found in right occipital cortex, shows that processing of left-sided figures, as compared to homogeneous stimuli, results in two phases of
significance for MT and LO conditions (but a similar pattern without TBS). These phases coincide with the two phases of activation of early visual cortex that was reported by Wokke et al (submitted) to be important for figure-ground segregation. Unfortunately, as Wokke and colleagues found different activity in response to stacks and frames in these two phases, we could only find those differences in response to general figure processing.
TBS was administered to the right side of the cortex, which would thus lead to an inhibition of processing in this hemisphere. Stimuli presented to the left are processed in the right visual cortex, thus finding more results with the left-sided stimuli is in line with the expectations. These findings could be interpreted as inhibiting LO would lead to general facilitation or increase in early visual cortex activity, which might spread to the left occipital cortex. TBS over MT could, in contrast, inhibit activity in the contralateral cortex, which might be compensated by an increase of differential activity in the ipsilateral hemisphere. Right-sided stimuli are processed by the left occipital cortex and should thus not be
influenced by right-sided TBS. However, the lack of significance differences between right-sided figure and homogeneous stimuli related activity in the left cortex without any TBS administration is an
unexpected finding. Furthermore, visual inspection of figure 4 results in doubts about the validity of the analysis carried out, as the significance of the difference wave (magenta colored) between figure and homogeneous-related activity does not seem to correlate with the amplitude of the same wave. For example, the difference in the MT condition and in right occipital cortex seems to be rather great as compared to the significant parts of other difference waves. Therefore, mistakes in the analysis and the plotting, such as confusing conditions, might have happened.
The comparison of stack- and frame-related activity did not reveal any significant differences in early visual cortex activity in response to any of the stimuli. This is unexpected as it has been shown before that, with and without interruption of regions with TBS, this comparison can successfully isolate activity related to complex figure surface segregation (Wokke et al.). This lack of significant results could be due to similar errors in the analysis as noted above. It is very unlikely that no differences can be found in early visual cortex in the processing of two different stimuli.
Similarly, the comparisons of the differences (figure-homogeneous and stack-frame) between the three conditions (No, MT, and LO TBS) did not result in significant differences. This might, at least in
the case of the stack-frame comparisons, be due to the lack of results found in the simple within-conditions comparisons.
The time-frequency analysis did also not result in significant differences between stimuli and conditions. One main factor for this lack of significant results is most likely the fact that no limitations were set beforehand on the time and frequency windows of interest. This resulted in 3276800 comparisons, which, when FDR corrected, should not be expected to leave many significant findings. Therefore, it is strongly suggested to limit the amount of comparisons, e.g. by using the same time window as in the ERP analysis (80 – 350 ms). Furthermore, even though the frequency range was already limited to 0 to 40 Hz, the analysis could have been driven more closely by the hypotheses, which could have resulted in limiting the frequency range to alpha (8-12 Hz) and theta (4-8 Hz) frequency bands. However, visual inspection of the complete time-frequency range shows that also later and higher-frequency points could be of interest. In particular, a more restricted analysis could reveal significant differences between stack- and frame-related activities, especially after left-sided stimulus presentation. It seems, as interpreted from visual inspection of figures 11 and 12, that the differential activity occurs earlier after TBS administration than without TBS. In the NO TBS condition, this activity also seems to be of lower frequency. In contrast, the differential activity in the LO condition occurs around stimulus presentation and the frequency is between 10 and 20 Hz. The lower frequencies of this distribution, which can be in particular found in the right early visual cortex and starting before stimulus presentation, seem to be alpha band activity, which is related to inhibition. This implies that more alpha band activity is found in these time-frequency points after presentation of stack compared to frame stimuli. Thus, inhibition of LO might result in a higher inhibition of early visual cortex after the presentation of the more complex stimuli than following the more basic frame stimuli. However, this activity is most likely not related to feedback in response to the stimuli, as the activity seems to start before stimulus onset. What is more, the main part of this differential activity seems to be in the beta band (12-30 Hz). Beta band activity has been associated with visual attention (Kinsey, Anderson, Hadjipapas, & Holliday, 2011), not necessarily with figure-ground segregation. However, if the here presented results of LO would reach significance, the difference in beta frequencies when isolating figure surface segregation-related activity would point towards a role of beta that goes beyond simple attention effects. If processing of stacks would require higher attention than processing of frames, the effect should be found in all three conditions. Thus, inhibition of LO might selectively lead to an enhancement of beta band activity for stacks (or decrease of beta for frames).
TBS over MT does result in differential activity that deviates from the pattern of activity found without TBS. However, this low-frequency activity is continuous throughout the trial and does not seem to be related to stimulus onset. Here, it could be helpful to clarify which, if any, of these time-frequency points would reach significance in a more restricted analysis, as suggested above. However, against
expectations, the differences in activity seem to be of very low frequency and not restricted to a time window.
Effects of TBS in the stack-frame comparisons can be clearly seen, as the activity related to left and right presentation clearly differ; furthermore, the time-frequency plots of the three TBS conditions also seem to differ. Interestingly, these differences between figure and homogeneous stimuli seem to be less distinctive. Further analysis could be very fruitful to shed further light on the nature of these differences.
The differences between figure and homogeneous stimuli do lead to clear regions of differential activity around the alpha range after stimulus presentation, and, in the case of right-sided stimuli also later (500 ms) in the higher beta range. However, it remains questionable whether the comparison of these activities between conditions would result in significant findings in a more restricted analysis. By visual inspection it can be said that the pattern of differential activity between conditions look very similar.
To summarize, we did not find support for our hypotheses. Inhibiting MT did not seem to result in a decrease in feedback to early visual cortex, as neither an increase in alpha band activity nor a decreased reactivation of the early visual cortex. LO did seem to be involved in the processing of complex features, due to the fact that inhibiting LO lead to differences in (beta) activity in early visual cortex in response to stack and frame stimuli. It seems that without the influence of LO, higher visual attention (or other, figure-ground-related beta activity) can be found in early visual cortex in response to stack stimuli. This could be interpreted as being in line with Wokke et al.s’ (submitted) suggestion of a compensatory push-pull interaction between the dorsal and the ventral stream. However, in this study, no clear contribution of MT has been found, therefore it remains unclear whether inhibition of LO automatically leads to a greater influence of MT. Other regions within the visual cortex could also play a role. Further analysis might be fruitful to shed
References
Appelbaum, L. G., Wade, A. R., Vildavski, V. Y., Pettet, M. W., & Norcia, A. M. (2006). Cue-invariant networks for figure and background processing in human visual cortex. The Journal of
neuroscience : the official journal of the Society for Neuroscience, 26(45), 11695–708.
doi:10.1523/JNEUROSCI.2741-06.2006
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrika, 93, 491–507.
Cowey, A., & Walsh, V. (2001). Vision: From Neurons to Cognition. null (Vol. 134). Elsevier. doi:10.1016/S0079-6123(01)34027-X
Cárdenas-Morales, L., Nowak, D. a, Kammer, T., Wolf, R. C., & Schönfeldt-Lecuona, C. (2010).
Mechanisms and applications of theta-burst rTMS on the human motor cortex. Brain topography,
22(4), 294–306. doi:10.1007/s10548-009-0084-7
Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics. Journal of Neuroscience Methods, 134, 9–21.
Dumoulin, S. O., Bittar, R. G., Kabani, N. J., Baker, C. L., Le Goualher, G., Bruce Pike, G., & Evans, A. C. (2000). A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning. Cerebral cortex (New York, N.Y. : 1991), 10(5), 454–63. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10847595
Enns, J. T., & Di Lollo, V. (2000). What’s new in visual masking? Trends in Cognitive Sciences, 4, 345–352. Farivar, R. (2009). Dorsal-ventral integration in object recognition. Brain research reviews, 61(2), 144–53.
doi:10.1016/j.brainresrev.2009.05.006
Ferber, S. (2003). The Lateral Occipital Complex Subserves the Perceptual Persistence of Motion-defined Groupings. Cerebral Cortex, 13(7), 716–721. doi:10.1093/cercor/13.7.716
Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in
Neurosciences, 15(1), 20–25. doi:10.1016/0166-2236(92)90344-8
Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Research, 41(10-11), 1409–1422. doi:10.1016/S0042-6989(01)00073-6
Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649– 677.
Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P., & Rothwell, J. C. (2005). Theta burst stimulation of the human motor cortex. Neuron, 45(2), 201–6. doi:10.1016/j.neuron.2004.12.033
Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurons in the cat’s striate cortex. Journal
of Physiology, 148, 574–591.
Kinsey, K., Anderson, S. J., Hadjipapas, A., & Holliday, I. E. (2011). The role of oscillatory brain activity in object processing and figure-ground segmentation in human vision. International journal of
psychophysiology : official journal of the International Organization of Psychophysiology, 79(3),
Lamme, V. A. F. (1995). The neurophysiology of figure-ground segregation in primary visual cortex.
Journal of Neuroscience, 15, 1605–1615.
Lamme, V. A. F., & Spekrijse, H. (1998). Feedforward, horizontal, and feedback processing in the visual cortex. Current Opinion in Neurobiology, 8, 529–535.
Lamme, V. a, & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 23(11), 571–9. Retrieved from
http://www.ncbi.nlm.nih.gov/pubmed/11074267
Likova, L. T., & Tyler, C. W. (2008). Occipital network for figure/ground organization. Experimental brain
research. Experimentelle Hirnforschung. Expérimentation cérébrale, 189(3), 257–67.
doi:10.1007/s00221-008-1417-6
Maunsell, J. H. R., & Newsome, W. T. (1987). Visual processing in monkey extrastriate cortex. Annual
Review of Neuroscience, 10, 363–401.
Mishkin, M., & Ungerleider, L. G. (1982). Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behavioural Brain Research, 6(1), 57–77.
doi:10.1016/0166-4328(82)90081-X
Perrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalography and Clinical Neurophysiology, 72(2), 184–187. doi:10.1016/0013-4694(89)90180-6
Raudies, F., & Neumann, H. (2008). Neural architecture of cortical motion feature detection, figure-ground segregation and selecitve visual attention. Perception, 37 ECVP Ab, 153.
Reddy, L., & Kanwisher, N. (2006). Coding of visual objects in the ventral stream. Current opinion in
neurobiology, 16(4), 408–14. doi:10.1016/j.conb.2006.06.004
Schenk, T. (2012). No dissociation between perception and action in patient DF when haptic feedback is withdrawn. The Journal of neuroscience : the official journal of the Society for Neuroscience, 32(6), 2013–7. doi:10.1523/JNEUROSCI.3413-11.2012
Schira, M. M., Fahle, M., Donner, T. H., Kraft, A., & Brandt, S. a. (2004). Differential contribution of early visual areas to the perceptual process of contour processing. Journal of neurophysiology, 91(4), 1716–21. doi:10.1152/jn.00380.2003
Scholte, H. S., Jolij, J., Fahrenfort, J. J., & Lamme, V. A. F. (2008). Feedforward and recurrent processing in scene segmentation: electroencephalography and functional magnetic resonance imaging.
Journal of cognitive neuroscience, 20(11), 2097–109. doi:10.1162/jocn.2008.20142
Wokke, M., Scholte, H. S., & Lamme, V. A. F. (n.d.). Opposing dorsal/ventral stream dynamics during figure-ground segregation.
Wokke, M., Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. (n.d.). Two critical periods of early visual cortex during figure-ground segregation. in press.
Zipser, K., Lamme, V. A. F., & Schiller, P. H. (1996). Contextual modulation in primary visual cortex.
de Haan, E. H. F., & Cowey, A. (2011). On the usefulness of “what” and “where” pathways in vision.
Trends in cognitive sciences, 15(10), 460–6. doi:10.1016/j.tics.2011.08.005
von der Heydt, R., Zhou, H., & Friedman, H. S. (2000). Representation of sterescopic edges in monkey visual cortex. Vision Research, 40, 1955–1967.
