perception.
Ayla Kruis*
Netherlands Institute for Neuroscience (NIN)
University of Amsterdam (UvA)
ABSTRACT
An essential feature of visual perception is the ability to discriminate foreground (figures) from background. This study investigated the neural effect of figure-ground segregation on one of the key low-level features of human perception: contrast. In order to gain insight into the neural effect of figure-ground modulation on contrast perception, an electrophysiological (EEG) experiment was performed. Event-related potential (ERP) measures of brain activity were recorded while observers viewed a partial checkerboard through a stereoscope, creating a clearly visible difference in ground and figure: the illusion that a contrast patch, a Gabor, was placed on foreground or background. Additionally, a psychophysical experiment was performed in order to obtain a detailed behavioral measure of the effect of figure-ground segregation on human contrast perception. We consistently found that the perceived contrast of the Gabor embedded in a texture seems to be lower on a figure compared to ground, possibly due to cross-channel masking. Complementing these findings, the results from the EEG study showed a difference between ERPs evoked by Gabors presented on figure-regions compared to Gabors placed on background-regions within a time-window of 150-250ms after onset of the Gabor-presentation. Additionally, a strong correlation was found between ERP amplitude and perceived contrast. In conclusion, the present study demonstrates that contrast perception, one of the low-level visual features, is modulated by figure-ground segregation, a high-level visual feature.
KEY WORDS: figure ground modulation (FGM), contrast perception, surround suppression, electro-encephalogram, event related potentials, electrophysiology, psychophysiology, cross-channel masking.
*Student nr: 10000034 (University of Amsterdam), e-mail address: a.kruis@nin.knaw.nl.
This research project was performed as a part of the Master Brain and Cognitive Science at the Netherlands Institute for Neuroscience under the supervision of Matthew Self, PhD in the Vision and Cognition Group.
[2]
INTRODUCTION
Visual perception can be seen as one of the most important human perceptual tools; most humans are highly dependent of their visual representation of a complex visual world. We perceive the world in scenes, with visual objects in their surroundings, embedded in a typical visual context. But how does visual context affect the elements constituting our global representation of the environment, and how is this processed in the human brain? In order to answer this question, a detailed investigation of the neural substrates of the visual system is required. The importance of this subject is reflected in the large number of studies addressing this issue over the last decennia, demonstrating different forms of context-dependent modulation of visual perception (e.g. Gilbert and Wiesel, 1990; Knierim & van Essen, 1992; Kovacs and Julesz 1993; Lamme, 1995; Yu et al. 2003; Supèr et al., 2010). Among the elemental features of visual perception is the segregation of objects from their background. This phenomenon, known as figure-ground segregation, is an example of a visual feature occurring in the higher visual areas, as it requires the integration of various cues from early lower visual areas.
Several decades ago, responses of visual neurons to a visual stimulus inside their classical receptive field (CRF) were shown to be modulated by stimuli outside their CRF (Allman et al., 1985). More recently, this type of contextual modulation was shown to occur with figure-ground elements in V1 of awake, behaving macaque monkeys. Studies investigating figure-ground segregation on a neural level demonstrate that the firing-rate of neurons in V1, one of the major early visual areas, depends on information provided by higher visual areas about whether their receptive field falls on a figure or a background (Lamme, 1995; Zipser, Lamme & Schiller, 1996). A late increased firing rate (starting at 100ms) was found in V1 neurons when their receptive fields fell on a figure-region compared to a background-region. These findings were proposed to reflect figure-ground segregation. In addition, as Zipser et al. demonstrated, contextual modulation recorded in monkey V1 neurons follows the perception of textures (Zipser, Lamme & Schiller, 1996). This implies that V1 plays an important role in the formation of a global representation of the world around us.
The modulation in firing-rate found in the studies mentioned above, known as figure-ground modulation (FGM), has been suggested to be the neural-correlate that accounts for perceptual figure-ground assignment. In this view, FGM occurs when global information from higher visual areas is provided to lower visual areas by intrinsic horizontal connections that connect surrounding neurons or (especially) by feedback projections (Lamme & Roelfsema, 2000; Roelfsema et al., 2002; Scholte et al., 2008).
[3]
Although the studies mentioned above, among others, have led to an increased understanding of the neural mechanisms underlying FGM, little is known about the neural substrates of figure-ground modulation of low-level visual features (such as contrast perception) in humans. In order to elucidate if and how low-level visual features are modulated by figure-ground segregation via feedback mechanisms, Self et. al (in prep.) measured the effect of figure-ground segregation on contrast perception in human subjects, using a psychophysical contrast-discrimination task. They demonstrated that apparent contrast perception of a probe stimulus is higher on figures compared to background. Based on these findings, they suggested this occurred because the neural representation of the probe was enhanced when presented on a figure.
Apart from the study by Self et al. (in prep.), to our knowledge, no other study has investigated whether contrast perception is modulated by figure-ground assignment in humans. An important point of concern, however, is a possible saliency effect of the stimuli used by Self et al. It is possible that the observed enhanced contrast perception on figures vs. background was mainly due to a higher saliency of figure regions. Attention is known to affect contrast perception (Carrasco et al. , 2004). Therefore, the increased contrast perception on figure regions might be due to participants directing their attention to the salient figure, rather than a pure measure of figure-ground segregation. An additional concern in the previously used stimuli is the first order Gabor patch (probe stimuli used for the contrast discrimination task), possibly forming a figure on its own on a figure region, thereby adding an extra layer to the stimulus and perhaps leading to the perception of a ‘figure on a figure on a background’ instead of a ‘figure on a background’ (Figure 1A and B).
In order to address the first (saliency-)issue, we have used a checkerboard pattern where figure- or background regions were constituted of multiple checks (figure 1B). Thereby, the saliency was equal for the entire stimulus surface. In order to solve the second concern, a second-order Gabor was employed in the present study: the Gabor patch was embedded in the texture itself of a figure- or ground region. The second order Gabor thereby allowed for a more direct measurement of the influence of the figure per se on contrast perception.
[4]
Figure 1. Examples of a first-and second order Gabor. In figure 1A, a figure region is placed on a ground region. A first order Gabor probe is placed on top of this figure region (fragment of the stimulus previously employed by Self et al., in prep). Figure 1B shows the type of stimuli as used in the present study. The figure region now consists of a partial checkerboard (5 checks), placed on a ground region. A second order Gabor is embedded in the structure of the middle check of the figure.
Overall, based on the studies demonstrating modulation of neural firing rate by figure ground segregation in monkeys (Lamme, 1995; Zipser, Lamme, & Schiller, 1996), the present study aims to investigate how figure-ground assignment affects the perception of contrast. In order to answer this question both on a behavioral (psychophysical) and neural (physiological) level, the study is divided into two complementary experiments, combining a psychophysical contrast-discrimination task and an electrophysiological (EEG) experiment, respectively. By these means, this study proposes an effective method to assess the role of figure-ground modulation on contrast perception, both on the behavioral and the neural level.
On a behavioral level, we hypothesize that Gabors will be perceived as higher in contrast when presented on a figure compared to a background. On a neural level, we expect an increased neural activity in figure-regions are compared to background-regions. More specifically, we expect a higher amplitude of event-related potentials (ERPs) in electrodes placed over early visual areas when Gabor patches are placed on a figure region compared to Gabor patches placed on ground regions. Finally, we hypothesize that the behavioral difference in perceived contrast is reflected in the difference in neural response evoked by contrast perceived on figure- or ground regions.
A
B
Figure
1st order Gabor
Ground
2nd order Gabor
Ground
Figure
[5]
GENERAL METHODS
Stimuli
All stimuli were constructed in MATLAB R2006b (MathWorks Inc.) using COGENT (developed by John Romaya at the LON at the Wellcome Dept. of Imaging Neuroscience) and were presented on a gamma-corrected CRT monitor at a viewing distance of 0.54m. All stimuli consisted of checkerboard textures (full screen checkerboard textures or partial checkerboards) composed of checks with orthogonally (45o or 135o) oriented line-elements (see figure 1B for an example of a partial checkerboard, and figure 6 for the distinction between a full and partial checkerboard). The screen (36.5o x 27.4o) was vertically divided into two equal halves (18.3 o x 27.4o ) in order to present the stimuli viewed though a mirror-stereoscope (see section below).
On every half of the screen, partial or full checkerboards (dependent on the precise configuration of the experiment) were made on a 10.5o x 10.5o square region, placed on a grey background with the same mean luminance as the checkerboard textures (38.8 cd.m-2). Both the partial and the full checkerboard figures were constructed by copying square regions (3o by 3o) of one texture onto the orthogonal texture (e.g., a 3o by 3o square of a texture with 45 o oriented lines was placed on a texture with 135 o oriented lines). Textures were composed of 15.910 lines of which the center coordinates were randomly chosen, with a length of 1.46 o and a thickness of one pixel (0.023 o). The lines had a high luminance (48.8 cd.m-2) and were drawn on a background with a low luminance (28.8 cd.m-2). The boundaries of the checks of the checkerboard were defined by the orientation difference of the texture (see also figure 1B).
In all experiments, perceived contrast was measured using second order Gabors (sinusoidal gratings, modulated by a Gaussian envelope), embedded in the texture of the central check of the checkerboard. These Gabors were created by modulating the luminance of the texture with a Gabor function comprising a sine-wave with a spatial frequency of 3 cycles per degree within a Gaussian envelope with a standard deviation of 1 degree, presented within a circular aperture of 1.15 o. integrating the sine wave of the Gabor with the line-elements of the texture of the central Gabor. The Gabors were cosine-phase with a phase of 180o, i.e. they had a dark-stripe in the middle, and they were always oriented vertically.
[6]
A
B
Stereoscope
Depth is perceived due to binocular disparity, defined as the difference in position between the visual stimulus presented to left eye and the right eye. When the two retinal representations of the image (from the left and the right eye) are combined, the image is perceived as three-dimensional. In the experiments throughout the study, all stimuli were viewed through a mirror-stereoscope. With the mirror-stereoscope, two separate images were presented to the left and right eye with a small binocular disparity, thereby creating the illusion of physical depth for the partial checkerboards. The mirror-stereoscope set-up is shown in figure 2, an example of a stimulus presented with binocular disparity is shown in figure 3.
The perception of contrast for Gabors perceived on figures or backgrounds was compared. In the (back)ground condition, the Gabor was embedded in the texture of the background and presented at the plane of fixation, at 0 disparity. The four checks of the partial checkerboard surrounding it were presented at a crossed disparity. I.e., the four checks presented to the right eye were shifted by 0.23° (to the right) relative to the background and Gabor presented to the left eye, so that they appeared to be shifted towards the observer in depth relative to the Gabor. In the figure condition, the Gabor was embedded in the texture of the central check of a pattern of five checks, presented at a 0 disparity level. In this condition, the five checks and the Gabor presented to the right eye were shifted by -0.23° (to the left) relative to the background presented to the left eye. This led to the perception that the background was shifted backwards relative to the figure and the Gabor. The Gabor was always presented at a the level of the fixation plane in order to ensure that participants fixated on the same depth plane in all conditions, thereby preventing fixation bias.
Figure 2. A Stimulus perception through a mirror – stereoscope. In this example, participants perceive a square through their left eye, a circle through their right eye. Both images are merged by the visual
[7]
system, leading to the perception of a circle placed in a square. B. Experimental set-up using a mirror stereoscope, side view. The subject perceives the stimulus through the stereoscope, the image projected in the left and right eye are separated by a vertical screen.
Figure 3. Example of employed stimulus. The red dashed line indicates the position of the screen separating the left from the right visual field when viewed through the stereoscope. The black dashed lines in the left half (not shown in the actual stimulus) demarcate the checks constituting the partial ground and the partial figure. For the partial ground, the Gabor is perceived to be part of the background, further away relative to the four checks surrounding it. For the figure, the Gabor is perceived as being part of the texture of the central check of the five checks forming the partial figure. The figure seems to be closer relative to the background texture. The image presented to the right eye is slightly shifted to create disparity.
[8]
Calibration
Some observers do not fixate accurately at the zero disparity fixation plane: their fusion angle (between the visual axes of the two eyes) may correspond to points slightly nearer or farther away. In order to correct for this fusion error (or fixation disparity) and to optimize the perception of disparity, a calibration screen was presented to every participant before each session (in all experiments). The distance between the images perceived by both eyes could be adapted until optimal fusion was reached (figure 4A). Subsequently, nonius lines were presented before every session of all experiments and adjusted to subjective alignment (figure 4B). Calibration settings were saved for each participant after every session, but re-adjusted if needed.
Figure 4. A. calibration screen. The distance between the image presented to the left and right eye, separated by a screen (indicated by the red dotted line), was adjusted until optimal fusion of the two images was attained. B. Nonius lines were horizontally shifted until participants indicated that the upper- and lower lines were perfectly aligned.
QUEST staircase procedure
In all experiments perceived contrast was measured by assessing the Point of Subjective Equality (PSE): the point at which participants were not able anymore to discriminate between the contrast of the test Gabor and the reference Gabor, placed on figure (F) or ground (G). The PSE was measured under control of the QUEST algorithm (Watson and Pelli, 1983). The contrast of the reference Gabor (30 % Michelson contrast) was chosen as a start value for the PSE. In every trial, participants were asked which Gabor was higher in contrast. Depending on the subject’s decision, the contrast of the test Gabor was adapted after every trial by QUEST in order to obtain a precise estimation of the point of subjective equality for all subjects.
[9]
PSYCHOPHYSICS
In order to investigate on a behavioral level whether contrast perception in humans is affected by figure-ground assignment, two psychophysical experiments were conducted. In the first experiment, participants were asked to judge the contrast of Gabor-patches placed on figure-and ground-regions. Based on studies demonstrating that neural activity is enhanced when contrast is perceived on a figure-texture as compared to ground-figure-textures, we expected a higher perceived contrast of Gabors placed on figures vs. backgrounds.
Experiment 1 – Methods
Participants
Six subjects (1 male, 5 female, mean age 27, sd. 4.6) gave their written informed consent to participate in the psychophysical experiment. All subjects had normal acuity (in some cases with corrective optics) and normal stereo vision. Two participants, including the author, were highly experienced in psychophysical tasks, the four other participants had no previous exposure to this experiment. The inexperienced observers were naive as to the purposes of the experiments.
Stimulus, task and behavioral measure
Participants performed 1200 trials, divided over three sessions of 400 trials. A calibration screen and nonius lines were presented and adjusted before every session. Every trial started with a fixation screen, displayed for 300ms at 0 disparity, followed by the textures only, for 300ms (see Figure 5). Subsequently, the Gabors (test and reference) appeared on the central checks. Test-and reference Gabors were pseudo-randomly presented on either figure or background, on the upper or lower half of the screen (randomized). The texture of the central check, in which the Gabors were embedded, were pseudo-randomly oriented 45° or 145°. Participants were instructed to indicate the Gabor with the highest contrast, by pressing the 1-or 2 key on the keyboard (1 for the upper Gabor, 2 for the lower Gabor). Participants had an unlimited response-time. The contrast of the test Gabor varied in contrast within the range of 25 – 35 % Michelson contrast, and was compared to a 30 % fixed contrast reference Gabor.
[10]
Calibration screen
Fixation screen,
300 ms
Textures only, 300 ms
Textures +
Gabors
Nonius lines
Figure 5. Sequence of the contrast-judgment task. Before every session, a calibration screen was presented and nonius lines were aligned for every individual subject. For every trial, a fixation screen was presented for 300ms, followed by the presentation of the texture (only), after which the Gabors were presented on the textures.
Experiment 1 - Results
The results of the six participants are shown in Figure 6. The contrast of the reference Gabor was always kept constant at 30% Michelson. The results as shown in Figure 6 represent the contrast value (averaged across participants and trials) at the point of subjective equality: the point at which participants were not able anymore to discriminate between the contrast of the test Gabor and the reference Gabor, placed on figure (F) or ground (G).
[11]
In order to ensure that the observed difference between Gabors placed on figure or background was not caused by a difference in orientation of the texture-lines of the central check, differences depending on orientation were analyzed. No significant difference was observed between Gabors placed on 135° or 45° oriented textures were perceived (paired t-test p > 0.05).
With high consistency across subjects, the results indicate a clear difference between perception of contrast when a Gabor is perceived on a foreground vs. background (p = 0.002, Wilcoxon signed rank test). In the ground (G) condition, the actual contrast of the test Gabor has to be on average 1,1% lower than the contrast of the (30% Michelson) reference Gabor in order to reach the point of subjective equality (Figure 6). In the figure (F) condition, the actual contrast of the test Gabor has to be on average 1,2% higher than the 30% reference contrast to reach the point of subjective equality.
Figure 6. Results of experiment 1 for each individual participant (bars indicate the average across participants). The x-axis shows whether the test Gabor was embedded in a ground texture (G), or a figure texture (F), and whether the lines of the central check, in which the Gabor was embedded, were oriented 135° or 45°. The y-axis shows the contrast of the test – Gabor. Perceived contrast for the ground condition is significantly higher than for the figure condition (p = 0.002).
[12]
Experiment 1- Discussion
We found consistent evidence that contrast perception is affected by the figure or ground structure on which a Gabor is placed. For the ground condition, the actual contrast of the test Gabor had to be around 1,2% lower than the contrast of the reference Gabor (which by definition was placed on a figure) in order to reach the level of subjective equality. This implies that when a Gabor was placed on a background, participants perceived the contrast of this Gabor higher than the Gabor placed on the figure. In the figure condition, the opposite effect was observed. The actual contrast of the test Gabor had to be higher than the contrast of the reference Gabor in order to be perceived as the same contrast. This means that when a Gabor was placed on a figure region, participants perceived the Gabor-contrast as lower than the Gabor placed on a background. The results were very consistent across all participants. Furthermore, the difference between contrast perception on figure or background was highly significant (p < 0.005, Wilcoxon signed rank test).
These results seem counterintuitive, since recent evidence showed an increase of perceived contrast on figures compared to backgrounds (Self et al, in prep.). A possible explanation could be that an enhanced neural response in the figure-region might lead to increased surround suppression, decreasing the perceived contrast of the test-Gabor . Reversely, inhibition of the neural response in the ground-region could lead to decreased surround-suppression, thereby releasing the suppression of the Gabor, leading to an enhancement of the perceived contrast. However, based on these results, it is not possible to say whether this modulation of contrast-perception is mainly due to the enhancement of figure-regions or suppression of ground-regions. Therefore, an additional experiment was conducted. In this experiment, instead of directly comparing Gabors placed on figure vs. ground regions, an additional condition, a full checkerboard, was added. The test-Gabor was now always placed on a (flat, 0-disparity) checkerboard and compared with the reference-Gabor, placed on either a partial figure or ground.
Experiment 2 - Methods
Subjects
Seven subjects (1 male, 6 female, mean age 24.1, sd. 4.7) gave their written informed consent to participate in the psychophysical experiment. All subjects had normal acuity (in some cases with corrective optics) and normal stereo vision. Three participants, including the author, were highly experienced in psychophysical tasks, the four other participants had no previous exposure to this
[13]
experiment. Three out of the four inexperienced observers received a short contrast modulation task prior to the experiment, in order to familiarize them with the discrimination of contrast differences.
Stimulus
In this experiment, test-and reference Gabors were still compared in a two alternative spatial forced-choice task (2AFC). In contrast to the previous experiment, ground-and figure condition were not directly compared. Instead, test Gabors were always placed on a full checkerboard, at a 0-disparity level, and compared to a reference Gabor placed either on a partial figure or a partial ground in a pseudo-randomized manner (Figure 7). Test and reference were randomly presented in the upper or lower position. As in the previous experiment, the task of the participants consisted of choosing the Gabor with the highest contrast, indicated by the 1-and 2 key on the keyboard (1 for the upper Gabor, 2 for the lower Gabor).
Figure 7. Example of employed stimulus in experiment 2. The black dashed lines (not shown in the actual stimulus) demarcate the checks constituting the partial ground, the partial figure or the checkerboard. In this example, the upper Gabor is part of a figure texture, and is compared to a Gabor embedded in the central check of a full checkerboard. The Gabor is always presented at a 0 disparity level.
[14] Reference condition
Con
tras
t
(%
M
ic
h
els
o
n
)
Participants performed 1200 trials, divided over three sessions of 400 trials. The sequence of the presentation of the stimuli is identical to the sequence employed in experiment 1 (a fixation dot was presented for 300ms, followed by the textures only, for 300ms, followed by the Gabors). In contrast to the previous experiment, participants compared a (test) Gabor on a full checkerboard with a 30 % Michelson contrast reference Gabor placed on either a figure or a background. Again, subjects were instructed to indicate the Gabor with the highest contrast, by pressing the 1-or 2 key on the keyboard (1 for the upper Gabor, 2 for the lower Gabor), with an unlimited response-time. The contrast of the test Gabor varied in contrast within the range of 25 – 35 % Michelson contrast, and was compared to a 30 % fixed contrast reference Gabor.
Experiment 2 - Results
Out of 7 participants participating in this control experiment, 3 were highly trained observers. The other 4 participants did not have prior experience in similar contrast judgment tasks, and performed a short contrast matching training task prior to the experiment in order to get acquainted with contrast judgment. One (inexperienced) participant was removed from the analysis due to extreme behavioral results. When averaged across participants, no significant difference was found between perceived contrast of reference Gabors placed on ground vs. figure (Figure 8, p > 0.05, Wilcoxon signed rank test).
[15] G-135° G- 45° F-135° F-45°
Reference condition
Con
tras
t
(%
M
ic
h
els
o
n
)
Figure 8. Results of experiment 2, averaged across all participants (with individual data shown with the differently colored symbols). On the x-axis, the contrast of the reference Gabor, embedded in a figure (F) or ground (G) structure, on a central check texture constituted of 135° or 45° oriented lines. The y-axis shows the contrast of the test Gabor in % Michelson) No significant difference was found for Gabors perceived on ground-or figure textures (p > 0.05).
Interestingly, when the perceived contrast of experienced observers was compared to the perceived contrast of inexperienced observers, a clear difference in contrast judgment was found. For the experienced observers, a significant difference between Gabors placed on figure or ground was found (Figure 9, p < 0.05, Wilcoxon signed rank test for every individual experienced participant). Furthermore, the results indicate that this difference is mainly due to an enhanced perceived contrast for Gabors placed on ground. This effect is highly consistent across the three experienced participants.
[16]
Figure 9. Results experiment 2, averaged across the experienced participants (with individual data shown with the differently colored symbols). On the x-axis, the contrast of the reference Gabor, embedded in a figure (F) or ground (G) structure, on a central check texture constituted of 135° or 45° oriented lines. The y-axis shows the contrast of the test Gabor in % Michelson) A significant difference was found between Gabors perceived on ground-or figure textures for all subjects (p < 0.05).
When the results of the inexperienced observers are analyzed separate from the results of the experienced participants, no significant difference was found between perception of contrast when Gabors were placed on a ground or figure (Figure 10). In addition, results across the inexperienced participants were highly inconsistent.
Figure 10. Results experiment 2, averaged across the inexperienced participants (with individual data shown with the differently colored symbols). On the x-axis, the contrast of the reference Gabor, embedded in a figure (F) or ground (G) structure, on a central check texture constituted of 135° or 45° oriented lines. The y-axis shows the contrast of the test Gabor in % Michelson. No significant difference was found between Gabors perceived on ground-or figure textures.
Reference
condition
G-135° G-45° F-135° F-45°Con
tras
t
%
ss)
M
ic
h
elso
n
)
[17]
Experiment 2 - Discussion
Inexperienced participants seem to perceive both reference Gabors placed on ground and figure higher than the (test) Gabors placed on the full checkerboard. This might be due to an attention effect: inexperienced observers might be biased by the saliency of depth-cues when compared to a flat checkerboard. This attention bias might lead to an enhanced perceived contrast of both Gabors placed on figure and ground when compared to Gabors on a full checkerboard. Based on the more consistent results of the experienced participants, it can be speculated that the difference in perceived contrast for Gabors on figure and ground regions is mainly due to enhancement of the Gabor presented on the ground. In other words, the suppression of the texture surrounding the Gabor placed on the ground-check might release the inhibition of the Gabor itself, leading to a higher perceived contrast. However, a follow-up study with additional (trained) participants is needed in order to investigate in more detail whether a difference in perceived contrast on figure or ground can be ascribed enhancement of figure-regions or suppression of ground-figure-regions.
ELECTROPHYSIOLOGY
Numerous previous studies have found an effect of figure-ground assignment on the neural response in V1 in awake, behaving macaque monkeys. With single-cell recordings, neural activity was recorded during viewing textured figure and ground displays (Lamme, 1995). Lamme used textured stimuli configured in such a manner that the receptive field of a V1 neuron under study received an identical pattern of stimulation from trial to trial. In spite of the identical stimulation of the receptive fields, V1 neurons responded more vigorously in trials in which orientation or motion of the texture on the receptive field belonged to a figure as compared to trials in which the texture was a background. The present psychophysical experiments (experiment 1 and 2) indicate that human contrast perception is affected by figure-ground assignment. In order to investigate these findings on a neural level and to relate human data to the data obtained in experiments with macaque monkeys, an EEG experiment was performed. We expected to measure a difference in event-related potential (ERPs) when a Gabor is perceived on a figure compared to when it is placed on a background.
[18]
Experiment 3 - Methods
Participants
Eleven healthy subjects (3 male, 8 female, mean age 23.5, sd. 1.5) with normal or corrected-to-normal vision and corrected-to-normal stereo vision gave their written informed consent to take part in the EEG experiment. Two participants, including the author, were highly experienced in psychophysical (discrimination) tasks, all other participants had little or no previous exposure to contrast-discrimination experiments. The participants received a small honorarium to compensate for their time.
Training
All of the 9 inexperienced subjects were given two short training sessions (2 x 15 mn) for familiarization with the task and contrast discrimination. The first training task consisted of an adapted (shortened) version of the task used in the psychophysical experiment. The second training task consisted of a contrast matching-version of the stimuli as employed for the psychophysics study. Instead of a forced choice procedure, participants were able to freely vary the contrast of the test Gabor to match the contrast of the test Gabor to the contrast of a reference Gabor. With this training task, participants were able to become acquainted with contrast discrimination.
Stimulus/task
For the EEG experiment, a temporal two alternatives forced choice version of the psychophysical experiments was designed (Figure 11). Stimuli were viewed through a mirror-stereoscope. In the psychophysical experiments, test-and reference Gabors were shown simultaneously on the screen. In the EEG experiment, test-and reference Gabor, pseudo-randomly placed on a figure or ground texture, were shown sequentially. Every condition (test/reference Gabor on figure/ground, orientation of the texture of the central check with 45°/135° oriented lines) was presented an equal number of times during every session. Every session consisted of 352 trials, with 8 repetitions of 11 contrast values for the Gabors. In contrast to the staircase procedure employed in the psychophysical experiments (experiment 1 and 2), a method of constant stimuli was used. In this method, eight different Gabor contrast-values were equally spaced by 0,75% around the reference contrast of 30 % Michelson, resulting in values ranging from 26,25 % up to 33,75 % Michelson contrast. Additionally, two extreme contrast values (20% and 40%) were added. Three sessions (or 1056 trials) were completed by all participants.
[19]
A
B
Figure 11. The stimuli employed in the EEG experiment. Dashed lines demarcating the checks are not shown in the real stimuli. A shows a Gabor embedded in the texture of the central check of a figure. The disparity of the background is uncrossed (-0.23°), the disparity of the partial figure (the five checks) is 0. B shows a Gabor embedded in the texture of the background. Here, the background with the embedded Gabor is at 0 disparity, with crossed disparity for the four checks surrounding it (thereby seemingly nearer than the Gabor on the background).
At the beginning of each trial, a fixation dot was presented for 500ms, followed the first texture, figure or ground, presented for 500ms. Subsequently, the test or reference-Gabor appeared embedded in the central check of the texture and remained on the screen for 500ms. Then, the second texture was presented (figure or ground, the opposite of the first texture), for 500ms. The second Gabor then appeared on the second texture (test or reference, dependent on the first Gabor, always the opposite). Test and reference stimuli were pseudo-randomly presented first or second in the sequence. Finally, the question ‘Which Gabor was higher in contrast, 1 or 2’ appeared on the screen. Participants had an unlimited response time, and responded by pressing the keys 1 or 2 on the keyboard in order to indicate which Gabor was higher in contrast (the first or second Gabor) (Figure 12).
[20] Fixation screen (500ms) 1st texture only (500ms) 1st texture + 1st Gabor (500ms) 2nd texture only (500ms) 2nd texture + 2nd Gabor (500ms) Question (until response) Fixation screen (500ms)
Figure 12. Schematic overview the stimuli presented during one trial of the EEG experiment. Before every session, a calibration procedure was applied. Every trial started with a fixation screen. After 500ms, the first texture (figure or ground) was presented, for 500ms. The Gabor (test or reference) then appeared embedded in the texture of the central check, for a duration of 500ms. Subsequently, the fixation screen was again presented for 500ms, followed by the second texture (figure or ground, opposite from the first texture) was presented for 500ms, followed by the Gabor (test or reference, opposite of the first Gabor), presented for 500ms. Finally, the question ‘Which Gabor was higher in contrast?’ appeared on the screen. The next trial started automatically after a response was given (i.e. when 1, for the first presented Gabor, or 2 for the second Gabor was pressed on the keyboard).
Experiment 3 - EEG acquisition
EEG measurements and pre-processing
EEG was recorded from the scalp while participants performed the behavioral task using a 256-Ag/AgCl electrode sponge-based EEG-cap (Hydrocel Geodesic Sensor Net, Electrical Geodesics, Inc.
[21]
(EGI)). Sponges were soaked for ten minutes in a 0.1 M KCl solution. Signals in all EEG channels were referenced to the vertex (Cz). Impedances were verified before the experiment and after every session (lasting approximately 30 minutes) and kept below 50 kΩ. Electrodes were refilled if necessary.
During the experiment, online sampling rate was 1000 Hz, with a 0.1 Hz high pass filter and a 50 Hz Notch filter. Event-related potentials (ERPs) were computed as the average ERP amplitude across all trials per condition. The behavioral data obtained in the EEG experiment was analyzed by averaging the ERPs and comparing the average amplitude of the ERPs measured in early visual areas when Gabor patches are placed on a figure region vs. Gabor patches placed on ground regions.
Raw data were pre-processed off-line before analysis, at a sampling rate of 500 Hz. A 0.5 Hz high pass filter was then applied. For event related potential (ERP) analysis, data was lowpass filtered at 30 Hz. Bad channels and bad samples (e.g. artifacts caused by eye movements (EOG) and muscle activation (EMG)) were removed before analysis. The signal from each electrode was re-referenced to the average signal across all electrode sites. Offline EEG analysis was performed using the Field Trip open source toolbox (Oostenveld, Fries, & Jensen, 2009; documentation and algorithms available at ru.nl/fcdonders/fieldtrip) in MatLab R2006b (The MathWorks, Natick, MA).
Statistical analysis
Differences between experimental conditions were ascertained using sample-by sample paired t-tests (Bonferroni corrected for multiple comparisons) one-way analyses of variance (ANOVA). An error probability of P = 0.05 was accepted as statistically significant throughout the study. Event-related potentials were averaged over a selection of 55 manually selected occipital electrodes. The obtained data were averaged over all selected electrodes and all subjects but one (excluded for extreme behavioral results).
Experiment 3 - Results
Behavioral results
Behavioral results obtained during the EEG experiment are shown in figure 13. The data show that when a test-Gabor is placed on a partial ground texture, the point of subjective equality (PSE) is lower than 30 % Michelson contrast. This indicates that at this PSE, the low physical contrast of the test-Gabor is perceived as the same contrast as the (30%) reference Gabor. This means that participants perceive the contrast of the test-Gabor as higher than it actually is, when placed on a ground-region. Reversely,
[22]
when a test-Gabor is placed on a figure region, the contrast value of the PSE is higher than 30% Michelson. This indicates that when a Gabor is placed on a figure region, participants perceive the contrast of the test-Gabor as lower than the physical contrast of the reference Gabor.
Figure 13. Behavioral result of the EEG experiment. On the x-axis the contrast of the test Gabor is indicated in % Michelson. On the y-axis, the probability (P) that the test Gabor is perceived as being higher in contrast as compared to the (30%) reference Gabor. P=0.5 is the Point of Subjective Equality (PSE, ±SE): the contrast level of the test Gabor at which participants perceive no difference between the contrast of the test- and reference Gabor. At the PSE, the probability of choosing the test Gabor as the highest in contrast is then 50%).
Test on ground
Test on figure
[23]
ERP Results
The difference between amplitude of ERPs evoked by the presentation of the Gabor on a ground vs. figure region is shown in figure 14. Average minimal ERP amplitude was calculated for the presentation of the reference Gabors, across all participants, all 55 selected occipital electrodes and first and second presentation (reference Gabor presented as a first or second stimulus within a trial). Data were baseline corrected from -0.05s (before Gabor onset) to 0.05s (after Gabor onset). Differences in ERP amplitudes significantly different from zero in a sample-by sample paired t-test (Bonferroni corrected for multiple comparisons) are indicated with a star (*).
Based on the time-window in which a significant difference was found, 3 components were selected for further analysis: component 1, in a time-window of 150-180ms after Gabor-onset, component 2, from 180-250ms and component 3, combining component 1 and 2 (150-250ms).
Figure 14. Difference in (minimal) ERP amplitude as a result of presentation of the reference Gabor on a figure or ground texture (ERP’s evoked by Gabors placed on figure-textures minus ERP’s evoked by Gabors placed on ground-textures). Differences in ERP amplitudes significantly different from zero (Bonferroni corrected for multiple comparisons) are indicated with a star (*).
[24]
Relation between behavioral results and ERP results
We hypothesized that presentation of Gabors placed on figure-textures and ground-textures would evoke significantly different ERPs. More precisely, based on the data obtained in macaque monkeys (Lamme, 1995; Zipser, Lamme & Schiller, 1996), we expected a higher ERP amplitude for Gabors presented on figures compared to Gabors presented on backgrounds. Our results showed a significant difference between the figure-and ground condition. Additionally, we expected that the magnitude of this difference in ERP would predict the magnitude of the absolute difference in contrast perception evoked by figure-and ground textures. In order to relate the behavior of the participants during the EEG experiment to the neural activity during the experiment, behavioral results were correlated with the ERP amplitude measured at the time of presentation of the reference stimuli.
For each of the three components selected in the analysis of the ERP data (component 1: 150-180ms, component 2: 180-250ms and component 3: 150-250ms), a correlation analysis was performed with the behavioral data of each individual participant (i.e. the perceived contrast difference for Gabors placed on ground minus Gabors placed on figures at the PSE). A behavioral difference was found for every subject, ranging from 0.15 % to 1.75% perceived contrast difference, indicating that when presented on a ground-texture and compared to a figure-texture, test-Gabors needed to be 0.15-1.75% higher in contrast that the reference in order to reach the PSE.
Component 1
For component 1 (150-180ms after Gabor onset, Figure 15), 60% explained behavioral variance (
R
2= 0.6) was found, indicating that the behavioral difference (the difference in perceived contrast for ground vs. figure) correlates with the mean ERP evoked by ground vs. figure. The lower the mean ERP (and therefore the higher the amplitude), the larger the behavioral difference. One outlier was removed from the analysis (z > ±2,5).[25]
Figure 15. Correlation of behavior and mean ERP amplitude for component 1 (150-180 ms after presentation of the Gabor). The x-axis indicates the behavioral difference in % (the difference in contrast perception for ground minus figure textures), the y-axis indicates the mean ERP amplitude for component 1). Results are shown for every individual participant, indicated as a blue dot. One outlier (indicated by a black circle) was removed from the analysis (z > ± 2,5). R²=0.60.
Component 2
A high correlation was found for component 2 (180-250 ms after Gabor onset, Figure 16): the explained variance is 84% (R²=0.84). This indicates that the ERP amplitude in this time window is strongly correlated with the contrast differences perceived by the participants for Gabors placed on ground- regions compared to Gabors placed on figure-regions. One subject with an absolute z-score > 2,5 for the residual was excluded from the regression analysis.
[26]
Figure 16. Correlation of behavior and mean ERP amplitude for component 2 (180-250 ms after presentation of the Gabor). The x-axis indicates the behavioral difference in % (the difference in contrast perception for ground minus figure textures), the y-axis indicates the mean ERP amplitude for component 2). Results are shown for every individual participant, indicated as a blue dot. One outlier (marked with a black circle) was removed from the analysis (z > ± 2,5). R²=0.84.
Component 3
When the mean ERP amplitude within the time-frame of component 1 and 2 (150-250 ms after Gabor onset) is correlated to the difference in contrast perception, 63 % of the variance can be explained (Figure 17, R² = 0.63).
Figure 17. Correlation of behavior and mean ERP amplitude for component 3 (150-250 ms after presentation of the Gabor). The x-axis indicates the behavioral difference in % (the magnitude of the difference in perceived contrast for ground minus figure textures), the y-axis indicates the mean ERP amplitude for component 3). Results are shown for every individual participant, indicated as a blue dot. R²=0.63.
[27]
Experiment 3 - Discussion
The results obtained in the EEG experiment show a significant difference in ERP amplitude for reference Gabors placed on figure-or ground regions. Furthermore, a high correlation was found between the difference in neural response evoked by Gabors placed on figure-or ground regions and behavioral differences in perceived contrast. The correlation was especially high for the second analyses component, within a window of 180-250 ms after onset of the Gabor presentation. In this time-window, 84% of the variance in behavior was explained. This implies that the difference in neural activity evoked by presentation of Gabor-patches on figure-or ground textures is a good predictor of the perceived difference in contrast depending on the figure-ground condition.
The behavioral results found in the EEG experiment were highly consistent with the results found in the psychophysical experiment, in spite of the use of a temporal forced choice task instead of a spatial forced choice task. This confirms that the results found in both experiments were not merely due to the sequence of the presentation of the stimuli.
Another alternative explanation for the obtained results might be a difference in neural response due to subtle differences of the figure vs. ground texture per se, due to a small difference in shape (the partial ground consisted of 4 checks, the partial figure of five checks) and absolute disparity. I.e, although the Gabor was always presented on a 0 disparity level and the relative difference between fore-and background was always the same, the absolute difference was not equal. In the ground condition, the Gabor was embedded in the texture at 0 disparity, the level of fixation, with four checks at a crossed disparity (0.23°) surrounding it. In the figure condition, the level of disparity was 0 for the Gabor (embedded in the central check of the five checks forming the partial figure) , and uncrossed (-0.23°) for the background ‘behind’ the partial figure. These differences in shape and disparity however did not seem to affect the results: the difference in perceived contrast correlates poorly with the difference in ERP amplitude evoked by the texture difference per se (R² < 0.10, data not shown).
ERPs were averaged across 55 selected occipital electrodes. In summary, results from the EEG experiment suggest that the ERP amplitude in the occipital area is affected by the figure-or ground context on which Gabor-patches are presented. Furthermore, the difference in ERP amplitude for the figure vs. ground condition predicts the behavioral difference in perceived contrast. A topographic exploration of the found effect, by analyzing smaller areas, would be of great interest.
[28]
GENERAL DISCUSSION
The current experiments examined the modulating effect of figure-ground segregation on human contrast perception. Combining psychophysical and electrophysiological experiments, we have found consistent evidence that contrast perception is influenced by figure-ground segregation. Based on data obtained in macaque monkeys and humans (Lamme, 1995; Zipser, Lamme & Schiller, 1996; Scholte et al. 2008), we originally hypothesized that due to enhanced neural firing evoked by figure regions, contrast perception would be enhanced when perceived on figure regions, and suppressed when perceived on ground regions. Surprisingly, the consistent and highly significant results obtained in the psychophysical and EEG experiments demonstrated the opposite effect. When a Gabor patch is embedded in a ground region, contrast is perceived as higher when compared to a Gabor placed on a figure region. Reversely, when embedded in a figure region, contrast is perceived as lower when compared to a Gabor on a ground region.
A possible explanation for these results could be cross-channel masking, a phenomenon that can occur when dissimilar target and mask patterns are used, such as gratings differing in orientation (e.g., Derrington & Henning, 1989; Foley, 1994; Olzak & Thomas, 1991). It is thought that the different orientations stimulate separate neural mechanisms (Saarela & Herzog, 2009). The lines of the texture surrounding the Gabor were always orthogonally oriented (45° or 135°), whereas the Gabor was always vertically oriented (90°). If neural activity is increased when a figure texture (constituted of orthogonally oriented lines) is presented, this may enhance the cross-channel masking effect when the vertically oriented Gabor is presented, embedded in the figure texture. An enhancement of the figure texture might lead to an increased inhibition of the differently oriented grating of the Gabor patch, resulting in decreased perceived contrast when a Gabor is placed on a figure region. Conversely, ground-textures might lead to a decrease in neural activity, suppressing the ground-texture. If the ground-texture is suppressed, the inhibition of the Gabor embedded in a ground region (caused by cross channel masking) might be released, leading to an enhanced contrast perception. Further exploration of the effect of cross channel masking on figure-ground modulation would be of great interest.
In the control psychophysical experiment, a large behavioral difference was observed between contrast judged by experienced observers versus naïve observers. This difference was not observed in the first psychophysical experiment nor the EEG experiment, in which a Gabor presented on a figure was always directly compared to a Gabor presented on a background. In the second psychophysical
[29]
experiment, a (test) Gabor of varying contrast was presented on a full (flat) checkerboard and compared to a (reference) Gabor presented on either a figure or a ground. It is possible that when figure and ground are compared directly, the perceived contrast difference between the Gabors is relatively large, as compared to the difference between a figure/ground stimulus to a full checkerboard. This might have increased the difficulty of the task used in experiment 2, leading to a significant difference between trained and naïve observers. This is in line with previous research, showing that extensive training has a large impact on the performance of visual tasks (e.g., Treisman, Vieira, & Hayes, 1992; Wolfe et al., 1989).
Additionally, it is possible that the (flat) checkerboard and the (disparity-based) figure/ground structures were not equally salient. Attention is known to affect contrast perception (Carrasco et. al , 2004). As a result, and in combination with the increase in difficulty of the contrast-discrimination task, inexperienced subjects might have been biased towards choosing the reference Gabor, placed on the relatively more salient figure or ground instead of a higher contrast per se. The consistent results of the experienced observers might suggest that the difference found between the perception of contrast on figure or ground can mainly be attributed to a suppression of the background texture. A follow-up study is however necessary in order to confirm these results. Importantly, all participants should be trained in contrast-discrimination before participating in the experiment. A training program could consist of a more extensive version of the contrast-matching task as employed before the EEG experiment, or a task where feedback (correct/incorrect) is provided after every trial.
The differences in contrast-perception across subjects as measured behaviorally in the EEG experiment was shown to be strongly correlated with differences in neural response evoked by Gabors placed on figures or backgrounds. This is consistent with other studies demonstrating variability in contrast-perception across subjects (Cannon & Fullenkamp, 1996; Yu et al., 2001). The difference in contrast perception for Gabors placed on figures vs. ground do not seem to result from subtle differences in shape of the stimuli (i.e. the number of checks of the partial figure/ground and the difference in absolute disparity). However, these results are based on the perception of the (fixed contrast) reference Gabor. It would be interesting to investigate whether the ERP amplitude was affected by physical contrast differences of the test Gabor.
The highest correlation between the difference in neural response evoked by Gabors placed on figure-or ground regions and behavioral differences in perceived contrast was found between 180-250 ms after Gabor onset. It is thought that in visual processing in monkeys, the fast feed forward sweep is
[30]
completed within approximately 100ms. It has been proposed that recurrent connections are involved in the visual tasks in which longer delays are obtained (Lamme & Roelfsema, 2000). An EEG/fMRI study performed by Scholte shows an increased response to figures compared to backgrounds in early visual areas at around 150ms after stimulus onset. Modulatory effects related to figure-ground segregation have been suggested to depend on feedback from higher visual areas (Hupé et. al, 1998; Lamme et al., 1998; Roelfsema, Lamme, Spekreijse & Bosch, 2002). Although the effect of figure-ground assignment on early visual processing has been well studied on a neural level in macaque monkeys, comparable studies in humans are by far not as numerous (although see e.g. Kovacs & Julesz, 1993; Scholte et al., 2006; Scholte et al. 2008). Our findings are consistent with the idea that recurrent connections are involved in contextual modulation, since recurrent processing of the Gabor stimulus might be altered by figure ground modulation. However, further research, involving a larger number of participants, is essential in order to provide more conclusive evidence.
In summary, the findings presented in this study lead to the conclusion that human contrast perception is affected by figure ground modulation. As demonstrated on a behavioral and neural level, contrast is perceived as higher on ground regions when compared to figure regions, providing a new and interesting perspective on the modulation of low-level visual features by higher visual processing in humans.
Acknowledgments
I am heartily thankful to my supervisor, Matthew Self, whose patience, help and support from the initial to the final level of the research project enabled me to develop a better understanding of the subject. I am grateful to Danique Jeurissen for her help with the EEG experiments and to Giovanni Piantoni for his generous help with the analysis of the EEG data. Lastly, I offer my regards to all of those who supported me in any respect during the completion of the project.
[31]
REFERENCES
Allman J., Miezin F., McGuinness E. (1985). Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons.
Annual Review Neuroscience 8, 407–430.
Cannon, M.W., Fullenkamp S.C., (1991) Spatial interactions in apparent contrast: Inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations, Vision
Research 31; 11, 1985–1998.
Cannon M. W., Fullenkamp S. C. (1996). Spatial interactions in apparent contrast: Individual differences in enhancement and suppression effects. Vision Research, 33, 1685–1695.
Derrington A. M., Henning G. B. (1989). Some observations on the masking effects of two-dimensional stimuli. Vision Research, 29, 241–246.
Gilbert C.D., Wiesel T.N. (1990) The influence of contextual stimuli on the orientation selectivity of cells in primary visual cortex of the cat. Vision Research 30, 1689 –1701.
Hupé, J.M. et al. (1998). Cortical feedback improves discrimination between figures and background by V1, V2 and V3 neurons. Nature 394, 784-787.
Knierim, J. J. & Van Essen, D. C. (1992). Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology, 67, 961-980.
Kovacs, I., Julesza, B. (1993) Closed curve is much more than an incomplete one: Effect of closure in figure-ground segmentation, Proc. Natl. Acad. Sci. 90, 7495-7497.
Lamme, V.A.F. (1995). The neurophysiology of figure-ground segregation in primary visual cortex.
Journal of Neuroscience., 15, 1605-1615.
Lamme, V.A.F. et al. (1998). Feedforward, horizontal and feedback processing in the visual cortex.
Current Opinion in Neurobiology 8, 529-535.
Lamme V.A.F., Roelfsema P.R. (2000) The distinct modes of vision offered by feedforward and recurrent processing. Trends Neuroscience 23, 571–579.
Luck, S.J., Girelli, M., McDermott, M.T., Ford, M.A. (1997). Bridging the Gap between Monkey Neurophysiology and Human Perception: An Ambiguity Resolution Theory of Visual Selective Attention. Cognitive Psychology 33, 64–87.
Meese T. S., Hess R. F. (2004). Low spatial frequencies are suppressively masked across spatial scale, orientation, field position, and eye of origin. Journal of Vision, 4, (10):2, 843–859.
[32]
Olzak L. A., Thomas J. P. (1991). When orthogonal orientations are not processed independently. Vision
Research, 31, 51–57.
Roelfsema, P. R., Lamme, V. A., Spekreijse, H., & Bosch, H. (2002). Figure-ground segregation in a recurrent network architecture. Journal of Cognitive Neuroscience, 14, 525-537.
Saarela T., Herzog M. (2009) Size tuning and contextual modulation of backward contrast masking,
Journal of Vision 9 (11) 21.1- 21.12.
Scholte, H. S., Witteveen, S. C., Spekreijse, H., & Lamme, V. A. (2006). The influence of inattention on the neural correlates of scene segmentation. Brain Research., 1076, 106-115.
Scholte, H. S., Jolij, J., Fahrenfort, J. J., & Lamme, V. A. (2008). Feedforward and recurrent processing in scene segmentation: electroencephalography and functional magnetic resonance imaging.
Journal of Cognitive Neuroscience 20, 2097-2109.
Supèr H., Romeo A., Keil M. (2010) Feed-Forward Segmentation of Figure-Ground and Assignment of Border-Ownership. PLoS ONE 19(5): e10705.
Treisman, A., Vieira, A., & Hayes, A. (1992). Automaticity and preattentive processing. American Journal
of Psychology, 105, 341–362.
Watson A.B., Pelli D.G. (1983) QUEST: a Bayesian adaptive psychometric method. Perception &
Psychophysics 33, 113–120.
Wolfe, J. M., Cave, K. R., Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology: Human Perception and
Performance, 15, 419–433.
C. Yu, S.A. Klein, D.M. Levi (2001) Surround modulation of perceived contrast and the role of brightness induction. Journal of Vision 1, 18–31.
Yu, C., Klein, S.A, Levi, D.M. (2003) Cross- and Iso- oriented surrounds modulate the contrast response function: The effect of surround contrast, Journal of Vision 3, 527-540.
Zipser, K., Lamme, V. A., & Schiller, P. H. (1996). Contextual modulation in primary visual cortex. Journal
