Spread of object based attention during perceptual grouping: evidence for the growth-cone model.

Spread of object based attention during perceptual grouping: evidence for the

growth-cone model.

Klaudia Ambroziak

Supervisor: Danique Jeurissen

Abstract

In this study we aimed to compare three models of attention spread during perceptual grouping: filling-in model, pixel-by-pixel model and the growth-cone model. To test the models we designed experiments in which we compared the reaction times for the conditions where attention spread close to the edges with the conditions where attention spread in homogenous area. Participants indicated whether two cues (red dots, one dot was always the same as fixation point) were on the same or on two different objects. Two experiments were conducted. In the first experiment all the target dots appeared on the same distance from the fixation point, but on different distances from the edges of the figure. In the second experiment we adjusted the stimuli and the task design to measure different distances from the fixation point. We analyzed the reaction times patterns for all the cue locations and compared them with the predictions of the three models. The results of both experiment 1 and 2 support the growth-cone model and strongly suggests that attentional spread is faster if it spreads over homogeneous areas of the object.

1. Introduction

We live in visually rich and clustered world. Our eyes are constantly bombarded with perceptual input. The visual system needs to choose which objects are relevant for behavior and then segregate them from other objects and the background. This process of segregation is usually assumed to be divided in two stages. The first stage is preattentive, whereas in the second stage, visual attention is brought into play.

Two types of visual attention can be distinguished: spatial and object-based attention. According to spatial attention models, attention works like a spotlight that can catch certain spatial location. In object-based models perceptual groups (i.e. objects) within a visual scene guide shifts of attention (Egly, Driver and Rafal, 1994). These two types of attention are not necessarily mutually exclusive but can interact with each other (Behrmann, Zemel, and Mozer, 1998; Soto and Blanco, 2004). A common method used to investigate both space-based attention and object-based effects is a spatial cuing paradigm (Egly et al., 1994; Hecht and Vecera 2007, 2012) where attention is directed to one of two objects using a peripheral cue. Targets can appear at the cued location (valid), at an uncued location but on the attended object (invalid same-object), or at an uncued location on the unattended object (invalid object). Object-based attention is illustrated by faster responses to invalid same-object trials than invalid different-object trials even though the distance between both cues and fixation point is the same.

In addition to behavioral measures, an important challenge is to address in perceptual grouping on the neural level. One of the neurophysiological theories that explains attentive and preattentive scene segmentation is the incremental grouping theory (Roelfsema, 2006), which introduces a distinction between base and incremental grouping. The base-groupings are preattentive and provide an initial analysis of the visual scene by applying grouping and segmentation cues in parallel across the entire image. These processes are coded by single neurons with small receptive fields tuned to feature conjunctions, for example a color and an orientation. They are computed with a great speed due to feedforward connections. The complexity of tuning, and therefore the extent of the base-groupings, increases in higher visual areas where receptive fields are larger. Nevertheless, not all possible feature combinations can be coded by single neurons. If grouping cannot be resolved by base-groupings, the second stage follows called incremental grouping. Incremental grouping enhances firing of many different neurons coding features that are bound together in perception. This process takes more time than base-grouping because it involves not only feedforward but also horizontal and feedback connections with other brain areas. This spread of enhanced neuronal activity corresponds to the spread of object-based attention labeling image elements.

However, it is not clear how object-based attention operates during grouping of an object. Several models were proposed to explain how attention spreads during perceptual grouping. According to the filling-in model (Paradiso and Nakayama, 1991) attention first connects boundaries throughout the visual field. Attentional spread starts at the outlines and then fills in the object area. The pixel-by-pixel model predicts spread of attention (Houtkamp et al., 2003) which starts from one point (fixation) and continues over the rest of the object with the same speed in all directions: the time needed to group an object depends on the shortest pixel-by-pixel distance through an object. Similarly, the eccentricity model predicts that attentional spread starts from the point of fixation and continues over the rest of the object but the speed of attentional spread depends on the Euclidean distance between points. The next model is the growth-cone model (Jeurissen and Roelfsema, 2012) which predicts faster spread of attention on homogenous areas. Like pixel-by pixel model and the eccentricity model, this model implies that attention spreads from one point (fixation) over the rest of the object. However, the growth-cone model takes into account the size of receptive fields in visual areas. Small receptive fields in lower visual areas are necessary to resolve the grouping at a small scale, whereas larger receptive fields in higher visual areas can be used to resolve grouping at a larger scale. Thus, in homogenous areas, slower horizontal connections on the lower levels can be ‘by-passed’ by feedback from higher visual areas that can very fast group together similar, large areas. In consequence, attention can spread faster over homogenous areas.

Evidence for the growth-cone model comes from a research of Jeurissen et al. (2012). In this experiment subjects had to indicate whether two cues were on the same or on two different objects. Participants were randomly assigned to one of the four stimulus sets that varied in their level of complexity: natural images, detailed cartoons, cartoon outlines, and scrambled cartoons. The authors analyzed whether the reaction time pattern for each condition was in accordance with the prediction of three models: growth-cone model, pixel-by-pixel model and the eccentricity model. Regression analysis of the RT-data showed that the explained variance was highest for the growth-cone model. Furthermore, this superior performance of the growth-cone model was very consistent for the various image conditions. These results demonstrated that attentional spread is faster if it spreads over homogeneous areas of the object and slower on small or narrow part of the object.

Other object-based attention studies showed the results that support the growth-cone model. For example, thestudy

by Hecht and Vecera (2012) examined the influence of an object’s structure on object-based shifts of visual attention. Observers viewed single-part objects and three-part objects in a spatial cuing paradigm (Egly et al., 1994). The inter-stimulus interval (ISI) between the cue’s offset and the target’s onset could vary (0, 100, 350 ms). Object-based attention effect (faster responses to invalid same-object trials than invalid different-object trials) emerged at all ISIs for single-part objects. However, for more complex objects this effect was only observed at a 350 ms ISI. Since reaction times in “invalid same-object” condition decreased as ISI increased, these results suggest that attention requires time to spread through complex objects. These findings are consistent with the growth-cone model. Because on more complex objects attention has to spread over heterogeneous areas, larger receptive fields in higher visual areas cannot be recruited and smaller receptive fields in lower visual areas are necessary.

Does object-based attention spread faster in homogenous areas? To answer this question the study was designed in which we compared the reaction times for the conditions where attention spread close to the edges with the conditions where attention spread in homogenous area. Participants indicated whether two cues were on the same or on two different objects. The cues were red dots: one dot was always the same as fixation point and the other could appear in different locations. This research was the follow up/continuation of the study by Jeurissen et al. (2012). In the previous experiment only a few locations within many different objects was tested. Here, we designed a stimulus that would generate very different reaction-time predictions for the filling-in, pixel-by-pixel, and growth-cone model and could thereby maximally distinguish between them. The eccentricity model would predict the same reaction times as the pixel-by-pixel model, thus we did not include it. Two experiments were conducted. In the first experiment all the target dots appeared at the same distance from the fixation point, but at different distances from the edges of the figure (Figure 1). In the second experiment we adjusted the stimuli and the task design to measure different distances from the fixation point (Figure 5). In both experiments we looked at the reaction times for different locations and compare results for dots that appeared close to the edges with the results for dotes that appeared farther from the edges. To test the models, we investigated whether the reaction time patterns were in accordance with the models’ predictions.

2. Experiment 1

Experiment 1 aimed to compare three models: filling-in model, pixel-by-pixel model and the growth-cone model. To test the models, we analyzed the reaction times for the conditions where attention spreads close to the edges with the conditions where attention spreads on the large homogenous central area of the object. Subjects performed a computer task in which they indicated whether two cues were on the same or on two different objects. The cues were red dots: one dot was always the same as fixation point and the other could appear in twelve different locations within each figure. All the target dots appeared on the same distance from the fixation point, but on different distances from the edges of the figure (see Figure 1 A and B).

Figure 2: The examples of stimuli used in the experiment 1: A) a dot that appeared far from the edges and B) a dot that appeared close to the edge.

Three models made different predictions on the time-course of attention spreads during grouping of an object (Figure 1). The filling-in model predicted that reaction times would be the shortest for the cues appearing close to the edges, and longer for all other cues. The pixel-by-pixel model predicted that reaction times for all cue locations would be the same, since they were all at the same distance from the fixation point. The growth-cone model predicted that the reaction times for the target dots appearing in the middle of the figure (further from the edges) would be the shortest, since on the distance between the fixation point and the dots appearing farther from the edges the attention spreads in homogenous area. Longer reaction times for the dots closer to the edges were expected.

A

C

Figure 2: Reaction-time predictions made by three models: A) the filling in model, B) the pixel-by-pixel model, C) the growth-cone model. Numbers from 0 to 72 on the X-axis represent the location of the dot in degrees of the circle e.g. the value “36 degrees” corresponds with the dot in the middle of the figure. The values on the Y-axis are shown for the comparison, the main point of this graph is the relative difference between the models, and absolute reaction-time values presented here are not relevant.

2.1. Materials and Methods

Twenty subjects (14 females and 6 males, aged 19-55, average 24.5) with normal or corrected-to-normal vision participated in this experiment. For their participation, subjects received monetary reward. All participants gave their written informed consent to participate in this study. The experiment was approved by the local ethics committee.

Stimuli

Subjects were shown two figures: triangle wedges formed by two 72 -degrees sectors of a circle (Figure 1 and 2). Both wedges had the same radius and shape. The cues were two red dots. One dot in the center of the screen was also the fixation point. The other, the target dot always appeared on the same distance of 250 pixels from the fixation point. The dot could appear on the same or different wedge on 12 locations (24 in total). Stimuli were presented on a white background. To make the task more difficult, on each trial wedges changed rotation or/and flip: wedge with the fixation point could appear either on the left or on the right side.

Task

First the red fixation dot appeared in the middle of the screen for 500 ms to indicate the start of the trial (Figure 3). Then, a stimulus was shown until participant gave the response. Subjects were asked to indicate by a button press whether the red dot appeared on the same (50% of the trials) or different (50% of the trials) figure as the fixation point. On each trial subjects received visual feedback about the correctness of their response for 1000 ms. If subject made an error, the same trial was repeated later, which resulted in 100% correct response for each participant. Subjects were instructed to focus on the fixation point during each trial. If subject made an eye movement the trial was aborted and repeated later.

Figure 3: Trial design in time: fixation (500ms), stimulus display (until response) and feedback (1000ms).

Procedure

Participants sit 50 cm from the computer screen. The subject’s head was stabilized on a chin rest and an infrared Eyelink-1000 (SR Research, Osgoode, ON, Canada) system monitored eye position to control for the eye movements. First, the calibration of the eye tracking system was done to determine where the subject’s pupil was and to map the gaze position onto the computer monitor via a tracking algorithm.

Thereafter, subject were shown a small number of testing trials chosen randomly from all possible conditions (target location (12) x wedge rotation (9) × left/right flip (2) x same/different figure (2)). Then, in the experimental task subjects performed 4 trials in each condition in randomized order, resulting in a total of 1728 trials. Each target location occurred 72 times within each edge. Each wedge type (rotation (9) x left/right flip (2)) occurred 8 times per block, 96 times in total. The task consists of 1 session divided into 12 blocks of 144 trials.

Presentation of stimuli, recording of eye position, and data analysis were implemented in Matlab (Mathworks, Natick, MA) using psychophysics and eyelink toolbox extensions.

2.2. Results

In the present study we analyzed the reaction times for the cues in 24 different locations: 12 locations for both same and different figure (Figure 4). We compared the results with the predictions of three models: filling-in model, pixel-by-pixel model and grown-cone model.

As predicted by the grown-cone model (see Figure 2), in the “same figure” condition, reactions times were significantly slower for the target dots that appeared close to the edges of the figure and faster for the dots farther away from the edges of the figure (Figure 4, blue bars). When the target dot was located on the other figure, we did not observe the similar pattern (Figure 4, grey bars). As expected, the reaction times in “other figure” condition were generally slower than the reaction times in “same figure” condition.

Figure 4: The results of the experiment 1: mean reaction times for 24 possible cue locations. Blue bars represent 12 locations on the “same” figure, and the grey bars represent 12 locations on the “different” figure. Bar twelve (last blue bar) and thirteen (first grey) present the reaction times for the cues located close to the border edge between two figures, whereas the first blue bar and the last grey bar (number 24) show the reaction time for the cues located close to the outer edges.

2.3. Discussion

The results of the experiment 1 showed that the prediction of the growth-cone model made the best fit to the data. As expected, the reaction times for the cues appearing close to the edges of the figure were longest, whereas the reaction times for the dots appearing farther from the edges were shortest. On the distance between the fixation point and the dots appearing farther from the edges the attention spread in homogenous area, thus this result show that grouping of similar, large areas is faster.

This superior performance of the growth-cone model was consistent with the results of the studies using natural images and cartoons (Jeurissen et. al 2012). In the previous experiments, however, only a few cue location within many different objects were tested. Here, we analyzed reaction times for 12 different cue locations within a single object. All the cues, however, always appeared at the same distance from the fixation point. The interesting question that emerged was whether the same reaction-time pattern can be found for different distances from the fixation point. Thus, the experiment 2 was conducted to investigate if the similar effect is present for many other location within a single figure including different distances from the fixation.

3. Experiment 2

In the experiment 1 all the target dots appeared at the same distance from the fixation point, but at different distances from the edges of the figure (Figure 2). In the experiment 2 we adjusted the stimuli and the task design to measure different distances from the fixation point (Figure 5). We analyzed the reaction times for different locations and compare results for dots that appeared close to the edges with the results for dots that appeared farther from the edges. Likewise in the experiment 1, we aimed to compare the 3 models: the filling-in model, the pixel-by-pixel model and the growth-cone model.

Figure 5: All possible location of the target dot in the experiment 2. In this image different colors were used to demonstrate different distances from the fixation point (red dot). In experiment all cue dots were always red.

Again, three models made different predictions on the time-course of attentional spread during grouping of an object (Figure 6). The filling-in model predicted that reaction times would be shortest for the cues appearing close to the edges, and gradually got longer for the cues closer to the middle of an object. The pixel-by-pixel model predicted that reaction times for cue locations farther from the fixation point would be longer, but would stay the same for all cues at the equal distance from the fixation point. The growth-cone model predicted that the reaction times would be modulated by both: the distance from the fixation point and the distance from the edges of the figure. Expected reaction times would be shortest for the cues appearing close to the fixation point, and gradually got longer for the cues farther from the fixation. However the reaction times would stay shorter for the cue location close to the middle of the figure, since on the distance between the fixation point and the dots appearing farther from the edges the attention spreads in homogenous area.

A

B

C

Figure 6: Predictions made by three models: A) the growth-cone model, B) the pixel-by-pixel model, C) the filling in model. Blue color indicates the shortest reaction times and red color indicates longest reaction times.

3.1. Materials and Methods

30 subjects (21 females and 9 males, aged 18-40, average 22.8) with normal or corrected-to-normal vision participated in this experiment. For their participation, subjects received monetary reward. All participants gave their written informed consent to participate in this study. The experiment was approved by the local ethics committee.

In the second experiment we used the same figures as in experiment 1. There were always two wedge figures and two red dots shown in the stimulus display. This time however, 6 different distances from the fixation point were introduced (Figure 5). The target dot could appear 60, 100, 150, 200, 250 and 294 pixels from the fixation points. There were 7 possible locations at the distance of 294, 6 locations at the distance of 250 and 200, 5 locations at the distance of 150 and 100, and 3 locations at the distance of 60 pixels. Thus, the target dot could appear in 32 locations within each figure (64 locations in total, see Figure 5). Stimuli were presented on a white background.

Task

The task in Experiment 2 was the same as in Experiment 1. The duration of the visual feedback on each trial was reduced to 500 ms to minimize the length of the whole experiment.

Procedure

As in experiment 1, the eye movements where monitored using Eyelink-1000 (SR Research, Osgoode, ON, Canada). Subjects performed 2 trials in each condition (location (32) × wedge type (8) x left/right (2) x same/different (2)), resulting in a total of 2048 trials. The experiment consists of 1 session divided into 16 blocks of 128 trials (2 trials per condition: location (32) × same/different (2)).

3.2. Results

In this experiment we analyzed the reaction times for the cues in 64 different locations: 32 locations for each “same figure” and “different figure” conditions (Figure 6). Again, we compared the results with the predictions of three models: filling-in model, pixel-by-pixel model and grown-cone model.

As predicted by the grown-cone model, in the “same figure” condition, reactions times were significantly slower for the target dots that appeared close to the edges of the figure and faster for the dots farther away from the edges of the figure. Importantly, this pattern was similar for different distances from the fixation point except for the cue locations very close to the fixation point. Additionally, as predicted by both the pixel-by-pixel model and the growth-cone model (but not filling-in model), reaction times got longer for the cues farther away from the fixation point. Furthermore, according to our expectation, when the target dot was located on the other figure, we did not observe the same pattern. This finding was consistent with the results of the experiment 1. Again, the reaction times in “other figure” condition were generally slower than the reaction times in “same figure” condition.

Regression analysis of the reaction times in the “same figure” condition revealed that the explained variance (R2_{) was}

the highest for the growth-cone model (69%) whereas for the filling-in and pixel-by-pixel models the explained variance (R2_{) was 48% and 15% respectively (Figure 8). The correlation between results predicted by the growth-cone}

model and the results obtained in experiment 2 was high (R = 0.82). For the pixel-by-pixel models this correlation was smaller (R = 0.69) and for the filling-in model the correlation was negative (R = -0.38).

Figure 7: The results of the experiment 2: mean reaction times for 64 possible cue locations. Color bars represent 32 locations on the “same” figure, and the different shades of grey represent 32 locations on the “other” figure. The same colors where used in Figure 5 to demonstrate different distances from the fixation point. Similarly, in this graph light blue bars represent the reaction times for the cues located close to the fixation point on the “same” figure, whereas the dark grey bars show the reaction time for the cues located close to the fixation point on the “other” figure. The first light blue bar show mean reaction time for the cues located close to the outer edge and the last light blue bar show mean reaction time for the cues located close to the border edge between two figures.

B

C

Figure 8: The correlation between the reaction times obtained in experiment 2 and the reaction times predicted by A) the growth-cone model, B) the pixel-by-pixel model, C) the filling in model.

3.3. Discussion

The results of the experiment 2 supports the growth-cone model by showing that attention spreads faster over homogeneous areas of the object and slower on parts of the object that are close to the edges. This reaction-time pattern was consistent for the various distances from the fixation point. Only for the distances very close to the fixation point the reaction times were similar for all cue locations, which can also be explained by the growth-cone model, since on short distances, small receptive fields are sufficient to resolve grouping and bigger receptive fields from the higher level are not engaged at this stage. The results of the experiment 2 strongly demonstrate that the prediction of the growth-cone model made the best fit to the data when compared to the predictions of filling-in

4. General Discussion

In the present study we aimed to compare three models of attention spread during perceptual grouping: filling-in model, pixel-by-pixel model and the growth-cone model. To test the models we designed experiments in which we compared the reaction times for the conditions where attention spreads close to the edges with the conditions where attention spreads in homogenous area. Participants indicated whether two cues were located on the same or on two different objects. Two experiments were conducted. In the experiment 1 all the target dots appeared on the same distance from the fixation point, but on different distances from the edges of the figure. In the experiment 2 we adjusted the stimuli and the task design to measure different distances from the fixation point. Results revealed that, when both dots were located on the same figure, reactions times were significantly slower for the target dots that appeared close to the edges of the figure and faster for the dots farther away from the edges of the figure. Since on the distance between the fixation point and the dots appearing farther from the edges the attention spreads in homogenous area, the results of both experiment 1 and experiment 2 support the growth-cone model.On the basis of this model, our findings can be explained by larger receptive fields from higher visual areas that were recruited to resolve grouping at a larger scale. In homogenous areas, slower horizontal connections on the lower levels can be ‘by-passed’ by feedback from higher visual areas that can very fast group together similar, large areas. In consequence, attention can spread faster over homogenous areas.

This explanation is also consistent with the incremental grouping theory according to which base groupings are done in parallel across visual scene, whereas incremental grouping is a serial processes requiring attention. In this framework grouping of an object requires incremental grouping and involves feedback projections from the higher visual areas. Previous studies (Houtkamp, Spekreijse and Roelfsema, 2003) provides evidence for a role of attention in contour grouping. The spatial distribution of attention was investigated in a curve tracing task where subjects had to trace a target curve connected to a fixation point. Results showed that reaction times significantly increased along with a number of intersections between the target and the distractor curves, demonstrating that contour grouping can be time consuming. Attention gradually spreads across the curve until all contour elements are grouped together. The present experiment generalizes the results from curve-tracing studies to perceptual grouping of larger areas. By showing the different reaction times for detecting targets in various locations within a figure, this study demonstrates that grouping of objects is time consuming and is resolved by serial processes. Therefore, the serial grouping observed in a curve-tracing task is not a special case but it also occurs for more complex figures.

A notable point is that in both experiments, the reaction-time pattern predicted by the growth-cone model was only found in a condition when the target dot appeared on the same figure as the fixation point. When the target dot was located on the other figure, the mean reaction times were very similar for all the locations. The lack of a similar reaction-time pattern in the “other figure” condition may suggest that subjects used different cognitive strategy in these trials i.e. subjects first grouped the figure with the fixation dot and when they did not find the target dot, they answered “different” even without grouping the other figure.

The superior performance of the growth-cone model in this research was consistent with the results of the studies using natural images and cartoons (Jeurissen et. al 2012). This suggests that the human brain uses similar strategies during perceptual grouping of simplified as well as natural objects. Moreover, other results of the object-based attention studies also seem to have a good explanation on the basis of the growth-cone model e.g., the delay of attention spread through complex objects (Hecht and Vecera, 2012).

References

Behrmann, M., Zemel, R. S., and Mozer, M. C. (1998). Object-based attention and occlusion: Evidence from normal participants and a computational model. Journal of Experimental Psychology: Human Perception & Performance, 24, 1011-1036.

Egly, R., Driver, J., and Rafal, R. D. (1994). Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161-177.

Hecht , L. N. Vecera , S. P. (2012). Attentional Selection of Simple and Complex Objects. Proceedings of the 12th

Annual Meeting of the Vision Sciences Society. Naples, Florida

Hecht , L. N. Vecera , S. P. (2007). Attentional selection of complex objects: Joint effects of surface uniformity and part structure. Psychonomic Bulletin & Review

Houtkamp, R., and Roelfsema, P. R. (2010). Parallel and Serial Grouping of Image Elements in Visual Perception .

Journal of Experimental Psychology: Human Perception and Performance.

Houtkamp,R., Spekreijse, H., Roelfsema, P.R. (2003) A gradual spread of attention during mental curve tracing.

Perception & Psychophysics, 65 (7), 1136-1144

Jeurissen, D. J. J. D .M. and Roelfsema, P. R. (2012) Image Parsing, From Curve-tracing to Natural Images. Proceedings

of the 12th Annual Meeting of the Vision Sciences Society. Naples, Florida

Paradiso, M. A. and K. Nakayama (1991). Brightness perception and filling-in. Vision Res 31(7-8): 1221-36.

Roelfsema, P.R. (2006) Cortical algorithms for perceptual grouping. Annu. Rev. Neurosci. 29, 203–227

Soto D, Blanco MJ. (2004) Spatial attention and object-based attention: a comparison within a single task. Vision Res. 44(1), 69-81.