Effect of a spatial probability map on irrelevant visual search in natural scenes
Paula Hoijer
Psychobiology, University of Amsterdam Bachelor project: Psychobiologie
L.K.A. Sörensen MSc dr. H. S. Scholte
Description
While searching for your laptop, plate or keys, there are positional regularities that support effective visual search (Võ et al., 2019 and Kaiser et al., 2019). This prior statistical spatial knowledge is developed by experience when searching for a certain objects in daily life. In the present study, our intention is to investigate the relationship between this statistical knowledge and visual search in natural scenes.
For visual search, the individual has to do both; locate an object and identify it. Peelen and Kastner (2014) suggest content specific What and location specific Where templates to guide visual search. The What template compares the characteristics of the target and distractor objects by integrating them, and the Where template represents the expected location of the target. Both templates can be influenced by scene context and prior knowledge of the type of scene. As most visual search studies are done in artificial scenes, the Where template, for instance, is influenced by artificial cues like arrows. These studies on visual search in artificial scenes show that targets in a likely location, assigned by an artificial cue, are easier to detect than targets in a non-likely location (Baker et al., 2004, Geng and Behrmann 2005, Peelen and Kastner, 2014). Thus, in artificial scenes we see how the What and Where template influence visual search performance.
Comparing visual search in artificial and natural scenes reveals a number of important differences. Despite there are no spatial cues like arrows or boxes present in daily scenes, an improved performance is seen when objects tend to be in a common location (Võ et al., 2019 and Kaiser et al., 2019, Stein and Peelen, 2017). Peelen and Kastner (2014) describe the four major differences between natural and artificial scenes; familiarity with the objects (e.g. we see bicycles almost on a daily basis), familiarity with the distractors (e.g. the person riding a bicycle), context information (e.g. visual properties and likely target locations) and familiarity of the scene (e.g. we are exposed to street scenes on a daily basis). Therewith, Eckstein et al. (2006) and Võ et al., (2019) claim that highly visible objects tend to serve as cues in object search tasks with natural scenes. Stein and Peelen (2017) predicted that when having previous knowledge about the target category this would increase the target detection sensitivity. So, when looking into visual search the most important difference between natural and artificial scenes is familiarity.
In addition to these scene differences, when comparing visual search tasks and the current study task, different kind of cues are identified: In artificial scenes typically stripes or dots indicate a probable location that improves visual search (Palmer, Ames and Lindsey, 1993; Palmer, Verghese and Pavel, 2000). In this study proposal, we intend to use a “category cue” which leads to observer-directed processing, also known as top-down attention, or goal-driven selection. Previous research has shown that there is a higher target sensitivity when a category word cue is present than when absent (Stein and Peelen, 2017). Hence, in this study we aim to use a word cue evoking top-down attention to evoke a higher target sensitivity.
Concerning visual search, attention plays a leading role. Theeuwes (2019) explains in a review how, in a priority map for attention, three aspects are integrated that account for selection priority; goal-driven, stimulus-driven and history-driven selection. The weights of these three aspects change in every new situation and combined they give a representation of the environment which influences the spatial selection priority. With regard to this study, a category cue might influence goal-driven selection and prior knowledge about natural scenes might influence history-driven selection. These types of selection influences visual search by directed attention. Moreover, Stein and Peelen (2017) distinguish two different types of attention; spatial and category-based attention, referring to the location and category of the target. In this study both, an independent an collaborative effect of these types of attention is reported; when looking for a specific object the spatial attention is guided to objects that fit the category-based attentional set, but also independently, they improve the visual search. This study demonstrate how essential the attentional onset is, prior to the scene onset, when looking at visual search in natural scenes.
When comparing the current study proposal with the previous studies, there are important differences which leads to the research question how prior statistical knowledge about natural scenes influences irrelevant visual search. The innovative component of our study is that, next to a categorial target, an unfamiliar target will be present in the images. In this study we intend to examine the influence of statistical spatial knowledge of natural scenes on visual search of irrelevant targets (Gabor patch) instead of relevant targets (category targets) by measuring the reaction time and
accuracy. All of the described studies that have used natural scenes also used objects. We tend to have constant interactions within our real world with targets like ‘bicycles’ and ‘stop-signs’. Therefore, when searching for these categories we expect these objects to be on a certain, likely, location. Otherwise, with an unfamiliar target like a Gabor patch, most participants have never been exposed to this target and do not have any predictions regarding a possible location in the presented scene. To understand the effect of natural scenes on visual search it is important to look into unfamiliar objects as well, since this will give information about the spatial attention engagement in these natural scenes. Including an unrelated target we expect to understand the integration of spatial and category-based attention better and get a better understanding of the influence of selection priority between goal-driven and history-goal-driven selection. Thus, by investigation of both, relevant and irrelevant targets, the effect of prior statistical spatial knowledge of natural scenes in visual search becomes evident.
When making a proposal for the research question we took different studies into account. First, in artificial scenes, results show how relevant cues leads to a decrease in reaction time and an increase in accuracy (Geng and Behrman, 2005). Secondly, Peelen and Kastner (2019) show how prior statistical spatial knowledge of natural scenes enhances visual search for relevant targets. When presenting a category cue before the image of a natural scene image, tending to look at likely
locations according to this category cue might lead to a decreased reaction time. In this study we want to replace the category target for an irrelevant target to investigate whether the effect of prior
knowledge of natural scenes and this category cue on irrelevant visual search will affect the outcome. Kaiser et al., (2019) observed continuous flash suppression (CFS) to measure the detectability of an object. They concluded that face parts break suppression faster when on a corresponding location. In this study the researchers did not only used relevant objects but also irrelevant objects; they observed the same effect, suggesting a higher sensitivity for high-level stimuli in typical real-world location. Therefore, based on previous publications and driven by the knowledge gap of how our searching capacity is influenced, we hypothesize that when a category cue is given, the subjects will have a quicker reaction time and an increased accuracy when the Gabor patch is on a likely location than when in a non-likely location. So, by using a category cue before presenting the natural scene will enhance statistical spatial knowledge regarding visual search, which will probably lead to a decreased reaction time and increased accuracy when an irrelevant target is located in a likely location.
To test this hypothesis we will examine two conditions; both conditions will have a category cue (e.g. ‘bicycle’) evoking statistical spatial knowledge. In one condition a Gabor patch will be on a likely location corresponding to the category cue, and in the other condition the Gabor will be on a less likely location. In the current study proposal ‘a likely location’ is distinguished by the most probable category target location. Participants’ task will be to indicate whether they saw the category target and/or Gabor patch. By using this approach, we expect to elucidate how irrelevant visual search in natural scenes works.
Experimental procedure
1.1 Study design
The following section will explain the structure of the experiment, composed by a visual search task in natural scenes. In the experiment there will two main conditions investigated; 1) trials with an irrelevant target on a likely location or 2) a non-likely location. This experiment will show the effect of a spatial probability map on visual search in a “within subject” design. A spatial probability map describes the most likely location of an object in a scene evoked by a category cue. When an object is located in a likely location we expect subjects to have a quicker reaction time than when an object would be in a non-likely location due to the probable developed probability map. In addition, a “within subject” design helps to reduce errors associated with individual or task differences which makes a design more powerful and requires less participants than a between design.
For every subject two conditions are presented; one with an irrelevant target (Gabor patch) on a likely location and one on a non-likely location. The Gabor patch is on a likely location when, according to the probability map, it is on a location where we would expect the category target to be. Both conditions will have a category word cue prior to every trial to evoke a spatial probability map. In figure 1 an overview of the main experiment and corresponding null and alternative hypotheses is
shown. For both conditions no category target will be present, but only a Gabor patch, since a second target could influence visual search (Stein and Peelen, 2017). Under the null hypothesis the location of the Gabor patch does not influence performance and under the alternative hypothesis it improves the irrelevant visual search performance. As explained in the introduction we expect the category cue to probably evoke a spatial probability map, which will guide visual search positively to the most likely location corresponding to the category, thus rejecting the null hypothesis.
Figure 1: Representing the design corresponding main hypothesis. Under the null hypothesis (H0) the location of the Gabor patch (Irrelevant target) does not influence response time. It does not matter if the target is inside or outside the spatial probability map evoked by the category cue (CUE). Under the alternative hypothesis (Ha) the spatial target location of the Gabor patch does influence performance. When the irrelevant target is on a likely location according to the spatial probability map it will improve visual search performance.
To elaborate on the categories and targets used in figure 1, an example is shown in figure 2 with the category target ‘bicycle’ and a Gabor patch in the bottom left side of the image. All pictures are from street scenes and have the category target ‘bicycle’, ‘bus’, ‘stop sign’ or ‘car’. These targets are all very well-known and the probability maps of these targets are all very different in shape and location.
Figure 2: Example of a picture used in the experiment. A street scene with the category target ‘bicycle’ and a Gabor patch on the bottom left side of the picture.
1.2 Procedure
The experimental procedure of the study design will be explained in this section. As on the date of submission of this preregistration the experimental part of this study has been negatively affected by COVID-19, only the pilot data of the category cue and target has been collected. Here we present a
preliminary analysis of the data and design of the study. Pilot data from only 6 participants has been collected, granting that this might be inadequate for a reliable data analysis. The following section will explain and support the adopted methodology in order of the experimental phase.
Every experiment starts with a short introduction and training. In the first 20 trials the image presenting time was 300 ms, so the participant could understand the task, and another 20 trials were used to get used to the presenting time of 150 ms step-by-step. After this introduction and training the main experiment starts, see figure 4 for the corresponding task flow of the experiment. In 80% of the trials there will be a category cue present (‘bicycle’, ‘car’, ‘bus’ and ‘stop sign’). In the other 20% of the trials, where there is no category cue present, there will be the non-category cue ‘READY’ (Stein and Peelen, 2017). In this condition subjects will probably only have a context probability map. These cues will be mixed and presented for 500 ms to insure that the subject developed these probability maps (Battistoni et al., 2018).
To make sure the participants are maintaining their focus on the same location, a fixation cross is presented in the middle of the screen. Even during the image presentation in the next step, the fixation cross will be present to maintain central fixation. This fixation will be measured using an eye tracker. In the next step, a picture of a street scene is displayed for 150 ms. These pictures are from the MS COCO database training set from the year 2017 (Lin et al., 2014). To make sure the selected pictures have a similar context, the selection was based on the feature ‘street’. Furthermore, the selected images only contain one category target, this target should be recognizable and the picture should not have any filters. All the pictures were formatted in the same spatial dimensions (224 x 224 pixels) but at least 80% of the original object has to be maintained. All the pictures will be black and white to avoid confounding factors of other colours. For the presentation duration Van Rullen and Thorpe (2001) concluded that differentiating targets from nontargets takes 150 ms when presented in natural scenes. In addition, analysis of the pilot data showed that, accuracy was sufficient when presented for 150 ms.
In these images Gabor patches and category targets are processed. In 80% of the trials, there will be a Gabor patch so the effect of irrelevant search with different cues and targets will become clear. When the Gabor patch is present it will be on a likely location on 50% of the time and
remainder 50% of the time on a non-likely location. For example, when the category cue is ‘bicycle’ the Gabor patch will be presented in the expected location of a bicycle. When in this example, the Gabor is presented on the least likely location of a bicycle, this means it is presented on a non-likely location according to the category cue. In the experimental analysis we will show how we determine these locations. The distinction of the spatial likelihood will allow us to check for the effect of the spatial probability map on irrelevant visual search and to answer the research question. Stein and Peelen (2017) presented a category target in two third of the trials, but since our main focus is the irrelevant visual search of Gabor patch and a possible contrary effect will evolve when two targets are present at the same time, only 10% of the total number of trials there will be a category target with no Gabor patch. This control analyses is to check whether a development of a spatial probability map is reached when primed with a category cue. In another 10% of the trials no targets will be presented at all, to control for random answering. The subjects task, is to indicate if the Gabor patch is present will be checked after every trial, since this is our main point of interest. The presence of the category target will only be checked in 20% of the trials, regarding time saving. Furthermore, weighing the fact that we want to have as many trials as possible against the subject’s attention span, the number of trials was settled on 1000 in total. See figure 3 for an overview of the trials.
Figure 3: Overview of the amount of trials and each condition. In bold the conditions of the main experiment are accentuated.
After every 50 trials, there is a short break where participants can see their overall reaction time, accuracy (correct rejected and correct accepted) and experimental progress. We believe this will motivate them to keep focussing and feel the drive to do their best in the following block. The whole experiment takes about an hour. From the pilot experiments we received the feedback that the total time was too long time to stay concentrated. This is why we decided to install a long break in the middle of the experiment where participants can get up and take some rest. For the pilot participants this break usually took about 5 minutes.
To correct for bias in the occurrence of all the different conditions described above (presence category cue, category target or Gabor patch and different categories), they will be randomized to avoid unexpected variables. Randomization is important for the reliability of the experiment and to eliminate any possible biases arising from experimental design. When using block randomization, each participant will randomly be assigned to trials with a Gabor patch on a likely location or on a non-likely location. Meanwhile, the amount of trials within each block will stay the same and the trials will be randomly randomized between all the blocks.
Figure 4: Task flow experiment. Starting with a (category) cue (500 ms) followed by a fixation cross (500 ms) and a street scene (150 ms). Subject have to indicate of the target was present and every 50 trials there is a feedback moment.
1.3 Data collection procedures
Participants will be recruited through the website lab.uva.nl. This is a site were (mainly psychology) students from the University of Amsterdam can participate in (psychology) researches like ours. The predicted average age will be 18-20 years old since the psychology students have to participate in these researches in their first year of their bachelors. They will receive 1,5 Student Credit or 15 Euro’s. When looking for participants they have to meet certain criteria; first of all, subjects must be between the age of 16 and 39, since attention can decrease in older people (Chikhaoui et al., 2019). Secondly, the subjects need to have normal or corrected vision (lenses/glasses), it is important being able to see images on the computer screen sharply and to be able to read the category cues in order to detect the targets. Thirdly, the subjects should not have any attention-deficit/hyperactivity disorder, like ADHD or Autism. This could influence the attention span, or other unforeseen effects (Arguin et al., 1993, O'riordan et al., 2001). In addition, all subject have to sign an informed consent prior to the experiment.
During the experiment the subject’s eye movement will be followed by an eye tracker while the subjects head is kept in place using a chin rest. This technique keeps track of eye movements and checks if the participant keeps fixated. Prior to the experiment a calibration and validation has to be done for eye tracker measurements. If a fixation is not detected for more than 500 ms, the following trial is deleted and a recalibration is triggered. The refresh rate of the minor LCD screen is 60 Hz and the screen for the subjects has a refresh rate of 120 Hz. The subject’s eyes must always be at 76 cm from the fixation cross on the screen, which is 28.5 cm (height) by 51 cm (width). The screen resolutions is 1024 pixels (height) by 1280 (width). The Gabor patch should be subtended 1.0° of the visual angle. Besides eye tracker data we also use behaviour data. Using keypress it is possible to measure the reaction time by looking at the time between the disappearance of the scene and the response (keypress). The response is determined by pressing the key ‘m’ (present) and ‘z’ (not
present) using a Qwerty keyboard. Accuracy is measured by looking for the correct rejections rate and hit rate.
Analysis plan
2.1 Sample size
In order to reach statistical differences in the collected data, we performed a statistical power analysis to quantify our hypothesis. We expect the reaction time will stay the equal or less when a category cue is present, subsequently a one tailed statistical test was applied. In the paragraph statistical analysis we explain the use of the Wilcoxon signed-rank test for the examination of our main hypothesis. An effect size of 0.5 is usually considered as an median effect size, which is an
requirement for this kind of study (Stein and Peelen 2017, Eckstein 2006). With a power of 0.95 and an alpha of 0.05, a total sample size of 47 subjects is needed based on the G*Power model (Faul et al., 2007). With this sample size we will be able to test our hypothesis.
2.2 Experimental analysis
To answer the main question how statistical spatial knowledge about natural scenes affects irrelevant visual search, we will divide the category images in likely locations or non-likely locations. Using the pilot data we came to the conclusion that the reaction time is not normally distributed (W = 0.736, p-value = 0.000), thus we need to use a non-parametric statistical test. When a cue is present we might develop a spatial probability map, as explained in the introduction. We estimated these probability maps by leveraging the object segmentation annotation to determine how often every pixel was part of a target object in the selected standardized images. These probability maps of all 4 category targets were divided into a 7x7 grid to determine the 10 most- and least-likely (see figure 5). We selected 10 different locations for both, the likely location and non-likely location since we want sufficient repetitions per location but also a deviation in locations, since it could become too predictable if e.g. only 5 locations were constantly selected. Since we have 4 categories and 2 locations per category (10 likely, 10 non-likely), there will be 80 Gabor patch locations. In total, there will be 600 trials for this experimental analysis so each location will be presented 7.5 times. In the middle of the screen the
A
B
C
D
fixation cross will be presented so this location is excluded from the possible Gabor patch locations. A Gabor patch (50 pixels) fits in each of the box locations (72 pixels). When a Gabor patch is on a likely location it is presented in one of the 10 most likely locations according to the category and, when on a non-likely location is presented, in one of the 10 least likely locations according to each category. Therefore regarding our main hypothesis, if prior statistical knowledge will influence irrelevant visual search, we will apply a non-parametric paired Wilcoxon rank test with two main groups (likely and non-likely location) and reaction time as dependent variable.
Figure 5: Spatial probability grid of the category targets. Using the spatial probability map of every target, a 7x7 grid is made to determine likely vs. non-likely locations. With the ‘-‘ sign the 10 least likely locations are indicated and with the ‘*’sign the 10 most likely locations are indicated.
2.3 Control analysis
During the development of the experiment, we introduced a couple of variations that made the trials more varied and allowed us to create a greater number of trials. To test whether these variations introduced a different response, we will look at the response time for detecting the Gabor patch using the pilot data. First of all, we want to see if the different category images differ in reaction time. If, for example, the detection of a bus is faster than a stop sign because of its size, we have to take this into account for our research question. In general, we expect the same reaction time for all different categories in the street scene; bicycle, car, bus, stop sign. Under the null hypothesis there is no significant effect in reaction time for the categories; under the alternative hypothesis of a significant effect in reaction time for one or more different categories. This analysis is a within-subject analysis since every subject has looked at all four categories. This hypothesis will be tested with a RM ANOVA. As the reaction time in the pilot data was not normally distributed, we recommend the use of a non-parametric alternative such as Friedman test. The independent variables are the four different categories and the dependent variable is reaction time. When testing for the different categories, no
significant difference was found (F = 6.200, p-value = 0.102) so, for the context probability map our primary conclusion is that no significant difference in irrelevant visual search is found between four different categories (see figure 6).
Figure 6: Control analysis for the different categories bicycle (s = 43.050, Md= 440.089), car (s =55.363, Md = 430.768), bus (s = 33.538, Md = 444.206) and stop sign (s = 56.417, Md = 430.912). There is no significant difference found between the categories (F = 6.200, p-value = 0.102).
Some values of the Gabor patch will vary to prevent additional influences on the reaction time. Subsequently, the Gabor patch will be orientated in different orientations [-90, 90 degrees] and will have different spatial frequencies [0.1, 0.25]. To be sure this variation won’t influence the results, we checked the effect variation on the pilot data and results show no significant effect for different Gabor’s orientations (ρ = -0.009, p-value=0.444) nor spatial frequency (ρ = 0.0153, p-value=0.2101), see figure 7.
Figure 7: correlation of Spatial Frequency (A) and Orientation (B) of a Gabor patch on reaction time in all subjects. Frequency is from [0.1, 0.25] and orientation from [-90, 90 degrees]. Using the Spearman correlation test no correlation is found for both variables (orientation ρ = -0.009, p-value = 0.444); SF ρ = 0.0153, p-value = 0.2101)
In addition, to control for the effect of the presence of the target, we investigated whether there is any effect on the reaction time when the Gabor patch is present or not by looking at the difference in response time. Will it take longer if you are searching for an object but cannot find it? Previous research confirms this expectation (Neider and Zelinsky, 2006) and so does our pilot data (W = 0.000, p-value = 0.028), see figure 8A. Since in the main experiment the Gabor patch is always present, we do not expect the difference in reaction time will influence the results. To confirm the assumption that the difference in response time for the presence of the Gabor patch will not influence the performance, we calculated the influence of the reaction time for correct rejected or correct accepted trials. The results confirm this (W = 1.000, p-value = 0.046), but since in both conditions
A
B
C
there will be correct accepted- and rejected trials, we expect this not to have any influence on the final results (see figure 8B).
In the hypothesis we make the assumption that the reaction time is greater when behaviour response is correct than when incorrect. This relation is confirmed in the pilot data (W = 0.000, p-value = 0.028), so when drawing conclusions from the reaction time, we can also make expectations about the accuracy, under reservation (see figure 8C)
Figure 8: (A) control analysis for the presence of the Gabor patch on the reaction time (W = 0.000, p-value = 0.028). (B) control analysis for the correct accepted and correct rejected trials (W = 1.000, p-value = 0.046) and (C) the analysis between accuracy and reaction time (W = 0.000, p-value = 0.028).
Furthermore, controlling for false positives will be carried out in 100 trials where a category cue with no Gabor patch nor category target will be present. This will ensure that the subjects will not respond randomly to the question if the Gabor patch or target is present. In another 100 trials, there will be a category cue with only a category target but no Gabor patch. In this way, we can control how a category cue evokes a spatial probability map.
Using the pilot data we will understand how the method, of a likely location, or a non-likely location works. As explained in the introduction, we might develop a so called context probability map (see 9B) when no cue is present. If an increased performance is observed when the irrelevant target is in a likely location compared to when it is placed in the periphery of the image and no category cue is present, this would mean that for irrelevant visual search no spatial probability map is necessary. See figure 9A for an overview of the experiment. The occipitotemporal cortex show how parts of the body on a likely location evoke a more distinct response pattern than when not in a likely location caused by our context probability map (Chan et al., 2010; de Haas et al., 2016; cited in Kaiser et al., 2019). Like street scenes, the spatial probability of body parts is influenced by our daily life experiences. Neider and Zelinsky (2006) on the other hand, studied the search direction of known and unknown objects. They concluded that observers’ gaze pattern did not differ between scene region. This finding leads to our hypothesis that the context probability map does not influence visual search of unknown objects. To test this hypothesis we looked at the pilot data and the relation between the reaction time and the degree of spatial likelihood by category. Under the null hypothesis there is no difference in reaction time between the Gabor patch being on a likely location or a non-likely location and under the alternative hypothesis there is a significant difference. Using a Wilcoxon rank test the null hypothesis is confirmed (W = 723.000, p-value = 0.158), see figure 9C for an overview of the results. Hereby we conclude that when developing a context probability map, this will not influence the visual search. Besides, we can conclude that the proposed method works and it is possible to distinguish a likely from a non-likely location.
B
C
Figure 9: statistical control analysis for the context probability map and the overall experiment. There is no category cue and only an irrelevant target (Gabor patch) on a likely or non-likely location (A). Using the context probability map (B) we could calculate the difference in reaction time between a Gabor patch on a likely location or non-likely location (W = 723.000, p-value = 0.158) (C).
2.4 Data exclusion
One approach to determine outliers is by looking at the pilot data. When looking at the reaction time as a response value, in some trials the measured values are out of the predetermined maximal range. An outlier is defined as being any point of data that lies over 1.5 interquartile ranges below the first quartile (Q1) or above the third quartile (Q3). When looking at figure 10 we see a boxplot with the reaction time of the pilot data where it is visible that a reaction time of ≤ 66.632 ms or ≥ 804.875 ms, will be considered as an outlier. In the pilot data this came down to an average of 28 outliers per subject. Furthermore, when using the eye-tracker, fixation between trials cannot be detected for more than 500 ms, consequently the following trial will be deleted and a recalibration is triggered.
Figure 10: Boxplot the reaction time of all trials from pilot data. The interquartile ranges are 343 ms (Q1), 427 ms (Q2) and 528 ms (Q3). Outliers lie 1.5 interquartile ranges below the first quartile (Q1) or above the third quartile (Q3); values below 66.632 ms or above 804.875 ms are outliers.
Discussion
Regarding the research question, if prior statistical knowledge about natural scenes can affect irrelevant visual search, there are no possible conclusions to be drawn yet. Unlike previous research, the intention is to present a Gabor patch in natural scenes instead of a categorical target. Expected is that a prior target cue could improve performance when the irrelevant target is on a likely location compared to an unlikely location (Võ et al., 2019 and Kaiser et al., 2019, Stein and Peelen, 2017). With the present pilot data, this hypothesis was not testable but there are two probable outcomes: 1) In the first one, there is no difference in visual search performance between a Gabor patch being on a likely location compared to a non-likely location suggesting that prior statistical knowledge about natural scenes would not influence irrelevant visual search. When there is no difference in reaction time this would not mean that there is no spatial probability map when a category cue is present, but it would implement that it does not influence irrelevant visual search. 2) In the second probable
situation, there is a significant difference in performance between the Gabor patch being in a likely location or not. It does not reassure a spatial probability map but it does give big empirical support. If no effect is found we will check in the control analysis a possible spatial probability map evokes and if it helps for relevant visual search.
In the introduction we describe two different types of attention by Stein and Peelen (2017); spatial and category-based attention, referring to the location and category of the target. Results demonstrate how essential the attentional onset is, prior to the scene onset, when looking at visual search in natural scenes. If the second probable situation would be true (2), this would support the concept of the importance of the attentional onset for spatial and category-based attention.
Additionally, as discussed in the introduction, three aspects integrate regarding selection priority; goal-driven, stimulus-driven, and history-driven selection (Theeuwes, 2019). These are divided into a priority map for attention and change in priority level in every new situation and combined they give a representation of the environment which influences the spatial selection priority. We expect a
category cue to influence the goal-driven selection and prior knowledge about natural scenes might influence history-driven selection. If the second probable scenario (2) and our expectations are true, this could signify the combined goal- and stimulus-driven selection having a higher priority than stimulus-driven selection. Alternatively, when no difference is found between a Gabor patch
presented on a likely location compared to an unlikely location, this could implicate a priority of stimulus-driven selection above history- or goal-driven selection.
As mentioned, the research question could not be tested yet but considering the pilot data, we could analyse the functioning of the methodology of using a likely and non-likely location. The distinction between the likely and non-likely locations was clear but, as expected, no significant difference is found. The pilot results suggest that a probable context probability map based on prior statistical knowledge does not influence visual search performance. Although we do expect to find a difference when a category cue is present (Stein and Peelen, 2017), there are some possible
complications for this methodology: The different categories have very different probability maps. For example, the category bus has all most likely locations, centered in the middle of the image close around the fixation cross, while the most likely locations for a car are located on the left and right sight of the image. When analysing the pilot data there was no significant difference found between the different categories but what caught the attention was the big variation in reaction time within those categories. This could lead to the expectation that some probability maps are way more specific than others.
The current study design has several limitations. For instance, some research indicates an influence of the number of objects in the scene on visual search. When a scene is more complicated reaction time increases (Geng and Behrmann, 2005; Peelen and Kastner 2014). In Lowe et al., (2016), results show the effect of the texture and evenness of the background. In addition, the feedback reported by the pilot participants was that it was easier to detect the Gabor patch when it was on a very light or dark background. In a follow-up study, we want to take these natural scene features into account. This might be possible by differentiating two groups; placing a Gabor patch on a simple or more complex surface without any object information. To prevent any confounds about the number of distractor objects, this will be set at 30 objects for both conditions (Peelen and Kastern, 2014).
In addition, Kaiser et al., (2019) observed continuous flash suppression (CFS) and concluded that face parts break suppression faster when on a corresponding location, suggesting a higher sensitivity for high-level stimuli in a typical real-world location. If there would be a difference in performance for the location of the Gabor patch, looking at CFS would be an interesting follow-up study to broaden the understanding of irrelevant visual search in natural scenes.
So, when results are found for the research question, if prior statistical knowledge influences irrelevant search in natural scenes, we will get a better insight into the functioning and processing of irrelevant visual search in natural scenes. In the current preregistration, we proposed a research design supported with pilot data. With this proposed design we will get more insight into the attentional selection and search behaviour
Acknowledgement
I would like to thank L.K.A Sörensen for their assistance during the internship and thoughtful feedback for this preregistration. Thank you S.H. Scholte for your support and giving us an insight in your lab.
References:
Arguin, M., Joanette, Y., & Cavanagh, P. (1993). Visual search for feature and conjunction targets with an attention deficit. Journal of cognitive neuroscience, 5(4), 436-452.
Baker, C. I., Olson, C. R., & Behrmann, M. (2004). Role of attention and perceptual grouping in visual statistical learning. Psychological Science, 15(7), 460-466.
Battistoni, E., Kaiser, D., Hickey, C., & Peelen, M. V. (2018). The time course of spatial attention during naturalistic visual search. Cortex.
Chikhaoui, B., Ruer, P., & Vallières, É. F. (2019, October). A Deep Learning Method for Automatic Visual Attention Detection in Older Drivers. In International Conference on Smart Homes and Health Telematics (pp. 49-60). Springer, Cham.
Chikhaoui, B., Ruer, P., & Vallières, É. F. (2019, October). A Deep Learning Method for Automatic Visual Attention Detection in Older Drivers. In International Conference on Smart Homes and Health Telematics (pp. 49-60). Springer, Cham.
Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual review of neuroscience, 18(1), 193-222.
Eckstein, M. P., Drescher, B. A., & Shimozaki, S. S. (2006). Attentional cues in real scenes, saccadic targeting, and Bayesian priors. Psychological science, 17(11), 973-980.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.
Folk, C. L., & Hoyer, W. J. (1992). Aging and shifts of visual spatial attention. Psychology and aging, 7(3), 453.
Geng, J. J., & Behrmann, M. (2005). Spatial probability as an attentional cue in visual search. Perception & psychophysics, 67(7), 1252-1268.
Hartley, A. A., Kieley, J., & Mckenzie, C. R. (1992). Allocation of visual attention in younger and older adults. Perception & Psychophysics, 52(2), 175-185.
Kaiser, D., Quek, G. L., Cichy, R. M., & Peelen, M. V. (2019). Object vision in a structured world. Trends in cognitive sciences.
Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., ... & Zitnick, C. L. (2014, September). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.
Lowe, M. X., Gallivan, J. P., Ferber, S., & Cant, J. S. (2016). Feature diagnosticity and task context shape activity in human scene-selective cortex. NeuroImage, 125, 681-692.
Neider, M. B., & Zelinsky, G. J. (2006). Scene context guides eye movements during visual search. Vision research, 46(5), 614-621.
O'riordan, M. A., Plaisted, K. C., Driver, J., & Baron-Cohen, S. (2001). Superior visual search in autism. Journal of Experimental Psychology: Human Perception and Performance, 27(3), 719. Palmer, J., Ames, C. T., & Lindsey, D. T. (1993). Measuring the effect of attention on simple visual search. Journal of Experimental Psychology: Human Perception and Performance, 19(1), 108.
Palmer, J., Verghese, P., & Pavel, M. (2000). The psychophysics of visual search. Vision research, 40(10-12), 1227-1268.
Peelen, M. V., & Kastner, S. (2014). Attention in the real world: toward understanding its neural basis. Trends in cognitive sciences, 18(5), 242-250.
Stein, T., & Peelen, M. V. (2017). Object detection in natural scenes: Independent effects of spatial and category-based attention. Attention, Perception, & psychophysics, 79(3), 738-752.
Theeuwes, J. (2019). Goal-Driven, Stimulus-Driven and History-Driven selection. Current opinion in psychology.
Vanrullen, R., & Thorpe, S. J. (2001). The time course of visual processing: from early perception to decision-making. Journal of cognitive neuroscience, 13(4), 454-461.
Võ, M. L. H., Boettcher, S. E., & Draschkow, D. (2019). Reading scenes: How scene grammar guides attention and aids perception in real-world environments. Current opinion in psychology
