Investigation of motor and visual space in the brain: optimizing mislocalization effects in a smooth pursuit paradigm

(1)

Investigation of motor and visual space in the

brain

Optimizing mislocalization effects in a smooth pursuit paradigm

Author: Anupama Nair

ABSTRACT

The fallibilities in our perceptual abilities are often deemed bizarre, as they appear to be counter-intuitive yet persistent. New discoveries in this area further highlight the limitations in our perceptual system, such as perception of locations in the world that are heavily influenced by eye movements. One such example is the robust phenomenon of mislocalization of a briefly flashed stimulus during smooth eye movements which is of primary interest in this study. In this experiment, we optimize the conditions under which mislocalization of a static stimulus occurs during smooth tracking of a moving object and test for the effects of eye movement direction. Subjects are asked to report the perceived displacement of a briefly flashed stimulus relative to another similar stimulus while maintaining steady pursuit of a sinusoidally moving target dot. Using a staircase procedure, the point of subjective equal location of the stimulus is determined and updated on every trial based on the participant’s previous response to that stimulus. This experimental paradigm is different from other common mislocalization paradigms which require participants to point to the perceived location of a target, for example, with a mouse pointer. Our approach enables us to capture the effects of mislocalization on a minute level while testing for interaction effects with the direction of pursuit. The study finds the difference between the mislocalization effects for the two kinds of eye movement direction to be significant indicative of the role of eye movement direction in localization.

Supervisor: Dr. Tomas Knapen

Co-assessor: Dr. Ilja Sligte

Vrije University, Amsterdam

(2)

2

Acknowledgements

My masters internship was an invaluable learning experience for which I

would like to thank Dr. Tomas Knapen immensely for his guidance, supervision,

patience and encouragement, as well as for pushing me to excel at all times. I

would also like to thank Daan Van Es, Ronald Dekker and Alban Voppel for their

helpful suggestions, critical feedback and constant support that my project and I

have greatly benefitted from. I am also grateful to Bronagh McCoy, Nicola

Anderson and Jarik den Hartog for investing time and effort into refining the

project and for allowing me to learn from their expertise. Last but not the least, I

would like to express gratitude to all my classmates, peers and friends who not

only participated in my study but also assisted me with my queries on numerous

occasions, without hesitation.

(3)

3

I. Introduction

Perception is a constantly constructed subjective representation of reality. Visual spatial perception is especially distorted during eye movements (Königs & Bremmer, 2010). For example, a distortion or compression of perceptual space is observed just before the onset of a saccade. Further evidence of such an active construction process is derived from a smooth pursuit paradigm. Pola (2002) defines smooth pursuit movement as “a slow to medium velocity movement that allows us to visually follow a moving target, and thus maintain our gaze on or near the target.” This ability of accurately tracking an object’s trajectory in physical space is most developed in primates (Lisberger, Morris & Tychsen, 1987).

In a localization paradigm involving smooth pursuit movements, subjects have to track a moving target with their eyes and are required to localize the final position of the target in their visual space. Research has consistently demonstrated subjects’ tendency to mislocalize the final position of the target in the direction of smooth pursuit, also known as forward shift (Hubbard, 1995). Similarly, stimuli flashed during smooth pursuit of a target are also mislocalized in the direction of smooth pursuit (Mita, Hironaka, & Koike, 1950 as cited in Brenner, Smeets & van den Berg, 2001) and away from the fovea also known as spatial expansion (Mitrani & Dimitrov, 1981), such that there is an enhanced effect of the forward shift for stimuli presented ahead of the smooth pursuit target (cumulative effect of forward shift and spatial expansion), emphasizing the fallibility of perception.

How robust is this mislocalization effect? A study as early as in 1924 established the distortion effects in perception caused by the tracking of a smoothly moving target. Hazelhoff and Wiersma (1924) proposed that mislocalization is the result of the temporal properties of the human visual system. In their studies, participants had to track a moving stimulus along a linear path, during certain periods of which brief flashes of stimuli were presented, to be localized by the participants. As expected, participants tended to mislocalize the stimulus in the direction of the pursuit object. According to the authors, the ratio between mislocalization and velocity of tracking is contingent on the processing time or the ‘perception time’ of a visual stimulus. The

(5)

5

processing time to register the brief stimulus leads to mislocalization (Hazelhoff and Wiersma, 1925 as cited in Mateeff, Yakimoff & Dimitrov, 1981 ).

In a similar study by Brenner, Smeets and van den Berg (2001), subjects were asked to align the position of a dot to another previously flashed dot (called the reference dot in this study) while pursuing a target ring moving sinusoidally back and forth across the center of the screen. The reference dot in this study flashed as the ring moved to the left, in predetermined positions above the ring on every trial, and always flashed at a moment when the ring passed exactly below it. The second dot flashed as the eyes moved to the right, and had to be aligned to the reference by manually repositioning the computer mouse in order for the two dots to be aligned on a vertical plane. The velocity of the ring was manipulated which led to the reference dot position being the determining factor of predicted error. A mislocalization error of 4 cm was found when subjects pursued the ring in complete darkness which corresponded to a timing error of 100 ms. Such a mislocalization effect was found to be persistent even with predictable stimuli, once again highlighting the fallibility of perception.

Such results were obtained with the use of static stimuli in a smooth pursuit task. What happens when moving stimuli are used in localization experiments? As for mobile targets, it is not just the actual momentum of the target that influences localization estimates but also the momentum of the representation of the target (called displacement representational momentum) that influences localization amongst other factors (Hubbard, 1995). In his study, Hubbard (1995) asked participants to pursue a linearly moving object on a screen till it disappears and then position their mouse to pinpoint the vanishing point (VP) of the stimulus. He found that with a horizontally moving target, the VP was localized further downward and more towards the direction of perception than it’s true VP. As for a vertically moving target, the observed forward shift declined with upward motion but persisted with downward motion, indicative of the implicit influence of gravity and momentum in our localization decisions. Moreover, the author emphasized the role of expectations about the target’s trajectory in the process of memory displacement. To support this statement, he found evidence using a visual barrier in a smooth pursuit paradigm wherein the target reversed its motion direction on encountering the barrier. With repeated presentations of such trials, the participants exhibited memory displacement in line with the anticipated motion of the target (reversal on reaching the barrier) which was in fact away from the current direction of motion (Hubbard, 1994; Hubbard & Bharucha, 1988;

(6)

6

Verfaillie & D’Ydewalle, 1991 as stated in Hubbard, 1995), emphasizing the role of expectations in representational momentum of the target.

This mislocalization effect is not solely a shortfall of the memory system but is evident in perception itself. In Stork, Neggers and Müsseler’s (2002) experiment, using a similar paradigm where participants were required to follow a moving stimulus along a circular trajectory and indicate its VP along this trajectory, participants consistently overshot the final position of the stimulus. The eye position of the participant at the time of stimulus disappearance was further along the actual VP of the stimulus. They found such an effect even when participants could intentionally control the disappearance of the stimulus along its trajectory, although it was found to be less drastic than when they had no control over its disappearance. These results were indicative of the fact that the oculomotor system does not function merely on the basis of visual stimulus characteristics but is also influenced by expectations and predictions of the observer regarding future stimulus positions (Stork, Neggers and Müsseler, 2002).

Watanabe (2005) conducted a similar study where a flash was presented either before, at, or ahead of the fixation point in the presence of horizontally moving bar stimuli and subjects had to localize the position of the flash. They found that when participants had to estimate the flash position with respect to fixation, their estimations were influenced by the moving bars in the direction of motion. This tendency which they refer to as position-capture effect was more pronounced when the flash was presented spatially ahead of the moving bar. In accordance with this finding, Mitrani and Dimitrov (1981) also found mislocalization to be greater when a stimulus appears in the region which has not yet been crossed by the fovea as compared to the region which has been passed by the fovea. They attribute this effect to greater perception times of stimuli in certain locations on the retina, as compared to the fovea. Similarly, Van Beers, Wolpert and Haggard (2001) found noteworthy localization errors in the hemifield ahead of the pursuit target, consistent with the direction of smooth pursuit. Moreover, Watanabe (2005) also found a reversal in position-capture direction (opposite to bar-stimulus motion) when the flash was presented far behind the moving bar in the incoming-field condition (which refers to the incoming motion of the bar stimuli on screen). They attributed this inconsistency in position-capture, which they refer to as asymmetric mislocalization, to the flash’s relation to the moving object (bar) and not with respect to the fixation. In short, in the incoming-field condition, the strength of the position capture effect was found to be an inverse function of the distance

(7)

7

between the flash and the moving object and a reversed position capture was observed when the flash was presented far behind the moving object.

In another study by Souman, Hooge and Wertheim (2006), subjects were to perform a localization and motion perception task wherein they were presented with a vertically moving stimulus during smooth pursuit of a horizontally moving target and they had to estimate the direction of the vertically moving stimulus in the motion perception task or the start or end points of the stimulus in the localization task. The experimenters further manipulated the duration of stimulus presentation (300, 700 or 1100 ms), the motion direction of the stimulus (up or down) and the direction of smooth pursuit (left or right). Results showed that the starting positions were more strongly mislocalized in the direction of the pursuit as compared to the ending positions which were also mislocalized, but to a lesser degree. This error was larger for longer stimulus presentation times. Moreover, both starting and ending positions were mislocalized in the vertical motion direction at some presentation durations. As for the motion direction task results, the participants’ judgments reflected a deviation of about 30° from the physical/ vertical direction towards the retinal image direction.

A difference was noted in the stimulus direction indicated from the localization data and the indicated stimulus motion data implying the use of different compensatory mechanisms for motion perception and localization during smooth pursuit. The authors further speculate that this conclusion can only be deemed valid if it is assumed that the perceived motion direction is the same as direction predicted from starting and end points during fixation.

To test if this assumption was true and if perceptual inaccuracies were indeed due to the smooth pursuit eye movements as against the inconsistencies between localization and motion perception, the researchers devised a clever paradigm where the same experimental setup was used with the exception of a stationary fixation target in place of a moving target to be pursued. They found that, with such a setup, the localization errors with respect to the start and end positions were found to be much smaller as compared to the errors during smooth pursuit. Similarly, there was lesser deviation of the average indicated direction from the actual physical direction of the stimulus. This study exemplifies the strong distortion effects of the smooth pursuit eye movements, suggesting that the compensatory mechanisms to enable such eye movements leads to a difference in the motion indicated in the location and motion perception performance.

(8)

8

For static stimuli presented in the periphery of the retina, the stimulus is localized closer to the foveal center as compared to its actual position, also referred to as the foveal bias(Van der Heijden, Van der Geest, De Leeuw, Krikke, & Müsseler, 1999; Kerzel, 2000; Kerzel, Jordan & Müsseler, 2001). Similarly, Mitrani and Dimitrov (1981) found that stimuli flashed several visual degrees from the fovea are perceived closer towards the fovea centralis. Contrary to expectations, they also found flashes presented at the end of their scale (or at the periphery in the direction of motion) to be mislocalized to a higher degree as compared to flashes presented at the beginning of the scale.

A. Factors affecting smooth pursuit:

i. Lighting conditions, perceived angular size

Kerzel, Aivar, Ziegler and Brenner (2006) also investigated the effects of external lighting and ambience on localization of stimuli during performance of a smooth pursuit task to determine if detection of visual referents in the light can impede localization. The results of their experiments showed that the lighting conditions had little influence on localization performance- flashes were consistently mislocalized in the direction of motion highlighting the robustness of the smooth pursuit phenomenon.

ii. Predictability

Predictability of a flash stimulus, also called temporal cuing, has little success in reducing mislocalization in smooth pursuit. (Rotman et al., 2002 as cited in Watanabe, 2005). As stated previously, while describing Stork, Neggers and Müsseler’s (2002) experiment, participants’ gaze continued to overshoot the final position of the stimulus, albeit to a lesser degree, even when they had control over the target’s disappearance.

B.

Errors in Smooth Pursuit

:

The errors observed during smooth pursuit may simply arise because retinal signals which reflect information about eye orientation at a moment of time may in fact differ from the moment of retinal stimulation (Rotman, Brenner, Smeets 2004). This lack of synchrony may be explained by neuronal delays between retinal and extra-retinal information (which is constantly

(9)

9

updated based on ongoing eye activity) that are not compensated for (Brenner et al., 2001). This neuronal delay may manifest as the distance of mislocalization, “...which corresponds with the time taken for information from retinal stimulation to reach the brain and for oculomotor commands to reach the eye muscles” (Brenner et al., 2001 as cited in Kerzel, Aivar, Ziegler, Brenner, 2005).

As for moving stimuli, it is speculated that the stimulus is mislocalized in the direction of smooth pursuit as a result of representational momentum (Kerzel, Jordan & Musseler, 2001). Alternatively, according to the perceptual account, the eyes overshoot the final position of the target and a foveal bias further contributes to the mislocalization of the stimulus in the direction of motion.

As is well established from the vast number of studies replicating the mislocalization effect in a smooth pursuit paradigm, perception is susceptible to a number of flaws, especially where eye movements are concerned. In our study, we were therefore interested in replicating such mislocalization effects using a similar paradigm but under optimal conditions to maximize this effect. These optimal conditions involve the recruitment of the most suitable variables to achieve a pronounced mislocalization effect.

(10)

10

II. Method

i. Participants

16 participants (9 males, 7 females) who were students and faculty members at the University of Amsterdam and Vrije University Amsterdam respectively, participated in the experiment. All participants had normal or corrected-to-normal vision. The data from one participant was excluded from analysis due to poor task performance.

ii. Task

The present experiment draws its basis from Brenner et al.’s (2001) experiment where participants had to align a flashing dot to a previously presented reference dot while keeping smooth pursuit eye movements of a sinusoidally moving ring intact.

In the present experiment, participants were to perform a similar localization task in the context of a smooth pursuit paradigm. The experiment was created on OpenSesame 2.9.7, and was displayed on a flat-screen LCD monitor (Height - 29.5 cms, width - 47.5 cms). The monitor was connected to a 1000 Hz Eyelink eyetracker which tracked the left eye. They were seated 70 cms from the experimental monitor and their head was rested comfortably on a chin-rest. The room in which the experiment was conducted was dimly lit. The paradigm consisted of a fixation dot (default opensesame filled white circle with an 8px radius and 2px black hole) at the center of the experimental window (512px). On pressing any key, the fixation dot traversed the screen smoothly either along a left or right trajectory on a horizontal plane with an amplitude of 409.6 pixels relative to the screen width (1024 pixels). This sinusoidal motion was created by changing the fixation dot’s position marginally with every refresh cycle of the monitor (which was set at 120 Hz and each refresh rate lasted approximately 8 ms) leading to the appearance of smooth motion. Upon reaching the left or right end coordinates of the experimental window (the end-points of the window width), the fixation dot reversed its direction and continued its sinusoidal motion along the horizontal plane. This motion continued for six minutes which comprised one experimental run. The velocity of the smooth pursuit fixation target (SP dot) was set at 0.5 Hz which matched one of the frequency manipulation settings in the original experiment by Brenner and colleagues (2001).

(11)

11

The experiment consisted of a single such run, which contained approximately 95 trials. Each trial consisted of the presentation of a reference stimulus flash, a test stimulus flash and an intertrial interval. Each pass of the smooth pursuit dot from extreme left to extreme right (or vice-versa) lasted 1000 ms. Therefore, each trial lasted approximately 4000 ms. The first flash (called the reference stimulus) was a red bar with a width of 10 pixels and a height of 100 pixels and always occurred 20 pixels above the SP dot, when the SP dot crossed the center, on random trials.The visual degree per pixel of the screen was 0.042 degrees and the size of the visual stimulus in degrees was 4.22 degrees. Invariably, the reference stimulus therefore also always flashed 20 pixels above the center of the screen, only when the SP dot passed below it. Thus, for the reference stimulus position, the retinal location of the stimulus and the spatial position of the eye were held constant across all trials. The second stimulus, the test stimulus, was a red bar with the exact dimensions and flashed 20 pixels below the SP dot. However, the test stimulus flashed only under certain conditions. First, it always flashed in the smooth pursuit pass following the reference stimulus presentation, i.e., if the reference stimulus was presented when the SP dot was moving from left to right, the test stimulus was always presented in the following SP dot motion from right to left and vice versa. Second, the test stimulus was always presented when the SP dot crossed the center coordinates on its way back - therefore, the spatial position of the eye was kept constant even for the test stimulus. Finally, the position of the test stimulus was determined by a staircase method. To elaborate, the starting position of the test stimulus was based on the staircase it belonged to. Four staircases were implemented in this experiment, with starting positions of -50px, -25px, 25px and 50px from the center which corresponded to 2.1 and 1.05 visual degrees. Hence, the test stimulus flashed at one of the four predetermined locations, two of which started from the left corner of the experimental screen and the other two from the right corner. Two of these staircases (staircase 0 and staircase 1) flashed on opposite ends of the screen when the pursuit eye movement was to the right and the other two staircases (staircase 2 and 3) flashed when the pursuit eye movement was to the left. Each staircase “remembered” the participant’s response to the stimulus and determined its next position for that staircase according to a predetermined step-size. On each trial, one of the four staircases was selected randomly and the position of the test stimulus was determined based on the initial starting position value of that staircase and the participant’s previous response. Thus, each staircase

(12)

12

worked independently in presenting the subsequent test stimulus. This was done so as to prevent participants from “guessing” the consequent positions of the test stimulus.

The participant was instructed to steadily pursue the SP dot, despite the presentations of the stimuli. Once the test stimulus was presented however, the participant had to compare the reference stimulus to the test stimulus by indicating whether the test stimulus was to the left or to the right of the reference stimulus by pressing either the left (for the former condition) or right (for the latter condition) arrow keys. This was a forced choice response, and the participant was compelled to decide only between the “left” and “right” responses while keeping smooth pursuit eye movements intact. Based on the participant’s response, the staircase selected in that trial “remembered” the given response and implemented a reversal in the presentation of the subsequent test stimulus. The step size for the first reversal was set at 10px and for every second reversal, the step-size value was multiplied by 0.8 to make it harder to gauge its position with respect to the reference stimulus. The staircase was constantly updated for every trial accordingly. (see Figure 1)

Figure 1. Task set-up.

(a) Fixation, followed by sinusoidal pass of SP dot

(b)Trial start

Ref - stim presentation (1000 ms)

(c) Test - stim presentation + response (1000 ms) (d) ITI (1000-2000ms) (e) staircase update

(13)

13

Pictorial representation of the experiment (a) The experiment begins with a fixation dot which stays on screen

until a key is pressed. (b) The dot then starts moving back and forth until a reference stimulus is presented right over the SP dot when it crosses the center. This pass marks the beginning of a trial and the direction of pursuit is chosen at random. (c) In the following pass (1000 ms later), the test stimulus is shown when the SP dot crosses the center which could be at one of the four pre-determined staircase locations. The participant indicates his response at this stage. (d) ITI = intertrial interval, which could last either for 1000 ms or 2000 ms marking the end of the trial. (e) The process repeats - the reference stimulus is presented again marking the beginning of a new trial, the staircase selected in that trial is updated and the test stimulus is presented based on the participant’s previous response to the staircase stimulus.

In this manner, the present experiment is similar to Brenner et al’s (2001) in that participants in both experiments were required to compare the second stimulus (test stimulus in our study) to the first reference stimulus while maintaining smooth pursuit of another target. However, in our experiment, the effect of mislocalization was tested for eye movements in both directions. The present experiment began with a practice block of 50 seconds containing presentations of both the reference and test stimuli to familiarize participants to the design of the experiment. The practice block was followed by the real experiment of six minutes, where pupil data was recorded.

iii. Treatment of collected pupil data

The edf files obtained from the eyetracker for each participant were subject to an analysis script which transferred the data from each run of the participant to HDF files which were readable. Moreover, blinks and saccades are detected in the participant’s data and interpolated. The pupil data was decimated using filters. (see Figure 2)

(14)

14

(b) Gaze data

Figure 2. Example pupil data treatment plots for subject 3 (a) pupil data recorded by the eyetracker is “cleaned”

out for every trial - by estimating the effects of blinks, saccades and foreshortening on the data and regressing the effects out. Here, we show interpolation over blinks in pupil data. The x-axis represents the time duration in seconds for one trial and the values on the y-axis represent pupil size (b) The sinusoidal eye movement gaze patterns on pursuing the SP dot, collected over the duration of an entire experimental run (right = up). The x-axis represents time duration for one trial in milliseconds and the y-axis represents time duration (1000 ms) for one pass of the eye from one end of the experimental screen to another.

iv.

Analysis of participants’ response

The test stimulus position values for each of the four staircases were recorded for each trial per participant. Since test stimuli for staircases 0 and 1 are flashed when the smooth pursuit eye movements are to the right, they are grouped together. Staircase 0 is flashed to the left of the screen center (or the reference stimulus) and staircase 1 is flashed to the right of the screen center, while the eye is moving to the right. Similarly, staircases 2 and 3 are grouped together on account of test stimuli being flashed when the smooth pursuit eye movements are to the left. The test stimulus for staircase 2 is flashed to the left of the screen center and that for staircase 3 is flashed to the right, while the eye is moving to the left. (see Figure 3)

(15)

15

Figure 3: Examples of staircase plots for two subjects. The yellow and green lines represent staircase numbers 0

and 1 that start from -50px and +50px to the center (0) respectively and are presented when the eye movements are from left to right. The red and blue lines represent staircase numbers 2 and 3 that start from -25px and +25px to the center (0) respectively and are presented when the eye movements are from right to left. In the first few trials, based on the participants’ responses, the staircase values are updated which leads them to converge to the center. The effects of mislocalization are seen best after this point when staircases 0 and 1 are estimated to be further left (negative) than its veridical position and staircase 2 and 3 are estimated to be further right (positive) than its veridical position

The mean of the last six test stimulus positions for staircase 0 and 1 were compared to the mean of the last six staircase positions of staircase 2 and 3. In the initial few trials of the experiment, it is relatively easier to indicate whether the test stimulus flash is to the left or right of the reference stimulus flash (at the center) because its position is noticeably distant from the center. Therefore, typically, most participants correctly estimate the test stimulus’ position relative to the reference stimulus which causes the staircases to converge towards the center. Once the test stimulus

(16)

16

approaches the center of the screen, comparison becomes more difficult. Mislocalization effects are hypothesized to be most pronounced in the last few trials of the experiment when it is fairly certain that the four staircases would have already converged towards the center and it is the trials after this point that are of most interest to our research question.

If mislocalization effects do indeed occur in the context of smooth pursuit, we expect the combined mean of staircase 0 and 1 to be in the negative in the last six trials. This is because, for staircases 0 and 1, the eye movements for the test stimulus passes are from left to right, which implies that the eye movement for the reference stimulus pass was from right to left (since the test stimulus pass always immediately follows the reference stimulus pass). This would imply that if mislocalization were to occur, it should lead people to mislocalize the flash in the direction of pursuit (to the right), especially once the test stimulus has converged to the center.

This is because, for the reference stimulus pass, as the eye is moving to the left, we expect the reference stimulus to be further mislocalized in the direction of pursuit (i.e. to the left). Similarly, for the test stimulus pass, as the eye is moving to the right, we expect the test stimulus to be further mislocalized to the right. The combined effect of the mislocalizations should heighten the perceptual difference between the reference and test stimulus, leading the observer to believe that the test stimulus is further to the right than its veridical position. As the user is tempted to respond “right” to this setting, the test stimulus for that staircase “jumps” backward (to the left), on its next presentation, due to the staircase reversal procedure. As this continues, the test stimulus positions of staircases 0 and 1 tend to remain to the left of the center

in the last few trials.

In a similar vein, if mislocalization should occur, the test stimulus positions of staircases 2 and 3 are expected to remain to the right of the center. Again, this would be due to staircase reversal procedures that make the test position “jump” to the right as the observer is increasingly convinced that the test stimulus is further to the left than it actually is. (see Table 1)

(17)

17 STAIR-CASE REF-STIM PASS DIRECTION TEST-STIM PASS DIRECTION TEST-STIM FLASH LOCATION EXPECTED VALENCE OF THE MEAN VISUAL

REPRESENTATION OF REF AND TEST STIM PASSES

0 Right to Left Left to Right Left Negative

1 Right to Left Left to Right Right Negative

2 Left to Right Right to Left Left Positive

3 Left to Right Right to Left Right Positive

Table 1: A summary of reference and test stimulus pass direction, location of stimulus and expected valence of the

mean for that staircase in the last six trials of the experiment.

To test for the significance of the difference between the two staircase means, a two-sided t test was conducted to test h0 against the obtained data.

v.

Testing the position-capture effect

By using a staircase method to present test stimulus flashes, we can capture, on a small scale, the effect of this mislocalization in smooth pursuit. It allows us to compare between the relative positions of test stimuli on an individual level to isolate those test stimuli spatial positions that are susceptible to mislocalization from those that are not (therefore allowing us to draw a

(18)

18

threshold). It also allows for comparison between mislocalization effects for stimuli being flashed on opposite ends of the screen, during the same eye motion direction. Therefore, it allows us to test the position-capture effect as well as its reversal, as put forth by Watanabe (2005). According to the position capture effect, flashes presented ahead of the fixation point should be more influenced by the moving stimulus (in this case, the SP dot) in the direction of motion, than flashes presented before the fixation point (Watanabe, 2005). Therefore, it would seem logical to compare localization performance between staircases 0 and 1 and between staircases 2 and 3, since they are both presented in the same eye movement direction and start at similar screen co-ordinates, but on opposite ends of the screen [-50px, 50px, -25px, 25px to the center respectively]. Thus, staircase 0 is flashed before the fixation point (the center) and staircase 1 is flashed ahead of the fixation point when the eye motion direction is from left to right (see Figure 4) and staircase 3 is flashed before the fixation point and staircase 2 is flashed ahead of the fixation point when the eye motion direction is from right to left.

Figure 4. Pictorial representation of staircase 0 and 1. Staircase 0 is flashed to the left of the center (and

therefore before fixation point) and staircase 1 is flashed to the right of the center (after fixation point). We expect greater number of trials for staircase 1 to reach the screen center.

We compared the number of trials it took for each staircase of a staircase set to converge to or closest to the screen center. If the position-capture effect occurred in our experiment, we should expect to see greater number of trials for staircases that are flashed ahead of the fixation point (staircase 1 and 2) to converge to the center since these stimuli would be subject to greater mislocalization, leading to more reversals in the opposite direction. If the staircases didn’t cross the center, we took into account trials for those staircases that came closest to the center (+/- 5px from screen center of 512px), but for convenience, we’ll use the phrase “converge to the center” for this comparison. If any one staircase widely deviated from the center, the staircase comparison for that participant was excluded (For this reason, data from 5 participants was

(19)

19

excluded from the comparison between staircase 0 and 1 and 3 participants were excluded from the comparison between staircase 2 and 3).

On average, it took 12.2 trials for staircase 0 to reach the center and 6.9 trials for staircase 1 to reach the center. As for staircases 2 and 3, an average of 3.82 and 4 trials were needed to converge to the center. These results were inconsistent with the position-capture effect. However, this could also be because the distance from the center for the two sets of staircases was not very large especially for staircase sets 2 and 3 (set at 25px to the left and right of the center, respectively). Besides, for some participants, staircase responses were outside the expected range leading to large variations in the number of trials needed to converge to the center. In their experiment, Watanabe et al. (2005) used a fixation stimulus (cross) that differed from the moving stimulus (horizontal bars). They found the asymmetric mislocalization effect only when the to-be-localized flash was localized with respect to the moving bars, and not to the fixation cross. They did this so as to determine whether the frame of reference for asymmetric mislocalization was fovea-centered or moving-stimulus centered. In our experiment, the fixation point is the moving stimulus itself, making it difficult to draw this distinction, since foveal attention is focused on the moving SP dot stimulus. Moreover, Watanabe et al. (2005) define the space ahead of the moving object as one where the to-be-localized stimulus will be displayed vs. the space before the moving object where the to-be-localized stimulus is already displayed. Therefore, prediction and expectation play an important role in their study. In our experiment, there was less scope for prediction because the staircases were selected at random and therefore also the test stimulus positions varied every trial. Also, since the test stimulus was always displayed when the SP dot reached the center, it didn’t matter too much if the test stimulus was presented before or ahead of the dot (the temporal dynamics of the test stimulus remained the same in every trial regardless of whether it was presented before or ahead of the SP dot).

(20)

20

III. Results

Gaze data across participants showed consistent smooth pursuit behaviour, across all trials, reflecting a pursuit gain of almost 1. 10 of the 15 participants tested showed behaviour consistent with expectations, i.e., the combined staircase means for staircase 0 and 1 for these participants was in the negative and the combined staircase means for staircase 2 and 3 was in the positive. (see Table 2)

MEANS OF STAIRCASE 0 & 1 MEANS OF STAIRCASE 2 & 3 DIFFERENCE

1 14.19 -3.42 17.61 2 27.33 29.33 -2 3 -31.8 70.19 -101.99 4 -1.4 1.07 -2.47 5 -4.89 -3.06 -1.83 6 -12.4 -15.5 3.1 7 5.97 -22.1 28.07 8 -9.01 9.96 -18.97 9 0.668 17.17 -16.502 10 -2.53 19.56 -22.09 11 -7.34 8.06 -15.4 12 -67.0 49.06 -116.06 13 12.10 8.03 4.07 14 -72.7 73.36 -146.06 15 -41.6 16.78 -58.38 MEAN -12.7 17.255

Table 2: Comparison of the mean staircase position values of 0 and 1 vs. mean staircase position values of 2 and 3

(21)

21

The t-test results of the group staircase means showed a significant difference between the means in the expected direction (t(27.99) = 2.24, p < 0.05).

non-stim

reference

test

(22)

22

(b) Staircase means

Figure 5. Group level data. (a) Sinusoidal waves representing eye movement from left to right (upward curve) and

vice-versa. An overlay of mean reference stimulus passes (yellow), test stimulus passes (red) on non-stimulus passes (blue) for all participants across all trials. The waves represent almost perfect smooth pursuit performance on a group level because the stimulus passes map on well onto non-stimulus passes (where there no lesser or no distractions in smooth pursuit activity). The plot also shows deviations in pursuit, reflected through error bands (b) Barplot representing the mean of the last six staircase positions for staircase 0 and 1 (left) and staircase 2 and 3 (right) respectively. As expected, the combined mean of the first two sets of staircases was negative and the combined mean of the last two sets of staircases was positive. Moreover, the difference between the two sets of means was statistically significant (p<0.05) indicative of the presence of mislocalization in our study.

A barplot representing the last six average positions of staircases with SEM plotted as error bars in blue

(23)

23

IV. Discussion & Final Thoughts

The aim of the present study was to optimize the conditions under which mislocalization occurs in the context of smooth pursuit. By designing a paradigm that best enhances this mislocalization effect, it would be possible to replicate the study while delving into neural processes (with fMRI) to determine which bottom-up and top-down processes contribute most to this perceptual effect.

The main effect we tried to replicate in our study was that of mislocalization in the context of smooth pursuit, specifically to test the effects of stimulus motion direction in localization. We expected the target stimulus to be mislocalized further along the direction of SP dot motion. For this purpose, we compared means of the first two sets of staircases (staircase 0 and 1) with the mean of the last two sets of staircases (staircase 2 and 3) and found the difference to be statistically significant at p<0.05. This implies that participants in our study did tend to mislocalize the test stimulus in the direction of motion, as expected, also implying that a noticeable forward shift in the direction of motion was found. This experiment was conducted in the presence of a structured background (where the edge of the monitor was clearly visible) which has been known to decrease the effects of mislocalization in such smooth pursuit paradigms (Brenner et al., 2001). It’s possible that pursuing the SP dot in complete darkness could accentuate the effect of mislocalization due to an absent frame of reference.

Some possible pitfalls of our study could be the presence of random responses as a result of smooth-pursuit induced-fatigue. The experiment lasted for six minutes with the SP dot oscillating at 0.5 Hz continuously for the entire duration, which could be challenging for most people. The verbal reports collected at the end of the experiment hinted at the fatiguing nature of the task. However, smooth pursuit movements were not found to be adversely affected, as observed in the gaze-data plot (see Figure 5(a)). Moreover, additional manipulations such as varying speeds of the SP dot, contrast values of the reference and test stimuli and differing luminance values of the screens were absent, which could serve as potential avenues for future research. It would also be interesting to look into the brain using high-end neuroimaging techniques such as layer-specific fMRI to see how bottom-up and top-layers interact in creating the perceptual mislocalization effect. Specifically, it would be interesting to verify if bottom-up layers on account of being detail-oriented, reflect reality but are influenced heavily by top-down layers that are shaped heavily by expectations.

(24)

24

V. References

Brenner, E., Smeets, J. B., & Van den Berg, A. V. (2001). Smooth eye movements and spatial localisation. Vision research, 41(17), 2253-2259.

Hubbard, T. L. (1995). Environmental invariants in the representation of motion: Implied dynamics and representational momentum, gravity, friction, and centripetal force. Psychonomic Bulletin & Review, 2(3), 322-338.

Kerzel, D. (2000). Eye movements and visible persistence explain the mislocalization of the final position of a moving target. Vision research,40(27), 3703-3715.

Kerzel, D., Aivar, M. P., Ziegler, N. E., & Brenner, E. (2006). Mislocalization of flashes during smooth pursuit hardly depends on the lighting conditions.Vision Research, 46(6), 1145-1154.

Kerzel, D., Jordan, J. S., & Müsseler, J. (2001). The role of perception in the mislocalization of the final position of a moving target. Journal of Experimental Psychology: Human

Perception and Performance, 27(4), 829.

Königs, K., & Bremmer, F. (2010). Localization of visual and auditory stimuli during smooth pursuit eye movements. Journal of vision, 10(8), 8-8.

Lisberger, S. G., Morris, E. J., & Tychsen, L. (1987). Visual motion processing and sensory-motor integration for smooth pursuit eye movements. Annual review of neuroscience, 10(1), 97-129.

Mateeff, S., Yakimoff, N., & Dimitrov, G. (1981). Localization of brief visual stimuli during pursuit eye movements. Acta Psychologica, 48(1), 133-140.

Mitrani, L., & Dimitrov, G. (1982). Retinal location and visual localization during pursuit eye movement. Vision research, 22(8), 1047-1051.

Pola, J. (2002). Models of the saccadic and smooth pursuit systems. InModels of the visual

system (pp. 385-429). Springer US.

Rotman, G., Brenner, E., & Smeets, J. B. (2004). Mislocalization of targets flashed during smooth pursuit depends on the change in gaze direction after the flash. Journal of

(25)

25

Souman, J. L., Hooge, I. T. C., & Wertheim, A. H. (2006). Localization and motion perception during smooth pursuit eye movements. Experimental brain research, 171(4), 448-458.

Stork, S., Neggers, S. F., & Müsseler, J. (2002). Intentionally-evoked modulations of smooth pursuit eye movements. Human movement science,21(3), 23-36.

Van Beers, R. J., Wolpert, D. M., & Haggard, P. (2001). Sensorimotor integration compensates for visual localization errors during smooth pursuit eye movements. Journal of

Neurophysiology, 85(5), 1914-1922.

Van der Heijden, A. H., Van der Geest, J. N., De Leeuw, F., Krikke, K., & Müsseler, J. (1999). Sources of position-perception error for small isolated targets. Psychological

Research, 62(1), 20-35.

Watanabe, K. (2005). Asymmetric mislocalization of a visual flash ahead of and behind a moving object. Perception, 34(6), 687-698.

Investigation of motor and visual space in the brain: optimizing mislocalization effects in a smooth pursuit paradigm