The influence of transiently learned expectations on illusions in
short-term memory
Pauline van Lieshout Bachelor Thesis
Supervisor: Yair Pinto Bachelor Psychobiologie
First Assessor: Yair Pinto Universiteit van Amsterdam
Second Assessor: Timo Stein Student Number: 11221828 Date: 24/01/2019
Abstract
Expectations influence our perception of the world. These expectations can either be deeply ingrained through a life-time of experience, or transiently learned to adapt to the current situation. It is not yet understood if these transient expectations influence short-term memory (STM) and perception. Here, we manipulated stimulus probability to evoke transient
expectations on the direction of movement of the stimulus. Participants were especially biased to estimate direction of movement towards the two motion directions most probable to occur if no stimulus was presented at all. In other words, when participants hallucinated the presence of dots, their illusory percept/memory was heavily affected by transiently learned expectations. Crucially, these illusory memories were affected by expectations more as memory decayed. These results suggest that not only is STM susceptible to transient expectations, but STM's susceptibility increases as bottom-up information fades from memory. These findings cast doubt on the reliability of eye witness memories, especially when these witnesses are biased by expectations and the time between witnessed event and report is prolonged.
Introduction
Our memories are an essential part of who we are. Without them, we would only be driven by external cues, unable to think about the past or predict the future. Memory is needed for everything from understanding sentences, navigating through space to recognizing others. However, these memories we rely so heavily on, are not always accurate. Long term, they can get heavily distorted by a number of variables, such as expectations, cultural influences or stress (Kim, Park, & Lee, 2014). These variables can distort encoding of memories as well as retrieval (Loftus, 2019). It is suggested that these false memories are the result of an adaptive system of the brain that integrates prior knowledge with memories, which is usually efficient in everyday life (Schacter, Guerin, & St. Jacques, 2011). However, in eye-witness reports, it can lead to false statements and in the worst case, wrongful conviction. In eye-witness reports, time is of the essence. While time between the event and the report increases, memories become more susceptible to expectations and therefore become less accurate (Barkowitz & Brigham, 1982; Tuckey & Brewer, 2003). This suggests that STM is viewed as being more reliable than LTM. This is intuitive, as it feels easy to remember something that happened a few seconds ago. But is our STM as reliable as we think it is? We argue that STM is sensitive to distortion as well.
Expectations are known to distort LTM, but what about STM? For instance, it has been found that the context of an event heavily influences what is remembered in a STM task. For instance, semantic context can provoke expectations on what was observed, leading to false memories (Deese, 1959; Gallo, 2010; Roediger & McDermott, 1995). In another example, our life-long experience with letters can be used to create illusions. De Gardelle, Sackur, & Kouider, 2009 found, in a STM task, that subjects were prone to confuse pseudo-letters (i.e. mirror-imaged or rotated letters) with normally oriented letters. They interpreted this as evidence that expectations influence visual perception. However, a study of Pinto, Seth & Otten (2019) has linked these illusions of perceiving pseudo-letters as real letters to STM instead of solely to perception. They found that the number of illusions increased as
participants held information over a short period of time, which would not be expected if the errors were only perceptual in nature. This provides evidence that not only perception can be influenced by expectation, but also STM. In all, it can be argued that expectations of low-level visual properties as well as high-low-level cognition of concepts influence STM.
However, these are examples of deeply ingrained expectations influencing STM. Deeply ingrained expectations are slowly formed over a life time, affecting all observations.
Therefore, it is likely that these expectations influence STM as well as perception. However, expectations are not always deeply ingrained, but can also be transiently learned. Transiently
learned expectations are quickly acquired and highly adaptable to context, only affecting perception in isolated situations. For instance, perceptual bias of an ambiguous motion stimulus can be heavily influenced by transient expectations (Sterzer, Frith, & Petrovic, 2008). Research also shows that transient and deeply ingrained expectations can interact, playing a crucial role in perception (Dogge, Custers, Gayet, Hoijtink, & Aarts, 2019; Seriès & Seitz, 2013). As transient expectations influence perception, it can be argued that they influence STM as well.
A simple way to manipulate transient expectations is through statistical learning. Implicit statistical learning relies on the brains ability to automatically detect regularities and direct attention towards them (Zhao, Al-Aidroos, & Turk-Browne, 2013). This attentional focus towards regularities is essential for statistical learning (Turk-Browne, Jungé, & Scholl, 2005). An experiment that uses statistical learning to induce transient expectations is the
experiment of Chalk, Seitz & Series (2010). They presented subjects with coherently moving dots within a central circle, e.g. all dots within this circle moved horizontally. Within one trial all dots moved in one direction, however, across trials motion direction varied. Moreover, unbeknownst to the subject, two motion directions occurred more often than the other directions. The moving dots were presented either very clearly (large luminance difference with the background), barely visible (minuscule luminance difference) or the dots were absent. Participants indicated if dots were present at all, and if so, in which direction they moved. When participants indicated the motion direction of barely visible dots, their
estimations tended towards one of the two more frequently occurring motion directions (i.e. the expected direction). Moreover, when no stimulus was presented at all, participants occasionally ‘hallucinated’ dots. Crucially, these hallucinated dots generally moved in the expected directions (or very close to them). Since participants estimated the direction of motion during presentation of the moving dots, the illusions were considered to result from perceptual processes (perceptual illusions). In a follow up experiment, the researchers discovered participants did not pick up on every statistical rule. In this experiment, the moving dots were presented in two colors, each color predictive of a direction most likely to occur. They found that depending on the degree of overlap of the statistical rules,
participants did not use the extra information provided by the color of the dots (Gekas, Chalk, Seitz, & Seriès, 2013; Seriès & Seitz, 2013). To summarize, statistical learning is an
automatic way to implicitly acquire transient expectations, given that attention is directed towards the stimulus. Learning multiple statistical rules at once has proven to be difficult as it is influenced by the degree of overlap between the rules.
To further understand the influence of transient expectations on STM, it is necessary to understand how these expectations can influence perception at all. It has been suggested that perception is shaped by predictive processing. Researchers found that in the brain’s hierarchy, higher brain areas send feedback to lower brain areas to influence the sensory signals coming in. This claim is highly supported by researches investigating neural processing. It is found that indeed expectations are integrated with sensory information at early stages in visual processing (Kok, Brouwer, van Gerven, & de Lange, 2013;
Summerfield, Trittschuh, Monti, Mesulam, & Egner, 2008) and auditory processing (Lange, 2009; Todorovic, van Ede, Maris, & de Lange, 2011). Aside from top-down influence, there is evidence that lower brain areas also send feedforward signals to higher brain areas
displaying the prediction error. The prediction error can cause higher brain areas to adjust expectations, minimizing the prediction error (Friston, 2010; Rao & Ballard, 1999). It is suggested that observers do so by using Bayesian strategy, combining a learned prior (expectation) with sensory information in a probabilistically optimal way to form perception (Knill & Pouget, 2004; Körding & Wolpert, 2004; Tassinari, Hudson, & Landy, 2006). In essence, perception is constructed by the integration of sensory signals with expectations by using a Bayesian strategy.
In short, so far, we know that deeply ingrained expectations have far reaching effects, from basic visual processing to storage of conceptual items in STM. However, the impact of transiently learned expectations is less clear. They seem to affect perception through predictive processing, but can they also affect STM? Pinto et al.'s (2019) research has showed that deeply ingrained expectations strongly impact STM, but it is not yet clear that transiently learned expectations can do so as well. One reason to be skeptical of this is that a STM sensitive to transiently learned expectations may not be robust enough to serve as a reliable guide. The main focus of the current research is to answer the question if transiently learned expectations can influence STM. Secondarily, the effect of transient expectations on perception is also investigated. It is hypothesized that transient expectations affect STM, resulting in memory illusions. This effect is expected to increase as memory decays. Additionally, it is hypothesized that perception is also influenced by transient expectations, which leads to perceptual illusions.
We employ 2 experiments. However, it is important to note that the 2 experiments are not independent as they are conducted in sequence and share the same participants. The first experiment (Experiment 1) is an exact replication of the experiment of Chalk et al. (2010). Coherently moving dots (either highly visible, barely visible or invisible) will be presented, during which participants estimate in which direction the dots moved. Two directions of
motion are more likely to occur than others. Following this estimation task, participants indicate whether or not they have seen a stimulus in the detection task. In Experiment 1, the effect of transiently learned expectations on perception is determined. To involve STM, a second experiment (Experiment 2) is added. In Experiment 2, STM is involved by
memorizing three displays of moving dots, instead of one. The three displays will be
presented quickly after one another and after witnessing all three displays, the target display is revealed (I, II or III). The participant executes the estimation task and the detection task for the target display only. This method ensures that participants hold three items in their
memory, during which it is hypothesized that expectations influence information stored in STM. Briefly, the aim of Experiment 2 is to measure the effect of transiently learned expectations on STM.
It is predicted that in Experiment 1, participants are biased to make estimations closer to the most frequently presented directions of motion. The estimation bias is not only expected to occur if the dots are barely visible but also if the dots are not visible at all. These predictions are based on the findings of Chalk et al (2010) and assume the presence of perceptual
illusions. Experiment 2 is expected to provoke stronger estimation bias than Experiment 1, as
Experiment 2 is more difficult to execute. Additionally, larger estimation bias is expected if the target is I vs. if the target is III. After all, the second and third display possess similarities to the first display and would therefore interfere with the memory of the first display (Burke & Srull, 1988; Ricker, Vergauwe, & Cowan, 2016). The participant is made more unsure, which is hypothesized to cause information stored in their STM to be more susceptible to the influence of expectation. If this difference in bias is found while comparing target I and target III, it would prove the existence of memory illusions. After all, if the illusions are only
perceptual, all three target displays would be equally susceptible to illusions, resulting in equally biased estimates for all three targets. If found, it would provide proof that when information is held in STM, transient expectations play an increasingly important role in estimating and detecting movement.
To ensure that the illusions are not just a vague gist of familiarity but clear memories
(Brainerd & Reyna, 2002), follow up research is required. In the follow up, participants should indicate how certain they are of their judgment. If participants report memory illusions with high confidence, this would strongly suggest that transiently learned expectations generate memory illusions instead of response biases.
Experiment 1
The aim of the first experiment was to determine whether transient expectations evoke perceptual illusions by using the exact method provided by Chalk et al (2010). Transient expectations were evoked by manipulating the probability of the occurrence of certain motion directions. These transiently learned expectations are predicted to bias the estimations of participants towards the most probable motion directions. Mainly, this estimation bias is expected to occur if no stimulus is presented, but a stimulus is detected. These
‘hallucinations’ of movement are perceptual illusions. Secondarily, estimation bias is also expected to occur if the stimulus is barely visible. To summarize, in Experiment 1 we examine the effect of transient expectations on estimation bias and perceptual illusions.
Method
Training
To ensure participants were motivated and possessed a minimal performance level on a visual memory task, they were trained on a cued change detection task. Stimuli were
presented on a 23-inch ASUS display (screen resolution 1980 x 1020) under control of a Dell Optiplex 9010 computer. All tasks were programmed in Matlab (version 2012b, Natick, Massachusetts: The MathWorks, Inc; 2010) using Psychophysics Toolbox extensions (Brainard, 1997; Kleiner et al., 2007; Pelli, 1997). For the training task, the refresh rate was set to 60 Hz.
In the training task, participants were presented with a memory display consisting of 8 white rectangular blocks (CIE x,y coordinates: 0.289, 0.317; luminance: 60.5 cd/m2; 1.16 x 0.29˚ in size). Each rectangle presented in the array was randomly allocated a vertical, horizontal or oblique (45˚ or 135˚) orientation. The rectangles were positioned in a circular shape (radius 4.68˚) around a red fixation dot (radius: 0.4˚, CIE: 0.641, 0.341; luminance: 11.9 cd/m2). The memory display was presented for 0.25 seconds, after which a blank screen was presented for 0.1-1 seconds. A cue (CIE:0.405, 0.521; luminance: 33.9 cd/m2) highlighting the location of one of the rectangles appeared 0.1 or 1 second after offset of the memory display and lasted for 0.5 seconds. The cue was either presented during the blank screen, or during the test display. The test display consisted of the exact same array of rectangular blocks, except in half of the trials the cued rectangle had changed orientation. Participants indicated
whether the cued rectangle had changed orientation or not by pressing a key (for task, see figure 1). Only after response, the test display disappeared. The training task consisted of 10 blocks of 42 trials each and took about 45 minutes to complete. Feedback on performance was provided after every block and every trial. If participants scored less than 70% correct on the last 5 blocks of the cued change detection task, they were excluded from taking part in
Experiment 1 and 2. About 20% of participants were excluded based on the training task. For a similar training procedure, see Pinto, Sligte, Shapiro & Lamme (2013) and Sligte, Scholte & Lamme (2008).
Participants
The exclusion criterium of >70% correct on the training task led to exclusion of 5 out of 23 participants. From the remaining 18 participants, 17 participants completed the full
experiment (12 female; age range 18-25 years, average age: 21.2). Only naïve participants with normal or corrected-to-normal vision were permitted to execute the experiment.
Participants were either rewarded with research credits or money (€10,- / hour). All participants gave their informed consent, approbated by the ethics review board of the Psychology Department at the University of Amsterdam.
Design
Each session, Experiment 1 was followed by Experiment 2. The total experiment consisted of 4 sessions of 90 minutes each. Each participant executed both Experiment 1 (30 min.) and Experiment 2 (60 min.) in this order each session. Experiment 1 consisted of 13 blocks of 34 trials, adding up to 442 trials per session and 1768 trials in total.
Stimuli
Participants were seated in a dimly lit room at a distance of 50 cm away from the screen with their forehead and chin positioned against a chin rest. The refresh rate was set to 100Hz. Over the background (CIE: 0.278, 0.303; luminance: 4.6 cd/m2) an annular circle was presented (radius: min. 1.5˚, max. 5˚). Within the annulus, a field of dots (density: 2 dots/degree) moved coherently at a speed of 9˚/sec.
Figure 1. Experimental set-up of the cued change detection training task. Two conditions are shown in which the cue is either presented during the blank screen (left), or the cue is presented together with the test screen (right). Participants estimated whether the cued rectangle had changed orientation, or stayed the same compared to the first display.
Procedure
Preluding each trial, participants fixated on a fixation dot (radius: 0.3˚, CIE: 0.285, 0.312; luminance: 23.8 cd/m2) for 400 ms. After fixation, the circular annulus appeared wherein moving dots were either very clearly visible (15% of trials; CIE: 0.280, 0.306; luminance: 8 cd/m2), barely visible (55% of trials; CIE: 0.278, 0.303; luminance: 4.7 cd/m2) or invisible (30% of trials; CIE: 0.278, 0.303; luminance: 4.6 cd/m2). Each participant was assigned a random reference angle, which remained the same for Experiment 1 and 2. If dots were barely visible, the motion direction could be either 0˚, 16˚, 32˚, 48˚ or 64˚ in respect to the reference angle. Each angle occurred with the same probability (P=0.11), except for 32˚, which occurred more frequently (P=0.55), manipulating participants expectations (see figure 2). If dots were presented very clearly, motion direction was either completely random on a third of the trials, or in the direction of 32˚ on the remaining two third of the trials. The luminance levels and motion directions were randomly intermixed.
Together with the annulus and moving dots, a red estimation bar (CIE: 0.486, 0.323; luminance: 11.7 cd/m2) appeared (see figure 3). The starting position of the estimation bar was random, pointing outwards from fixation to the smallest circle of the annulus (radius: 1.5˚). Participants were instructed to estimate the direction of motion of the dots by pointing the estimation bar in the direction of movement
and by clicking the mouse. From now on this will be referred to as the ‘estimation task’. The moving dots remained visible until an estimation was made, or 3750 ms had elapsed. If no dots were visible, participants were instructed to either make a random estimate, or wait until 3750 ms had elapsed. After the estimation task, participants were presented with a display, divided by a line, stating ‘Dots’ on the right side and ‘No dots’ on the left side of the display. Participants where prompted to indicate whether or not they had seen the moving dots by clicking the right (‘Dots’) or left (‘No dots’) side of the screen. This will be referred to as the ‘detection task’. After performing the detection task, the fixation dot reappeared, preluding the next trial. The location of the cursor was set to the middle of the display before the estimation and the detection task. Feedback on detection performance was provided every
9
Figure 2. The probability distribution of each of the motion directions to occur in trials with barely visible dots. Note that two motion directions (+32˚ and -32˚) are more probable to occur than the other motion directions. All displayed values are in respect to the reference angle, which was different for each participant.
Figure 3. Experimental set-up of one trial in Experiment 1. Participants fixated for 400 ms, after which a circular annulus appeared containing dots that were either highly visible, barely visible or absent. The dots moved coherently, although direction of movement differed each trial. During presentation of the dots, participants estimated the direction of movement of the dots with the red estimation bar. After doing so, or after 3750 ms had passed, the detection task appeared. Participants clicked on the right or left side of the screen to indicate if they had seen ‘Dots’ (right) or ‘No dots’ (left). -64 -48 -32 -16 0 16 32 48 64 0 0.1 0.2 0.3
Probability of motion direction in trials with barely visible dots
Angle (deg) P ro b ab ili ty ( P )
trial by means of a green (correct) or red (incorrect) fixation dot, which preceded the next trial. Additionally, participants received feedback on their detection rate (percentage correct responses on the detection task) and estimation error (mean deviation of estimates from the true direction of movement) every block (34 trials).
Analysis
Before analysis, a number of trials were excluded. Incongruent trials, in which a stimulus was detected but no estimation was made, were excluded from analysis. Also, the first 3 blocks (102 trials) were excluded from analysis to adjust the display settings (brightness) for each participant so that the barely visible dots were near-threshold. Additionally, based on the experiment of Chalk et al. (2010), two exclusion criteria were employed to make sure participants stayed motivated throughout the experiment. The exclusion criteria were based on the performance on trials with clearly visible dots only. Participants were required to reach
a minimum detection rate of 80% and an estimation error lower than 30˚. Based on these preregistered criteria, 2 out of 17 participants were excluded from analysis.
The effect of transient expectations on perception was measured by computing estimation bias towards the motion directions most probable to occur. When no stimulus was presented, yet a stimulus was detected, estimation bias was determined by calculating the deviance of each estimation to the closest frequently presented angle (32˚, in respect to the reference angle). The mean deviance shows how strongly participants are biased to estimate
movement towards these directions. If participants made completely random estimates, mean deviance from the most probable directions is expected to be 58˚ (see figure 4). To elaborate, if for instance the reference angle was 0˚, the circle could be divided into two hemispheres. If estimates were random within each hemisphere, the average estimation would be 90˚ in the right hemisphere, and 270˚ in the left hemisphere. These mean estimations would be 58˚ away from most probable motion directions 32˚ (+32˚ right hemisphere) and 328 (-32˚, left hemisphere) in respect to the reference angle. If the mean deviance is smaller than 58˚, it indicates that perceptual illusions were biased towards the most probable angles. By means of a Paired Samples T-test, the deviance of estimates from expected angles is compared to the deviance of random estimates.
If the stimulus was barely visible and it was correctly detected, estimation bias was
calculated for each presented motion direction (16˚, 32, 48˚ and 64˚ in respect to the
11
Figure 4. visual indication why 58˚ is the mean deviance from the most probable motion directions if estimations are random. If the reference angle is 0˚ (this differs for each participant), the circle can be divided into two hemispheres (dashed line). If participants made random estimations in each hemisphere, their mean estimation would be 90˚ in the right hemisphere and 270˚ in the left hemisphere. The distance between the random estimates and the closest frequently presented angle should be 58˚. If the distance is smaller than 58˚, estimations are biased towards the most probable motion directions.
reference angle). This was done by determining the deviance of the mean estimated angle from the actually presented motion direction. For each motion direction, bias of the right and left side of the reference angle were combined (e.g. -16 and +16, considering the direction of the bias) before comparisons. Estimation bias at motion direction 16˚ and 48˚ were
compared to estimation bias at motion direction 32˚.
For exploratory purposes, on no-stimulus trials, the mean deviance of estimations from the most probable angles was also calculated if the stimulus was undetected, to correct for response bias. Additionally, we analyzed if reaction time on the detection task influenced estimation bias in the case of a non-presented, detected stimulus. First, the median reaction time was determined, and data was divided into two groups: 1. Short reaction time and 2. Long reaction time (reaction time lower and higher than the median reaction time,
respectively). Then, the mean deviance of the estimations to the most probable angles was determined for each group.
Results
Mainly, we found that estimations were biased towards the most probable angles in trials where a stimulus was detected, but no stimulus was presented (see figure 5). It was found that the
mean deviance from the most probable motion directions (M = 39.3˚, SD = 14.8˚) was smaller than to be expected if estimates would be random (58˚), t(14) = 10.3, p < .001.
Secondarily, estimation bias was also found if the dots were barely visible and detected. In concordance with the experiment of Chalk et al. (2010), estimation bias at 48˚ (M = -16.7˚, SD = 14.4˚) was smaller than the estimation bias at 32˚ (M = -4.5˚, SD = 13.2˚), t(14) = -2.4, p < .001. This indicates estimates are negatively biased towards 32˚ when the actual
Figure 5. This image shows the mean deviance of estimates from expected angles if no dots were presented. The dots were either detected(grey) or undetected (black). Note that the deviation of perceptual illusions (trials in which a stimulus was detected) was significantly smaller (*) than random estimates (dashed red line) and were therefore biased towards the most frequently presented angles.
Detected Undetected 0 35 70 0 38.95 Undetected; 48.69 0
Experiment 1: estimation bias of perceptual illusions
D ev ia n ce f ro m e xp ec te d a n gl es ( d eg )
direction of the dots is 48˚. However, unlike in the earlier mentioned research, this
difference in estimation bias between 32˚ (M = -4.5˚, SD = 13.2˚) and 16˚ (M = -2.5˚, SD = 16˚) was not found, t(14) = -0.1, p = .648). Indicating estimates were not biased towards 32˚ when the actually presented motion direction was 16˚.
For exploratory purposes, we compared estimation bias of detected and undetected no-stimulus trials to control for response strategies (see figure 5). No significant difference in estimation bias was found between detected (M = 39.3˚, SD = 14.8˚) and undetected (M = 44.2˚, SD = 16.4˚) stimuli, t(14) = -0.9 , p = .22. Also, estimations in undetected trials were not completely random (58˚), t(14) = 3, p = .016. Also, a significant difference in estimation bias was found between the short reaction time group (M = 34.7˚, SD = 37˚) and the long reaction time group (M = 42.7˚, SD = 40˚), t(14) = -0.5, p < .001. These results indicate that short reaction times lead to stronger estimation bias towards expected angles.
Main findings
Most importantly, we found that estimates were biased towards the most frequently presented motion directions when no stimulus is presented, but a stimulus was detected. These results show that perceptual illusions do occur as a result of transient expectations. Secondarily, estimates on trials with barely visible dots were also biased towards the most probable angles, although evidence was inconsistent. Our results largely replicate the results of the experiment of Chalk et al. (2010). However, the effect of transient expectations on STM has yet to be discovered and will be discussed in Experiment 2.
Experiment 2
The focus of Experiment 1 was to determine whether transient expectations evoke perceptual illusions. However, it remains unclear if these transient expectations have a similar effect on STM. Therefore, Experiment 2 was added. By adding a memory load, the effect of transient expectations on STM can be determined. It is predicted that as memory decays, expectations play an increasingly important role in estimation behavior.
Method
Participants
The same group of participants that executed in Experiment 1, participated in Experiment 2.
Design
Each session, Experiment 2 followed Experiment 1. The total experiment consisted of 4 sessions of 90 minutes each. Each participant executed both Experiment 1 (30 min.) and Experiment 2 (60 min.) in the same order each session. Experiment 2 consisted of 18 blocks of 15 trials, adding up to 270 trials per session and 1080 trials in total.
Stimuli
The stimuli used in Experiment 2 were identical to the stimuli mentioned in Experiment 1.
Procedure
Trials of Experiment 2 consisted of not one, but three consecutive displays presenting moving dots (memory displays). The probability of the presented motion direction and
visibility of the dots on each memory display was identical to those described in Experiment 1 (e.g. see figure 2). Also, all visual stimuli discussed below possessed the same visual
properties as described in Experiment 1.
Prior to the memory displays, every trial began by fixating on a fixation dot for 400 ms. Then, preluding each of the three memory displays, the number ‘I’, ‘II’ or ‘III’ was presented for 500 ms, dependent on which of the three displays was about to be presented (see figure 6). Every time a number was presented, a memory display was presented afterwards for 1500 ms. After observing the last memory display, participants fixated for 500 ms. Then, the target display was displayed (I, II, or III) and remained visible for 500 ms. The target was II on 20% of the trials, less often than target I and III (40% of trials each). Following the target display, the estimation task appeared (see Experiment 1 for a more detailed description). Participants had to estimate the direction of the moving dots of the target display only and had unlimited time in doing so. If no dots were visible, participants were instructed to make a random estimate. After making an estimate, the detection task appeared. Participants were prompted to indicate whether they had seen ‘Dots’(click right) or ‘No dots’(click left). After the detection task, the next trial began. The fixation dot preluding the next trial provided feedback on the detection performance of the previous trial. The fixation dot was either green or red, indicating a correct or incorrect response. Also, block (15 trials) feedback was provided on their detection rate (percentage correct responses in the detection task) and estimation error (mean deviation of the estimate from the true direction of movement).
Analysis
Before analysis, a number of trials were excluded. Firstly, only trials in which the dots were absent were analyzed. Secondly, incongruent trials in which a stimulus was detected, but no estimation was made were excluded from analysis. Thirdly, the first block (15 trials) was excluded. The exclusion criteria were preregistered based on the exclusion criteria employed in Chalk et al. (2010). The exclusion criteria of Experiment 2 were identical to the exclusion criteria of Experiment 1 (detection rate: 80%, mean estimation error: 30˚). The same two participants that were excluded in Experiment 1 did not meet criteria again in Experiment 2 and were therefore excluded from analysis.
Estimation bias on trials with a non-presented, but detected stimulus was determined in the same way as described in Experiment 1. Firstly, on these trials, the mean deviation of estimates from the most frequently presented motion directions was determined. Again, this mean deviation will be compared to the mean deviation of random estimates (58˚) to check for potential estimation bias. More interestingly, the mean deviation from the most probable angles is compared between target I and target III. This way, it would become clear if
estimation bias increases as memory decays. Additionally, the mean deviance of estimations from the most probable angles is compared between Experiment 1 and Experiment 2. All comparisons will be made by using the Paired Samples T-Test in Microsoft Excel.
The exploratory analyses were the same as described in Experiment 1.
15
Figure 6. Experimental set-up of one trial of Experiment 2. Participants were presented with three memory displays containing motion stimuli (either highly visible, barely visible or absent) lasting 1500 ms each. Each memory display was preceded by the number I, II or III, dependent on which display would be presented. After viewing all three memory displays, the target display was revealed. Then, the estimation task followed, in which participants estimated the direction of movement of the target display only. After doing so, the detection task appeared. Participants clicked on the right or left side of the screen to indicate if they had seen ‘dots’ (right) or ‘no dots’ (left) in the target display. The target in this example is I, but could be either of three displays in the actual experiment.
Results
If no stimulus was presented, yet a stimulus was detected, estimations were not completely random. Instead, the mean deviance of estimates (M = 37.8˚, SD = 12.5˚) from the most frequently presented motion directions was smaller than to be expected if estimates were random (58˚), t(14) = -6 , p < .001. More interestingly, we found that when the target was I (M = 32.9˚, SD = 11.4˚), estimations were biased towards the motion directions most probable to occur more than when the target was III (M = 43.9˚, SD = 15.5˚), t(14) = -2.2, p = .003. This suggests that the estimations were increasingly influenced by
expectations as memory decayed (see figure 7).
Figure 7. This image shows the mean deviance of estimates from expected angles if no dots were presented, but dots were detected. This mean deviance is depicted for target I (grey) and target III (black). Note that the mean deviance of target I is significantly smaller than the mean deviance of target III (*). The dashed red line
symbolizes random estimates.
Additionally, a comparison was made between estimation bias of Experiment 1 (M = 39.3˚, SD = 14.8˚) and
estimation bias of Experiment 2 (M = 37.8˚, SD = 12.5˚). Against predictions, the estimates of Experiment 2 were not biased more towards the expected angles than the estimates of
Experiment 1 t(14) = 0.31, p = .92 (see figure 8).
Exploratory tests revealed that a significant difference in estimation bias
16
Target I Target III 0 35 70 0 Target I; 32.85 43.87 0 58 58 58Target III; 58
Experiment 2: Estimation bias of memory illusions; target I vs. target III
Deviance from expected angles Random estimates D ev ia n ce f ro m e xp ec te d a n gl es ( d eg ) Experiment 1 Experiment 2 0 35 70
Experiment 1 v.s. Experiment 2: no difference in estimation bias
Deviance from expected angles
D ev ia n ce o f es ti m ati o n s fr o m e xp ec te d a n gl es ( d eg )
between detected (M = 37.8˚, SD = 12.5˚) and undetected (M = 49.4˚, SD = 19.8˚) stimuli was found, t(14) = -1.9, p = .028. Also, the deviation of estimations on undetected trials was not significantly smaller than the deviation of random estimates (58˚), t(14) = 1.5, p = .215. Additionally, the effect of reaction time on estimation bias found in Experiment 1 was also found in Experiment 2. The difference in bias became apparent whilst comparing the short reaction time group (M = 34.6˚, SD = 36.4˚) to the long reaction time group (M = 40.2˚, SD = 39˚), t(14) = -0.4 , p < .001. These results indicate that short reaction times led to stronger estimation bias towards expected angles.
Main findings
These results confirm that transient expectations biased estimates towards the most frequently presented motion directions when no stimulus is presented, yet a stimulus was detected. Crucially, the estimation bias was stronger if participants held information over time and interference. As memory decayed, STM grew more sensitive to distortion by transient expectations, which led to strongly biased memory illusions. However, unexpectedly, no difference in estimation bias was found between Experiment 1 and Experiment 2.
Discussion
The results from Experiment 2 show that transiently learned expectations can affect STM. As memory deteriorated, memory illusions grew increasingly biased, showcasing the effect transient expectations have on STM. However, no difference in estimation bias was found between Experiment 1 and 2. The results from Experiment 1 show that perception is also affected by transiently learned expectations. Perceptual illusions were strongly biased as a result of transient expectations. Also, if dots were barely visible, transient expectations somewhat biased estimation behavior of participants (although evidence was inconsistent). To summarize, general estimation bias and perceptual/memory illusions did occur as a result of transient expectations influencing STM and perception.
Unpredicted results
Some results were different from what was predicted. For instance, estimations in Experiment 1 and Experiment 2 were equally biased. This could be explained by a methodological difference between the two experiments. In Experiment 1, participants estimated direction of movement of the dots during their presentation. Whereas in
Experiment 2, participants estimated direction of movement after presentation. The method of Experiment 1 led to greater accuracy of estimations (average estimation error: 37˚) in trials
with barely visible dots while being compared to Experiment 2 (average estimation error: 52˚). It is possible, that when no dots were visible, yet dots were detected, estimations in Experiment 2 were also less accurate than in Experiment 1. This could cause estimations of ‘hallucinated dots’ to be less biased towards the most probable directions of motion, since they generally were less accurate. It could explain why an effect of memory on estimation bias could be found between target I and III in Experiment 2, but not between Experiment 1 and 2.
Another unpredicted result is that if the dots were barely visible in Experiment 1, estimation bias towards the most probable angles occurred if the direction of movement was 48˚, but not if it was 16˚. This is in contradiction with the experiment of Chalk et al. (2010). Perhaps this could be explained by the fact 16˚ is still fairly close to both frequently presented motion directions (16˚ away from one, 48˚ away from the other), and 48˚ is only close to one. Supporting this claim, estimates at movement direction 64˚ were strongly negatively biased towards the expected direction of motion (32˚). This direction of motion is 32˚ away from the expected direction of motion, but estimations were still is strongly drawn towards it. Thus, it could be possible estimations at motion direction 16˚ were biased towards the expected motion direction that is 48˚ away, as well as the expected motion direction that is 16˚ away. Also, by including Experiment 2, expectations in Experiment 1 could be even stronger than in the experiment of Chalk et al. (2010). This could explain why estimations in the current study are biased towards both of the expected directions of motion if the direction of motion is 16˚, instead of just one.
Response strategy
Since two motion directions were presented much more often, it could be argued that participants estimated based on a response strategy. For instance, they could have
developed an automaticity to make estimations towards these motion directions, regardless of whether the stimulus was detected or not. If this was the case, when no stimulus was presented, estimations on detected trials should be equally biased as estimations on
undetected trials. If no response strategy is used, estimations on undetected trials should be completely random. We found that in Experiment 1, estimation behavior did not differ
between detected and undetected trials (although the mean deviance does differ, see figure 5). Additionally, estimations on undetected trials were not random, and therefore biased. However, in Experiment 2, the complete opposite was found. Estimations of detected trials were biased more than estimations of undetected trials. Moreover, there was no statistical difference between random estimates and estimates on undetected trials. It seems as if
participants developed a more automatic response strategy in Experiment 1 than in Experiment 2.
A possible explanation could be the time pressure to make an estimation in Experiment 1 (time to make an estimation: 3750 ms), which is absent in Experiment 2 (time to make an estimation: ). Time pressure can lead to the application of more heuristic strategies in performing a task (Verplanken, 1993). It could be argued that time pressured participants used a more ‘practical’ estimation strategy, such as always estimating in the most probable angles, regardless of detection. However, it still remains remarkable that overall, undetected stimuli seem to be slightly biased towards the expected angles. It could be plausible that automatic behavior has a small effect on the estimations of participants. Although it could also be argued that this automatic behavior is simply the result of expectations as well.
Reaction time and confidence
An interesting find from this study is that reaction time on the estimation task influenced estimation bias in Experiment 1 and Experiment 2. This applies to trials with no presented stimulus, where a stimulus was detected. We found that short reaction time led to stronger estimation bias, while long reaction time led to weaker estimation bias. Short reaction time is negatively related to confidence (Geller & Whitman, 1973). This suggests that high
confidence is indirectly related to stronger estimation bias in these experiments. In other words, if participants are more confident they have seen the dots move, they are biased to estimate in the expected angles more than when they are less confident. This is promising, as it would suggest that transiently learned expectations generate actual memory/perceptual illusions instead of response biases. However, reaction time is only an implicit way to
measure confidence. Thus, for further research, a confidence interval should be added to the task to confirm this hypothesis.
How transient expectations influence STM and perception
Overall, our findings largely replicate the results of the research of Chalk et al. (2010). Firstly, the current research confirms that implicitly learning a transient rule leads to estimation bias and perceptual illusions. This discovery is consistent with numerous experiments that investigated the influence of expectations on perception (de Gardelle, Sackur, & Kouider, 2009; Dogge et al., 2019; Rao & Ballard, 1999). Our results also show that transient
expectations can be learned implicitly and automatically by the visual system (Turk-Browne et al., 2005). Additionally, we provide evidence that expectations influence STM, for an increase of bias was found if information was held over time and interference. These results are in concordance with earlier research on memory illusions/false memories (Pinto et al.,
2019; Deese, 1959; Roediger & McDermott, 1995). Uniquely, additional to these findings, we found that memory illusions also appear if expectations were transiently learned, instead of deeply ingrained.
To further understand how expectations influence perception and STM, we revisit the idea of predictive processing. Predictive processing states that perception relies on sensory
information and expectations. The weight of each of their influence on perception is determined by certainty (Friston, 2009). In the current study, the certainty of sensory information was manipulated by lowering the luminance of the dots to near-threshold.
Expectations were manipulated by implicit transient learning of stimulus probability. This way, the weight attributed to sensory information is lowered, causing expectations to have a larger relative influence on perception. This is demonstrated by participants hallucinating the presence of moving stimuli, while nothing is actually presented to them. They rely so heavily on their transient expectations (and less on sensory information), that it leads to
hallucinations. More interestingly, these illusions increase if information is held in STM. This could be explained in the context of predictive processing as well. In Experiment 2, the certainty of sensory information weakened as memory decayed over time and interference, causing the weight of expectations on perception to increase. In other words, as participants are more unsure of what they have seen as time goes by, they rely more on their
expectations and less on sensory information.
But how does this interaction of sensory information and expectations work exactly? It is suggested that observers used a Bayesian strategy. As in predictive processing, Bayesian strategy also states that our reliance on expectations increases as sensory information becomes more uncertain. In this strategy, a learned prior (expectation) is combined with sensory information in a probabilistically optimal way to form perception (Knill & Pouget, 2004; Körding & Wolpert, 2004; Tassinari et al., 2006). Although our data was not fitted against a Bayesian model, the experimenters of Chalk et al. (2010) found that participants did use a Bayes optimal strategy to estimate direction of movement. Since the experimental procedure of the current study is very similar to the earlier mentioned research, it is plausible that participants adapted a Bayesian strategy as well. This can apply to Experiment 1, but also to Experiment 2. The adaptation of a Bayesian strategy in a STM task (like Experiment 2) could be the focus of further research.
Implications of memory illusions
Implications of the current study are mainly focused on the effect of transient expectations on STM. Up until now, most research on illusory memories has been conducted using semantic
information (Deese, 1959; Roediger & McDermott, 1995) or letters (Pinto et al. 2019). These types of information are deeply ingrained over a lifetime. Uniquely, our study provided evidence that STM is also susceptible to transient expectations. Additionally, the current research provided evidence that expectations of general visual features like movement can affect STM as well as letters or words. Therefore, it could be argued that memory illusions consistently occur in visual cognitive processing. This would mean that from the moment of perception to STM, expectations increasingly change our view of a witnessed event. Memories get more and more distorted as sensory information fades, even if the event happened just a few seconds ago. This would imply that STM is not at all as reliable as we think it is. In most situations, this process is adaptive as it allows us to make the most probable guess of what actually occurred. However, in eye-witness reports, it can have drastic consequences. The statements of eye-witnesses could be especially biased if
expectations are strong and the time between the witnessed event and report is prolonged. A limitation of the current experiments is that they are not necessarily similar to a natural environment. Investigating the effect of transient expectations on STM in a more natural setting could be an interesting suggestion for further research.
In conclusion, our experiments provided evidence that STM and perception are susceptible to transient expectations. Time and time again, perception and memory are proven to rely on more than what is actually there. Expectations play an important role in perception and memory, changing not only what we see, but also what we remember.
(Chalk, Seitz, & Seriès, 2010)
References
Barkowitz, P., & Brigham, J. C. (1982). Recognition of Faces: Own‐Race Bias, Incentive, and Time Delay. Journal of Applied Social Psychology.
https://doi.org/10.1111/j.1559-1816.1982.tb00863.x
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision. https://doi.org/10.1163/156856897X00357
Brainerd, C. J., & Reyna, V. F. (2002). Fuzzy-trace theory and false memory. Current
Directions in Psychological Science, 11(5), 164–169.
https://doi.org/10.1111/1467-8721.00192
Burke, R. R., & Srull, T. K. (1988). Competitive Interference and Consumer Memory for Advertising. Journal of Consumer Research. https://doi.org/10.1086/209145 Chalk, M., Seitz, A. R., & Seriès, P. (2010). Rapidly learned stimulus expectations alter
perception of motion. Journal of Vision. https://doi.org/10.1167/10.8.2
de Gardelle, V., Sackur, J., & Kouider, S. (2009). Perceptual illusions in brief visual
presentations. Consciousness and Cognition, 18(3), 569–577. https://doi.org/10.1016/j.concog.2009.03.002
Deese, J. (1959). On the prediction of occurrence of particular verbal intrusions in immediate recall. Journal of Experimental Psychology. https://doi.org/10.1037/h0046671
Dogge, M., Custers, R., Gayet, S., Hoijtink, H., & Aarts, H. (2019). Perception of action-outcomes is shaped by life-long and contextual expectations. Scientific Reports, 9(1), 1– 9. https://doi.org/10.1038/s41598-019-41090-8
Friston, K. (2009). The free-energy principle: a rough guide to the brain? Trends in Cognitive
Sciences. https://doi.org/10.1016/j.tics.2009.04.005
Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews
Neuroscience. https://doi.org/10.1038/nrn2787
Gallo, D. A. (2010). False memories and fantastic beliefs: 15 years of the DRM illusion.
Memory and Cognition, 38(7), 833–848. https://doi.org/10.3758/MC.38.7.833
Gekas, N., Chalk, M., Seitz, A. R., & Seriès, P. (2013). Complexity and specificity of experimentally-induced expectations in motion perception. Journal of Vision. https://doi.org/10.1167/13.4.8
Geller, E. S., & Whitman, C. P. (1973). Confidence ill stimulus predictions and choice reaction time. Memory & Cognition. https://doi.org/10.3758/BF03198121
Kim, K., Park, K. K., & Lee, J. H. (2014). The influence of arousal and expectation on eyewitness memory in a virtual environment. Cyberpsychology, Behavior, and Social
Networking. https://doi.org/10.1089/cyber.2013.0638
Kleiner, M., Brainard, D. H., Pelli, D. G., Broussard, C., Wolf, T., & Niehorster, D. (2007). What’s new in Psychtoolbox-3? Perception. https://doi.org/10.1068/v070821
Knill, D. C., & Pouget, A. (2004). The Bayesian brain: The role of uncertainty in neural coding and computation. Trends in Neurosciences. https://doi.org/10.1016/j.tins.2004.10.007 Kok, P., Brouwer, G. J., van Gerven, M. A. J., & de Lange, F. P. (2013). Prior expectations
bias sensory representations in visual cortex. Journal of Neuroscience. https://doi.org/10.1523/JNEUROSCI.0742-13.2013
Körding, K. P., & Wolpert, D. M. (2004). Bayesian integration in sensorimotor learning.
Nature. https://doi.org/10.1038/nature02169
Lange, K. (2009). Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and Cognition.
https://doi.org/10.1016/j.bandc.2008.06.004
Loftus, E. F. (2019). Eyewitness testimony. Applied Cognitive Psychology. https://doi.org/10.1002/acp.3542
Pinto, Y., Seth, A., & Otten, M. (2019). Seeing ɔ, remembering c: Illusions in short-term memory
Pinto, Y., Sligte, I. G., Shapiro, K. L., & Lamme, V. A. F. (2013). Fragile visual short-term memory is an object-based and location-specific store. Psychonomic Bulletin and Review. https://doi.org/10.3758/s13423-013-0393-4
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision. https://doi.org/10.1163/156856897X00366 Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: A functional
interpretation of some extra-classical receptive-field effects. Nature Neuroscience. https://doi.org/10.1038/4580
Ricker, T. J., Vergauwe, E., & Cowan, N. (2016). Decay theory of immediate memory: From Brown (1958) to today (2014). Quarterly Journal of Experimental Psychology.
https://doi.org/10.1080/17470218.2014.914546
Roediger, H. L., & McDermott, K. B. (1995). Creating False Memories: Remembering Words Not Presented in Lists. Journal of Experimental Psychology: Learning, Memory, and
Cognition. https://doi.org/10.1037/0278-7393.21.4.803
Schacter, D. L., Guerin, S. A., & St. Jacques, P. L. (2011). Memory distortion: An adaptive perspective. Trends in Cognitive Sciences. https://doi.org/10.1016/j.tics.2011.08.004 Seriès, P., & Seitz, A. R. (2013). Learning what to expect (in visual perception). Frontiers in
Human Neuroscience. https://doi.org/10.3389/fnhum.2013.00668
Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. (2008). Are there multiple visual short-term memory stores? PLoS ONE. https://doi.org/10.1371/journal.pone.0001699
Sterzer, P., Frith, C., & Petrovic, P. (2008). Believing is seeing: expectations alter visual awareness. Current Biology. https://doi.org/10.1016/j.cub.2008.06.021
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M. M., & Egner, T. (2008). Neural repetition suppression reflects fulfilled perceptual expectations. Nature Neuroscience. https://doi.org/10.1038/nn.2163
Tassinari, H., Hudson, T. E., & Landy, M. S. (2006). Combining priors and noisy visual cues in a rapid pointing task. Journal of Neuroscience.
https://doi.org/10.1523/JNEUROSCI.2779-06.2006
Todorovic, A., van Ede, F., Maris, E., & de Lange, F. P. (2011). Prior expectation mediates neural adaptation to repeated sounds in the auditory cortex: An MEG study. Journal of
Neuroscience. https://doi.org/10.1523/JNEUROSCI.1425-11.2011
Tuckey, M. R., & Brewer, N. (2003). The Influence of Schemas, Stimulus Ambiguity, and
Interview Schedule on Eyewitness Memory Over Time. Journal of Experimental
Psychology: Applied. https://doi.org/10.1037/1076-898X.9.2.101
Turk-Browne, N. B., Jungé, J. A., & Scholl, B. J. (2005). The automaticity of visual statistical learning. Journal of Experimental Psychology: General. https://doi.org/10.1037/0096-3445.134.4.552
Verplanken, B. (1993). Need for cognition and external information search: Responses to time pressure during decision-making. Journal of Research in Personality.
https://doi.org/10.1006/jrpe.1993.1017
Zhao, J., Al-Aidroos, N., & Turk-Browne, N. B. (2013). Attention Is Spontaneously Biased Toward Regularities. Psychological Science.
