Validating rapid iconic memory assessment
Abstract
Iconic memory is the most detailed, yet also the fastest decaying part of the visual memory. Its rate of decay naturally becomes faster when people age, but this process happens a lot sooner for cognitively declining individuals. As such, iconic memory performance happens to be a strong predictor for cognitive decline and also Alzheimer’s disease. Since Alzheimer’s can be prevented, but not reverted, this study could make a vast improvement for those who will later develop the disease.
Iconic memory is traditionally assessed through the partial report task. However, this task takes over an hour to complete, making it unfit for self-measurement and for generating reference groups. Furthermore, these tests aren’t available to the public.
Here we replicate shortened method of measuring iconic memory in the browser. The method can be shortened due to its use of Bayesian adaption, which enhances the rate at which information is gained, resulting in less trials being required. The adaptive method showed to produce equal results as the longer non-adaptive method. However, both methods were not precise enough to accurately assess individuals. Conclusively, we found several different areas for optimization, which will be implemented in a future version of this experiment.
Kas van 't Veer
Programme: MSc Brain & Cognitive Sciences
Research project no.: 1
Academic year: 2016-2017
Time span: February 6
th- June 30
thSupervisor: dr. Ilja Sligte
Co-assessor: prof. Jaap Murre
Credits: 26
Introduction
Shortly after a visual stimulus disappears, we still have a detailed impression of what it looked like. This is because our iconic memory holds detailed snapshots of the visual images we have just seen (Coltheart, 1980). These snapshots last for less than a second, but their enormous capacity allows us to act upon intricate details. Iconic memory is used for many cognitive abilities, such as detecting rapid changes in images, viewing videos as a fluent motion and performing well in sports (Sperling, 1960; Dick, 1974; Becker, Pashler, & Anstis, 2000; Adam & Wilberg, 1992). Having been studied for well over half a century, iconic memory is a widely known part of human cognition.
Why should we study iconic memory?
In recent years, several studies suggested that iconic memory may help in predicting
cognitive decline. This idea started when a well-employed 58-year old subject was found to have a substantial lack of iconic memory, while not experiencing any apparent problems (Lu, Neuse, Madigan & Dosher, 2004). Nevertheless, this subject was forced to quit his job 2 years later due to recently developed memory problems and was later diagnosed with
Alzheimer's disease (AD). This peculiar case instigated an experimental study that found that the duration of iconic memory was not only significantly reduced in Alzheimer's patients, but also in people that were at risk of developing the disease later on. It is also important to consider that present medication for Alzheimer's disease can merely slow the progression of the disease, but not reverse its course. If Alzheimer's disease can be accurately predicted earlier on, preventive treatment can be started during an earlier stage. This would allow patients to retain more of their cognitive ability during the rest of their lives. Therefore, this study aims to turn a traditional iconic memory experiment into a public online assessment application to reduce the impact of this life-altering disease.
Traditional measuring of iconic memory
Iconic memory is typically measured with the partial report procedure (Sperling, 1960). In this classic paradigm, participants are asked to remember a briefly appearing (10-250 ms) memory array, consisting of three rows of letters. Other varieties with different stimuli, a circle of letters for instance, have been used as well (Averbach & Coriell, 1961; Sperling, 1967). Traditionally, two conditions are compared: whole report, in which the entire array has to be reported and partial report, in which only a single row has to be reported. The row to report was not disclosed until after the memory array disappears (Figure 1).
Figure 1. Example of one trial in a traditional partial report procedure. Reproduced from "qPR: An adaptive partial-report procedure based on Bayesian inference" by J. Baek, L.A. Lesmes, and Z.L. Lu, Journal of Vision, 16(10), 25-25.
During both conditions, the screen turns blank after the stimulus has been shown for the desired amount of time. In the partial report condition, subjects are informed about which row to report by a visual or auditory cue, but only after a time delay. It was found that participants were able to report a much higher amount of letters (76% vs 38%) during partial report. This would not have been possible without a very brief, high capacity type of visual memory, thus showing evidence of its existence. In the whole report condition, the information retrieval process takes enough time for the items in the iconic memory to decay before they can be retrieved and reported. In the partial report condition however, the amount of letters to report is much smaller, which drastically reduces this effect. However, during partial report, the iconic memory of the subject decays during the delay (at which point the screen is blank). As such, by varying the delay in a follow-up experiment, the amount of information that remains in the iconic memory was estimated at different points in time after the stimulus. By taking many measurements over a wide span of these delays, a curve can be fitted that represents the rate of decay (Figure 2).
Figure 2. Example of the iconic memory decay curves three different groups of participants. The amount of correct responses is portrayed using the sensitivity index, d-prime. Reproduced from "qPR: An adaptive partial-report procedure based on Bayesian inference" by J. Baek, L.A. Lesmes, and Z.L. Lu, Journal of Vision, 16(10), 25-25.
The curve is exponential in nature and clear effects of aging and cognitive decline are visible between different reference groups. The partial report procedure makes it possible to clearly map out these differences in iconic memory decay, despite it being very short lived.
Disadvantages of the partial report procedure
However, there are a few problems with the partial report (PR) paradigm if it were to be used for the prediction of Alzheimer's disease. First of all, no accurate reference data is currently available for iconic memory performance for all age groups, education levels and cognitive health. It is impossible to draw a safe conclusion about the state of a participant's iconic memory without being able to refer to means or safe ranges for each type of group. Secondly, there are some motivational issues concerning iconic memory measurement. In order to get a precise measurement of the participant's iconic memory, the procedure requires many trials,
making it last 45-60 minutes. This long duration not only makes it harder to generate reference data, but may also complicate assessment. Boredom or loss of focus is a common problem during long repetitive experiments. This may severely compromise the accuracy of the results or even cause dropouts, especially to people that aren't used to long tedious experiments. Furthermore, there are not yet any tools available online with which people can accurately assess their own iconic memory. The motivational issues may pose an even larger problem in an online environment, where participants may be even less focused. As such, the task needs to be shortened and be more appealing to be used effectively as a public online tool. Conclusively, multiple interrelated problems hinder iconic memory assessment from being used for the prediction of Alzheimer's. A browser-based test that is shorter and more fun needs to be developed and tested in a lab to generate reference groups.
qPR as a better alternative to partial report
One example of an improved partial report method that measures more quickly, called quick partial report (qPR), was published last year (Baek, Lesmes & Lu 2016). Contrary to the traditional PR procedure, qPR uses a circular array of letters of which a single letter has to be reported (Figure 3).
Figure 3. Example of one trial in the qPR procedure. Reproduced from "qPR: An adaptive partial-report procedure based on Bayesian inference" by J. Baek, L.A. Lesmes, and Z.L. Lu, Journal of Vision, 16(10), 25-25.
The letter to be reported is indicated by an arrow cue, which is shown after a variable delay, starting as soon as the stimulus appears. This delay between the stimulus onset and the cue is defined as stimulus onset asynchrony (SOA) and can be either 0, 30, 60, 140, 300, 650, 1400 or 3000 ms. qPR uses an adaptive algorithm, based on Bayesian inference, to find the optimal SOA for each trial. The optimal SOA is found by re-estimating the iconic memory decay curve after each trial and subsequently choosing the most informative SOA. The
mathematical implementation will be discussed shortly but can be found in more detail in the the original publication or in the source code in the supplementary materials (Baek et al., 2016). The iconic memory decay curve which is updated after each trial consists of three parameters:
a0, the probability correct after all iconic memory is decayed, forms the asymptote of the curve
τ, the parameter indicating the magnitude of the decay, equal to the amount of seconds it takes for the percentage correct to reach a0 + (a1 – a0) · 0.37
The resulting curve estimates the probability of a correct response as a function of the SOA, where x is the SOA in seconds:
pc(x) = a0 + (a1 – a0)e-x/τ
The role of the different parameters can be easily displayed when the curve is plotted (Figure 4):
Figure 4. Example of an iconic memory decay curve of a healthy individual. Reproduced from "qPR: An adaptive partial-report procedure based on Bayesian inference" by J. Baek, L.A. Lesmes, and Z.L. Lu, Journal of Vision, 16(10), 25-25. During the experiment, the estimated decay curve for the participant is updated after every trial. To select the stimulus for the next trial, the qPR determines cue delay x that would maximize the expected information gain about all of the parameters of the iconic memory decay function (Baek et al., 2016). This makes qPR need less trials for estimating a decay curve of equal accuracy compared to traditional PR, which picks every SOA equally, regardless of the expected information gain.
How qPR may solve problems
qPR makes a good candidate for a public test, since it is much shorter and therefore less boring. In the original experiment, qPR was found to be 3-5 times as efficient as an
equivalent non-adaptive test, making it need less than 10 minutes to estimate a decay function with reasonable accuracy and precision (Baek et al., 2016). This will accommodate users of a public test, since they will probably perform the test during an internet browsing session and/ or on mobile devices, during which their attention span may be limited. Additionally, a shorter task that's easier to complete will also simplify the process of creating reference data for different age groups. However, in order to actually use qPR for public online
measurements, it first needs further validation. In the original experiment, the method was extensively validated through simulations, but merely three participants were used for the behavioural part of the study. To predict Alzheimer’s in an individual, the procedure first
needs to show that it’s able to estimate the individual’s decay curve correctly. This study will therefore address this issue by attempting to further validate qPR through a behavioural experiment.
The experiment and its hypotheses
The original qPR experiment will be rewritten as a web page application. This will be compared against a visually identical web page application that’s longer and non-adaptive, called MCS (Method of Constant Stimuli), using a within-subjects design.
The hypotheses are as follows:
Both conditions are expected to perform equally on estimating the a0, a1 and τ of each participant's' curve.
Both conditions are expected to perform differently on estimating the a0, a1 and τ of each participant’s curve when the MCS condition is cut to the same length as the qPR. Participant’s individual a0, a1 and τ are expected to strongly correlate with one another
between conditions.
Sequence effects are expected, as such, the score in both conditions is expected to be affected by the sequence in which they are performed
Methods
First, our improved browser-version of the traditional qPR algorithm was tested in a lab environment.
Sample group
40 students (10 male, 30 female) between the ages of 18 and 25 (mean 23.375) participated in the experiment. All subjects were not dyslectic and had normal or corrected-to-normal vision. Participation was part of the students' study course or rewarded by financial compensation. The experiment was approved by the local ethics committee and all subjects gave their written informed consent.
Technical implementation
The experiment is largely the same as the original experiment (Baek et al., 2016), but several modifications were made to make it fit better as an online test in the future. For each trial a circle with 8 equally spaced letters was used instead of 10 letters, in order to avoid confusion about the location of the letter. The possible letters were changed to C, D, T, N, R, S, V and Z, as to reduce ambiguity between letters, for example O and C. The order of the letters was randomized for each trial. Participants were seated 60 cm away from the screen, while the diameter of the circle was 9.19 visual degrees. Participants answered by pressing the appropriate letter key on the keyboard. The font used for the letters was Arial, 84pt. The stimulus time was increased from 20 to 250 ms, to make the test more suitable for future elderly users and to reduce the effect of different screen types of future online users. The delay between stimulus and cue was defined as interstimulus interval (ISI) instead of SOA. As such, the delay was measured starting after the stimulus disappeared, instead of starting at the onset of the stimulus. The possible ISI's were changed to 0, 50, 100, 200, 350, 600, 1000 and 1500 ms, to create a more even logarithmic distribution and also to fit the standard 60Hz refresh rate used by most devices for future online testing. The drastic reduction of the 3000
ms upper bound of the ISI range of the original study was due to 1500 ms of delay being enough for all information in the iconic memory to decay (Figure 3 and 4; Baek et al., 2016). After this delay, the cue appeared in the form of an arrow that was 2.82 visual degrees in length. Its location was chosen randomly for each trial. A fixation cross was visible throughout the entire experiment. Subjects were informed about the correctness of each answer by the fixation cross turning red or green (for 1000 ms) after each response. After this period, the fixation cross disappeared for a duration of 500 ms. Afterwards, the fixation cross reappeared in black again and the next trial started after 1000 ms.
Condition and trial blocking
The experiment consisted of two conditions: adaptive (qPR: quick Partial Report) and non-adaptive (MCS: Method of Constant Stimuli). Both conditions were performed by every subject in counterbalanced order. 192 trials were performed during the adaptive condition and 768 trials during the non-adaptive condition. Subjects were allowed to take a short break after every 48 trials. The non-adaptive test ran through a randomized permutation of all possible ISI's every 8 trials. During the adaptive test, the ISI with the highest expected information gain was chosen through the original qPR algorithm (Baek et al., 2016). Our experiment implemented one change to the original algorithm, as the entropy function was changed to the derivative of its original. This was done because this change had led to more accurate results during previous simulations. All other methods with regards to the Bayesian adaptation algorithm were kept the same as in the original publication. The implementation can be found in its entirety in the supplementary materials.
Hardware and software specifications
As the experiment ran entirely in the browser (Google Chrome), it was programmed using a combination of HTML, CSS, PHP, JavaScript and MySQL. The lab computer ran a local server that contained the software, built on XAMPP v5. As such, no remote connections needed to be made, as the experiment could easily be accessed through the localhost address. The operating system used on the computer was a stripped-down version of Windows 7. The display that was used was an Asus VG236HE (120Hz, 2 ms) and the keyboard was a
Gigabyte Aiva Osmium. Since 120Hz is the exact double of the intended refresh rate of 60Hz, all the timings were still in perfect alignment with the refresh rate. All random choices throughout the experiment were made by JavaScript's built-in Math.random() function. Due to the computational complexity of the adaptive algorithm in JavaScript, there was a slight delay (<300ms) between the response and the onset of the feedback during qPR.
Warranting accurate timings
Timing of the stimuli was controlled for using JavaScript's built-in requestAnimationFrame() function (Window.requestAnimationFrame() - Web API Interfaces | MDN., 2010). This function waits until the next display refresh and subsequently reports the amount of time spent waiting for that very refresh. By combining this with recording the time at which the function was called, this allows tight control and measurement of the duration of stimuli and also allows measuring the frame rate of the browser window (often equal to the display's refresh rate). For instance, when the system needs to wait for 50 ms at a refresh rate of 120Hz, we will keep calling the window.requestAnimationFrame() function, each time evaluating the amount of time spent waiting for the next refresh. This would typically be 6 refreshes at 120Hz, though the browser may not always deliver a frame in time for a refresh and cause a so called "dropped frame". But in our implementation, even in the rare event in
which a frame gets dropped, the timing will still be perfect most of the time, since the code double-checks whether the last frame update actually happened within a single refresh. Should it detect a dropped frame, it will add one additional refresh to the counter, resulting in the system correcting itself. The only situation in which the timing will be off is when a dropped frame happens to occur right at the last refresh upon which is to be waited. Even in this case, the incident will be reported in the data, so that it may be excluded from the analysis. This system was created to aid future online testing, because dropped frames are expected to occur more often in (slower) mobile devices. In any case, the effect of dropped frames is kept to an absolute minimum using this approach.
Results
Performance of qPR in estimating decay curves of individuals
The goal of this experiment was to validate the ability of the qPR procedure to accurately estimate the decay curve. In order to evaluate this, each participant's decay curve was estimated at the end of every condition and the equality of each parameter between both conditions was determined using a paired samples t-test (Figure 5).
Figure 5. Shows the distribution, mean and the means' standard error for all of the participant's parameters for both conditions, together with paired samples t-test results for each parameter. Every distribution includes all 40 subjects due to the within-subjects design. The possible range of each parameter was set at 0 to 1.
The paired samples ttests showed that neither a0 (t(39) = 1.838, p = 0.074), a1 (t(39) = -0.631, p = 0.532) or τ (t(39) = -1.393, p = 0.171) was estimated differently by the different conditions. However, the plots show that the distributions of the τ-parameter are noticeably wide for both conditions, covering nearly the entire possible range. This issue was further
investigated by calculating the correlations between both conditions for each parameter (Figure 6).
Figure 6. Correlations between both conditions for each parameter. Calculated over all conditions of all 40 subjects.
The conditions show a striking lack of correlation on the τ-parameter (r = -0.024, p = .881), while the conditions did correlate on a0 (r = 0.852, p < 0.001) and a1 (r = 0.171, p < 0.001). The goal of the experiment was to be able to estimate the decay rate for each individual. Unfortunately, τ is the parameter solely responsible for the rate of decay. Plotting individual decay curves for some participants gives insight as to why these problems happen. Some individual curves show a lack of exponentiality and/or a very limited range between a0 and a1, while the qPR condition shows oversampling at the extremes of ISI range for many subjects (Figure 7).
Figure 7. Iconic memory decay curve of three individuals in the conditions given. Subject 17 in the MCS conditions shows a notable lack of range on the y-axis. Subject 38 in the qPR condition shows a linear decay curve. Subject 29 is an example of a correct fit. Both examples of the qPR condition show oversampling at the extremes of the ISI range.
Displaying the occurrence of each ISI in the qPR condition a histogram confirms that oversampling at the extremes is a problem across all participants (Figure 8).
Figure 8. Total occurrence of each ISI in the qPR condition across all participants. The extremes of the ISI range show problems of oversampling the outermost ISI's.
Performance of qPR in estimating the decay curve of the entire population (exploratory)
Further tests were performed to see whether the task was able to estimate iconic memory decay on a population level instead. Fitting decay curves for the entire population for both conditions shows that both methods are well capable of finding a good curve on a population level (Figure 9).
Figure 9. Population curves for each condition. Each participant's average performance for every ISI has also been plotted. The diameter of the dots indicates the amount of corresponding trials. Each sample is linearly weighed depending on its amount of corresponding trials.
To statistically evaluate equality of the parameters between the population curves, a Monte Carlo analysis was used with 4 million samples (Figure 10).
Figure 10. Monte Carlo simulation plots for each parameter on the entire population. Each sample flips the conditions around for a single random participant in the population. After each sample, the parameter difference between the conditions is recalculated and added to the plot. The blue line indicates the difference between both conditions at the start of the simulation.
The Monte Carlo analysis shows that both conditions return equal estimations of a0 (p = 0.313), a1 (p = 0.217) and τ (p = 0.357) for an α of 0.025 (two-tailed).
Is qPR actually better?
In order to evaluate whether the qPR actually needs less trials for the same result, the results of the MCS condition were cut to the same length of the qPR (192 trials) and compared with three paired samples t-tests once again (Figure 11).
Figure 11. Shows the same analysis as in figure 5, except that the MCS has been truncated to the same length as the qPR (192 trials).
There was again no difference in parameters between the qPR and the MCS for a0 (t(39) = -0.680, p = 0.500), a1 (t(39) = -1.081, p = 0.286) and τ (t(39) = 0.752, p = 0.456). On average the p-values have moved further above the threshold of significance, suggesting that the truncated results of the MCS match the results of the qPR better. In contrast, a slightly weaker correlation profile was found than before, though still similar; a0 (r = 0.358, p =
0.023) and a1 (0.751, p < 0.001) show significant correlations, while τ does not correlate (r = -0.016, p = 0.921). Another Monte Carlo analysis again showed no differences on a0 (p = 0.448), a1 (p = 0.078) or τ (p = 0.390) on a population level for an α of 0.025 (two-tailed). Plots for these tests (using the truncated MCS) can be found in the supplementary material. To further investigate the performance of both algorithms, each participant's decay curve was estimated at each point in time for the first 192 trials of both conditions in a progression plot (Figure 12).
Figure 12. Progression of the parameters of the individual decay curves for some participants. Subject numbers are noted in the upper-right corner for each graph. Plots for the entire population of the experiment are available in the supplementary material.
The progression plots show that the qPR algorithm does not converge any faster than the MCS. It also shows the lack of range between a0 and a1 problems and a large instability of the τ-parameter, explaining the lack of correlation that was found previously. It further shows that the parameters sometimes converge to a different level in each condition, indicating the possibility of a sequence effect.
Does the order of conditions influence the score?
The progression plots suggested the possibility of a sequence effect due to the same
parameters converging to different values between conditions (Figure 12). To investigate this issue, each of the estimated parameters of each participant's individual curve in the truncated MCS condition was subtracted from the same parameter in the qPR condition and compared using an independent samples t-test (Figure 13).
Figure 13. The value of each subject’s estimated parameter in the truncated MCS condition was subtracted from its value in the qPR condition. The plot shows the mean, its standard error, the distribution and the p-value of the independent samples t-test. Every distribution contains 20 out of 40 subjects, due to the condition order being a between-subjects variable. Violin plots of the distributions of actual parameters split on condition and condition order are available in the supplementary materials.
a0 showed a difference on basis of the order of conditions (t(39) = -2.192, p = 0.035), with subjects performing better when the condition was performed last. No difference was found for a1 (t(39) = -1.002, p = 0.323) and τ (t(39) = 0.124, p = 0.902), though the distributions suggest that a1 may have shown a difference as well when a larger sample size would have been used. The lack of difference on the τ-parameter was expected due to the random nature of its estimation in this experiment.
All scripts used for analyses in this section can be found in the supplementary materials.
Discussion
Summary of the results
The goal of the experiment was to be able to estimate the iconic memory decay curve faster with the qPR procedure. Both algorithms succeeded in this when estimating the decay curve for the entire population. Unfortunately, our version of the qPR procedure failed to show faster or more accurate estimation of the decay rate of individuals, due to oversampling the smallest and largest ISI. Therefore, it would not be able to accurately find the decay curve of an individual in a short amount of time in its current state.
The control condition also showed an inability to estimate the iconic memory decay curve of individuals. This overarching problem seems to have been caused by the a0 and a1 being too close to each other, which in turn caused a general inability to estimate the τ-parameter in both of the conditions. The lack of range between a0 and a1 caused many decay curves to be estimated as linear curves, resulting in an unrealistic τ-parameter such as 0 or 1. This effect did not happen on a population level, at which these aberrations were averaged out and proper decay curves were able to be fit for each condition.
Lastly, a0 was significantly affected by the sequence in which the conditions were performed. a1 may also have been affected, but this was not clear due to the low number of subjects in both sequences. As a result, the decay curves estimated by both algorithms are inherently different for the same subject, complicating statistical comparison between the two.
The inability to estimate curves of individuals has been caused by multiple problems with the implementation. These problems and their hypothesized improvements will be discussed in separate subsections of this paragraph.
Improper difficulty of the task
One of the components on which experiment has to be improved, is the range of difficulty. The task was both too easy and too hard in different areas. Many subjects had their a0 estimated at values of 0.5 or higher, which is much higher than the experiment we attempted to replicate (Baek et al., 2016). Furthermore, many subjects had their a1 estimated at values lower than 0.75, which is lower than the original experiment. Since iconic memory decay is defined as a1 converging to a0, our experiment left little range for the decay to happen in. Many participants had decay curves spanning only 0.1-0.2 on the entire proportion correct axis of range 1 (Figure 7, leftmost plot). With such a small vertical range, even low amounts of noise in the data will mask the exponential nature of the decay curve, making the best possible fit linear instead.
When the curve has a linear shape, the best fit for the τ-parameter will either be 1 (the most shallow slope possible) or close to 0 (an almost instantly decaying curve). Surprisingly, a τ close to 0 also tends to fit a linear decay curve well. In this case, the estimated curve makes a very low, but also very steep drop from the performance measured at 0ms until the
performance at 50-100ms, after which it will continue linearly. This explains why τ may switch back and forth between 1 and values close to 0 for some of the participants.
Unfortunately, the steepness of the decay curve is the most discerning attribute of cognitively declining people (Figure 2). As such, τ is the most valuable parameter to measure and any disturbance in it will render the application useless for its goal of predicting cognitive decline.
The narrow range of the decay curve raises some concern with regards to the experimental setup. Since the baseline performance at a0 is considered to be originating only from working memory, while any performance above a1 is considered to be originating from the iconic memory, we must conclude that very little iconic memory has been measured at all. To improve upon this issue in the future, we could start introducing negative ISI's or switch to using SOA's. This will result in the cue being shown slightly before the stimulus disappears for some trials. Consequently, less information about the stimulus in the iconic memory will have been decayed at this point, therefore raising a1. Additionally, we could increase the amount of letters in the circle. This will reduce the chance of correctly guessing the letter and also reduce the chance that the letter to be reported at 1500 ms will be present in the working memory, both lowering a0. These changes will increase the range between a0 and a1 by increasing the contribution of the iconic memory to the task. As a result, the τ-parameter will be estimated more accurately, in turn making the task better at distinguishing people with cognitive decline.
Imbalanced sampling of the ISI's
the parameters together. It treats all the parameters equally in this regard. Since a1 (starting performance) is entirely captured by the shortest ISI and a0 (baseline performance) is entirely captured by the longest ISI, the longest and shortest ISI contain a high amount of
information. In contrast, the information about the τ-parameter is divided among all of the ISI's. This causes the longest and shortest ISI to be much more attractive for the qPR procedure to choose, since these contain all of the information about one parameter (a0 of a1), instead of the other ISI's, which contain just a small amount of information about τ. This makes qPR tend to neglect τ, the most important parameter for our goal of predicting
cognitive decline.
Future research using an adaptive procedure can be improved in this regard. The simplest adjustment would be to keep using the qPR procedure, but apply a weighing factor to the parameters. For instance, the information about τ may be weighed twice as strong as information about a0 or a1. This would reduce the degree to which qPR will neglect the τ parameter. Another option would be to determine a0 and a1 in the beginning of the experiment, after which they will no longer be able to change. Then, the procedure could focus entirely on τ, eliminating the risk of neglecting it in favour of a0 and a1. Whichever method will be best is up for debate. The effect of increasing the range of the difficulty of the task has to be evaluated first before deciding how to adapt the algorithm.
Implications of the sequence effects
Since the order in which the conditions were performed influenced the estimated decay curve, comparing subjects between conditions becomes more complicated. When comparing on a population level, effects of condition sequence are easily averaged out, but the goal of our experiment was to estimate the decay curves of individuals. Some individuals showed increasing performance over time, likely due to training, while others suffered a decrease of performance over time, likely due to boredom and loss of focus. Because the effect of order is not unidirectional, an individual's score will unpredictably differ between conditions. In order to reduce this source of noise, a design with alternating, blocked conditions may be
implemented. This will equalize the order effects among both conditions, whether the effects are positive or negative. Of course, sequence effects are common to a within-subjects design and an obvious solution would be to use a between-subjects design. Unfortunately, our paradigm is about evaluating two methods in their accuracy of assessing individuals. This makes a within-subjects design nearly mandatory, since without data of the same individual using a different method, these comparisons can't be made. A blocked within-subjects design is therefore the most promising option
Final thoughts and future improvements
In its current state, our implementation of the qPR procedure is not fit for accurately measuring the decay rate of an individual's iconic memory. It is therefore also not fit to predict cognitive decline. Many problems have been discussed that have degraded the results. Nevertheless, all of the encountered problems with the implementation have granted us great insight as to how the algorithm can be improved. For this reason, the experiment shall be repeated in the future after being improved in all the areas that have shown to cause problems in its current state. Should the new version prove to work well in a lab environment, it will be made available to the public in order to start generating reference groups. When this public version of the test has been completed enough times to build reference groups, another study with cognitively impaired patients has to be performed in order to see whether it is capable of
differentiating between them and healthy participants of the same age. If we can manage to complete all of these requirements, the future version of the test will indeed help in early recognition of cognitive decline after all and therefore improve the lives of those affected by starting preventive treatment at an earlier stage.
References
Adam, J. J., & Wilberg, R. B. (1992). Individual differences in visual information processing rate and the prediction of performance differences in team sports: A preliminary investigation. Journal of Sports Sciences, 10(3), 261-273.
Averbach, E., & Coriell, A. S. (1961). Short-term memory in vision. Bell Labs
Technical Journal, 40(1), 309-328.
Baek, J., Lesmes, L. A., & Lu, Z. L. (2016). qPR: An adaptive partial-report procedure based on Bayesian inference. Journal of Vision, 16(10), 25-25.
Becker, M. W., Pashler, H., & Anstis, S. M. (2000). The role of iconic memory in change-detection tasks. Perception, 29(3), 273-286.
Coltheart, M. (1980). Iconic memory and visible persistence. Perception &
psychophysics, 27(3), 183-228.
Dick, A. O. (1974). Iconic memory and its relation to perceptual processing and other memory mechanisms. Attention, Perception, & Psychophysics, 16(3), 575-596.
Irwin, D. E., & Thomas, L. E. (2008). Visual sensory memory. Visual memory, 1(9), 9-43.
Lu, Z. L., Neuse, J., Madigan, S. A., & Dosher, B. A. (2004). Fast Decay of Iconic Memory in Observers At-Risk for Alzheimer's Disease. Journal of Vision, 4(8), 767-767.
Rensink, R. A. (2014). Limits to the usability of iconic memory.
Sperling, G. (1960). The information available in brief visual presentations.
Psychological monographs: General and applied, 74(11), 1.
Sperling, G. (1967). Successive approximations to a model for short term memory. Acta psychologica, 27, 285-292.
Window.requestAnimationFrame() - Web API Interfaces | MDN. (2010). Retrieved
June 26, 2017, from
