Name: Bas Vegt
Student number: 1522876 Date: 2020-07-21
Supervisor: Roy de Kleijn Second reader:
Word count: 8439 Cognitive Psychology
Thesis Msci Applied Cognitive Psychology
Differential Learning Effects in a
Probabilistic Serial Reaction Time
Task.
Abstract
Human behaviour consists largely of elemental tasks, which when performed in sequence make up larger actions. In the trajectory serial reaction time (SRT) task, participants move their cursors to one of four stimuli located at corners in a square-shaped grid. This task has been used to study implicit learning of motor skills. Of interest to the present study are results from Kachergis, De Kleijn, et al. (2014) in which subsequences were not learned unilaterally. Preliminary analysis carried out after the fact suggests this effect cannot be fully ascribed to diagonal or in-frequent moves making certain subsequences harder. The present study aims to determine whether the effect is due to variations in distance, whether it is cor-related with frequency, and whether its orientation frame is absolute or relative. 52 participants divided into two conditions, each of which saw subsequences in a different orientation, performed an online version of the trajectory SRT task. We discuss three main findings. First, no evidence was found to suggest relative distance played a role. Second, moves which occurred frequently were associated with lower RTs. Third, relative RTs per subsequence were the same between the two conditions, indicating the effect is relative to the orientation of moves made during training. Together these results lead us to conclude that higher frequency of certain moves during training leads to better post-training performance of sub-sequences containing these moves. Post-hoc analyses suggest this relationship might not be entirely straight-forward. In light of these outcomes and their impli-cations, we discuss several opportunities for future research, as well as limitations and alternate interpretations.
Keywords: Sequential action; sequential learning; serial reaction time task; implicit learning; movement trajectory; action sequencing, visuospatial reference frame, statistical learning
Introduction
Background
On any given day, people perform countless actions, many of which might seem elemental on the surface. In reality, nearly all human behaviour can be broken down into several components. Indeed, even something as simple and routine as brewing your morning cup of coffee consists of numerous sequential actions: Grabbing a cup, placing it under the coffee machine, filling the machine with beans, adding water, etc. To perform these tasks effectively, a person must know what steps are required and what steps these steps consist of in turn.
Imagine you could not fetch your own coffee. Instead, you were to precisely instruct another person (or robot) to do it for you. Suppose this assistant knows nothing about fetching coffee or any other drink. Would the above set of instruc-tions suffice? The first instruction, "grabbing a cup", immediately raises several questions. How does grabbing work? Where is the cup? For that matter, what precisely is a "cup"? If the cup is in a cupboard, the assistant will need to move a hand to the cupboard’s door handle, open the cupboard, move a hand to a cup, and clasp it, all in order to carry out the very first instruction.
In this way, each of the instructions must be further distilled into distinct sub-tasks. More often than not, these can be distilled further still. At the deepest levels, a person or robot needs to recognise all of the required objects on sight or touch, which in itself is no small feat. Advanced agents might think ahead to plan a next action before the current one is entirely resolved. For instance, sometimes a cup can be grabbed much more efficiently if one hand is already moving towards the to-be-opened cupboard while another is still in the process of opening its door. All of these considerations and more occur each time you fetch a cup, in processes which take place within milliseconds.
Sequentiality in Language
Language, too, has a sequential structure. Words can be seen as action sequences governed by grammar as a set of rules. These rules dictate which sounds (or sym-bols) may occur in sequence, and which may not. Before a person can apply these
rules, they have to learn them, as is the case for any non-innate skill. This learning process occurs in part at school, through explicit explanations of rules and defi-nitions which broaden one’s understanding of a language. But early development of this skill takes place to a large extent in the early years of infancy (Goswami, 2008).
It is well documented that early language acquisition revolves to a large extent on context (Waxman and Lidz, 1998), but this is not the only piece to the puzzle. Before infants are able to associate words with objects or events, they need to know where one word ends and another begins. Conventional speech does not reliable provide acoustic breaks between words (Kuhl, 2004). Instead, human infants attend to clues in at least two components of speech. The first of these components is rhythm (Jusczyk et al., 1999): subtly emphasised (or stressed) syllables typically indicate the beginning of a word (in English). The second source of clues brings us back to the sequential nature of language.
The syntax of natural human languages can be expressed as a set of elemen-tal components. In the case of spoken language these elements are sounds (or phonemes); in written language they would be symbols. These elements, or sub-actions, are accompanied by transition frequencies which indicate how likely each sub-action is to follow any given one. Humans (Saffran, Newport, et al., 1996), especially human infants (Saffran, Aslin, et al., 1996), pick up on these transition probabilities.
Thus the structure of a language can be instinctively understood through clues in the transitional probabilities of its elements: a "y" is quite likely to be followed up by an "ou", but after that, a functionally endless variety of sounds could fol-low. So, "you" is probably a valid word. By this process, sets of phonemes are recognised as words. In the same manner, sets of words can be recognised as larger phrases or a sentence. For instance, while it is true that a great many sounds could follow "you", it is more likely to be followed by "are" than by "is", hint-ing to which of these options is appropriate. This is the nature of early language acquisition.
Studying Sequential Learning
At the heart of human language acquisition, and sequential learning generally, is a broader statistical learning mechanism which allows for the development of numerous skills (Kelly and Martin, 1994). These sequential learning processes can be represented in certain research paradigms. The serial reaction time (SRT) task, first established by Nissen and Bullemer (1987), is one such paradigm. In this task, participants in two conditions were presented with four buttons and a set of corresponding lights, which flashed consecutively. Participants were instructed to push the corresponding button in response to each light stimulus. The order of these stimuli were determined randomly in the control condition, whereas they repeatedly followed a predetermined order (i.e. a sequence) in the experimental condition.
Nissen and Bullemer (1987) showed that subjects were largely unaware of the fact that any fixed sequence occurred repeatedly but nonetheless showed faster reaction times after training than did subjects presented with random stimuli. This paradigm can thus be used to study the learning process for implicit motor skills.
Previous Research
An ongoing line of research (de Kleijn, 2017; de Kleijn et al., 2018; Kachergis, Berends, et al., 2014; Kachergis, De Kleijn, et al., 2014) adapted the SRT task into a paradigm which involves moving a cursor, rather than discrete button presses. This adaptation enabled researchers to study anticipatory movement. Moreover, this so-called trajectory SRT task Kachergis, Berends, et al., 2014 presented stim-uli in subsequences which in turn occurred in a random order, in contrast to the original SRT task Nissen and Bullemer, 1987 which presented stimuli in fully pre-set sequences. Nevertheless, participants’ performance improved in the trajectory SRT task just as it had in the original.
Note that the order of stimuli within a given subsequence did not change, despite subsequences themselves having been presented in a random order. Thus, stimuli which were part of the same subsequence more often appeared directly succeeding one another, than did those between two different subsequences (i.e. the transition from the last stimuli of one subsequence and the first of another).
Once more, the learning effect was apparent in the form of a reduction in reaction times: As the training progressed, participants were faster to respond to stimuli within subsequences than in transitions between them, and showed larger speed-up in a repeating deterministic condition than in a random condition (Kachergis, Berends, et al., 2014).
One of the earlier articles (Kachergis, De Kleijn, et al., 2014) from this line of research highlighted certain ambiguous results which they suggested warranted further research. They found that some of their subsequences saw more improve-ment over time than others. Furthermore, the subsequences differed substantially in terms of overall reaction times as well. See figure 1 for the subsequences in question and figure 2 for their corresponding reaction times.
Figure 1: The subsequences used in Kachergis, De Kleijn, et al. (2014)
Figure 2: Reaction times in ms per subsequence, after learning. (Block 4.) Bars show +/-1SE.
This effect could not be accounted for, by any known phenomena, in the orig-inal study (Kachergis, De Kleijn, et al., 2014). It is not known whether this effect
was motor-related or cognitive, nor whether it concerned a learned or innate ten-dency. Considering the nature of the topic at hand, one central question is whether the observed effects were visuospatial in nature.
Within the subsequences in question, all four stimuli locations were not pre-sented equally often. Due to this fact, it is possible that during training, partici-pants learned to tend towards a certain corner or side of the screen, as that corner or side was more likely than the other(s) to contain the next stimulus. It is also possible that people have an innate tendency to pay more attention to certain sides or corners of a screen than others.
Present Study
Goals
The present study attempted to investigate the different reaction times per subse-quence observed in previous research (Kachergis, De Kleijn, et al., 2014). To this end, it replicated the paradigm from that study and added a novel condition to test new hypotheses, detailed below. The aim of the present study was to determine which, if any, of these hypotheses is or are accurate.
Hypotheses
Based on the spatial attributes of the various subsequences (see figures 1 and 2), our understanding of the mechanisms behind sequential learning, and preliminar-ily evaluation of data from previous research (Kachergis, De Kleijn, et al., 2014), we formed the following hypotheses:
1. Distance effect:
Subsequences which include more diagonal movements might tend to take longer to execute than those which do not, because these movements requires the participant traverse their cursor for a larger distance than do non-diagonal movements.
2. Frequency effect:
Subsequences which include more uncommon movements might tend to take longer to execute than those which contain only common movements (i.e.
movements that also appear in other subsequences), because the latter are more readily available in participants’ memory.
3. Visuospatial effect:
It has been observed that participants find certain subsequences more difficult to learn than others. Reasoning in terms of a visuospatial aspect, it can be said that within the chosen set of subsequences movements originating and/or end-ing in certain locations were more difficult to learn than those which did not. If this aspect was indeed the determinate one, then these same moves should always prove difficult, regardless of which subsequences are used and thus which other moves were trained. Thus, subsequences including certain loca-tions might be more difficult to learn than others, due to certain characteristics of those locations.
General Setup
The distance hypothesis was tested in two ways: by theoretical correction and by empirical correction. For the correction based on theory, the reaction times for di-agonal movements were corrected by the factor of the extra distance geometrically required for these movements versus non-diagonal ones. Due to the stimuli’s ori-entation, the relevant ratio could be easily calculated for all diagonal movements using the Pythagorean theorem.
For the correction based on empirical data, the reaction times for diagonal movements were again corrected by a ratio representing the difference between diagonal and non-diagonal movements, this time determined by previously mea-sured reaction times. As comparison material, we used data from a trajectory SRT task with random rather than repeating stimuli sequences (Kachergis, Berends, et al., 2014), that is to say, without any learning effects potentially causing noise. This data allowed us to determined precisely how much longer (if at all) partici-pants typically take to complete diagonal rather than non-diagonal movements in a vacuum, disregarding other effects. This ratio was then applied to the data in the same manner as in the theoretical correction.
The frequency hypothesis was tested by determining the correlation between frequency and reaction time for each possible movement. We measured reaction
time per possible move from one corner to another ("syllable"), instead of per sub-sequence of four moves ("word"). Perhaps participants anticipated certain moves or locations to occur more than others, and were quicker to perform these ex-pected moves, and slower to perform unexex-pected ones. If this were the case, we expected the quicker moves to be those which occur more frequently, thus giving participants cause to anticipate them occurring frequently in the future.
Testing the visuospatial hypothesis required a modification of the trajectory SRT task used previously. An adaptation was implemented which employs two between-participant conditions, on of which contained the "original" subsequences, while in the other all subsequences were "rotated" by 90 degrees. That is to say, when comparing the two conditions, the orientation of all subsequence moves are offset relative to the stimuli locations (as illustrated in figure 3).
Figure 3: The subsequences used in the present study.
(a) The original subsequences. (b) The turned subsequences.
The original study found different response times pertaining to each of the six subsequences used, with participants roughly scoring worst for subsequence "E" and best for "A" (see figure 2). The present study contained two conditions: one in which the subsequences were identical to the original study, and one in which they were turned sideways. The visuospatial hypothesis predicted that the identical subsequences would yield equivalent response times, whereas the turned subsequences would not.
Methods
Design
The data points of the current study stemmed from a new adaptation of the same trajectory serial reaction time (SRT) task (Kachergis, Berends, et al., 2014). Par-ticipants were instructed to move their cursor as fast and accurately as possible to any target that changed from white to green. After arriving at the highlighted green stimulus (the other three stimuli stayed white), another stimulus was highlighted after a 500 ms inter-stimulus interval (ISI). Participants completed 4 blocks of 20 trials, each of which contained a series of 12 locations (i.e., 3 subsequences). There was a short break after every block in which participants could rest their eyes and hand.
In each block, each of the 6 action subsequences appeared 10 times, randomly distributed. The 6 subsequences were also guaranteed to be the first six that were presented. though the order in which each of the six appeared was also random. A unique sequence order was determined using these parameters for each partic-ipant. Word transition frequency was not uniformly random, as no repetitions in subsequence nor stimulus were allowed.
After training, participants were asked to provide (i.e. type) the numbered locations of any action subsequences they recalled from training.
The purpose of the current study was to replicate the general findings of the aforementioned study (Kachergis, De Kleijn, et al., 2014) and investigate the asymmetric learning effects in a new circumstance, namely within a rotated refer-ence frame. To this end, the current study’s design included two between-subjects conditions. In one condition, subsequences contained the same stimuli, in the same order, as was the case in previous research. This condition will be referred to as the original condition and served as a control in our analysis. In the other condition, called the "turned condition", subsequences where shifted so that stim-uli which corresponded to one corner in the original condition now correspond to the one located one step further, in clockwise direction. (See figure 3.) This was the experimental condition.
partici-pants, that is, the time between the appearance of a stimuli and the instant their cursor touched any part of it, measured in milliseconds. The level of measurement was ratio. The independent variable was which condition participants were in.
Participants
Participation in this study occurred via the internet, as did the recruitment. Par-ticipants signed up via the website ul.sona-systems.com and were connected to a webpage on a personally ran server. A total of 52 participants, 67% female, 33% male (0% other), and aged 18 to 28 (m=20), who consisted of Leiden Univer-sity Students from the Faculty of Social and Behavioural Sciences, took part in our study in exchange for one course credit. In addition, we promised the fastest overall participant an additional payout of 10 euros.
There were no inclusion or exclusion criteria, nor any drop-outs. The present study was reviewed and approved by the Psychology Research Ethics Committee of Leiden University and was conducted in accordance with the applicable laws and guidelines.
Apparatus
The task used to measure reaction time (in ms) in this study was made in Mi-crosoft Visual Studio 2019 (MiMi-crosoft Visual Studio Community 2019, 2019) using javascript and PHP. The central functionality ran on a series of javascript packages from JSPsych (de Leeuw, 2015). The script was run on an apache server, managed through WinCSP (Prikryl, 2020). The data analysis was performed in RStudio (RStudio Team, 2019).
The stimulus display consisted of four white squares (location 1 = upper left, 2 = upper right, 3 = lower left, 4 = lower right), displayed constantly. Participants used their desktop or laptop computer. As such, monitor size and resolution could not be set or determined, but the program was set to display each stimulus as 80 pixels on each side, separated by 440 pixels of white space.
This task was based on those used in previous research (de Kleijn, 2017; de Kleijn et al., 2018; Kachergis, Berends, et al., 2014; Kachergis, De Kleijn, et al.,
2014). Half of the stimuli subsequences used in the present study are identical to the ones used in those studies, with the other half being the same subsequences, but rotated clockwise once. These adjusted subsequences made up the experimen-tal "turned" condition. The created task was tested by a small pilot of students to ensure it worked appropriately.
Reaction times were used to determine the extent to which participants im-proved their performance in the various subsequences and within-subsequence action moves. The task took participants an average of 18 minutes to perform. After training, participants were asked to provide (i.e. type) the numbered loca-tions of any action subsequences they recalled from training.
Procedure
Participants took part in this study on a time of their choosing (before a deadline) and from their chosen location, presumably their desk at home or equivalent. After sign-up, participants where asked to connect to a webpage using a browser on their desktop or laptop computer. On this webpage, they were presented with some basic information concerning the study and asked for their informed consent. Participants were given the option to ask questions prior to participation in the form of e-mail. All further instructions participants received came in the form of text on this webpage. After agreeing to continue, participants filled in a simple questionnaire before moving on to the main task. This questionnaire consisted of demographic questions as well as some questions to test whether participants had read and understood their instructions. We also asked participants whether they would use a mouse or trackpad, although we informed participants that a mouse was preferred.
Due to the online nature of this study, we did not offer any opportunity for questions from this point on until the task was completed, since this would have necessitated unpredictable intermissions, which could have caused substantial noise in our data due to the central role of memory and attention in this study. Partic-ipants were of course free to abort their further participation in the study at any time if they so desired, but would not then be allowed to participate again.
pos-sible to any target that changed from white to green. There was a short break after every block in which participants could rest their eyes and hand. After training, participants were asked to provide (i.e. type) the numbered locations of any action subsequences they recalled from training. Participants typically spent a little over 21 minutes on the study from start to finish.
Analysis
To determine whether general learning occurred, we ran a one-way ANOVA as well as Bayesian ANOVA on block vs reaction time (RT).
To determine whether the same subsequence effect occurred as was the case in Kachergis, De Kleijn, et al. (2014), we ran a one-way ANOVA as well as Bayesian ANOVA on subsequence vs RT.
To test the distance hypothesis, we corrected diagonal distance, first by the geometrically calculated difference between diagonal and non-diagonal distances, then by determining the difference between reaction times for diagonal versus non-diagonal movements in previous data (Kachergis, Berends, et al., 2014). We tested for each of these corrected sets of data whether subsequence RT values aligned, or still differed from each other significantly, using a one-way ANOVA.
To test the frequency hypothesis, we performed a simple t-test for correlation between frequency and reaction time for each possible movement.
To test the visuospatial hypothesis, we performed a one-way ANOVA as well as Bayesian ANOVA on condition vs RT. In the same way, some additional inter-action effects were tested: block*condition on RT, as well as subsequence*condition on RT, as well as the three-way interaction between block, condition and subse-quence.
For all tests, we considered outlier data to be any individual movements which took longer than 1500 ms, as this probably indicated participants were distracted or had a momentary lapse of attention, neither of which are relevant to the phe-nomena under investigation. We also omitted the first move of each block from our analyses, as these starting moves are few in number and, lacking a set starting location, are poorly represented in and representative of the list of possible moves and thus the data.
In order to perform the listed tests, the following assumptions and criteria were checked: the residuals should be normally distributed, the groups should be equal in variance, and samples should be independent. No further calculations were needed to be applied to the raw data files in order to get operationalised scores, as the scripts we used provide reaction times in ms.
Results
Initial Description of Data
Median reaction time was 415 ms (SD: 204). 238 individual reaction times (out of 49920) were longer than the aforementioned outlier threshold of 1500 ms and were thus removed. All participants’ data were otherwise complete and no partici-pant had to be removed due to missing or erroneous data. As such, the final N was 52. Retaining or removing outliers did not have a swaying effect on the obtained results. No violation of assumptions were encountered in any of the following tests.
Replicating Key Effects
General learning effects were evaluated using a one-way ANOVA as well as Bayesian ANOVA. ANOVA indicated a significant main effect of block (F(3, 150) = 11.354, p < .001). Bayes factor in favor of a main effect of block was 6.54 x 1041, which is considered decisive evidence in favor of an effect.
Subsequence effects at block four were evaluated using a one-way ANOVA as well as Bayesian ANOVA. ANOVA indicated a significant main effect of subse-quence (F(5, 250) = 10.568, p < .001). Bayes factor in favor of a main effect of subsequence was 2.50 x 108. This is considered decisive evidence in favor of an effect.
Figure 5: Reaction time in ms per subsequence in block four. Bars show +/-1SE.
Testing the Distance Hypothesis
To test the distance hypothesis, we corrected for diagonal distance, first by the cal-culated difference between the respective distances (figure 6a), then by determin-ing the difference between reaction times for diagonal versus other movements in previous data (Kachergis, Berends, et al., 2014) (figure 6b). Both of the re-sulting datasets were then evaluated using one-way ANOVAs as well as Bayesian ANOVA. ANOVA indicated a significant effect of subsequences at block four af-ter geometric correction (F(5, 250) = 26.486, p < .001) as well as afaf-ter empirical correction (F(5, 250) = 10.686, p < .001). Bayes factor in favour of a main effect of subsequence at block four, after geometric correction and empirical correction respectively, were 8.21 x 1032and 3.47 x 108. This is considered decisive evidence in favor of an effect in both cases.
Testing the Frequency Hypothesis
To test the frequency hypothesis, we took the median reaction times for each pos-sible move from one stimuli to another as well as the occurrence frequency of
Figure 6: ’Corrected’ reaction times
(a) Reaction times in ms per subse-quence in block four, corrected by ge-ometrically calculated ratio.
(b) Reaction times in ms per subse-quence in block four, corrected by em-pirically determined ratio.
each move. We then evaluated the correlation between these two datapoints using Pearson’s correlation coefficient. Blocks 1-3 (i.e. blocks in which most training would occur) were used to evaluate the frequencies and block 4 (i.e. after partic-ipants had a chance to grow familiar with the data) was used to evaluate reaction times. This test revealed a statistically significant negative correlation between frequency and reaction time (r(10) = -.716, p = .009). See figure 7.
Testing the Visuospatial Hypothesis
To test the visuospatial hypothesis, the interaction effect of subsequence*condition at block four was evaluated using a two-way ANOVA as well as Bayesian ANOVA. ANOVA indicated no significant interaction effect of subsequence*condition (F(5, 250) = 2.217, p = .053). Bayes Factor in favor of an interaction effect of condi-tion*subsequence was 6.21 x 10-3. This is considered decisive evidence against an effect.
Other Effects
To ensure we did not overlook anything, all other possible interaction effects were evaluated.
Effects of condition were evaluated using a one-way ANOVA as well as Bayesian ANOVA. ANOVA indicated no significant main effect of condition (F(1, 50) =
Figure 7: Frequency vs reaction time for all moves
(a) Occurrence frequency of each move in first three blocks.
(b) Reaction time in ms per move in the fourth block. Bars show +/-1SE.
Figure 8: Reaction time in ms across the four blocks, grouped by condition. Bars show +/-1SE.
.351, p = .556). Bayes factor in favor of a main effect of condition was 1.25 x 10-1. This is considered substantial evidence against an effect.
Interaction effects for condition * block (i.e. different learning effects per condition) were evaluated using a two-way ANOVA as well as Bayesian ANOVA. ANOVA indicated no significant interaction effect of block*condition (F(3, 150) = .255, p = .858). Bayes factor in favor of an interaction effect of block*condition was 2.36 x 10-4. This is considered decisive evidence against an effect.
Interaction effects for block * subsequence * condition were evaluated using a three-way ANOVA as well as Bayesian ANOVA. ANOVA indicated no significant three-way interaction effect (F(15, 750) = 1.120, p = .334). Bayes factor in favor of an interaction effect of condition * block * subsequence was 1.17 x 10-9. This is considered decisive evidence against an effect.
Exploratory Analysis
Moves per Condition
Considering the correlation between reaction time (RT) and frequency per move was significant, we saw fit to calculate the move * condition effect of the data in order to shed more light on this finding. Interaction effect of move * condition was significant (F(11, 550) = 25.954, p < .001). Bayes factor in favor of an interaction effect of move*condition was 1.98 x 10128. This is considered decisive evidence in favor of an effect.
Figure 9: Reaction time in ms per move in the fourth block, grouped by condition. Bars show +/-1SE.
Number of Stimuli Locations
Based on a notation in Kachergis, De Kleijn, et al. (2014), we tested the effect of the number of different locations touched during each subsequence, using a one-way ANOVA as well as Bayesian ANOVA. (I.E. subsequence A, B, C and D all touch three locations, while E and F touch four.) ANOVA indicated a significant main effect of location (F(1, 51) = 65.562, p < .001). Bayes factor in favor of a main effect of location was 4.62 x 1024. This is considered decisive evidence in favor of an effect."
In other words, participants responded slower to subsequences which required moving the cursor to four distinct locations, than to those which included one location two times. One interpretation of this finding is that participants had a harder time learning subsequences which involve a greater amount of diverging information.
Figure 10: Reaction time in ms across the four blocks, grouped by number of locations in the subsequence. Bars show +/-1SE.
Use of Device
Participants in this study were allowed to use trackpads, instead of conventional computer mouses, to perform the task. Thus, the collected data can be separated in two groups, differentiated by device usage. In order to check whether these two data groups are comparable enough as to allow for them to be treated as a homogenous dataset, we tested the main effect of device, as well as the interaction effect of device * block, using a one-way ANOVA and Bayesian ANOVA.
ANOVA indicated a significant main effect of device (F(1, 50) = .351. p = .003). Bayes factor in favor of a main effect of device was 4.42. This is considered substantial evidence in favor of an effect. ANOVA did not indicate a significant interaction effect for block * device (F(3, 150) = 1.947, p = .124). Bayes factor in favor of an interaction effect of block * device was 1.66 * 103. This is normally considered decisive evidence in favor of an effect. These contradictory outcomes are discussed further below.
Figure 11: Reaction time in ms per move across the four blocks, grouped by device. Bars show +/-1SE.
Overview of the Results
The subsequence effect observed in this and and previous research was not ac-counted for by correcting for diagonal distance. Move frequency and RT were negatively correlated, which could play a role in the subsequence effect. The newly introduced condition did not seem to influence the learning effect or final reaction times per subsequence.
Discussion
The aim of the present study was to replicate and further investigate earlier find-ings made by Kachergis, De Kleijn, et al. (2014). We sought to test various pos-sible explanations. To this end, we used an adaptation of their trajectory serial reaction time task, with the addition of two between-subjects conditions. Partici-pants in each condition were presented with the stimuli subsequences in a different orientation.
Interpretation of the Results
We tested several hypotheses that could explain the effect in question. First, we checked whether the increased distance of diagonal movements could account for the effect. The results did not support this hypothesis, since the data still contained substantial subsequence effects after our attempts to correct reaction times based on both theory and previous data. Next, we tested for a correlation between fre-quency and reaction time. This revealed a negative correlation. Lastly, we checked whether the effect was affected after all stimuli subsequences were turned side-ways. This was not the case; respective performance across subsequences was the same in both conditions. Based on these results, the exact spatial location of the stimuli does not seem to be relevant to this effect. Miscellaneous interaction effects were evaluated as to be thorough but none proved significant at this stage. Post-Hoc analyses revealed a number of effects we had not anticipated prior to our data collection. These effects are all addressed in our discussion below.
Conflicting Results
Curiously, post-hoc tests revealed a significant effect between move and condi-tion, whereas no effect was found between subsequence and condition. This is noteworthy considering subsequences are entirely made up of these moves. One explanation of this could be that the interaction between move and condition is not a straight-forward linear one, but more complicated in nature, thus obscur-ing the effect when moves are represented as a cluster (i.e. subsequence). This speculation is elaborated on in the future research subsection below.
What is more, post-hoc testing revealed that within our data, the difference between reaction times of moves within subsequences on the one hand, versus moves between subsequences on the other, manifests as early as the first time the subsequences are presented. In other words, subsequences were differentiated from the start. Although this difference did grow, as expected, over the course of the subsequent moves and blocks, it was not expected that the effect would be present at all on the very first trials.
Previous research has seen within-versus-between-subsequence effects after one iteration at the earliest (Kachergis, Berends, et al., 2014). Participants had not seen the subsequences prior to participation and thus should have had no knowl-edge of them at that point, as they had not yet had a chance to learn them. For this reason it is unexpected to find this effect during the first iteration of subse-quences. One possible explanation could be that participants completed moves in a certain rhythm, taking tiny breaks more often after every fourth move. This could be either due to some instinctive tendency or possibly because participants encountered comparable actions of similar lengths in previous studies in which they participated.
Lastly, post-hoc testing revealed that RT was different for participants who used a trackpad rather than a mouse. However, we obtained conflicting results regarding the learning effect associated with different devices. We were not able to cut through this discrepancy. Thus, we cannot say anything about the nature of this particular interaction effect (or lack thereof) based on the outcomes of the present study.
Alternate Interpretations
It is well-documented that an individual’s preferred reading or scanning direc-tion is associated with their performance on visuospatial tasks (Kazandjian and Chokron, 2008; Kessler and Oberauer, 2015). Such research suggests participants exhibit a bias which can be predicted based on whether they are accustomed to left-to-right or right-to-left scripts. This effect could play a part in the varying re-action times across subsequences. However, our results indicated that these vari-ations carry over across conditions with different subsequences, and that which
particular moves are performed well can be predicted based on which occur fre-quently in training. This outcome makes it unlikely that scanning direction plays a critical role in this phenomenon.
Limitations
Data for this study was not collected in a lab-setting. Instead, participants partic-ipated in this study from their homes using their own computers. There are some potential drawbacks to this approach when compare to a traditional lab-based test-ing that should be mentioned.
Participant Focus and Motivation
Of central concern is participants’ focus. Although participants were asked to take the task seriously, we cannot be certain they did not, for instance, attempt partic-ipation while watching a tv-show in the background, or were in any other way not fully focussed on the task at hand. The average amount of time participants took breaks between blocks was around 48 seconds (m=30), which is perfectly in compliance with their received instruction: "not more than a minute or so". However, several participants’ breaks lasted several minutes: 20 participants took at least one break longer than 60 seconds. Of these, 8 took at least one break of over 120 seconds. The longest two breaks, which lasted nearly 6 and 10 minutes respectively, were both taken by the same participant.
It is unknown what participants were doing during these breaks or if any of them were due to technical difficulties. The participant in question (among others) also featured slower reaction time in their final block than any previous block. In fact, more than one participant seemed to exhibit, for all intents and purposes, inverted learning curves. These participants might have suffered from a loss of concentration over time. Perhaps these participants took longer breaks because they were unmotivated to complete the task in a timely fashion. Regardless of the reason behind these longer breaks, they are likely to have interfered with learning processes to some extent.
Use of Trackpad Could Distort Data
Another factor introduced as a product of the online data collection setting is the use of different input devices. Unlike previous studies in the current line of research, the present study allowed participants to participate using a trackpad. This was done because participation was to occur online and not all participants would have had access to a working computer mouse from home. All participants were asked to use a mouse if they had one available. Nevertheless, 30 subjects out of the total 52 participated used a trackpad.
Post-Hoc analysis revealed a substantial difference in overall reaction time (RT) between participants using different input devices. However, results were ambiguous as to whether or not differences existed between them in terms of RT improvement across blocks. This is to say that it is not proven within the present study that trackpads are a suitable substitute for use in studies of this sort, and whether they are a source of noise in the present study.
Note that the use of one input device or the other was not an experimentally controlled condition in the present study. This decision was left up to participants themselves, hopefully determined by the simple fact of who among them owned a computer mouse, and who did not. We did not think to regulate groups based on device because we did not anticipate trackpad users would make up a noteworthy portion of participants, let alone the majority. Any relevant participant character-istics associated with device ownerships, should they exist, are thus unaccounted for.
Finally, note that participants indicated only in which of two categories their device belonged. Beyond this basic information, nothing is known about the par-ticular devices participants used, which likely varied in terms of size, weight, sensitivity, quality, etc. As such, we do not know how much distance needed to be traversed in space of typical movements of either device, nor how heavy or reliable these movements were. All this is to call for caution when making any conclusions regarding the use of a mouse, versus that of a trackpad, in performing the trajectory SRT task, based on the present study’s data.
Implications
The present study is the latest in a line of research investigating learning processes of sequential actions. It tested several hypotheses to account for the observed ef-fects. Despite some methodological limitations, we successfully replicated previ-ous research and observed similar results in terms of subsequence effects. Com-paring our different conditions produced results suggesting that the exact spatial orientation plays no critical role in the subsequence effects observed in this re-search, nor did whether a given movement is diagonal or not. What did seem to play a role is the relative frequency of each move. Further research is required for a firmer grasp on the exact nature of this role.
Future Research
Based on the aforementioned results, more frequently appearing moves appear easier to learn, but we have not establish exactly how this trend manifests in the subsequences themselves. Future research could investigate the nature of this relationship to determine, for instance, whether reaction time is influenced more heavily by the number of relatively rare moves within a subsequence or rather the rarity of the least (or most) common individual move.
Direction Versus Destination
On the topic of moves, the present research determined reaction times and fre-quencies of movements from one corner to another, thus creating a pair of two stimuli locations. In other words, moves were presently defined in terms of their direction. Alternatively, moves could be defined by their target location by group-ing them based solely on their destination stimulus, regardless of where a move started. Considering moves in this way would not evaluate each possible move in turn, but compare the four stimuli locations to one another. There are method-ological benefits and drawbacks to both approaches. Future researchers must de-termine their preferred method based on the specific hypotheses they wish to test.
Implicit Versus Explicit Learning
There is also the question of how much participants were aware of learning re-curring sequences. When asked to list any sequences they could remember, par-ticipants typically listed a few at most, if they get any correct at all. Despite displaying increased performance, explicit knowledge of the stimuli order was absent more often than not, as has been documented before (Kachergis, Berends, et al., 2014). Despite this general rule, some participants in our study did dis-play a better knowledge of the subsequences than others. Future research could investigate whether recollection after the fact bears any relationship, positive or negative, to the learning process or reaction times generally.
Scanning Directionality
Within the demographic tested, it is likely that most if not all participants left-to-right readers. Future research could ask participants in which language they typically read and write, and compare the two groups. However, since these two demographics are not typically found closely mixed within one population sam-ple, researcher might need to explicitly recruit right-to-left readers in order to form a usable sample.
In addition to aforementioned effects of individual’s scanning directions as a result of their native script, it is also possible that potential scanning direction ef-fects should be identified in terms of clockwise vs counter-clockwise direction, rather than strictly left-to-right or right-to-left. This would be harder to test, since the dials of most contemporary clocks in the world run in the same direction, due to their heritage in sun-dials which were originally made in the northern hemi-sphere. As such, nearly all humans who have a concept of "clockwise" think of it as the same orientation. However, research has found certain differences between individuals regarding this orientation. For one, while nearly all right-handed peo-ple draw circles counter-clockwise, this is not the case for left-handed peopeo-ple (van Sommers, 2011). However, the same research also indicated the mechanics of drawing half-circles and quarter-circles are not equivalent to those of drawing circles.
making full rotations, be they clockwise or counter-clockwise, it is difficult to say exactly how any scanning directionality, should one be discovered, would impact performance on the task. It has proven difficult to produce consistent findings in this topic of research (Karev, 1999), and much of it distinguishes only horizontal or vertical directionality rather than a circular orientation of direction. It is also not clear whether any effects would indicate a causal link between handedness and scanning direction or whether the two are both determined by an underlying neurological cause. While future research could sample participants based on the aforementioned tendencies, it remains to be seen whether this approach would bear any fruit.
Number of Locations in a Subsequence
Kachergis, De Kleijn, et al. (2014) named several characteristics of the subse-quences used in that as well as the present study, one of which was the number of different locations each subsequence contains. Post-hoc analysis of the present data suggested this characteristic might be a key factor behind the effect under investigation. This could be one avenue future research could take.
Within the current paradigm, the possible subsequences have few degrees of freedom due to certain restrictions: each subsequence must consist of four acti-vated stimuli, where each actiacti-vated stimuli can be any one of four stimuli locations (corners of the screen). What is more, the same stimuli cannot occur twice in a row. Within these parameters, the only possible number of different locations that can be fitted within one subsequence is two, three, and four. Of theses, differ-ent subsequences of length two would necessarily consist of only one possible ’shape’, that is, a line. Contrast this to subsequences of length three and four, which can take on various shapes. This is to say that the only feasible possible subsequences to use in the current paradigm consist of either three or four stimuli locations.
This limitation presents a challenge for any research further delving into the possible connection between reaction times and the amount of locations visited within a single subsequence. Analyses carried out to determine relevant effects would benefit from a broader numeric scale so that different levels can be
com-pared, rather than just two (these two being either 3 locations or 4 locations). Future research could employ five rather than four stimuli locations, arranged in a pentagon, and increase the subsequence length to five accordingly.
Explicit Recall and Device Use
Research suggests that people retain relatively more information from a learning task when some form of difficulty causes it to demand more attention (Diemand-Yauman et al., 2011). Tasks with such "desirable difficulties" demand more cog-nitive engagement, which leads to deeper processing, thus facilitating encoding.
Applying this finding in the present context, participants using a trackpad might spend more attention during the task than those using a mouse, as the latter is generally thought of as more ergonomic and easy to use without much thought by most computer users. Thus, the use of a trackpad could lead to higher recollec-tion of subsequences. On the other hand, if the increased difficulty is too high, the use of a trackpad might function as extraneous cognitive load rather than desir-able difficulty (Sweller et al., 1994). In other words, it could also take away from the learning process rather than add to it. It also remains to be seen whether a higher degree of explicit recall would be beneficial or detrimental to performance in terms of reaction times.
As stated previously, we cannot comment based on the present study on any theories regarding the use of various input devices in the trajectory SRT task. Were this topic to be taken on by future research, it is advisable to do so in a traditional lab-setting, where factors such as screen size and cursor sensitivity can be precisely controlled, and devices are randomly designated to each participant. In addition, a method would need to be devised for measuring and determining recall. This might seem a trivial matter, and indeed collecting the information is simple enough- only ask your participants to list or reproduce the movements for any sequences they remember after participation, optionally after a delay. The dif-ficulty lies in determining based on that information how well a given participant, in fact, remembers any subsequences. Particularly, it is not obvious how partially correct answers would score, e.g. listing three out of four links of a subsequence, or conversely, listing one move too many.
References
de Kleijn, R. (2017). Control of complex actions in humans and robots (Doctoral dissertation). Leiden University.
de Kleijn, R., Kachergis, G., & Hommel, B. (2018). Predictive Movements and Human Reinforcement Learning of Sequential Action. Cognitive Science, 42, 783–808. https://doi.org/10.1111/cogs.12599
this is the one that also had blind exploration learning
de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral ex-periments in a web browse. Behavior Research Methods, 47(1), 1–12. https://doi.org/10.3758/s13428-014-0458-y
Diemand-Yauman, C., Oppenheimer, D. M., & Vaughan, E. B. (2011). Fortune fa-vors the: Effects of disfluency on educational outcomes. Cognition, 118(1), 111–115. https://doi.org/10.1016/j.cognition.2010.09.012
Goswami, U. (2008). Cognitive Development: The Learning Brain. Psychology Press.
Jusczyk, P. W., Houston, D. M., & Newsome, M. (1999). The Beginnings of Word Sememtation in English-speaking Infants. Cognitive Psychology, 207, 159–207.
Kachergis, G., Berends, F., Kleijn, R. D., & Hommel, B. (2014). Trajectory Effects in a Novel Serial Reaction Time Task. Proceedings of the 36th Annual Conference of the Cognitive Science Society, (July). https://doi.org/10. 13140/2.1.2370.9763
Kachergis, G., De Kleijn, R., Berends, F., & Hommel, B. (2014). Reward effects on sequential action learning in a trajectory serial reaction time task. IEEE ICDL-EPIROB 2014 - 4th Joint IEEE International Conference on Devel-opment and Learning and on Epigenetic Robotics, 415–420. https://doi. org/10.1109/DEVLRN.2014.6983017
Karev, G. B. (1999). Directionality in right, mixed and left handers. Cortex, 35(3), 423–431. https://doi.org/10.1016/S0010-9452(08)70810-4
Kazandjian, S., & Chokron, S. (2008). Paying attention to reading direction. Na-ture Reviews Neuroscience, 9(12), 965. https://doi.org/10.1038/nrn2456-c1
Kelly, M. H., & Martin, S. (1994). Domain-general abilities applied to domain-specific tasks: Sensitivity to probabilities in perception, cognition, and lan-guage. Lingua, 92(100), 105–140. https://doi.org/10.1016/0024-3841(94) 90339-5
Kessler, Y., & Oberauer, K. (2015). Forward scanning in verbal working memory updating. Psychonomic Bulletin and Review, 22(6), 1770–1776. https : / / doi.org/10.3758/s13423-015-0853-0
Kuhl, P. K. (2004). Early language acquisition: Cracking the speech code. Nature Reviews Neuroscience, 5(11), 831–843. https://doi.org/10.1038/nrn1533 Microsoft Visual Studio Community 2019. (2019). Visual Studio.
Nissen, M. J., & Bullemer, P. (1987). Attention Requirements of Learning Evi-dence from Performance Measures. Cognitive Psychology, 19, 1–32. Prikryl, M. (2020). WinSCP. https://winscp.net/
RStudio Team. (2019). RStudio: Integrated Development Environment for R. http: //www.rstudio.com/
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274(5294), 1926–1928. https : / / doi . org / 10 . 1126/science.274.5294.1926
Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996). Word segmentation: The role of distributional cues. Journal of Memory and Language, 35(4), 606–621. https://doi.org/10.1006/jmla.1996.0032
Sweller, J., Chandler, P., Cognition, S., Sweller, J., & Chandler, P. (1994). Why Some Material Is Difficult to Learn Linked references are available on JSTOR for this article : Why Some Material Is Difficult to Learn. 12(3), 185–233.
van Sommers, P. (2011). The production of curvilinear forms. Drawing and Cog-nition, 72–94. https://doi.org/10.1017/cbo9780511897672.005
Waxman, S. R., & Lidz, J. L. (1998). Early Word Learning. In D. Kuhn, R. S. Siegler, W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology: Cognition, perception, and language(5th ed.). John Wiley & Sons Inc.
