The Influence of Time Pressure on the Accuracy of
Mathematical Skills
Femke van Son 10533176
Bachelor Thesis Methodological Psychology Supervised by Leendert van Maanen
University of Amsterdam June 24 2016
Abstract
Time pressure seems to influence the accuracy on different tasks, but most of all on visual tasks. This study investigates the influence of time pressure on four different tasks: the flanker task, lexical decision task, math task and flash task. The focus in this article will be on the mathematical task. For this task, a distinction between easy and hard trials has been made. Results indicate that time pressure indeed influences the accuracy on
mathematical trials. The more time pressure is present, the less accurate both easy and hard trials were made.
A new way of living has arrived in which people feel there are never enough hours in the work day, also known as the phenomenon of “time famine” (Perlow, 1999). Indeed, it is likely that anyone reading this article has a daunting to-do list on the agenda. There is even a new law proposal in France that will forbid emailing after work hours to reduce the work pressure and the 24/ 7 mentality (The Washington Post, 2016). Due to this time famine phenomenon, almost every task takes place under some kind of time pressure. It is still unknown how exactly increasing time pressure influences the daily performed tasks. When time pressure is indeed raising, it is important to achieve knowledge about
performance changes due to this fact. This study will investigate the influence of time pressure on daily tasks.
However, daily tasks are a widely concept that will be difficult to study. Daily tasks can differ from answering work mail, to determine how to manage a business. One aspect this different tasks have in common, is that they all require some kind of strategie to resolve (Degenhardt- Burke, 2010). As van Maanen, de Jong and van Rijn (2014) stated; almost all intentional behavior is the result of applying strategies to problems. A task that needs strategy to resolve can been used in order to study the performance on daily tasks. Earlier research on the influence of time pressure mainly focusses on visual tasks. Stins, Polderman, Boomsma and de Geus (2007) investigated if the speed of responding affected the accuracy on the Eriksen Flanker task and the spatial conflict task, both visual
performance tasks. They concluded the faster the participants responded, the lower the percentage of correct responses. Reinagel (2013) has also shown how, again, the accuracy on a visual task performance, increases with slow reaction times. However, little strategy is needed to perform on visual tasks and are therefore less generalizable to daily tasks.
An example for a task where strategy is needed is a mathematical task.
Mathematical tasks are of prime importance in everyday life, enabling performance of calculations and comprehension of number (Sigmundsson, Polman & Lorås, 2013). Mathematical skills are learned early in the childhood all over the world, regardless specific cultural differences. It keeps evolving in every event till the adulthood
(Butterworth, 2005). This skill is required every single day, where practice and experience leads to individual changes in the ability to solve mathematical trials. Mathematics is not a
series of disconnected routines to be memorized, it involves connected ideas that can be used to solve all sorts of problems (Stallings, 2007). Where it depends on the task or circumstances what kind of strategie will be chosen.
The chosen strategy for every task can therefore depend on the present of time pressure while performing the task. Time pressure has been defined widely, and its operationalization has differed greatly from study to study (Svenson & Maule, 1998). In this study, time pressure refers to the time participants have to respond. When they have less time to respond in comprehension with other trials, the expectation is this time pressure will place a burden on cognitive load and hence reduces processing capacity (Noda, Takai & Yoshida, 2007). Due to this burden on cognitive load, the available
strategies to resolve a mathematical task can be chosen differently under time pressure. In order to determine how time pressure can influence the strategies for solving math tasks, and thereby the accuracy on the tasks, we need to define the strategies for math solving with the present of time pressure and exactly what kind of math tasks need to be solved.
First, the different tasks used in this study; the mathematical tasks that need to be solved are subtraction, addition, dividing and multiplying. This are the most researched operators and are the pedagogical and, to a large extent, the conceptual basis of other aspects of arithmetic. Therefore the focus in this study for the mathematical task will be on these four topics. Second, the strategies that will be used to solve the tasks. The expected strategies for these trials are based on the problem solving strategies posed by
Posamentier and Krulik (1998). They constructed ten different strategies. Due to the type of mathematical tasks and the deadline, we choose two strategies from Posamentier and Krulik which can be applied in this study:
1. Intelligent guessing and testing
In this strategy an answer is quickly chosen, with no or just little evidence for the right answer. A characteristic of this strategy is a low value of accuracy.
2. Logical reasoning
Although solving any problem requires logical thinking or reasoning, some problems depend upon logical reasoning as the primary strategy for solving them. In this strategy an inferences is made, which leads to a second inference and son on. The social process continues until the problem has been resolved. This strategy has in general a high accuracy value.
These two strategies will lead to two different groups of behavior patterns; the guessers and the solvers. Math tasks are known to be difficult for many people, what can lead to a large group of people who will immediately guess instead of solve. This group will therefore be called “the guessers”. It is expected that the guessers will response fast and that the accuracy in this group will lay around the guessing point. The people who do not guess will use a strategy to solve the task, in order to resolve mathematical tasks, logical reasoning is required. This group will be called “the solvers”. This logical reasoning will take some time, the expected reaction time for the solvers will therefore be longer than the reaction time of the guessers. The solvers will most likely find the right answer, that leads to the expectation that the accuracy in the solver group will be higher than in the guesser group. Because of a deadline during the tasks, a third group of behavior pattern will be expected. This group is called “the procrastinators”, these are the people who postpone their answer until they are mandatory to respond. This group will have the longest reaction times. The accuracy will be a little lower than the accuracy of the problem solvers, there where people who do not know the right answer yet, will give their response as well.
The presence of time pressure will most likely influence the third group the most. The guesser and solver group are expected to stay mainly the same, an earlier deadline will not influence the strategy that is chosen at the beginning of solving the task. On the contrary, the earlier deadline will cause that the procrastinator need to respond faster. The average reaction times of this group will therefore be closer to the average reaction times for the solver group. The expected low accuracy values for the procrastinators group will be even lower when time pressure is present than when there is no time pressure. Due to
this time pressure the procrastinators will have, as said earlier, a burden on cognitive load and are therefor expected to make more mistakes on the task when the deadline arrives.
Method Participants
The subject group consisted of thirty students with an age range from 19 to 24 years old (mean = 21.37, SD = 1.13) with 7 males and 23 females The participants could receive a participation point as a reward for enrolling in the experiment. The experiment consisted of four separate tasks: the flanker task, the lexical decision task, the math task and the flash task. The main focus in this article lies on the math task (for more information about the other tasks see Colden, 2016; Kevenaar, 2016; Meurs, 2016). These four tasks were executed in both a high time pressure condition and a low time pressure condition. In order to execute all the tasks properly, mastering the dutch language is a requirement. All other languages were excluded from the experiment. At the beginning of the experiment all the participants received further information on the study and signed an informed consent form.
Materials
The four tasks were created in PsychoPy (v.182.01). The math task contains 106 different mathematical items which can be divided in two categories; easy and hard. The Basic Knowledge in Mathematics test (BKM) has been used very often to see children development in mathematics (Sigmundsson, polman & Lorås, 2013). The results of the BKM suggests that addition and subtraction is more easy for most children than
multiplying and dividing. Therefor the easy category consists of addition and subtraction items. In the hard category the items consist of multiplying and dividing. The participants had two answer options; A or B, where fifty percent of the correct answers was A, and the other fifty correct answers B. The experiment was administered on computers with
Procedure
The experiment was executed at the Psychology lab of the University of
Amsterdam, the entire session lasted approximately 45 minutes per participant. Each participant completed the tasks individually.
In the design of the study is accounted for counterbalancing. All the subjects fulfilled the four tasks, that were constantly administered in a different order. Participant one started with the flanker task, followed by the lexical decision task, math task and at last the flash task. Participant two started with the lexical decision task, followed by the math task, the flash task and ended with the flanker task. Participant three started with the math task, followed by the flash task etcetera. Half of the participants started each task in the high time pressure condition, the other half started in the low time pressure condition.
After each trial the participant received feedback, there were three different kinds of feedback. The first one was: “Correct and on time”, when the answer was correct and on time. The second one was: “Correct but too slow” when the answer was correct, but slower than the deadline. The last one was: “Incorrect”, when the answer was incorrect regardless in which time frame the participant answered. In the high time pressure condition participants needed to give their answer in maximal four seconds, in the low time pressure participants had six seconds to give their answer. The participants could see how much time they had left in a decreasing time bar. When participants could not react on time, they could still fill in their answer. After approximately each 25 items was a short break. Any questions about the study were answered by the researcher after all the tasks were fulfilled.
Data analysis
A two-way repeated measures ANOVA was performed on the percentages correct responses with trial type (hard vs. easy) and time pressure condition (high vs. low). There is also a two-way repeated measures ANOVA performed on the reaction times with the same trial type (hard vs. easy) and time pressure condition (high vs. low).
The influence of time pressure on the accuracy on mathematical tasks can best be seen by plotting the conditional accuracy function (CAF) (White, Ratcliff & Starns, 2011).
In which accuracy for the mathematic task is displayed for responses that will fall within different bins. For construction of the CAF, each participants reaction time on a trial was classified into one second bins. For both easy and hard trials the average percentage of correct answers in each bin is computed. These percentages were then averaged across all participants, resulting in the CAFs. An upper and lower limit to the RT-bins was set, even though one could construct CAFs spanning the entire RT-range (Baroni, Pellicano, Lugli, Nicoletti & Proctor, 2012). The upper limit is set on 6 seconds, the motivation for this cut-off is that almost all the participants answered in this time period. There are a few trials that took longer than this time period, but this is most likely due to noise. The lower limit starts at 0 second.
It is expected that the results of the CAF will show the three different groups of strategies in each condition. Which will lead to low accuracy percentages for fast reaction times; the guessers, with increasing accuracy for increasing reaction time; the solvers and a decreasing accuracy for reaction times around the deadline; the procrastinators. In the high time pressure condition, the trials were made under more time pressure due to this time pressure trials will be made less accurate. What will probably lead to a faster
decreasing accuracy in the CAF for the procrastinators in the high time condition in comparing with the low time pressure condition.
In addition to the CAF, a distribution for the reaction times on all trials will be constructed for the four different conditions (trials: hard vs. easy, time pressure: high vs. low). Where most of the reaction times for the trials will be found around the deadline. The distribution of the high time pressure condition is expected to be narrower than the distribution of the low time pressure condition. Because of the high time pressure, the procrastinators will need to choose their answer faster than when there is low time pressure. This shorter time frame will lead to a narrower distribution.
In order to detect if there is indeed a mixture distribution of three different reaction time groups for solving the mathematical tasks, a cluster analysis is conducted. What is be done by using the R package Mclust (Farley & Raftery, 2006). This is a package for normal mixture models. It selects the optimal number of components in a dataset according to the Bayesian Information Criterion (BIC). The BIC is as an aid in choosing between competing
modeles. It is defined as -2Lm + mlnn. Where n is the sample size, Lm is the maximized log-likelihood of the model and m is the number of parameters in the model. The favorable model is the model with the smallest BIC (Swarz, 1978). Expected, with the earlier
provided hypothesis of three different groups of strategies, that this analysis will favor a three component model. And as addition to that expectation, the high time pressure condition will have a smaller BIC for a model with three components than the low time pressure condition. Due to time pressure, those three groups will be more present in the high time pressure condition.
At last, the four different tasks; the flanker task, the lexical decision task, math task and flash task, will be compared by estimating the association between the skewness values of the tasks. Expected is that all four tasks will correlate with each other. There where they all study the accuracy of cognitive functioning under time pressure.
Results
The average percentages correct answers for the hard and easy items and the average reaction time per time pressure condition are shown in Table 1. What indicates that the manipulation of the independent variables; time pressure and different trials, was successful. The average reaction time in the high pressure condition was higher than in the low time pressure condition (2.37 vs. 3.15). The easy trials have a higher percentage of correct responses than the hard trials (70.71 vs. 56.86). A further analysis of the main effects and interaction effects will show if these differences are indeed significant.
Tabel 1
Average reaction time (RT) and standard deviations (between the parenthesis) for the two different trial types (easy and hard) and time pressure conditions (high and low). Followed by the percentage for correct answers for the different trials types and time pressure conditions.
The two-way repeated measures ANOVA revealed that there was a main effect of time pressure , F(1, 29) = 81.17, p < .001 on reaction time. Indicating that the reaction time in the low time pressure condition is significantly higher than in the high time pressure condition. Manipulation of the duration of the time bar influences the reaction time of participants, even when they could still give their answer if the time bar has ended. Next, there was an effect of reaction time on the two different types of trials (hard vs. easy), F(1, 29) = 7.829, p = 0.009. Contrary to the prior hypothesis, the reaction time for hard trials were shorter than for the easy trials (2.67 vs. 2.84). There was no two-way interaction effect of reaction time between the time pressure condition and the trial types, F(1, 29) = 0.009, p = 0.926. Figure 2a shows likewise that there is no interaction effect, there where the two lines do not cross each other. This means that the effect of the variable time pressure does not depend on the type of trial or vice versa.
Figure 1
Deadline (long and short) and trial type (hard and easy) as function of reaction time (A) and accuracy (B), with error bars.
The main effect of accuracy on the time pressure condition was significant as well, F(1, 29) = 5.86, p = 0.02. Participants made significant more trials correct when they were in the low time pressure condition in comparison with the high time pressure condition. Also an effect of accuracy on the type of trials has been found, F(1, 29) = 48.77, p < 0.001. As shown in Tabel 1 and Figure 1b, easy trials had in both time pressure conditions a higher level of accuracy than the hard trials. And again, there is no interaction effect found for accuracy between the time pressure condition and the trial types, F(1, 29) = 0.17, p = 0.683.
The CAF will give an insight on how the duration of reaction time relates to the accuracy of performance within the time pressure conditions and the different trial types. The CAF is shown in Figure 3. First of all, it becomes clear in this figure as well, that the hard trials in both time conditions has a lower percentage of accuracy than the easy trials. Where the accuracy for hard trials in the high time pressure condition even stay around
the guessing point. There is a missing value for the hard trials and high time pressure condition in the figure at a reaction time of 4.5. This point has therefor a percentage correct of zero. Nevertheless, the expected tendency for the three different groups of strategy can be found in the CAF. The trials with earlier reaction times score low on accuracy, where trials with increasing reaction times slowly raises the accuracy and around the deadlines the accuracy for all conditions decreases. What gives evidence for the three different groups of strategies. For the slow reactions times the strategy is mainly guessing resulting in a low accuracy, followed by the strategy of solving what results in increasing accuracy followed by the last strategy; postponing resulting in again a lower accuracy. This lower accuracy is more present in the high time pressure condition than in the low time pressure condition. Indication time pressure indeed influence the accuracy on mathematical tasks. At the longest reaction times, the accuracy on trials increases. What could imply that there may be a fourth strategy group, who are less sensitive to achieve the deadline and more sensitive to give the correct respons instead. The analysis of mixture design could answer this question.
Figure 2
Conditional Accuracy Function (CAF). Mean percentage of correct responses as a function of RT-bin.
How many trials were made at which reaction time can be found in Figure 4. Including a cut-off line for the deadlines in the high and low time pressure condition. The deadline in the high time condition was set to 4 seconds, in the low time pressure
condition the deadline was set on 6 seconds. However, most responses for the trials are given two seconds before the deadline. The reaction time distribution for the high time pressure condition is narrower than the reaction time distribution for the low time pressure condition. Indicating that, as expected, due to the high time pressure the trials made with the postponing strategy need to be made faster.
Figure 3
Distribution of the amount of trials (hard vs. easy) and the corresponding reaction time for each time condition (high time pressure and low time pressure condition and easy or hard trial). With two cut-offs for the deadlines, the deadline in the high time pressure condition was 4 seconds, the deadline in the low time pressure condition was 6 seconds.
The results for the mixture distribution analysis on reaction time for all trials are shown in Figure 4, giving zero to five number of components and the corresponding BIC value. The analysis selected for both time pressure conditions an univariate unequal variance (E) model with three components. As can be found in Figure 4a and 4b, the bic values is the smallest for a model with three components. In addition, the optimal BIC value for the high time pressure condition (BIC = -3581.27), is smaller than the optimal BIC value of the low time pressure condition (BIC = -5045.47). What leads to the conclusion that there are indeed three different groups of strategies in the distribution of reaction time of all participants on all trials and that those three groups are more present in the high time pressure condition.
Figure 4
Analysis of a Mixture distribution in reaction time the low time pressure condition (4a) and the high time pressure condition (4b). Corresponding BIC value for number of components for both
univariate unequal variance (E) and univariate variable variance (V).
B A
When conduction the mixture distribution for every participant separated, results show a different number of components by choosing the best model. Where most
participants have a best fitting model with one component, only three participants had the best fitting model with three different components. The three different groups of strategies are absence in the reaction time of every participant separated, what could indicate that the participants most likely use only one or two strategies to solve the math tasks.
To see whether participants have the same reaction behavior pattern for the four different tasks, the association between the paired samples has been estimated. This is done by conducting a test based on Pearson’s product moment correlation coefficient. Which computes a test of the association value being zero (Best & Roberts, 1975). The test-statistics and p-values are shown in Table 2. To interpret those values, a correction for the problem of multiple comparisons has to be made. The problem of multiple comparison occurs when there is more than one statistical test executed. This leads to an increasing alpha, what on that turn, increases the hazard to find a significant association between two variables, what is actually caused by chance (Miller, 1981). The Bonferroni correction is a method used to counteract the problem of multiple comparisons, whereby each individual hypothesis is tested at a significance level of ⍺ / m, where m stands for the amount of different tests that where conducted (Abdi, 2007). In this case, the significant level is 0.05 / 6 = 0.008.
The correlation between lexical decision task and the flash task, and the math task and the flash task are both significant, P > 0.008. What indicates that there is an association between these tasks and participants are likely to have the same response behavior in those tasks.
Only the flanker task is uncorrelated with any other task, what indicates that the reaction behavior for these trials differ from the other trials. As Colden (2016) stated, this could be due to the short deadlines that were set for both time conditions. It is possible that the participants felt the time pressure in both conditions and therefor more fast responses in the low time pressure condition were found.
Tabel 2
Estimate association between the four different tasks; flanker, lexical decision (LD), math and flash task. Including the degrees of freedom (DF), the test statistic and the p-value.
Discussion
In this study, evidence is found for the influence of time pressure on the accuracy of mathematical skills. The accuracy decreases when time pressure is present. For the
distribution of reaction times for all trials, a mixture design of three components is found. This supports the hypothesis of there being three different groups of strategies for solving the mathematical tasks. However, no evidence is found that this mixture design of reaction time is present for each separated participant. Indicating that participants do not use three different strategies, but choose one or two strategies to solve the mathematical tasks.
There were some limitations within the present study that should be addressed in future attempts. First, the percentage of correct responses for the hard trials is really close to the guessing point (56.86), what may prove that the hard trials were too difficult to make. Regardless from the time pressure condition, very few participants could answer the questions correctly. This may cause a high guessing value for the hard trials, where participants did not even bother to solve the task. This could also be an explanation for
why the average reaction time for the hard trials is lower than the average reaction time for the easy trials. It may be argued that the four chosen topics: addition, subtraction, dividing and multiplying requires a different kind of skill to solve. Where multiplying and dividing is too difficult for most people (Edelman, 1992; Leversen, Haga, Sigmundsson, 2012).
Second, an important issue is the lack of a manipulation check. At the end of the experiment was no questionary to find out if the participants actually felt the differences in the time pressure conditions. Although the responses were faster in the high time pressure condition in comparison with the low time pressure condition. It would be interesting to know if the participants felt the time pressure as well. And if so, in what kind of extent. When the participants did feel the time pressure, it is more likely that they experienced a burden on cognitive load and therefor made more mistakes. If participants did not felt the time pressure but only reacted on the visual faster decreasing time bar, the feeling of a burden on cognitive load will be less present.
Finally, in future research on mathematical tasks and response times, it might be useful to reconsider the deadlines for both time conditions. Participants answered at an average of two seconds before the deadline was set. Further research could explain if a shorter deadline would give the expected reaction times, which will be around the
deadline. Or if by adjusting the deadline the reaction behavior will maintain the same, and participants will respond even faster than they did in this study.
References
Abdi, H. (2007). Bonferroni and Šidák corrections for multiple comparisons. In Salkind, N. J. Encyclopedia of Measurement and Statistics. Thousand Oaks, CA: Sage.
Baroni, G., Pellicano, A., Lugli, L., Nicoletti, R., & Proctor, R. W. (2012). Influence of Temporal Overlap on Time Course of the Simon Effect. Experimental Psychology, 59(2), 88-98. doi:10.1027/1618-3169/a000130
Best, D. J., & Roberts, D. E. (1975). Algorithm AS 89: The Upper Tail Probabilities of Spearman's rho. Applied Statistics, 24, 377–379.
Butterworth, B. (2005). The development of arithmetical abilities. J Child Psychol & Psychiat Journal of Child Psychology and Psychiatry, 46(1), 3-18. doi:10.1111/j.
1469-7610.2004.00374.x
Colden, A. (2016). The effects of urgency on reaction time and accuracy in perceptual tasks. University of Amsterdam, bachelor thesis.
Degenhardt-Burke, C. (2010). New strategies to boost your career. On Wall Street, 51-52. Edelman, G. M. (1992). Bright air, brilliant fire: on the matter of the mind. New York: Basic Farley, C., & Raftery, A. E. (2006). Mclust version 3 for R: Normal Mixture Modeling and
Model-Based Clustering. National Institutes of Health.
Kevenaar, S. (2016). De effecten van tijdsdruk op cognitieve beslistaken. University of Amsterdam, Bachelor thesis.
Leversen, J. S., Haga, M., & Sigmundsson, H. (2012). From Children to Adults: Motor Performance across the Life-Span. PLoS ONE, 7(6). doi:10.1371/journal.pone. 0038830
Maanen, L., Jong, R. D., & Rijn, H. (2014). How to Assess the Existence of Competing Strategies in Cognitive Tasks: A Primer on the Fixed-Point Property. PLoS ONE, 9(8). doi:10.1371/journal.pone.0106113
Miller, R. G. (1985). Simultaneous statistical inference. New York: Springer-Verlag. Noda, M., Takai, J., & Yoshida, T. (2007). The Influence of Time Pressure on
Mood-Congruent Effects: Evaluating Products With Limited Information. Journal of Applied Social Psychology, 37(2), 403-425. doi:10.1111/j.0021-9029.2007.00166.x
Perlow, L. A. (1999). The Time Famine: Toward a Sociology of Work Time. Administrative Science Quarterly, 44(1), 57. doi:10.2307/2667031
Posamentier A. S. and Krulik S. (1998). Problem-Solving Strategies for Efficient and Elegant Solutions. A Research for the Mathematics Teacher, California: Corwin Press, Inc.
Reinagel P. (2013). Speed and accuracy of visual motion discrimination by rats. PLoS ONE, 8.
Sigmundsson, H., Polman, R. C., & Lorås, H. (2013). Exploring Individual Differences In Children's Mathematical Skills: A Correlational And Dimensional Approach 1. Psychological Reports, 113(1), 23-30. doi:10.2466/04.10.pr0.113x12z2
Stallings, L. L. (2007). See a different mathematics. Mathematics Teaching In the Middle School, 13(4), 212-217.
Stins, J. F., Polderman, J. C., Boomsma, D. I., & Geus, E. J. (2007). Conditional accuracy in response interference tasks: Evidence from the Eriksen flanker task and the spatial conflict task. Advances in Cognitive Psychology, 3(3), 409-417. doi:10.2478/
v10053-008-0005-4
Svenson, O., & Maule, A. J. (1993). Time pressure and stress in human judgment and decision making. New York: Plenum Press.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6(2), pp. 461-464
Turner, K. (2016). France Might pass a law that makes it illegal to send after-hours work emails. The Washington Post. From https://www.washingtonpost.com/news/the- switch/wp/2016/05/12/france-might-pass-a-law-that-makes-it-illegal-to-send-after-hours- work-emails/.
White, C. N., Ratcliff, R., & Starns, J. J. (2011). Diffusion models of the flanker task: Discrete versus gradual attentional selection. Cognitive Psychology, 63(4), 210-238. doi:
