The effect of time pressure on the speed-accuracy trade-off during math test

The$effect$of$time$pressure$on$the$speed1accuracy$trade1off$during$a$

math$test$

Rekentuin,'to'test'children'on'their'math'performance'in'a'high?'and'low?pressure'

situation.' This' is' compared' with' the' math' performance' test' “Tempo' Toets' Rekenen”'(TTR)'and'additionally'a'structured'interview'was'held.'The'hypothesis' is'that'the'effect'of'time'pressure'differs'between'high'and'low'math'performers.' However,' the' results' did' not' show' any' effect' of' time' pressure' on' accuracy' and' response' time.' The' interview' revealed' that' the' children' we're' not' distracted' or' stressed'due'to'time'pressure.'Nonetheless,'the'study'shows'some'directions'for' future'research.''

! 2! ! Since! the! development! of! modern! ICT! resources,! some! primary! schools! in! the! Netherlands!have!adopted!online!educational!learning!environments!for!children.!One!of! the! many! advantages! of! an! online! learning! environment! is! the! opportunity! to! automatically!adapt!the!level!of!training!to!the!child’s!ability.!“Rekentuin”!is!an!example! of! an! adaptive! learning! and! testing! environment! and! is! developed! by! the! company! “Oefenweb”! (Van! der! Maas,! Klinkenberg,! &! Straatemeier,! 2010).! ! Rekentuin! provides! children! the! opportunity! to! practice! their! math! skills!through! games.! An! advantage! of! this! practice! method! is! that! children! enjoy! these! games,! which! may! lead! to! more! motivation.!!

! Another!advantage!of!online!testing!is!the!ability!to!measure!the!response!time! during! the! test.! In! Rekentuin! the! score! is! a! weighted! combination! of! the! accuracy! and! speed! of! the! response.! Speed! and! accuracy! can! be! modelled! with! a! speedRaccuracy! tradeRoff! (Van! der! Linden,! 2007).! This! tradeRoff! describes! a! negative! correlation! between! speed! and! accuracy,! where! more! speed! is! associated! with! less! accuracy.! The! scoring!rule!of!Rekentuin!takes!this!tradeRoff!into!account!and!is!presented!in!Figure!1! (Klinkenberg,!2014;!van!der!Maas!&!Maris!(2012).!!

!

Figure! ! 1! High! speed,! high! stakes! scoring! rule,! where! d! is! the! time! limit! and! a! is! a! scaling! factor! (Klinkenberg,!2014;!Van!der!Maas!&!Maris!(2012).!

Figure 1: High speed high stakes scoring rule, where d is the time limit and a is a scal-ing factor.

The high speed high stakes scoring rule thereby offers a solution for guessing as well as the speed accuracy problem. Given that the certainty about an answer increas-es with time (an assumption in almost all decision theoriincreas-es), there is an optimal mo-ment for actually responding. Interestingly, Maris and Van der Maas (2005) have proven, providing that the scoring rule is a sufficient statistic for measuring ability, that the model for the probability for answering correctly is identical to the most fre-quently used model in assessment, namely the two parameter logistic model (Van der Linden & Hamleton, 1997). The discrimination parameter is shown to be equal to the time limit d for the item. This elegant result offers many opportunities for, among others, adaptive testing with reaction times. Maris & van der Maas (2011) have to this end derived all relevant conditional probability distributions.

It is still relevant that respondents understand the rule. Through digital assessment this rule can easily be visualized. Figure 2 shows an implementation of this rule in the

!

! While!playing!a!game,!the!child!can!earn!a!maximum!of!20!coins!or!a!minimum!of! R20!for!each!item.!The!child!can!see!the!coins!and!with!each!second!a!coin!disappears.! Another!aspect!of!this!rule!is!punishing!an!incorrect!answer!with!losing!coins!to!prevent! gambling.!The!children!are!aware!of!the!“High!speed,!high!stakes”!scorings!rule,!because! the! faster! he/she! answers,! the! more! coins! the! child! gets.! The! scoring! rule! per! item! proposed! by! Maris! and! Van! der! Maas! (2012)! is:! (acc! *2R1)! *! (dRRT),! where! accuracy! (acc)!is!0!(incorrect)!or!1!(correct).!d!is!the!time!limit,!which!is!20!seconds!in!Rekentuin! and!RT!is!the!speed!of!the!child!to!give!an!answer.!A!fast!wrong!answer!will!result!in!a! very! negative! score! (Figure! 1).! The! advantage! of! taking! response! time! into! account! is! that!it!provides!much!more!information!about!the!underlying!ability!of!the!child,!even! when!the!items!are!easy.!!

! However,!children!and!primary!schools!have!complained!about!the!stress!caused! by! the! time! pressure.! The! teachers! of! the! children! have! complained! that! the! coins! counting!down!distract!the!child!from!the!tasks!and!evoke!stress,!especially!with!the!less! skilled!children.!This!suggests!that!stress!induced!by!time!pressure!could!influence!the! math!performance.!For!this!study!it!is!interesting!to!examine!this!effect!of!time!pressure! on!analysing!and!measuring!math!performance.!!

! Research! shows! that! a! stressful! environment! can! adversely! affect! the! math! performance!of!people!(Beilock!&!Carr,!2005;!Beilock,!2008).!!Although!some!individuals! maybe! motivated! by! stress! to! perform! well,! other! individuals! may! perform! at! their! worst.! This! is! based! on! Distraction2theories,! which! suggest! that! pressure! occupies! the! working!memory!with!thoughts!about!the!situation!and!its!importance!instead!of!paying! attention! to! the! execution! (Beilock,! 2004;! Lewis! &! Linder,! 1997).! A! stressful! environment!places!individuals!on!a!dual!task:!execution!of!the!task!and!worries!about!

the! performance.! Recently,! researchers! in! cognitive! and! social! psychology! have! found! that!especially!the!high!performers!(HPs),!based!on!their!working!memory!capacity,!are! influenced! negatively! by! pressure! on! their! accuracy! and! response! time,! while! the! low! performers! (LPs)! do! not! suffer! under! pressure! (Beilock! &! Carr,! 2005;! Beilock,! 2008;! Ashcraft! &! Kirk,! 2001).! In! the! study! of! Beilock! (2005)! participants! were! tested! on! a! computer! with! low! and! high! demanding! math! problems.! ! The! participants! completed! this! task! in! either! a! lowR! or! a! highRpressure! test! environment.! The! highRpressure! environment!was!created!in!the!form!of!monetary!incentives,!peer!pressure!and!social! evaluation.!The!results!of!the!study!showed!that!especially!the!high!performers!“choked! under!the!pressure”!in!the!high!demanding!math!problems.!!

! If! pressure! targets! especially! HPs,! it! would! have! serious! implications! for! interpreting! the! performance! with! highRstakes! tests.! It! would! suggest! that! individuals! with!the!highest!competence!in!math!are!the!ones!that!will!not!succeed!under!pressure.! Secondly,! these! results! raise! some! interesting! questions,! because! it! contradicts! what! teachers! say! about! Rekentuin.! They! say! especially! that! children! with! low! performance! are! distracted! under! pressure.! This! would! suggest! there! is! a! different! effect! of! time! pressure!on!accuracy!and!reaction!time!with!lowR!and!highRlevel!performers.!!

! Also,!research!shows!a!different!effect!of!time!pressure!between!genders.!While! women! tend! to! underperform! the! men! in! a! highRpressure! math! test,! women! greatly! increase!in!performance!in!a!lowRpressure!math!test,!such!that!they!outperform!the!men! (Shurchkov,! 2012;! Niederle! &! Vesterlund,! 2010).! This! is! especially! found! on! difficult! math!tests!and!not!on!easy!tests.!

! ! !

THE'CURRENT'EXPERIMENT'

In!this!study,!we!will!explore!what!the!effect!is!of!time!pressure!on!the!speed!accuracy! tradeRoff! during! a! math! test.! To! answer! this! question! the! math! skills! of! children! from! different!primary!schools!have!been!tested.!The!test!was!made!up!of!three!parts.!!First!to! validate! the! ability! of! the! children’! math! skills,! a! test! called! “! Tempo! Test! Rekenen”! (TTR)!is!done.!The!second!test!is!an!online!game!that!resembles!Rekentuin,!where!the! children!have!to!sum!two!numbers!and!get!rewarded!with!coins!for!a!correct!answer.! The! same! High2Speed9High2Stakes! rule! as! in! Rekentuin! is! applied,! but! the! children! will! perform!under!both!lowRpressure!and!highRpressure!situations.!The!last!part!of!the!test! is! a! structured! interview! with! the! children! individually! where! they! were! asked! about! their! experience! with! the! tests.! With! these! three! parts! combined,! the! effects! of! time! pressure!can!be!investigated.!!

! Firstly,!in!this!study!will!be!examined!whether!the!participants!are!aware!of!the! coins! counting! down! in! the! highRpressure! condition.! This! will! be! checked! in! the! structured!interview.!If!the!children!don’t!see!the!difference,!we!can’t!conclude!that!they! are!aware!of!the!time!during!the!test.!!We!expect!that!the!children!are!aware!of!whether! the!coins!are!on!the!screen!or!not.!The!second!question!is!whether!time!pressure!has!an! effect!on!the!accuracy!and!response!time!performance.!The!hypothesis!is!that!there!will! be!a!difference!in!accuracy!and!response!time!between!the!conditions.!! ! !To!further!analyse!the!effect!of!time!pressure!on!accuracy!and!response!time,!the! last! research! subject! will! contain! the! influence! of! math! performance.! The! first! hypothesis,!based!on!the!complaints!of!the!teachers,!is!that!children!with!a!lower!score! on!the!math!test!will!perform!significantly!worse!on!their!response!time!and!accuracy! when! there! is! time! pressure.! In! the! second! hypothesis,! based! on! the! literature,! it! is! expected! that! people! with! high! scores! on! math! will! be! influenced! negatively! by! time!

pressure!on!their!response!time!and!their!accuracy.!The!children!with!a!lower!score!on! the! math! test! will! not! perform! significantly! worse! on! their! response! time! and! their! accuracy.!!

! Additionally! some! possible! individual! differences! will! be! investigated,! like! age! and! gender! (Klinkenberg,! 2014).! We! expect! that! there! will! be! a! significant! effect! for! gender! associated! with! accuracy! in! accordance! with! the! literature! that! women! underperform!in!stressful!environments.!!

! At! last! an! overall! analysis! will! be! conducted! with! all! the! conditions! of! the! experiment!on!accuracy,!response!time!and!on!the!score!based!on!the!‘High!Stakes,!High! Speed’!scoring!rule!used!in!Rekentuin.!!!The!first!two!conditions!of!the!experiment!are! coins!on!the!screen!or!no!coins,!the!main!focus!of!this!study.!The!other!two!conditions! contain!different!scoring!rules:!A!wrong!answer!is!sanctioned!with!a!loss!of!coins!and! with!the!other!rule!a!wrong!answer!is!not!sanctioned.!!

! In! this! study! the! aim! is! to! investigate! the! effect! of! time! pressure! on! the! performance! of! children! during! a! math! test.! From! a! psychometric! point! of! view! it! is! interesting!to!see!how!the!child’s!ability!can!be!reflected!best.!Also,!the!study!can!give!! some! insights! in! how! to! improve! Rekentuin,! so! children! will! be! motivated! to! keep! playing.! ! ! ! ! ! ! !

METHOD' Participants''

Data! from! 197! children,! aged! eight! till! thirteen,! of! four! Dutch! primary! schools! from! grades! four! till! eight! were! analysed.! At! the! primary! schools! that! used! Rekentuin! and! participated! in! the! experiment,! both! teachers! and! children! had! complaints! concerning! the! coins.2 For! this! study! passive! informed! consents! were! sent! to! the! parents! of! the! children.! Participants! were! divided! into! an! LP! group! (N=100)! and! HP! group! (N=90),! using! a! median! split! of! the! average! of! their! scores! on! the! TTR! per! grade! (Median:! 8th_!

Grade=26,!7th_{!Grade=!25,!6}th_{!Grade=!23.5,!5}th_{!Grade=!17,!4}th_{!Grade=!13,!3th!Grade=!11).!!} ' Materials' The!“Tempo!Test!Rekenen”!(TTR)!was!used!in!this!study!to!test!the!math!ability!of!the! participants.!This!test!requires!participants!to!complete!as!much!math!items!as!possible! in!one!minute,!for!example!“5+7=!“.!The!TTRRscore!is!based!on!the!total!of!the!correct! answers!(range!0R50).! ! Table!1.!Four!games!of!the!computer!test!with!the!time!pressure!conditions!and!the!scorings!rule! conditions.!The!games!are!presented!in!random!sequence!per!participant!

Game' Time'Pressure' Scorings'rule'

1' Coins!on!the!screen! Penalty! 2' No!coins!on!the!scree! Penalty! 3' Coins!on!the!screen! No!penalty! 4' No!coins!on!the!screen! No!penalty! ! ! !

! The!computer!test!is!built!in!the!experimental!control!system!“PsychoPy”!(Peirce,! 2007)! written! in! the! programming! language! Python! (Van! Rossum,! 2003).! The! test! consists! of! four! games:! Two! of! them! have! coins! counting! down! on! the! screen! (time! pressure! condition,! see! Table! 1! and! Figure! 2)! and! two! games! have! no! coins! on! the! screen!(no!pressure!condition).!!Another!scorings!rule!is!investigated!in!the!four!games:! two!of!the!games!punish!a!wrong!answer!by!reducing!the!coins!(penalty!condition)!and! two!of!them!don’t!punish!a!wrong!answer!(no!penalty!condition).!The!four!games!with! different! conditions! are! presented! in! a! random! sequence! per! participant.! Also! the! 20! items!in!each!games!increase!in!difficulty!from!easy!to!difficult.!The!item!difficulties!are! based!on!the!responses!of!all!children!that!play!Rekentuin.!!

!

Figure! 2.! A! screenshot! of! the! computer! test! built! in! PsychoPy.! The! coins! can! be! seen! on! the! screen! counting!down!each!second.! ! ! As!mentioned!before,!after!the!computer!test!a!structured!interview!was!used!to! ask!about!the!experiences!of!the!children!with!the!test!and!to!verify!whether!they!were! aware!of!the!differences!between!the!games.!!The!researcher!asked!question!about!the! coins!on!the!screen!and!about!the!scorings!rule,!for!example!‘were!the!coins!always!on! the!screen!during!the!games?’!and!‘Do!you!like!the!coins!on!the!screen?’.!Based!on!the!

reactions!of!the!participants!the!researcher!scored!from!1!till!4!(1:!Yes,!2:!Neutral,!3:!No,! 4:! Don’t! Know).! ! The! degree! of! awareness! of! the! coins! on! the! screen! is! scored! with! 1! (Yes)!or!2!(No).!!

!

Procedure'

Participants! were! tested! individually! on! a! computer! in! groups! of! ten! at! their! own! primary! school! in! a! quiet! environment! apart! from! their! classroom.! The! participants! were! first! asked! to! fill! in! their! gender,! age! and! grade.! Secondly,! they! were! instructed! about!the!TTR!test,!to!answer!as!much!math!items!in!one!minute!on!paper.!After!the!TTR! test!they!were!instructed!about!the!computer!test.!The!differences!between!the!games! were!explained!beforehand.!Each!game!also!started!with!an!instruction!on!the!screen,! for!example!“The!coins!can!be!seen!on!the!screen.!The!coins!will!decrease!with!a!wrong! answer.!Click!on!the!spacebar!to!continue”.!!After!the!computer!test!participants!were! randomly!asked!for!an!interview.!The!interview!was!held!in!a!separate!room!apart!from! the!other!participants.!! ! RESULTS' 86%!of!the!participants!of!the!interviewees!reported!to!have!seen!the!difference!in!coins! on!the!screen.!Only!22%!of!the!participants!stated!in!the!interview!to!be!distracted!by! the!coins!and!16%!were!stressed!because!the!coins!were!on!the!screen.!And!more!than! half!of!the!participants!(54%)!would!like!to!keep!the!coins!on!the!screen!in!“Rekentuin”.!! ! Combining!all!the!participants,!the!results!from!the!computer!test!indicate!a!nonR significant!difference!in!accuracy!between!the!timeRpressure!(M=!30.14,!SD=!7.39)!and! no! timeRpressure! condition! (M=! 29.78,! SD=! 7.47),! t2 (174)! =! 1.34,! p=! .18.! ! Also,! the!

response! time! is! not! significantly! different! between! the! timeRpressure! (M=3.86,! SD=! 0.54)!and!no!time!pressure!condition!(M=23.82,!SD=0.53),!t2(172)=!1.85,!p=!.07.!!!

!

Table! 2.! Means! of! the! response! time! (log! transformed)! and! accuracy! in! the! performance! group! (low! performance,!high!performance)!and!the!time!pressure!conditions!(coins,!no!coins)!of!the!computer!test! ! Performance! Group! Time! Pressure! Conditions!

!Response! time! ! !!!!!Accuracy! ! !

! ! Mean!! SD2 N2 Mean! SD2 N2 LP! Coins! 4.03! .53! 91! 28.16! 8.11! 90! ! No!Coins! 3.97! .53! 89! 27.93! 7.81! 90! HP! Coins! 3.66! .50! 79! 32.49! 5.90! 79! ! No!Coins! 3.66! .48! 79! 32.32! 6.12! 79! ! ! Additionally,!a!2x2!factorial!mixed!ANOVA!was!conducted!on!response!time,!with! time!pressure!(coins,!no!coins!condition)!as!a!within!subjects!variable!and!performance! (LP,!HP!group)!as!the!independent!between!subjects!variable.!!The!main!effect!of!time! pressure!on!response!time!was!not!significant,!F!(1,334)=!0.267,!p2=!.61!(Table!2).!The! main!effect!of!performance!on!response!time!was!significant,!F2(1,334)=!36.108!p2<!.00.! The!response!time!was!higher!in!the!LP!group!than!in!the!HP!group!(Table!2,!Figure!3).! The!interaction!effect!between!time!pressure!and!performance!was!not!significant,!! F2(1,334)=!0.219,!p=!.64.!!

! ! Accuracy!on!the!computer!test!was!also!analysed!in!a!2x2!factorial!mixed!ANOVA! with! time! pressure! (coins,! no! coins! condition)! as! a! within! subjects! variable! and! performance! (HP,! LP! group)! as! a! between! subjects! variable.! The! assumption! of! homogeneity! of! variance! was! moderately! violated,! F! (3,334)=2.75,! p=! .04.! The! main! effect!of!time!pressure!on!accuracy!was!not!significant,!F!(1,336)=!.127,!p=!.72!(Table!2).!! The!main!effect!of!performance!on!accuracy!was!significant,!F2(1,336)=!33.001!p2<!.00.! The!accuracy!was!higher!in!the!HP!group!than!in!the!LP!group!(Table!2,!Figure!4).!The! interaction! effect! between! time! pressure! and! performance! was! not! significant,! F! (1,334)=!.001,!p=!.98.!!!

Figure!3:!!Mean!response!time!(s)!of!the!HP!(N=90)!and!LP!group!(N=100)!in!the!coins!and!no!coins! condition.!!The!error!bars!shown!are!standard!deviation.!

!

Figure!4.'Mean!accuracy!in!number!corrects!of!the!computer!test!of!the!HP!(N=90)!and!LP!group!(N=100)! in!the!coins!and!no!coins!condition.!The!error!bars!shown!are!standard!deviations.!!

! Furthermore,!some!individual!differences!in!response!time!and!accuracy!with!age! and! gender! as! covariates! have! been! investigated! separately! in! 2x2! factorial! within! ANCOVA’s.!The!covariate,!gender,!was!not!significantly!related!to!response!time,22 F2(1,332)!=!.282,!p=!.30.!The!interaction!effect!between!gender!and!performance!was!not! significant!for!response!time,!F!(1,332)=!.139,!p=!.71.!There!was!also!a!nonRsignificant! relation!between!gender!and!accuracy,!F!(1,!332)!=!.576,!p=!.45.!No!interaction!effect!has! been!found!between!gender!and!performance,!F!(1,332)=!.004,!p=!.95.!!! ! The!covariate,!age,!was!significantly!related!to!response!time,!F2(1,336)=!49.607,! p<!.00!(Figure!5).!Likewise,!accuracy!related!significantly!to!the!covariate!age,2F!(1,336)=! 100.45,! p<! .00! (Figure! 6).! The! interaction! effect! between! age! and! performance! for! response! time! was! nonRsignificant,! F2 (1,336)=! .004,! p=! .90.! Regarding! accuracy,! the! interaction!effect!between!age!and!performance!was!also!not!significant,!!

!

Figure!5.!!Mean!response!time!(s)!measured!against!age!in!the!coins!and!no!coins!conditions.!The!error! bars!shown!are!standard!deviations.!

!

Figure! 6.! Mean! number! corrects! of! the! computer! test! measured! against! age! in! the! coins! and! no! coins! conditions.!The!error!bars!shown!are!standard!deviations.!' !

!

Table!3.!Means!of!the!response!time!(log!transformed),!accuracy!and!score!based!on!the!‘High!Stakes,!High! Speed’! scorings! rule! in! the! scorings! rule! (penalty,! no! penalty)! against! time! pressure! (coins,! no! coins)! conditions!of!the!computer!test! ! ! ! ! Finally,!three!repeated!measures!ANOVA’s!were!conducted!to!compare!the!effect! of!the!time!pressure!and!the!scorings!rule!conditions!on!accuracy,!response!time!and!on! the!score!based!on!the!‘High!Stakes,!High!Speed’!scoring!rule!used!in!Rekentuin!(Table! 3).! Firstly,! there! was! a! nonRsignificant! main! effect! of! time! pressure! on! accuracy,! F2 (1,196)=! 1.804,! p=! .18.2 We! found! a! nonRsignificant! main! effect! of! the! scoring! rule,! F! (1,196)=!.118,!p=!.73.!The!interaction!effect!between!the!scoring!rule!and!time!pressure! on! accuracy! was! significant,! F2(1,196)=! 4.718,! p=! .03! (Figure! 7).! ! A! combination! of! no! penalty! and! coins! resulted! in! the! highest! mean! of! accuracy! and! a! combination! of! no! penalty!and!no!coins!in!the!lowest!mean!of!accuracy!(Table!3).!!

Scorings! rule!

Time!Pressure!! !Response! Time! !!!!!Accuracy! ! Score! !

! ! Mean!! SD2 Mean! SD2 Mean! SD2

Penalty! Coins! 1.91! .30! 14.91! 3.85! 9.05! 3.53!

! No!Coins! 1.94! .30! 15.00! 3.94! 9.25! 3.60!

No!Penalty! Coins! 1.95! .30! 15.23! 3.90! 9.25! 3.62!

!

Figure!7.'Mean!number!corrects!of!the!computer!test!measured!with!the!no!penalty!and!penalty!scorings! rule!in!the!coins!and!no!coins!condition.!The!error!bars!shown!are!standard!deviations.!!

' '

' Secondly,! a! repeated! measures! ANOVA! was! also! performed! on! response! time.!

The!main!effect!of!the!scoring!rule!was!nonRsignificant,!F2(1,172)=!.419,!p=!.52.!The!main! effect!of!time!pressure!was!also!nonRsignificant,!F2(1,172)=!3.399,!p=!.07.!!However,!the! interaction! effect! of! time! pressure! and! the! scorings! rule! on! response! time! was! significant,!F2(1,172)=!13.783,!p!<!.00!(Figure!8).!!Just!as!accuracy,!a!combination!of!no! penalty!and!coins!resulted!in!the!highest!mean!of!response!time!and!a!combination!of!no! penalty!and!no!coins!in!the!lowest!mean!of!response!time!(Table!3).!

!

Figure!8.'Mean!response!time!(s)!measured!with!the!no!penalty!and!penalty!scorings!rule!in!the!coins!and! no!coins!conditions.!The!error!bars!shown!are!standard!deviations.!

! !

! At! last,! the! score! based! on! the! ‘High! Stakes,! High! Speed’! scoring! rule! used! in!

Rekentuin!was!also!analysed!with!a!repeated!measures!ANOVA.!The!main!effect!of!the! scoring! rule! was! nonRsignificant,! F2 (1,174)=! 1.183,! p=! .28.! The! main! effect! of! time! pressure! was! also! nonRsignificant,! F2 (1,174)=! 1.229,! p=! .27.! ! The! interaction! effect! of! time!pressure!and!the!scorings!rule!on!response!time!was!also!nonRsignificant,!!

! Figure!9.'Mean!score!computer!test!measured!with!the!no!penalty!and!penalty!scorings!rule!in!the!coins! and!no!coins!conditions.!The!error!bars!shown!are!standard!deviations.' ! ' DISCUSSION' In!this!study!we!examined!the!effect!of!time!pressure!on!accuracy!and!response!time!of! math! performance,! specifically! with! children! that! play! Rekentuin.! ! The! results! of! the! computer!test!did!not!show!any!effect!of!time!pressure!on!accuracy!and!response!time.! The! hypothesis! of! a! difference! in! accuracy! and! response! time! between! the! conditions! can!be!rejected.!An!analysis!over!all!the!conditions!did!show!interaction!effects!between! the!scorings!rule!and!time!pressure!on!response!time!and!accuracy.!A!combination!of!no! penalty!and!time!pressure!resulted!in!the!highest!mean!of!accuracy!and!response!time! and!a!combination!of!no!penalty!and!no!time!pressure!in!the!lowest!mean!of!accuracy! and!response!time.!However,!any!effect!disappeared!when!we!analysed!the!score!used!in!

Rekentuin,!where!accuracy!and!response!time!is!combined!following!the!formula:!(acc! *2R1)!*!(dRRT)!(Maris!&!Van!der!Maas,!2012).!The!interaction!effects!can!be!explained! due!to!the!speedRaccuracy!trade!off!(Van!der!Linden,!2007).!! ! Also!the!performance!groups,!based!on!their!TTR!score,!analysed!separately!did! not!yield!a!significant!effect.!We!expected!an!increase!in!response!time!in!the!LP!group! in!the!coins!condition!and!a!decrease!in!accuracy!in!the!HP!group!in!the!coins!condition.! The! results! reveal! that! accuracy! increased! in! both! the! performance! groups! with! the! coins!condition.!! ! However,!the!direction!of!the!effects!was!in!accordance!with!the!hypothesis!for! the!LP!group!in!the!response!time,!where!the!response!time!of!the!LP!was!higher!when! the!coins!were!on!the!screen.!This!is!consistent!with!the!complaints!of!the!teachers!that! it’s!especially!the!low!level!performers!who!are!distracted!by!the!coins.!The!interview! results!showed!that!the!children!were!aware!of!the!coins,!but!most!of!them!stated!they! were!not!stressed!or!distracted!by!the!coins.!Additionally,!more!than!half!of!the!children! would!choose!to!keep!the!coins!on!the!screen!in!Rekentuin.!! ! However,!literature!(Beilock,!2005;!2008;!Ashcraft!et!al,!2001;!Lewis!et!al,!1997;! Shurchkov,! 2012)! showed! an! effect! of! time! pressure! on! math! performance,! while! this! study!has!not!found!a!significant!effect.!This!discrepancy!could!be!due!to!the!difficulty!of! the!math!ability.!The!literature!revealed!that!time!pressure!has!the!biggest!effect!on!high! demanding!math!problems.!For!most!of!the!children!math!sums!are!not!that!difficult!and! are!used!to!doing!this.!!Another!explanation!could!be!that!the!level!of!time!pressure!in! the! computer! test! was! not! large! enough! for! the! children! to! have! an! effect! on! the! performance.!

! Furthermore,! the! computer! test! was! a! good! indication! of! math! performance! in! accordance! with! TTR.! Both! main! effects! of! performance! group! were! significant! in! the!

2x2! designs! for! accuracy! en! response! time.! However,! TTR! as! an! indication! of! math! performance! has! it’s! own! time! pressure,! and! this! aspect! can! have! an! influence! on! the! comparison!with!the!computer!test.!That!is!why!we!chose!not!to!consider!performance! level!as!a!continuous!variable,!but!to!use!TTR!to!split!two!performance!groups.!!

! Also,! the! data! for! accuracy! did! not! meet! the! assumption! of! homogeneity! in! variances.!This!causes!the!F!values!to!be!slightly!affected,!so!the!results!with!measures!of! accuracy!should!be!analysed!with!coercion.!The!cause!of!the!different!sample!sizes!in!the! conditions!is!due!to!missing!values!and!the!median!split!of!TTR!choice,!where!scores!on! the!exact!median!have!been!put!in!the!high!performers!group.!!

! In!short,!this!study!has!not!showed!any!effect!of!time!pressure!that!distracts!or! stresses! math! performers.! Following! the! results! of! this! study! there! doesn’t! seem! any! reason! for! Rekentuin! to! be! concerned! about! the! effect! of! the! coins! on! the! screen! with! regard! to! measuring! the! child’s! ability.! Future! research! could! use! another! math! performance!test!measured!without!time!pressure!to!compare!with!the!computer!test.!It! could! be! interesting! to! use! TTR! as! a! continuous! variable! in! the! comparison.! Also,! this! test!showed!that!the!children!we’re!not!so!much!distracted!or!stressed!by!the!coins!on! the! screen.! By! creating! higher! stakes! test! and! more! time! pressure! a! potential! effect! might! be! detected.! In! following! research! not! only! visual! aspects! but! also! audio! visualisation!can!be!used!to!represent!counting!each!second.!By!understanding!more!of! the!effect!of!time!pressure!on!ability,!we!will!be!more!equipped!to!analyse!and!measure! math!performance!and!keep!children!motivated!for!learning.!! ! ! ! !

REFERENCES' '

Ashcraft,! M.H.,! &! Kirk,! E.P.! (2001).2 The2 relationships2 among2 working2 memory,2 math2 2 anxiety,2 and2 performance.! Journal! of! Experimental! Psychology:! General,! 130,!

! 224–237.!

Beilock,!S.!L.,!&!Carr,!T.!H.!(2005).2When2high9powered2people2fail2working2memory2and2 2 “choking2under2pressure”2in2math.2Psychological!Science,!16,!101R105.!

Beilock,! S.! L.! (2008).! Math2 performance2 in2 stressful2 situations.! Current! Directions! in! ! Psychological!Science,!17,!339R343.! Dennis,!I.,!&!Evans,!J.!S.!B.!(1996).!The2speed9error2trade9off2problem2in22 2 psychometric2testing.!British!Journal!of!Psychology,!87,!105R129.! Henmon,!V.!A.!C.!(1911).!The2relation2of2the2time2of2a2judgment2to2its2accuracy.!! ! Psychological!Review,!18,!186.! Phillips,!L.!H.,!&!Rabbit,!P.!M.!(1995).2Impulsivity2and2speed9accuracy2strategies2in22 2 intelligence2test2performance.!Intelligence,!21,!13R29.! Klinkenberg,!S.!(2014).2High2Speed2High2Stakes2Scoring2Rule.!In!Computer!Assisted!! ! Assessment.! Research! into! ERAssessment! (pp.! 114R126).! Springer! International! ! Publishing!

Lewis,!B.!P.,!&!Linder,!D.!E.!(1997).!Thinking2about2choking?2Attentional2processes2and22 2 paradoxical2 performance.! Personality! and! Social! Psychology! Bulletin,! 23,! 937R

! 944.!

Maris,!G.,!&!van!der!Maas,!H.!(2012).!Speed9accuracy2response2models:2Scoring2rules22 2 based2on2response2time2and2accuracy.!Psychometrika,!77,!615R633!

2 model2 can2 account2 for2 the2 accuracy2 and2 reaction2 time2 of2 value9based2 choices2 2 under2high2and2low2time2pressure.!Available!at!SSRN!1901533! Niederle,!M.,!&!Vesterlund,!L.!(2010).!Explaining2the2gender2gap2in2math2test2scores:2The22 2 role2of2competition.!The!Journal!of!Economic!Perspectives,!129R144.! Van!Rossum,!G.,!&!Drake,!F.!L.!(2003).!Python2language2reference2manual2(p.2144).22 ! Network!Theory.! Peirce,!J.!W.!(2007).!PsychoPy—psychophysics2software2in2Python.!Journal!of!! ! neuroscience!methods,!162,!8R13.! Shurchkov,!O.!(2012).2Under2pressure:2gender2differences2in2output2quality2and2quantity22 2 under2 competition2 and2 time2 constraints.! Journal! of! the! European! Economic! ! Association,!10,!1189R1213.!

Van!der!Maas,!H.,!Klinkenberg,!S.,!&!Straatemeier,!M.!(2010).!Rekentuin.2nl:2Combinatie22 2 van2oefenen2en2toetsen.2Examens,!10R14!

Van!der!Linden,!W.!J.!(2007).2A2hierarchical2framework2for2modeling2speed2and2accuracy22 2 on2test2items.!Psychometrika,!72,!287R308!

Wickelgren,! W.! A.! (1977).! Speed9accuracy2 tradeoff2 and2 information2 processing2 2 dynamics.!Acta!psychologica,!41,!67R85.!