Strategies in the Number Game
R.E.L. Hetem∗University of Amsterdam rogierhetem@gmail.com
Abstract
This study examines whether items from the Number Game compatible with the Proximity heuristic are easier and whether this difference in difficulty disappears for the participants that are trained in the Working Backward heuristic (N=56). Nineteen participants were trained in the Working Backward heuristic, nineteen were trained in the Proximity heuristic and the others were not trained. The items compatible with the Proximity heuristic were easier. This difference in difficulty between the two item types did not disappear after a Working Backward training. These results show that the difference in difficulty is partly but not entirely due to the use of strategies.
I. Introduction
T
he number game is presented in the ed-ucational web environment "The Math Garden". The task in this game is to form a goal-number with a set of start-numbers. Ev-ery start-number (say, 2, 3, 5) must be used exactly once. By using the four arithmetic op-erators (addition, subtraction, multiplication and division) one can form the goal-number (say, 11). There are no restrictions placed on the use of these arithmetic operations (solu-tion: 2×3+5 = 11). In this bachelor thesis I discuss our attempt to test the hypothesis, derived from Van der Maas and Nyamsuren (2015) whether items compatible with a specific strategy are easier. As an addition I also dis-cuss our attempt to teach participants an other strategy.The Math Garden
"The Math Garden is an online computer adaptive learning system for math" (Van der
Maas and Nyamsuren, 2015, p. 2). It is used by more than 1400 school in the Netherlands (Rekentuin, 2015). The basis for the "Math Gar-den" is provided by the Computer Adaptive Practice (CAP) system (Klinkenberg et al., 2011). This CAP system is an on the fly ability and difficulty estimator. It is based on the Elo (1978; as cited in Klinkenberg et al. 2011) rating sys-tem. This rating system is a dynamic paired comparison model. Each player is given an abil-ity rating which is updated after every game, based on the match result. "The updated ability estimation depends on the weighted difference in match result S and the expected match result E(S).The expected match result is a function of the difference between the ability estimations of both players." (Klinkenberg et al., 2011, p. 1815). Because the ability estimation can not always be accurate (e.g. for new players or players that had not been playing for a considerable time), Glickman (1995; as cited in Klinkenberg et al. 2011) proposed the K factor to represent this uncertainty. The higher the K factor, the more uncertain the ability estimation of a player is.
In the CAP system one player is replaced with an item. This is presented in the following equations.
ˆθj = θj+Kj(Sij−E(Sij)). (1)
ˆβi =βi+Ki(E(Sij−Sij)). (2)
Here the ˆθj represents the ability rating of
player j, as a function of its current rating plus the expected probability E(Sij)of winning mi-nus the weighted difference between the item and the players score Sij multiplied with the
uncertainty factor K. The ˆβ represents the item difficulty. The K factor in this equation is a function of the uncertainty of the rating, which is represented with U.
Kj =K(1+K+Uj−K−Ui). (3)
Ki = K(1+K+Ui−K−Uj). (4)
When there is no uncertainty the value of the K factor will be 0.0075. The K+=4 and K−=0.5. This are the weighted rates of the uncertainty of player j and item i. The U is determined as following: ˆ U=U− 1 40+ 1 30D. (5)
Were the U =1, there is assumed that after ev-ery administration the uncertainty of the play-ers and items decreases. Therefor the uncer-tainty decreases to 0 after 40 administrations. While it increases, to a maximum of 1, after
D days of not playing. The CAP system also incorporated a speed accuracy rule (Maris and Van der Maas, 2012). The score Sij is a
func-tion of player j respons x in time tij within the
time limit di on item i. This is scaled by the
discrimination parameter a.
Sij = (2xij−1)(aidi−aitij). (6)
Lastly, the expected score (E(Sij)) can be
in-ferred from this model. This is estimated by using the ability of player j (Sij), the difficulty
rating of item i (βi), the time used (tij) and the
discrimination parameter of item i (ai). This
will give the following equation.
Sij =aidi e2aidi(θj−βi)+1 e2aidi(θj−βi)−1− 1 θj−βi (7)
The validity and the reliability of the CAP sys-tem seems to be very promising (Klinkenberg et al., 2011, p. 1822). This in combination with the high frequency of responses (50.000 responses a day) (Klinkenberg et al., 2011, p. 1824), this must constitute into a very precise estimation of the item difficulty.
The Proximity Heuristic
The relative success of players in the Num-ber Game is due to a heuristic (Van der Maas and Nyamsuren, 2015, p. 4). A heuristic is a type of strategy that uses a practical (relatively easy) methodology not guaranteed to find the correct or best solution, but minimizes the com-putational effort one has to perform (Willing-ham, 2007, p. 408). The heuristic which players seem to use in the Number Game is the Prox-imity heuristic. When the ProxProx-imity heuristic is being used (in general), a person tries to mini-mize the distance between his(her) current state and the desired state. This is done in the fastest manner possible, while never increasing the distance between the desired state and the cur-rent state. This heuristic will guide toward the desired state in many real-life situations. An
simple example is; "When a person is thirsty, the desired state is having something to drink, while in the current state this person does not have something to drink. In this case (following the Proximity heuristic) this person (given that this person is at his(her) own home) will walk toward the refrigerator and grab something to drink. This is completed by taking (literally) the shortest route and by never moving further away from the refrigerator during this route." As probably can be imagined, this strategy is not guaranteed to solve a problem. This can easily be demonstrated by using the famous Tower of Hanoi as an example (see Figure 1).
Figure 1: Tower of Hanoi
Image derived from: http://www.cs.brandeis.edu/ storer/Jim-Puzzles/ZPAGES/zzzTowersOfHanoi.html
Rules
1. You can move only one ring at the time. 2. You can move only the top ring on a peg. 3. You cannot put a larger ring on top of a
smaller ring.
The task in this game is to move all rings from peg A to peg C, while following the Rules (Will-ingham, 2007). If you are not familiar with the Tower of Hanoi, first try to solve the puzzle. You will soon realize that during this puzzle you are sometimes forced to move further away from the desired state, because to accomplish the game it is necessary to move rings that are on pin C (the desired state) away from it.
Proximity in the Number Game
Van der Maas and Nyamsuren (2015) ex-plain how the Proximity heuristic works in the context of the Number Game. They divided the Proximity heuristic into two parts. The use of the shortest route (part 1) and the getting at the desired state in the fastest manner possible (part 2). If a player in the Number Game wants to take the shortest route (part 1) the player starts by selecting the two highest start-numbers. If the player than decides to get to the desired state as fast as possible (part 2), the player uses these two highest numbers and tries to get as close as possible toward the goal-number. For a better understanding of how the Proximity heuristic works in the Number Game, see Ex-ample 1.
A Number Game task
Leftside =start-numbers
Rightside =goal-number
1 10 100| + − ×÷ |999
Part 1 = Selecting the two highest numbers.
100 and 10
Part 2 = Getting as close as possible to the goal-number.
100×10=1000
Part 3* = Wrap up.
1000 - 1 = 999
Example 1: Proximity in the Number Game
As can be seen in Example 1, a third part is added (part 3*). This is not necessarily part of the Proximity heuristic. Only it follows very logically after the first two steps (otherwise a player will never actually get to the desired state).
Testing the Proximity Hypothesis
Van der Maas and Nyamsuren (2015) tested their hypothesis whether people tend to auto-matically use the Proximity heuristic. This was completed through analyzing the items diffi-culty rating (which is established with the CAP system) to see whether items compatible with the Proximity heuristic were indeed easier. A lin-ear regression confirmed this hypothesis. Items that are compatible with the Proximity heuristic are indeed easier (β = −8.22, t(985)=−7.095, p < 0.001). These results appeared to be very plausible, because even when tested in specific items (e.g. items that could only be solved with subtraction or multiplication) the items most compatible with the Proximity heuristic were always the easiest. This indicates that people have an automatic tendency to use the Proximity heuristic. However, there is a possible alternative explanation which would clarify why items compatible with the Proxim-ity heuristic are easier in the Number Game. Namely, it should be due to the layout of the Math Garden. In the Math Garden a player must select a start-number than an operator and than a start-number again (by clicking on it), also in this sequence. Than the numbers and operator will appear in a box, in the mid-dle of the screen in the given order. Also the answer of this operation will be given and that answer will be the first (new) number being used during the next operation. Thus, for the next operation a player now only needs to se-lect an operator and another start-number. This is much conform the Proximity heuristic and therefor might be the reason items compatible with this heuristic are easier.
Before I continue with the hypothesis we are going to test, I will first introduce the Working Backward heuristic which seem to be compatible with many of the Number Game items. However before I can encounter this
specific strategy, I first need to take a few steps back and briefly take problem solving in general into account.
Problem Solving in General
If your original state of being is not the state you would like to be in you have a prob-lem, according to the definition of the word problem (Willingham, 2007). "In the search for a solution, a problem solver may use strategies" (Bédard and Chi, 1992, p. 137). These strate-gies can form the key in solving a problem (Willingham, 2007), because the chosen strat-egy plays a considerable role in how difficult a problem is for someone. This is also confirmed by the results of Van der Maas and Nyamsuren (2015), where the items compatible with the Proximity heuristic were found to be easier. By using strategy "A" it will be hard (if even pos-sible) to find an answer to your problem, like solving the Tower of Hanoi with the Proximity heuristic (see Figure 1). This while by using strategy "B" your problem will be solved in just a few seconds. Literature suggest that a difference between novices and experts (in specific problems) is that experts take more time to understand the problem, they can make an ill-defined problem a well-defined problem (Chen, 2000). Of course this is due to the fact that experts have more knowledge about a certain type of problem, while novices do not. However this makes experts choose the right strategy, while novices are more likely to just "jump into" the problem. This emphasize the importance of the usage of strategies in prob-lem solving. There is much (more) literature on problem solving and the strategies that are being used (e.g. Willingham (2007); Voss et al. (1983); Chi et al. (1981); Bédard and Chi (1992); Scarpelli et al. (1996); Chen (2000)). Some of these articles even established to create true
algorithms for their strategies (e.g. Chen (2000); Scarpelli et al. (1996)).
The Working Backward Heuristic
In the Working Backward heuristic one starts at the desired state and tries to work back to the starting state (Willingham, 2007). There does not seem to be a more complete descrip-tion of this heuristic. This heuristic is useful when the desired state is known but the initial (or start) state is not, although it is not neces-sary to have an unknown starting state for this strategy to be worth using. An example is the following: "After reading this sentence, you will realize that the the brain doesn’t recognize a second "the"." (Ashley, 2015). Most people do not recognize the second the in that sentence (Ashley, 2015). While if you read that sentence backward you will definitely notice the second the. There are lots of different examples, es-pecially within the context of puzzles where the Working Backward heuristic is very useful (A more comprehensive example can be found in chapter 12 of Willingham (2007)).
Working Backward in the Number Game
As the most featured aspect of the Work-ing Backward heuristic, one starts at the goal state instead of the starting state. In the context of the Number Game the player will begin by inspecting the goal-number. Now (part 1) the player has two options, the first is (option 1) to divide the goal-number by one of the start-numbers. When it is not possible to use option 1, choose the second option namely, (option 2) divide one of the start-numbers by the goal-number. No matter which option is chosen this will create a new number. The second step (part 2) will be to form this newly created num-ber with the remaining start-numnum-bers. For a
better understanding of the Working Backward heuristic see Example 2.
A Number Game task
Leftside =start-numbers
Rightside =goal-number
1 10 100| + − ×÷ |900
Part 1 = Choosing option 1 or 2.
900÷100=9
Part 2 = Form the new created number (9) with the remaining start-numbers.
10 -1 = 9
Part 3* = Wrap up.
(10-1)×100=900
Example 2: Working Backward in the Number Game
As can be seen in Example 2, a third part is added (part 3*). This is not necessarily part of the Working Backward heuristic. Only it follows very logically after the first two steps (oth-erwise a player will never get to the desired state). In this part the player writes out the answer (equation) in a "normal" (not backward) manner. This is quite easy, because within this strategy there are only two possible ways to write out the answer depending on whether a player used option 1 (divided goal-number by number) or option 2 (divided a start-number by goal-start-number). If option 1 is used, a player always writes the answer as following; place brackets around part 2 (10-1) and multi-ply it with the start-number that was used to divide the goal-number in part 1 (100). This is the case in example 2. When option 2 is being applied, also place brackets around part 2 and divide the remaining start-number by it.
It is All About the Strategy
Along the road of getting more and more familiar with the Number Game it became clear to us (the author and fellow researchers) that almost all items were either solvable by using the Proximity heuristic or solvable by using the Working Backward heuristic. Although there was a difference in degree to how compatible the different items were with the used strategy, it seemed very obvious that the items were almost all solvable with one of these strategies. All that needed to be done was to expand the strategies a little bit to make them suitable for more items. This "expansion" could easily be made. The only thing a player needs to do in case of failure while using a strategy, was to rerun the entire strategy algorithm, but use a different combination of available numbers instead.
The New Hypothesis
"The independence of the Proximity heuris-tic from necessary operations lead to another prediction that items that are more compatible with the Proximity heuristic should be easier than items that are less compatible indepen-dently of types of operations involved" (Van der Maas and Nyamsuren, 2015, p. 8). Hitherto this seems to be the case, as Van der Maas and Nyamsuren (2015) graphically showed that items that used the exact same start-numbers (1, 10, 100) the items compatible with the Proximity heuristic were, again the easiest. This was even the case when items needed to be solved with the exact same type of operations (e.g. goal-number 999 or 900, in both items one needs to use subtraction and multiplication only in opposite order, the easiest is conform the Prox-imity heuristic). In Figure 2 a simplification is made of their graph. This simplification does
only show the difficulty of four items of each type (compatible or not compatible with the Proximity heuristic).
Figure 2: Difference in difficulty between items, with the same start-numbers (1,10,100) compatible with the Proximity heuristic or not compatible.
As can be seen in Figure 2, there appears to be a sudden "jump" (the large gap between the squares and the triangles ) between the diffi-culty of items compatible with the Proximity heuristic and the items that are not compatible with the heuristic. The hypothesis that items, more compatible with the Proximity heuristic should be easier than items that are less com-patible independently of types of operations involved seems to be confirmed by the data.
We propose two hypotheses. The first hy-pothesis; items compatible with the Proximity heuristic are easier, independent of the Math Garden. The second hypothesis; the difference in difficulty between the two types of items will disappear if the participant is being trained with the Working Backward heuristic.
II. Methods Participants
The 56 participants were mainly gathered via posted flyer’s and through the official psy-chology and communication research website from the University of Amsterdam, were stu-dents can see and register for all the ongoing research at the University of Amsterdam. Most of the participants were first year psychology students and received a so called colloquium point for their participation. The other partici-pants were acquaintances of the experimenters and were so whole-souled to participate for free. The average age was 21.9 with a standard deviation of 3.4 years.
Materials
Arithmetic
The arithmetic ability was measured via the standardized TempoTest Automatiseren (TTA) of basic arithmetic ability (De Vos, 2010). The worksheet contained 40 sums in increasing dif-ficulty order with the possible operations of addition, subtraction, multiplication and devi-sion. The participants were instructed to solve as many sums as possible within one minute, by writing down the answer. The score given was the total number of correctly written an-swers.
Training
All participants watched an instruction movie on a computer screen. The movies lasted between six and eight and a half min-utes. In all these movies the camera is pointed at a paper, so only the hands of the instructor can be seen while he writes down the sums. The same instructor is recorded through all
three different movies and the same type of paper and pen is used. In all movies a strategy (which strategy, depends on the condition) is explained via examples, so the participants can directly see how the strategy should be applied to the sums. In all the movies we use the word "method", instead of using names as Proximity or Working Backward.
After the instruction movie, all participants completed three training sums (compatible with their strategy). During these training sums the experimenter is aloud to help and clarify the method, if it is not understood correctly by the participant. In the next inden-tions the content of the movies will be outlined.
Proximity condition(Prox.condition) The first example given is; 5 7 9 11| + − ×÷ |22. First (step 1) select the two highest start-numbers. Second (step 2) get as close as possible with the selected numbers to the goal-number. Last (step 3) form the remaining value with the unused start-numbers. In this example one should select 11 and 9 (step 1). Then get as close as possible to 22, this is possible by adding 9 to 11, this will give 20 (step 2). The remaining value is now 2 (the distance from 20 to 22). This value can be formed with the unused start-numbers. Namely, by subtracting 5 from 7 (step 3). Thus, the answer is, 11+9+ (7−5) =22. After this example there follows an comparable example. In the last example the method seems to fail. But if the method fails start over and this time instead of selecting the highest numbers, select the highest and the third highest number than repeat the procedure. If this fails also, start again and select the highest and fourth highest number this procedure goes on until the sum is solved, or until no more combinations can be made at the first step.
The first example given is 3 4 5 | + − ×÷ |
35. First (step 1) you check whether the goal-number can be divided by one of the start-numbers (option 1). If this is not the case, check whether a start-number can be divided by the goal-number (option 2). In the first ex-ample it is possible to divide 35 by 5, this will form 7. Then (step 2) see whether it is possible to create this newly formed number with the remaining start-numbers. In this case this is possible by adding 3 by 4. The way the answer should be written is as following, (3+4) ×5. Thus, when the first step during the method is a division, in the answer your last operation is a multiplication with that same start-number. In the third example the (step 1) goal-number is not dividable by any of the start-numbers (option 1). This time one of the start-numbers is divided by the goal-number (option 2). The next step (step 2) will be the same as when option 1 is chosen. The answer should be writ-ten different. Here the start-number used in step 1 should be divided by the newly formed number in step 2.
Control condition (Control group) A math-ematical trick concerning the multiplication table of eleven is shown and explained. An example used in this movie is 14×11. The trick goes as following; you select the left number (from, in this case 14) and place that on the left position of the answer. Then you take the right number and place that on the right position of the answer. This will give you "1 . 4", with the "." being the middle number that we must form. To obtain this number add the left by the right number (1+4), this will give you 5. The answer should than be, 154. This trick works with all numbers you multiply with 11, except numbers lower than 10. This is shown with multiple examples.
Pretest and Posttest
The pretest and posttest consisted both out of five items compatible with the Proximity heuristic (e.g. 3 20 100 | + − ×÷ | 83) and five items compatible with the Working Back-ward heuristic (e.g. 6 10 100 | + − ×÷ | 940). The order of Proximity and Working Backward items was in alternating sequence. The pretest and posttest were of approximately of the same difficulty, although we did not test this. All tests and forms were given on paper.
Measures
AccuracyThe pretest and posttest were being checked by the experimenters. The items were given a score of 1 when a correct answer was given or a score of 0 when an incorrect answer was given.
Response Time The time a participant took per item was recorded with a hand-stopwatch by the experimenter. When a participant did not finish an item within the time limit of 60 seconds the experimenter reported 61 seconds. When an item was answered incorrect (as de-termined during the accuracy check), the time was also changed to 61 seconds.
Scoring A combination of accuracy and re-sponse time is created according to the "Correct Item Summed Residual Time" (CISRT) (Van der Maas and Wagenmakers, 2005). The score is calculated by multiplying the accuracy (1 for correct, 0 for incorrect) with the time limit (60 seconds) minus the response time of the partic-ipant. This will give the following formula.
Acci(MT−Ti). (8)
Were Acci represents the accuracy on item i.
MT represents the time limit (Maximum Time), and the Ti represents the response time on item
i.
Manipulation checksAs manipulation checks we created a form with the following state-ments: "I participated seriously" and "I under-stood the method, proposed in the instruction movie". Answers were given on a five point Likert Scale (1=completely disagree, 5=com-pletely agree).
Procedure
After the participants arrived at the lab, they were provided with the informed consent and completed the TTA (De Vos, 2010). Than the participants read an explanation form of the Number Game and completed ten (easy) practice sums. Hereby the participants were told that they should ask questions if there was something unclear to them. When this prac-tice phase was completed, the experimenter explained that the test was about to start. The participants had to complete the ten sums in the order that they were presented. The par-ticipant may only continue with the following sum when the previous sum was completed or, if the time limit of 60 seconds was passed. Whether the time limit was passed was indi-cated by the experimenter. After completing the pretest the participants watched a movie, which movie depended on the condition the participant was assigned to. After this movie the participant completed three training sums, with possible guidance of the experimenter. Than they completed the posttest, in the same fashion as the pretest. At last the participants had to fill out a form with questions concerning the manipulation check.
III. Results
The manipulation checks showed that all the participants participated seriously (M=4.9, SD=0.3). Also, all participants understood the method, proposed in the movie they watched (M=4.8, SD=0.4). Three conducted t-test showed that there was no difference in scores, between the conditions on the TTA.
A two-way ANOVA for the accuracy rate on the items compatible with the Proximity heuris-tic, with a between-subject variable "condition" (Prox.condition, Work.condition and Control group) and with a within-subject variable "time" (pretest and posttest) was conducted. These results are graphically presented in Fig-ure 3.
Figure 3:The total accuracy rating on items compatible with the Proximity heuristic, dependent on condition. At pretest and posttest.
There was a significant main effect of condi-tion, F(2)=5.257, p < .01. The post hoc t-tests re-vealed that the Work.condition had a significant lower accuracy rate than the Prox.condition, during the posttest t(36)=3.261, p < .01.
There was a significant main effect of time F(1)=19.68, p < .001. The post hoc t-tests re-vealed that the Work.condition had a significant lower accuracy rate at the posttest as during the pretest t(18)=3.314, p < .01. As did the Control group, t(17)=2.675, p < .05.
There was a significant interaction between condition and time, F(2)=12.01, p < .001. This effect indicates that the Working Backward train-ing which the Work.condition received had a negative effect on accuracy rate on the items compatible with the Proximity heuristic.
A two-way ANOVA for the accuracy rate on the items compatible with the Working Back-ward heuristic, with a between-subject variable "condition" (Prox.condition, Work.condition and Control group) and with a within-subject variable "time" (pretest and posttest) was con-ducted. These results are graphically presented in Figure 4.
Figure 4: The total accuracy rating on items compatible with the Working Backward heuristic, dependent on condition. At pretest and posttest.
There was a non-significant main effect of condition, F(2)=0.058, p = .944.
There was a significant main effect of time F(1)=10.229, p < .01. The post hoc t-tests revealed that the Work.condition had a significant higher accuracy rate during the posttest t(18)=4.789, p < .001.
There was a significant interaction between condition and time, F(2)=5.377, p < .01. This effect indicates that the training which the Work.condition received improved their accu-racy rate on items compatible with the Working Backward heuristic. Because there was no main effect of condition, this also indicates that the accuracy of the Work.condition at the pretest was probably lower (although not significant) than the accuracy rate of the other conditions. The exact proportions correct (accuracy rate) are displayed in Table 1.
A two-way ANOVA for the scores (based on the CISRT) on the items compatible with the Proximity heuristic, with the between-subject variable "condition" (Prox.condition, Work.condition and Control group) and with the within-subject variable "time" (pretest and posttest) was conducted.
There was a significant main effect of condi-tion, F(2)=3.74, p < .05. The post hoc t-tests revealed that the Work.condition had a sig-nificant lower score than the Prox.condition at the posttest, t(36)=4.745, p< .001. The Work.condition also had a significant lower score than the Control group at the posttest, t(35)=2.712, p< .05.
There was a significant main effect of time, F(1)=15.12, p < .001. The post hoc t-tests revealed that the Work.condition had a significant lower score during the posttest, t(18)=4.805, p< .001. The Control group also had a significant lower score during the posttest, t(17)=2.278, p< .05.
There was a significant interaction effect be-tween condition and time F(2)=11.43, p < .001. This effect indicates that the Working Backward
Table 1: Proportion correct on the two types of items, dependent on the training received.
Pre Post
Training Prox. items Work. items Prox. items Work. items
Prox.-Training .90 .55 .96 .64
Work.-Training .95 .45 .83 .75
Control .97 .48 .88 .61
training which the Work.condition received had a negative effect on the score on the items compatible with the Proximity heuristic. These results are graphically presented in Figure 5.
Figure 5: The mean scores on items compatible with the Proximity heuristic, dependent on condition. At pretest and posttest.
A two-way ANOVA for the scores (based on the CISRT) on the items compatible with the Working Backward heuristic, with the between-subject variable "condition" (Prox.condition, Work.condition and Control group) and with
the within-subject variable "time" (pretest and posttest) was conducted. These results are graphically presented in Figure 6.
Figure 6: The mean scores on items compatible with the Working Backward heuristic, dependent on condition. At pretest and posttest.
There was a non-significant main effect of condition, F(2)=0.074, p < .93.
There was a significant main effect of time, F(1)=30.46, p < .001. The post hoc t-tests revealed that the Work.condition had a significant higher score during the posttest, t(18)=4.187, p< .001. The Control group also had a significant higher
Table 2: Mean scores on the two types of items, dependent on the training received.
Mean (SD)
Pre Post
Training Mean Prox. items Mean Work. items Mean Prox. items Mean Work. items
Prox.-Training 40.6 (13.0) 20.0 (20.2) 43.0 (9.11) 22.6 (17.1)
Work.-Training 44.1 (9.5) 15.5 (18.8) 31.3 (18.3) 24.3 (16.7)
Control 44.9 (8.0) 16.2(18.2) 40.2 (13.6) 25.7 (19.3)
score during the posttest, t(17)=3.891, p< .01. There was a non-significant interaction ef-fect between condition and time, F(2)=3.044, p = .056. This indicates that the Working Backward training did not have an effect on the scores on items compatible with the Working Backward heuristic. The exact scores are displayed in Table 2.
Figure 7: Mean score on all pretest items. All participants included.
A paired t-test over the scores of all partic-ipants on the pretest, with all items included showed a significant difference between the two types of items t(55)=15.465, p < .001. These results are graphically presented in Figure 7.
A paired t-test over the scores on the posttest, solely for the Work.condition with all items included still showed the same differ-ence between the two types of items t(18)=2.209, p < .05.
IV. Discussion
The study reported here shows that items compatible with the Proximity heuristic, inde-pendent of the Math Garden are easier, as our hypothesis predicted. These results are also in accordance with the results found by Van der Maas and Nyamsuren (2015). This indicates that people have an automatic tendency to use the Proximity heuristic.
This study also shows that a Working Back-ward training does not make the difficulty dif-ference between the items compatible with the Proximity heuristic and other items disappear. This was not according to our hypothesis. The
only results necessary to show this support against our hypothesis were the results coming from the "scoring". However, these results in-dicate that the Working Backward training did not have any positive effect at all and it would appear that these results are possibly due to a failure of the training itself. However, as the accuracy results show, only the participants that received the Working Backward training im-proved significantly, during the posttest on items compatible with the Working Backward heuristic. Although this training effect was to small to provide significant differences between groups, I argue that this was due to the fact that the participants were trained too short (less than ten minutes). I do belief a longer Working Backward training could yield into significant differences between the conditions. With this statement I do not try to argue that the differ-ence between the items compatible with the Proximity heuristic and other items, could dis-appear after the training. Our results would be strongly in conflict with such a statement. As it shows that even while the Work.condition improved in accuracy on the items compatible with the Working Backward heuristic and at the same time decreased in accuracy on items com-patible with the Proximity heuristic, there still was a significant difficulty difference between those two types of items.
It is possible that our experiment suffers from experimenter effect(s). Because of the combination; timing the participants and the use of a paper and pen design, we were some-what forced to be in the same room as the par-ticipant during the pretest and posttest. Of course it is possible to let the participants time themselves. However, we saw more flaws in such a design than our current design, there-for our choice. Following, there were a few participants that made it explicitly clear that they found it a bit uncomfortable that the
ex-perimenter was present during the test. This indicates that it is not unlikely that some partic-ipants did not perform on their highest capaci-ties possible. This is something experimenters should definitely take into account when set-ting up their research, as we also tried too (e.g. We decided not to be in the room during the instruction movie, so the participants would at least not be distracted during the movie).
Concluding, strategies seem to play a role in solving Number Game sums, as the items compatible with the Proximity heuristic are less difficult. Because the Working Backward training did not make the difficulty difference between the two types of items disappear, it indicates that other factors, beside strategy, also play an important role in solving Number Game sums. With "other factors" one should think of, the actual operations necessary and the amount of start-numbers. These other factors were also re-vealed by Van der Maas and Nyamsuren (2015) and till so far cannot be reduced to another strategy.
References
Ashley, M. (2013 (accessed June 25, 2015)). Working backward to solve problems - Maurice Ashley. https://www.youtube.com/watch? v=v34NqCbAA1c.
Bédard, J. and Chi, M. T. (1992). Expertise. Cur-rent directions in psychological science, pages 135–139.
Chen, S.-M. (2000). Fuzzy backward reasoning using fuzzy petri nets. Systems, Man, and Cy-bernetics, Part B: CyCy-bernetics, IEEE Transactions on, 30(6):846–856.
Chi, M. T., Feltovich, P. J., and Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive science, 5(2):121–152.
De Vos, T. (2010). Tempo Test Automatiseren. Am-sterdam: Boom Test Publishers.
Klinkenberg, S., Straatemeier, M., and Van der Maas, H. (2011). Computer adaptive practice of maths ability using a new item response model for on the fly ability and difficulty es-timation. Computers & Education, 57(2):1813– 1824.
Maris, G. and Van der Maas, H. (2012). Speed-accuracy response models: Scoring rules based on response time and accuracy. Psy-chometrika, 77(4):615–633.
Rekentuin (2009 (accessed June 20, 2015)). on-line rekenen oefenen voor scholen en families. http://www.rekentuin.nl.
Scarpelli, H., Gomide, F., and Yager, R. R. (1996). A reasoning algorithm for high-level fuzzy
petri nets. Fuzzy Systems, IEEE Transactions on, 4(3):282–294.
Van der Maas, H. and Nyamsuren, E. (2015). The number game: an educational game and challenge for cognitive modelling. Manuscript in preparation.
Van der Maas, H. L. and Wagenmakers, E.-J. (2005). A psychometric analysis of chess ex-pertise. The American journal of psychology, pages 29–60.
Voss, J. F., Greene, T. R., Post, T. A., and Pen-ner, B. C. (1983). Problem-solving skill in the social sciences. The psychology of learning and motivation, 17:165–213.
Willingham, D. T. (2007). Cognition: The think-ing animal. Pearson/Prentice Hall Englewood Cliffs, NJ.
