The Number Game

(1)

The Number Game

Student: Wessel de Jong Studentnumber: 10206620

Words: 6174

Attendant: dhr. prof. dr. H.L.J. (Han) van der Maas Institute: University of Amsterdam

(2)

ABSTRACT – The alleged NP-Complete Number Game, used in Math Garden, assists in educating basic arithmetic operations and problem solving. Math Garden data has shown a sudden jump in complexity between different instances of this game. It is hypothesized that inclinations in heuristic use is what underlies this sharp transition. We propose a heuristic based on backward reasoning, which should perfectly apply to harder instances of the Number Game. This heuristic was

contrasted with an intuitive heuristic based on forward reasoning. Participants were either trained in the forward or backward heuristic. Results again showed a sharp transition in complexity. However, there was no effect of training the backward

heuristic.

Math garden is a highly popular educational website for assisting primary school teachers in training their pupils in maths and other kinds of problems. This computer-adaptive learning system is used by more than 1500 primary schools in the Netherlands (Klinkenberg, et. al. 2011). By playing different kinds of games pupils are introduced with the basic principles of math. One of the promising games in the Math Garden is the Number Game, a game designed to improve players’ skills in addition, subtraction, multiplication and division as well as their general problem solving skills.

In the Number Game players are given a set of Sn positive integers and a certain target number T. The goal of the game is to find T with the given set of integers using the operators ‘+’, ‘−‘, ‘×’ and ‘÷’. All integers in the given set have to be used precisely one time. The same operator can however be used multiple times and not all operators have to be used. To illustrate the number game consider the following example.

2 3 11 | 66

In this example players are asked to manipulate the set of 2, 3 and 11 (Sn = {2, 3, 11}) in such a way so the target number 66 (T = 66) is found. The solution for this instance of the number game is easily found. By only using multiplication T can be found, 2 x 3 x 11 = 66. A much harder example is the Number Game in which given the set 1, 3, 4 and 6 (Sn = {1, 3, 4, 6}) the target number 24 (T = 24) must be found.

(3)

An important feature of the Number Game is it’s alleged NP-Completeness. Before showing that some instances of the Number Game are NP-Complete the theory of NP Completeness will be discussed. In Computer Science a much-discussed topic on problem solving is the distinction between P and NP problems. When an algorithm in polynomial time can find a solution for any kind of problem, the problem is called P (Torenvliet, 2014). This class of problems forms the lowest level in a hierarchy of problem complexity. The next level in this hierarchy would be problems for which only exponential time algorithms are known. Problems, for which only exponential time algorithms are known, are NP problems. For these problems the best solution is often ‘brute force’, examining every possible answer until one finds the correct one (Willingham, 2007). With a growing problem size the time needed to solve these problems grows exponentially. As can be expected these problems are often unsolvable for humans. However, should a solution for these problems be given, than it is easy to check this solution in polynomial time.

The fact that for NP-problems no polynomial time algorithms are known does not mean that they do not exist. Within NP-problems there is a subclass of problems for which it seems very likely that no polynomial time algorithms exist. Problems in this subclass are known as NP-Complete problems. If a polynomial time algorithm exists for one of the problems in this subset, by definition such an algorithm exist for all problems in this subclass. In such a situation it also means that a polynomial time algorithm exists for all NP problems.

How to prove that a problem belongs to the subset of NP-Complete problems? This can be done by a method called reduction (Torenvliet, 2014). A problem is proven to be NP-Complete when all instances of a certain known NP-Complete problem can be reduced to an instance of this problem. In this way the Number Game can be shown to be NP-Complete by reducing all instances of a popular NP-Complete problem, namely the Partition Problem (Kurzen, 2011).

In the Partition Problem you are given a set of n positive integers. The goal is to separate this set into two non-overlapping subsets of which the sums are as nearly equal as possible (Hayes, 2002). In an ideal situation the sums would be exactly the same, but this can only be done if the sum of the entire given set is even. The Partition Problem is a well-known NP-Complete problem.

It can be shown that the Partition Problem can be reduced to those instances of the Number Game where the only operators allowed to use are ‘+’ and ‘-‘ and the

(4)

target number is 0 (T = 0). If we take for example the Partition Problem with the given set of integers Sn = {2, 10, 3, 8, 5, 7, 9, 5, 3, 2}. Our task is to divide this set into two sets with an equal sum. This is equivalent to asking to find the target number 0 (T = 0) with only the operators ‘+’ and ‘-‘. The solution would be “(10 + 7 + 5 + 3 + 2) - (9 + 8 + 5 + 3 + 2)”. The equivalence between the problems is easy spotted in this example, two sets with equal sums are formed and than subtracted from each other. Consequently, the Number Game is proven to be NP-complete for those instances where only the operators ‘+’ and ‘−‘ are allowed to be used.

To prove that the Number Game is also NP-Complete for those instances where players are allowed to use all four operators is not as easy. Intuitively however, one would be inclined to say this would be the case. It would seem that adding ‘×’ and ‘÷’ to the operable operators would only increase the problem size of the Number Game, and thus the time needed to solve these problems. Thereby, the instances of the Number Game with only the operators ‘×’ and ‘÷’ are equivalent to those instances with only ‘+’ and ‘−‘ if we make use of an ln-transformation. An instance of the Number Game where we would take the natural logarithm of the set integers and the target number and the only operators allowed to be used are ‘×’ en ‘÷’, would be equivalent to an instance where only ‘+’ and ‘−‘ can be used with the same integer set. Although the NP-Completeness of all Number Games seems to be logical, a formal proof is needed to make any claims about this alleged feature of the Number Game.

The NP-Complete status of the Partition Problem does not mean that all instances of this problem are NP-Complete. If we look at the above first example of this problem we would not be inclined to say this is a NP-Complete problem, the example seems easy to solve. The question arises which Partition Problems can be labelled NP-Complete and where the threshold lies between Partition Problems that are NP-Complete and those that are not.

It has been shown that the spectrum of Partition Problems has both easy and hard regions and a sharp boundary between these regions (Hayes, 2002). This sharp phase transition has been found in many NP-Complete problems (Mertens, 2001). Gent and Walsh (1996) introduced the control parameter k on which this phase transition depends. This parameter k corresponds to the ratio of the number of significant bits in encoding the set integers and the number of n integers (Mertens, 2001; Mertens, 2003). Below the critical value kc Partition Problems seem to have a

(5)

special property, which makes it possible to solve these problems with a polynomial algorithm (Mertens, 2003). Above this value instances of the Partition Problem can be considered NP-Complete. Simulations have shown that the property determining kc is

the probability of a perfect partition (Figure 1). This probability jumps from 1 to 0 if k crosses kc. Different ways of determining kc in each instance of the Partition Problem

have been proposed (Gent & Walsh, 1996; Mertens, 2001; Borgs, Chayes, & Pittel, 2001).

Fig. 1. Number of nodes needed to solve Partition Problems with growing sets of

random 25-bit integers. The complete balanced largest differencing method (Mertens, 2001), BLDM, is used to optimally solve these problems. As can be seen the number of nodes needed by the complete BLDM algorithm to solve these problems grows exponentially with increasing sets of integers up to a certain point. From this point the number of nodes needed decreases to the point where this number is equal to the number of set integers. It can be seen that at this transition point the probability of a perfect partition jumps from 0 to 1.

It is still an open question whether this kind of phase transition can also be found in the Number Game. A simulation study could answer this question. If such a phase transition would apply to the Number Game it should be considered what underlies this transition and where the border between the easy and hard phase lies. It should be noted however that human cognition probably is not able of solving these instances. The NP-Complete instances of the Partition Problem have average integer sets of more than 100 integers. These instances seem to be unsolvable for human cognition. Although we expect above example with Sn = {1, 3, 4, 6} and T = 24 to be

(6)

NP-Complete, the NP-Completeness label would possibly only apply to those instances of the Number Game, which can be studied in the Computer Sciences. It is however striking that a phase transition can also be found in instances of the Number Game, which can be solved by human cognition. Although it is not claimed these phase transitions are similar, both transitions seem to be grounded on the same basis, namely a sudden jump in the complexity level of different instances of the Number Game. This transition will be considered next.

Math Garden has shown that children are surprisingly good in solving the Number Game, considering the complexity of this game. Data have however shown there is a sharp transition between certain kinds of instances and the rate by which these instances are solved. This transition can be seen in figure 2. Points in this figure represent different instance of the number game were the set 1, 10 and 100 is offered (Sn = {1, 10, 100}). These instances differ in the target number that should be found. With each point the target number is depicted. It can be seen that there exist a sharp transition between the instances to the left and those to the right. The instances to the left are rated much easier then the ones to the right of the horizontal axis. What could be underlying this transition?

We hypothesize that this transition can solely be understood from heuristic use in solving the Number Game. What makes the harder instances in figure 2 harder can only be understood in light of inclinations of human reasoning in problem solving.

Analyses of data from Math Garden have shown that pupils’ success in solving the Number Game can be explained by an inclination towards a heuristic, which will be called the proximity heuristic from now on (Van der Maas & Nyamsuren, 2015). This heuristic resembles well-known greedy heuristics

(Torenvliet, 2014). In this heuristic the two largest numbers in the given integer set are selected. This is the greedy part of the heuristic. After selecting the largest

numbers, an operator is selected which results in the highest convergence to the target number. With the remaining numbers in the integer set a way is sought to find the target number.

(7)

Fig. 2. Phase transition in complexity of different instances of the Number Game. There seems to be a

sharp transition observable along the horizontal axis. Easier instances are perfectly solvable with the

proximity heuristic.

It can be easily shown that this heuristic perfectly applies to the easier instances of the Number Game in figure 2. The inclination of pupils towards this heuristic explains why these instances are successfully solved most of the times and thus rated easier. If we for example take the Number Game with the integer set 1, 10 and 100 (Sn = {1, 10, 100}) and the target number 1001 (T = 1001). Applying the proximity heuristic we would first select the largest numbers in the set, 10 and 100. Second, we would select the operator, which results in the highest convergence to the target number with these numbers. Here we would choose multiplication and end up with 1000. The remaining 1 can be used to find the target number. The harder instances in figure 2 do not seem to comply so perfectly to this heuristic.

We propose a heuristic, which would increase the success rate of the harder instances in figure 2. This heuristic should be easily applicable to all these harder instances. A throughout examination of the instances of the Number Game in figure 2 reveals an important distinction between the easier and the harder items. This

distinction has played an important role in problem solving literature, namely the distinction between forward reasoning and backward reasoning (Willingham, 2007). These terms were first introduced in studies to expertise in physics. A distinction was

(8)

made between reasoning forward from the starting state to the goal state of the problem space and reasoning backward from the goal state to the starting state.

Backward reasoning is usually thought to be a more sophisticated strategy than forward reasoning (Larkin, McDermott, Simon, & Simon, 1980). The heuristic we

propose includes working backwards and shall be called the backwards heuristic. Where the proximity heuristic reasons form the integer set to the target number, the

backward heuristic works the other way around; a solution is sought by reasoning

from the target number to the integer set. Although the backwards heuristic is not applicable to all instances of the Number Game, it can be shown to be perfectly applicable to the harder instances of the Number Game from figure 2. As can be deduced from the following explanation of this heuristic, it is perfectly applicable to those instances of the number game of which the solution ends with a multiplication or division. These instances seem not to be solvable by the proximity heuristic, and are therefor considered harder.

With the backward heuristic first it is examined if the target number is divisible by one of the given integers. If possible the target number is divided by this set integer. Then a way is sought to form the outcome of this division with the remaining integers in the set and some operator(s). If possible, a multiplication with the previously selected integer is all that is needed to find the target number. When the target number is not divisible by one of the integers, one examines whether a set integer is divisible by the target number. Again this division is made. With the remaining integers a way is sought to form this division outcome. If it happens to be possible to form the division outcome with the remaining integers, dividing the first selected integer by this result brings the solution.

Take for example the Number Game where the integer set 1, 10, 100 (Sn = {1, 10, 100}) and the target number 1010 (T = 1010) are offered. Applying the proximity

heuristic will not help finding the solution in this case. However, the target number

can be divided by the integers 1 and 10. Dividing 1010 by 1 gives 1010. With the remaining integers 10 and 100 it is not possible to find 1010. If we divide 1010 by 10 however, we end up with 101. We should now search for a way to form 101 with the remaining integers 1 and 100 and this is done by using addition, as can be easily seen. Now we only have to multiply this addition by the previous selected integer 10, which results in the following solution: (100 + 1) * 10. In another example with the integer set 3, 12, 28 (Sn = {3, 12, 28}) and target number 7 (T = 7) a further step is needed. In

(9)

this example the target number is not divisible by one of the integers. However the given integer 28 is divisible by the target number 7, which would result in 4. Next, 4 must be formed with the integers 12 and 3, which can be done by another division. Finally, dividing the first selected integers 28 by the division of 12 and 3 brings the solution: 28 ÷ (12 ÷ 3). As can be seen, in both examples the solution is found by reasoning from the target number to the integer set.

An alternative explanation for the phase transition in figure 2 can be found in the design of the Number Game in Math Garden. It can be argued that the way the Number Game is presented provokes using forward reasoning. Figure 3 shows the way the Number Game is presented in Math Garden. The screenshot shows that pupils are not allowed to apply any operations to the target number 2 (T = 2). First two integers and an operator are selected after which the result of this operation forms the first integer in a next operation. This way it is obliged to reason step by step from the integer set to the target number. The proximity heuristic seems to be perfectly compatible with this kind of design. Not being allowed to operate on the target number makes applying the backward heuristic much harder. This might also explain the phase transition found in figure 2. Because pupils are stimulated to use forward

reasoning by the design of Math Garden, the instances that comply with this kind of

reasoning will be solved much easier. The instances that require backward reasoning however will be rated much harder.

(10)

If inclinations in heuristic use are what really underlie the sharp transition phase found in figure 2, players trained in applying the backwards heuristic should have a higher success rate on the harder items. These items should then be rated as less complex and the sharp transition should disappear. The current research explores if by teaching the backwards heuristic the success rate of solving the Number Game

increases. It is yet unclear if the backward heuristic really is an efficient way of solving the Number Game. Thereby, for educational purposes, it is explored if this heuristic can be efficiently taught by means of an instruction video.

Before these questions can be answered, it should again be shown that human cognition really has an inclination towards the proximity heuristic in this kind of problem solving. It is of great importance to show that it is not the way the Number Game is presented in Math Garden that induces a preference for forward reasoning. Using a different mode of presentation than made use of in Math Garden, it should be shown that instances complying with this heuristic are mostly successfully solved and have a higher success rate in comparison with instances that do not comply with this heuristic before any intervention has taken place.

(11)

Method

Participants

Fifty-six college students between the age of 18 and 40 participated in this study. Participants were recruited through an online add on the website of the psychology research institution of the University of Amsterdam. Participants were assigned to one of three conditions. Nineteen participants were assigned to the forward condition and the backward condition and eighteen participants were

assigned to a control condition. If needed, participants could be rewarded with course credits.

Materials

To make sure participants were acquainted with the basic arithmetic skills of addition, subtraction, multiplication and division, the ‘Tempo Test Rekenen’ (De Vos, 2010) was used. This is a short questionnaire in which participants are asked to solve as many basic arithmetic tasks as possible in 60 seconds. The test consists of 5 columns of which we only used the last one. This column consists of 40 items, which are a combination of addition, subtraction, multiplication and division tasks. The different items are ordered on their difficulty.

Instruction videos were used for the heuristic training. For every condition a different instruction video was made. All three videos lasted for approximately 8 minutes. In the videos we made use of blank sheets of paper on which some instances of the Number Game were sketched. Only the blank sheet and the writing hand of the narrator were visible in the videos. By means of different examples of the Number Game the successive steps of both the proximity heuristic and the backward heuristic were presented.

Participants in the forward condition were presented an instruction video of the proximity heuristic, which was modified a little from the way it is presented above. In the instruction video participants were told that they were going to learn a trick, which could help them with solving the Number Game. The first and the second step of the heuristic were presented as above; first select the two highest integers and then select the operator that will result in the highest convergence to the target number with these integers. After these steps, try to find the target number with the remaining integers. If however these steps would not lead to a solution, then go back

(12)

to the first step. This time, do not select the two highest integers, but the highest integer and the third highest integers. With these integers follow the same steps in the same order. If these integers also do not lead to a solution, select the highest integer and the fourth highest integer, and go on like this until the whole integer set is checked from the highest to the lowest integer. In this video the following examples were used: (1) Sn = {5, 7, 9, 11} and T = 22, (2) Sn = {3, 1, 11} and T = 33, (3) Sn = {4, 5, 1} and T = 24.

In the backward condition participants were presented an instruction video of the

backwards heuristic. This heuristic was precisely presented as above. In this video the

following examples were made use of: (1) Sn = {3, 4, 5} and T = 35, (2) Sn = {2, 6, 11} and T = 33, (3) Sn = {3, 12, 28} and T = 7, (4) Sn = {10, 50, 100} and T = 5.

Participants in the control condition were presented an instruction video of an arithmetic trick that could help when multiplying a number between 11 and 99 with 11. The trick works as follows: when multiplying a number between 11 and 99 with 11, all that needs to be done is write done the first and the second digit of the number that is being multiplied by 11. In between these digits write the sum of both digits of the number that is being multiplied by 11, this gives the result of the multiplication. Different examples of multiplication with 11 were used.

Participants were presented with a pre and post-test (before and after showing the instruction video), consisting of 10 instances of the Number Game. The instances selected resembled used items from the database of Math Garden. Items were selected on their difficulty, where mostly items with an average difficulty were selected. The difficulty of these items is determined by computer adaptive testing results, whereby difficulty of an item is determined by its response accuracy and the response times needed to answer the item (Maris, & Van der Maas, 2012). The items selected all started with the integer set 1, 10, 100 (Sn = {1, 10, 100}). To prevent training effects we modified this integer set to equivalent sets like 2, 10, 100 (Sn = {2, 10, 100}. Both

the pre and the post-test consisted of 5 items on which the proximity heuristic seems to be perfectly applicable, which we will call the forward items from now on, and 5 items on which the backwards heuristic seems to be perfectly applicable, which we will call the backward items. A blank box intended for the participant’s response followed every item, making sure the items did not stimulate making use of forward

(13)

Three different scoring methods were applied. Response accuracy and response times were used as different measures of the participants’ results. Each item had a maximum response time of 60 seconds. A third scoring method applied combines response accuracy and response time. The method used is called correct item summed residual time (CISRT) (Van der Maas, & Wagenmakers, 2005). The CISRT score depends on how much time is left of the maximum time allowed if and only if the response is correct. If the response is incorrect the assigned score is 0. If the response is correct, the assigned score is calculated as follows: 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 = 𝐴𝐴𝑠𝑠𝑠𝑠_𝑖𝑖(𝑀𝑀𝑀𝑀 − 𝑡𝑡_𝑖𝑖), where

i determines the item, Acc denotes the accuracy of the response, MT the maximum

completion time and t the response time. When the maximum completion time is exceeded a minimum score of 0 is assigned. The maximum score would be 60 with a correct response.

Procedure

Participants were seated behind a computer in a quite research room. During the research the researcher was seated next to the participant. After answering some demographical questions participants were presented with the ‘Tempo Test Rekenen’ (De Vos, 2010). Time was measured with a simple stopwatch by the researcher. Participants were instructed to complete as many items as possible in 60 seconds. It was advised not to dwell on items they could not answer but to skip these items.

After this task a tutorial of the Number Game was presented. Participants were asked to attentively read the tutorial. After reading the tutorial the researcher checked if the tutorial was understood and could provide extra information when needed. Participants were then presented with 10 practice items to make sure the Number Game was understood. These items were straightforward instances of the Number Game, mostly with integer sets of only 2 integers. During this practice the participant could still ask questions if anything was unclear.

The participants were then presented with the pre-test. It was not allowed to skip items until the maximum response time was exceeded. When participants had given their response they could immediately move on to the next item. After finishing the pre-test the researcher opened one of the instruction videos on the computer and left the research room. Thereafter participants were asked if they had any questions about the content of the video. Three training items were presented to practice the presented heuristic. For each condition, training items were selected on which the

(14)

trained heuristic was perfectly applicable. With these items there was no maximum response time and asking questions was allowed. Afterwards the participants were presented with the post-test in the exact same fashion as the pre-test. Finally, participants were asked to fill in a short survey about their experience of the test. They were then thanked and could be rewarded course credits if needed.

(15)

Results

The data showed no missing values, all data were used in the analysis.

Participants who gave an incorrect response were assigned a maximum response time (60 seconds) on the specific item. First, raw mean response times, accuracy rates and CISRT scores were assessed (Table 1 – 3).

Table 1: Mean and standard deviation response times.

Pre Post

Forward Backward Forward Backward

Forward Condition 19.44 (7.86) 40.44 (13.76) 17.02 (5.02) 37.74 (12.63) Backward Condition 15.88 (6.20) 44.99 (12.80) 28.87 (11.39) 35.97 (12.19) Control Condition 15.07 (5.40) 44.28 (12.82) 19.92 (8.52) 34.68 (12.69) Mean 16.79 (6.49) 43.24 (13.13) 21.94 (8.31) 36.13 (12.50)

Table 2: Mean and standard deviation accuracy rates.

Pre Post

Table 3: Mean and standard deviation CISRT scores.

Pre Post

(16)

Because the CISRT scores combine response accuracy and response times, data analysis was mainly focused on these results. Paired sample t-tests were

conducted between the mean CISRT scores on the forward and backward items of the pre-test for all conditions. It was shown that in all conditions the CISRT scores on the forward items of the pre-test were significantly higher then the CISRT scores on the backward items of the pretest (forward: t(18) = 7.015, p < .05; backward: t(18) = -10.167, p < .05; control: t(17) = -10.736, p < .05).

A repeated measures ANOVA has been conducted with the mean CISRT scores from table 3. This repeated measures ANOVA has been conducted once for the forward items and once for the backward items. Condition was used as the between subjects variable and time (pre or post-test) as the within subject variable. There was a significant effect of time, F(1, 53) = 15.121, p < .05. CISRT scores differed

significantly between the forward items of the pre and the post-test. In addition, there was a significant interaction between time and condition, F(2, 53) = 11.430, p < .05. Inspection of figure 4 shows that this interaction can mainly be understood by the strong decrease in CISRT scores of participants in the backward condition. Posthoc paired sample t-tests showed that both the backward condition and the control condition scored significantly lower on the post-test (backward: t(18) = 4.805, p < .05; control: t(17) = 2.278, p = .036.

Fig. 4. Mean CISRT scores

of all conditions on forward items of pre and post-test.

(17)

For the backward items there was a significant effect of time, F(1, 53) = 30.463, p < .05. There was however no significant interaction between time and condition, F(2, 53) = 3.044, p = .056. Figure 5 shows that almost the exact same growth in CISRT score applies to the control condition and the backward condition.

Fig 5. Mean CISRT scores of all conditions on backward items of pre and post-test.

Above results do not show any effect of the heuristic training on the backward items. However, a repeated measures ANOVA with the mean accuracy responses on the backward items gave some interesting results. Again condition was used as the between subjects variable and time (pre or post-test) as the within subject variable. There was a significant effect of time, F(1, 53) = 13.45, p < .05. However, this time there was also no significant interaction between time and condition, F(2, 53) = 2.42,

p < .05. Figure 6 shows the interaction plot of the mean accuracy responses on the

backward items. Although there was no significant interaction, it is highly interesting to note that paired sample t-tests showed that only the backward condition grew

(18)

significantly between the pre and post-test (forward: t(18) = -1.323, p = .202; backward: t(18) = -4.789, p < .05; control: t(17) = -1.644, p = .119).

(19)

Discussion

This research examined if by training the backwards heuristic, which is based on backward reasoning, the success rate of solving the Number Game increases. Results show that such increase in success rate, which was determined by response accuracy and response time, did not take place. After being trained in applying the

backward heuristic, participants’ success rate on the backward items of the post-test

did not increase. Striking was however, that the response accuracy on the backward items of participants in the backward condition did show a significant increase between the pre and post-post. It seems that by being trained participants in the backward condition acquired the necessary skills to give a correct response to a high proportion of the backward items in the post-test. It should be noted that this is a very cautious conclusion, since there was no such increase in CISRT scores.

This study again clearly revealed a strong phase transition between certain instances of the Number Game. The results convincingly show that the instances of the Number Game that comply with forward reasoning are solved much easier than all other instances. This confirms our expectation that human cognition has an inclination towards forward reasoning in this kind of problem solving. By showing this

inclination in heuristic use can also be found making use of a mode of presentation that does not stimulate forward reasoning, it can be concluded that the design of Math Garden does not play a decisive role in causing the phase transition of figure 2.

It remains unclear however what underlies this phase transition. Our study seems to disconfirm that the phase transition is based on heuristic use alone. The backward

heuristic we proposed did not cause a disappearance of the phase transition, as was

hypothesized. This is a peculiar finding, since the proposed heuristics can be traced back to the inner structures of the instances of the Number Game. It can be easily shown that backward reasoning perfectly applies to the harder instances in figure 2, the solutions of these instances all end with a multiplication or a division. The

backward heuristic fits to these instances accordingly. The proximity heuristic, on the

other hand, perfectly applies to the easier instances in figure 2. Both heuristics will not be of any help when applied to instances on the other side of the phase transition. These facts all point to our proposed foundation of the phase transition.

A possible and very plausible alternative explanation for not finding the

(20)

during this research. The demonstrated growth in accuracy rates on backward items after receiving a backward heuristic training seems to indicate there actually is a distinction between the easier and harder instances of the Number Game as we

proposed. This distinction can however only be demonstrated by means of a thorough training. Training in our research only consisted of watching a short instruction video of the appropriate heuristic. In these videos the heuristics were demonstrated with a few examples after which it was expected participants could apply the presented heuristic. For the forward training this seems to have worked, the success rate on forward items clearly showed an increase. This increase can however be explained by the intuitive nature of this heuristic. As was concluded, human cognition shows an inclination towards forward reasoning. The instruction video on the proximity

heuristic could then for example only be a confirmation of the already existent

solving strategies, or a completion of a present inclination in our cognition.

Backwards reasoning is not intuitive like this. To master this kind of problem solving

strategies it can be expected that a lot of attention and exercise is needed. An instruction video of eight minutes would not be enough for a sufficient grasp of this kind of reasoning. This also seems to be in accord with the finding that the success rate on forward items actually decreased for the backward condition. The insufficient grasp of the backward heuristic could have caused a lot of uncertainty in solving all instances of the Number Game. Vainly trying to apply the backward heuristic resulted in a lot of disturbance in solving the forward items. Further research should invest more in a decent training. Participants should be offered more time to practice applying the presented heuristic. This way a good grasp of both the proximity

heuristic and the backward heuristic could be guaranteed. This could assure better

conditions for bringing to the light the potential foundation of the discussed phase transition.

If heuristic use is what underlies the phase transition, and so it shows backwards

reasoning is suitable for solving harder instances of the Number Game, finding a

suitable training could be of great educational importance. Such training could be easily incorporated in educational systems like Math Garden. This would possibly lead to a growth of pupils’ expertise in simple arithmetic operations.

Besides educational goals, further research should also be aimed at the alleged NP-Completeness of the Number Game. First and foremost a full reduction to a known NP-Complete problem should be made, to formally proof the Number Game is

(21)

NP-Complete. If the Number Game turns out to be NP-Complete, it would be very interesting to draft an algorithm which can optimally solve forward, backward and even harder instances of the Number Game. A simulation study could then be conducted to examine if another phase transition can be found in the spectrum of the Number Game. As mentioned, phase transitions are common in NP-Complete problems. Finding and knowing what underlies such a potential phase transition would be another step forward in fully understanding this highly interesting game.

For now it is concluded that backward heuristic training does not result in the disappearance of the presented phase transition and thus does not cause a higher success rate in solving the Number Game. However, this research has shown signs the phase transition is actually based on the distinction between forward reasoning and

backward reasoning. To completely throw away the ideas proposed in this paper

(22)

References

Borgs, C., Chayes, J., & Pittel, B. (2001). Phase transition and finite-size scaling for the integer partitioning problem. Random Structures and Algorithms, 19, 247-288.

Gent, I. P., & Walsh, T. (1996). Phase transitions and annealed theories: Number partitioning as a case study. In W. Wahlster (Ed.), Proceedings of the 12th

European Conference on Artificial Intelligence (ECAI-96) (pp. 170-174).

New York : John Wiley & Sons.

Hayes, B. (2002). The easiest hard problem. American Scientist, 90, 113-117. Klinkenberg, S., Straatemeier, M., & Van der Maas, H. L. J. (2011). Computer

adaptive practice of Maths ability using a new item response model for on the fly ability and difficulty estimation. Computers & Education, 57, 1813-1824.

Kurzen, L. (2011). Some ideas for the Numbers Task. Unpublished manuscript, University of Amsterdam.

Larkin, J., McDermott, J., Simon, P. D., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science, 208, 1335-1342.

Van der Maas, H. L. J., & Nyamsuren, E. (2015). The number game: An educational

game and challenge for cognitive modelling. Unpublished manuscript,

University of Amsterdam.

Van der Maas, H. L. J., & Wagenmakers, E. (2005). Psychometric analysis of chess expertise. The American Journal of Psychology, 118, 29-60.

Maris, G., & Van der Maas, H. L. J. (2012). Speed-accuracy response models: Scoring rules based on response time and accuracy. Psychometrika, 77, 615-633.

Mertens, S. (2001). A physicist’s approach to number partitioning. Theoretical

Computer Science, 265, 79-108.

Mertens, S. (2003). The easiest hard problem: Number partitioning. In A. Percus, G. Istrate & C. Moore (Eds.), Computational complexity and statistical physics (pp. 125-140). Oxford: Oxford University Press.

Torenvliet, L. (2014). Algoritmen en complexiteit. Unpublished manuscript, University of Amsterdam.

(23)

Willingham, D. T. (2007). Cognition: The thinking animal (3rd ed.). Englewood Cliffs: Pearson/Prentice Hall.

(24)

First steps in proving the Number

Game is NP-Complete.

Student: Wessel de Jong Studentnumber: 10206620

Words: 1465

Attendant: dhr. prof. dr. H.L.J. (Han) van der Maas Institute: University of Amsterdam

(25)

The Number Game is a highly interesting game that shows much potential in assisting teaching basic arithmetic operations and problem solving in primary education. Although it is hypothesized this game is NP-Complete, it still remains unanswered if the Number Game is part of this subclass of problems. To prove the Number Game is NP-Complete a full reduction of all instances of an already known NP-Complete problem to an instance of the Number Game is needed (Torenvliet, 2014). In this way Kurzen (2011) has proven the Number Game to be NP-Complete if only the operators ‘+’ and ‘−‘ are allowed to be used. Kurzen (2011) made use of a reduction of all instances of the well-known NP-Complete Partition Problem. In this problem players are asked to form two non-overlapping sets with equal sums, given a set of n positive integers. Given the same set of integers, an instance of the Number Game with target number 0 (T = 0) and operators ‘+’ and ‘−‘ would be equal to an instance of the Partition Problem. It can be shown that all instances of the Partition Problem are equal to instances of the Number Game when the only operators allowed to be used are ‘+’ and ‘−‘. Consequently, part of the Number Game is proven to be NP-Complete. Thereby, only ‘×’ and ‘÷’ instances are equal to only ‘+’ and ‘−‘ instances when made use of an ln-transformation. However, a formal prove of the alleged NP-Completeness of all instances of the Number Game, including those in which the four basic operators are allowed to be used, is still missing. Although this paper will not present a full reduction from a known NP-Complete problem, some first steps in determining the complexity of the Number Game will be taken. It will be examined what the consequences are for this complexity if multiplication and division are allowed.

First, only adding multiplication will be considered. Naturally it would seem that instances of the Number Game with operable operators ‘+’, ‘−‘ and ‘×’ are harder than those instances with only ‘+’ and ‘−‘. Adding an extra operator would only make the problem space bigger. With every consecutive step in solving the Number Game there is an extra option to consider, which will accumulate to significant more potential solutions to consider using a ‘brute force’ solving strategy (Willingham, 2007). With a growing integer set, the time needed to solve the Number Game should be expected to grow on an even faster rate. Considering the Number Game with only ‘+’ and ‘−‘ is proven to be NP-Complete and adding ‘×’ would result in even harder problems, surely it seems logical that those instances with ‘+’, ‘−‘ and ‘×’ are NP-Complete. Intuitively, this NP-Complete label seems to presuppose that with an

(26)

ever-growing integer set, the Number Game becomes harder every time an integer is added to the given set. The opposite seems to be true. Allowing for multiplication results in the possibility of applying a certain solving strategy which cannot be applied to only ‘+’ and ‘−‘ instances. This solving strategy makes the instances with large integer sets easier to solve than those with small integer sets. With the possibility of

multiplication the Number Game can be solved by first finding the target number with just a subset of the given integer set. It is then only needed to form a 0 with again just a subset of the remaining integers. This 0 can be multiplied by the sum of the unused integers to remain with 0. All that needs to be done now is adding to or subtracting from the previously formed target number the remaining 0. This strategy still seems very hard for those instances with small integer sets, after all one needs the

opportunity to separately form the target number and a 0 with just a subset of the integer set. However, the probability of this being possible grows with a growing integer set. At a certain point in this growth it will be possible to easily find the target number and a 0, while only using a small subset of the total integer set. The difficulty of the ‘+’, ‘−‘ and ‘×’ instances of the Number Game grows with an increasing integer set unto a certain point, the point after which the probability of forming the target number and a 0 exceeds a certain threshold. After this point the difficulty will decrease because of the growing probability of forming the target number and a 0.

This phase transitions from an exponential growth to a sudden decrease in dissolution time are common in NP-Complete problems (Mertens, 2001). Figure 1 shows such a transition for the Partition Problem, which seems to resemble the expected phase transition in the Number Game. Each data point represents the average of 100 random problem instances with 25-bit integers (Mertens, 2001). The number of nodes needed to solve the Partition Problem grows exponentially to the point where the probability of a perfect partition jumps from 0 to 1. After this point the number of nodes needed decreases. It would be interesting to conduct a similar simulation study with the ‘+’, ‘−‘ and ‘×’ instances of the Number Game and a growing random integer set. An algorithm should be created that matches above solving strategy. First, the algorithm should find the target number and a 0 to go through the steps corresponding to this strategy. If this turns out to be impossible, the algorithm should alter it’s solving strategy and continue with a ‘brute force’ strategy. This algorithm should then be used to solve instances of the Number Game with a growing random integer set. It should be examined if this simulation would show a

(27)

similar sudden jump in number of nodes needed to solve the selected instances. Beforehand it is not at all clear this jump will be as sudden as shown in figure 1. With the Partition Problem the probability of a perfect partition suddenly jumps from 0 to 1. With the Number Game the probability of finding the target number and a 0 seems to steadily rise with a growing integer set. The proposed simulation study can, therefore, bring some more clarity on the complexity of the Number Game.

What applies to multiplication also applies to division. When considering ‘+’, ‘−‘ and ‘÷’ instances of the Number Game it still seems logical that with a growing integer set the difficulty of the Number Game grows. However, division also offers the opportunity to ‘cancel out’ the rest of the integer set by finding a 0. Following the same line of thought as the solving strategy with multiplication, in this case one again would need to find the target number and a 0 with just a subset of the total integer set. Although it is not possible to divide the sum of the remaining unused integers by 0, it is possible to divide 0 by this sum. In this manner one would end up with a 0 and the target number and all that remains to be done is a simple addition or subtraction. Again adding an extra operator to the set of operable operators brings strategic benefits. The same algorithm applied to the only ‘+’, ‘−‘ and ‘×’ instances should be applied to conduct a simulation study with only ‘+’, ‘−‘ and ‘÷’ instances. It is expected the result of this study will be very similar to the result of the multiplication simulation study.

It has been shown that adding multiplication and division makes solving the Number Game easier with large integer sets. Both operators offer the possibility to ‘cancel out’ most of the given integer set. Instances of the Number Game with large integer sets growing to infinity, which are expected to be hard to solve, become very easy to solve. Only a small subset of the integers needs to be considered. However, this does not necessarily have to mean the Number Game is not NP-Complete. As has been discussed, phase transitions are common in NP-Complete problems. Therefor, further research should mainly focus on finding an algorithm that can optimally solve the actual Number Game with all four basic operators. The proximity heuristic and the

backward heuristic are not sufficient because they are only able to solve a small

subset of the whole spectrum of the Number Game. Simulation studies with such an algorithm could reveal more proof of the Number Game actually being NP-Complete. Such studies can however never result in conclusive conclusions about the

(28)

NP-Completeness of the Number Game. We therefore like to invite the mathematicians to come up with a formal proof showing the Number Game really is NP-Complete.

(29)

References

Kurzen, L. (2011). Some ideas for the Numbers Task. Unpublished manuscript, University of Amsterdam.

Mertens, S. (2001). A physicist’s approach to number partitioning. Theoretical

Computer Science, 265, 79-108.

Torenvliet, L. (2014). Algoritmen en complexiteit. Unpublished manuscript, University of Amsterdam.

Willingham, D. T. (2007). Cognition: The thinking animal (3rd ed.). Englewood Cliffs: Pearson/Prentice Hall.