ACT-‐R modeling to investigate the effect of multiple graphical representations on
fraction learning
Margreet Vogelzang
1648497 July 2012
Master's Thesis
Human-‐Machine Communication University of Groningen, the Netherlands
Internal supervisor:
Prof. Dr. Niels Taatgen (Artificial Intelligence, University of Groningen)
External supervisor:
Dr. Vincent Aleven (Carnegie Mellon University, USA)
Abstract
To improve children's understanding of fractions, many curricula have started to use graphical representations (GRs). These would help children to relate to the problem more than when only a symbolic representation is shown. Moyer et al.
(2002) state that the use of GRs can be especially helpful in a computer tutor, because of the interactions that are not possible on paper. The Rational Number Project (RNP, Cramer et al., 1997) recommends using multiple GRs (MGRs) in one curriculum: potentially children could then combine the benefits of the separate GRs.
Rau et al. (in press) performed a classroom study with an intelligent tutoring system for fraction learning with 4th and 5th grade students. The study had a between-‐subjects design with one single GR (SGR) condition and multiple MGR conditions. The MGR conditions tested which pattern of presentation (schedule of practice) of GRs in an MGR tutor is optimal for learning. The results show that the MGR tutor improves children’s knowledge more than the SGR tutor. Furthermore, small differences between the MGR conditions were found.
This project used the data from the experiment of Rau et al. (in press) for the development of three ACT-‐R models. The models are used to investigate the specificity of students' knowledge when learning fractions, especially when using MGRs. It is expected that before using the tutor children have a few general strategies they apply to many different problems, and after the tutor children learned more specific strategies. The different models that are fit to the data represent different levels and types of specialization. Based on students' correct strategies and error strategies, the models compare whether students are more apt to use specific strategies per GR or per problem type. The results show that the tutor increases the number of specific strategies children use per problem type. Moreover, students in the SGR tutor condition sometimes generalize different from students in the MGR conditions.
Keywords: Multiple representations, fractions, cognitive modeling, model-‐data fit.
1. Introduction ... 4
1.1 Graphical representations compared ... 5
1.2 Multiple vs. single graphical representations ... 6
1.3 MGR patterns of presentation ... 7
1.4 Cognitive models ... 10
2. Research questions ... 13
3. Method ... 15
3.1 Data analysis ... 15
3.1.1 Subject selection ... 15
3.1.2 Strategy identification ... 15
3.1.2.1 Examples ... 16
3.1.3 Final data ... 23
3.2 ACT-‐R model ... 25
3.2.1 General assumptions ... 25
3.2.2 Design and model settings ... 26
3.2.3 Strategy selection ... 27
3.3 Parameter fitting of multiple models ... 28
3.4 Information criteria ... 31
4. Results ... 33
4.1 How to compare models ... 33
4.2 Relative model fits ... 34
4.2.1 All problems included ... 34
4.2.2 Pairwise comparison of GRs ... 36
5. Conclusion ... 41
6. Discussion and Future research ... 44
7. Acknowledgements ... 47
8. References ... 47
Appendices ... 51
Appendix A: Strategy data ... 51
Appendix B: Example test ... 55
Appendix C: Complete relative model fits ... 65
4
1. Introduction
Children’s knowledge of fractions is not as it should be; research assessing the achievement of elementary and secondary students in the United States showed that 60% of the 4th grade students (age 9 -‐ 10) did not know whether 1/4 was greater than 1/5 and 50% of the 8th grade students (age 13 -‐ 14) could not order three fractions (NAEP, 2007). To improve the mathematical knowledge of children, including their understanding of fractions, many curricula have started to use graphical representations (GRs). Working with graphical representations provides for more robust learning, as they provide a way for children to relate to the problem (e.g. DeWindt-‐King & Goldin, 2003; Mack, 1995); a fraction presented graphically can seem more familiar than one represented symbolically. Moreover, by using a GR, important aspects of the learning content can be emphasized. The National Council of Teachers of Mathematics (NCTM) wrote that using GRs to depict fractions should be standard in any school’s mathematics program (NCTM, 2000).
Furthermore, the NCTM encourages schools to use technology to teach mathematics, as it is essential and encourages students’ learning (NCTM, 2000). An important use of technology in children’s mathematics learning is the use of computer tutors. Computer tutors are computer-‐
based instructional technologies that are designed to aid children in learning a specific subject.
A specific kind of a computer tutor is a cognitive tutor (in the literature also referred to as an ITS, an intelligent tutoring system). A cognitive tutor is a computer-‐based tutor in which the instructions displayed are based on a cognitive model of the competence of the student using it (Anderson et al., 1995). It has been shown that cognitive tutors can have advantages over traditional methods for learning in numerous areas of mathematics (Ohlsson, 1991; Anderson et al., 1995; Koedinger & Aleven, 2007; Koedinger & Corbett, 2006). These tutors have the advantage of providing step-‐by-‐step guidance, allowing children to work at their own pace, and the advantage that they can adapt to each individual child: The level of the posed questions can be adjusted to the level of each child. Children can ‘learn by doing’ and receive feedback on their mistakes, even at an intermediate step in the answer process. Because of this immediate feedback, students are forced to evaluate their own actions and answers immediately. This includes an evaluation of the actions that were done with the GRs (Sarama &
Clements, 2009).
Moyer et al. (2002) state that GRs can be especially useful in a computer tutor, because of the interactions that are not possible on paper (dragging, dropping, automatic updating of figures etc.). For example, Rau et al. (2009) used a cognitive tutor with GRs that change interactively, so that children learn to draw parallels between their own actions with the GRs and the resulting changes in the symbolic representations.
This paper will focus on the specific area of fraction learning with GRs, to be able to offer more detailed insights into the challenges of the field.
5 1.1 Graphical representations compared
Although there is a general consensus about the use of GRs to help children understand fractions better, it is not clear which one is best. Numerous different GRs are possible, for example, area models (such as circles and rectangles), linear models (such as number lines), or discrete models (such as coins or other sets of objects) (Rau et al., 2009).
Different GRs have different advantages. The number line for example fits the theory of mathematics learning called ’magnitude representation’, which states that children have to learn to understand that all real numbers have magnitudes that can be ordered (Siegler et al., 2011). This theory suggests that a number line would be the most useful GR for teaching children fractions, because it helps to view fractions as numbers with magnitudes which can be assigned to specific locations on the number line. On the other hand, Cramer et al. (2008) found that children performed better when using a circle representation than other GRs (including a number line) when they were asked to compare two fractions and find common denominators. Mack (1995) found that circle representations allow children to link their existing knowledge of dividing and sharing to fractions. Partially in line with the research that promotes using circles, Caldwell (1995) found that area models in general promote fraction learning.
By contrast, Hiebert and Tonnessen (1978) found that when young children were asked to divide candy amongst some stuffed animals, they performed better with a discrete set of candies than with a string of licorice to divide up or a circular clay pie. Their explanation is that when dealing with separate pieces of something, children can start working on the solution without seeing the set as a ‘whole’ and without deciding beforehand how large the parts for each party are going to be (no division is needed). This however would cause one to think that, even though the task was performed correctly, the child still does not have a conceptual understanding of fractions. Hiebert and Tonnessens explanation of the results is consistent with the theory of Piaget et al. (1960) that making fourths is easier than making thirds in e.g. a circle representation, because fourths can be solved by first making halves and dividing those up again, while thirds have to be constructed directly from the whole.
Besides having advantages, different GRs also have different disadvantages. A specific aspect of number line estimations is that children do not always display a feeling for linearity when using a number line; estimates of 3-‐ and 4-‐year-‐olds for the numbers 0 to 10 follow a logarithmic pattern, while the estimates of 5-‐ and 6-‐year-‐olds follow a linear pattern (Berteletti et al., 2010). Therefore, area models such as circles and rectangles could make it easier for children to understand that all sections of a fraction (a GR of a fraction) should have the same size. On the other hand, area models like circles could have the downside that it is difficult for children to divide them up into pieces. The difficulties that children have with circles may stem from their informal knowledge: knowledge formed by experiences in real-‐life (Mack, 2001). In daily life children encounter many situations with sets (e.g. a number of cookies) that need to be divided, not with one cookie that needs to be divided in a number of pieces. Therefore, it might seem unnatural to children to divide up one ‘item’ into more than two parts (two parts are more natural because halves occur frequently in daily life).
In addition to the research on single GRs and the differences between them, there has been research on the use of multiple GRs in one curriculum: potentially children could then combine the benefits of the separate GRs (e.g. Rational Number Project (RNP, Cramer et al., 1997)). The next section will discuss this possibility.
6
1.2 Multiple vs. single graphical representations
Research suggests the use of multiple graphical representations (MGRs) in learning materials can improve student learning when compared to using a single graphical representation (SGR) in complex domains (Schnotz & Bannert, 2003; Ainsworth et al., 2002). However, which GRs should be combined, and how they should be combined, is still to be determined for many task domains. What is clear is that children must get enough practice and perform different tasks with the representations to be able to benefit from using MGRs (De Jong et al., 1998). Practice is needed for children to find a link between the GRs and therefore learn to understand the deeper concepts (National Mathematics Advisory Panel, 2008). More precisely, students have to conceptually understand each one of the representations to be able to make a meaningful connection between them (Ainsworth, 2006).
The Rational Numbers Project (RNP) is a complete fraction learning curriculum that uses MGRs and focusses on making connections between these GRs, to be able to gain an advantage from using them (Cramer et al., 1997). Cramer at al. (2002) compared the RNP curriculum to commercial curricula that were used at schools at that time. In a number of specific areas of fractions (concepts, order, transfer and estimation), 4th and 5th grade students using the RNP curriculum outperformed those using the commercial curricula.
Rau, Aleven and Rummel (2009) performed a study focusing on the difference between using an SGR and using MGRs in a computer tutor. Their research used five different GRs, each chosen to suggest a certain interpretation of fractions: number line, circle (pie chart), rectangle, stack, and set (from Kaminski, 2002). There were two tutor conditions: an SGR condition showing only a number line representation and an MGR condition using all five GRs consecutively (each problem was seen once with each GR). The study was done with 6th grade students.
Half of the students in each tutor condition group were asked to self-‐explain the connection between the GR and the symbolic representation. The students were provided with feedback at all steps of the tutor. Rau, Aleven and Rummel (2009) expected that using the MGR tutor would lead to better learning than the SGR tutor. Because the MGR tutor required linking the different representations, the researchers anticipated that students in the MGR condition would benefit from self-‐explanation.
The results show that the MGR condition aids learning, but only when students are asked to self-‐explain. One explanation for this result is that when a child is taught fractions using different GRs, he/she will learn the concepts more robustly, because there is a greater need to understand the underlying concepts when switching GRs. This is especially true when self-‐
explaining, because the link between the GR and the symbolic representation is made explicitly. Note, however, that students in the MGR condition performed worse than in the SGR condition when no self-‐explanation was necessary. This indicates that MGRs without explicit linkage might be confusing rather than helpful.
Also note that the in the described experiment five different GRs were used, which might be too many and therefore the link between the GRs might be more difficult to make. The optimal number (and type) of GRs to be used in an MGR tutor is still open to investigation.
7 1.3 MGR patterns of presentation
In order to get closer to developing the optimal fraction tutor system, Rau et al. (in press) investigated which pattern of presentation of the GRs in an MGR tutor would be optimal. The patterns of presentation could influence how well children understand each GR and how well they can find a link between them. Prior research suggested that the pattern of presentation of problem types influences learning. Interleaving problem types seems to encourage deep processing as opposed to blocking problem types, and thus improve learning (De Croock et al., 1998). Interleaving patterns of presentation might have a similar effect. In an earlier study, Rau et al. (2010) found that the benefits from interleaving problem types are larger than the benefits of interleaving GRs, but the question remained if interleaving GRs in addition to interleaving problem types could improve children’s learning further.
In the experiment of Rau et al. (in press) 290 4th and 5th grade students (age 9 -‐ 12) used a cognitive tutor for about five hours. While using the tutor, the children were presented with 108 problems. Each of these problems used a circle, rectangle or number line representation, but only one representation per problem was shown. Because the experiment was designed to investigate the influence of different patterns of GRs on learning, there were a number of different patterns of presentation designed to control the changes from one GR to another.
The experiment used five experimental conditions: Blocked, Moderate, Interleaved, Increased and SGR.
In the Blocked condition children saw blocks of 36 problems with the same GR, before changing to the next. Therefore, there were a total of three blocks of 36 problems, and no GR reoccurred after having switched to another GR. The Moderate condition showed the problems in blocks of six with one GR, before switching to another GR. In the Interleaved condition the GR was changed after each problem. The Increased condition slowly decreased the size of the blocks of problems with one GR, starting with blocks of 12 problems per GR, and ending with blocks of one. Finally, there was an SGR control condition, in which only one GR was used (either circle, rectangle or number line).
As in the tutor of Rau, Aleven and Rummel (2009), this tutor gave feedback at each step and hints were available if children needed them. The GRs were interactive and could be modified by e.g. selecting, dragging and dropping elements of the GRs and clicking partitioning buttons.
The different types of tutor problems required different reasoning processes. The six problem types were: Identifying fractions, Making representations, Reconstructing the unit from proper, unit and improper fractions and Naming improper fractions.
These were presented in an interleaved manner, in which after each 6 problems the type changed. This way of scheduling problem types was most successful in the study of Rau et al. (2010).
The students were tested in a pre-‐, immediate post-‐ and delayed posttest. These tests each contained 25 questions, which also varied in problem type. Each type of question that used a GR occurred on the tests three times, once with each GR (circle, rectangle and number line).
Additionally, there were questions without a GR that contained story problems with fractions.
An example test is given in Appendix B.
The results of the study showed that students improved on number line test items regardless of which tutor condition they were in. The MGR conditions proved to be better than the SGR condition on a number of problems types (although not all) and students retained their newly gained knowledge longer: the students in the MGR condition performed better on number line and area model items on resp. the posttest and the delayed posttest than students in the SGR condition (see Figures 1 and 2). When examining the difference between the MGR conditions
8
all three types of interleaved condition performed better than the blocked condition.
Figure 1: The average scores on area model items (circle and rectangle) in the MGR and SGR conditions on the three tests (error bars 95%)
Figure 2: The average scores on number line items in the MGR and SGR conditions on the three tests (error bars 95%)
Rau et al. (in press) conclude from these results that MGRs can aid students’ learning, even when the tutor has only been used for a short amount of time. These benefits will still be present one week after the tutor usage, when the delayed posttest was taken.
The results look promising because the MGR tutor improves children’s knowledge more than
9
the SGR tutor. As stated before, it is generally assumed that this effect occurs because MGRs encourage deep processing and through this a robust understanding of the concept of fractions is developed. However, these terms are relatively vague and it remains unclear how exactly MGRs affect fraction learning.
10 1.4 Cognitive models
The traditional approach for testing the effect of tutors has been to perform classroom experiments. These experiments have been very informative and can confirm there is an effect of using MGRs, but this approach does not seem to give an answer to the question stated above; how do MGRs affect fraction learning? To investigate this question, an alternative approach has to be taken. One option is to examine problem solving with cognitive models. A cognitive model can simulate human problem solving and might therefore give new insights into how students learn from specific problem.
A cognitive architecture that can be used for modeling the solution process of problems such as fractions is ACT-‐R (Adaptive Control of Thought-‐Rational; Anderson et al., 2004). This architecture is a useful tool to model and explain human behavior and cognition. It ensures psychological plausibility as it is constructed to reflect assumptions about human cognition and is based on experimental data. An ACT-‐R model is a process model: It can solve problems and, on the basis of a model, predictions can be made about the solution process: which choices are made, and how (with which steps) problems are solved.
The ACT-‐R architecture has a number of elements that one should know to be able to grasp the general idea of a model. Based on human cognition, ACT-‐R has two basic memory modules: the declarative memory and the procedural memory. The declarative memory contains all factual knowledge, while the procedural memory contains productions; knowledge about how to do things. The ACT-‐R architecture is especially useful for modeling cognitive tasks, such as learning and understanding. ACT-‐R models have been used for modeling and investigating numerous types of problem solving and learning tasks, e.g. word learning tasks (Pavlik & Anderson, 2008), time estimation tasks (Taatgen et al., 2007) and mathematical tasks (Koedinger & McLaren, 1997).
Models in ACT-‐R can be built up in numerous ways, depending on the task at hand. In this research, the process of solving fraction problems will be modeled. But even for this specific problem there are multiple ways to construct a model. An important part of building a problem solving model is to determine how decisions are made: are they made at a set point in time or along the way? In our models, we make the assumption that solving a fraction problem starts with a decision as to what solution strategy one is going to (attempt to) apply.
This decision of which solution strategy is going to be used can be modeled in ACT-‐R in more than one way. The main two options are based on activation and utility, and will be discussed below.
Activation-‐based selection is done as follows: A chunk is a piece of knowledge in the declarative memory (DM), the memory that contains factual knowledge. In the case of strategy selection, each strategy is represented by a chunk. Every chunk has an activation that represents an estimation of how relevant that piece of knowledge is in the current context.
When searching for a solution strategy to apply to a problem, a retrieval request is made to the DM to retrieve a chunk (strategy) that fits the specifications of the problem. The chunk with the highest activation will be retrieved from the DM and is used by the model. The activation is influenced by a noise component, so that not always the same chunk is retrieved, but still chunks with higher activations have a higher chance of being used. The base-‐level activation (activation without noise) of a chunk increases every time it is encountered (seen in the world or retrieved from memory). The activation of a chunk decreases if it is not encountered for a while. Furthermore, the base-‐level activation of a chunk increases if a chunk that is encountered is associated with it. This effectively means that not only the activation of the encountered chunk in increased, but also the activation of chunks that are related to it and
11
that are related to the current context. The strength of the association between the encountered and the related chunk determines how much the activation of the related chunk is increased.
Activation-‐based strategy selection has the upside of being relatively simple, but the downside that it is difficult to include the effect of feedback; once a chunk is retrieved its activation will increase, regardless of the outcome of the application of this strategy. So, to select the correct strategy with activation-‐based selection, the model will have to be presented with many examples of this strategy; it is based primarily on repetition.
Utility-‐based selection means the selection of production rules. Production rules are rules that are executed in order to get to a solution to a problem. These production rules are assigned with a utility. This utility can be updated; every time feedback on a solution is given, the utility of all previously used production rules is adjusted (positively or negatively, depending on the correctness of the solution). Therefore, utility-‐based selection does not depend on repetition, but on feedback: It does not necessarily need examples of the correct strategy to update its knowledge; it can use a penalty for the wrong strategies (resembling reinforcement learning).
So, the type of strategy selection would depend on the task at hand and the way in which this task is approached. The choice between activation-‐based learning and utility-‐based learning will have consequences when making tutors: is repetition or feedback the key to learning correct strategies?
An example of a cognitive model of early algebra problem solving is given by Koedinger and McLaren (1997). They built two types of models. First a model they call type I, which decides on the strategy to apply through production rules and utility. In this model, each strategy has its own production rule and the utility of the production rules directly determines which strategy gets executed. However, a strategy in which a miscalculation is made has to be included as a separate strategy in this model, while in reality the error might be made halfway through executing a correct strategy.
Secondly, Koedinger and McLaren (1997) built a model type II, in which not every strategy was explicitly represented by a production rule, but the execution of production rules in a certain sequence led to the completion of a strategy; different strategies had overlapping production rules and errors could be made during the process of solving a problem, instead of explicitly deciding which strategy (including faulty strategies) is going to be applied beforehand. This model is therefore more realistic and, as their results show, performs equally well when compared to experimental data as the type I model.
The type model II supports the notion of implicit rather than explicit knowledge and errors (through implicit strategy selection). To be able to draw this conclusion, the ACT-‐R modeling was particularly useful, as it gave insight into the solution process itself rather than just analyzing the outcome (the answers of the students). Koedinger and McLaren (1997) conclude that their results support the notion that the algebraic knowledge that was tested is acquired by doing more than by directly communicating strategies (by e.g. a teacher).
Goldstein and Gigerenzer (2002) performed an experiment in which American students had to estimate which of the two given German cities was bigger. They found that if students are only familiar with one of the cities (only one is available in memory), they tended to choose this one as being bigger (recognition heuristic). This heuristic shows better results than when both cities are familiar and a choice has to be made. So, the availability of cities in memory plays a
12
large role in this decision process. The heuristic shows that the idea of activation and availability in memory that is used in ACT-‐R is supported by experimental data; it is a widely accepted idea in (cognitive) science.
Schooler and Hertwig (2005) used data of the time and frequency with which German cities were mentioned in a specific newspaper. They made an ACT-‐R model and gave the data of the cities as input, shaping the DM of the model. When letting the model do the same task as the students did (determine which city is bigger), the activation of a chunk (city) in the DM of the model is a predictor of the chance that a student will recognize a city. The model fit the data well, which is a confirmation of the way memory and activation are used in ACT-‐R.
Another example of activation-‐based decision making is described by West et al. (2005); the game of rock-‐paper-‐scissors. When deciding what to do, a prediction can be made about what the opponent is going to do, based on the sequences of actions he/she has shown before. In their model, West et al. (2005) represent these sequences by chunks, and so the base-‐level activation of the sequence most likely to be used (the one that has been used most before), will be highest and will therefore be retrieved. This way, the model predicts the moves of the opponent and responds accordingly.
The studies described above give an impression of the different possibilities of investigating data with cognitive models and the additional information that can be gained through a cognitive model as opposed to an experiment. Cognitive models could also be applied to the field of fractions, in order to investigate the question of how MGRs affect fraction learning.
13
2. Research questions
This project investigates the influence of different GRs on children’s understanding of fractions. Children generally perform better after working with multiple graphical representations than with a single one. Why is this? What influence do the MGRs have on children’s understanding of fractions? We are specifically interested in finding out whether children’s knowledge and solution strategies are general or specific: do children know how to apply one solution strategy to multiple problems of the same type but with a different GR, or do they develop a different solution strategy for each problem with a different GR? Does a tutor influence the development of specific or general solution strategies?
So, the project also investigates the generality of strategies used on different GR; how well do children generalize over GRs? Are there two GRs that children generalize over more than others? If children know the correct way to answer a fraction problem with a circle, can they apply the same solution strategy to the same problem with a rectangle, or do they not see the similarities between the problems. The project investigates three GRs: circle, rectangle and number line. Are the area models more alike (like suggested in the Introduction), and therefore easier to generalize over than the number line?
Finding answers to these questions will provide insight into the effect that MGR tutors have on children when learning fractions. Understanding why MGR tutors aid learning could help to improve future tutors. Understanding which GRs children can generalize over and which not can be useful information when designing a tutor or educational system, as one can account better for what children will generally understand and what not.
To investigate the raised questions, we will take a closer look at the previously discussed data of Rau et al.’s (in press) tutor experiment though a cognitive model. The original experiment consisted of a pretest, a tutor, a posttest and a delayed posttest. Modeling children working with the tutor itself is complicated and, since we are interested in the effects of the tutor, looking at the pre-‐ and posttest data will suffice for this project. From the data, information was extracted that helped shape the cognitive models; what types of errors do children make?
How frequent are these error types?
Three ACT-‐R models were constructed to simulate a child solving the fraction problems presented in the pre-‐ and posttest. A model-‐based analysis was done to test the specificity of children’s incorrect and correct solution strategies that were extracted from the data.
Specifically, the models were used to investigate the generality/specificity of the strategies that children use to solve the fraction problems. The models focus on a number of possible specifications of knowledge that children may know and use: if there are no specific, only general, strategies known, children apply the same strategy to many different problems. This is an overgeneralization, which leads to many incorrect answers. If there are specific strategies known per GR, then the same strategy will be applied to different problems using the same GR. This will also lead to many incorrect answers, as the different problem types often need different strategies to be solved, even though the same GR is used. If children know a specific strategy for each problem type, the same strategies are applied to different questions with the same problem type, even though the problems use different GRs. This type of strategy specialization is most useful, as problems of the same type need the same strategy to be solved.
Which type of specialization do children use? And does the difference in performance between
14
the posttests of the SGR and MGR-‐condition groups stem from a difference in the use of specific strategies? Do children know more specific strategies per problem type after working with MGRs in the tutor? A child with little experience with fractions might have few strategies and apply those generally. The goal of an MGR tutor is to teach children specific problems per problem type. However, if the emphasis of the tutor is wrong or children misunderstand it, they might learn specific strategies per GR.
We expect to see a difference in specialization between children in the SGR and children in the MGR tutor conditions. The children in the MGR conditions are expected to develop a more robust understanding of fractions, and more knowledge of strategies that can be applied to different GRs, and therefore specify more per problem type. This specialization does not only include correct strategies, but also incorrect strategies that are used to solve problems. If children use the same incorrect strategies for problems with the same problem type but different GRs, the generalization they learned to make is good, but the strategy they use is not.
Still, this type of specialization is seen as a good thing, as children did learn to generalize over GRs.
Children in the SGR condition are expected to specify more per GR, using one strategy for multiple problem types with one GR. They are expected answer most of the questions with the practiced GR correct. However, they will not apply the same strategies to problems with unpracticed GRs, as they did not learn to make a connection between the practiced and unpracticed GRs.
Furthermore, both the circle and the rectangle are area model items so we expect children to apply the same strategy to circle and rectangle problems more often than to one of those GRs and the number line. This would be because the circle and rectangle are graphically more alike, and because the strategies that are needed to solve circle and rectangle problems are more similar to each other than to strategies needed to solve number line problems.
15
3. Method
3.1 Data analysis
The original data of Rau et al. (in press) consisted of a pretest, tutor, posttest and delayed posttest with 587 4th-‐ and 5th-‐grade students. In their paper, this number was reduced to 290 students, excluding students who missed at least one test day and who completed less than 67% of all tutor problems. The current analysis however, because it does not look at individual learning, but at the average performance over participants, did not exclude students who missed one or more test days. This way more data remained for analysis.
Three different test forms (with isomorphic problems) were used in the original experiment;
each child saw each test once (pre-‐, post-‐ and delayed posttest). The order of the test forms was counterbalanced over the subjects across pre-‐, post-‐ and delayed posttest.
The data that were used in this project were only the data from the pre-‐ and posttest. These data were used to analyze the influence of the tutor conditions, without looking at the logged data from the tutor itself. The pre-‐ and posttest originally consisted of 25 questions (problems) each. Most, but not all, of these problems used a GR which was circle, a rectangle or a number line. Every type of problem that used one GR also occurred with the other GRs; these problems occurred in groups of three, each of the same problem type but with a different GR. This means all children saw all GRs the same number of times in the pre-‐ and posttest.
From the pre-‐ and posttest 15 problems were selected for further analysis; 10 problems from the original tests were excluded. These were problems that did not involve any GR at all and problems that were too different from the other problems to be able to investigate if the solution strategies for that problem were also used for other problems. Examples of the original 25 problems are shown in Appendix B. The problem numbers that were used for further analysis can be found in the table in Appendix A. The five problem types of these problems are: Identifying fractions, Making representations and Reconstructing the unit from a proper, a unit and an improper fraction.
3.1.1 Subject selection
As described before, the experiment of Rau et al. (in press) used 5 experimental conditions:
Blocked (BL), Moderate (MO), Interleaved (INT), Increased (INC) and single graphical representation (SGR). For the models, a stratified random subset of the data from the pre-‐ and posttest was used: 27 subjects were randomly selected in each condition to be evaluated further. This means 27 pretests and 27 posttests in the MGR conditions, and 9 pre-‐ and posttests of each of the three SGRs in the SGR condition (circle, rectangle and number line), as they were treated in the experiment as one condition, but this way an equal contribution from each of the different GRs is ensured. However, because it is interesting to also look at the differences between the SGRs and just 9 tests is a very small number, 27 pre and posttests were selected in each SGR as well; the extra 18 tests per SGR were not used when comparing overall (MGR+SGR) effects.
3.1.2 Strategy identification
All answers on the pre-‐ and posttest were categorized. The answers given by the children were written answers to questions such as ‘where is ⅚ on the number line?’. After examining the
16
answers given by the children, correct and error categories were determined per problem. This was done on the basis of the analysis done by Rau et al. (in press), and by determining what the most frequent answers were. The categorization in this research was done by one grader, although the categories were loosely based on the previous work of Rau et al. (in press). This means no inter-‐rater reliability score can be computed, and it cannot be stated that the categorization was done truly objective. However, at a later time a second grader could still review the data and a reliability score could be computed retroactively.
All given answers were placed in a category. After the categorization, each problem had two to seven categories, of which one correct, one called ‘other’ that included all answers that did not fit into any of the other categories, and a number of error categories. Categories overlapped between different problems where possible, so that later the specialization could be examined.
Finally, the answer categories were translated back to strategies, as it is not so much the final answer that is important as the strategy that was used to get there. This makes it easier to find the common strategy used for e.g. circle and number line answers, because the answers are not the same but the reasoning that was used to get to the answer was. From now on, categories will be referred to as strategies. This notion encompasses both correct and incorrect strategies. Examples of some answer strategies are presented in the next section. An overview of all strategies can be found in Tables 6a and 6b, at the end of the next section.
3.1.2.1 Examples
This section discusses some examples of presented problems and their answers, to get a better idea of what the data look like. From every group of three problems of the same type with different GRs one will be discussed. All examples are of a different problem type. Underneath each example a small table is given with an overview of the strategies used for that problem and how often they were used in the pre-‐ and posttest. A complete overview of all strategies per problem can be found in Appendix A.
Figure 3: An example of a problem of the type ‘Identifying fractions’
The first type of problem was a relatively easy one, answered correctly by most children, of the type ‘identifying fractions’, in which a graphically displayed unit fraction had to be named (Figure 3).
The wrong answers given were mostly children that counted the sections wrong. However, this was such a small percentage that the incorrect answers were all classified in the ‘other’
strategy.
17
Problemno.
GR Problem type Strategy pretest posttest
2 Rectangle Identifying fractions correct identify 98.5 99.3
Other 1.5 0.7
Table 1: Strategy data for problem number 2
Figure 4: An example of a problem of the type ‘Making representations’
Figure 4 is an example of a problem of the type ‘making representations’, in which the child is asked to draw a fraction given a unit such as a circle, rectangle, or a number line with point “1”
marked (the fractions are 2/3, 3/4 and 5/6). In this problem two types of errors were commonly made. The first one is a size error, in which a child does not understand that all sections of a fraction should have the same size. An example of such an error is shown in Figure 5, where a child was trying to draw 5/6. The second frequently made error is to take the figure as being one section (figure as section error), and draw multiple of these sections. An example of this is shown in Figure 6, where 5/6 was being drawn as well.
Figure 5: An example of a ‘size’ error
Figure 6: An example of a ‘figure as section’ error
