Scaffolding in Classroom:
Analysing Diagnosing in teacher-student interactions from both
a quantitative and a qualitative perspective
Maria Angelakopoulou
1586475
University of Leiden
Supervisors:
dr. K. F. Stroet, University of Leiden dr. F. J. Glastra, University of Leiden
2 CONTENTS ABSTRACT 3 INTRODUCTION 4 THEORETICAL FRAMEWORK 5 RESEARCH DESIGN 14 RESULTS 18
CONCLUSION AND DISCUSSION 29
3
Abstract
The present study emanates from recent educational scientific research and focuses on the first step of the scaffolding model of contingent teaching, diagnosing the student’s actual academic level. One English teacher was selected, trained and videotaped while teaching, pre- and post-scaffolding training. The research questions to be answered were two, whether scaffolding training can be linked to an increase in teacher-student interactions with several diagnostic questions and what factors, other than the training itself, may influence teachers’ diagnostic behavior. With respect to the first research question, statistical analysis confirmed that the number of interactions with two or more diagnostic questions increased post-training. Regarding the second research question, qualitative analysis didn’t confirm that time availability may influence teacher’s diagnosing, as expected. However, two other factors, the interaction starting point and the difficulty in differentiating between diagnostic and intervention questions, have emerged as influential factors here. Conclusions, implications and limitations of these findings are discussed.
4
Introduction What is scaffolding?
Educational reforms and cut-downs are currently a reality for many countries. To cope with these changes and still keep the level of teaching and that of national test scores high, teachers, schools, and educational researchers are asked to enrich their teaching practices and undertake innovative ones. Out of such innovations in education, scaffolding emerges as a popular concept among educators and educational scientists. This theoretical framework stresses the necessity for teachers to adapt their practices to student level and needs. Although several scaffolding definitions have been given in the past, scaffolding is now broadly defined as support given by teachers after exploration of students’ actual academic level (Van de Pol, Volman, & Beishuizen, 2012).
Recent scientific research has provided educators with a scaffolding model including predefined steps. The initial step, what is known as diagnosing, is when teachers apply strategies to discover students’ actual academic level. Based on current scientific research findings, diagnosing - as part of scaffolding - is hard to implement in practice. Preliminary research on training diagnosing within scaffolding has presented ambiguous findings, positive evidence on improving the quality of diagnostic strategies but not the quantity (Van de Pol, Volman, Oort, & Beishuizen, 2014). Another study has pointed at time limitations being a possible culprit for teachers facing difficulties diagnosing (Van de Pol, Volman, Oort, & Beishuizen, 2015). Drawing on these findings, the present study aspires to examine the effects of training on a single teacher’s diagnostic behaviour and to discover other factors that may intervene in this behaviour.
Specifically, two research questions were asked:
1. Will there be an increase in the number of teacher-student interactions containing several diagnostic questions after the scaffolding training?
2. Where can the variation in the use of diagnostic questions be attributed to?
The present Master’s Thesis was written by one Master’s student of the University of Leiden. Its content firstly involves the theoretical underpinning of scaffolding, its characteristics and its teaching model. Then, previous scientific research on diagnosing in practice is introduced. Afterwards, the study’s two research question are formulated. The case study, the training programme and the research design are extensively explained and these are
5
followed by both quantitative and qualitative research results. Lastly, results are explained and linked to previous theory and practical and scientific implications are discussed.
Theoretical Framework Theoretical Background of Scaffolding
In scientific literature, scaffolding is usually accompanied by the words “support” and “adaptation” (Pino-Pasternak, Whitebread, & Tolmie, 2010). To help students reach the highest academic level of their abilities, teachers first discover what their current academic level is and, based on the information they collect, they provide fitting support. At this point, it should be stressed that scaffolding isn’t interchangeable with the word “support”. Teachers offering help based solely on their judgment do not necessarily practice scaffolding. Support provided when not needed, underrates a student’s abilities. Support omitted when actually needed, overrates one’s abilities. Support appropriate to one’s needs is desired. (Van de Pol et al., 2012).
What makes scaffolding attractive for educators and researchers is the vivid image the word brings to mind. Just like a literal scaffold, temporarily erected to help construct a building, scaffolding in education aims at providing support, is short-term and it is withdrawn when no longer needed (Van de Pol et al., 2012).
Scaffolding originates from sociocultural (Vygotsky, 1978) and constructivist theories (e.g., Duffy & Cunningham, 1996). Vygotsky’s Zone of Proximal Development (ZPD) theoretical framework, in particular, laid the groundwork for the introduction of scaffolding within education. He defined the ZPD as ‘the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined by problem solving under adult guidance’ (Vygotsky, 1978, p. 86.) ZPD spans the distance between the current level of one’s performance and the highest performance level possible, had the learner been helped by a knowledgeable other (Vygotsky, 1978). Scaffolding can be described as temporary support given within one’s ZPD, support helping one reach the maximum of their ability (Mercer & Littleton, 2007; Roehler & Cantlon, 1997; Stone, 1998a; Stone, 1998b; Van de Pol et al., 2014).
6
Van de Pol, Volman, and Beishuizen (2010) conducted a literature review of scaffolding studies. They emphasized the disambiguation around a widely-accepted definition of scaffolding in literature. Some scholars have expressed their concern about the term being applied too loosely or being falsely synonymous with support of any kind (Pea, 2004; Puntambekar & Hübscher, 2005). The reviewers concluded that the best scaffolding definition is that of Wood, Bruner, and Ross (1976) and Wood, Wood, and Middleton (1978), who had defined scaffolding as ‘a process that enables a child or a novice to solve a problem, carry out a task or achieve a goal which would be beyond his unassisted efforts’ (Wood, Bruner, & Ross, 1976, p. 90).
Key Scaffolding Characteristics
In their definition, Van de Pol and co-researchers (2010) have underlined three scaffolding characteristics. The first characteristic is contingency. Contingency refers to support that depends on and is adjusted to the learner’s current level of knowledge, competency and needs. Van de Pol, Volman, Oort and Beishuizen (2015) tried to answer why support must be contingent upon the individual by reasoning of how support relates to information processing and its link to long term memory models as well as availability of cognitive mechanisms. These factors’ relationship with contingent teaching will be explicated in the next paragraph.
Available pieces of information in the long term memory form a mental model when linked to each other. Processing new information that is relevant to the available information is added to the mental model when the learner is actively engaged. New information that is irrelevant to existing long-term memory information won’t be added to the mental model. Linking this to giving support in scaffolding, if an educator gives more support than necessary, the cognitive demand is not big enough for the student to actively process information and they make no new connections with the long-term memory, so information processing remains superficial (Wittwer & Renkl, 2008; Wittwer, Nückles, & Renkl, 2010). If an educator gives less support than necessary, the demand then is too big and the student doesn’t have the available mental schemata to create new connections; information processing will again be superficial (Wittwer et al., 2010). If, however, support matches the student’s actual level, the latter will be able to associate current knowledge and new information and to incorporate new information in the long-term memory (Webb & Mastergeorge, 2003; Wittwer et al., 2010). In
7
light of this cognitive theory, this becomes a sound and strong argument in favour of practicing scaffolding and offering contingent support.
The second scaffolding characteristic is fading of the support. When the learner shows signs of progress, help must be reduced, so they can approach independency. Fading differs from contingency because it refers to taking support away from the learner, while contingency is about adjusting support to the learner’s level.
As scaffolding fades away, responsibility must be gradually passed back to the learner; that is, the learner must gradually gain back the control over learning. This the third scaffolding characteristic, the transfer of responsibility.
The Model of Contingent Teaching
Various researchers have established a variety of strategies fostering scaffolding. Drawing upon the work of Ruiz-Primo and Furtak (2007), Van de Pol, Volman, Oort, and Beishuizen (2014) have created a model of four steps a teacher can follow when they scaffold, namely the model of contingent teaching (Figure 1).
Figure 1. The Model of Contingent Teaching.
The initial step, Step 1, is to diagnose a student’s existing academic level and strategies by posing diagnostic questions, also known as diagnostic strategies. These can be (a) asking the student questions and (b) reading students’ work, with the intention of discovering what the students do and do not know. Drawing on the student’s responses, the teacher will determine if further diagnostic questions are needed. If not, the teacher moves to Step 2.
To establish common understanding between the two parties, the educator will pose questions to check if they have understood the student correctly. In checking the diagnosis,
8
the teacher also uses questioning as in the previous step. Van de Pol and her colleagues mention in
their 2014 article that the questions in this case are meant to evoke an answer, not simply be rhetorical questions; this is a pre-requisite for a teacher utterance to be step 2. A teacher must be certain before he moves on to the third step, not simply assume that he has the right diagnosis. The second step was integrated with the first step in the work of Ruiz-Primo and Furtak (2007), but Van de Pol, Volman, and Beishuizen (2012) argued that it should be a separate step. Their argument is that the establishment of shared understanding should be explored separately from initial diagnosing, in that no successful intervention can take place if teachers have wrong impressions on their students current academic level; the only way to make a correct judgement is by having a separate step of checking the teacher’s diagnosis.
Providing support in the form of intervention strategies is Step 3, according to the model. With regard to intervention strategies, Van de pol and colleagues (2010) have integrated the two classification systems by Tharp and Gallimore (1998) and that by Wood and colleagues (1976) into a framework of intervention means. Giving feedback (1) is about giving information on the student’s progress. Giving hints (2) is leading the student towards a certain direction, using clues. Giving instruction (3) alludes to the teacher explicitly mentioning what needs to be done and how. Explaining (4) is a step further than instructing and it’s about giving detailed information. Modelling (5) pertains to demonstrating and imitating, while questioning (6) concerns posing questions to students (Tharp & Gallimore, 1998).
The last step of the model is checking the new learning and this step was proposed by the participating teachers themselves in the 2014 Van de Pol and colleagues study. For this step, the teacher asks questions to confirm that the students have integrated the new learning, such as ‘Do you understand that?’ or ‘Is that clear?’ It was argued by the researchers that this step should follow the intervention strategies (step 3) as teachers checking what the students have learned is useful feedback on the success of the scaffolding. To illustrate the use of steps in practice, an example is provided next (Example 1).
Teacher What is the question? Can I see the reply? [step 1: Diagnostic Strategy]
Student 1 Here…I said…uhhhhh…. ‘Should I buy her a present?’ And then I said, ‘Shall I bought a present for her?’ But that’s just all right…
9 Student 1 Buy
Student 2 Buy
Student 3 Shall I buy her? Yeah… But did you also say that? (looking at student 3) [step
2: Checking the diagnosis]
Student 3 In my opinion….
Teacher Shall I…buy her a present (again, looking at student 3) [step 2: Checking the
diagnosis]
Student 3 No…Oh yeah…yeah
Student 4 But ‘bought’ is still good, right?
Teacher Shall I buy... is a good sentence. [step 3: Intervention strategy] Student 4 Yes. But ‘a present for her’…instead of…’ buy her a present’?
Teacher Yes, absolutely, this is just a difference in the sentence that has nothing to do with the verb…. Yeah that is good for sure…both good, yeah.
So, which is the correct answer? [step 4: Checking student’s learning] Student 3: Shall I buy her a present.
Teacher: Yeah, that’s it.
Example 1. An example of the contingent teaching model steps.
In this example, the teacher approaches a group of student who have a question and he first attempts to diagnose by asking to read a piece of written work (line 1). One of the student states the disagreement among the group, ‘Should I buy her a present?’ or ‘Shall I bought a present for her? (line 2). The teacher wants to collect information and so he asks, ‘shall I...?’ expecting the right answer (line 3). This is step 1. Students 1 and 2 give the correct form of the verb, ‘buy’. The teacher asks two questions to student 3 to verify his good understanding of her also knowing the ‘Shall I buy’ correct sentence (lines 6, 8). This is step 2. Student 3 replies correctly but then student 4 mentions a misconception and makes a mistake (line 10). The teacher then intervenes and provides support by pointing out the correct sentence, ‘shall I buy... is a good sentence’ (line 1). This is step 3. In the end, the teacher asks for someone to repeat and establish the new learning ‘So which is the correct answer?’ as a way of concluding the interaction. This is step 4. This teacher-student interaction fragment is an excellent example of small-group contingent scaffolding. The teacher collected information on what each student knew, checked his understanding of the student he wasn’t completely sure he had understood, provided support to the student still not grasping the grammar phenomenon and asked a final question to verify the students’ new learning.
10
Previous Research on the promotion of scaffolding
Following the conceptualization of scaffolding, the theoretical framework will continue with introducing past scientific literature findings, necessary to clarify what has already been investigated and to make a connection with the present study’s research questions. Early research on the promotion of scaffolding showed that teachers’ knowledge on scaffolding can increase after learning the scaffolding theory and strategies (Bliss, Askew, & Macrae, 1996). Implementation of scaffolding for the same study, however, wasn’t successful, possibly due to teachers’ low confidence on their achievement. In another study by Seymour and Osana (2003), results were more positive for scaffolding practice after a Professional Development Programme (PDP) was used. Still, Seymour and Osana recognized that their training programme required improvement since they had noticed persistent misconceptions on teachers’ scaffolding knowledge. They suggested that a more elaborate PDP should be created, one using video fragments of teachers’ actual teaching practice, relating the videos to the scaffolding theory, and of teachers reflecting on how they can promote their scaffolding strategies.
In an exploratory study, Van de Pol and colleagues (2012) trained teachers in a PDP which they designed themselves based on older scaffolding research. They investigated whether teachers’ scaffolding knowledge and practice of the model of contingent teaching steps would increase. They found teachers knowing more about scaffolding after the training and they found them capable of applying diagnostic strategies (step 1) and intervention strategies (step 3).
The step of checking the diagnosis remained relative low (step 2) in this study. This was interpreted by the teachers’ negative or indifferent attitude towards the step. Two out of four teachers mentioned they hadn’t understood its function and they believed it was unnecessary. The two other teachers had an opposite opinion, but only one of them managed to show small increase in its use. The fourth teacher didn’t implement it whatsoever, possibly because she didn’t fully understand it, as researchers ascertained.
Van de Pol and colleagues (2012) explicitly state that their findings in increased use of diagnostic strategies were positive as, in previous research, diagnostic strategies were found to be scarce in classroom practice (Elbers, Hajer, Jonkers, Koole, & Perenger, 2008; Lockhorst,
11
Wubbels, & van Oers, 2010; Van de Pol, Volman, & Beishuizen, 2010). They claimed the use of videos as part of the training programme to having significantly contributed in the overall success of the study.
Following this, research findings on diagnostic strategies within a scaffolding context will be presented as this is the focal point of this study. Moreover, research findings on the
importance of training diagnostic strategies, on how easy to apply them, and on how effective the training is will be discussed in the next paragraphs.
Scientific Research Results on Diagnosing? Why is Diagnosing so important?
Van de Pol and colleagues (2014) conducted more studies to test the efficiency of their adapted PDP model in promoting teachers’ scaffolding practice. They reported results for each one of the four contingent teaching model steps, however, only step 1 results will be presented in this study. The focus of the study is on identifying the effects of training on teachers’ diagnostic behaviour and discovering more factors, other than the training itself, that would be related to it.
Previous literature by Puntambekar and Hübscher (2005) and Dewey (1990) shows that diagnosing is a pre-requisite for contingency, and thus for scaffolding, as the contingency element lies at the heart of scaffolding. Diagnosis is the scaffolding element mainly used in scaffolding definitions (Maybin, Mercer, & Stierer, 1992; Wood et al., 1976). Chiu (2004) has claimed that successful diagnosing before giving support is a good indicator of how effective the support will be. It can be, thus, deducted that diagnosing should always precede support if one wants to perform high-yielding scaffolding.
In the study mentioned above, Van de Pol and colleagues investigated teachers’ development in quantity and quality of the steps in a naturalistic classroom context. They assigned teachers in one PDP-trained and one non-PDP-trained groups. Concerning diagnosing, they found an increase in diagnostic strategies for the teachers participating in the PDP. This increase wasn’t significant, but it was observed that the PDP-trained teachers had used more diagnostic strategies of a high quality (questions evoking students’ elaborate demonstrations) than the control group of teachers. The researchers claimed this was an important result as diagnostic strategies have been previously found difficult to implement in classroom (Wittwer & Renkl, 2008).
12
Nevertheless, the finding of the non-significant increase in the quantity of diagnosing questions post-training is at first surprising. The researchers sought to justify this result by mentioning that it has been repeatedly found hard for teachers to apply diagnosing in classroom (Begeny, Krouse, Brown, & Mann, 2011;Wittwer & Renkl, 2008; Wittwer et al., 2010). Extending this outcome in a later study (2015), Van de Pol and colleagues associated teachers’ difficulty in diagnosing with the element of time. Diagnosing was argued to be a time-consuming process for teachers whose time inside the classroom is valuable and should be carefully managed.
The present study was based on the aforementioned scientific research body in an attempt to extend the findings and examine them from a slightly different point of view. The first research question to be answered here is whether there will be an increase in the number of teacher-student interactions carrying several diagnostic questions after training. This question differs from questions asked in previous research. What was measured in the 2014 study by Van de Pol and colleagues was the total number of diagnostic questions posed by two teacher groups (trained and untrained). The total number of questions was calculated for all the measured lessons one teacher gave. An increase was found (although insignificant) in diagnostic questions spanning a series of lessons.
Even though, this Van de Pol finding gives some useful feedback, it doesn’t tell the whole story, it says nothing about how the diagnostic questions are spread out across the teachers- student interactions. It would be possible that there are one or two interactions with many diagnostic questions and a lot of interactions with one or zero diagnostic questions. If only a few interactions raise the total number, it can’t be argued that the training is effective. But Van de Pol’s question formulation doesn’t allow to examine how the diagnostic questions are spread out, it’s an uncontrolled factor.
On the contrary, the first question posed in the present study targets specific teacher-student interactions, those including several diagnostic questions, and explores whether the number of interactions - not the number of diagnostic questions - increased after training. To explain the logic behind this research question, positive evidence on many interactions with several diagnostic questions post-training would imply that the training influences educators to ask more diagnostic questions within an interaction. More diagnostic questions within an interaction may lead to a more elaborate diagnosing. Successful diagnosing is the important first step towards successful scaffolding. Hence, positive evidence on more diagnosing amounts to positive evidence on the implementation of scaffolding. This is a strong argument in favour of the training programme.
13
This first research question is relevant to both scientific research and practice. With regards to scientific research, potential positive evidence will favour training in being incorporated in future research and more educational research on scaffolding might take place. Regarding educational practice, favourable evidence on training diagnosing skills may result in teachers’ further professionalisation by receiving extra training on scaffolding. This may eventually lead to teachers having one more innovative teaching method in their ‘toolkit’ and them feeling more confident in diagnosing their students.
The second question to be answered, how the teacher’s diagnostic behaviour can vary after training and where this variation can be attributed to, is also relevant to educational science and practice. With this question, it’s insinuated that factors other than the training itself may affect diagnosing. Discovering them would be opening a window to other perspectives for scaffolding research as other elements can emerge as important and be tested in future research. The research question is also relevant to practice. Researchers will discover new elements to incorporate in the training or elements to avoid using. In doing so, they influence teachers’ training programmes or at least they should influence them to some extent (after all, informing and improving practice is one main purpose of educational research). Better training directly affects teachers’ extended education and professionalisation.
The study designed for the purposes of this paper is a case study, with one participating male Dutch teacher of English. Firstly, the teacher’s natural teaching style was videotaped, he then received a short version of Van de Pol’s scaffolding training programme (2012, 2014) and afterwards his teaching was videotaped again after training. The research design will be described in detail in the next section.
The study seeks to answer the following questions:
1. Will the number of teacher-student interactions containing several diagnostic questions increase after training on scaffolding and its steps?
It is expected that the number of teacher-student interactions including several diagnostic questions will increase after the training (Hypothesis 1).
2. Where can the variation in the use of diagnostic questions post training be attributed to? In their 2015 study, Van de Pol and colleagues have found time limitation to be a deterring factor for teachers’ diagnosing when applying scaffolding. The same may occur in the present study as well, or it may happen that other unknown factors could emerge.
14
Research Design Procedure
This study is an exploratory case-study focusing on how teachers’ scaffolding practice can be promoted by a shorter and less intensive version of the Professional Development Programme (PDP) created and first used by Van de Pol and colleagues (2012, 2014, 2015). Firstly, the PDP (based on Van de Pol and colleagues 2012) was agreed with a fellow Master student and the mutual Thesis supervisor. It was decided that the main part of the programme would be kept and that adaptations would be made to suit the teacher.
One volunteer, a secondary education English Teacher working in a public school in Leiden, the Netherlands was the participant. During the first meeting, the theme of the scientific study was briefly discussed. A consent letter was given to parents and legal guardians. No parent expressed disagreement.
Firstly, two pre-measurement lessons were videotaped using a video-camera, during which the teacher’s own teaching style was recorded. After a week, two training sessions with the teacher followed, both of which were videotaped. During the training, the theory of scaffolding and the steps of contingent teaching were stressed. A power-point presentation with the summary of the training content was given to the teacher. After that, two project lessons followed, during which the teacher attempted to implement scaffolding in a small-group work. The second project lesson took place the exact following day after the first lesson.
Few hours after the second project lesson, a reflection session with the teacher took place. During this session, the teacher was shown video fragments of his interaction and questions were posed regarding his use of the steps of the model of contingent teaching. Two more project lessons followed after a week. After a two-month’ vacation break, two more projects lessons were recorded as well as a one-hour long interview with the teacher.
The two researchers transcribed all material in English and worked together to differentiate between teacher and student turns. All teacher utterances were regarded as teacher turns, while student turns were all student utterances. Afterwards, the four steps of the
15
model of contingent teaching were identified for each one of the teacher turns by both researchers.
Participants
The participant was one male English teacher, of Dutch nationality and of approximately 45 years of age. He is a holder of a Bachelor’s degree in English Language and Literature, awarded by the University of Groningen, and works at a secondary education school in the city of Leiden, the Netherlands. The school was described by the team leader and the teacher himself as an innovative school.
Two classes participated in this study, one pre-university education class (vwo) and one class combining pre-university education and senior general secondary education students (havo). According to the Dutch education system, secondary education in The Netherlands can have four forms. The pre-vocational education (VMBO), the senior general secondary education (HAVO), the pre-university education (VWO) and the practical training (PRO). HAVO and VWO classrooms together are usually occupied by 40% of all students on a national level every year (www.nuffic.nl). Both classes consisted of 12-14 year old students, both males and females. Video recordings of the first class took place at the end of the 2014-2015 school year, while recordings of the second class took place at the beginning of the next school year.
The participating researchers were two Master students, both studying “Educational Studies” at the University of Leiden, The Netherlands. The two researchers worked together in all practical matters of the study, such as planning the training, training the teacher, recording the lessons etc. The material was used for both Master’s Theses. It should be stressed here that the two students addressed different research questions in their Theses.
Materials
A video camera and a wireless microphone were used as recording devices for all pre- and post-training observations as well as for the interview and reflection session. The produced videos were transcribed and analyzed with the help of one windows media player programme.
The Lesson
All lessons were English language lessons. The content focused on grammar and reading comprehension. Occasionally, instructions were given about exercises, homework or
16
tests. These parts of the lesson were excluded as practical issues. The lesson language used were both English and Dutch, but all material was transcribed in English by the two researchers.
Training
Professional Development Programme (PDP).
The PDP was created by Van de Pol and her various co-researchers (Van de Pol et al., 2012, 2014). In these studies, the PDP has produced positive results in terms of teachers’ increased scaffolding knowledge and practice. The PDP used in the present study was slightly different due to time and participant limitations. It was based on the model of contingent teaching, including all four steps described in the theoretical framework section. Both researchers participated in all training sessions. The programme started with the first teacher-researchers meeting. During this session, the study was broadly discussed. Questions were posed to discover the teacher’s current knowledge on scaffolding. The discussion involved a brief explanation of the Vygotskian theory of the ZPD and the concept of scaffolding.
The main body of the training programme consisted of two training sessions which took place after the two pre-training observation lessons. During the first, the teacher was introduced to the scientific research underpinning the theory of scaffolding. Then, he was presented with the general outline of the model of contingent teaching, which was followed by an extensive explanation of the first step, diagnosing. This first meeting lasted approximately an hour.
The second training session was longer, almost two hours. During this session, the remaining three steps were discussed and several teacher-student interactions from the two pre-training lessons were used as exercises on explaining and practicing with the model. For instance, the teacher was given an example and asked to initiate scaffolding. One of the trainers would assume the role of the student. The teacher was asked to try and apply all four steps. After each example, a reflection on his practice followed. In the end, a power point presentation with the model was used to recapitulate. The session ended by giving the power point presentation to the teacher in paper and asking him to study it again before the first project lesson.
17
For the sake of analysis, only video fragments that focused on the English lesson content were selected. Any utterance of practical nature (such as discussing homework or school trips) was not taken into account. The instances between teacher-students that were chosen were called interactions. Two definitions of the concept of interaction were given. Interaction is defined as the teacher’s approaching a student or a group of students and discussing the lesson content. The interaction ends when the teacher walks away or when the discussion shifts to practical details. The second definition involves the teacher addressing a student from a distance in a whole-classroom context, while the discussion is about the lesson content. This type of interaction ends when either of the two parties ends the discussion.
All video material, pre- and post-training lessons, reflection session, training and interviews were transcribed by both researchers in English. All transcribing sessions took place in a designated Leiden University room, where privacy was assured. Some interactions of the pre-training lessons body were lost due to lack of sound in the video.
Analytical Plan
Firstly, inter-coder reliability was calculated for all the lesson transcriptions, pre- and post-training. This type of reliability determines whether two independent coders evaluate the same characteristics reaching the same assumptions. In the present study, the two researchers/ students were the two independent coders. Inter-coder reliability was calculated by percent agreement, calculated by hand as in Neuendorf (2002). Only the teacher turns were coded as one of the four steps of the model of contingent teaching. Both researchers coded the steps individually and then they compared their selections. All teacher turns that were coded as the same step by both coders were added up. The number of times the coders agreed on a step was divided by the total number of teacher turns (Neuendorf, 2002).This process was followed for each one of the steps separately, so step 1 reliability etc. and for overall agreement as well.
Statistical analyses on SPSS was used for descriptive statistics and the exploration of the first research question, namely to explore if there would be an increase in the number of interactions with several diagnostic questions after-training than before. Descriptive statistics were run to measure how many pre- and post-training interactions were in total and how many of them contained diagnostic questions or not. The hypothesis stipulated that the number of interactions containing two or more diagnostic questions would be higher for the post-training lessons than before.
18
Two variables were used here. The first was the number of interactions containing a pre-specified amount of diagnostic questions. This categorical variable had two values: interactions with zero or one diagnostic questions and interactions with two or more diagnostic questions. The second variable was another categorical variable, the measurement occasion, with two values: pre-training and post-training interactions. To examine what the relationship between these two variables is, a chi-square test for independence was used. This test calculates the significance value/score for the relationship (desired significance level was Sig. < = .05).
To address the second research question, namely to examine what the variation in post-training diagnosing can be attributed to, qualitative analysis was used. Already knowing from previous scientific research that variation in diagnosing can still occur after training (Van de Pol et al., 2012), it was intended in the present study to search interactions, compare them and make assumptions on why there could be such variation. Keeping this specific research question in mind, there was a thorough examination of post-training video fragments and respective interactions. For every interaction, the number of diagnostic questions posed was considered. Two video fragments showing divergence in diagnosing were selected, one during which three diagnostic questions were asked and one during which none was asked.
A comparison of these two fragments’ diagnostic phase took place in terms of quality/context. These two particular fragments were selected because they were very different in terms of the teacher’s diagnostic actions. On one hand he posed several diagnostic questions and on the other hand none. Both excerpts took place after-training and the first excerpt chronologically preceded the second. Comparing these two different interactions was thought to reveal details on what the teacher’s logic is behind this diverse use of diagnosing. Also, unpredicted elements were expected to emerge as influencing the teacher’s judgment on how he should scaffold (or not). What’s more, a relevant fragment of the interview was selected and linked to the two interactions’ findings. Conclusions were drawn upon connecting the data with previous research findings on diagnosing in scaffolding.
The illustration of the findings takes place primarily through the observations of teacher-student interactions in videos. Firstly, the two interactions are presented. Then, the two excerpts are extensively described and, lastly, the observations are associated with the hypothesis. The teacher’s and students’ names have been changed for reasons of participant privacy and confidentiality.
19
Reliability
Reliability was satisfactory both for the overall agreement and for each one of the four contingent teaching model steps. Regarding the first, overall percent agreement – for all teacher turns coded - was 81.5%. With respect to the four steps, percent agreement on step 1 was 84%, agreement on step 2 was 100%, agreement on step 3 was 82% and agreement on step 4 was 79%.
Descriptive Statistics
All teacher turns were 153, out of which, 9 were pre-training turns and 144 post-training ones (Table 1).
Table 1
Teacher Turns Categorized by Measurement Occasion and Diagnostic Quality No Diagnostic Phase Diagnostic Phase Total Condition: Pre-training 7 2 9 Post-training 112 32 144
Total number Total Number 119 34 153
In this table, turns are categorized based on two variables, either they are turns of pre- or post-training interactions or they are diagnostic questions or not. In the pre-training condition, two turns were diagnostic strategies, while seven were not. In the post-training condition, 32 turns were diagnostic questions, while 112 were not. An important finding here is that the post-training teacher turns are many more than the pre-training ones. This is a result of the fact that post-training lessons recorded were more than the pre-training ones and because the teacher had longer interactions with his students post-training, as he was advised to do.
There is another finding worth stressing here. If the number of non-diagnostic questions is divided by the number of diagnostic questions, both for pre- and post-training, so 7/2 and 112/32 respectively, the produced result is for both conditions is 3.5. This means that
20
the teacher didn’t ask more diagnostic questions in total post-training. There are more because the interactions were more in number and they were longer. Nevertheless, this isn’t the question the first research question aimed at answering. The question here is whether there will be a raise in the number of interactions including several diagnostic questions after the training, so there is an interest in specific interactions not the total number of diagnostic questions.
Research Question 1: Quantitative increase in interactions with two or more diagnostic strategies.
A chi-square test for independence was performed to compare two categorical variables, namely interactions with a pre-defined number of diagnostic questions and measurement occasion. The first variable included two values, interactions with zero or one diagnostic questions and interactions with two or more diagnostic questions. Pre-training and post-training were the values of the second variable. As can be seen on Table 2 below, interactions with two or more diagnostic questions did increase in number after the training, while the number of interactions with zero or one diagnostic questions remained the same after training.
Table 2
Chi-Square Test for Independence
Number of Interactions
< = 1 2,00 + Total
Condition: Pre-training 4 0 4
Post-training 4 10 14
Total number Total 8 10 18
The significance value and effect size for this test is depicted in Tables 3 and 4 respectively.
Table 3
21
Value df Asymptotic Significance
Pearson Chi-Square 6.4 1 .05
Notes. * p < .05.
The association between the variables reached statistical significance, χ2 (1, n = 18) = 3.8, p = .05, phi = .60, results found in Table 3 above and Table 4 below. The corrected test value, 3.8 is significant. The calculated effect size, represented by the Phi value (Table 4), is large according to Cohen’s (1988) criteria of .10 for small effect, .30 for medium effect, and .50 and bigger for large effect.
Table 4
Phi Coefficient Indicator of Effect Size Value
Phi .60
N of Valid Cases 18
Research Question 2: Factors affecting teacher’s diagnosing behaviour post-training.
For this research question, qualitative analysis was used. Two post-training interaction fragments are presented, described and compared. The two fragments show important differences in how the teacher reacts in terms of diagnosing. By exploring this variation in diagnostic behaviour, insightful assumptions were made as to what influences the teacher with respect to diagnostic strategies. For the purposes of this study, all text material was written in English. In addition, all dialogues were written in English, even though, in reality, teacher and students spoke in both Dutch and English with each other.
In Excerpt 1, the English teacher asked students to work on a reading comprehension exercise. An English text about a disabled female pilot is given to students accompanied by
22
reading comprehension questions in Dutch. The part of the text that relates to the interaction fragment here was the following:
A disabled woman, Debbie Grice (31) from Lockington, has been presented with her ‘wings’ after she completed a course in learning to fly. Debbie who suffers from a chronic illness which affects her muscles, has now got a full pilot’s license and can fly across the UK.
The multiple choice question intended for the students was (translated in English from the original text in Dutch):
What is it meant here with the sentence ‘Debbie Grice has been presented with her ‘wings’ (paragraph 1)?
Debbie received
A an offered flying training
B an airplane adapted to her [disability] C an awarded flying diploma
The teacher asked students to be divided in groups and work together by comparing their answers. The students were instructed to ask the teacher for help, should they had any questions. At some point, the teacher saw a female student making a gesture calling him and he approached the student desk, where one more female students sat. The participating student and the teacher are addressed as Jane and Mr. Collins in Excerpt 1 below.
Excerpt 1
Mr. Collins: Tell me.
Jane: It’s said here that she was presented with an airplane… Mr. Collins: Yes.
Jane: But then it’s said that she has learned how to fly…after she has learned how to fly, but is it then… ‘an offered flying training’, ‘an awarded flying diploma’ or ‘an airplane adapted to her [disability]’?
Mr. Collins: What do you think? Which one do you think fits the question best? Jane: I think…the ‘flying diploma’?
23
Jane: Well, it’s said here that she after…that she has learned to fly.
Mr. Collins: Yes, I think you’re right. Let’s say, I will read this one more time (reads aloud from the text)…
‘A disabled woman, Debbie Grice from Lockington, has been presented with her ‘wings’ after she completed a course in learning to fly’.
What does this mean? .... ‘After she completed a course in learning to fly’? Jane: After she learned to fly…
Mr. Collins: And… so, after she did a course… And what do you get then often if you have done a course? You do another course… Let’s say, a ‘baking cookies course’ or something…What do you usually get? Some kind of…
(Interaction continues)
In this Excerpt, the teacher was confronted with the student’s doubts about which of the three choices corresponds better to the text. Jane read a part from the text, ‘It’s said here that she was presented with an airplane’ in an attempt to show that she had already found the part of the text that would give her the answer. She seemed uncertain of which choice was a better translation of this part of the text in Dutch (the question is in Dutch). The teacher seized the opportunity to respond with a question himself, the first diagnostic question here, ‘What do you think? Which one do you think fits the question best?’ (The two questions here are taken as one, since they carry the same meaning and they serve the same purpose).
Jane responded with what she thought was best, ‘the flying diploma’ (still with some doubt indicated by the accompanying verb ‘I think’). Mr. Collins’s response is another diagnostic question, ‘Why do you think that? Where do you see that in the text?’ Jane pointed at text and translated in Dutch, ‘that she learned to fly’. Mr. Collins confirmed Jane’s answer by telling her that she was probably right and he then prompted her to read along while he read the text aloud. After reading the part he had chosen, he posed a third diagnostic question saying, ‘What does this mean? ‘After she completed a course in learning to fly’. Jane replied with the correct translation in Dutch.
What it’s observed in this excerpt is that Jane had already figured out the answer to some extend but remained skeptical as to whether it was the right choice. She initiated the interaction by calling the teacher and communicated her thoughts to him. The teacher diagnosed by posing three diagnostic questions, to all of which Jane gave the correct answer. After these questions, the teacher started giving support to the student by using intervention
24
strategies; giving an example about the ‘baking cookies course’ and hinting about diploma are his way of helping the student finding the word ‘diploma’ in the text.
Next, follows Excerpt 2, a part of a different interaction, one that took place exactly six days after the first interaction. As mentioned before, this fragment was selected for comparison with the previous one because the teacher reacted completely different in terms of diagnosing. A true contrast of diagnosing post-training can be observed by their comparison. This comparison can give useful information on the effectiveness of the training and as to why the teacher diagnoses differently. The student talking here, Peter, was sitting next to another male student and they were working together. The teacher approached the two boys and checked their work.
Excerpt 2
Mr. Collins: So… you have the answer there. I see a... read this well...you say... ‘you went to go to the swimming pool’.
You just want to use a past tense here, right? Peter: But I say… ‘do you went’…
Mr. Collins: Which part of the sentence should go? Peter: Go…
Mr. Collins: And then… What’s written there then? Peter: Went…
Mr. Collins: If you only remove ‘go’, then it says, ‘do you went to… to the swimming pool’
Peter: Do you….
Mr. Collins: ‘You went to the swimming pool’. Ok, that’s clear… then we take a look at the question… and then you say, ‘do you go to the swimming pool’, is that correct? Peter: But it’s, ‘do you went to…’
25
Mr. Collins: Let’s go back… do you went... you realise, in any case that a part of the question has to be in past tense?
Other student: If it is past tense, it is still ‘did’ right?
Mr. Collins: Yes, well done. And then? And what do we do then with ‘go’? (addressing Peter)
Peter: Went…
Mr. Collins: And why do you think that? Peter: Because it is the past tense.
Mr. Collins: Yes, that is a regular sentence. But if it’s a question? [Peter stays quite]
Mr. Collins: I will do this with another word… Uhhh… ‘I liked’… ‘Did you…’? Peter: Like? (not clear if he said ‘like’ or ‘liked’)
Mr. Collins: ‘Did you liked’ or ‘did you like it’? Peter: Like?
Mr. Collins: Is there a ‘d’ at the end or not? Peter: I think so… because it’s the past tense.
Mr. Collins: What do you want… uuuh.. Think a bit about the Present Simple rule… How does that go? How often can you use the final –s?
Peter: Like…
Mr. Collins: So, it’s actually the same, you can sort of apply this rule in the past tense, but then there’s no final –s, but then you add a final –d because it’s the past, do you get it?
Peter: Yeah…
Mr. Collins: So, we’ll do this one more time…So, for example…. uhhh… ‘They drove to the hospital’… and then you want to say…how do you say that then?
Peter: They…
Mr. Collins: Be careful! Peter: They…drove…
26
(The teacher turned his back to the students to write it on the board, in the meantime, the other student whispered the right answer to Peter) Peter: Drive?
Mr. Collins: Yes, that’s what I was trying to say… ‘Did they drive’, yeah? (interaction continues)
In this excerpt, the teacher approached the student and read his work from the book. He found a grammatical mistake and started saying, ‘I see a…’ Before he finished the sentence, he prompted the student to re-read what he had written, ‘you went to go to the swimming pool’. The mistake the teacher had spotted is the double use of the verb ‘to go’, once in the Past Simple tense and once as a full infinitive. He asked the question, ‘you just want to use a past tense here, right?’ as a way to confirm aloud his realisation. This is not a diagnostic question, its nature is rather rhetorical. Mr. Collins didn’t await for Peter’s response and, even though, Peter said in a somewhat low voice, ‘but I say… do you went’, Mr. Collins asked his next question, ‘Which part of the sentence should go?’ as a way of helping his student correct his mistake, of giving support. Peter gave the correct answer, so the teacher asked him next to re-formulate the correct sentence, again using a prompting question. It should be stressed here that these last two questions are not diagnostic questions but intervention questions. To distinguish between the two, the first is used to gain information on the student’s actual level for informing oneself, whereas the latter is used as a way of giving support to the student.
Comparing the two interaction fragments, a link should be, firstly, created with past literature. As discussed in the theoretical framework section, time limitation has been found an influential factor in teachers’ diagnostic behaviour in the past (Van de Pol et al., 2015). In this case, it can be concluded that time availability isn’t an issue for any of the two interactions. During the first interaction, the teacher invested time in hearing the student out and posed three diagnostic questions to gain information on what Jane already knew. In the second interaction, Mr. Collins spent a considerable amount of time giving intervention (as can be seen by the length of the excerpt) so it can be argued that, in both cases, the teacher wasn’t pressed for time. This factor can be excluded as a possibility of explaining the diagnosing variation between the two fragments.
With respect to other emerging factors, it is worth discussing how the two interactions start. In the first interaction, a student called the teacher and expressed her incredulity at her
27
choice of answer. Jane had tracked down the part of the text where the answer lied. She suspected the answer is ‘flying diploma’ but thought it was best to check with the teacher. Mr. Collins posed three good diagnostic questions, so he did perform the first step of scaffolding successfully. In the second fragment, the teacher approached the two students and read one’s work. He found a grammatical mistake and asked a rhetorical question to simply affirm his understanding of Peter’s knowledge. He then proceeded to intervention. In one case, there is a student question, accompanied by her reading the part of the text that she thought gave the answer. In the other case, the teacher spotted a mistake, asked a rhetorical question and then went on to intervention.
A logical deduction that can be drawn is that the starting point seems to make a difference in how the teacher reacts when he initiates scaffolding. Jane presented no mistakes to the teacher. She started reading aloud text sentences and then the multiple choice question, stating that she felt a bit unsure. The teacher must have suspected Jane already knew the answer and that she was for some reason confused. He decided to diagnose in order to elicit the correct answer from Jane, so he used his personal judgement to draw an assumption, the assumption that Jane already knew something about the question.
On the contrary, during the interaction with the boys, the teacher saw the mistake and probably assumed that Peter somehow lacked knowledge on the grammatical phenomenon. This assumption the teacher might have made is not unreasonable, he saw the mistake on the book. He might have assumed that the student lacked in knowledge around the phenomenon at hand. And this is probably why he posed a rhetorical and not a diagnostic question. Rhetorical is a question asked to express an already-formulated point rather than receive an answer. The teacher must had already drawn a conclusion, must had already offline diagnosed when he spotted that mistake. So he probably felt no need to actually perform the diagnosis and, consequently, moved into giving support. This emerges as an important result in relation to the second research question, it can be deduced that the interaction starting point, whether it is a question, a doubt or a mistake, seems to affect greatly the teacher’s diagnostic behaviour.
A second finding implicates the quality of the questions Mr. Collins asked. Questions like: ‘Which one do you think fits the question best?’ or ‘Where do you see that in the text?’ and ‘What does this mean?’, questions the teacher asked in the beginning of the first excerpt, were meant to elicit information on what the student knew, namely to be part of a diagnostic process. However, excerpt 2 questions such as: ‘Which part of the sentence should go’ or ‘…and then you say, ‘do you go to the swimming pool’, is that correct?’ and ‘Did you liked’
28
or ‘did you like it?’ were questions/hints aiming at helping the student find the correct answer, intervention questions.
Linking these last questions with what the teacher said about diagnosing during the interview after the videotaped lessons, provides an important result. When asked if he thought he had understood what diagnosing in scaffolding means and whether he had performed diagnosing successfully, the teacher said the following:
Yeah, I was thinking about diagnosing …uh….because I think that’s why you ask the questions all the time…and…. I think I know for most of the topics what are the relevant questions, and if they can’t answer those then I have to change tac[tic] and ask questions so that they find them…like, I give them an example and they have to work out what that example fits in with that question.
… But I think the diagnosis part is very important…I think I will always ask a student about ‘ok, what is it then that you don’t understand?’ Very often students just say… ‘I don’t understand’… What is it you don’t understand? ... What do you understand is another important one…so, If I ask you about this…. it’s like this…so tell me again …uh…and then, for instance, I ask them to translate, and they automatically do it… And I say ‘see you do understand it!
What can be assumed by the teacher’s words is that he seems to believe that diagnostic questions and intervention questions don’t differ. He says, ‘that’s why you ask the questions all the time’, but diagnostic questions are not questions to be asked all the time, there are questions one asks at the beginning of an interaction to discover the student’s current level. Mr. Collins also said in the interview, ‘I think I know for most of the topics what are the relevant questions’ Diagnostic questions are not pre-defined questions, they are dependent on what the student says in the beginning of the interaction because they aim at exploring each unique student’s actual knowledge and that makes them unique for every interaction.
The sentence, ‘if they can’t answer those, then I have to change tac[tic]’, also shows that the teacher had used intervention questions in the place of diagnostic questions because diagnostic questions are not a tactic one should use to help students; that’s the role of intervention questions. From this part of the interview, it becomes obvious that in his mind there is no distinction between questions one asks to elicit information for his own benefit (to diagnose the student) and questions asked to help the student learn (to give support).
29
This last finding can be linked to the lack of diagnostic questions in the second excerpt. The teacher had been trained on the principles of scaffolding and on each step of the Contingent Teaching Model and still misinterpreted the different role of diagnostic and intervention questions. It’s possible that the he believed that the intervention questions he asked were diagnostic questions. And so, instead of diagnosing, he might have unconsciously provided support already at the beginning of the interaction. Thus, the teacher here faced a clear difficulty in distinguishing among diagnostic and intervention questions. To summarise the findings for the second research questions, it’s concluded that time availability doesn’t affect the variation in teacher’s diagnosing, but the interaction starting point and the misinterpretation of diagnostic questions do.
Conclusion and Discussion
The present Master’s Thesis aspired to present new scientific conclusions on diagnosing as part of the scaffolding process, drawing on previous studies (Van de Pol et al., 2010, 2012, 2014, 2015). It aimed at exploring teachers’ diagnostic patterns, both quantitatively and qualitatively. This goal was translated into two research questions. Will the number of teacher-student interactions containing several diagnostic questions increase after training on scaffolding and its steps? Where can the variation in the use of diagnostic questions post-training be attributed to?
To examine whether the number of interactions having several diagnostic questions post-training will increase, a statistical test for the relationship between the variables ‘pre- / post-training’ and ‘number of interactions’ was used. A chi square test for independence showed that there was, indeed, a post-training increase in the number of interactions with several diagnostic questions. The hypothesis can be confirmed with confidence, as the test reached statistical significance and the corresponding effect size was found to be large. This
30
piece of evidence is partly in line with Van de Pol and colleagues’ findings. In her 2012 study – along with colleagues Volman and Beishuizen – Van de Pol had found preliminary evidence on the increase of the total number of diagnostic strategies per teacher for a small sample (finding which she interpreted with caution).
The present study’s finding is important as it diverges from Van de Pol’s finding. The latter only showed an increase in the total number of diagnostic questions. It wasn’t clarified how the diagnostic questions were spread throughout the interactions. This research question formulation doesn’t fully allow to conclude that the increase is effective in stirring teachers towards better diagnosing behaviour because it can’t be checked whether the raise was equally spread among the interactions. Nor if there was an unusually high number of diagnostic questions within one interaction.
Contrariwise, the present study’s finding permitted controlling these factor by comparing specific interactions, those containing at least two diagnostic questions. Such pre- and post-training interactions were compared and an increase was found for the latter. This result reveals it’s possible to benefit from the training, in terms of diagnosing, and learn to apply diagnostic strategies more often after training.
With respect to descriptive statistics, no increase in the use of diagnostic strategies in total was found after the training. This is in line with Van de Pol and colleagues (2014), who found a statistically significant increase in the quality of diagnostic questions but a non-statistical increase in the quantity of diagnostic questions after an eight-weeks PDP. Again, this last increase was about the total number of diagnostic questions corresponding to one teacher which differs from what was expected and was found for Hypothesis 1, an increase in the number of interactions with several diagnostic strategies.
With regard to the second research question, exploring factors affecting the teacher’s diagnostic behaviour, qualitative analysis presented interesting findings. Firstly, with regard to time limitations meddling with diagnosing, it was found that time wasn’t an obstacle for the teacher. Two interactions were chosen from all material for the purposes of the present study and, in these two, the teacher performed diagnosing very differently. By comparing these specific interactions, with the teacher successfully diagnosing in one but not in the other, assumptions could be drawn on whether time was an obstacle for diagnosing. The teacher spent a considerable amount of time with both his students so it was concluded that the variation in diagnosing can’t be explained by time pressure.
This result isn’t in line with what Van de Pol and colleagues found in their 2015 study. Time pressure was then shown to deter teachers from doing diagnosing. A possible
31
explanation for the difference between the two studies could be this study’s participant and his organisational skills. The teacher might be managing time pressure well and/or being used to managing organisational matters effectively, possibly in contrast with the other study’s participants.
Another possibility here is this study’s participating school and its educational approach. The teachers’ team leader had described the school as innovative, which the teacher also confirmed during the first meetings. An innovative school is more open to modern educational methods. So the teacher might have already incorporated non-traditional teaching methods in his style, being influenced by the school’s progressive approach. Indeed, throughout all post-training project lessons he seemed comfortable with splitting the class in groups, letting students work individually and giving himself time to focus on a student/group of students. This reasoning is in line with what Van de Pol, Volman and Beishuizen had argued in their 2011 study, that innovative schools influence the teachers in acquiring non-traditional teaching techniques.
Since the time limitation hypothesis wasn’t supported by the results, other culprits were looked for in the two transcripts after qualitatively analyzing the transcripts. One conclusion made was that the interaction starting point could have played a crucial role in teacher’s diagnosing. The situation during which a student poses questions to the teacher while showing signs of some understanding on the material already, may give the impression of a partial understanding to the teacher and the latter may diagnose to discover what is already known and what remains to be learned. If the situation is different though, and the student makes a mistake, this starting point may send signals and hints to the teacher about the student’s current lack of knowledge and so they may feel they have already uncovered it and there is no need in further diagnosis. This unconsciously formed reasoning is a possible association the teacher may create. This finding warns teachers and educational scientists that they be aware of their prejudiced personal judgement affecting the scaffolding principles in practice.
The second speculation made here was the difficulty in distinguishing between diagnostic and intervention questions and how it may prevent teachers from posing diagnostic questions, hence the variation in the use of diagnosing after training. Even though, the role of the two was distinguished in the training the teacher was confused about it, as observed in the interview. Consequently, it can be said that the quality of the asked questions is another important factor influencing diagnosing in scaffolding. The last two findings haven’t been clearly reported by previous research so far but they do point at the conclusion several other
