Running Head: CHUNKS IN L2 FRENCH WRITING
The impact of Dynamic Usage-Based and Structure-Based instruction on the development of chunks in L2 French learners’ written production.
Laura Vandendorpe, S4072219
Word Count: 15,962
MA thesis, Department of Applied Linguistics, Faculty of Arts,
Rijksuniversiteit Groningen
Supervisor: Prof. Dr. Merel Keijzer
Second Reader: Prof. Dr. Marjolijn Verspoor
Table of Contents
Acknowledgements p.4
Abstract p.5
1. Introduction p.6
2. Literature Review p.8
2.1 A Dynamic Usage-Based Approach p.8
2.2 L1 development from a Usage-Based perspective: learning through chunks
p.9
2.3 Defining chunks p.10
2.4 Chunks in L2 learner development p.13
2.4.1 Chunks: reflective of authentic language use p.13 2.4.2 Chunks as an L2 proficiency measure p.14 2.4.3 Emergence of grammar through chunks p.16 2.4.4 Processing chunks and the implications for CAF p.17
2.5 L2 writing from a DUB perspective p.18
2.6 Structure-Based and Dynamic Usage-Based Pedagogy p.20 2.6.1 Theoretical frameworks: SB or DUB instruction? p.20 2.6.2 Empirical research: SB or DUB instruction? p.21
2.7 Statement of Purpose p.24 3. Methodology p.25 3.1 Participants p.26 3.1.1 SB instruction p.27 3.1.2 DUB instruction p.27 3.1.3 Assessment preparation p.28 3.2 Materials p.28 3.3 Procedure p.29
3.3.1 Defining and measuring chunks p.29
3.3.2 Identifying chunks p.30
3.4 Statistical Design and Analysis p.32
4. Results p.35
4.1 Overall Chunk Use p.35
4.2 Use of Different Chunk Types p.37
5.1 Influence of Instruction on Overall Chunk Use p.46 5.1.1 Higher chunk coverage in DUB learners p.46
5.1.2 Dispersion of chunk coverage p.50
5.2 Influence of Instruction on Different Types of Chunks p.52
5.2.1 Partially schematic chunks p.53
5.2.2 Fully fixed chunks p.55
5.2.2.1 Collocations p.56
5.2.2.2 Discourse chunks p.57
6. Conclusion p.59
6.1 Summary of Research Findings p.59
6.2 Limitations p.60
6.3 Future Directions p.60
References p.62
Appendices p.67
Appendix A: Written Examinations p.68
Appendix B: VBA Script p.73
Acknowledgements
Firstly, I’d like to thank Merel, my supervisor, for introducing me to Wim and the larger project, which allowed me to research something I’ve been truly passionate about. I have also appreciated your constant support and feedback throughout the process, especially given the current circumstances. Thank you to also to Wim, for letting me work with your data and be part of this exciting research. I wouldn’t have had the chance to do something I’ve loved so much without either of you! Finally, I’d like to thank both Etienne and my mum. Coping with life in ‘lockdown’ while finishing a Master’s was stressful at times, but your support made all the difference.
Abstract
Chunks are considered to be reflective of authentic and proficient language (i.e. Verspoor & Smiskova, 2012), as well as limiting second language (L2) processing costs, which can ultimately increase fluency (Forsberg, 2010). Understanding how instructional approaches enable the successful development of chunks is thus of utmost importance, in order to determine how L2 pedagogy can successfully prepare learners for authentic, proficient and fluent language production. The present study compared the production of chunks in learners’ written French after six years of instruction in two different programmes: an explicit
Structure-Based (SB) programme, and an implicit Dynamic-Usage Based (DUB) programme. In order to obtain a fully reflective comparison of how each type of instruction promotes chunk development, the overall chunk coverage, alongside the use of different types of
chunks, were compared. The results indicated that, after six years of instruction, DUB learners used more chunks overall. However, subsequent analysis revealed that, in most chunk
categories, there were remarkable similarities between the groups. In fact, the only significant differences were that DUB students used more collocations and discourse chunks than SB students. Finally, as the present research formed part of Gombert’s (forthcoming) doctoral research, the results were also considered in light of previously found complexity, accuracy and fluency (CAF) measures. This tentatively suggested that DUB students’ increased complexity and fluency scores may have resulted from their increased chunk usage.
Keywords: chunks, L2 French, Dynamic Usage Based instruction, Structure-Based
The impact of Dynamic Usage-Based and Structure-Based instruction on the development of chunks in L2 French learners’ written production.
In an increasingly globalised world, there is a need for language education which sufficiently prepares learners to become proficient and fluent second language (L2) users, with an understanding of how language is used authentically. A Dynamic Usage-Based (DUB) approach to pedagogy, in which language learning is considered to occur through frequent and repetitive exposure to language in authentic contexts (Verspoor & Nguyen, 2015), aims to do this by providing learners with enough authentic L2 input for language forms to become entrenched in their memory systems (Rousse-Malpat & Verspoor, 2018). These entrenched forms, or ‘chunks’, can be used to produce authentic language (Verspoor, Schmidt & Xu, 2012), as well as to achieve functional communication while incurring fewer processing costs (Forsberg, 2010). It is, therefore, of evident interest to consider how different L2 instructional approaches may enable chunk entrenchment, in order to prepare learners to become successful L2 users, produce authentic language, and communicate while being less burdened by processing costs.
Previous research has demonstrated that learners in high-input, Usage-Based teaching programmes use more chunks (i.e. Verspoor & Smiskova, 2012; Piggott, 2019) than learners taught on the basis of pedagogical approaches with less authentic input and instead an explicit focus on form. Furthermore, studies involving complexity, accuracy and fluency (CAF) as measures of language proficiency have demonstrated that students within Usage-Based programmes produce more fluent and complex language (Rousse-Malpat & Verspoor, 2012; Piggott, 2019; Gombert, forthcoming), than students in explicit programmes. At the same time, Myles (2012) suggested that learners may rely on chunks to produce more complex and fluent language, as chunks are stored and processed as single units. In light of this, it can be considered that analysing the use of chunks within the context of pre-existing CAF results
may reveal more about the relationship between the measures, and whether generally increased CAF measures in Usage-Based students are, indeed, a result of their increased chunk use. Moreover, chunks reflect idiomatic production, and are thus an arguably more comprehensive measure of L2 proficiency than traditionally employed CAF measures. Overall, it is clear that analysing chunks alongside CAF measures delves deeper into the complexities of L2 development, highlighting how varying instructional methodologies prepare learners to actively use the L2, and produce language that is idiomatic and authentic, as well as complex, accurate and fluent.
The present study, therefore, compares the use of chunks in L2 French written
production of Dutch secondary school students after six years of either Structure-Based (SB) or DUB instruction. It predicted that, after six years of instruction, DUB students will use more chunks overall, due to increased L2 exposure and thus increased opportunities for entrenchment. However, in order to obtain a full reflection of how each instructional approach facilitates chunk development, it is also necessary to consider how groups fare in their use of different chunk types. As this research forms part of a larger study, namely Gombert’s (forthcoming) doctoral dissertation, the results can be interpreted in the context of previous findings on the dataset. In particular, Gombert analysed a variety of proficiency measures, including CAF, morphosyntactic profiling, and holistic ratings from expert teachers.
Gombert’s (forthcoming) results have clear value in comparing the effectiveness of each type of instruction; however, as previously highlighted, analysing the use of chunks in addition can provide further understanding as to how each instructional approach enables L2 written development, particularly in terms of idiomaticity and authenticity. Moreover, comparing L2 writing in particular is of interest due to the fact that explicit teaching methods are generally considered essential in achieving writing skills (Gunnarsson, 2012). Finally, the value of the present research must be emphasised in terms of its longitudinal approach, comparing L2
written production after six years of instruction. As highlighted by Rousse-Malpat and Verspoor (2012), the effects of DUB instruction take longer to become apparent, due to the time needed for sufficient entrenchment.
Firstly, in Chapter 2, a DUB approach to language development and language teaching will be considered, with attention paid to the role of chunks in first language (L1) and L2 development. The notion and operationalisation of a ‘chunk’ will subsequently be defined. Previous research will be evaluated, including the implications of chunks on CAF measures. Next, the use of chunks within L2 writing will be explored, and the differences between SB and DUB pedagogical approaches will be outlined, drawing on results of previous empirical studies in relation to the aforementioned proficiency indicators. The present study and its results are subsequently discussed in Chapters 3, 4 and 5, analysing firstly how the overall quantitative use of chunks differs between groups, and subsequently looking in more detail at how different types of chunks are used. The results of the present study are considered within the context of Gombert’s (forthcoming) previous findings, with attention to whether chunk usage can provide a tentative explanation for the increased complexity and fluency of DUB students. Finally, the limitations and implications for further research are considered.
2. Literature Review
2.1 A Dynamic Usage-Based Approach
A DUB approach to language combines Usage-Based theory with Complex Dynamic Systems Theory (CDST), with the underlying premise that language is formed and learned through use (Rousse-Malpat & Verspoor, 2012). From Emergentist and Usage-Based perspectives, meaning is the foundation of language, since language is communicative and cannot exist without meaning (Tyler & Ortega, 2018). In this respect, language consists of meaningful chunks constructed together through use (Tyler & Ortega, 2018). Therefore,
unlike Nativist theories such as Universal Grammar, a Usage-Based perspective suggests that successful language acquisition is derived from frequent exposure to meaningful language, rather than from learning rules (Tyler & Ortega, 2018). Furthermore, the notion of frequency is particularly important: the more frequently a certain construction is heard, the more
strongly it becomes entrenched in the memory system (Verspoor & Behrens, 2011). On the other hand, CDST suggests that different linguistic subsystems (such as grammar, phonology, etc.) interact with one another dynamically (Rousse-Malpat & Verspoor, 2018), meaning that language development is non-linear and highly dependent on external factors such as
language exposure and the learner’s environment (de Bot, Lowie & Verspoor, 2007). In other words, frequency is once again a defining factor, and external influences have a substantial effect on language acquisition. In combining these theories, a DUB approach advocates that language acquisition emerges from repetitive, iterative, and authentic language exposure, preferably within meaningful contexts. This enables language to become consolidated and entrenched (Rousse-Malpat & Verspoor, 2018). Moreover, structural properties such as grammar and syntax are not the foundations of learning. Rather, language is a combination of meaningful units organised into so-called form-meaning pairs (Verspoor & Nguyen, 2015). Language thus emerges through use and exposure, and the goal of language lies in its communicative intent.
2.2 L1 Development from a Usage-Based Perspective: Learning Through Chunks
If language is socially driven and meaning-based, the motivation for language
acquisition is arguably to be part of a social group and express meaning (Verspoor & Behrens, 2011). This is notably true within first language acquisition, as children mimic caregivers’ linguistic patterns to express basic needs, intentions, and desires (Verspoor & Behrens, 2011). Moreover, research from an Emergentist perspective has demonstrated that, in early stages of L1 acquisition, children rely on a predisposed set of linguistic items (i.e. Tomasello, 2000).
These include ‘holophrases’ (fixed parts of utterances repeated from adult language) and schemas, which consist of constructions with open slots, allowing a child to insert words from their repertoire (Tomasello, 2000). Children begin communicating with these pre-constructed chunks, which are initially stored and understood as whole items, before becoming
increasingly creative in their use of schemas (Tomasello, 2002). In terms of Usage-Based linguistics, this process is considered the emergence of grammar, as children begin to understand both the utterance as a whole, as well as the roles of its parts (Tomasello, 2000).
2.3 Defining Chunks
Schemas, or ‘chunks’, are present within both L1 and L2 development, and are frequently used meaningful units that convey communicative functions. Wray (2000) states that one of the main characteristics of chunks is their representation of preferred ways of expressing certain concepts within speech communities, resulting in their extensive use and consequently ingrained associations. It is, ultimately, this frequency that drives the
entrenchment process, enabling both L1 and L2 learners to consolidate these form-meaning mappings until they are automatised (Verspoor & Nguyen, 2015). In consequence, only the most conventionalised language forms, which occur frequently within a specific community, can become entrenched over time with enough repetitive exposure. These can include
relatively short compounds and collocations, but also larger strings of idiomatic language (Verspoor & Smiskova, 2012). As Forsberg (2010) additionally highlights, many social routines contain chunks, in which a specific linguistic pattern is repeated in accordance with social conventions. If even an otherwise proficient learner breaks these conventions, it can signal them as a non-expert language user (Wray, 2000). This is also reflected in genre and discourse conventions, in which non-native language is often marked by stylistically inappropriate choices (Forsberg, 2010). Therefore, it is evident that chunks are vital for authentic language production.
These schemas which represent preferred and highly occurring linguistic forms have been labelled differently throughout previous literature, including, but not limited to,
‘chunks’, ‘formulaic sequences’, ‘conventionalised ways of saying things’, and ‘multiword expressions’ (Verspoor et al., 2012; Wray, 2000; Forsberg, 2010; Langacker, 2008; Myles, 2012). Adding to complexity, as Verspoor and Smiskova (2012) highlight, chunks can be difficult to operationalise. Nevertheless, the following aspects are repeatedly cited: frequency of occurrence; association of words (notably by native speaker norms); and comprehension and production of chunks as a whole (Wray, 2000; Myles, 2012; Forsberg, 2010; Verspoor & Smiskova, 2012). Based on these characteristics, the present study defines what we from now on term ‘chunks’ as frequently used linguistic forms that represent native speakers’ preferred way of expressing a concept, and are generally learned, processed and produced as whole linguistic items.
The operationalisation of chunks is more challenging than their definition, as Verspoor et al. (2012) highlight. Nevertheless, Verspoor et al. (2012) provide a comprehensive
overview of chunk categories, based on a combination of previously identified chunks from various studies (see Verspoor et al., 2012). As demonstrated by Table 1, Verspoor et al. (2012) distinguish between ‘partially schematic’ chunks and ‘fully fixed’ chunks. This refers to the level of productivity required by the learner, and therefore, as Verspoor and Smiskova (2012) argue in reference to this distinction, provides a highlyillustrative reflection of L2 development by accounting for the productivity differences between types of chunks. For example, partially schematic chunks require that the learner fills a ‘slot’ with a lexical item, such as ‘je suis très content que’ (‘I am very happy that’), in which ‘content’ may be replaced by another lexical item. On the other hand, fully fixed chunks are stored in the lexicon as whole items, and thus can be used without modifications. For example, collocations, such as ‘tomber malade’ (‘to become ill’; literally ‘to fall ill’), ‘prendre une décision’ (‘to make a
decision’; literally ‘to take a decision’), are evidently important for accurate and authentic language use, as awkward or incorrect collocational choices can mark otherwise natural L2 users as non-expert language users (Wray, 2000). Fully fixed chunks are relatively easily implemented once they have been entrenched, as there is no productivity required within the chunk itself.
Table 1
Categorisation of chunks, adapted from Verspoor et al. (2012, p.250)
Label Definition Examples*
Partially schematic chunks
Structures Fixed expression with slot-fillers Trois mois par an, la majorité de gens, plus de désavantages que d’avantages
Complements Verb with a complement (infinitives, gerunds, nominal sentences, or reflexives)
Il est important de…, je pense que…, avoir besoin de…, en sachant que…
Fully fixed chunks
Compounds Fixed combinations of nouns, adjectives, prepositions, or particles
Tout le monde, petits-enfants, quelquefois
Particles Verbs or nouns with prepositions or particles, including phrasal verbs
À l’avenir, continuer à, à cause de, penser à, risquer de,
Collocations Collocating nouns, adjectives, verbs, adverbs, prepositions and/or pronouns
Passer l’aspirateur, prendre soin, prendre une décision, chaque année
Fixed phrases Conventional combinations of words, often idiomatic, usually consisting of more than two words
Je suis d’accord, il y a, tourner en rond, faire la sourde oreille, couronné de succès
Discourse Any form of chunk with a discourse function
Pour finir, ainsi, de toute façon, tout d’abord, premièrement
*The examples are taken from the present study, in order to provide French examples, where English examples were provided in the original source.
Thus far, much chunk research has concerned the English language, including the operationalisation presented above. While, to the best of the author’s knowledge, this categorisation has not been applied to other languages, the typological similarities between English and French suggest that it may, indeed, be applied to French. If so, a common categorisation framework would enable French chunk development research to compare the development of productive and non-productive chunks, as well as enabling cross-linguistic comparisons. The present study, therefore, intends to use this categorisation, determining its effectiveness for French chunk development.
2.4 Chunks in L2 Learner Development
2.4.1 Chunks: reflective of authentic language use. In terms of second language
development, it has been suggested that chunks are reflective of native-like language
(Verspoor & Smiskova, 2012), and that advanced learners use more chunks (Forsberg, 2010). Hou, Loerts and Verspoor (2018) suggest that the lexicon consists of more than individual words, thus learners must understand how words are combined to produce native-like
language. From a DUB perspective, learners with more target language exposure are likely to use more chunks, due to increased opportunities for entrenchment. Indeed, Verspoor and Smiskova (2012) demonstrated that high-input learners produced more chunks than low-input learners, although this difference was not significant. They compared chunk development in L2 English writing of 22 Dutch school students. Participants were taken from a low-input group (2 hours a week of English classes with a non-native teacher) and a high-input group (a semi-immersion programme with half of content classes in English with both native and non-native teachers). The authors analysed texts produced by both groups over two years,
concluding that the high-input group produced slightly more chunks overall. Despite the lack of significance, these results nevertheless align with a DUB perspective, as students with increased target language exposure produced more chunks. Furthermore, the non-significant
results may be explained by the prominent role of English in Dutch society. As highlighted by Piggott (2019), Dutch students receive extensive out-of-class English exposure, notably through television and other media. Consequently, Verspoor and Smiskova’s (2012) results may be explained by the uncontrollable influence of out-of-class input to which learners may have been exposed.
Examining individual chunk categories, Verspoor and Smiskova’s (2012) high-input participants used significantly more conventionalised sentence starters, which the authors argued are reflective of native-like production. These are partially schematic chunks,
suggesting that frequent exposure to authentic target language can result in more authentic L2 production, and possibly more productive chunks. Finally, the authors traced the development of one learner from each instructional group, concluding that the high-input learner used a wider range of chunks, as well as sounding more authentic (Verspoor & Smiskova, 2012). However, chunk development in both groups was slow, only increasing rapidly after 18 months of instruction (Verspoor & Smiskova, 2012). These results therefore support the claim that learners with high levels of target language exposure produce more chunks, and thus more authentic language. This highlights the importance of instruction on chunk development, as this chapter will later explore in more detail. However, due to an apparent slow
development, exposure is needed over an extended period, and thus chunk development research must take this into account through longitudinal analysis.
2.4.2 Chunks as an L2 proficiency measure. Alongside reflecting authentic
language use, research has also indicated that chunks are indicative of proficiency level (Verspoor et al., 2012; Forsberg, 2010; Hou et al., 2018). For example, Verspoor et al. (2012) concluded that chunks were one of the best proficiency level discriminators in English learner writing. The authors analysed written data from 489 Dutch school students with varying proficiency levels, enrolled in a pre-university programme (either mainstream education with
2-3 hours of English a week, or an English semi-immersion track). Their study compared various objective measures from a DUB perspective, finding chunks to be extremely clear indicators of proficiency (Verspoor et al., 2012). As Table 1 previously demonstrated, Verspoor et al. (2012) distinguished between fully fixed and partially schematic chunks. The authors used this distinction to compare the development of each type of chunk, ultimately revealing that productive and non-productive chunks showed different developmental patterns. In particular, the authors noted an increased use of fixed chunks in higher levels of proficiency (Verspoor et al. 2012), which suggests that higher proficiency learners are lexically more similar to native speakers, for example in their use of compounds and collocations. In other words, as previously suggested, higher proficiency learners who
resemble native speakers use formulaic language and sequences of preferred words, which are notably present in their lexical and collocational choices.
Likewise, Hou et al. (2018) investigated chunk development in 18 Chinese advanced learners of English over a period of 18 months, concluding that more proficient learners used more chunks in their written English, especially collocations. This further suggests that higher proficiency learners use authentic combinations of lexical items, thus sounding more native-like in their language production. Employing the aforementioned classification by Verspoor et al. (2012) of fully fixed and partially schematic chunks, it was discovered that overall,
learners used more fully fixed chunks (in Hou et al. referred to as lexical) than partially schematic chunks, even at the highest levels of proficiency. Finally, Hou et al. (2018) considered a variety of ways to measure chunk usage, highlighting the reliability of chunk coverage (number of words in chunks divided by number of words in text), due to the consideration of both length and number of chunks.
In terms of French chunk development, Forsberg (2010) analysed L2 French oral production of native Swedish speakers across four different proficiency levels. These ranged
from teenage beginners after one month of French instruction, to adult ‘very advanced speakers’ who had spent at least 4.5 years in France. Each group contained 6 participants, whose production was analysed for chunk use, and subsequently compared to French natives. Forsberg (2010) considered chunk quantity, category distribution and frequency, finding quantity to be the most predictive measure of proficiency. In particular, chunk use increased with proficiency level, and there were limited differences between native speakers and ‘very advanced’ learners (Forsberg, 2010). These results, therefore, clearly align with Verspoor et al. (2012) and Hou et al. (2018) in terms of chunks being indicative of proficient language use, despite Forsberg (2010) examining another language. In addition, while Forsberg’s (2010) categorisation of chunks was limited to lexical, grammatical or discursive chunks, she did also indicate that lexical chunks involving open slots, such as ‘ça fait + TIME UNIT’ (‘since + TIME UNIT’), were the most difficult to acquire. This further highlights the difference in the development of productive and non-productive chunks, emphasising the need for chunk analysis to consider this distinction. However, as Forsberg’s (2010) categorisation did not include open-slot chunks as a separate category, Verspoor et al.’s (2012) categorisation arguably remains optimalfor comparing the development of productive and non-productive chunks.
2.4.3 Emergence of grammar through chunks. In terms of early L2 development,
Myles, Hooper and Mitchell (1998) identified the prevalence of chunks in beginner-level learners through observing the oral production of 16 British secondary school pupils in their second year of French instruction. Three types of chunks were identified in beginner French learner production due to their frequent over-extension: certain question forms, certain
negative forms and certain first-person singular forms. For example, ‘j’aime’ (I like) was used as ‘elle j’aime le shopping’ (‘she I like shopping’; likely intended meaning ‘she likes
prefabricated linguistic constructions to achieve communicative goals, before ‘unpacking’ chunks to gradually use the components individually in novel utterances. This ‘unpacking’ provides evidence for the emergence of grammar through chunks, as previously discussed in terms of Emergentist and Usage-Based theories. As a result, it can be suggested that learners use chunks for bootstrapping purposes, meaning that they are capable of internalising
grammatical rules on the basis of these entrenched chunks (Myles et al., 1998). This is further explored by Towell (2014), who suggested that sufficient exposure results in learners’
recognition of ‘surface patterns’, from which they can deduct grammatical rules. Consequently, both L1 and L2 learners can acquire grammatical rules without an overt awareness of such rules. This was, indeed, demonstrated by Perera (2001), who analysed the use of prefabricated language in four Japanese learners of English between 3 and 5 years old, enrolled in an immersion programme. In early stages of language development, novel
utterances were rarely formed without prefabricated chunks, and these chunks enabled the discovery of target language rules (Perera, 2001). This clearly aligns with Myles et al.’s (1998) argument of ‘unpacking’ chunks to learn how individual elements combine before using these elements more productively. Overall, it is thus clear that chunks play an important role in second language development, by increasing productive fluency and aiding learners to discover grammatical rules. This further illustrates the interest in understanding how L2 instruction enables learners to acquire and use chunks.
2.4.4 Processing chunks and the implications for CAF. Moreover, Myles (2012)
later argued that the results from Myles et al. (1998) demonstrated that learners produce forms that may exceed their own linguistic competences. She highlights that pre-learned sequences permit communication despite limited linguistic ability, enhancing CAF in early learner production. This indicates that formulaic expressions are memorised as a whole, without understanding their individual components. In consequence, processing, storage, and retrieval
of these whole items require less effort or linguistic knowledge (Wray, 2000). Likewise, Forsberg (2010) suggested that learners use chunks to achieve functional communication (which, as previously mentioned, is the goal of language acquisition) and reduce cognitive load. Therefore, in light of the suggestion that chunks result in increased CAF, it is clear that analysing the use of chunks in L2 development can provide important information pertaining to language development, and possibly even more so than CAF measures, which do not necessarily account for the idiomaticity and authenticity of language production. Evaluating chunks in relation to CAF may additionally explain any increased CAF measures, as the research presented thus far has suggested that learners who use more chunks are likely to produce more complex, accurate and fluent language. This, alongside the presented evidence of chunks being reflective of proficient and native-like language use (i.e. Verspoor &
Smiskova, 2012), as well as enabling implicit discovery of rules, demonstrates the necessity of considering chunks in order to obtain a comprehensive understanding of learner
development.
2.5 L2 Writing from a DUB Perspective
While the research presented thus far has demonstrated that the use of chunks is present within all areas of language, the present study focuses on written learner production. Schoonen, Snellings, Stevenson and Van Gelderen (2009) argue that writing consists of the following stages: planning, linguistic formulation, and production. As writing allows more time for reflection than oral production, more attention can be afforded to theformulation stage, in which linguistic structures and appropriate lexical choices, such as collocations, are selected (Schoonen et al., 2009). In an L2, retrieval, selection, and grammar and vocabulary processing are less automatic than in the L1, meaning that formulation is a slower process requiring more cognitive demands (Gunnarsson, 2012). In consequence, L2 writers tend to spend most of their production time on the formulation of low-level linguistic aspects, such as
grammar. Moreover, L2 French writing in particular, as Gunnarsson (2012) explains, is highly cognitively demanding due to the prevalence of silent morphology (i.e. il mange ‘he eats’ and ils mangent ‘they eat’, both pronounced /ilmɑ̃ʒ/), further increasing the need for attention to
low-level elements in L2 French writing. Combining this with learners’ limited attentional resources, especially in earlier stages of mastery, means a prioritisation of certain linguistic elements when relying on explicit knowledge (Gunnarsson, 2012). However, Forsberg (2010) suggests that French learners benefit from processing gains and reduced cognitive effort when using chunks, as chunks are thought to be processed as whole items. Therefore, learners may rely on prefabricated chunks as a means of enabling written production and achieving
communicative goals by helping to compensate for any limited L2 automation, particularly in French.
This was, indeed, demonstrated by Benevento and Storch (2011). Their study investigated how different areas of L2 French writing developed over a six-month period, using written production of 15 Australian secondary school students. They ultimately found a strong reliance on chunks, notably at the start of the six months. However, learners gradually developed an increased ability for creative chunk use with developing proficiency. In terms of the previously explored chunk types, it could be considered that this increased creativity refers to an increase in partially schematic chunks, although it must be noted that the authors did not consider this distinction. Finally, their results indicated that the increase in creative chunks in French learner writing corresponded with an increase in complexity and accuracy measures, further supporting Myles’ (2012) claim that chunks enable increased CAF in L2 production.
2.6 Structure-Based and Dynamic Usage-Based Pedagogy
2.6.1 Theoretical frameworks: SB and DUB instruction. As previously highlighted,
the type of instruction learners receive is likely to impact their chunk development, as learners acquire chunks from frequent exposure to authentic language. SB instruction, however, is explicit, rule-based and centred around a structurally oriented syllabus, in which linguistic items are generally learned separately rather than as part of meaningful combinations
(Lightbown & Spada, 2013). As a result, it can be argued that this analytic approach does not promote chunk development, as learners are encouraged to rely on explicit knowledge, bottom-up strategies, and grammatical rules. Furthermore, SB instruction is carried out primarily via the L1, and any target language exposure is modified to provide opportunities for grammatical practice (Lightbown & Spada, 2013). This further reduces opportunities for chunk development due to reduced authentic target language exposure, and therefore leads to less entrenchment. Wray (2002) additionally suggests that such lack of exposure results in chunks failing to become fully entrenched into the memory system, and learners thus
potentially attempting to complete the chunk using their own linguistic resources, leading to errors. Finally, the prioritisation of accuracy in SB instruction does not create the necessary conditions for learners to take risks in language production (Lightbown & Spada, 2013). In terms of L2 written production, therefore, SB students are likely to use fewer chunks overall, possibly using fully fixed chunks rather than partially schematic chunks due to less creative language use. Overall, it is apparent that SB instruction does not necessarily provide the opportunities required for successful chunk development.
On the other hand, DUB instruction emphasises frequent exposure to authentic and meaningful language. DUB-aligned instructional approaches are founded on the assumption that language consists of meaningful units that are combined in conventionalised ways (Rousse-Malpat & Verspoor, 2018). Commonly used language forms become entrenched
through frequent and repetitive exposure (Rousse-Malpat & Verspoor, 2018), and DUB instruction relies on implicit recognition and deduction of patterns from input (Andringa, de Glopper & Hacquebord, 2011). From this repeated exposure, learners create form-meaning mappings, in which entrenched language forms are associated with specific meanings, thus enabling language development. Moreover, Wray (2000) suggests that formulaic language is hard for learners to acquire, as classroom language is generally not reflective of authentic language use, creating fewer opportunities for entrenchment. However, the highly authentic language used in DUB pedagogy creates the optimal conditions needed for chunk
development, further supporting the argument that DUB learners will use more chunks.
An example of a DUB-aligned approach is the Accelerative Integrated Method (AIM; Maxwell, 2001), involving extensive use of the target language to maximise input. All language is meaningful and contextualised, with some scaffolding aids such as the use of gestures (Maxwell, 2001) to ensure that input remains comprehensible even for beginners, and that focus remains on meaning rather than form. AIM also promotes productive language use through encouraging the reuse and manipulation of sequences from the input (Rousse-Malpat & Verspoor, 2012). These productive activities increase in complexity, with a consistent focus on fluency and meaning (Maxwell, 2001). As discussed, this repetitive exposure is thought to enable the implicit discovery of patterns (Rousse-Malpat & Verspoor, 2018), as well as strengthen memory traces due to ample opportunities for entrenchment (Verspoor & Smiskova, 2012). It is, therefore, clear that AIM enables more opportunities for chunk entrenchment than SB approaches, due to increased exposure to authentic language.
2.6.2 Empirical research: SB or DUB instruction? SB and DUB teaching represent
examples of explicit and implicit teaching, respectively. As demonstrated, explicit instruction contains a structural and grammatical focus, whereas implicit instruction involves exposure to language and a focus on meaning (Andringa et al., 2011). Research into the effectiveness of
each style has suggested that explicit teaching is more effective than its implicit counterpart (Lightbown & Spada, 2013). As a result, the popularity of this instructional method has remained evident (Rousse-Malpat & Verspoor, 2018). However, as Rousse-Malpat and Verspoor (2018) emphasise, studies upon which these conclusions are based largely considered short-term interventions involving discrete grammar tests, enabling a bias to explicit teaching forms. As a result, more recent research has started to use free production, thus providing a more representative view of language competences.
For example, Rousse-Malpat and Verspoor (2012) compared proficiency and
grammatical accuracy of L2 French oral skills. The authors demonstrated the effectiveness of a DUB-oriented Focus on Meaning (FoM) pedagogy through the comparison of two groups of secondary school French learners: one group with two years of traditional Focus on Form (FoF) teaching, and the other with two years of the aforementioned FoM method, AIM. It was concluded that, while there was no difference in grammatical accuracy, the FoM students had higher oral proficiency scores (Rousse-Malpat & Verspoor, 2012). The authors suggest that their results do not align with previous FoF and FoM comparisons due to their use of free-response testing methods, which allows learners to display their full range of productive language competences (Rousse-Malpat & Verspoor, 2012). Furthermore, it is argued that the unique longitudinal stance of their study is another reason for the difference in results, as DUB methods may take longer for the effects to become fully apparent (Rousse-Malpat & Verspoor, 2012). In particular, while fluency effects may be more immediately recognised, the results of this study demonstrate that, given enough time, learners are also capable of bootstrapping grammatical and structural rules without a need for explicit awareness of rules, as previously discussed.
Moreover, Piggott’s (2019) study largely corresponds with Rousse-Malpat and Verspoor’s (2012) findings. She studied the difference in receptive and productive skills
between explicit and implicit FoF in Dutch learners of English during their first two years of secondary school. In terms of written production, it was concluded that both approaches were effective, but facilitated different areas of language. In particular, the explicit group had higher accuracy, but the implicit group showed higher complexity and fluency levels. Within her study, Piggott (2019) operationalised fluency as both text length and chunk ratio, further indicating that implicit L2 instruction with high amounts of input enables successful chunk development. Furthermore, Piggott (2019) tentatively attributes higher complexity scores in the implicit group to the use of memorised chunks, suggesting that this may have led to increased subordination and coordination, thus giving the impression of more complex writing. Evidently, this aligns with Myles’ (2012) suggestion that CAF measures may be influenced by chunk use, and further supports the argument that DUB students are likely to produce more chunks.
Finally, Gombert (forthcoming), considered the written production of 56 French learners, enrolled in either SB or DUB classes in a Dutch secondary school. Learners’ French written production in free response tasks demonstrated that, in general, the DUB group’s L2 French writing products were more complex and fluent than those of their SB counterparts. Furthermore, DUB students scored better in terms of accuracy, holistic ratings by expert teachers and morphosyntactic profiling, although the differences were non-significant. These results align with both Piggott (2019) and Rousse-Malpat and Verspoor (2012), suggesting that, overall, DUB learners appear to produce more complex and fluent language, as well as further highlighting the necessity for both free-response tasks and longitudinal research as a means of fully reflecting learner capabilities under DUB instruction. Nevertheless, as
discussed, increased CAF measures may be a result of increased chunk usage, which Gombert (forthcoming) did not consider. Therefore, it can be argued that, to obtain a full reflection of language proficiency, chunks must be analysed alongside traditional measures such as CAF.
2.7 Statement of Purpose
The present study, therefore, builds on Gombert’s research, analysing the use of chunks in written production of Dutch learners of L2 French after six years of either SB or DUB instruction. The literature presented thus far has demonstrated the value of chunk analysis in assessing second language development, proving that chunks are reflective of native-like language use, permitting learners to produce complex, accurate and fluent
language. It is for this reason that it can be considered of utmost interest to explore the use of chunks in learners of different instructional methods to determine how best to equip learners with the linguistic skills needed to communicate effectively, proficiently and authentically. Moreover, considering these results in the context of previous CAF findings may shed further light on the apparent phenomenon for increased complexity and fluency measures in DUB learners. Finally, as Rousse-Malpat and Verspoor (2012) explain, longitudinal research is vital when considering the impact of instruction on L2 development. Therefore, the present study evaluates chunk production after six years of instruction.
In order to meet these aims, the following research question will be considered: • To what extent do SB and DUB learners of French differ in their use of chunks after
six years of instruction?
This main research question is operationalised by means of two sub questions for analysis:
1. Is there a significant difference in the overall use of chunks by SB and DUB learners after six years of instruction?
2. How do SB and DUB learners differ in their use of fully fixed and partially schematic chunks after six years of instruction?
In light of the literature discussed thus far, it is predicted that there will be a difference between the use of chunks in SB and DUB learners. Notably, in relation to the first research
question, it is predicted that, in general, DUB learners will use more chunks than SB learners. This is due to the increased exposure to authentic target language, expectedly resulting in more entrenchment and thus more production of chunks (Verspoor & Smiskova, 2012; Piggott, 2019). Furthermore, the fact that previous analyses have demonstrated that DUB students attained increased complexity and fluency scores provides further support that they may have used more chunks due to the reduced cognitive effort required in chunk production (Forsberg, 2010), allowing attentional resources to be spent elsewhere. In response to the second research question, it is considered that DUB learners will have an increased use of both types of chunks. Firstly, they are likely to use more partially schematic chunks due to the tendency for DUB instruction to encourage manipulation of target language compared to SB learners’ concern for accuracy in L2 production (Rousse-Malpat & Verspoor, 2012;
Lightbown & Spada, 2013). Secondly, DUB learners are predicted to use more fully fixed chunks due to their increased exposure to such chunks, allowing more opportunities for entrenchment.
3. Methodology
The present study formed part of a larger study: Gombert’s (forthcoming) doctoral dissertation. That study was a longitudinal comparison of L2 French written skills after six years of either SB or DUB instruction, operationalised in terms of various proficiency measures. The present study extends on Gombert’s project in that it analyses the use of chunks, aiming to determine whether a difference exists between SB and DUB students after six years of instruction. Essays were collected from each group, and subsequently coded for chunk types. An Independent Samples t-test was conducted on the overall chunk use per group, and quantitative and qualitative analyses were conducted concerning the use of different types of chunk by each group. Such a mixed-methods design enabled a deeper
understanding of the differences between the groups, as well as allowing the results to be generalised across the wider population.
3.1 Participants
A total of 61 Dutch students were included, split into two cohorts. All participants were enrolled in the same school in Groningen, the Netherlands, and were in their final year of pre-university education (Dutch VWO track). Students had undergone six years of L2 French instruction as part of their secondary education, so from approximately 12 to 18 years old. The first cohort was the control group, containing 29 students (7 males and 22 females), who were all part of one intact class. This group finished their six years of L2 French
instruction in the spring of 2016, having been taught on the basis of a SB method. The second cohort was the experimental group, which comprised 32 students (6 male and 26 female), also part of one single class. They finished their education in the spring of 2017, after six years of DUB French instruction. As previously noted, a longitudinal stance is particularly important when considering DUB methodologies, due to the increased time required for the effects to become clear (Rousse-Malpat & Verspoor, 2012).
For the first three years of their L2 French instruction, both groups had had two 50-minute lessons per week, followed by three 50-50-minute lessons per week in the final three years, resulting in a total 450 hours of instruction. Alongside this, 170 hours of online
homework was administered to each group over the six years. Therefore, both groups had had an equal 620 hours of instruction at the time of testing in their final year, and both with the same class teacher. It is important to acknowledge, as Gombert (forthcoming) notes, that limited French exposure in the Netherlands means that students were very unlikely to have had any additional regular exposure to the language outside their respective classes.
Therefore, the two groups can be considered equal in this sense, with only the method of instruction differentiating them.
3.1.1 SB instruction. The SB group’s instruction consisted of an explicit focus on
grammar and lexis, using traditional structure-based coursebooks: Grandes Lignes (Bakker et al., 2005) in lower years, followed by Libre Service (Breek et al., 2003) in the final three years. Grammatical rules were explicitly explained through the L1, and classroom instructions were also predominantly in the L1.Figure 1 illustrates that the target language, French, was used minimally in the classroom, limited to activities specifically targeting oral skills. Receptive skills were primarily practised at home. In terms of writing instruction,
grammatical accuracy and structure were emphasised. This was initially achieved through translations and closed writing exercises targeting specific linguistic elements, later extending to the composition of formal letters.
3.1.2 DUB instruction. On the other hand, the experimental group was taught using a
program that was in line with a DUB perspective. The first three years of French were taught via AIM (Maxwell, 2001), consisting of lessons based on narrative dialogues, initially emphasising listening and oral skills (see section 2.6.1). Gestures were used to aid
comprehension, as classes were almost entirely conducted in the target language (see Figure 1). Subsequently, the final three years consisted of AIM extended (AIMe), a method devised by Gombert (forthcoming), the teacher of these classes. This method is an extension of AIM, created for older students preparing to take their final exams in the Netherlands1. Teaching via AIMe involved a continuation of the extensive exposure to authentic target language, but primarily in written form. Productive skills were developed in class, while receptive skills were predominantly developed at home – in line with what was done for the SB group. Finally, in terms of writing skills, students participated in guided writing activities related to
the aforementioned in-class narrative dialogues that are the basis of AIM instruction, and subsequently in free writing narrative and argumentative tasks.
Figure 1. Estimated percentage of in-class languages spoken. Estimations were
provided by the class teacher.
3.1.3 Assessment preparation. Within the aforementioned 450 hours of instruction,
both groups received 30 hours of preparation for their final written exams (the data used for the present study). This included receptive and productive activities on topics including abortion, immigration, living for a thousand years, tattoos and stay-at-home fathers. Both groups were presented with audio-visual input on these topics during class. Follow-up activities included constructing arguments in the target language and guided writing tasks, in which students were prompted with ideas and linguistic forms.
3.2 Materials
The data for the present study consist of students’ final written examinations. These were free response written tests, in which students had to respond to prompts based on one of the previously studied topics. As highlighted by Rousse-Malpat and Verspoor (2012), free response tests provide learners with the opportunity to demonstrate their full array of
linguistic competences, resulting in a more reflective account of students’ language skills than
0% 20% 40% 60% 80% 100%
Control (SB) Experimental (DUB)
Es timated p erce n ta ge o f cla ss time Instructional Group
Estimated distribution of in-class language
L2 (French) L1 (Dutch)
writing exercises that only elicit certain constructions, such as discrete grammar tests. Students completed these exams in both December and April of their respective years. However, for scope reasons, only the April exams were used for each instructional group. This later exam was selected as it was considered to be more reflective of students’ linguistic capabilities, due to the higher contribution of this exam to their final grade, thus yielding more effort from students. Moreover, these exams took place later in the year right before their final examination and were therefore deemed to be a good reflection of all the students had learned during their French classes. Consequently, the present study considered 56 essays (24 that were written by SB students, 4 male and 20 female; 32 by DUB students, 6 male and 26 female). In other words, not all students completed this April exam due to absences on the day. During the exam, students were presented with two of the topics, and self-selected one for their essay in which they were required to write a minimum of 200 words. As the SB group and the DUB group came from different school year cohorts, they did not all receive the same exam. Rather, SB students had the choice between writing on the subject of ‘living for 1000 years’ and ‘migrants crossing the Mediterranean’. The following year, DUB students had the choice between ‘legalising abortion in Chili’ and ‘stay at home fathers’ (see Appendix Afor the materials used in their respective exams). Essentially, each student had the choice of two topics, and eventually wrote one essay on their chosen topic.
3.3 Procedure
3.3.1 Defining and measuring chunks. The present study operationalised chunks as
presented in Table 1. As argued in section 2.3, Verspoor et al. (2012) provide one of the most comprehensive operationalisations of chunks, thus the decision was made to use this in the present study although it had not yet been considered for a language other than English. Therefore, in order to ensure its appropriateness for intermediate-level French, a random sample of five SB and five DUB essays were selected and coded by the researcher. Piloting
the methodology revealed that the operationalisation was also effective for use in French. However, many students used conventional opinion-giving chunks, such as ‘je pense que’, ‘je trouve que’ and ‘j’espère que’ (‘I think that’, ‘I find that’ and ‘I hope that’, respectively).
Therefore, to render the categorisation suitable for the present study, the original definition of a complement (“verbs with infinitives, gerunds, nominal sentences or reflexives as
complement”; Verspoor et al., 2012, p.250) was expanded to include these opinion-giving sentence starters with relative pronoun phrases as a complement, and this was consistently applied to all the L2 written French learner data.
3.3.2 Identifying chunks. Written examinations were administered in a computer
laboratory at the learners’ school. Students had 45 minutes to complete their essays digitally, with the option of using supportive tools (i.e. Microsoft Word spellcheck), but no internet access. Essays were collected in the form of Microsoft Word files, and subsequently
‘cleansed’, involving removal of anything personally relating to the author (such as names), as well as any messages intended for the teacher. Subsequently, each essay was coded for
chunks by the author of the present study. This was done using a Visual Basic for
Applications (VBA) script (see Appendix B), previously created by van der Ploeg (2017) for Piggott’s (2019) research. In order to ensure the script’s suitability for the present study, several slight modifications were made. Firstly, a function was added to ensure that chunks that appeared across multiple essays were recorded in the Excel file multiple times (for example, each time ‘je pense que’ was identified in an essay, this new identification was recorded in the script). This was necessary for a subsequent calculation of chunk coverage per text, as we needed to know not just how many times a chunk appeared, but specifically in which texts the chunk appeared. Next, another function was added to ensure that these repeated chunks were only searched for once in ensuing essays, to avoid false measures.
Each essay thus took the form of a Microsoft Word Document and the VBA script was a .dotm file, allowing macros to be used within the Word document to permit communication between the Word and Excel files. In other words, any macro command executed in Word resulted in the appropriate information being stored in the Excel file. The following outlined procedure was conducted using the modified VBA script. Firstly, treating each essay
individually, the script was imported to the Word file via the .dotm template, and a macro shortcut was used to manually add each chunk to the corresponding Excel file. Chunks were identified based on the author’s assessment, and a native French speaker was present
throughout the coding for a second native opinion. Regarding embedded chunks, the longest chunk was counted to ensure reliable results and avoid repetition in counting.
After identifying a chunk and executing the appropriate macro commands, the chunk was stored in an Excel file alongside the following information: classification (structures, complements, particles, etc.); the text in which the chunk was found; and whether its form was correct. Form was considered incorrect when the chunk contained spelling, gender, agreement or conjugation errors. For example, ‘toute le monde’ (correct form: ‘tout le monde’) and ‘d’autre choses’ (correct form: ‘d’autres choses’) were accepted, as these were minor agreement errors. These examples also highlight Gunnarsson’s (2012) aforementioned argument that French contains much silent morphology, as these forms have little or no discrepancies between their pronunciation, especially to a non-native French user. For these reasons, such chunks with an incorrect form were accepted. However, expressions such as ‘à l’un côté’ (correct form: ‘d’un côté’) were not counted as chunks, as the errors pertain to an
incorrect prepositional choice, indicating that the linguistic sequence had not been entrenched, and was thus not stored as a chunk. Likewise, linguistic sequences resembling chunks but with word-order errors were not considered, as it can be assumed that the learner had not successfully learned this chunk as a whole, and it arguably cannot be considered a chunk.
Within the Word document itself, the chunk underwent formatting changes: italics and red font. This was done automatically by the VBA script, and allowed the researcher to visually identify which chunks had already been detected by the VBA script and added to the Excel file. Alongside identifying and recording chunks, the total number of words was also
computed and inserted into the Excel file, in order to calculate a chunk coverage per text (to be discussed below). This was done by manually highlighting the text and executing another macro command.
The process outlined above was then repeated for each essay. However, after the first essay, each new document was firstly scanned for previously identified chunks, using another macro command. This made the process of identification easier and more reliable. Any duplicates found in subsequent essays were automatically changed to italics and a green font by the VBA script, indicating that the chunk had been pre-identified. Any identified chunks were also automatically added to the Excel file. After this step, the essay was subsequently manually coded for any new chunks by the researcher, which were added to the Excel file, as explained above. Using Excel meant that it was easy to manually modify the classification of a chunk, or even remove it entirely, if required, further ensuring a more reliable identification process. This could be done in the Excel file at any stage of the analysis. Finally, once all 56 texts had been coded, each text was inspected a second time to ensure that no chunks had been missed, and thus further ensuring reliability in the identification of chunks.
3.4 Statistical Design and Analysis
In order to answer the first research question, whether the overall use of chunks differed between instructional groups, the independent variable under consideration was type of instruction. This was a nominal variable with two levels: SB and DUB. The dependent variable was chunk coverage (all types combined). This figure reflected the percentage of the text containing chunks, and was calculated by the total number of words in chunks divided by
the text length at large (following Hou et al., 2018). As Hou et al. (2018) explain, analysing chunk coverage accounts for both length and frequency, as higher proficiency learners may use longer, but consequently fewer, chunks. Furthermore, calculating a chunk coverage per text also accounts for differences in text length, as the texts of the present study ranged from 200 to 733 words. Chunk coverage was calculated by using Excel to determine the number of words per chunk. From this, it was possible to calculate the total number of words in chunks per text, which was subsequently divided by the text length. Multiplying the chunk coverage by 100 then generated this number as a percentage.
Statistical analyses were then conducted. Firstly, data were inserted in R (version 3.6.1) using the “readxl” package (Wickham & Bryan, 2019). Descriptive statistics were calculated using “psych” (Revelle, 2018) and data were plotted in a boxplot to provide an initial overview. Assumptions of normal distribution and homogeneity of variance were checked in R, using packages “pastecs” (Grosjean & Ibanez, 2018) and “car” (Fox & Weisberg, 2019), respectively. Data were found to show both a normal distribution and homogeneity of variance; therefore, an Independent Samples t-test was conducted, where p<.05. The effect size was subsequently calculated. In the hopes of learning more concerning
chunk use by each group, a correct/incorrect form ratio was calculated per group, determining the percentage of chunks containing spelling, gender or conjugation errors. This was done by firstly calculating the number of chunks used overall by each group, and subsequently
calculating the number of chunks with a correct or incorrect form in each group. Using these figures, it was possible to calculate a ratio of correct/incorrect chunks per group. However, statistical analyses were ultimately not conducted, due to the extreme similarities between the groups, indicating no pertinent findings (to be discussed in section 4.1).
For the second research question concerning each group’s use of partially schematic and fully fixed chunks, the difference between the types of chunks used by each group was
analysed. Using Excel, a chunk coverage was calculated per type of chunk (structures, complements, compounds, particles, collocations, fixed phrases, and discourse) for each essay. This was done by calculating the sum of words in each type of chunk, divided by the text length, subsequently multiplied by 100. On the basis of these figures, a chunk coverage was additionally calculated for partially schematic and fully fixed chunks in each text (see section 2.3 for more detail). Next, statistical analyses were conducted in R. Assumptions were checked using the packages “car” (Fox & Weisberg, 2019) and “rstatix” (Kassambara, 2020). All assumptions were met, although two univariate outliers were found in the DUB group’s fully fixed chunks results. These outliers were not considered to be extreme and, moreover, one was above and one was below the mean. Therefore, the decision was made to leave these data points in. Descriptive statistics were performed, and a boxplot was created using
“ggplot2” (Wickham, 2016). Subsequently, a Multivariate Analysis of Variance (MANOVA) was performed, p<.05.
Next, a boxplot (Wickham, 2016) was created using the chunk coverage for each of the seven types of chunks, providing further information as to how each instructional group differed in their use and distribution of each type. In other words, not only was it investigated whether one group used more chunks compared to the other, we also wanted to see if the type of chunk differed. Based on this graph, subsequent statistical analyses were conducted for chunk types indicating a difference between the groups. Assumptions were checked (Grosjean & Ibanez, 2018; Fox & Weisberg, 2019), and Independent Samples t-tests were conducted, p<.05. Effect sizes were also calculated, using “rstatix” (Kassambara, 2020) and “coin”
(Hothorn et al., 2008) for cases where non-parametric t-tests were required. Finally, for chunk types in which there was a difference between the groups, the Excel file was consulted to consider qualitatively how chunk usage differed between each group.
4. Results
The current chapter outlines the results of the present study. Firstly, overall chunk use is investigated, pertaining to research question one. Secondly, the use of different types of chunks in each group is analysed, in answer to research question two.
4.1 Overall Chunk Use
Table 2 presents the overall chunk coverage per instructional group, calculated by the number of words in chunks divided by the total number of words in the entire text,
subsequently multiplied by 100 (see section 3.4.1).
Table 2
Overall chunk coverage by instructional group
SB (N=24) DUB (N=32) Mean 28.22 32.83 SD 4.99 6.2 Median 28.25 33.34 Minimum 17.82 17.53 Maximum 42.04 43.51
In response to the first research question, Table 2 demonstrates that, on average, students with six years of DUB instruction had a higher chunk coverage (M= 32.83, SD=6.2) in their output than students with six years of SB instruction (M= 28.22, SD=4.99).
Nevertheless, the minimum and maximum coverages suggest that both groups had similar overall distributions. Figure 2 visualises this average higher chunk coverage in DUB students, revealing more concerning the dispersion of chunk coverage in both groups.
Figure 2. Overall chunk coverage by instructional group.
As can be seen, the DUB students, while having a higher chunk coverage on average, were also less homogeneous as a group. Figure 2 also demonstrates that SB participants showed limited dispersion in their upper quartile, although one outlier had a relatively high chunk coverage. After checking assumptions, data were confirmed to be normally distributed and to show homogeneity of variance. Therefore, an Independent Samples t-test was
conducted. This revealed that, on average, the higher chunk coverage attested for the DUB students (M= 32.83, SD=6.2) compared to their SB counterparts (M= 28.22, SD=4.99) was significant, t(53.7)-3.08, p=0.003 , 95% CI [-7.60, -1.60], and of a medium effect size, r²=0.14.
An error ratio was also calculated per group, revealing that the SB group had a correct/incorrect form ratio of 92%/8%, whereas the DUB group had a correct/incorrect form ratio of 91%/9%. Due to the lack of incorrect chunks in both groups, no further analyses were conducted. However, only chunks with spelling, gender or conjugation errors were accepted
but marked having an incorrect form. Therefore, these results do not reflect the use of linguistic sequences that resemble chunks, but that contain word order, word choice or other more severe errors (see section 3.2.2 for examples).
4.2 Use of Different Chunk Types
Table 3 below presents the coverage of partially schematic and fully fixed chunks by each group. As a reminder, partially schematic chunks are those in which there is an ‘open slot’, whereas fully fixed chunks can be used without modifications, and thus require less productivity (see section 2.3).
Table 3
Partially schematic and fully fixed chunk coverage by instructional group
SB (N=24) DUB (N=32) Partially Schematic Chunks Mean 10.57 10.91 SD 2.92 4.19 Median 10.88 9.57 Minimum 3.96 4.38 Maximum 16.8 18.75 Fully Fixed Chunks Mean 17.65 21.92 SD 4.77 4.9 Median 17.63 21.58 Minimum 5.94 10.43 Maximum 28.72 34.34
Table 3 demonstrates that, in terms of the second research question, DUB students, on average, had a slightly higher mean partially schematic chunk coverage (M=10.91, SD=4.19) than SB students (M=10.57, SD=2.92), but at first glance this difference seems minimal. Once
more, however, DUB students were less homogeneous as a group. Furthermore, comparison of the medians demonstrates that the SB students, in fact, had a higher median (Mdn=10.88) than DUB students (Mdn=9.57). This further highlights the similarity between the groups, but also indicates that DUB students showed more variation and generally higher partially
schematic chunk coverages. On the other hand, in terms of fully fixed chunks, there were clearer differences between the groups. Once more, DUB students had, on average, a higher fully fixed chunk coverage (M=21.92, SD=4.9) than SB students (M=17.65, SD=4.77). These results are visualised in Figure 3, in which it is apparent that the instructional groups differed more in their use of fully fixed chunks than partially schematic chunks.
Figure 3. Partially Schematic and Fully Fixed chunk coverage by instructional group.
After checking assumptions, a one-way MANOVA was conducted, revealing there to be a significant effect of instruction regarding the use of chunk types, F(1,54)=5.5, p=.007. This was of a large effect size, partial η2=0.57. In particular, on average, DUB students had a
SD=2.92), but this difference was non-significant, F(1,54)=0.11, p=.74. Secondly, DUB
students had a higher fully fixed chunk coverage (M=21.92, SD=4.9) than SB students (M=17.65, SD=4.77), and this difference was significant, F(1,54)=10.65, p=.002. In order to explore these results in further depth and obtain a more comprehensive answer to the second research question, a chunk coverage per type of chunk was calculated, visualised in Figure 4. As Table 1 in Chapter 1 demonstrated, structures and complements are both types of partially schematic chunks, which require certain slots to be filled. Compounds, particles, collocations, fixed phrases and discourse chunks, on the other hand, are types of fully fixed chunks,
meaning they are stored and processed as whole items (Verspoor et al., 2012).
Figure 4. Chunk coverage by instructional group for each type of chunk.
As can be seen, there were many similarities between the groups, especially in terms of interquartile ranges. Turning firstly to the two types of partially schematic chunks
