Formulaic sequences in the L2 Predicting advanced learners, receptive versus productive knowledge and difficulties in measuring with frequency bands

(1)

Formulaic sequences in the L2

Predicting advanced learners, receptive versus

productive knowledge and difficulties in measuring

with frequency bands

DAVID ORRANTIA GARCÍA S3486664

MA in Applied Linguistics

Faculty of Liberal Arts

University of Groningen

Supervisors:

Rasmus Steinkrauss

Sake Jager

14030 words

22/06/2018

(2)

(3)

native-likeness due to the advantages their use involves. However, they are commonly acquired at a slower pace than general vocabulary and present strange constructions even for very advanced learners. This study aims to test whether the use of FL distinguishes highly advanced from intermediate speakers, if receptive knowledge precedes productive knowledge similarly to general vocabulary and the extent to which the frequency range in which a formulaic sequence appears plays a role in its acquisition. The results indicate that C1 learners are better at overall and productive knowledge of FS but not at receptive knowledge, pointing to formulaicity as a predictor of language proficiency. In addition, receptive knowledge was confirmed as a preceptor of the productive counterpart, similar to other vocabulary items despite FS late acquisition, indicating that learners might know the meaning of a FS but not be able to actively retrieve it. Finally, significant differences were not found for frequency of appearance, contradicting previous studies but revealing the difficulties in testing FS with frequency band and an arguable dissimilarity between natives and non-natives depending on the input they receive.

(6)

1. Introduction

In recent years, formulaic language has become is an increasingly important research topic in Applied Linguistics due to its pervasiveness in L1 speakers. In the words of Ellis (2001), "speaking natively is speaking idiomatically” (p.45). Formulaic sequences (FS) compromise between a third and a half of speech (Erman, Denke, Fant, & Forsberg Lundell, 2015) and entail sociofunctional and psycholinguistic advantages (Carrol & Conklin, 2014; González Fernández & Schmitt, 2015). Nevertheless, research has proven that it is not until an advanced stage in the learning process that learners actively employ these (González Fernández & Schmitt, 2015), otherwise common expressions for native speakers (Erman et al., 2015) and significantly lag behind other general vocabulary aspects/items (Wray, 2002). As a consequence of the lack of knowledge of FS, those advantages tend to be limited, especially for beginner and intermediate learners. The reasons for the lag in acquisition include the lack of salience of formulaic sequences in speech (Boers, Lindstromberg, & Eyckmans, 2014), a lower quality and quantity of input in second language classes (Wray, 2002) and the slower pace of acquisition when compared to natives (Ellis, 2001). This leads to formulaic language only showing up in higher levels of proficiency (Erman et al., 2015; Schmitt, 2004; Edmond & Gudmestad, 2014). Therefore, it seems appropriate to test the use of formulaic sequences at an intermediate (up to B2, according to the CEFR) and advanced stage (from C1 onwards) in order to determine if the use of FS distinguishes intermediate from advanced learners in order to compare the results and contrast them with previous theory.

Two major theories have tried to explain the acquisition process for FL for native and non-native speakers of a language. The main difference between the two

(7)

approaches is based on whether chunking occurs in non-native speakers or not; a chunk is a combination of words which appears together more often than would be expected by chance (Ellis, Simpson-vlach, & Maynard, 2016, p. 380). The first theory is based on a neo-Firthian approach (Durrant & Schmitt, 2010) and advocates for a fundamental difference in language acquisition: chunking happens for natives whereas non-natives, with their more mature cognitive system, lean towards breaking chunks into their fundamental units –words– (Wray, 2002; Wray & Perkins, 2000). Hence, this caps their formulaic language production and the advantages of their use. The second theory, following a Usage Based (UB) approach, argues for chunking taking place for both natives and non-natives (Ellis, 2001). However, as high frequency items are more likely acquired by non-natives (González Fernández & Schmitt, 2015), and FS are not as frequent or salient in speech as other linguistic items (Boers et al., 2014), this leads to the lag in acquisition.

In spite of their contribution to explaining the acquisition of formulaic sequences, these approaches solely investigate the development of the productive knowledge. There is little research on receptive knowledge of formulaic sequences; this is, if learners know the meaning of a formulaic sequence (FS) when presented with its form. Laufer & Goldstein (2004) define receptive knowledge as the capacity to “comprehend the input” (p. 404) whereas the productive counterpart implies retrieval. If FS knowledge lags behind for L2 speakers, it might be because their productive knowledge is poor. However, that does not necessarily imply that the same applies for receptive knowledge. Therefore, receptive knowledge is described as a previous step towards productive knowledge; more explicitly, productive recall is the final step in a hierarchy in which receptive recall precedes it. Therefore, testing and comparing the two of them

(8)

might prove the precedence relation as well as helping in determining the distance and level of correlation between the two. In sum, the comparison between passive and active recall of formulaic sequences will give insight into whether one precedes the other or if they are independently stored and learned.

The present study will therefore attempt to find further insight on three topics surrounding formulaic in language speakers: firstly, whether the leap from the Common European Framework of Reference for Language (CEFR) B2 to C1 level significantly increases FS knowledge and, therefore, if formulaic language is a predictor of language proficiency. It is expected that C1 participants will have an overall better knowledge since previous studies link effective acquisition of FS to advanced stages and C1 complies with the criterion as the CEFR considers this stage the first advanced level (Council of Europe, 2001). The second research question will expand on whether receptive knowledge precedes receptive knowledge FL, as it does in general vocabulary development (Laufer, Elder, Hill, & Congdon, 2004; Laufer & Goldstein, 2004). In order to test this hypothetical parallel, CATSS, the test the authors employed, provides a framework splitting receptive and productive knowledge. Although it originally tested individual words, FS do comply with the premises of the test. This leads to expect a positive outcome. Lastly, the third research question will investigate the effect of frequency of occurrence by means of frequency bands. It is expected that, as González Fernández & Schmitt (2015) explain, the more frequent FS will be the best acquired.

In the first place, this text will introduce the theoretical background, consisting firstly in the review of the advantages of formulaic language and the problems surrounding acquisition for non-natives and the theories explaining both. Next, a review of the main theories on acquisition and use of formulaic language will provide a

(9)

framework for the cited issues for non-natives and expand on their cause, followed by contrasting the theories on FS with the difference between receptive and productive language regarding formulaic sequences. This will lead to the thesis statement, including the research questions and hypotheses. Next, the methods section will explain the particularities of the testing and the operationalization of the variables and constructs, leading to the results. This section will analyse the data obtained after testing, which the discussion section will link to the theory and hypotheses tested. Finally, the conclusion section will display the conclusions and limitations of the study plus suggestions for further study.

(10)

2. Background

2.1. What is formulaicity? Finding a proper definition

Formulaic language (FS) represents an omnipresent phenomenon comprising between a third and a half of speech (Erman & Warren, 2000; Foster, 2001 as quoted in Carrol & Conklin, 2014) and it is in dire need of study. The following definition represents the baseline for all research surrounding the topic:

[A formulaic sequence is a] sequence, continuous or discontinuous, of words or other elements, which is, or appears to be, prefabricated: that is, stored and retrieved whole from memory at the time of use, rather than being subject to generation or analysis by the language grammar (Wray & Perkins, 2000, p. 1).

As Boers et al. (2006) comment, the definition “acknowledges the fuzzy nature of […] ‘formulaic sequences’” (p. 246) meaning that these multiword expressions are not an easily distinguishable category following a set of rules or sharing the linguistic properties of the words composing them. In other words, they are holistically stored and retrieved (Wray & Perkins, 2000; Wray, 2002). As such, they are processed as complete chunks of information retrieved from memory (Wray & Perkins, 2000) in a similar fashion as individual words or phrases (Altenberg, 1998, as quoted in Wood, 2006). Furthermore, the definition implies that what is holistically retrieved for some speakers will not be so for others (Boers et al., 2006). In sum, Wray proposes a definition wide enough to encompass all possible chunks and open enough so that it might apply to different individuals in a different manner.

(11)

2.2. Pervasiveness of idiomaticity in the L1.

Formulaic language is present in everyday language for native speakers of a language and represents one of the defining aspects of native-speakerness (Ellis, 2001). As a consequence, its mastery is a basic aspect for fluent and idiomatic language use, thus implying sociofunctional and psycholinguistic advantages (Conklin & Schmitt, 2008; González Fernández and Schmitt, 2015). With respect to psycholinguistic advantages, the use of formulaic sequences (FS) has been proven to reduce the demands on working memory (Conklin & Schmitt, 2008), which is limited and easily overloaded with individual lexical items in on-line planning. The previously explained holistic storage and retrieval of chunks leads to quicker and easier processing (Pawley & Syder, 1983; Conklin & Schmitt, 2008). Similarly, this contributes to the enhancement of real-time communication (Boers, Eyckmans, Kappel, Stengers, & Demecheleer, 2006), a sociofunctional advantage. Moreover, speaker fluency is related to FS use since it improves “rate or speed of speech, pause times and frequencies, and the length of fluent runs of speech between pauses” (Wood, 2006, p. 14). Ultimately, the frequent use of formulaic sequences in native speech responds to a need to achieve “maximally rapid intelligibility” (Ellis, 2001, p. 47).

Non-natives also benefit from these advantages “once they have encountered formulaic sequences in the L2 with enough regularity” (Carrol & Conklin, 2014, p. 786). Several studies conducted on processing for native and advanced non-native speakers reflect that both groups show shorter reaction times and lower error rates in FS than non formulaic sequences (Jiang and Nekrasova, 2007). Moreover, they read formulaic sequences faster than non formulaic alternatives (Conklin & Smith, 2007), thus giving proof to the stated advantages and their availability for natives and advanced

(12)

non-natives. Nonetheless, non-natives are generally slow acquiring L2 collocations (Durrant & Schmitt, 2010), without an active use until very advanced stages in language development (Erman et al., 2015; Schmitt, 2004; Edmond & Gudmestad, 2014). In addition, odd combinations are commonly present in terms of native-likeness even at this level (Boers et al., 2014), thus limiting the aforementioned advantages for L2 speakers.

2.3. L2 speakers. Problems when learning formulaic sequences

There are several possible reasons for the lag in acquisition of FS for non-natives. Whereas natives are sensitive to the degree of coherence of words, non-natives notice the frequency of a whole sentence (Ellis, Simon-Vlach and Maynard, 2008). However, it appears that the main difference between natives and non natives is founded on processing and frequency. More specifically, a lack of salience of FS in speech contributes to the lag in acquisition (Boers et al., 2014) as the characteristics of FS do not make them semantically or perceptually salient in speech: firstly, an absence of semantic salience means that, although the new elements tend to be paid more close attention than familiar words, collocations are frequently made up of common words and as such do not receive such attention. Therefore, their acquisition is not enhanced by the presence of new words which the learners tends to pay more attention to. Secondly, collocations are not common in speech, meaning that there is not a perceptual salience (Bybee, 2002 as quoted in Boers et al., 2014) and non-natives share a difficulty in the processing of collocations. This leads L2 learners to overuse a particular set of formulaic sequences instead of progressively learning over time, and thus under-developing formulaic language (Wray, 2012).

(13)

Furthermore, frequent and communicatively meaningful collocations are learnt faster by non-natives (Ellis, 2001), so the acquisition of non-frequent formulaic sequences tend to be capped for L2 learners (Boers, Lindstromberg, & Eyckmans, 2014). A deficiency in frequency combined with the range of variability is identified as a factor affecting statistical learning (Boers et al., 2014); the fact that FS are not common if compared with individual words (Durrant & Schmitt, 2010) and that formulaic sequences with very similar meanings are not always realized with the exact same combinations of words hinders the acquisition process for non-native speakers. In sum, formulaic language is a challenge even for “very advanced L2 speakers” (González Fernández & Schmitt, 2015, p. 99) and tends to be limited in an L2 class as the teachers, mostly L2 speakers as well, do not provide enough formulaic sequences for take-up to happen. Thus, this negatively reinforces the aforementioned low frequency of some FL and lack of salience as the main problems for L2 learners, reducing even more the less frequent sequences and the salience for take-up.

2.4.Theories on Formulaic Language functioning and acquisition for L1 and L2

Several studies have attempted to go in depth into the similarities and differences in the use and acquisition of formulaic sequences between native and non-native speakers in order to relate them to the late acquisition of FS by…, with two partially complementary approaches standing out: the neo-Firthian approach, with Allison Wray as the main developer, and a Usage Based approach (Durrant & Schmitt, 2010). Their main point of agreement lies on the role of chunking in the L1. The neo-Firthian tradition analyses how collocations “predict each other” (Durrant & Schmitt, 2010, p. 164). This means that collocations are made up of words which “occur together more

(14)

often than would be expected by chance” (Ellis, Simpson-Vlach, & Maynard, 2016, p.380). Despite this phenomenon originally described as a textual one, neo-Firthians use it in psycholinguistic terms (Durrant & Schmitt, 2009) to refer to the acquisition and use of chunks. Usage Based theories, likewise, claim that, by associative learning, words that co-occur together are stored in the learners’ long term memory as a chunk (Ellis, 2001; Ellis et al., 2016). This process is seen as recursive, so chunks are usually combined to form larger chunks (Durrant & Schmitt, 2010). The formation and recursion of chunks constitutes the essential process of formulaic language acquisition and use as the definition by Wray & Perkins (2000) illustrates and both approaches agree on this process. However, there is a major point of divergence between these theories: the availability of that process for teenage and adult learners of a second language in non-native settings.

According to neo-Firthian theories, the differences in use between L1 and L2 speakers occur because of the disparity in acquisition conditions between L1 learners – mostly children– and L2 adult learners, determined by social and cognitive factors (Wray, 2002). Regarding the social aspect, teenagers and adults learning a language in a non-native setting predominantly learn in a classroom setting, with an encouragement on focus on grammar by the teacher (Wray, 2002) and the introduction of words instead of chunks or sequences (Durrant & Schmitt, 2010). Moreover, classroom learners sometimes use non-native sequences and avoid native-like ones since the strongest pressure comes from their L1 group, usually sharing the same deficiencies as they do (Wray, 2002) and thus reinforcing the circle. In contrast, L1 learners in natural settings are exposed to massive unfiltered input and the social pressure comes from the target language community. Although L2 adult learners in natural settings do not face this

(15)

issue, the above mentioned cognitive factors limit the advantages if compared with L1 learners. Regarding these cognitive aspect, Wray (2002) argues for L2 learners having difficulties due to their “mature mental faculties of adult learners” (Durrant & Schmitt, 2010, p. 167), meaning that being teenagers or adults, their literacy leads them to break down the components of formulaic sequences into words as they tend to “feel uncomfortable with not knowing how a memorized string breaks down” (Wray, 2002, p. 206) and thus disregard the chunk. Wray tends to associate the potential chunk uptake to intentional memorization (Durrant & Schmitt, 2010).

In contrast with this theory, a Usage Based (UB) approach argues for chunking as a phenomenon driving language acquisition both for natives and non-natives (Ellis, 2001). Within this approach, several studies suggest that, similarly to individual words, formulaic sequences acquisition responds to frequency: high frequency items are more likely acquired for non-natives (González Fernández & Schmitt, 2015) and not so frequent “but strongly associated forms” which natives use are often ignored (Durrant and Schmitt, 2010, p. 170). Therefore, the acquisition for natives and non-natives works in a similar fashion but at a completely different pace. An eventual acquisition of chunks for non-natives is linked to an accommodation to the input with a sensibility to association strength, similar to natives, instead of intentional memorization (Durrant & Schmitt, 2010) as Wray would propose. Besides, native speakers are exposed to countless more quality input than non-natives in non-natural contexts.

In addition, UB approaches draw on the Lexical Approach by Lewis (1993 as cited in Boers et al., 2006), which relies on “the power of awareness-raising to trigger acquisition through imitation of sequences” (Boers et al., 2006, p. 248). As a consequence, enhancing chunk-noticing is a means to maximize the effect of the

(16)

authentic input learners are exposed to (Durrant & Schmitt, 2010). Usage Based theories take a further step and argue for a solely implicit process of chunk-noticing, relying on the learners cognitive abilities, in which L2 learning is similar to L1.

In conclusion, the main difference between theories lies in the difference in the acquisition process of FS for natives and non-natives. Whereas the Neo-Firthian approach identifies type of instruction and the functioning of adult cognitive processing as the main factors, the UB one highlights frequency of input as the key element. Furthermore, they provide different explanations for the advantages for native speakers and problems of acquisition but also fail to explain others. With respect to the problems in acquisition of FS that non-natives show, it seems that neo-Firthian theories relate these to the lack of semantic and perceptual salience of formulaic language which is problematic for non-native speakers due to cognitive factors specific to this group (Wray, 2002). This indicates that non-natives have inherent difficulties when processing FS which natives do not show. Regarding the social aspect, the difficulties found in frequency might be related to how the context, i.e. predominantly classroom settings, limits the quantity and quantity of formulaic sequences, resulting in poor acquisition. Nevertheless, this approach does not explain how it is possible for advanced non-native speakers in non-natural settings to benefit from the cognitive advantages of employing FL demonstrated in various studies (Conklin & Smith, 2007; Jiang and Nekrasova, 2007).

On the other hand, UB theories do agree with the diagnosed sociofunctional and psycholinguistic advantages for natives and advanced learners of a language as well as with the problems stated for native acquisition. Since chunking is available for non-native speakers, the aforementioned advantages are a result of input and implicit chunk

(17)

noticing (Durrant & Schmitt, 2010). In the case of problematic issues when acquiring FL, both lack of salience and frequency are valid explanations within a UB approach. As non-natives tend to have less exposure to formulaic sequences, the implicit process of chunk noticing is hindered. The class environment in which non-natives learn tends to lack formulaic language, thus making word combinations making up chunks less frequent and less likely to be taken up, affecting statistical learning (Boers et al., 2014). Therefore, UB approaches offer a more complete explanation for the causes of the problems in non-native FL acquisition and the advantages achieved by successful learners.

2.5. Productive versus productive knowledge

In spite of their contribution to explaining the acquisition of formulaic sequences, these approaches tend to reflect, above all, on the development of productive knowledge of formulaic sequences. Productive knowledge is defined as knowledge “used in speaking and writing and involves going from the meaning to the word form” (Nation, 2001, p. 359). Although this definition addresses the productive knowledge of vocabulary in general terms, it also applies to formulaic language since it addresses two aspects that FS also meet. Firstly, productive knowledge assumes the understanding of the meaning of a FS if the learner is able to retrieve the form; in other words, productive knowledge presupposes receptive knowledge. Secondly, speaking and writing are the realization of productive knowledge. These also tend to be the means of assessing FL in most of the studies conducted, with experiments measuring the output of diverse formulaic sequences after a period of enhanced exposure (Carrol & Conklin, 2014; Durrant & Schmitt, 2010; Wood, 2006), recordings of speech (Gholami, Karimi, &

(18)

Atai, 2017), text samples or written tests (González Fernández & Schmitt, 2015). These tests, due to their written and spoken language focus, only research the productive aspect of formulaic language, disregarding the receptive side. Hence, researching receptive and productive knowledge all together is worthwhile since it helps to improve the understanding of the functioning of learners’ knowledge and the result in each of the components provides insight on how to improve teaching and learning practice (Nguyen & Webb, 2017).

On the other hand, the definition by Nation (2001) also reveals what research is lacking. When a correct form of a formulaic sequence (or any other vocabulary item) is not retrieved, this does not necessarily imply that the meaning is not known; a learner may understand the meaning when supplied with the form (Laufer & Goldstein, 2004). Furthermore, the tests analysing written and spoken production of second language learners do not effectively assess receptive knowledge since their tests involve speaking and writing tasks and analysing FL output. Thus, receptive knowledge could be assessed by testing the understanding of the meaning of a formulaic sequence.

2.6. CATSS: A possible solution

When approaching knowledge of meaning, Laufer & Goldstein (2004) argue for four hierarchical degrees of knowledge in agreement with previous definitions of lexical knowledge as a progressive ability (Faerch et al., 1984; Palmberg, 1987 as quoted in Laufer et al., 2004). Apart from the aforementioned passive versus active knowledge distinction (Nation, 2001), there is a distinction between recognition and recall. Consequently, the four degrees imply two pairs of dichotomies: “supplying the form for a given meaning versus supplying the meaning for a given form” (Laufer & Goldstein,

(19)

2004, p.405) and “being able to recall versus only being able to recognize (whether form or meaning)” (p. 406). In other words, these two dichotomies are operationalized as a four-step-hierarchy which assumes that recall is more difficult than recognition and active (or productive) knowledge is more difficult to attain than passive (or receptive) knowledge. Then the most advanced degree of knowledge is reflected in active recall and the least advanced knowledge is passive recognition. In order to test this, the authors developed the Computer Adaptive Test of Size and Strength (henceforth, CATSS). The model exercises tested in CATSS work as follows: Active recall implies supplying the target word when its first letter and its meaning are provided. Passive recall tasks require providing the meaning embedded in a short sentence which includes the target word. Active recognition tasks consist in choosing the target word from several possible choices (the target word and distracters). Passive recognition tasks, in opposition, request subjects to select the correct definition or paraphrases for the target word. Figure 1 provides an example of each of the exercises.

Active recall

Turn into water: m ______________

Passive recall

When something melts it turns into ______________

Active recognition

Turn into water

a. elect b. blame c. melt d. threaten

(20)

Passive recognition

Melt

a. choose b. accuse

c. make threats d. turn into water

Figure 1. Example of the four types of exercises to test the hierarchy of CATSS.

Adapted from Laufer et al. (2004).

The test provides two separate manners of testing subjects. As its name indicates, there is a computer version, but also a paper one. The computer version fully relies on the explained hierarchy, assuming that, if the active recall is supplied, testing less advanced stages is not needed as it represents the highest point of that hierarchy (Laufer et al., 2004). Therefore, computer-based CATSS retests only those items not correctly retrieved. Nevertheless, the paper based alternative tests the four different hierarchical stages (Laufer & Goldstein, 2004). The results in these two studies revealed the hierarchical division assumption to be valid except for the active and passive recognition distinction, for which Laufer et al. (2004) found no significant difference.

Although the CATSS testing conducted by Laufer and Goldstein (2004) is targeted towards testing general vocabulary knowledge –more specifically, single words–, it is adaptable to formulaic sequences since these follow the same three basic assumptions established by the authors: there is a form-meaning connection, knowledge of meaning is hierarchical and the overall number of items known is more important than the depth of knowledge. The first condition is based on the theory that, in a test of associations, the meaning of the target word is commonly known (Laufer et al., 2004). For formulaic language, this would imply the knowledge of the meaning of whole chunk regardless of

(21)

the knowledge of individual words they are made of. In other words, chunking is available for non-native speakers. Following a UB approach, this knowledge is attainable for non-native learners and, as a result, the presupposition is met. The second condition implies that FS would follow the same acquisition pattern as other vocabulary items, i.e. that some speakers might be able to recognize a formulaic sequence but not yet able to retrieve it. Although this does not need to be the case, separately testing this in a similar way as Laufer et al. (2004) did would allow to circumvent this assumption and test if it is correct. The third claim relates to the theory on how frequency affects acquisition. According to UB principles, the deficiency in native-likeness for L2 speakers stems from the overuse of a set of formulaic sequences with a high frequency rate and failure to learn formulaic sequences which are not very frequent but still prominent in natives’ discourse (Durrant & Schmitt, 2010). Moreover, as implications of the definition by Nation (2001) reveal, learners might have a wider knowledge of FS in a receptive state, being able to understand the meaning but not to retrieve it. Only measuring depth is possible to determine if this is the case. Additionally, an adaptation of CATSS has already been successfully conducted for formulaic language and vocabulary testing (Alali & Schmitt, 2012).

Furthermore, CATSS, as a size test, analyses size of knowledge using frequency bands. The tests includes random words from different frequency levels and assumes their representativeness for the entirety of vocabulary in those bands (Laufer et al., 2004), a common procedure in vocabulary size testing. However, results prove that the distance between some of the bands (e.g.: 2000-3000 words) is not large enough to be significant. On the other hand, theory highlights the influence of frequency on how second language speakers learn formulaic sequences and the difference it makes on

(22)

which FS are learnt (Durrant & Schmitt, 2010; Ellis, 2001; González Fernández & Schmitt, 2015). Therefore, using different frequency bands to test formulaic language size will shed light on the usefulness of this procedure. Additionally, this information will lead to further conclusions on the adequacy of an adaptation of CATSS to test formulaic sequences, the accuracy of frequency bands and the testing procedures of CATSS as a whole in relation to the theory.

Seeing as there are many unexplored implications for the receptive-productive dichotomy and its connection to the problem of frequency for non-natives, and the uncertainty of when effective acquisition occurs and if the cited dichotomy plays a role, further research is required.

2.7. Statement of purpose

On the basis of the discussed literature, formulaic language is an omnipresent phenomenon deserving proper research due to its contribution to native-likeness. The discussed advantages for native speakers are usually not present in non-native learners, who experience difficulties when learning formulaic sequences until very advanced stages of learning, with persistent odd constructions even at that level (Boers et al., 2014). Therefore, previous research seems to point out that the use of formulaic language distinguishes intermediate from advanced non-native speakers. The Common European Framework of Reference (CEFR) employs the term “proficient users” for those learners at or above a C1 level (Council of Europe, 2001). Taking the European framework into consideration, it is possible to establish a contrast between B2, an intermediate stage and C1, an advanced one. In consequence, in order to identify whether there is a significant difference between these two levels in formulaic use and if

(23)

FL use distinguishes C1 from B2, receptive and productive knowledge for these two levels need to be investigated.

Furthermore, productive knowledge represents the final stage in vocabulary development, with the assumption that receptive knowledge is previously acquired (Laufer et al., 2004; Laufer & Goldstein, 2004). CATSS offers a model which integrates both receptive and productive knowledge, testing both separately (Laufer & Goldstein, 2004). Formulaic sequences, being vocabulary items, may fall under the scope of CATSS and the hierarchy it proposes despite their acquisition happening later other vocabulary items. However, it is not clear if FS are similar to other vocabulary items, precisely because of that delay in acquisition, presence of strange constructions and underdevelopment plus the absence of a significant base of studies on the role of receptive knowledge in acquisition. Thence, testing the assumption that receptive recall precedes productive recall will provide further information on whether formulaic sequences follow the same learning process as other vocabulary items do as well as assessing the distance and correlation between receptive and productive knowledge. In short, the test will determine if one precedes the other or if they are independently stored and learned.

Finally, one of the most relevant aspects discussed is frequency of input. Not only are UB approaches based on the centrality of frequency (and input) when learning a language (e.g.: Ellis, 2001), frequency effects is also the main factor for the lag of acquisition in non-native speakers (Boers et al., 2014), resulting in an overuse of a limited set of expressions (González Fernández & Schmitt, 2015) even for advanced learners. CATSS proposes a division between bands of frequency in speech. This division needs to be adapted due to the lesser frequency of FS. Thus, it may be possible

(24)

to shed light over the hypothetical the role of the frequency of input by means of measuring a difference in acquisition between different bands of frequency of use. Additionally, testing the role of frequency may also provide help in quantifying the effect of the lack of salience for non-native speakers in non-natural contexts.

2.7.1. Research questions

The first research question relates to whether formulaic language is a good predictor of language development in non-native learners. Previous research agrees on a slow development if compared with other vocabulary components (Boers et al., 2014), resulting in an absence of formulaicity until advanced stages of the language development process. Therefore, considering a CEFR C1 level the fist advanced stage (Council of Europe, 2001), this might be the crucial stage for FL acquisition. Since the theory argues for a division between receptive and productive knowledge, three separate questions need to be addressed. First, whether the overall knowledge of FS is significantly higher for C1 than for B2. It is hypothesized that, since C1 is an advanced stage and B2 is not, FL proficiency will be better. The second and third questions refer to receptive and productive knowledge separately. For active knowledge, it is expected that proficiency will increase significantly as previous studies point towards advanced stages of development being the ones in which FS use is achieved and these studies measured productive knowledge. For passive knowledge, the developed theory assumes that it is a prerequisite for the development of the active counterpart. Therefore, if active knowledge increases, so should passive knowledge.

The second research question builds on how active recall precedes passive recall in the case of general vocabulary (Laufer et al., 2004; Laufer & Goldstein, 2004) and if it

(25)

is possible to prove so for the specific case of formulaic language. Precedence implies a temporal and possibly causal relationship, which is difficult to measure when the test is only taken once. Nevertheless, it is predicted that, since the other two assumptions for the adapted test –CATSS– are met and formulaic sequences are part of general vocabulary despite its lagged acquisition for non-natives, receptive recall will, overall, be better than productive recall. Therefore, the assumption is hypothesized to be met.

The third research question will investigate the extent to which the frequency range in which a formulaic sequence appears plays a role in its acquisition. It is hypothesized that, since non-native speakers tend to learn the most frequent items better, ignoring other also relevant but not as frequent forms (González Fernández & Schmitt, 2015), that the first frequency band will be significantly better than the second and the second will be better than the third, complying with the theory developed.

(26)

3. Methodology

3.1. Participants

Two pools of participants participated in the test. The first one compromised 17 Dutch students at the Rijksuniversiteit Groningen (RUG) with Dutch as their L1 and English as their L2. They were enrolled in English for Academic Purposes (EAP) courses and belonged to the following bachelors: European Languages, International Relationships and English majors. They were split into two groups according to their estimated proficiency level, grouping the ones belonging to English with EAP 1 as B2 according to the Common European Framework of Reference (n=18) and EAP 3 students as C1 (n=10). The potential participants were invited to take part into the research by their course instructors and by email and slides at the end of a class. The second group consisted of 27 Spanish students and workers who had been studying English in non-natural settings for a long time, lived in Spain their whole lives and held a Cambridge FCE (n=7) or CAE diploma (n=20). The potential participants were recruited via individual messaging and/or social media. No distinction was made between the Spanish and the Dutch group because the subjects in both groups were studying English in a non-native environment and that was the minimum requirement set, together with the B2 and C1 level of English. Moreover, the number of subjects was still reduced without the division and it would only called the validity of the experiment into question.

The study was performed via an online test based on CATSS especially designed for the research. These students were fully aware that the data extracted from their results would contribute to research; the rules of the university dictate it is compulsory to do so. Moreover, they were informed of the necessity of completing the

(27)

whole test so that the program used is able to store the data. This condition also needed to be met in order for them to enter a contest for a 15 euro discount voucher in Amazon or Netflix and they would choose either of them. After the raffle took place, all of the participants received an email stating whether or not they won and were thanked for their participation.

3.2. Materials

3.2.1. Target vocabulary: formulaic language

The study tests the knowledge of formulaic language in English, the L2 of the subjects. The formulaic sequences chosen belong to a selection of the most frequently used idioms in the corpora created by Liu (2003) after examining three spoken American English corpora: Corpus of Spoken, Professional American English (Barlow, 2000); Michigan Corpus of Academic Spoken English (Simpson, Briggs, Ovens & Swales, 2002), and Spoken American Media (Liu, 2002). The selected FS are part of Appendix B, which displays the “Most Frequently Used Idioms Across Three Corpora of Spoken American English (in order of Frequency)” (Liu, 2003, p. 692), distributed in three bands: 50 or more tokens per million words (TMWs), 11 to 49 TMWs and 2 to 10 TMWs. A total of 10 items were randomly selected and tested from each of the bands following the assumption that any of the items in a frequency band is a representative sample of the whole list, in accordance with the original CATSS testing, following the general assumption for tests measuring vocabulary (Laufer et al., 2004; Laufer & Goldstein, 2004). Although the original test included wider bands (e.g.: 2000-3000 TMWs), formulaic sequences are less common in speech, reducing the range of occurrence.

(28)

3.2.2. Test instrument: CATSS-based test

In order to test the subjects, an online test with its roots in the Computer Adaptive Test of Size and Strength (CATSS) has been developed. The adapted version puts its focus on formulaic language instead of vocabulary and only tests the two final steps in the hierarchy formulated by Laufer & Goldstein (2004), this is, receptive and productive recall. The particular examples have been created by the researcher, an advanced non-native speaker with experience on testing and assessed and improved by a non-native speaker to ensure adequacy and evade non-native-like expressions that might hinder the understanding of the task for the participants. Productive recall exercises involve a fill-in-the-gap mechanism with instructions to retrieve the meaning of a missing word or words in a sentence using the explanation or synonym in brackets plus the first letter of the formulaic sequence as a hint (see figure 2), similar to the original test. In addition, since the original CATSS tested individual words, the number of words each collocation has is provided in order to ensure that the level of difficulty is similar. Correctly inferring the expression from the meaning equals productive recall knowledge of the expression.

(29)

Figure 2. Example of a productive recall exercise. Expected solution: ’Go on’.

Receptive recall exercises require the subjects to infer the meaning of a collocation from the collocation itself and the first letter of a missing word or words in the explanation (see figure 3). Although the original CATSS did not include either the first letter hint or the number of words, they have been added due to the greater difficulty in defining formulaic sequences. Correctly guessing the missing word or words of implies receptive recall of the FS. Appendix 1 displays all the questions asked to the subjects –both productive and receptive– and Appendix 2 shows all the answers in the correct order.

Figure 3. Example of a receptive knowledge exercise. Expected solution: ‘up till’

The tool employed to create the test was Qualtrics Survey Software (Snow & Mann, 2013), chosen due to its adaptability and ease in order to create a questionnaire and a survey within the same framework. The software enables researchers to create conditions to trigger further questions, display the results in a clear way and offers a wide range of options on how to present the questions. Furthermore, after the completion of the task, it allows to download the data in order to conduct a proper

(30)

analysis. The conditions established in the test included the obligation to answer each exercise (either correctly or incorrectly) and the randomness in the presentation of the exercises.

3.3. Procedures

The study was performed via online testing in one go. The overall study consisted of two different parts. The first one involved personal data and the second one consisted in the test explained above. After both parts were completed, the subjects were thanked for their participation and offered the chance to participate in a contest in which a 15 euro voucher for Amazon/Netflix was the prize. The e-mail address of all the subjects who had agreed to participate in the raffle was entered in a randomizer to determine the winner and notify whether they had won or not. The various email address were stored separately from the rest of the data and they are only used for the contest and the subsequent notification for the winner.

The first part of the test elicited the proficiency level of English of the subjects so as to comply with the first research question. The options for proficiency were B2 (English

Proficiency), C1 (EAP 2: Argumentation), B2 (other) and C1 (other). Since there were

four different groups of participants, the division intended to facilitate the choice for the subjects and allow further analysis between groups if needed and the degree they studied. This means that Dutch subjects should choose between the two courses they were taking (English Proficiency or EAP 2: Argumentation) and, since the Spaniards were chosen because they held an English proficiency diploma (either in B2 and C1), they had to choose between the remaining two (B2 (other) and C1 (other)). After a

(31)

revision of the results, the degree studied was considered non relevant and no further action was taken on them.

The second part of the test displayed the test exercises in a random order. In contrast to the original CATSS online test, the 60 exercises were completely randomized, regardless of whether they targeted receptive or productive formulaic language knowledge. In doing so, receptive and productive versions of the same item were tested independently. Even though the computerized version of CATSS would only test receptive knowledge if the elicited productive equivalent was not correctly retrieved, the test in this study elicits both so as to compare the results and test the assumed hierarchical relation between stages in vocabulary/FS knowledge. In addition, the inclusion of 60 exercises was one of the main reasons for not including receptive and productive recognition, since it would double the time the subjects would need to employ, resulting in possible dropouts.

The test contained the three frequency bands developed by Liu (2003) explained in the previous section. Each of the three bands included 20 items: 10 for productive and 10 for receptive knowledge, testing the same 10 idioms and adding up to 60 individual exercises. Moreover, participants were required to fill in every answer even if they did not know what to respond. As with the original CATSS, the productive version prompted the subjects to actively recall the item by providing the meaning, and the first letter of the idiom within a context. Provided that formulaic sequences do contain more than a word, the number of words was added to facilitate the task. On the other hand, the receptive version required the subjects to provide a definition for each FS in the context of a sentence. Again, the number of words per gap in the definition was given to force a response as similar to the preconceived one during the design of the test as

(32)

possible, although answers fitting the criteria which were not that one were also allowed.

The experiment took between seventeen minutes and five hours per student. It is hypothesized that the test was left open for that span of time as there was no activity for the most extreme times.

To conduct the analysis of the results, the results from Qualtrics (Snow & Mann, 2013) were downloaded into a Microsoft Excel (Microsoft Corporation, 2007) spreadsheet. Each participant’s answer was operationalized as 1 or 0, representing correct and incorrect respectively and based on the following rules: firstly, words with a small typo not inhibiting the understanding of the response were considered correct and so were words containing tense mistakes (such as third person singular s and the use of present simple instead of past simple, or vice versa). Secondly, collocations missing a word or with a different particle were assumed as incorrect since the experiment intends to test chunk knowledge and incorrect combinations do not match with the FS meaning. In order to analyse the data and conduct descriptive statistics and the corresponding test, GraphPad Prism 7 (GraphPad Software) was employed. The program was chosen due to the simplicity in entering the data, which needed just to be translated from Excel, and the user-friendliness when conducting the tests, giving access to a variety of options including testing the assumptions, the selection of the interval of confidence or the experimental results (one or two/tailed, paired or unpaired for t-tests, etc).

With regards to the drop-out participants in the study, Table 1 displays the participant pool, reflecting the number of participants in each category, the number of drop-outs and the number of subjects completing more than 90% of the test. Tests with

(33)

a ninety percent completion were considered finished and the remaining answers operationalized as 0 because of their non-completion.

Table 1. Completion and mortality rates.

Groups Total

enrolment

Completed test Completed >90% Drop-out B2 (Dutch) 18 8 2 8 C1 (Dutch) 10 6 1 3 B2 (Spanish) 7 4 0 3 C1 (Spanish) 20 13 1 6

Table 1 shows a high attrition rate, with up to a 44% dropout in the case of the B2 participants and 30% in the case of C1. In terms of nationalities, the dropout rate was similar, with a 39% for Dutch participants and 33% of the Spaniards dropping out. These results allow for two interpretations: the test was too long and forcing the participants’ response even if they did not know the answer was not a good choice. The test needed to be exhaustive enough to test both frequency bands and receptive and productive knowledge. Hence, 60 questions were needed, which might have been overwhelming for some of the subjects. Secondly, forcing the answer was a means to ensure that the subjects did not mindlessly skip over those exercises which, although they might know the answer to, did not seem as easy as others. In other words, this attempt to ensure the reliability of the test entailed a drawback.

(34)

3.4. Design and Analyses

3.4.1. RQ1: Is there a difference between B2 and C1 in FS?

FS acquisition is allegedly not effective in SLA until a very advanced stage due to input and uptake. In other words, whereas C1 learners are supposed to productively employ FS, B2 are not. Because of this, testing the development of productive and receptive knowledge between these stages separately provides information on whether FS use predicts proficiency.

Considering the proficiency level of English as the independent variable, operationalized as a nominal variable with two levels (B2 and C1 or upper-intermediate and advanced) and knowledge of English idioms as three different dependent variables, operationalized as interval variables (overall knowledge, receptive knowledge and productive knowledge), the following hypotheses are proposed:

1. Hoa: there is no significant interaction between proficiency level of English and overall knowledge of Formulaic sequences. Hence, the H1a: there is a difference in overall knowledge of FS depending on the level of English proficiency. A t-test will t-test the hypothetical relation between two groups of FL use. An alpha level of .05 is chosen because this particular case does not require a stricter alpha-level. One-tailed testing is employed since the literature section seems to predict a positive outcome.

2. Hob: there is no significant interaction between proficiency level of English and receptive knowledge of Formulaic sequences. Hence, the H1a: there is a difference in receptive knowledge of FS depending on the level of English proficiency. A t-test will test the hypothetical relation between two groups of FL use. An alpha level of .05 is chosen because this particular case does not require

(35)

a stricter alpha-level. One-tailed testing is preferred since there is no prediction of a negative outcome in the discussed literature.

3. Hoc: there is no significant interaction between proficiency level of English and productive knowledge of Formulaic sequences. Hence, the H1b: there is a difference is productive knowledge of FS depending on the level of English proficiency. A t-test will test the hypothetical relation between two groups of FL use. An alpha level of .05 is also chosen for this test. One-tailed testing is employed because there is no evidence in the reviewed background pointing towards a reduction of receptive knowledge in the course of development.

3.4.2. RQ2: Does active knowledge precede passive for FS?

Laufer and Goldstein assume that passive recall is a pre-requisite for active recall. Therefore, correct answers for active knowledge are supposed to correlate with correct responses for passive knowledge, but correlation does not necessarily mean causation. In consequence, three different tests need to be conducted and their joint result will lead to a conclusion on whether the assumption is met for FS.

The first test would imply comparing receptive and productive exercises answers per student following this criterion: if receptive knowledge precedes productive knowledge, the results for the former should be equal or better than for the latter. In other words, incorrect receptive response of and correct productive response of the same FS id considered a violation. This analysis provides a percentage of violations and compliances with the criterion.

The second test is a comparison between the average score between all the participants and all the exercises for receptive and productive knowledge separately. If

(36)

the receptive mean is higher than the productive mean, this by and large suggests the assumption is correct.

Finally passive knowledge is operationalized as an independent variable operationalized as an interval (mean of all receptive answers per participant) and productive knowledge as a dependent one, operationalized as an interval (mean of all productive answers per participant), the following H0 is selected: there is no significant correlation between passive and active knowledge of FS. Therefore, the H1: there is a correlation between passive and active knowledge of FS. A Pearson correlation will test the hypothesis. Similarly to RQ1, the alpha level is set at .05 in a one-tailed test.

3.4.3. RQ3: Is there a significant difference between the 3 bands of frequency?

Frequency is a key element for FS to be acquired and the most common items tend to be the best learnt for non-native speakers. Hence, using a test with FS from three different bands of frequency might provide information on how different the acquisition of FS is depending on the band regardless the stage of knowledge (productive or receptive) and the level of the learners (B2 or C1).

Regarding frequency as an independent nominal variable with three levels (50 or more TMWs, 11 to 49 TMWs and 2 to 10 TMWs) and the score on the exercises regardless the type of knowledge as an interval variable, the following H0 is proposed: there is no significant interaction between the band of frequency of a FS and score. Thus, the H1: there is difference in score depending on the band of frequency of a FS. A one-way ANOVA will test the hypothesis. The alpha level is established at .05 as it is common in social sciences.

(37)

4. Results

This chapter outlines the results of the analyses following a similar order as the Methods section. The results include the descriptive statistics of the variables, the outcome of the statistical tests proposed to analyse the data plus its interpretation. Specifically, the labels and categories will be justified, followed by the presentation of the results and its interpretation. All three research questions are based on quantitative analyses although the second one has an exploratory component due to the difficulty in testing precedence reliably.

4.1. Difference between B2 and C1 in Formulaic Sequences

The results for the difference between B2 and C1 emerge from the average score of all the participants within the category for each answer. Three different t-tests examine the hypothetical difference between B2 and C1 in receptive, productive and overall formulaic sequences knowledge.

4.1.1. Difference between B2 and C1 in Overall Formulaic Sequences

After plotting the data, the descriptive statistics for B2 (N = 60, m = 0.48, SD = 0.24) and C1 (N = 60, m = 0.56, SD = 0.24) overall FS knowledge point towards normal distribution in both cases. The analysis of descriptive data also reveals little dispersion and similar maximum and minimum values (see table 2). After conducting a Shapiro-Wilk normality test, normality is confirmed in the two sets of data (p (B2) = .33; p (C1) = .13).

(38)

Values B2 (n = 14) C1 (n = 21)

Mean 0.48 0.56

Max 1 0.95

Min 0 0

SD 0.24 0.24

After testing the assumptions, a Welch’s t-test is administered, since there are two separate groups and no similar SD can be assumed. The results show that C1 overall Formulaic Sequence knowledge (M = 0.56, SD = 0.24) is better than overall B2 Formulaic Language knowledge (M = 0.48, SD = 0.24). The difference is significant (t (118) = 1.84, p = .034) and the effect size is very small (r2 = 0.027).

Figure 4. Boxplot displaying the differences between B2 and C1 groups. 0 0,2 0,4 0,6 0,8 1 1,2 B2 C1 M e an S co re

(39)

The boxplot seems to comply with the results of the t-test. It shows that although the median is somewhat similar for both groups, the distribution is quite different. For B2 the distribution is even whereas for C1 there is less agreement between participants above the median than below. The maximum and minimum are similar for the two groups but both the upper and lower quartile in C1 show higher values. Therefore, it reflects what the Welch’s t-test asserts plus providing extra information on how the B2 group is more consistent than the C1, especially in the upper quartile, in contrast with what the descriptive data revealed.

4.1.2. Difference between B2 and C1 in Receptive Formulaic Sequences

With regards to receptive knowledge specifically, the descriptive statistics indicate that there is little dispersion, that the minimum and maximum values are similar for both B2 receptive formulaic knowledge (N = 14, m = 0.49, SD = 0.2) and C1 receptive formulaic knowledge (N = 21, m = 0.57, SD = 0.23) and point towards normal distribution. A Shapiro-Wilk normality test confirms this, with p-values above the significance level (p (B2) = .6; p (C1)= .51). Table 3 provides further visualization of the data distribution.

Table 3. Descriptive statistics of B2 and C1 groups for receptive knowledge.

Values B2 Receptive (n = 14) C1 Receptive (n = 21)

Mean 0.49 0.57

Max 0.97 0.91

(40)

SD 0.2 0.23

Since there are two separate groups and equal standard deviation cannot be assumed, a Welch’s t-test needs to be employed. The result revels that there is no significant difference between B2 and C1 in receptive knowledge (t (82) = 2.291, p > .05). Therefore, the H0b hypothesis cannot be rejected.

Figure 5. Boxplot displaying B2 and C1 receptive knowledge differences.

The boxplot reflects that there is a difference in distribution between B2 receptive and C1 receptive groups despite de median being similar. The score is more dispersed in the lower quartile for B2 than in the upper quartile. For the C1 group, the score of the upper quartile is mostly similar to the lower quartile. In addition, although the maximum is higher for the B2 group, the upper quartile is comparatively higher for the C1 group. The minimum and lower quartile ranges are similar for both groups. Overall, despite the

0 0,2 0,4 0,6 0,8 1 1,2 B2 Receptive C1 Receptive M e an sco re

(41)

dissimilarity in how the data is distributed for each group, the boxpot mostly agrees with the results of the t-test, indicating that the differences between groups are not great enough to be significant.

4.1.3. Difference between B2 and C1 in Productive Formulaic Sequences

Regarding productive knowledge of formulaic sequences for B2 (N = 30, m = 0.43, SD = 0.22) and C1 knowledge (N = 30, m = 0.56, SD = 0.23), the analysis of descriptive statistics show little dispersion and similar maximum values, yet sensibly different minimum values, as table 4 reflects. However, plotting the data reveals negative kurtosis for C1. A Shapiro-Wilk normality test indicates normal distribution, displaying non significant p-values (p (B2) = .26; p (C1) = .11).

Table 4. Descriptive statistics of the results of B2 and C1 groups for productive

knowledge.

Values B2 Productive (n = 14) C1 Productive (n 0= 21)

Mean 0.44 0.55

Max 1 0.95

Min 0.07 0.19

SD 0.22 0.24

A t-test revealed that, on average, subjects with a CEFR C1 level of English (M = 0.55, SD = 0.24) seemed to show better knowledge of productive formulaic language

(42)

than CEFR B2 (M = 0.44, SD = 0.22). This difference was significant t (58) = 2.01, p = .049) and the effect size was very small (r2 = 0.007).

Figure 6. Boxplot displaying B2 and C1 productive knowledge differences.

The boxplot reflects what the Welch’s t-test has shown. At first glance, the distribution of scores is dissimilar in the two groups. Firstly, the data is more equally dispersed for C1 than B2 productive knowledge. B2 also shows more dispersion in the lower quartile, whereas C1 does the opposite. Additionally, the maximum and minimum values are further from both quartiles in B2, showing more diverse participants than for C1 productive knowledge. Finally, although the maximum values are higher for B2, the boxplot provides additional information on how the overall data is consistently higher for C1, thus complying with the results of the t-test.

0 0,2 0,4 0,6 0,8 1 1,2 B2 PROD C1 PROD M e an S co re

(43)

4.2. Receptive knowledge precedence towards productive knowledge in FS.

As stated in the methods section, there is no conventional test to investigate causality. Therefore, there needs to be an alternative which points towards a hypothetical causality relation. In light of the assumption that receptive knowledge precedes productive knowledge, three different tests were conducted; the first two were exploratory and the third one was quantitative. The first one compared the number of cases in which the assumption was met for exercises in which the receptive and the equivalent productive FS were requested for each participant. The second one compared the average score the participants got in receptive exercises and in the receptive counterpart regardless the level of proficiency. The third test analyses the hypothetical correlation between average receptive and productive score per student, again disregarding the level of proficiency.

4.2.1. Assumption meeting in overall results

The overall results display 1037 pairs of observations. After comparing the number in which the score for a receptive answer was equal or better than for a productive one, it was determined that in 81.87% of the cases the assumption was met. This implies that, out the 1037 pairs of observation compared, 849 represented cases in which receptive knowledge was equal or superior to productive knowledge and 188 (18.13%) violate the assumption that receptive knowledge of FS precedes productive knowledge (see figure 7). Therefore, the assumption is met in over four times more cases than it is violated.

(44)

Figure 7. Pie chart of the compliances/violations of the assumption distribution.

4.2.2. Comparison between receptive and productive knowledge.

In addition, the calculation of the average for receptive and productive exercises regardless of the participant reports that the mean from every receptive exercise for all the participants was 0.56 whereas the mean for productive exercises was 0.49. Complying with the assumptions from the first test, the average for receptive knowledge is higher than for productive, suggesting the assumption is correct.

4.2.3. Correlation between receptive and productive knowledge between subjects.

In order to test the hypothetical correlation between receptive and productive knowledge in each of the participants, the mean of all the exercises for receptive and the mean for all productive exercises was calculated for every participant. A Shapiro-Wilk normality test revealed normal distribution (p (Receptive) = .22; p (Productive) = .48).

Receptive-productive pairs

Assumption met Assumption violated

(45)

The conditions for a Pearson correlation are then met. There is a significant positive relationship between the mean scores per participant in receptive knowledge exercises and scores in productive knowledge exercises, Pearson r (18) = 0.758, p < 0.001 (one-tailed). The size effect is r2 =0.57, indicating that 57% of the data comparing receptive and receptive knowledge means in each participant can be explained. This is a moderately large effect size. The scatter plot below seems in agreement with the Pearson correlation, displaying a strong positive linear relationship with the residuals forming an upward trend.

Figure 8. Scatter plot of the correlation between mean scores in each participant.

4.3. Difference between bands of frequency

The results for the difference between bands of frequency are derived from the average score of all the participants within the band for each answer. An ANOVA tests the hypothetical difference between Band 1, Band 2 and Band 3 in Liu’s (2003) “Most Frequently Used Idioms across Three Corpora of Spoken American English (in order of

0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 0 0,2 0,4 0,6 0,8 1 Por d u ctiv e kn o wl e d ge Receptive knowledge

Formulaic sequences in the L2 Predicting advanced learners, receptive versus productive knowledge and difficulties in measuring with frequency bands