A validation study: is prediction in eye-tracking a measure of
implicit processing?
U
NIVERSITEIT
VAN
A
MSTERDAM
Thesis Master Linguistics of European Languages: Spanish March 2018
Supervision: Sible Andringa Name: Elisabet García González
Student number: 10404503
Abstract
: This study investigates the effects of metalinguistic information in the online L1 and L2 processing of Differential Object Marking (DOM). This research also addresses the validity of measures of implicit knowledge in language processing. Our results show that Spanish native speakers are able to predict the presence of an animate object through grammatical cues, and so are advanced Dutch language learners of Spanish after explicit instruction of the grammatical rule. The retrospective reports indicate that Spanish native speakers are less aware of the linguistic structure than L2 speakers, even after explicit instruction of the target structure. Additional switch analysis shows that L2 speakers can also predict when unaware, but the effect seems driven by those speakers who once were taught the rule. Our results raise interesting questions about the role of awareness in native versus L2 speakers, and with respect to the circumstances in which prediction may occur in experimental settings. These findings contribute to the on-going discussion of sources of knowledge and awareness, but also deepen into the validity and reliability of current measures of implicit and explicit knowledge.INDEX
1. The importance of awareness in SLA 3
2. Measures of knowledge: explicit vs implicit 4
2.1 Measures of awareness in second language acquisition 5
2.2 The visual eye paradigm as a measure of implicit processing 6
2.2.1 Prediction: if it does not always occur, is it necessary for language processing? 7 2.2.2 Do all speakers predict? The importance of individual differences 9
3. This study 9
4. Method 10
4.1 Target structure (DOM) 10
4.2 Materials 11 4.3 Procedure 11 4.4 Participants 11 4.4.1 Spanish participants 12 4.4.2 Dutch participants 12 5. Analysis 12
5.1 Eye-tracking data analysis 12
5.2 RT analysis 13
5.3 Debriefing and awareness analysis 13
6. Results 13 6.1 Eye-tracking results 14 6.2 RT results 16 6.3 Switch analysis 17 6.4 Debriefing results 18 6.4.1 Spanish group 18 6.4.2 Dutch group 19 6.5 Vocabulary test 19 7. Discussion 20 8. Concluding remarks 23 Acknowledgements 24 References 25
1. The importance of awareness in SLA
The concepts of attention and awareness are an unresolved topic in the multiple debates in the field of second language acquisition. Much of the discussion arises from questioning the usefulness of making learners explicitly aware of the rules of the language they are trying to master. While the exact role of attention and awareness in SLA is not completely clear, it is generally considered that awareness is the product of a sufficient degree of attention (Rebuschat, 2015).
The two extreme positions in relation to the role of awareness are the following. First, it has been argued that learning without awareness is impossible. Conversely, scholars have claimed that some degree of awareness is a requirement for learning to take place. Schmidt’s (1990) noticing hypothesis is central in this discussion. This hypothesis holds that for learning to occur some degree of heightened awareness from the input is needed. Others have argued that learning without awareness may also occur. These discussions have inevitably led to the great question about how to define awareness.
What is clear from these discussions is that awareness is a factor to be reckoned with when measuring learner’s ability or progress. While awareness in first language does not immediately comes into questioning, it is a subject that has drawn several lines of research in second language acquisition. This interest probably started when Krashen (1981) put forward his challenging distinction between acquisition and learning, claiming that one involves no awareness for the linguistic items that are learned, while the other depends on awareness. Krashen argued that successful SLA depends on acquisition, not learning, and thus some of the controversy is an issue of definition. As has been pointed out before (Godfroid et al., 2015), awareness is a dichotomous construct, whereas attention is a continuous variable. Awareness is important for learners to make hypotheses and understand how the target language functions, and therefore is part of the process of learning. What is important to clarify is that awareness occurs somewhere in the process of language learning, but it is the product of awareness what linguists asses, interpreting it into a dichotomous variable.
Thus, the assessment of awareness is somewhat more problematic, and interferes with the core understanding of awareness. The process of becoming aware may develop along time, but how scholars measure awareness is by targeting the product, the outcome of awareness, namely, the verbalized understanding of learner’s intake. But what might be the effect of awareness of linguistic rules in speakers that are learning a foreign language? And how does being (un)aware affect
linguistic processes? This study assess awareness through verbal reports, with the intention to tap into Spanish native speakers’ and Dutch L2 learners’ processing of Differential Object Marking in Spanish.
2. Measures of knowledge: explicit vs implicit
There are two crucial issues that the current methodology fails to disentangle. Firstly, we must wonder what measures can unequivocally tap into implicit processing. Grammaticality Judgement Tasks (GJT) do not seem to be the ideal measure for studying predictive behavior nor for implicit processing, as many have shown its difficulty to tap into different language processes unequivocally (Ellis 2004, 2005; Rebuschat, 2015). Instead, measuring predictive behavior by means of timed, online techniques may show signs of implicit processing. Secondly, we need to understand what the role of awareness in language predicting behavior is. Whether speakers are aware of relevant linguistic information has been used as means to study predictive behavior in learners of an L2. In this respect, we tap into speaker awareness in reacting behavior by means of verbal reports.
While measuring awareness through the assessment of specific components of knowledge has proven difficult, teasing apart implicit and explicit processing is not less challenging. Research has found evidence that different type of learners should have an advantage for a specific process: L1 speakers may be better at implicit tasks, but not necessarily at explicit tasks, for which L2 speakers might have had specific instruction (Ellis, 2005). However, research has found great variance in how these two groups of speakers perform in such tasks, suggesting that the type of test used might affect participant performance. Grammaticality Judgment Tasks (GJT) are still the most frequent measure in L2 learning. While timed GTJ should test implicit knowledge, which is automatized and accessed more rapidly, untimed GJT test explicit knowledge, which requires more time to process and reflect on language. Nonetheless, Ellis argues that the use of GJTs can lead to the erroneous outcome if not conducted carefully, namely, we may test explicit knowledge when we intended to asses implicit knowledge, or vice versa. In a study that evaluated five different tasks, Ellis found a difference between grammatical and ungrammatical sentences within the untimed task: while grammatical sentences seemed to tap into implicit knowledge, ungrammatical sentences affected explicit knowledge. Besides the timed and untimed GJT, an oral narrative, a metalinguistic test and imitation test were also performed. Interestingly, the untimed GJT appeared to correlate with metalinguistic knowledge more than the metalinguistic test itself. Thus, it appears that the
testing conditions eg.: highly automatic tasks, rather than the type of task itself, may be more informative of the language processes we aim to measure. While Ellis’ findings raise the point that no measure can ever perfectly test linguistic processes and awareness, that does not minimize the importance of awareness in language processing. Conversely, Dienes et al. (2010) recognize the limitations of conscious perception of linguistic structures, but claim that factors as familiarity of the structure and the combination of verbal reports with other measures are reliable tools to measure speaker awareness. It seems clear that, while verbal reports may not be the absolute reliable measure to asses awareness, they have been used across fields of research and can certainly provide highly useful information.
In relation to this, we may wonder: is it possible to learn without awareness? What is the role of awareness in (implicit) language processing? Does awareness affect L1 and L2 speakers in the same way? In this study, we precisely dive into these issues and argue for the possibility of learning to take place through unaware, implicit processes. Following we present the current view on the validity of measures of implicit knowledge in L2 acquisition.
2.1 Measures of awareness in second language acquisition
Second language acquisition research (SLA) has addressed the importance of awareness in language learning. Yet, measuring whether speakers are (un)aware of a linguistic rule is a very complex matter. Rebuschat (2015) stresses that awareness is often assessed by the use of verbal self-reports of participants: those who are able to verbalize something relevant in language are classified as aware. Thus, speaker awareness is evaluated based on the ability to put into words the understanding of something that occurs in language (Ellis, 2005).
Since awareness in SLA research is often determined by tests of metalanguage or retrospective verbal reports, assessing awareness and consequently explicit knowledge can be problematic. According to Ellis, metalanguage is not an essential component of explicit knowledge, as it only is verbalized understating of rules that apply to linguistic phenomena, e.g. ‘‘In the case of dative alternation, there are some verbs like explain that require the indirect object to be realized as a prepositional phrase rather than as a noun phrase.’’ (Ellis, 2004: 240). Thus, the ability to express metalinguistic information — although might correlate with awareness at some level — does not imply it. Ellis (2004) argues that analyzed (explicit) knowledge and metalanguage must be measured independently and proposes specific guidelines for their assessment. Firstly, a test should
analyze learners’ explicit knowledge and distinguish between that and their language aptitude. The focus of a test should be on a specific aspect of language, e.g. grammar. It should also target learners’ awareness of the linguistic differences between the L1 and the L2 (Ellis, 2004, 266). Thus measurements of metalanguage should be secondary and should target receptive rather than productive knowledge of metalanguage, e.g. if speakers are asked to evaluate a linguistic rule, rather than requested to formulate it.
Ellis (2005) challenges the use of verbal reports as measures of awareness, which are so often used in the field of L2 acquisition. In his psychometry study, Ellis (2005) used a battery of tests to measure participants’ implicit and explicit processing. A knowledge test and a metalanguage test were used as measures for explicit knowledge, which he found could account for most of the variance. Yet, the metalanguage test seemed to be a better measure for implicit processing than for explicit.
Attempting to assess implicit knowledge, some studies have used techniques that target highly rapid and automatic responses, such as the ability to predict linguistic information in eye-tracking tasks (Huettig, 2015; Andriga & Curcic, 2015). While various studies have used prediction in eye-tracking to investigate implicit knowledge, the relation between awareness and prediction is highly understudied.
2.2 The visual eye paradigm as a measure of implicit processing
Recently, researchers have introduced the use of online processing measures such as ERP (Tanner & van Hell, 2014), eye-tracking (Huettig 2015; Andringa & Curcic, 2015) or self-paced reading (Robert & Felser, 2011) to assess how speakers process language. Speakers are faced to provide rapid reactions to linguistic stimuli in an effortless, unconscious way. Eye tracking data has been used in the field of SLA (and L1) research to investigate how individuals react to syntactically ambiguous sentences or complex syntactic structures (Dussias & Sagarra, 2007, in Godfroid & Winke, 2015). The goal of this study is precisely to confirm whether eye tracking is valid tool to study access of implicit knowledge.
The visual world paradigm measures anticipatory effects (such as prediction). Such effects can occur when a particular word contains information (eg.: grammatical gender) that allows the speaker to disambiguate between different items presented to them visually (Godfroid & Winke, 2015). Predicting behavior is, in theory, unconscious, and could be evidence for implicit processing.
Most research has used this type of methodologies to study anticipatory behavior in the processing of eg.: grammatical gender in L2 (Hopp, 2012; Dussias et al., 2013) or artificial languages (Andringa & Curcic, 2015). While the use of eye-tracking measures does not imply that the particular structure will be process implicitly, it reflects the application of explicit knowledge (Andringa & Curcic, 2015), and perhaps, the absence of it may reflect implicit processing of linguistic structures. Godfroid & Winke (2015) suggest that eye-tracking data provides insight of automatic and effortless processing. Yet, researchers need additional evidence to establish learners’ consciousness involved in such processes, namely, a triangulation of eye-tracking data, verbal reports and ERPs. Another issue challenges the validity of online measures; whether learners can access linguistic knowledge during real-time processing. For this reason, Godfroid & Winke (2015) support Ellis’ proposal of the use of timed and untimed tests GJTs to measure different constructs that can me labeled as implicit or explicit processes. In this study, we combine eye-tracking techniques with debriefing data to establish a relation between speaker’s implicit processing of language, their awareness of linguistic cues and consequent ability to predict information based on them. In the following section, we present different views on the effect of prediction in language processing.
2.2.1 Prediction: if it does not always occur, is it necessary for language processing?
Eye-tracking has been used to investigate predictive linguistic behavior in speakers first language. Such anticipatory behavior has been argued to be a fundamental property of language processing as it occurs effortlessly and unconsciously (Ferdermeier, 2007). There is evidence that suggests that some core features of language, such as grammatical gender, are easily used as cues for predictive behavior in speakers L1. Dussias et al., (2013) found that Spanish native speakers used gender’s determiner to predict the object of a sentence. Nevertheless, this is not always the case for L2 speakers, who may not process language in the same manner as L1 peers. In the same study, Dussias et al., (2013) found that L2 speakers of Spanish could only use predictive processing when they were very proficient.
Both structural differences and language proficiency affect L2 processing, therefore, can we claim that that prediction is a core feature of language processing? Huettig & Mani (2016) argue that prediction can occur in specific contexts in which it provides a helping hand for language understanding, but it is not necessary for it. According to them, prediction is subject to individual
differences (eg.: in listening skills or working memory). Thus, not all speakers are able to predict upcoming linguistic information. In other words, prediction might help language comprehension, but it is not a requirement. Additionally, input that is not optimal can hinder prediction, whereas it can certainly be “encouraged” by the specific experimental setting (Huettig & Mani, 2016)
In order to address whether speakers predict linguistic information, it is crucial to understand what prediction actually is and how it takes place. Huettig (2015) addresses four main issues about prediction (why, what, how, when). Firstly, why prediction may occur may be explained by potential benefits such as faster processing or reduced ambiguity. For instance, it could be the case that less natural environments, such as experimental settings either trigger or impede prediction. On the one hand, prediction could compensate for the lack of crucial cues in natural communication. On the other hand, it might precisely be those cues of natural communication what benefit prediction. Nevertheless, whereas many studies find prediction in upcoming syllables and other aspects of phonology and phonetics, they also seem to be linked to individual processing differences or particular contexts.
Secondly, we must ask ourselves what prediction actually is, namely, what type of information is used to predict an upcoming word and what kind of representations are consequently activated (Huettig, 2015). Case-marking (Kamide et al., 2003), prosody (Weber et al., 2006), and visually presented events (Knoeferle et al., 2005) have been found to encourage language prediction. However, those findings do not provide sufficient evidence to claim that prediction is fundamental for language processing (Huettig, 2015). What is clear is that there is a need for some kind of linguistic cue for prediction to take place.
When prediction occurs has been difficult to address due to the different methodologies used
in studies. It may occur less in challenging situations or with less proficient users (Huettig, 2015), for instance, it may be less likely for speakers to predict in a non-native language. Finally, how does prediction occur, we can only assure that it is highly determined by individual differences such as working memory, cognitive efficiency, or age, but much research is needed to understand the interplay between those and many other factors (Huettig, 2015).
The pivotal questions raised by Huettig (2015) make us question what speakers and under what circumstances may benefit from prediction. Is it possible to predict in experimental settings? Can non-native speakers use prediction cues? and would all speakers be able to use them in a similar way? Does awareness encourage prediction? Our study addresses all this questions by evaluating awareness of DOM in native and non-native speakers via eye-tracking.
2.2.2 Do all speakers predict? The importance of individual differences
Huettig and Mishra (2014) have also found effects for the effect of literacy in speakers’ ability to predict. Most studies investigate language processing on highly literate speakers (students), which are the WEIRDest (Western Educated Industrialized Rich Democratic) segment of the world population and thus do not accurately represent the majority of the world population, according to the authors. In a review of studies about the cognition of (i)literate speakers, they found that auditory prediction is little or no affected by low literacy. To the contrary, visual prediction seems to be affected by the level of literacy when cultural symbols are presented in a manner that resembles the written modality (eg.: black lines on white). Furthermore, the majority of studies used an offline method, which could impact the effects of phonological representation, more than phonological processing. In sum, the question is whether literacy itself improves processing and whether this effect can be found in day to day communication, as not everybody will be in a context of literacy from which they could benefit from their literacy skills (Huettig & Mishra, 2014)
The studies above mentioned suggest that prediction does not necessarily occur depending on speakers L1 or L2, and that it may not be a core feature of language processing. Instead, prediction may be the result of a strategic behavior on which (some) speakers can rely. We have some understanding of what speakers are able to predict and in which contexts, but the factors are varied, from individual differences to the necessity of the context or reliability of the input. This study does not aim to dive into all possible factors for prediction to take place, but we tap into the relation between awareness and prediction. Specifically, we ask the question whether awareness can enhance prediction in language processing and what it entails for L1 and L2 speakers.
3. This study
This study seeks to investigate to what extent prediction in language processing is implicit. Considering the current difficulty of measures to unambiguously measure implicit knowledge, this study uses the world eye paradigm in an attempt to access implicit processing in speakers. We do so by studying the predictive behavior of L1 and L2 speakers of Spanish. This study thus aims to answer the following research question
RQ1:To what extent is prediction in language processing an implicit measure of knowledge?
The role of awareness in language learning has been considered crucial. However there is not clear evidence that awareness is correlated with the ability to predict upcoming information. This paper presents evidence that suggest that prediction may occur when unaware, in both L1 and L2 speakers. We support this claim with debriefing data from participants which indicates that prediction might be a strategic behavior in some, but not all speakers. Thus this study aims to answer the following research question:
RQ2: How does awareness affect predictive behavior?
4. Method
4.1 Target structure (DOM)
This study investigated Differential Object Marking (DOM) in Spanish, a morpho-syntactic structure that is constrained by the semantic properties of animacy, specificity and telicity (Guijarro Fuentes, 2012) Animacy is the most prominent feature of DOM and a grammatical rule established 1
that animate direct objects are preceded by the proposition a ('to') (1), whereas inanimate objects are not (2).
(1) La niña vio a esta mujer.
The.FEM girl saw to this.FEM woman ‘The girl saw this woman.’
(2) La niña vio esta caja.
The.FEM girl saw this.FEM box ‘The girl saw this box.’
The items presented in this experiment were constrained to the feature of animacy only and the instruction participants
1
4.2 Materials
The experiment consisted on 72 trials of experimental and filler items in the implicit and search phase and 36 experimental items in the explicit phase. Experimental items were simple sentences with transitive verbs including a subject and an object introduced with the demonstrative este ‘this’. Filler trials were main sentences with the presentative construction esto es un ‘this is a’ in Spanish.
4.3 Procedure
The experiment was conducted in a phonetics laboratory with no acoustic disruptions. The materials were presented using E-prime software on a computer screen (Tobii), placed 50 to 60cm away from the participant's eyes. Participants listened to the experimental items while looking at two images on the screen. The eye-tracker Tobii 120 was calibrated for every participant prior to the experiment.
All participants listened to a short instruction of the test after and were offered the chance to ask questions to the experimenter. Participants were explained to press two keys (z or m) that matched the correct image. The experiment consisted of three phases: implicit, search and explicit. The implicit and search phases consisted on randomized experimental and filler trials, and the explicit phase contained experimental trials only. The implicit phase was introduced first and it was followed by an instruction that requested participants to reflect about grammar. The search phase was meant to awareness about the purpose of the experiment. Immediately after, participants listened to the explicit instruction of the DOM rule and then performed the explicit phase of the experiment. Instructions were given in Dutch for the Dutch group and in Spanish for the Spanish group.
4.4 Participants
Prior to the experiment, all participants completed a questionnaire that contained information about their vision, hearing and linguistic knowledge. It was important that participants in the Dutch group did not speak a third language in which the DOM feature exists to assure they would not have implicit knowledge about the rule.
4.4.1 Spanish participants
The Spanish group consisted of 20 native speakers of Spanish (12 female, 8 male) who were originally from Spain, Mexico, Chile and Argentina. They were between 20 and 34 years old (mean age: 26.1) and all of them had normal hearing and vision. All of these participants spoke English at intermediate to advanced level and some of them spoke Dutch as well. The first five participants tested in this group performed the two phase (implicit-explicit) experiment, before it was implemented with the intermediate 'search' phase.
4.4.2 Dutch participants
The Dutch group included 22 native speakers of Dutch (15 female, 7 male) who were born in The Netherlands (only one of them was born in Belgium). They were between 22 and 38 (mean age: 23.4) years old and had normal hearing and vision . The first nine participants of this group 2
performed the two-phase (implicit-explicit) experiment, before it was implemented. All participants spoke English fluently and some spoke other languages such as French, German or Italian, but none of them spoke a language with Differential Object Marking. Approximately a third of the participants in this group were or had been students of Spanish at a Bachelor's level or had studied the language in a university context (a minor, as a second language, etc.). All participants were paid 10 Euros for their participation in the experiment.
5. Analysis
5.1 Eye-tracking data analysis
Participants' gaze preferences were needed to perform the statistical analysis. These were obtained by subtracting participants’ eye fixations towards the non-target areas (incorrect image and other parts of the screen) from the fixations towards the correct image. Gaze preferences were obtained for every participant for each trial and phase. The relevant phases were: (1) gaze preferences from the onset of the sentence to the onset of the preposition a in the animate trials and the demonstrative
este in the inanimate trials ;(2) gaze preferences from the onset of the preposition a to the onset of
Participant 911 had strabismus but she did not report this in the questionnaire previous the appointment
2
to the experiment. It was difficult to calibrate the eye-tracker with this participant and she might have to be excluded.
the noun in the animate trials, and from the end of the demonstrative (este/esta) to the onset of the target noun, which disambiguated the target picture; (3) gaze preferences from the onset of the target noun to the end of the sentence. Gaze data outside these critical regions were dismissed. In this paper, we only focus on the gaze preferences from phase 2, which corresponds to the critical region or area of interest.
Data was aggregated by subject and analyzed with a linear mixed-effect lmer model in R. Fixed effects were option status (levels: same vs. different), ot1 and ot2. These factor were modelled to explore possible interactions. Eye-tracking data were split into the two conditions
animate and inanimate for the analysis and a different model was run for each test condition. Subject and Test item (ID) were set as random effects.
5.2 RT analysis
Reaction time (RT) analysis was run on experimental items in both groups of participants. In the mixed effect model Trial ID and Subject were modelled as random effects, whereas fixed effects were Option status (levels: same vs. different), Group (levels: Spanish native speakers vs. Dutch native speakers) and Test condition (levels: implicit, explicit and search).
5.3 Debriefing and awareness analysis
Awareness data was collected through the debriefing following the experiment. This data included participants’ perception of their awareness and ability to predict during the implicit and explicit phases of the experiment, as well as their knowledge of the rule prior to its explicit instruction. This information was coded in the data as binary factors aware implicit, aware explicit, knew the rule
report prediction, but were not included in the mixed effect model. In the case of the Dutch
participants, vocabulary scores were also incorporated to the dataset.
6. Results
For the purpose of this paper, the results described in this section focus on the animate conditions only, although the study of the inanimate conditions is necessary for the reliability of our results.
6.1 Eye-tracking results
For the proper analysis of gaze preferences, participants with more than 20% of missing gaze in phase 2 (critical region) were excluded (participant 920, 701 and 710). Track loss was analyzed and trials with more than 50% track loss were removed. The remaining trials were aggregated by subject across the response window.
For the Dutch group, the mixed effects model showed a main effect for option status in the explicit phase (p = 0.05) but not in the implicit phase (p = 0.08), meaning that participants showed different looking behaviors when the target items had the same or different animacy. In particular, participants looked more to the target animate image than to the inanimate one in the explicit phase only. This is shown in Figure 1 (b) by an increase followed by a decrease in the proportion of looks to the correct animate image, from which prediction due to the presence of the preposition can be interpreted. In the explicit phase, an interaction between option status and the quadratic time vector (ot2) was found: looks overtime follow a decreasing slope of ot2 in the implicit phase and an increasing slope of ot2 in the explicit phase. This interaction means that participants’ preference to look at the correct animate image was greater in the trials were items had different animacy. In the implicit phase, there was a three-way interaction between the quadratic vector, option status and the linear vector and option status and the quadratic vector, as shown in Table 1 and Figure 1 (a): while looks overtime fit the linear vector (ot1) for option status same, they fit the quadratic vector (ot2) for option status different.
Table 1. Fixed effects for animate trials on phase 2 for the Dutch group
Implicit Explicit Pr(Chi) Pr(Chi) OptionsAStatusC 0.076979 . 0.05208 . ot1 0.061595 . 0.96127 ot2 0.000106 *** 0.76425 OptionsAStatusC:ot1 3.286e-07 *** 0.19226 OptionsAStatusC:ot2 6.226e-05 *** < 2e-16 ***
! ! (a) Phase: implicit (b) Phase: explicit
Figure 1. Proportion of looks overtime of the Dutch speakers to the correct image in the animate condition
The data from the Spanish group revealed a main effect for option status in both the implicit (p=0.05) and the explicit (p= 0.05) conditions, which means that participants exhibited different looking behaviors towards the correct image in the two conditions: same and different. Specifically, these participants looked more to the target animate image than to the inanimate when the target items had different animacy. It follows that the Spanish group was able to predict on the basis of DOM in both the implicit and the explicit phases. This is shown in Figure 2 by the gradual increase of looks overtime towards the correct image in both implicit and explicit conditions when items had different animacy. Contrary to the Dutch group, we found main effects for the linear (ot1) and the quadratic vectors (ot2) for the Spanish groups, in both implicit and explicit conditions: looks overtime fit the linear vector (ot1) in both implicit and explicit phases when option status was
different; while they fit the quadratic vector (ot2) in both implicit and explicit phases when option status was same. Similar to the results found for the Dutch group, the analysis of the Spanish group
revealed a positive interaction between option status and the quadratic time vector (ot2), in both
same and different conditions (see the decreasing slope of ot2 in Figures 2 (a) and 2 (b). Figures 1
and 2 show that Spanish and Dutch native speakers exhibit different behaviors in their processing of DOM, even though the eye-tracking data of the Dutch group may be interpreted as prediction.
Table 2. Fixed effects for animate trials on phase 2 for the Spanish group
! !
(a) Phase: implicit (b) Phase: explicit
Figure 2. Proportion of looks of the Spanish speakers to correct image in the animate condition
6.2 RT results
Reaction time analysis was run in order to evaluate the interaction between option status, group and
test condition. If participants used the presence of DOM as a cue for choosing the correct image, a
significant effect should be found for option status, i.e. animacy of the object would prompt faster responses; test condition i.e. faster responses in trials with disambiguating animacy; group i.e. if Spanish native speakers showed an advantage over Dutch speakers. The analysis showed no significant main effects of option status (p= 3.1), group (p=0.83) nor test condition (p=5.2). Neither did option status, test condition or group revealed an interaction (p=1.7). These results mean that participants did not show a speed advantage in the processing of DOM in the presence of the preposition a in neither condition, regardless of whether they belonged to the Dutch or the Spanish group.
Implicit Explicit
Pr(Chi) Pr(Chi) OptionsAStatusC 0.0502095 . 0.0502095 . ot1 7.923e-14 *** 7.923e-14 *** ot2 0.0012485 ** 0.0012485 ** OptionsAStatusC:ot1 0.1283688 0.1283688 OptionsAStatusC:ot2 0.0005099 *** 0.0005099 ***
6.3 Switch analysis
A switch analysis was carried to evaluate if, and if so, at what point in time, participants switched to the correct picture (AOI). The goal is to identify whether participants are more quick at switching to the target image depending on their awareness, and or whether they are native or L2 speakers. The switch analysis was carried for the implicit and explicit conditions. Furthermore, we studied the consistency of the switches to the correct image over time. The main effect reveals that participants were more likely to switch to the target image than away from it when the proposition ‘a’ was used in the disambiguation trials, namely, when only one of the images was animate. Via switch analysis, all participants show this behavior regardless the language group in both implicit and explicit phases. In the implicit phase the effect is significant from frame 54 for Spanish speakers and from frame 80 for Dutch speakers. In the explicit phase, the difference reaches significance in earlier frames; at frame 38 for Spanish speakers and 14 for Dutch speakers. The fact that Dutch speakers show fairly faster reaction than Spanish speakers is a surprising finding that will be addressed in the discussion. The importance of the significant differences is that they all occur before frame 90, as shown in figure X and Y, before he onset of the noun, which provides evidence of prediction based on the DOM rule.
Table 3. Time frame at which switch effect is significant.
(a) Phase: implicit (b) Phase: explicit
Figure 3. Proportion of looks of the Spanish speakers to correct image in the animate condition
Implicit Explicit
Spanish group 54 ms 80 ms
Dutch group 38 ms 14 ms
A validation study: Is visual world eye-tracking suitable
for studying implicit learning?
Why visual world eye-tracking?
►Technique is used to demonstrate that language users can predict upcoming information
on the basis of some linguistic cue
► Prediction occurs within a few hundred milliseconds, which suggests automatic and
potentially implicit processing. Therefore, prediction may be evidence that L2 learners have
learned particular L2 structures up to a point that little or no conscious effort is required to
process them (Godfroid & Winke 2015; Morgan-Short, Faretta-Stutenberg, & Bartlett-Hsu,
2015 ).
► The attraction of the technique: it does not require ungrammatical items to assess L2
learners’ knowledge, which have been claimed to trigger awareness and to affect the learning
processes (e.g., Rebuschat et al 2015).
RQ: Can prediction be the result of implicit, unaware processes?
Sible Andringa &
Maja Curcic
EUROSLA
Reading, August 2017
La mujer mira
a este
perro.
‘The woman looks at the dog.’
Paradigm
predictive looks (during ‘al ese’) to the correct image
whenever the presence/absence of the DOM-marker
is predictive
20 Spanish
L1 speakers
20 Dutch
learners of
Spanish
Test 1 –eye-tracking when
unaware
No instruction
Test 2 – eye-tracking when
aware
Test 3 – Grammaticality
judgement & debriefing
Instruction: Apply DOM!
Design
Within-participant design to learn how
presence/absence of awareness affects eye
movement behaviour
Target structure
Differential Object Marking (DOM) in Spanish:
Animate object is preceded by preposition,
inanimate object is not.
• La mujer vio
a
esta niña.
• La mujer vio esta casa.
Dutch does not make an animacy distinction
Results: Growth analysis, animate trials only, before noun onset
Implicit phase
Explicit phase
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Results: Switch analysis, animate trials only
Implicit phase
Explicit phase
without rule knowledge
with rule knowledge
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Discussion
• L1 speakers predict when unaware, but
awareness leads to more and earlier predictions.
• Switch analysis suggests that L2 learners also
predict when unaware; the effect seems driven by
learners who remembered once being taught the
rule
• L2 learners show clear prediction when aware:
Strategic?
• DOM-based prediction is not necessarily, but can
be the result of implicit, unaware processes in L1
speakers; slight evidence for this in L2 speakers
• In this paradigm, debriefing is important to
establish if participants were aware.
In all cases
there is an
interaction
between
development
over time and
trial type
Significant difference from frame 54 Significant difference from frame 38 Significant difference from frame 80 Significant difference from frame 14 to 86A validation study: Is visual world eye-tracking suitable
for studying implicit learning?
Why visual world eye-tracking?
►Technique is used to demonstrate that language users can predict upcoming information
on the basis of some linguistic cue
► Prediction occurs within a few hundred milliseconds, which suggests automatic and
potentially implicit processing. Therefore, prediction may be evidence that L2 learners have
learned particular L2 structures up to a point that little or no conscious effort is required to
process them (Godfroid & Winke 2015; Morgan-Short, Faretta-Stutenberg, & Bartlett-Hsu,
2015 ).
► The attraction of the technique: it does not require ungrammatical items to assess L2
learners’ knowledge, which have been claimed to trigger awareness and to affect the learning
processes (e.g., Rebuschat et al 2015).
RQ: Can prediction be the result of implicit, unaware processes?
Sible Andringa &
Maja Curcic
EUROSLA
Reading, August 2017
La mujer mira
a este
perro.
‘The woman looks at the dog.’
Paradigm
predictive looks (during ‘al ese’) to the correct image
whenever the presence/absence of the DOM-marker
is predictive
20 Spanish
L1 speakers
20 Dutch
learners of
Spanish
Test 1 –eye-tracking when
unaware
No instruction
Test 2 – eye-tracking when
aware
Test 3 – Grammaticality
judgement & debriefing
Instruction: Apply DOM!
Design
Within-participant design to learn how
presence/absence of awareness affects eye
movement behaviour
Target structure
Differential Object Marking (DOM) in Spanish:
Animate object is preceded by preposition,
inanimate object is not.
• La mujer vio
a
esta niña.
• La mujer vio esta casa.
Dutch does not make an animacy distinction
Results: Growth analysis, animate trials only, before noun onset
Implicit phase
Explicit phase
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Results: Switch analysis, animate trials only
Implicit phase
Explicit phase
without rule knowledge
with rule knowledge
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Discussion
• L1 speakers predict when unaware, but
awareness leads to more and earlier predictions.
• Switch analysis suggests that L2 learners also
predict when unaware; the effect seems driven by
learners who remembered once being taught the
rule
• L2 learners show clear prediction when aware:
Strategic?
• DOM-based prediction is not necessarily, but can
be the result of implicit, unaware processes in L1
speakers; slight evidence for this in L2 speakers
• In this paradigm, debriefing is important to
establish if participants were aware.
In all cases
there is an
interaction
between
development
over time and
trial type
Significant difference from frame 54 Significant difference from frame 38 Significant difference from frame 80 Significant difference from frame 14 to 86(a) Phase: implicit (b) Phase: explicit
Figure 4. Proportion of looks of the Dutch speakers to correct image in the animate condition
6.4 Debriefing results
Participants were debriefed between the implicit and explicit phases and after the experiment. After the implicit phase, participants had to indicate whether they knew the purpose of the experiment or not. Once the experiment was completed, they were questioned about their impressions of the test. The purpose of debriefing participants was to identify awareness of the DOM rule at any point of the experiment. This data was taken into account for the evaluation of the eye-tracking analysis.
6.4.1 Spanish group
Debriefing of the Spanish group showed that participants were not aware of the purpose of the experiment and had not noticed the DOM structure during phase 1. Only one participant reported to be aware of this. Most participants thought that the experiment was about demonstratives and gender and some thought it was about how quickly they could relate words and images. Only half of the Spanish participants could tell the purpose of the experiment after it was completed, but when specifically asked about the instruction, most participants reported they had found it helpful and agreed with the rule. This means they had understood the purpose of the experiment even though they could not explain it. One third of the participants said the instruction had sometimes made it more difficult to react to the correct picture because they were trying to answer on the basis of the instruction in the trials where animacy was not disambiguating . As for the other participants, they all found the instruction helpful.
A validation study: Is visual world eye-tracking suitable
for studying implicit learning?
Why visual world eye-tracking?
►Technique is used to demonstrate that language users can predict upcoming information
on the basis of some linguistic cue
► Prediction occurs within a few hundred milliseconds, which suggests automatic and
potentially implicit processing. Therefore, prediction may be evidence that L2 learners have
learned particular L2 structures up to a point that little or no conscious effort is required to
process them (Godfroid & Winke 2015; Morgan-Short, Faretta-Stutenberg, & Bartlett-Hsu,
2015 ).
► The attraction of the technique: it does not require ungrammatical items to assess L2
learners’ knowledge, which have been claimed to trigger awareness and to affect the learning
processes (e.g., Rebuschat et al 2015).
RQ: Can prediction be the result of implicit, unaware processes?
Sible Andringa &
Maja Curcic
EUROSLA
Reading, August 2017
La mujer mira
a este
perro.
‘The woman looks at the dog.’
Paradigm
predictive looks (during ‘al ese’) to the correct image
whenever the presence/absence of the DOM-marker
is predictive
20 Spanish
L1 speakers
20 Dutch
learners of
Spanish
Test 1 –eye-tracking when
unaware
No instruction
Test 2 – eye-tracking when
aware
Test 3 – Grammaticality
judgement & debriefing
Instruction: Apply DOM!
Design
Within-participant design to learn how
presence/absence of awareness affects eye
movement behaviour
Target structure
Differential Object Marking (DOM) in Spanish:
Animate object is preceded by preposition,
inanimate object is not.
• La mujer vio
a
esta niña.
• La mujer vio esta casa.
Dutch does not make an animacy distinction
Results: Growth analysis, animate trials only, before noun onset
Implicit phase
Explicit phase
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Results: Switch analysis, animate trials only
Implicit phase
Explicit phase
without rule knowledge
with rule knowledge
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Discussion
• L1 speakers predict when unaware, but
awareness leads to more and earlier predictions.
• Switch analysis suggests that L2 learners also
predict when unaware; the effect seems driven by
learners who remembered once being taught the
rule
• L2 learners show clear prediction when aware:
Strategic?
• DOM-based prediction is not necessarily, but can
be the result of implicit, unaware processes in L1
speakers; slight evidence for this in L2 speakers
• In this paradigm, debriefing is important to
establish if participants were aware.
In all cases
there is an
interaction
between
development
over time and
trial type
Significant difference from frame 54 Significant difference from frame 38 Significant difference from frame 80 Significant difference from frame 14 to 86A validation study: Is visual world eye-tracking suitable
for studying implicit learning?
Why visual world eye-tracking?
►Technique is used to demonstrate that language users can predict upcoming information
on the basis of some linguistic cue
► Prediction occurs within a few hundred milliseconds, which suggests automatic and
potentially implicit processing. Therefore, prediction may be evidence that L2 learners have
learned particular L2 structures up to a point that little or no conscious effort is required to
process them (Godfroid & Winke 2015; Morgan-Short, Faretta-Stutenberg, & Bartlett-Hsu,
2015 ).
► The attraction of the technique: it does not require ungrammatical items to assess L2
learners’ knowledge, which have been claimed to trigger awareness and to affect the learning
processes (e.g., Rebuschat et al 2015).
RQ: Can prediction be the result of implicit, unaware processes?
Sible Andringa &
Maja Curcic
EUROSLA
Reading, August 2017
La mujer mira
a este
perro.
‘The woman looks at the dog.’
Paradigm
predictive looks (during ‘al ese’) to the correct image
whenever the presence/absence of the DOM-marker
is predictive
20 Spanish
L1 speakers
20 Dutch
learners of
Spanish
Test 1 –eye-tracking when
unaware
No instruction
Test 2 – eye-tracking when
aware
Test 3 – Grammaticality
judgement & debriefing
Instruction: Apply DOM!
Design
Within-participant design to learn how
presence/absence of awareness affects eye
movement behaviour
Target structure
Differential Object Marking (DOM) in Spanish:
Animate object is preceded by preposition,
inanimate object is not.
• La mujer vio
a
esta niña.
• La mujer vio esta casa.
Dutch does not make an animacy distinction
Results: Growth analysis, animate trials only, before noun onset
Implicit phase
Explicit phase
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Results: Switch analysis, animate trials only
Implicit phase
Explicit phase
without rule knowledge
with rule knowledge
Spanish
L1
speakers
Dutch L2
speakers
of
Spanish
Discussion
• L1 speakers predict when unaware, but
awareness leads to more and earlier predictions.
• Switch analysis suggests that L2 learners also
predict when unaware; the effect seems driven by
learners who remembered once being taught the
rule
• L2 learners show clear prediction when aware:
Strategic?
• DOM-based prediction is not necessarily, but can
be the result of implicit, unaware processes in L1
speakers; slight evidence for this in L2 speakers
• In this paradigm, debriefing is important to
establish if participants were aware.
In all cases
there is an
interaction
between
development
over time and
trial type
Significant difference from frame 54 Significant difference from frame 38 Significant difference from frame 80 Significant difference from frame 14 to 86The rule about the use of the preposition with animate direct objects had been explicitly taught to all European Spanish speakers either in primary or secondary school. As for the speakers who spoke American varieties of Spanish, most of them had not received explicit instruction but they generally agreed with the rule. All participants of this group except for four reported they had used the rule and had been able to predict the correct image on the basis of DOM.
6.4.2 Dutch group
The debriefing of the Dutch group differed from that of the Spanish group. Firstly, these participants had different levels of proficiency (advanced and intermediate). However this did not seem to determine patterns in their responses to the questionnaire. Similarly to the Spanish group, only two participants knew what the experiment was about after phase 1, but the majority of them understood the purpose of the experiment after its completion, unlike half of the Spanish participants. Hence, the level of awareness, as reported by the participants, was higher in the Dutch group than in the Spanish group. This group generally found the instruction helpful to perform the explicit phase of the experiment. Only five participants reported it had made it more difficult because they were thinking about the rule or because they answered too quickly when hearing the preposition, causing them to make mistakes.
Only eight participants had received explicit instruction about the rule, mostly those who had studied Spanish in university or secondary school. These participants had learned the language in an explicit and more formal environment, contexts that predominantly teaches European Spanish. 3
Dutch participants, except for three, also reported they had been able to predict the correct image during the explicit phase on the basis of the preposition. This differentiates greatly the Dutch group from the Spanish group, and it is even noticed in the faster reaction times of the Dutch participants in the explicit phase. We will further address this in the discussion.
6.5 Vocabulary test
The Dutch participants performed the vocabulary test DELE after the experiment. The test consisted of a multiple choice task and a fill in the text task. The test took between fifteen to
In Spain, students in both primary and secondary school receive very explicit instruction about the 3
five minutes depending on the proficiency of the speaker. Participants could score advanced (39-50), intermediate (30-39) or low (< 29), however, the former two categories were combined under intermediate as the individual scores of the participants that scored low did not differ considerably those of the intermediate group. Of the total of twenty two Dutch participants, fifteen were advanced speakers. Eight participants encountered difficulties with one or more words during the experiment. This does not seem to correlate with the level of proficiency but with particular items. The words carro 'car' and lápiz labial 'lipstick' were unknown or sounded funny to three participants. The words bocadillo 'sandwich' and tenedor 'fork' were problematic for two different participants. One speaker reported some words were not familiar for him at first but he was able to remember them once the pictures appear.
7. Discussion
In this study, we investigated 1) to what extent prediction in language processing is an implicit measure of knowledge, and 2) how awareness affects predictive behavior. We used eye-tracking measures that tested whether the awareness of the DOM rule in Spanish influenced its implicit and explicit processing in native as well as Dutch L2 speakers of Spanish. On the assumption that Spanish native speakers are able to predict the presence of an animate object on the basis of the preposition a (Curcic et al., 2014; Andringa & Curcic, 2015), we expected that the Spanish group would be able to predict regardless of metalinguistic information, thus, in both implicit and explicit phases of this experiment. Our results confirmed our expectations: Spanish native speakers showed prediction when they heard the preposition in trials were animacy of the target items was different. However, verbal reports do not indicate overall awareness of the rule during the experiment in Spanish speakers, but they were all taught the rule explicitly during school. On the contrary, most Dutch participants indicated awareness of DOM in the explicit phase, and some had been taught the rule in Spanish as a foreign language class.
Spanish speakers were able to predict when unaware, but awareness, as reported in the debriefing data, led to faster prediction. Dutch native speakers were not expected to predict the correct animate object in the implicit phase of the experiment. Our first analysis confirmed this expectation. However, further evaluation carrying out a switch analysis showed that Dutch speakers predict in the implicit phase when they have some knowledge of the rule, particularly, when they
were taught the rule at some point. We interpret this knowledge as evidence for awareness of DOM in this experiment.
Participants’ debriefing is crucial in the interpretation of our eye-tracking results. Following Godfroid & Winke (2015), the combination of online data and debriefing data is a essential to understand speakers processing of a language, specially when is a second language. It is important to address that awareness was assessed previous to and after the experiment, as suggested by Leow (2001), who claims that learners should be assumed to be aware unless proven the opposite. By debriefing participants prior to, during and after the experiment, we are more likely to accurately asses their awareness. Interestingly, awareness of the rule was still limited after its explicit instruction, according to participants’ report. A larger number of Dutch participants than Spanish participants established they were aware of the rule in the explicit phase. However, both groups showed DOM-related prediction in the explicit phase. It is important to mention that although both groups showed a significant effect for DOM prediction in this second phase, reactions were considerably faster for the Dutch speakers than the for the Spanish speakers. The experimenter reported that this difference was tangible during the experiment, as Dutch speakers were substantially faster at pressing the keys when they heard the preposition (e.g.: they barely waited for the onset of the noun). This was not the case for the Spanish speakers.
The rapidity of Dutch speakers’ reactions deserves some reflection. Two possibilities are considered. Firstly, it could be the case that Spanish speakers were indeed unaware of the DOM rule as self reports indicate, and their ability to predict does not depend on awareness but on other factors. This could also be interpreted as a sign that awareness may lead to faster reactions, if we consider that Dutch speakers who reported to be aware in the explicit phase were faster than Spanish participants, who generally claimed to be unaware. It has been suggested that prediction depends on multiple factors such as literacy (Huettig and Mishra, 2014) or even individual cognitive differences (Tanner & van Hell, 2014). A second possibility is that Dutch speakers are using some kind of strategic behavior, perhaps enhanced by the fact that they learned the language in more explicit contexts and are better able to make use of explicit (metalinguistic) information when provided. There is evidence of L2 speakers being benefited in explicit tasks compared to, for instance, bilingual heritage speakers, who mostly learn the language in naturalistic contexts. While the two possibilities are not mutually exclusive, we are inclined to support the second, for which we believe to have more evidence based on our experiment. These results tell us something about the relation between awareness and prediction. It seems that prediction comes from different sources for the two group of speakers. It is clear that prediction is a resource that native speakers use
overall. However, there seems to be an important role of explicit awareness for the Dutch speakers that are able to predict. It could be that the metalinguistic information provided during the experiment causes Dutch speakers to rely on less linguistic information than Spanish speakers might do, increasing their reaction response compare to native speakers.
This bring us to Huettig’s (2015) arguments about the necessity of prediction in language processing. It seems that our results could be interpreted as further evidence that prediction is not needed for language processing, but it certainly can be used by some speakers under suitable circumstances. One of them could be certain level of awareness. While this does not seem crucial for native speakers, it appears to beneficial for L2 speakers who show some level of non-reported awareness of a linguistic rule they once learned. In the case of Spanish speakers, their awareness of this rule could perhaps be supported by having been taught this linguistic rule in primary school. As we know, DOM is something that Spanish native speakers are explicitly taught during childhood. We must acknowledge that these results raise some questions about the understanding of awareness in the literature, namely that being aware is some sort of dichotomous state, unlike the process of learning, which belongs to a continuum (Hama & Leow, 2010). It may be worth considering whether awareness can also be placed in a continuum. The fact that speakers who remembered being taught the rule were able to predict despite self-report unawareness suggests that the assimilation of linguistic rules does not mean direct awareness of it. Something that is clear, as suggested by Ellis, is that verbal reports might not be the perfect measure of awareness. Nevertheless, we believe that thorough debriefing of participants, in combination with online measures of language, gives us fair insights of speakers knowledge of language. Further research should make an effort to debrief participants in great detail, as we believe can help research understand speakers’ linguistic processes.
To this respect, our results might tell us something about how awareness affects predictive behavior, and whether prediction can be a sign of implicit processing. This might indeed be the case, considering that native speakers used prediction in both phases of the experiment despite reporting not being aware, and so did some L2 speakers. In any case, our results show that being aware of a linguistic rule improves the rapidity of reaction to linguistic stimuli.
These results raise some crucial questions about the relation between awareness and implicit based on self-reports. If we consider participants’ ability to report their level of awareness a valid assessment, we may wonder whether the implicit and explicit phases in our experiment truly targeted implicit and explicit sources of knowledge respectively. Importantly, the main difference between Spanish and Dutch native speakers was the incapability of the latter to predict in both
phases of the experiment, which correlates with the expectations for implicit and explicit learning abilities of both groups. Nevertheless, the fact that most Spanish speakers still reported to be unaware of this rule in the explicit phase leads to the question whether explicit instruction of the rule truly prompted explicit knowledge. Furthermore, the Dutch native speakers only showed prediction once they were aware of the rule, hence there are crucial differences between native speakers and second language speakers in relation to awareness and DOM-related prediction.
Moreover, it is important to consider whether the non-naturalistic environment of the experiment made it possible for participants to truly be unaware, even though they reported to be. This taps into Ellis’ (2004) description of explicit knowledge and metalanguage. The fact that participants were not able to verbalize their linguistic knowledge, does not exclude the possibility that they were indeed aware. It is important to stress that not only the conditions (experimental vs naturalistic, availability of enough cues, etc.) are key for prediction to occur. It may very well be the case that only some speakers, regardless of whether it is their first or second language, are more likely to use prediction skills. Our debriefing data could be interpreted as evidence for this, as only when carefully evaluating individual responses could we relate certain levels of awareness to prediction abilities.
Overall, we can argue that our results showed important differences in the linguistic behavior of native and second language speakers in relation to the processing of DOM, but also crucial differences after explicit instruction of a grammatical rule. Our study has further investigated the relation between speakers’ self-reported awareness and their processing of grammatical rules in implicit and explicit knowledge. This study also raises questions for further research as the distinction between implicit and explicit knowledge in relation to speakers’ awareness is not yet clear. Ultimately, if metalanguage might not unambiguously exhibit speaker awareness, implemented tests might be able to properly asses awareness.
8. Concluding remarks
This study investigated the effects of metalinguistic information in the online L1 and L2 processing of Differential Object Marking (DOM). This research also addresses the validity of eye-tracking as measure to tap into implicit language processing. Generally, our results show that Spanish native speakers are able to predict the presence of an animate object through grammatical cues, and so are advanced Dutch language learners of Spanish after explicit instruction of the grammatical rule. The retrospective reports indicate that Spanish native speakers are less aware of the linguistic structure
