Do presupposition triggers influence the felicity of voice mismatched sentences?
Margreet Vogelzang Bachelor’s thesis
s1648497 M.Vogelzang@ai.rug.nl Artificial Intelligence department School of Computing and Cognition
University of Groningen Advisor: Jennifer Spenader
Abstract
Kehler (2002) claims that sentences with resemblance relations prefer parallel syntactic structures, while for cause-effect relations this is irrelevant, thus voice mismatches are dispreferred in resemblance relations.
Adding a presupposition trigger like too might help re- pair the damage made by the mismatch, making such sentences more felicitous.
In a sentences judgment task experiment, we asked Dutch speakers to judge the felicity of resemblance and cause-effect sentences with and without syntactic voice mismatches. An additional factor is the presence or ab- sence of the presupposition triggers too (ook) in resem- blance relations and toch in cause-effect relations.
The results show that adding a presupposition trigger doesn’t affect matched resemblance sentences, but sig- nificantly improves mismatched resemblance sentences.
Adding a presupposition trigger also doesn’t signifi- cantly improve matched or mismatched cause-effect sen- tences. Both resemblance and cause-effect sentences are not significantly worsened by mismatch, so our results can’t be said to support Kehler’s theory for Dutch sen- tences.
Keywords: presupposition triggers, anaphors, coher- ence relations, pronoun interpretation
1 Introduction
Presupposition triggers are words that trigger an implicit assumption about the world, generally treated as background information, and whose meaning is unaffected by logical operators. Examples are words like too and again; an example senctence is shown in (1).
(1) John got cake with his coffee again.
In this sentence, the presupposition trigger again gives you the presupposition that at some time in the past John got cake with his coffee too, at least once. Some presupposition triggers can be bound anaphorically, like again in sentence (2).
(2) John got cake with his coffee on Thursday and he got cake with his coffee again today.
In this case, the second part of the sentence states
something which also happened before, and is mentioned in the first part of the sentence. The pre- suppiosition trigger therefore makes an explicit link between the clauses and so the relationship between the two parts of the sentence is emphasized.
Little research has been done however on the in- fluence of presupposition triggers on different types of sentences. An experiment of Arregui et al. (2006) shows that sentences containing verb phrase ellipsis with ‘flawed’ antecedents can be improved by adding a presupposition trigger.
Different presupposition triggers are appropriate for different types of sentences. The type of relationship between two clauses is called the type of coherence relation or the coherence type. Hume (1748) distin- guishes three types of coherence relations, naming resemblance, cause-effect and contiguity in time or place.
Sentences (1) and (2) are both sentences with a resem- blance relation, since there are similar events described in both clauses of the sentence. In this experiment only two of the coherence relations will be taken into account; the resemblance and the cause-effect relation.
Examples of these coherence relations are stated in sentence (3)-(6) [(3) and (4) as given by Kehler et al.
(2008) p.12, example (11)].
(3) Samuel threatened Justin with a knife, and Erin blindfolded him. [resemblance]
(4) Samuel threatened Justin with a knife, and Erin stopped him. [cause-effect]
(5) Justin was threatened by Samuel with a knife, and Erin blindfolded him. [resemblance]
(6) Justin was threatened by Samuel with a knife, and Erin stopped him. [cause-effect]
Noticable is the use of a pronoun in all the ex- amples. The pronoun in the second clause of each sentence refers to a noun in the first clause. The pronoun is therefore another way to make sure the reader links the second clause back to the first one; to know what the pronouns antecedent is.
Theoretically, the pronouns in sentence (3)-(6) could
refer to both Samuel and Justin, as they are both male.
People have a certain way of interpreting the pronoun in the different sentences though, that depends on a variety of features of the sentence, e.g. the grammatical role of the pronoun, the coherence relation between the two clauses or the context in which the sentence places the pronoun.
Kehler (2002) and Kehler et al. (2008) have argued that the type of coherence relation between two clauses influences pronoun interpretation. They claim pro- nouns in resemblance relations such as (3) have differ- ent interpretation preferences than pronouns in cause- effect relations like (4).
Simple rules to explain this difference in interpreta- tion preference are not easily formulated. E.g., a possi- ble heuristic is to prefer to interpret pronouns as hav- ing an antecedent of the same grammatical role as the pronoun. But this heuristic can’t explain the difference in interpretation as the pronoun carries the same gram- matical role in sentence (3) and (4), but in sentence (3) the pronoun refers to the object of the first clause (Justin) while in (4) it refers to the subject (Samuel).
According to Kehler et al. (2008), simple strategies for pronoun interpretation that say e.g. object pro- nouns prefer object antecedents, work for resemblance relations like (3), but are irrelevant for cause-effect relations like (4), and so preferences for pronoun inter- pretation can be changed by changing the coherence relation (confirmed by Wolf et al. (2004)). In sentences with a resemblance relation a parallel structure is pre- ferred, and that explains why mismatched sentences like (5), in which the voice of the two clauses doesn’t match, sound less felicitous than sentences like (3), in which both clauses are active. Also, a more felicitous sentence than (5) would be (7), because both clauses are passive.
(7) Justin was threatened by Samuel with a knife, and he was blindfolded by Erin. [resemblance]
An explanation for the loss in felicity in sentences with a resemblance relation when there is no parallel syntactic structure could be that it makes the anaphor interpretation difficult because simple strategies are no longer possible.
In a study testing Kehler’s theories with verb phrase ellipsis, Frazier and Clifton Jr. (2006) unexpectedly found that people disliked syntactic mismatches re- gardless of whether they appeared in resemblance or cause-effect sentences. The difference between these experiments was, as noted by Frazier and Clifton Jr.
(2006), that their sentences with a resemblance rela- tion didn’t have the presupposition trigger too in them, while most of Kehlers sentences did. This however would not explain the difference in results in the cause-
effect sentences, as both Kehler et al. (2008) and Frazier and Clifton Jr. (2006) didn’t use a presupposition trig- ger.
As mentioned before, Arregui et al. (2006) found that sentences containing verb phrase ellipsis with
‘flawed’ antecedents are interpreted as better when a presupposition trigger is present. Also, Hendriks (2004) suggested that the presence of a presupposition trigger like too might help repair the damage made by the voice mismatch, making such sentences more felicitous. The presence of a presupposition trigger could give the hearer an alternative way to recognize how the events occuring in the clauses are linked. An example of a mismatched sentence with a presupposi- tion trigger is (8).
(8) Justin was threatened by Samuel with a knife, and Erin blindfolded him too. [resemblance]
In a sentence judgment task experiment, we ask Dutch speakers to judge the felicity of resemblance and cause-effect sentences with and without voice mismatches. An additional factor is the presence or absence of the presupposition trigger too (in Dutch ook) and the Dutch presupposition trigger toch. Testing in Dutch is particularly useful because the Dutch trigger toch marks denial of expectation, a type of cause-effect relation, and a presupposition trigger to mark such a relation doesn’t exist in English.
Like the sentences used in the experiment by Kehler et al. (2008) and Frazier and Clifton Jr. (2006) our test sentences contain an anaphoric device in the second clause to force the reader to make an explicit link be- tween the two clauses (Frazier and Clifton Jr. (2006) used verb phrase ellipsis, Kehler et al. (2008) used pro- nouns). Our sentences contain an object pronoun so we can study how the presence of the presupposition trigger influences the pronoun interpretation and the felicity of the sentence.
In this way we can test if, as Frazier and Clifton Jr.
(2006) found, mismatched sentences are infelicitious regardless of the type of coherence realtion, or if, as Kehler et al. (2008) state, only the mismatched re- semblance sentences are infelicitious and mismatching doesn’t influence the felicity of cause-effect sentences.
We can also check if presupposition triggers can repair the damage done by the mismatch.
Our prediction is that the presence of the presuppo- sition trigger will improve the mismatched sentences because it offers listeners an alternative means for in- terpreting the coherence relation. Also, our expecta- tion is that mismatch will degrade sentences with re- semblance relations but will not affect sentences with cause-effect relations, because only in sentences with resemblance relations a parallel structure is preferred.
2 Experiment 1:
Pronoun interpretation preference
2.1 IntroductionAs abovementioned, all sentences can be interpreted in two different ways; grammatically, the pronoun could refer to both the subject and the object in the first clause. The sentences were carefully formulated so the context would link the pronoun to the object in the first clause. Experiment 1 was done as a pretest to make sure that in all the experimental sentences the pronoun referred to the object, so that if we asked participants how felicitous a sentence was in experiment 2, all par- ticipants judging the sentence were using the same in- terpretation, in which the pronoun refers to the object.
The participants were presented with the sentences and had to report which noun they thought the pronoun re- ferred to; the subject or the object.
2.2 Participants
20 students and teachers from the University of Groningen participated in this study. All participants were native speakers of Dutch.
2.3 Materials
An example of the eight forms of a sentence used in this experiment, with their translations, are (9)-(16).
(9) Harry bedreigde Lars, en Mieke intimideerde hem. [Harry threatened Lars, and Mieke intimidated him.]
(10) Harry bedreigde Lars, en Mieke intimideerde hem ook. [Harry threatened Lars, and Mieke intimidated him too.]
(11) Harry bedreigde Lars, maar Mieke redde hem.
[Harry threatened Lars, but Mieke saved him.]
(12) Harry bedreigde Lars, maar Mieke redde hem toch. [Harry threatened Lars, but Mieke saved him toch.]
(13) Lars werd door Harry bedreigd, en Mieke in- timideerde hem. [Lars was threatened by Harry, and Mieke intimidated him.]
(14) Lars werd door Harry bedreigd, en Mieke in- timideerde hem ook. [Lars was threatened by Harry, and Mieke intimidated him too.]
(15) Lars werd door Harry bedreigd, maar Mieke redde hem. [Lars was threatened by Harry, but Mieke saved him.]
(16) Lars werd door Harry bedreigd, maar Mieke redde hem toch. [Lars was threatened by Harry, but Mieke saved him toch.]
All sentences consist of two clauses, that are con- nected by a conjunction. In the first clause, the subject and object of a transitive verb are male, referred to by a proper name. The second clause starts with the
conjunction, to indicate the type of coherence relation between the two clauses (resemblance with and (in Dutch en) or cause-effect with but (in Dutch maar)).
This second clause contains a female subject and a male object pronoun, which will refer to one of the male nouns in the first clause. The transitive verb connecting the subject and object in the second clause can either be a synonym (in resemblance relations) or an antonym (in cause-effect relations) of the verb in the first clause.
2.4 Design
The experiment used a 2x2x2 within-subjects de- sign, varying the presence of a presupposition trig- ger, the kind of coherence relation and the voice match/mismatch. This means each sentence had eight different forms, like can be seen in (9)-(16). In total there were 28 different test sentences and 32 fillers (the test sentences can be found in appendix A). Each par- ticipant was shown 60 sentences; one variant of each test sentence and the 32 fillers. This means each partic- ipant saw each test condition two to four times.
For the experiment a total of eight different lists were created, so that every list contained one of the condi- tions of a test sentence. For each participant one list was selected randomly. The list of sentences presented to the participant was pseudo-randomized, so that the test sentences and the fillers were mixed and there were no long sequences of the same experimental condition type. This minimizes possible sequence effects in the lists.
2.5 Procedure
The participants were tested individually in an online test. First they were presented with a list of ques- tions about what languages they spoke, where they had lived and what their native language was. Then they received an instruction saying they should select the name of the person they thought the pronoun was most likely to refer to.
When starting with the actual test, the sentences were shown one by one, with the question ‘What does he/him refer to?’, and the three possible answers, the first two being the names of the subject and object in randomized order and the last one being the option
‘both are possible’. The instructions given before the start of the test said they could only mark ‘both are possible’ if they thought it was just as likely that the pronoun would refer to the subject as to the object.
The participant selected an answer and then pressed the ‘next’ button to go to the next question. All 60 sentences were presented this way. The complete ex- periment took approximately ten minutes.
2.6 Results
Because participants who filled out ‘both are possible’
thought both options were equally likely, those exam- ples may show the same felicity of the sentence in both interpretations. So, what should be checked is if there are sentences that are interpreted only in the ‘wrong’
way (pronoun referring to the subject). In 22 out of the 28 sentences this was the case, but only six sen- tences exeeded a 10% margin (detailed information can be found in appendix B). These sentences will be taken out of the test set; 22 sentences remain to be evaluated as test sentences in experiment 2.
3 Experiment 2:
Judging the felicity
3.1 IntroductionIn the second experiment participants were asked to judge on a Likert scale with seven grades, how felici- tous a sentence sounds. The goal of this experiment is to test whether or not presupposition triggers influence the felicity of the mismatched sentences. We can also check if the influence of mismatching is seen in sen- tences with a resemblance relation and sentences with a cause-effect relation, or only in sentences with one of the two coherence relations.
3.2 Participants
48 students from the Psychology Department at the University of Groningen participated in this study for course credit. All participants were native speakers of Dutch.
3.3 Materials
The type and build of the sentences used in this exper- iment are the same as used in experiment 1.
3.4 Design
The design is the same as that of experiment 1. There were only 22 test sentences used in experiment 2 though, because six sentences were excluded on the ba- sis of the results of experiment 1. All the test sentences used can be found in appendix A.
3.5 Procedure
The participants were tested individually in an online test. First they were presented with a list of ques- tions about what languages they spoke, where they had lived and what their native language was. Then they received an instruction saying they should judge the felicity of a sentence according to how felicitous they though most people would find the sentence. They were given seven options, presented horizontally on a Likert scale with seven grades, running from ‘very bad’
to ‘very good’, with ‘neutral’ in the middle.
When starting with the actual test, the sentences were shown one by one, with the question ‘How felici- tous is this sentence?’, and the seven possible answers.
The participant selected an answer and then pressed the ‘next’ button to go to the next question. 60 sen- tences were presented this way. The complete experi- ment took approximately fifteen minutes.
3.6 Results
Prior to further analysis, seven participants were ex- cluded because they took the test in less than five min- utes; this is not enough time to read and evaluate all 60 sentences seriously. Also, four participants were ex- cluded because their results showed long repeated se- quences of the same answers, which shows they did not take the test seriously. So, after exclusion, 38 par- ticipants remained for analysis.
Because participants were asked to judge the felicity of a sentence on a Likert scale with seven grades, the anwers, ranging from ‘very bad’ to ‘very good’, can be converted to grades from respectively one to seven.
Using the abovementioned grades, a within-subject analysis of variance (ANOVA) was done to evaluate the effects between the three conditions. Significant ef- fects were only found between the presence of a pre- supposition trigger and the kind of coherence relation (F(1, 37) =17.600; p<0.001). Detailed information can be found in appendix C.1.
Figure 1: resemblance relation; matched and mis- matched, with and without presupposition triggers
An ad hoc paired t-test shows that mismatched re- semblance sentences improved significantly by adding a presupposition trigger (t(37) = −5.439; p<0.001). It also shows that the felicity of mismatched cause-effect sentences does not significantly change by adding a presupposition trigger (t(37) =1.761; p=0.087). Check appendix C.3 for detailed information on the outcomes of the t-test. The descriptive statistics can be found in
Figure 2: cause-effect relation; matched and mis- matched, with and without presupposition triggers
appendix C.2. The resulted means are also shown in figure 1 and 2.
Unlike to our expectations, no significant difference is found between matched resemblance sentences and mismatched resemblance sentences without presuppo- sition triggers (t(37) =1.941; p=0.060). The difference in felicity between mismatched resemblance sentences and matched resemblance sentences with presupposi- tion triggers is also not significant (t(37) = −1.899; p= 0.065).
In the cause-effect sentences there is no large effect to be seen between the matched and mismatched con- ditions both with and without a presupposition trigger (resp. t(37) = −1.417; p=0.165 and t(37) = −1.433; p= 0.160). The mismatched conditions were even slightly better than their matched forms (figure 2).
4 General results & discussion
As mentioned before, Kehler (2002) and Kehler et al.
(2008) state that mismatching will decrease the felic- ity of sentences with a resemblance relation, beacuse it makes it harder to correctly interpret the pronoun, but will not affect the felicity of sentences with a cause- effect relation. Our results do not show a significant difference in felicity between matched and mismatched resemblance sentences without the presence of a pre- supposition trigger. Also, no difference is found be- tween matched and mismatched cause-effect sentences, both with and without a presupposition trigger. Be- cause these results are not significant, they can’t be used as evidence for the theory of Kehler (2002) and Kehler et al. (2008), or for the theory of Frazier and Clifton Jr. (2006), who state mismatching will decrease the felicity of both resemblance and cause-effect sen- tences.
What should be mentioned is that in their experi- ment, Kehler et al. (2008) use sentences of the sub-class parallel relations as examples of resemblance relations
and sentences of the sub-class result relations as ex- amples of cause-effect relations, so they check the co- herence relations as stated by Hume (1748) for a more specific category. In our experiment, parallel relations, referred to as resemblance, and denial of expectation relations, referred to as cause-effect, are used in the test sentences. Denial of expectation and result rela- tions are different sub-classes of cause-effect relations, but it could be that sentences belonging to the differ- ent sub-classes have slightly different rules as to what the interpretation preferences of the anaphora are or how felicitous sentences with specific relations natu- rally sound.
It should also be noted that it is hard to extract valid conclusions from these results, since the number of par- ticipants and the number of sentences that remained for the analysis was lower than expected and hoped for. This was partially caused by doing the test online;
it gives participants more freedom to fill out nonsense.
Also, a number of effects that did show were not sig- nificant. With a larger number of sentences and par- ticipants these effects might have been significant, but for now we’ll have to base our conclusions only on the significant effects. Also, some sentences are naturally more felicitous than others, e.g. because some verbs are used more often, and it has proved to be difficult to come up with many good test sentences.
If we also take a look at the difference in size be- tween the effects that were not significant, it is clear the improvement of matched resemblance sentences over mismatched resemblance sentences without a presup- position trigger is larger than the the improvement of matched over mismatched cause-effect sentences with- out a presupposition trigger; matched cause-effect sen- tences were even slightly worse than their mismatched forms. We still can’t draw any definite conclusions from these results, as they are not significant, but it can be said that the results do gravitate towards sup- porting Kehler’s theory more than towards supporting Frazier and Clifton’s theory.
Furthermore, the results show that matched sen- tences with a resemblance relation are hardly affected by adding a presupposition trigger. The reason for this could be that the matched sentences were already good, so adding a presupposition trigger did not make a difference. Mismatched resemblance sentences how- ever improve significantly by adding a presupposition trigger, supporting the theory of Hendriks (2004) and reproducing the results of Arregui et al. (2006), only now using a pronoun as an anaphoric device instead of verb phrase ellipsis. Adding a presupposition trig- ger to mismatched cause-effect sentences does not have a significant effect though.
What will remain unclear is why adding a presuppo- sition tigger to a matched or mismatched cause-effect
sentence does not improve the sentence siginificantly.
An explanation for the difference in effect between the two presupposition triggers used in this experiment could be that some presupposition triggers are more commonly used than others, and ook is more com- monly used in Dutch than toch. But that does not ex- plain why toch is still used in Dutch, while it doesn’t make sentences more felicitous. It might be that some or all sentences were not appropriate for the use of toch, because of the coherence relation or for some other rea- son, but new research would have to be done on what kind of sentences toch is normally used in in Dutch.
What also can’t be explained by the theories men- tioned in the introduction is why matched resemblance sentences with presupposition triggers are not more fe- licitous than mismatched resemblance sentences with presupposition triggers. Because even though the pre- supposotion trigger gives an alternative mean for in- terpreting the coherence relation, the mismatch is still present, so it would be expected that matched sen- tences will still be more felicitous than mismatched sentences. It might be the case that the reader is so glad to recieve another way to discover the coherence relation between the two clauses that he/she grades the sentence very high, but a matched version should of course always sound more felicitous than a mis- matched version of a sentence.
Concluding, our results show that a presupposi- tion trigger significantly improves the felicity of mis- matched resemblance sentences, so the presence of the presupposition trigger too can help repair some of the damage made by the voice mismatch. Futhermore, our results show that both resemblance and cause-effect sentences are not significantly worsened by mismatch, so they are not in support of the theory of Kehler (2002) and Kehler et al. (2008) that mismatching only affects the felicity of sentences with a resemblance re- lation, but also don’t support the theory of Frazier and Clifton Jr. (2006), who state mismatching will worsen both resemblance and cause-effect sentences. Finally, adding a presupposition trigger doesn’t significantly improve the felicity of cause-effect sentences, and more research should be done on the effect of the presuppo- sition trigger toch on different types of sentences.
References
Arregui, A., Clifton Jr., C., Frazier, L., & Moulton, K.
(2006). Processing elided verb phrases with flawed antecedents: The recycling hypothesis. Journal of Memory and Language, 55(2), 232–246.
Frazier, L., & Clifton Jr., C. (2006). Ellipsis and dis- course coherence. Linguistics and Philosophy, 29, 315–
346.
Hendriks, P. (2004). Coherence relations, ellipsis, and
contrastive topics. Journal of Semantics, 21(2), 133–
153.
Hume, D. (1748). An inquiry concerning human under- standing (1995edition ed.). The Liberal Arts Press, New York.
Kehler, A. (2002). Coherence, reference, and the theory of grammar. CSLI Publications.
Kehler, A., Kertz, L., Rohde, H., & Elman, J. L. (2008).
Coherence and coreference revisited. Journal of Se- mantics, 25, 1–44.
Wolf, F., Gibson, E., & Desmet, T. (2004). Discourse co- herence and pronoun resolution. Language and Cog- nitive Processes, 19(6), 665–675.
Appendices
A Test sentences
#. 1. resemblance, match, without presupposition.
2. resemblance, match, with presupposition.
3. cause-effect, match, without presupposition.
4. cause-effect, match, with presupposition.
5. resemblance, mismatch, without presupposition.
6. resemblance, mismatch, with presupposition.
7. cause-effect, mismatch, without presupposition.
8. cause-effect, mismatch, with presupposition.
1. (a) Henk moedigde Stijn aan, en Floor prikkelde hem.
(b) Henk moedigde Stijn aan, en Floor prikkelde hem ook.
(c) Henk moedigde Stijn aan, maar Floor ontmoedigde hem.
(d) Henk moedigde Stijn aan, maar Floor ontmoedigde hem toch.
(e) Stijn werd door Henk aangemoedigd, en Floor prikkelde hem.
(f) Stijn werd door Henk aangemoedigd, en Floor prikkelde hem ook.
(g) Stijn werd door Henk aangemoedigd, maar Floor ontmoedigde hem.
(h) Stijn werd door Henk aangemoedigd, maar Floor ontmoedigde hem toch.
2. (a) Sander pakte Jelmer, en Cindy hield hem vast.
(b) Sander pakte Jelmer, en Cindy hield hem ook vast.
(c) Sander pakte Jelmer, maar Cindy liet hem vrij.
(d) Sander pakte Jelmer, maar Cindy liet hem toch vrij.
(e) Jelmer werd door Sander gepakt, en Cindy hield hem vast.
(f) Jelmer werd door Sander gepakt, en Cindy hield hem ook vast.
(g) Jelmer werd door Sander gepakt, maar Cindy liet hem vrij.
(h) Jelmer werd door Sander gepakt, maar Cindy liet hem toch vrij.
3. (a) Robin wees Dirk af, en Tamara verliet hem.
(b) Robin wees Dirk af, en Tamara verliet hem ook.
(c) Robin wees Dirk af, maar Tamara accepteerde hem.
(d) Robin wees Dirk af, maar Tamara accepteerde hem toch.
(e) Dirk werd door Robin afgewezen, en Tamara verliet hem.
(f) Dirk werd door Robin afgewezen, en Tamara verliet hem ook.
(g) Dirk werd door Robin afgewezen, maar Tamara accepteerde hem.
(h) Dirk werd door Robin afgewezen, maar Tamara accepteerde hem toch.
4. (a) Peter liet Karel achter, en Laura negeerde hem.
(b) Peter liet Karel achter, en Laura negeerde hem ook.
(c) Peter liet Karel achter, maar Laura bleef bij hem.
(d) Peter liet Karel achter, maar Laura bleef toch bij hem.
(e) Karel werd door Peter achtergelaten, en Laura negeerde hem.
(f) Karel werd door Peter achtergelaten, en Laura negeerde hem ook.
(g) Karel werd door Peter achtergelaten, maar Laura bleef bij hem.
(h) Karel werd door Peter achtergelaten, maar Laura bleef toch bij hem.
5. (a) Tim kraakte Willem af, en Marrit haalde hem neer.
(b) Tim kraakte Willem af, en Marrit haalde hem ook neer.
(c) Tim kraakte Willem af, maar Marrit prees hem.
(d) Tim kraakte Willem af, maar Marrit prees hem toch.
(e) Willem werd door Tim afgekraakt, en Marrit haalde hem neer.
(f) Willem werd door Tim afgekraakt, en Marrit haalde hem ook neer.
(g) Willem werd door Tim afgekraakt, maar Marrit prees hem.
(h) Willem werd door Tim afgekraakt, maar Marrit prees hem toch.
6. (a) Thijs prees Ivo, en Anne verheerlijkte hem.
(b) Thijs prees Ivo, en Anne verheerlijkte hem ook.
(c) Thijs prees Ivo, maar Anne haalde hem neer.
(d) Thijs prees Ivo, maar Anne haalde hem toch neer.
(e) Ivo werd door Thijs geprezen, en Anne verheerlijkte hem.
(f) Ivo werd door Thijs geprezen, en Anne verheerlijkte hem ook.
(g) Ivo werd door Thijs geprezen, maar Anne haalde hem neer.
(h) Ivo werd door Thijs geprezen, maar Anne haalde hem toch neer.
7. (a) Mik accepteerde Herman, en Naomi aanvaardde hem.
(b) Mik accepteerde Herman, en Naomi aanvaardde hem ook.
(c) Mik accepteerde Herman, maar Naomi wees hem af.
(d) Mik accepteerde Herman, maar Naomi wees hem toch af.
(e) Herman werd door Mik geaccepteerd, en Naomi aanvaardde hem.
(f) Herman werd door Mik geaccepteerd, en Naomi aanvaardde hem ook.
(g) Herman werd door Mik geaccepteerd, maar Naomi wees hem af.
(h) Herman werd door Mik geaccepteerd, maar Naomi wees hem toch af.
8. (a) Harry bedreigde Lars, en Mieke intimideerde hem.
(b) Harry bedreigde Lars, en Mieke intimideerde hem ook.
(c) Harry bedreigde Lars, maar Mieke redde hem.
(d) Harry bedreigde Lars, maar Mieke redde hem toch.
(e) Lars werd door Harry bedreigd, en Mieke intimideerde hem.
(f) Lars werd door Harry bedreigd, en Mieke intimideerde hem ook.
(g) Lars werd door Harry bedreigd, maar Mieke redde hem.
(h) Lars werd door Harry bedreigd, maar Mieke redde hem toch.
9. (a) Samuel trapte Justin, en Marja schopte hem.
(b) Samuel trapte Justin, en Marja schopte hem ook.
(c) Samuel trapte Justin, maar Marja verdedigde hem.
(d) Samuel trapte Justin, maar Marja verdedigde hem toch.
(e) Justin werd door Samuel getrapt, en Marja schopte hem.
(f) Justin werd door Samuel getrapt, en Marja schopte hem ook (g) Justin werd door Samuel getrapt, maar Marja verdedigde hem.
(h) Justin werd door Samuel getrapt, maar Marja verdedigde hem toch.
10. (a) Leon misleidde Simon, en Rianne verraadde hem.
(b) Leon misleidde Simon, en Rianne verraadde hem ook.
(c) Leon misleidde Simon, maar Rianne waarschuwde hem.
(d) Leon misleidde Simon, maar Rianne waarschuwde hem toch.
(e) Simon werd door Leon misleid, en Rianne verraadde hem.
(f) Simon werd door Leon misleid, en Rianne verraadde hem ook.
(g) Simon werd door Leon misleid, maar Rianne waarschuwde hem.
(h) Simon werd door Leon misleid, maar Rianne waarschuwde hem toch.
11. *
(a) Frank bestal Nick, en Petra bedroog hem.
(b) Frank bestal Nick, en Petra bedroog hem ook.
(c) Frank bestal Nick, maar Petra beschermde hem.
(d) Frank bestal Nick, maar Petra beschermde hem toch.
(e) Nick werd door Frank bestolen, en Petra bedroog hem.
(f) Nick werd door Frank bestolen, en Petra bedroog hem ook.
(g) Nick werd door Frank bestolen, maar Petra beschermde hem.
(h) Nick werd door Frank bestolen, maar Petra beschermde hem toch.
12. *
(a) Rick steunde Maarten, en Lieke hielp hem.
(b) Rick steunde Maarten, en Lieke hielp hem ook.
(c) Rick steunde Maarten, maar Lieke bekritiseerde hem.
(d) Rick steunde Maarten, maar Lieke bekritiseerde hem toch.
(e) Maarten werd door Rick gesteund, en Lieke hielp hem.
(f) Maarten werd door Rick gesteund, en Lieke hielp hem ook.
(g) Maarten werd door Rick gesteund, maar Lieke bekritiseerde hem.
(h) Maarten werd door Rick gesteund, maar Lieke bekritiseerde hem toch.
13. (a) Bas verdedigde Koen, en Julia beschermde hem.
(b) Bas verdedigde Koen, en Julia beschermde hem ook.
(c) Bas verdedigde Koen, maar Julia viel hem aan.
(d) Bas verdedigde Koen, maar Julia viel hem toch aan.
(e) Koen werd door Bas verdedigd, en Julia beschermde hem.
(f) Koen werd door Bas verdedigd, en Julia beschermde hem ook.
(g) Koen werd door Bas verdedigd, maar Julia viel hem aan.
(h) Koen werd door Bas verdedigd, maar Julia viel hem toch aan.
14. (a) Tijn dankte Cas af, en Hanna keerde hem de rug toe.
(b) Tijn dankte Cas af, en Hanna keerde hem ook de rug toe.
(c) Tijn dankte Cas af, maar Hanna charmeerde hem.
(d) Tijn dankte Cas af, maar Hanna charmeerde hem toch.
(e) Cas werd door Tijn afgedankt, en Hanna keerde hem de rug toe.
(f) Cas werd door Tijn afgedankt, en Hanna keerde hem ook de rug toe.
(g) Cas werd door Tijn afgedankt, maar Hanna charmeerde hem.
(h) Cas werd door Tijn afgedankt, maar Hanna charmeerde hem toch.
15. (a) Chris waardeerde Ben, en Margo bewonderde hem.
(b) Chris waardeerde Ben, en Margo bewonderde hem ook.
(c) Chris waardeerde Ben, maar Margo minachtte hem.
(d) Chris waardeerde Ben, maar Margo minachtte hem toch.
(e) Ben werd door Chris gewaardeerd, en Margo bewonderde hem.
(f) Ben werd door Chris gewaardeerd, en Margo bewonderde hem ook.
(g) Ben werd door Chris gewaardeerd, maar Margo minachtte hem.
(h) Ben werd door Chris gewaardeerd, maar Margo minachtte hem toch.
16. (a) Erik bond Vik vast, en Mirjam ketende hem.
(b) Erik bond Vik vast, en Mirjam ketende hem ook.
(c) Erik bond Vik vast, maar Mirjam maakte hem los.
(d) Erik bond Vik vast, maar Mirjam maakte hem toch los.
(e) Vik werd door Erik vastgebonden, en Mirjam ketende hem.
(f) Vik werd door Erik vastgebonden, en Mirjam ketende hem ook.
(g) Vik werd door Erik vastgebonden, maar Mirjam maakte hem los.
(h) Vik werd door Erik vastgebonden, maar Mirjam maakte hem toch los.
17. *
(a) Joost aaide Tom, en Ina streelde hem.
(b) Joost aaide Tom, en Ina streelde hem ook.
(c) Joost aaide Tom, maar Ina verfoeide hem.
(d) Joost aaide Tom, maar Ina verfoeide hem toch.
(e) Tom werd door Joost geaaid, en Ina streelde hem.
(f) Tom werd door Joost geaaid, en Ina streelde hem ook.
(g) Tom werd door Joost geaaid, maar Ina verfoeide hem.
(h) Tom werd door Joost geaaid, maar Ina verfoeide hem toch.
18. (a) Rutger haatte Frits, en Ellen verafschuwde hem.
(b) Rutger haatte Frits, en Ellen verafschuwde hem ook.
(c) Rutger haatte Frits, maar Ellen had hem lief.
(d) Rutger haatte Frits, maar Ellen had hem toch lief.
(e) Frits werd door Rutger gehaat, en Ellen verafschuwde hem.
(f) Frits werd door Rutger gehaat, en Ellen verafschuwde hem ook.
(g) Frits werd door Rutger gehaat, maar Ellen had hem lief.
(h) Frits werd door Rutger gehaat, maar Ellen had hem toch lief.
19. (a) Pim nodigde Hugo uit, en Wilma vroeg hem mee.
(b) Pim nodigde Hugo uit, en Wilma vroeg hem ook mee.
(c) Pim nodigde Hugo uit, maar Wilma negeerde hem.
(d) Pim nodigde Hugo uit, maar Wilma negeerde hem toch.
(e) Hugo werd door Pim uitgenodigd, en Wilma vroeg hem mee.
(f) Hugo werd door Pim uitgenodigd, en Wilma vroeg hem ook mee.
(g) Hugo werd door Pim uitgenodigd, maar Wilma negeerde hem.
(h) Hugo werd door Pim uitgenodigd, maar Wilma negeerde hem toch.
20. (a) Roy liet Erwin toe, en Sarah keurde hem goed.
(b) Roy liet Erwin toe, en Sarah keurde hem ook goed.
(c) Roy liet Erwin toe, maar Sarah sloot hem buiten.
(d) Roy liet Erwin toe, maar Sarah sloot hem toch buiten.
(e) Erwin werd door Roy toegelaten, en Sarah keurde hem goed.
(f) Erwin werd door Roy toegelaten, en Sarah keurde hem ook goed.
(g) Erwin werd door Roy toegelaten, maar Sarah sloot hem buiten.
(h) Erwin werd door Roy toegelaten, maar Sarah sloot hem toch buiten.
21. (a) Olivier verbeterde Oscar, en Iris corrigeerde hem.
(b) Olivier verbeterde Oscar, en Iris corrigeerde hem ook.
(c) Olivier verbeterde Oscar, maar Iris beloonde hem.
(d) Olivier verbeterde Oscar, maar Iris beloonde hem toch.
(e) Oscar werd door Olivier verbeterd, en Iris corrigeerde hem.
(f) Oscar werd door Olivier verbeterd, en Iris corrigeerde hem ook.
(g) Oscar werd door Olivier verbeterd, maar Iris beloonde hem.
(h) Oscar werd door Olivier verbeterd, maar Iris beloonde hem toch.
22. (a) Thomas stalkte Klaas, en Karin viel hem lastig.
(b) Thomas stalkte Klaas, en Karin viel hem ook lastig.
(c) Thomas stalkte Klaas, maar Karin alarmeerde hem.
(d) Thomas stalkte Klaas, maar Karin alarmeerde hem toch.
(e) Klaas werd door Thomas gestalkt, en Karin viel hem lastig.
(f) Klaas werd door Thomas gestalkt, en Karin viel hem ook lastig.
(g) Klaas werd door Thomas gestalkt, maar Karin alarmeerde hem.
(h) Klaas werd door Thomas gestalkt, maar Karin alarmeerde hem toch.
23. *
(a) Gert beledigde Fred, en Bette kwetste hem.
(b) Gert beledigde Fred, en Bette kwetste hem ook.
(c) Gert beledigde Fred, maar Bette behoedde hem.
(d) Gert beledigde Fred, maar Bette behoedde hem toch.
(e) Fred werd door Gert beledigd, en Bette kwetste hem.
(f) Fred werd door Gert beledigd, en Bette kwetste hem ook.
(g) Fred werd door Gert beledigd, maar Bette behoedde hem.
(h) Fred werd door Gert beledigd, maar Bette behoedde hem toch.
24. *
(a) Guus dankte Aaron, en Lisa loofde hem.
(b) Guus dankte Aaron, en Lisa loofde hem ook.
(c) Guus dankte Aaron, maar Lisa verachtte hem.
(d) Guus dankte Aaron, maar Lisa verachtte hem toch.
(e) Aaron werd door Guus gedankt, en Lisa loofde hem.
(f) Aaron werd door Guus gedankt, en Lisa loofde hem ook.
(g) Aaron werd door Guus gedankt, maar Lisa verachtte hem.
(h) Aaron werd door Guus gedankt, maar Lisa verachtte hem toch.
25. *
(a) Floris joeg Kelvin op, en Maria jutte hem op.
(b) Floris joeg Kelvin op, en Maria jutte hem ook op.
(c) Floris joeg Kelvin op, maar Maria remde hem af.
(d) Floris joeg Kelvin op, maar Maria remde hem toch af.
(e) Kelvin werd door Floris opgejaagd, en Maria jutte hem op.
(f) Kelvin werd door Floris opgejaagd, en Maria jutte hem ook op.
(g) Kelvin werd door Floris opgejaagd, maar Maria remde hem af.
(h) Kelvin werd door Floris opgejaagd, maar Maria remde hem toch af.
26. (a) Kees berispte Kay, en Renske vermaande hem.
(b) Kees berispte Kay, en Renske vermaande hem ook.
(c) Kees berispte Kay, maar Renske complimenteerde hem.
(d) Kees berispte Kay, maar Renske complimenteerde hem toch.
(e) Kay werd door Kees berispt, en Renske vermaande hem.
(f) Kay werd door Kees berispt, en Renske vermaande hem ook.
(g) Kay werd door Kees berispt, maar Renske complimenteerde hem.
(h) Kay werd door Kees berispt, maar Renske complimenteerde hem toch.
27. (a) Gerard ondervroeg Sjors, en Marloes verhoorde hem.
(b) Gerard ondervroeg Sjors, en Marloes verhoorde hem ook.
(c) Gerard ondervroeg Sjors, maar Marloes stelde hem op zijn gemak.
(d) Gerard ondervroeg Sjors, maar Marloes stelde hem toch op zijn gemak.
(e) Sjors werd door Gerard ondervraagd, en Marloes verhoorde hem.
(f) Sjors werd door Gerard ondervraagd, en Marloes verhoorde hem ook.
(g) Sjors werd door Gerard ondervraagd, maar Marloes stelde hem op zijn gemak.
(h) Sjors werd door Gerard ondervraagd, maar Marloes stelde hem toch op zijn gemak.
28. (a) Stefan verdacht Juri, en Roos beschuldigde hem.
(b) Stefan verdacht Juri, en Roos beschuldigde hem ook.
(c) Stefan verdacht Juri, maar Roos vertrouwde hem.
(d) Stefan verdacht Juri, maar Roos vertrouwde hem toch.
(e) Juri werd door Stefan verdacht, en Roos beschuldigde hem.
(f) Juri werd door Stefan verdacht, en Roos beschuldigde hem ook.
(g) Juri werd door Stefan verdacht, maar Roos vertrouwde hem.
(h) Juri werd door Stefan verdacht, maar Roos vertrouwde hem toch.
* removed from the test set in experiment 2
B Results experiment 1
Answers are shown per question and subject. The last column shows the number of ones per sentence, which are the number of answers that say the pronoun refers to the subject. The sentences with more than two ones were removed from the test set in experiment 2.
subject
question 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # 1s
1 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 3 0
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
3 2 2 2 2 2 3 2 2 2 3 2 2 1 2 2 1 2 2 2 2 2
4 2 2 2 2 3 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 0
5 1 3 3 3 2 2 2 2 2 2 2 2 3 2 2 2 2 3 2 1 2
6 2 2 2 2 2 2 3 2 2 1 2 2 1 2 2 2 2 2 2 2 2
7 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 3 1 1
8 2 1 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
9 2 3 2 3 2 2 2 2 2 2 2 2 3 2 1 3 1 3 3 3 2
10 2 3 3 2 2 2 2 2 2 3 2 2 3 2 1 2 3 2 2 2 1
11 1 3 2 2 2 2 2 1 2 1 2 2 3 3 2 2 2 3 2 3 3
12 3 2 2 2 2 1 2 1 2 1 2 2 3 2 2 2 2 2 2 2 3
13 2 3 3 2 2 2 2 2 2 2 2 2 2 1 2 2 2 3 2 1 2
14 2 2 2 2 3 2 3 2 2 3 2 2 2 2 2 1 2 2 2 2 1
15 2 2 2 2 3 2 2 1 2 2 2 2 2 2 2 1 2 2 2 2 2
16 2 2 2 2 3 2 2 2 1 2 2 2 2 2 2 1 2 2 2 2 2
17 2 3 2 2 3 2 2 2 2 2 2 2 3 2 1 1 1 2 2 3 3
18 3 2 2 2 1 2 2 2 3 2 2 2 2 2 2 2 2 2 1 2 2
19 2 2 2 2 2 2 2 2 2 2 2 2 1 2 3 2 2 2 2 2 1
20 2 2 2 2 2 2 2 2 2 1 1 2 3 3 2 2 2 2 2 2 2
21 2 3 1 2 2 2 2 2 2 2 2 2 2 1 2 2 2 3 2 3 2
22 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 1 2 1
23 2 2 2 2 3 2 2 1 3 3 1 2 3 2 2 2 2 2 2 1 3
24 2 3 2 3 1 1 2 2 2 3 1 2 3 2 2 1 2 2 2 2 4
25 3 3 2 2 3 2 2 2 1 2 1 2 2 1 3 2 1 2 2 3 4
26 2 2 2 2 3 2 2 2 2 2 2 2 1 2 3 2 2 2 2 2 1
27 2 2 2 2 3 2 2 2 3 3 2 2 3 2 2 2 2 2 2 2 0
28 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
