The Importance of Being Civil
Examining the effects and comparing sources of political incivility on intended political engagement
Danny Bowles
Student ID: 11571535
Master’s Thesis
Graduate School of Communication
Master’s Programme Communication Science Supervisor: dr. mr. J.H.P. (Joost) van Spanje Date of Completion: 01/02/2019
Abstract
Political discussions can often turn uncivil. This is the case at the highest levels of politics, but also in the political talk of ordinary citizens. Conventional wisdom suggests that incivility is a negative force in politics. However, the effects of exposure to political incivility and the explanation of these effects remain ambiguous. Moreover, little empirical work has been done on the subject outside USA. Conducting an experiment with a conveniently-gathered, EU-wide sample (N = 185), this paper analyses the effects of exposure to political incivility on intention to engage with politics, and how this can differ according to the source of incivility. It was found that exposure can depress future engagement in particular electoral campaigns. When comparing sources, I find that the incivility of ordinary citizens is a more relevant cue with which people make the decision to engage in politics, as opposed to politicians’ incivility. Exposure to what this paper will refer to as “mass incivility” has a negative effect on intention to engage compared to elite incivility. According to the findings here, the affective mechanism of disgust does not explain these effects. Among other things, the results suggest that ordinary citizens as message senders should be incorporated more into future studies on political communicative processes. They also provide implications for us all on how we choose to conduct our politics.
Word Count = 7,496
Introduction
The public’s view of politics is defined not just by its outcomes, but also by the way it is conducted. In political discourse, discussions can often turn uncivil. Using social media and exposure to political media reveals this reality to anyone. Civility is a long-studied idea and political norm, going back to Aristotle (Papacharissi, 2004). Adherence to this norm is considered by some to be a pre-requisite for some vital ideals of democracy (Delli-Carpini, Cook & Jacobs, 2004; Schudson, 1997). The perceived proclivity of incivility in
contemporary discourse is a concern held by some political commentators, with suggestions that it leads to sub-optimum outcomes, and turns people off politics (Herbst, 2010; Lecheler, 2013; Lewis, 2018; Sapiro, 1999).
However, the empirical evidence of the effects of incivility is unclear (Brooks & Geer, 2007; Mutz & Reeves, 2005). Nor is it obvious how exposure to incivility can influence our political attitudes and behaviours. Moreover, any evidence is limited to the American political context. The limited research done in Europe on incivility demonstrates its presence in British (Harris, 2001) and Austrian (Jenny et al., 2017) parliamentary debate. Beyond incivility research, there is mounting interest in populism, and its communicative style, particularly in Europe. A signature aspect of its style is to proudly disobey cultural and political mores, through exclusionary rhetoric and provocation (Heinisch, 2003; Nai, 2018; Wodak, 2013), which can be seen as incivility. When exposed to this, and other
manifestations of political incivility, does our view of politics and intention to get involved in the process change? This paper provides an introductory analysis of the effects of incivility on political engagement in Europe.
Work on incivility and other communicative processes in politics usually concerns politicians’ rhetoric. Politicians are, after all, thought to be the primary definers of the political landscape (Zaller, 1992). However, we are exposed to incivility in a wide variety of contexts and from a variety of sources. Those viewing the Brexit debate in the UK, the recent abortion referendum in Ireland, the gilets jaunes movement originating in France, among a long list of political campaigns, will see that incivility is an aspect of mass political discourse. We have more and more access to the thoughts of ordinary citizens in politics, through social media (Anderson et al., 2014; Papacharissi, 2004; Rains et al., 2017; Sveningsson, 2014) and, perhaps relatedly, political media making greater use of citizen voices (Kleemans, Schaap & Hermans, 2017; Lefevere, De Swert & Walgrave, 2012). Indeed, when discussing political incivility, politicians often refer to the messages they receive from voters, not fellow politicians (Lecheler, 2013; Lewis, 2018). Evidence suggests that the exclusionary and exaggerated rhetoric of populism has a growing place in mass political discourse (Rooduijn, 2014). Those around us, in our social network and beyond, are influential in our decision to engage in politics, so the cues we take from the way they conduct their politics may be very relevant and influential (Levinsen & Yndigegn, 2015; McClurg, 2006; Mutz, 2002; Torcal & Maldonado, 2014). With this study, I focused on the distinction between two sources of incivility, “mass incivility” in popular discourse, and “elite incivility” in politicians’ discourse.
I sought to test the effect of this incivility on intention to engage in the European Parliament (EP) elections in May 2019, as well as general future political engagement. I wanted to test the effect of incivility in relation to a real political campaign. However, we might expect any negative effects to be magnified for a low-salience election, providing a potential challenge to the wider application of the findings. In view of this, I tested the effects of incivility also on intention to engage in future political campaigns in general.
This paper makes three principal contributions to the study of incivility. First, I will study the effects of incivility in the European context. Second, I will introduce the distinction of sources, by establishing the concept of “mass incivility”, and compare its effects to those of “elite incivility”. Third, the paper will seek to understand if the affective reaction of disgust can explain the effect of incivility on political behaviours and attitudes. This paper will present the results of an experiment conducted on a conveniently-gathered, EU-wide participant pool (N = 185) to answer the research question: To what extent and how does exposure to different sources of incivility affect intention to engage in the political process?
Theoretical Framework
Scholars of incivility have struggled to clearly define the term (Benson, 2011). They agree that civility, the antonym, is related to social norms of politeness and courtesy. These norms are particularly strong in that non-adherence is quite conspicuous (Brown & Levinson, 1987). Central to politeness theory is the concept of “face”, that is, people’s need to be respected (Brown & Levinson, 1987). Incivility threatens one’s “face”; in other words, it is some disrespectful act. However, disrespect depends on the perception of the receiver, and the given social norms of a situation, rather than it being a stable set of behaviours
(Muddiman, 2013; Phillips & Smith, 2003). Even disagreement with one’s point of view can be considered uncivil (Muddiman, 2013). It is then essential to provide a coherent conceptual definition of incivility to use in this study, that can apply across the EU. My working
definition is taken from two seminal works on the subject; using gratuitous asides in political discussion that are purposefully inflammatory and disrespectful (Brooks & Geer, 2007; Mutz & Reeves, 2005). This definition is shared by multiple authors on the subject (Gervais, 2014, 2017; Hwang, Kim & Huh, 2014; Rains et al., 2017; Sydnor, 2015).
According to this definition, incivility is a discursive tone. Only a couple of words can turn a civil statement into an uncivil statement (Brooks & Geer, 2007; Gervais, 2014). Saying that someone is a “liar” is not necessarily uncivil and is an essential part of political discourse but calling someone a “deceitful little liar” is disrespectful and inflammatory. This is not more informative but is a pointed personal attack. The manifestations within this definition are manifold: name-calling, sarcasm, misrepresentative exaggerations, stereotyping, etc. (Mutz & Reeves, 2005; Sydnor, 2015). We can look at incivility as a continuum: civil utterances are on one side of the scale; outrageous claims, lewd remarks, prejudiced statements and efforts to limit one’s political rights are at the other end (Borah, 2013). The manifestations within the present definition are somewhere in the middle. Borah (2013) and Sobieraj and Berry (2011) analyse the more extreme end of the spectrum.
According to Papacharissi (2004), my definition unnecessarily limits the sphere of legitimate discourse. Her definition, favoured by Anderson et al. (2014, 2018) and Rossini (forthcoming), says that incivility is that which impedes the ideal of deliberation. Rossini (forthcoming) argues that what I and others have defined as incivility can be necessary political rhetoric. Certainly, the manifestations that we see within the present definition represent minor infractions, but they are still purposefully face-threatening acts. We are more often exposed to these minor forms of incivility in daily discussion and social and political media, so their effects are more relevant (Brooks & Geer, 2007). Papacharissi’s (2004) definition of incivility is less a matter of rhetorical tone but of statements that directly
impinge upon the political rights of others. This is something closer to anti-democratic speech than incivility (Sydnor, 2015).
The Effects of Incivility
Incivility is particularly evident in political discourse as a way of performing identity (Jamieson et al., 2017; Rains et al., 2017). It can be a useful tool for politicians to initiate group mobilisation by highlighting distinctions between the in-group and the out-group. The personal effects of this identity performance depend on whether one is in the in-group or not (Gervais, 2017). Incivility also increases the chances of political messages being picked up and highlighted by the media (Bradley & Lang, 2007; Geer, 2012; Harcup & O’Neill, 2017; Mutz, 2006, 2007). Empirical research has found that the presence of incivility in political media is increasing (Sobieraj & Berry, 2011). This identity performance by elites can mobilise partisans to use the incivility in their own discourse, especially on social media (Gervais, 2017). It can be especially useful in the relative anonymity of online platforms (Rains et al., 2017; Rowe, 2015). Uncivil comments provoke more interactions, as in-group members “like” and members of the out-group tend to reply with incivility of their own as a means of retribution (Andersson & Pearson, 1999; Papacharissi, 2004; Rains et al., 2017). Both in political and social media, the identity performance of incivility tends to be highlighted.
Incivility emphasises differences between individuals and groups, to the extent that there appears to be little, if any, mutual respect between them. When used in a piece of text, more severe manifestations of incivility make the information appear more biased (Anderson et al., 2018; Borah, 2013). It also affects one’s opinion on an issue; incivility encourages polarisation, as those exposed to it sort themselves out into more clearly defined in- and out-groups (Hwang, Kim & Huh, 2014; MacKuen et al., 2010; Mutz, 2006). When political elites adopt incivility in their messaging, important political attitudes can be negatively affected. Satisfaction and trust in the political process declines, as does our own perceived ability to affect politics (Borah, 2013; Hwang, Kim & Huh, 2014; Mutz & Reeves, 2005). Our
impressions of politics are negatively affected when it turns uncivil, as it appears to be an area in which mutual respect is not present. However, political behaviours, like engagement, which I am testing here, tend to be more stable (Lau, Sigelman & Brown Rovner, 2007).
Mass Incivility
Here, I will make an important distinction in types of incivility. Political incivility is not solely the realm of politicians (Anderson et al., 2014; Papacharissi, 2004; Rains et al., 2017). In this paper, I distinguish between mass incivility, that of ordinary citizens, and elite incivility, that of politicians. Exemplification is an established field in mass communication; news often uses personal cases to aid the description of phenomena (Zillmann & Brosius, 2000). What of the exemplification of political processes? I expect that there is a different judgement being made when we see people like ourselves being uncivil compared to observing the same behaviour among politicians.
We know that the source of political and persuasive communication matters (Gotlieb & Sarel, 1991; Hartman & Weber, 2009). Citizen voices in political advertisements have been found to add to the credibility and persuasiveness of a message (Franklin Fowler et al., 2014). These voices represent the on-the-ground realities of policies. Likewise, citizen voices in the news make stories more relevant and realistic to the audience (Lefevere, De Swert & Walgrave, 2012; Vliegenthart & Boukes, 2018). Extending this to our case, mass incivility in news provides realistic and relevant evidence of incivility in political discourse. This is more relevant because it is more likely to affect us if we were to engage in politics.
Incivility between politicians suggests that the practitioners of politics are
disrespectful. In this case, we can engage in an appropriate reprisal act (Durkheim, 1960); if we reject their incivility, we can vote them out of office. The same possibility is not afforded to us if fellow citizens are uncivil. We may also afford elites some slack because they are
involved in high-stakes discussions in which they are authoritative, so their incivility is more reasoned. Mass incivility suggests that personal disrespect is an aspect of even low-stakes political discussions, where nothing of great importance is being decided. According to exemplification theory, the use of citizen voices in news suggests that the mass of the population or a particular group support a given statement (Zillmann & Brosius, 2000). Evidence of mass incivility implies that the mass, or a large group within it, believes this is a fitting way to conduct politics. It suggests that politics is uncivil, not just its professional practitioners. We can view the quarrels of political elites distantly and garner some
entertainment from their incivility, which can dampen any negative effects (Brooks & Geer, 2007; Funk, 2001). This distance does not necessarily exist when we see ordinary citizens being uncivil. Then the threat of interpersonal conflict in politics becomes more apparent and realistic (Mutz, 2002).
Political Engagement
This paper will look at the effects of incivility on intention to engage in politics. Democracy relies on a citizenry that engages with the political process (Martin & van Deth, 2007). Political engagement refers to citizens’ attention to political institutions and processes (Barrett & Zani, 2015). This attention can come in the form of awareness, opinions, feelings and actions (Barrett & Zani, 2015). This is broader than, but may be the basis of, the concrete actions of political participation. Political engagement requires resources: time, money, education, among others (Verba, Schlozman & Brady, 1995). When the threshold for engagement is raised, when more resources are thought to be needed, the intention to participate in the process should reduce.
I expect this to be the case especially for the EP elections. These are “second-order” elections, in that they are considered less important than national elections (Reif & Schmitt,
1980). Some scholars have argued that their low salience can be counteracted by more conflict in news coverage of European Union (EU) affairs (Schuck, Vliegenthart & de Vreese, 2014). I argue that anything that raises the threshold for engaging with EU politics, for example, making political debate in Brussels and Strasbourg appear more conflictual and uncivil, can work to turn voters even more apathetic to these elections. There is believed to be less at stake in European elections (Franklin, 2014); incivility in low-stake scenarios is likely to be perceived as less acceptable.
There is some limited evidence that exposure to incivility can encourage engagement. Brooks and Geer (2007) found no effect of incivility on intention to vote or political interest, but they argue that the trend in the data suggests that incivility can mobilise engagement. Firstly, political interest is thought to be relatively stable over one’s lifetime (Prior, 2010). Moreover, the data provided no statistically significant evidence that mobilisation was occurring. To my knowledge, no succeeding research has supported their suggestion.
Negativity, which highlights the problems with political opponents in political messaging in a much softer way than incivility, has been shown by scholars to encourage engagement (e.g. Arceneaux & Nickerson, 2010; Delli-Carpini and Keeter 1996), though this claim is very much contested (e.g. Ansolabehere & Iyengar, 1995; Kahn & Kenney, 1999). Any potential mobilising effect may be limited to particular election scenarios (Djupe & Peterson, 2002; Martin, 2004) and personal traits (Clinton & Lapinski, 2004). Potentially, incivility can work in the same way.
My reading of the research on incivility and negativity sides with the demobilisation hypothesis. Ansolabehere and Iyengar (1995) and Kahn and Kenney (1999) argue that the cynicism and personal interest associated with simply negative political messaging can depress turnout. However, this effect might only occur in the long-term, after re-iterated exposure (Lau, Sigelman & Brown Rovner, 2007; Sobieraj & Berry, 2011). The effects of
incivility are thought to be more severe than those of negativity (Kahn & Kenney, 1999; Lau & Pomper, 2001). Mutz and Reeves (2005) and Borah (2013) have found it reduces our perceived ability to affect politics. This would necessarily influence our intention to engage in the process. Reactions to everyday incivility are linked to aversive strategies (Phillips & Smith, 2003). According to social psychology when agreement is not thought possible, the logical next step is to ignore the issue (Green, Visser & Tetlock, 2000). Even exposure to disagreement in our social networks can cause us to avoid political engagement because of the personal and interpersonal cost (Mutz, 2002; Levinsen & Yndigegn, 2015; Sveningsson, 2014; Torcal & Maldonado, 2014). Incivility magnifies this potential cost. Mass incivility provides a particularly relevant threat of disrespect when engaging in politics at even the lowest levels.
I hypothesise that incivility will impact negatively on participants’ intention to engage in politics. The more relevant threat of mass incivility will have a stronger depressing effect than elite incivility. I believe these effects will be observed for both general future political engagement and engagement with the May 2019 EP elections, but the low-salience nature of EU elections will mean there will be a stronger effect for EU engagement.
H1: Participants exposed to mass incivility will have lower intention to engage, (H1a) in politics generally and (H1b) the EP elections particularly, than those exposed to mass civility.
H2: Participants exposed to mass incivility will have lower intention to engage, (H2a) in politics generally and (H2b) the EP elections particularly, than participants exposed to elite incivility.
Disgust
Past analyses suggest that a physiological mediator can explain the effect of incivility on political attitudes and behaviours (Gervais, 2017; Mutz & Reeves, 2005). We can assume that the nature of the physiological arousal caused by incivility is negative, but negative emotions have distinct elicitors and associated behaviours (Petersen, 2010; Vandenbroek, 2012). I propose that the physiological mediator here is disgust. Negative political messaging has been shown to encourage disgust in participants (Kahn & Kenney, 1999). Sydnor (2015) finds incivility provokes disgust, more than anger or anxiety.
Disgust is a gut reaction used to fight and avoid poisonous substances (Rozin, Haidt & MacCauley, 2016; Sullivan & Masters, 1987). Ideas and actions can also be poisonous; in this case disgust becomes a moral judgement (Vandenbroek, 2012). When exposed to political incivility, disgust can be a moral judgement on the uncivil tone and non-adherence to social norms in political discourse (Schnall et al., 2008; Vandenbroek, 2012). Then, politics itself becomes a poisonous idea, and something to be avoided (Vandenbroek, 2012). Though subjects in the elite and mass incivility experimental groups will be exposed to the same words, I expect those in the mass condition will present higher levels of disgust. Social norms of politeness change according to the situation (Rossini, forthcoming). The norms, and thus the moral judgement caused by non-adherence, will be stronger in the low-stakes
scenario of mass political discussion.
H3: Exposure to mass incivility will result in higher levels of disgust towards the tone of the discourse, compared to mass civility.
H4: Those exposed to mass incivility will be more disgusted by the tone of the discourse than those exposed to elite incivility.
H5: The relationship between exposure to incivility and intended political
engagement, both for (H5a) politics generally and (H5b) the EP elections particularly, will be mediated by feelings of disgust towards the tone of the discourse.
H6: The relationship between source of incivility and intended political engagement, both for (H6a) politics generally and (H6b) the EP elections particularly, will be mediated by feelings of disgust towards the tone of the discourse.
Methods
These hypotheses were tested using a survey-embedded, online experiment.
Experiments offer a tractable way of examining causal effects. Through the manipulation of a few key words, I can isolate the effects of incivility, as I have defined it. The respondents for the study were gathered conveniently within my social network. Participants were also encouraged to share the study. The data was collected between 6 and 15 December 2018, with a total respondent pool of 215, reduced to 185 with the removal of uncompleted surveys.
Participants were exposed to one of three stimuli which came in the form of almost identical tweets.1 Each tweet referred to an “interesting article” and shared an image that was created to look like an extract of an article on a news website. Each extract was manipulated according to incivility and the source of the incivility. We are often exposed to political news through social media, where one can share clips of news articles. Providing only an extract meant details that may have affected the results beyond incivility and source of incivility were limited (Brooks & Geer, 2007; Gervais, 2017). The articles concerned the EU’s
Common Agricultural Policy (CAP). It is a relevant, EU-wide issue, but one of which people rarely have detailed knowledge (European Commission, 2016). High familiarity with a topic mitigates the effects of incivility (Anderson et al., 2014). I also wanted to avoid an issue for
which subjects had clear prior opinions, as like-minded and dissenting incivility produce distinct outcomes (Gervais, 2017).
In the mass civility and incivility experimental conditions, participants were told that the extract came from an article where journalists interviewed ordinary citizens across the EU in areas where agriculture is particularly important to the economy. The extract consisted of a discussion between two locals, Peter and Ivan, who disagreed on whether the CAP budget should be reduced. It was important that each respondent could think the messages were attributable to citizens from their country. Otherwise, subjects could isolate the incivility to a different country than their own, and it would have little bearing on their decision to engage in politics in their country. For this reason, the names Peter and Ivan were chosen for their geographical ambiguity and the country from which they came was not mentioned. Their arguments were taken largely from Pe’er et al. (2017). The substantial content of the discussion was held constant between the uncivil and civil conditions, with only the tone changing. For example, in the civil condition Peter claimed that Ivan “did not fully understand the problems farmers face”, while in the uncivil condition Peter said Ivan was “utterly clueless and stupid”. The main point of the argument is the same, that Ivan did not understand the issue. While the civil statement limits the misunderstanding to this issue, the uncivil statement is much more pointed, calling Ivan’s intelligence into question. It is disrespectful and deliberately inflammatory.
In the elite incivility condition participants were informed that the extract came from an article where Members of the European Parliament (MEPs) were interviewed for their opinions on the CAP. The text was identical to that of the mass incivility condition, but instead of Peter and Ivan being interviewed, Peter Jacobs MEP and Ivan Lind MEP were interviewed. These names were fabricated, which provides a challenge to the external validity
of the study. However, using real MEPs would have given geographical cues and challenged the internal validity.
Serving as an attention and manipulation check, participants were asked if the extract they read was based on the discussion of politicians. 46.3% of participants in the mass conditions believed that it was based on the discussion of politicians. 30.6% in the elite incivility group indicated that the article was not based on the discussion of politicians. There are various potential explanations for this, which are perhaps working in tandem. Firstly, the question was poorly-worded, leading to misinterpretations. Perhaps participants believed the question was asking whether the discussion was based on what Peter and Ivan had heard from politicians. However, this would not have been an issue for those in the elite condition. It may be that participants did not believe that Peter and Ivan were ordinary citizens, but politicians. The details of the CAP are not largely understood by ordinary citizens, so participants might have assumed anyone with such knowledge and strong opinions on the subject were politicians.2 Possibly, the participants did not pay attention to the stimuli, which can be a particular concern for web-based surveys (Sendelbah et al., 2016). More simply, perhaps the stimulus materials were not believable. The MEPs were fabricated, and it may not have been believable that two members of the public were asked for their opinions on CAP in a news article. This places a doubt on the interpretation and validity of the results. Though deleting data and limiting statistical power is a last resort, hypotheses of this study were based on participants’ understanding of the source of the incivility. It was decided to remove those who failed the manipulation check from the analyses.
2 We might then expect those in the uncivil condition to be more likely to believe that Peter and Ivan were
politicians because they apparently held their views on this niche topic more strongly, but this was not the case. The manipulation check was asked with a one-to-seven Likert scale, with seven meaning that the participant strongly agreed that the article was based on the discussion of politicians. Those in the mass incivility condition were no more likely to think that the article was based on the discussion of politicians (M = 3.98, SD = 1.71), compared to the mass civility condition (M = 4.03, SD = 1.57), t(120) = .17, p = .869.
This left a valid respondent pool of 108,3 divided unevenly across the three experimental conditions.4 The sample was mostly male (54.2%), young (Mdn = 25) and educated (82.4% had obtained at least an undergraduate degree). The participants came from twelve countries across the EU, but 49.1% came from Ireland.5
The manipulation of incivility was successful. Participants’ perception of the hostility and impoliteness (r = .78) in the article they had read was measured and combined to create an overall perceived incivility measure. Those in the two uncivil conditions thought the discussion in the article was more uncivil (M = 5.42, SD = 1.02) than those in the civil condition (M = 3.94, SD = 1.02), t(106) = 7.08, p < .001. There was no difference in perceived incivility between the elite and mass incivility conditions, t(71) = .41, p = .684. Dependent Variable
Intended political engagement was measured in two ways. First, participants were asked to think about politics generally, and the likelihood on a one-to-seven scale of them engaging in given political activities in the future, e.g. campaign or canvas for a political party.6 The items were taken from Barrett and Zani (2015). Participants also gave the likelihood of them engaging in actions related to the 2019 EP campaign, again on one-to-seven scales. This was a shorter list of activities with items taken from Strohmeier et al.
3 A power analysis using the G*power computer program was conducted based on previous findings on
incivility and source credibility in order to estimate the required sample size for this study (Faul et al., 2009). Though these studies are not being replicated here, they do provide a useful guideline. Mutz and Reeves (2005) found a small to medium effect size of incivility on trust in the political system and Hwang, Kim & Huh (2014) observed a similar effect size on the expectations of public deliberation. The power analysis indicated that a sample size of 102 (between only two conditions) would be necessary to detect medium effects (d = .5) with 80% power using a t-test between means with alpha at .05. Any potential effect of the source of incivility is harder to analyse but I used Franklin Fowler et al. (2014) as the lead. They also found a small to medium effect size of source on political message credibility. I conservatively estimate that a total sample across three
conditions of at least 153 would be necessary to replicate previous effect sizes. For this reason, I will interpret p < .1 as significant values in this paper.
4 30 respondents saw the mass incivility stimulus, 35 were exposed to mass civility and 43 were in the elite
incivility group.
5 Randomisation checks showed there was random allocation of the demographic variables across conditions.
See Table A1.
(2017). Principal component analysis, with Direct Oblimin rotation, was conducted to determine the factor structure of both sets of items. One factor was extracted for the general intended political engagement items, explaining 42.22% of the variance in the 10 items.7 The scale was reliable, α = .87 (M = 4.41, SD = 1.15). One factor was extracted for the EU engagement items, explaining 55.08% of the variance, creating a reliable scale, α = .86 (M = 4.54, SD = 1.53).8
Disgust
Disgust was measured with self-reporting. The accuracy of this method is doubted as we want to test one’s affective reactions and self-reporting necessarily requires some
cognitive assessment (Ritchie et al., 2016; Sullivan & Masters, 1987). However, emotional self-reporting is regularly-used in scientific studies (e.g. Sydnor, 2015; Vandenbroek, 2013). Participants were asked to indicate their feelings of disgust and sickness (r = .73) towards the discussion in the extract they had read, on a one-to-seven scale. The two values were
combined, with seven indicating an extreme feeling of disgust (M = 2.8, SD = 1.53).
Control Variables
Before exposure to the stimulus, participants completed a pre-test survey measuring relevant control variables. Ideology was measured using a 0-10 (left-right) scale (M = 3.44, SD = 1.76). Participants also indicated their political interest, from none at all (1) to a great deal (5) (M = 3.65, SD = 1.10). Goldstein’s (1999) conflict avoidance scale was used to measure orientation towards conflict, which has been found to be an important moderator for the effects of incivility (Mutz & Reeves, 2005; Sydnor, 2015). This scale was measured using
7
The sum of squared loadings was 4.22. Factor loadings were: .39, .75, .81, .51, .74, .53, .73, .50, .66, .76.
seven-point Likert scales and was reliable, α = .79. A score of seven suggested a high orientation towards conflict (M = 3.92, SD = 1.15).9
Results
Hypothesis 1
To test whether exposure to mass incivility provoked lower intention to engage in politics compared to exposure to mass civility, two t-tests were conducted with intended future political engagement and intended engagement with the EP campaign as dependent variables.10 Mean values for each of the experimental conditions and dependent variables can be found in Table 1 below. The findings suggest that intention to engage in the EU election campaign was significantly affected by exposure to uncivil discourse, in the expected direction. Those in the mass incivility condition had lower intention to engage in the
campaign (M = 3.89, SD = 1.57) compared to those in the mass civility condition (M = 4.75, SD = 1.64), t(63) = 2.15, p = .035, d = .69. Those exposed to mass incivility (M = 4.05, SD = 1.04) did not have a significantly lower intention to engage in non-specified future political campaigns than those in the mass civility condition (M = 4.49, SD = 1.27), t(62.88)11 = 1.53, p = .136. Hypothesis 1a is rejected and Hypothesis 1b is confirmed.
Hypothesis 2
To test whether exposure to mass incivility reduced intention to engage in politics compared to elite incivility, two t-tests were conducted. With regards general future
engagement, those exposed to mass incivility (M = 4.05, SD = 1.04) had significantly lower
9
These control variables were successfully randomly assigned across the three experimental conditions. Figures for the association of experimental condition to control and demographic variables can be found on Table A1 in Appendix A.
10 A MANOVA with the two dependent variables may have been conducted, but the small number of
participants and unequal distribution of respondents meant some of the assumptions for MANOVA tests were not met.
intention to engage than those exposed to elite incivility (M = 4.60, SD = 1.68), t(69.32)12 = 2.01, p = .048, d = .50. Hypothesis 2a is confirmed. The depressing effect of mass incivility was particularly prominent in relation to the EP campaign, with a mean difference of .93 (SE = .35), t(71) = 2.77, p = .007, d = .77.13 Hypothesis 2b is confirmed.
Fig. 1: Mean Values (with standard error bars) for Intended Political Engagement and Engagement
with the Upcoming EP Elections
Notes: The mean difference between Mass Incivility and Elite Incivility for Political Engagement (p =
.048) is significant. For EU Political Engagement, the mean differences between Mass Incivility and Mass Civility (p = .035) and Mass Incivility and Elite Incivility (p = .007) are significant.
Hypothesis 3
According to the findings, exposure to incivility can affect intention to become involved in politics, in particular scenarios, and the source of this incivility matters. I posited that the affective mechanism of disgust might explain this relationship. First, I hypothesised that those subject to mass incivility would indicate higher levels of disgust towards the tone of the discussion in the article than those reading the article with a civil discussion. This proved true, as can be seen in Table 1, t(63) = 3.85, p < .001, with a very large effect size of d = .97. I also hypothesised that the levels of disgust would be higher for those exposed to mass
12
The assumption of equality of variances was not met.
13 See Table 1 for Mean values.
2 3 4 5 6
Political Engagement EU Political Engagement Mass Incivility Mass Civility Elite Incivility
incivility as opposed to elite incivility. This was not the case, MDIFF = .04, t(71) = .1, p = .917, despite source of incivility affecting intended political engagement. Hypothesis 3 is
confirmed, but Hypothesis 4 is rejected.
Table 1: Means and Standard Deviations in parentheses for each Dependent Variable according to
Experimental Condition
Experimental Condition Dependent Variable
Intended Political Engagement Intended EU Political Engagement Disgust Mass Incivility 4.05 (1.04) a 3.89 (1.57) a b 3.18 (1.40) b Mass Civility 4.49 (1.27) 4.75 (1.64) a 1.94 (1.19) b c Elite Incivility 4.60 (1.28) a 4.81 (1.29) b 3.22 (1.59) c Total 4.41 (1.15) 4.54 (1.53) 2.80 (1.53)
Notes: a Significant difference, p < .05, b significant difference, p < .01, c significant difference, p < .001
The rejection of Hypothesis 4 means Hypothesis 6 must be rejected. With no relationship between the source of incivility and disgust, there can be no mediation of the observed relationship between source and future political engagement. Also, there was no observed effect of exposure to incivility on general intended political engagement, so
Hypothesis 5a is rejected. The only model on which I can test the mediating effect of disgust is the relationship between exposure to incivility and intention to engage in the EU elections. A mediation model (Model 4) was estimated using PROCESS analysis (Hayes, 2018), conducted with 5000 bootstrap samples, with exposure to incivility as independent variable, the intention to engage with the EP elections as the dependent variable and disgust as the mediator. The results show no mediation (indirect = .29, boot SE = .24, 95% BCI [-.11, .82]). Hypothesis 5b is rejected.
I conducted model variation tests to examine if the given models are consistent under variability, using different subsamples. Conducting the tests with the original respondent pool, including those who failed the manipulation check, there is a main effect of exposure to incivility on general intended political engagement, EU engagement and disgust, but there is no effect of source of incivility on any of the dependent variables. This stands to reason as the sources were not as distinct if we take the results of the larger participant pool. I conducted the same tests with ten random selections of 70% of the final respondent pool. This
necessarily limits the statistical power of the tests, nevertheless, the effect of exposure to incivility on disgust was found in all ten cases. The other observed models did not hold in all cases.14
Theoretically, I would expect that the effect of incivility has most impact for those who are not interested in politics. These people are less likely to be habitually engaged in the political process (Kahn & Kenney, 1999; Torcal & Maldonado, 2014). Those with low political interest would then be more likely to change their intention to engage in the face of the threat of incivility. However, conducting the same tests with only those with low political interest shows no replication of the observed models.15 This appears not to be down to the smaller sample size, but the mean values per condition follow different trends to what we saw in the original analyses. The observed effects are only present for high-interest people, which seems to run in the face of the theory and challenges the robustness of the present findings.
Conclusion
This paper has found people who read an uncivil political discussion between two ordinary citizens are significantly less inclined to engage in the political process, compared to
14 The effect of source of incivility on intended EU engagement was replicated in six of the ten tests. The effect
of source on general intended engagement and exposure to incivility on intended EU engagement were both seen in three of the ten tests.
those who read the same discussion but with a more civil tone. This effect was limited to the EP elections in May 2019. When distinguishing between mass incivility and elite incivility, those exposed to the former had significantly lower intention to engage in politics, for both the EP elections and all future political campaigns. How these relationships occur is not clear from the findings. While incivility, compared to civility, did produce more disgust, this reaction did not explain intention to engage in politics, nor did it differ according to the source of incivility. According to the robustness checks, the effects found are not stable under variability, and they are limited to those with high political interest. This limits their
generalisability. Due to the nature of the sample, and further limitations to the research which will be discussed, this is only a preliminary analysis into the importance of incivility,
particularly mass incivility, in European politics. It has, nonetheless, produced results worthy of consideration.
We see here that incivility can be an influential mechanism in European politics. Incivility is part of political discourse in Europe (Harris, 2001; Jenny et al., 2017), and is maybe growing with the rise of populism and its rhetoric (Heinisch, 2003; Nai, 2018; Rooduijn, 2014; Wodak, 2013). The findings here suggest that exposure to incivility can affect people’s political behaviour and deserves continued study. Providing evidence that politics is a venue of disrespectful conflict can make people avoid it altogether. The effect, however, is limited to the second-order EP elections. The threat of incivility may be worth facing when the political campaign is important, but not in the case of low-stakes elections. It has been argued that one way to boost the salience of the EU elections is for media to
highlight conflict in policy debate (Schuck, Vliegenthart & de Vreese, 2014). These results suggest that extra efforts need to be made to make these campaigns palatable for voters, if the goal is to increase engagement, or, at least, avoid disengagement. It may also be that
case, rather than referring to prospective political engagement, with no given timeframe, made the risk of being subject to incivility, if one were to engage in the process, more relevant and threatening. While the demobilising effect of incivility is only partially supported here, there is no evidence for Brooks and Geer’s (2007) argument that incivility can mobilise the electorate.
For scholars to further understand the impact of incivility, an understanding of the source of incivility is important. The findings here show that mass incivility can depress intention to engage in politics more than exposure to elite incivility, and this is not limited to the EP elections. Elite incivility did not depress intended political engagement any more than mass civility. I suggested that the role of source is important because of the different social norms that may apply. In the low-stakes political discussion of ordinary citizens, pointed and personal attacks would be considered more excessive, and suggestive that politics, not just politicians, is uncivil. However, the elite incivility was just as uncivil and disgusting as mass incivility. Then I suggest that mass incivility poses a more direct risk to people if they are to engage in politics as opposed to the incivility of political elites. The fact that the results were observed for only those highly-interested in politics, those most likely to engage in politics, may back this up, as they would be most at risk to the incivility. What goes on around us, within our social networks and between citizens whose discussions we are exposed to, matters when we make the decision to engage in politics (Levinsen & Yndigegn, 2015; McClurg, 2006; Mutz, 2002; Torcal & Maldonado, 2014). How ordinary citizens conduct their politics is more relevant to our own prospective political engagement than what
politicians do. Future research should incorporate the communicative effects of the ordinary citizen in other communicative processes in politics such as persuasiveness, negativity and anti-democratic rhetoric.
Exactly how mass incivility works to reduce one’s intention to engage in politics is not clear from this study. The gut reaction triggered by the rhetorical tone did not concern the source of incivility, as I expected it would. It was the cognitive thought of intention to engage in politics that differed according to the source. I argue that the participants’ disgust towards the politicians’ rhetoric was less relevant in their decision to engage in future politics. On the other hand, disgust may not be an important emotion in politics. Making the decision to engage in politics and determining political attitudes are highly-cognitive considerations, far-removed from gut reactions. Certainly, its effects are not uniform according to how it is elicited. Before disregarding the idea that disgust, or another emotional reaction, can explain the political effects of incivility, forthcoming research should use more sophisticated methods to measure emotional response (Scherer, 2005).
We are often exposed to the political discourse of ordinary citizens. Through the internet we have a direct access to mass debate. On radio phone-ins we hear people like us debate issues. In our daily lives, we can see and be involved in political discussions. Moreover, the use of citizen sources in news is increasing (Kleemans, Schaap & Hermans, 2017), and this is more prevalent closer to a public vote (Hopmann & Shehata, 2011). If this discourse turns uncivil, then there can be an impact on public political engagement. The realities of social media (Papacharissi, 2004; Rains et al., 2017) and traditional media (Funk, 2001; Harcup & O’Neill, 2017; Mutz, 2006, 2007) mean that incivility tends to get
emphasised. This is especially concerning when it comes to the European elections. There are consequences for public political engagement to the editorial decisions of media, and
communicative choices of politicians and ordinary citizens alike. When we choose to be uncivil in political discussion this can make politics unpalatable to those around us.
According to the findings here, elite incivility does not decrease intention to engage in politics any more than mass civility. However, it can have indirect negative effects. Gervais (2017) has found that partisans imitate their elites’ uncivil rhetoric in their own discourse. Some political commentators have defended incivility (e.g. Cooper, 2018; Signorile, 2018), especially in USA, as a way to challenge and counteract the rhetoric of President Trump. While it may work to counter the obnoxious demagoguery of some political leaders, because they do not listen to or engage with civil discussion, this tactic can turn people off politics if it becomes a part of mass political discourse.
There are questions that emerge from this analysis to guide future research, some of which have already been mentioned. Ideally, the effects of exposure to political incivility could be tested using a panel study, because in the real world, our exposure is re-occurring. I expect the effects to be more long-lasting and powerful after re-occurrence (Lau, Sigelman & Brown Rovner, 2007; Sobieraj & Berry, 2011). Subsequent studies can also work to
understand the personal differences in the effects and sources of incivility. Who exactly is pushed away from, or even enticed towards, political engagement when exposed to incivility, and does this change according to the message sender?
Perhaps a more pertinent question for future research is why exactly mass incivility has a negative effect on intention to engage in politics compared to elite incivility? I observed that there was no difference in feelings of disgust towards their respective incivility, so it is not to do with contextual norms. I have put forward the explanation of relevance. This would suggest that what ordinary citizens do is a much more important consideration than elite processes in understanding political behaviour, and maybe attitudes. This theory could be confirmed or rejected in future research. The effect of uncivil rhetoric, and the relevance of this uncivil threat, at different levels of politics and social closeness could be tested. It may be
that using made-up European politicians to represent political elites increased the distance between ordinary citizens and elites and reduced the relevance of the elite incivility. I would suggest for future studies to use real local politicians, rather than national politicians because they are more well-known and would provide clear ideological cues, in their stimuli. This would limit the social and geographical distance, between ordinary citizens and politicians, and will tell us if the incivility of citizens has a greater negative effect than the incivility of politicians at all levels. The effect of the source of incivility was observed for individuals that were unknown to the participants, and who could have feasibly been from all over the EU. We might expect the incivility of friends, acquaintances, even people known to be from one’s country, to have an even greater negative effect on intention to engage. The political actions of those in our close social network are particularly influential in our future political
behaviour (Levinsen & Yndigegn, 2015; McClurg, 2006). If the relevance of the threat does explain the importance of the source observed here, then the incivility of peers and friends is likely to turn us off political engagement even more than simply “mass incivility”. We could then extend the importance of social closeness and relevance to more political processes.
There are a number of limitations to this study, some of which have already been addressed. Primarily, it is difficult to comprehensively trust the data analysis. Robustness checks suggest that the models do not hold under variability. I would have expected the findings to hold for participants with low political interest, but they were only present for those with high political interest. This is an important caveat to the observed findings, which is made bigger by the fact that the respondent pool was relatively politically interested. Perhaps these findings are not generalisable to the wider population, but only to those with high political interest. A large proportion of the sample (41.62%) failed the manipulation check according to the source of incivility. The potential reasons for this have already been
deliberated, but it does suggest that the stimulus was not believable for many respondents, which necessarily limits any generalisability.
The effects of incivility were studied here using a niche issue, the CAP, news of which people outside of the farming community rarely seek out. This was done to limit ideological cues and the role of prior opinion but does reduce the external validity. I would suggest that if scholars are to tackle this question again, they use stimuli that cover a range of political issues. For many people, particularly those not interested in politics, discussion regarding agricultural policy is not relevant, so incivility in this discussion will not affect their intention to engage in politics. Perhaps this is why we see the effects are limited to those with high political interest, as they are more likely to understand the salience and relevance of the CAP budget. Using multiple issues, of both high and low salience, will make the incivility relevant to all participants, making the results more reliably generalisable, while still limiting the effect of prior opinion, which alters the effects of incivility (Anderson et al., 2014; Brooks & Geer, 2007; Gervais, 2017).
Despite the limitations, findings that have real-world implications were observed. Incivility is present in European political discourse and may be growing. The concerns of some political commentators (e.g. Herbst, 2010; Lecheler, 2013; Lewis, 2018) regarding civility, or lack thereof, in public discourse seem to be well-founded. This is not because incivility between politicians turns us off politics, but because it can become an element of popular discourse. This then provides a relevant threat of disrespect to people who might consider getting involved in politics. As a result, those exposed to incivility in mass discourse can become apathetic. This applies for the more minor manifestations of incivility that we can be exposed to every day. These results provide clear implications for how we all choose to conduct our politics.
References
Anderson, A. A., Brossard, D., Scheufele, D. A., Xenos, M. A., & Ladwig, P. (2014). The “Nasty Effect”: Online Incivility and Risk Perceptions of Emerging Technologies. Journal of Computer-Mediated Communication, 19(3), 373–387. DOI:
10.1111/jcc4.12009.
Anderson, A. A., Yeo, S. K., Brossard, D., Scheufele, D. A., & Xenos, M. A. (2018). Toxic Talk: How Online Incivility Can Undermine Perceptions of Media. International Journal of Public Opinion Research, 30(1), 156-168. DOI:10.1093/ijpor/edw022.
Andersson, L., & Pearson, C. (1999). Tit for tat? The spiraling effect of incivility in the workplace. Academy of Management Review, 24(3): 452–471. DOI:
10.5465/amr.1999.2202131.
Ansolabehere, S., & Iyengar, S. (1995). Going negative: How attack ads shrink and polarize the electorate. New York: Free Press.
Arceneaux, K., & Nickerson, D. W. (2010). Comparing Negative and Positive Campaign Messages. American Politics Research, 38(1), 54–83.
DOI:10.1177/1532673x09331613.
Barrett, M., & Zani, B. (2015). Political and civic engagement: Multidisciplinary perspectives. London: Routledge.
Benson, T. (2011). The rhetoric of civility: Power, authenticity, and democracy. Journal of Contemporary Rhetoric, 1(1), 22–30.
Borah, P. (2013). Interactions of News Frames and Incivility in the Political Blogosphere: Examining Perceptual Outcomes. Political Communication, 30(3), 456-473, DOI: 10.1080/10584609.2012.737426.
Bradley, M. M., & Lang, P. J. (2007). Emotions and Motivation. In J. T. Cacioppo, L. G. Tassinary & G. G. Berntson (Eds.), Handbook of Psychophysiology (3rd ed.) (pp. 581-607). Cambridge, UK: Cambridge University Press.
Brooks, D. J., & Geer, J. G. (2007). Beyond negativity: The effects of incivility on the electorate. American Journal of Political Science, 51(1), 1-16. DOI: 10.1111/j.1540-5907.2007.00233.x.
Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage. Cambridge, UK: Cambridge University Press.
Clinton, J. D., & Lapinski, J. S. (2004). Targeted Advertising and Voter Turnout: An
Experimental Study of the 2000 Presidential Election. The Journal of Politics, 66(1), 69-96. DOI: 10.1046/j.1468-2508.2004.00142.x.
Cooper, R. (2018, June 25). In defense of incivility. The Week. Retrieved from:
https://theweek.com/articles/780998/defense-incivility.
Delli-Carpini, M. X., Cook, F. L., & Jacobs, L. R. (2004). Public deliberation, discursive participation and citizen engagement: A review of the empirical literature. Annual Review of Political Science, 7(1), 315-344. DOI:
10.1146/annurev.polisci.7.121003.091630.
Delli-Carpini, M. X., & Keeter, S. (1996). What Americans Know About Politics and Why it Matters. New Haven: Yale University Press.
Djupe, P. A., & Peterson, D. A. M. (2002). The Timing and Impact of Negative
Campaigning: Evidence from the 1998 Senatorial Primaries. Political Research Quarterly, 55(4), 845–860. DOI:10.1177/106591290205500406.
Durkheim, E. (1960). The Division of Labour in Society (G. Simpson, Trans.). Glencoe, Ill.: The Free Press of Glencoe.
European Commission. (2016). Special Eurobarometer 440: Europeans, Agriculture and the CAP. Retrieved from:
http://ec.europa.eu/commfrontoffice/publicopinion/index.cfm/Survey/getSurveyDetail /yearFrom/1974/yearTo/2016/search/cap/surveyKy/2087.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. DOI:10.3758/brm.41.4.1149.
Franklin, M. N. (2014). Why vote at an election with no apparent purpose? Voter turnout at elections to the European Parliament. Swedish Institute for European Policy Studies, 2014(4). Retrieved from:
http://www.sieps.se/en/publications/2014/why-vote-at-an- election-with-no-apparent-purpose-voter-turnout-at-elections-to-the-european-parliament-20144epa/.
Franklin Fowler, E., Lawler, P., Linden, M., Loomis, E., Wulderk, Z., & Baum, L. (2014). A Messenger Like Me: The Effect of Average Spokespeople in Campaign Advertising. Paper prepared for the Annual Meeting of the American Political Science Association Conference, Washington D.C., 30 August 2014. Retrieved from:
https://ssrn.com/abstract=2453472.
Funk, C. L. (2001). Process Performance: Public Reaction to Legislative Policy Debate. In J. R. Hibbing and E. Theiss-Morse (Eds.), What is it About Government that Americans Dislike (pp. 193-204). Cambridge, UK: Cambridge University Press.
Geer, J. G. (2012). The News Media and the Rise of Negativity in Presidential Campaigns. PS, 45(3), 422–427. DOI: 10.1017/S1049096512000492.
Gervais, B. T. (2014). Following the News? Reception of Uncivil Partisan Media and the Use of Incivility in Political Expression. Political Communication, 31(4), 564-583, DOI: 10.1080/10584609.2013.852640.
Gervais, B. T. (2017). More than Mimicry? The Role of Anger in Uncivil Reactions to Elite Political Incivility. International Journal of Public Opinion Research, 29(3), 384-405. DOI: 10.1093/ijpor/edw010.
Goldstein, S. (1999). Construction and Validation of a Conflict Communication Scale. Journal of Applied Social Psychology, 29(9), 1803-1832. DOI: 10.1111/j.1559-1816.1999.tb00153.x.
Gotlieb, J. B., & Sarel, D. (1991). Comparative advertising effectiveness: the role of involvement and source credibility. Journal of Advertising, 20(1), 38–45. DOI: 10.1080/00913367.1991.10673205.
Green, M. C., Visser, P. S., & Tetlock, P. E. (2000). Coping with Accountability Cross-Pressures: Low-Effort Evasive Tactics and High-Effort Quests for Complex Compromises. Personality and Social Psychology Bulletin, 26(11), 1380–1391. DOI:10.1177/0146167200263006.
Harris, S. (2001). Being Politically Impolite: Extending Politeness Theory to Adversarial Political Discourse. Discourse & Society, 12(4), 451–472. DOI:
10.1177/0957926501012004003.
Hartman, T. K., & Weber, C. R. (2009). Who said what? The effects of source cues in issue frames. Political Behavior, 31(4), 537–558. DOI: 10.1007/s11109-009-9088-y. Hayes, A. F. (2018). Introduction to Mediation, Moderation, and Conditional Process
Analysis: A Regression-based Approach (2nd Ed.). London: The Guildford Press. Heinisch, R. (2003). Success in Opposition–Failure in Government: Explaining the
Performance of Right-Wing Populist Parties in Public Office. West European Politics, 26(3), 91–130. DOI: 10.1080/01402380312331280608.
Herbst, S. (2010). Rude Democracy: Civility and Incivility in America. Philadelphia: Temple University Press.
Hopmann, D. N., & Shehata, A. (2011). The contingencies of ordinary citizen appearances in political television news. Journalism Practice, 5(6), 657–671. DOI:
10.1080/17512786.2011.603967.
Hwang, H, Kim, Y., & Huh, C. U. (2014). Seeing is Believing: Effects of Uncivil Online Debate on Political Polarization and Expectations of Deliberation. Journal of Broadcasting & Electronic Media, 58(4), 621-633. DOI:
10.1080/08838151.2014.966365.
Jamieson, K., Volinsky, A., Weitz, I., & Kenski, K. (2017). The Political Uses and Abuses of Civility and Incivility. In K. Kenski & K. H. Jamieson (Eds), The Oxford Handbook of Political Communication. Oxford: Oxford University Press. Retrieved from:
http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199793471.001.0001/ oxfordhb-9780199793471-e-79.
Jenny, M., Haselmayer, M., Rudkowsky, E., Wastian, M., Emrich, S., & Sedlmair, M. (2017). Incivility in Austrian parliamentary debates: A supervised sentiment analysis of parliamentary speeches. Paper prepared for the Workshop Political Incivility in Parliament, Party Competition and Political Communication, ECPR Joint Sessions of Workshops, Nottingham, 25-30 April 2017. Retrieved from:
https://www.researchgate.net/publication/316668097_Incivility_in_Austrian_parliame ntary_debates_A_supervised_sentiment_analysis_of_parliamentary_speeches.
Kahn, K. F., & Kenney, P. J. (1999). Do Negative Campaigns Mobilize or Suppress Turnout? Clarifying the Relationship between Negativity and Participation. The American Political Science Review, 93(4), 877-889. DOI: 10.2307/2586118.
Kleemans, M., Schaap, G., & Hermans, L. (2017). Citizen sources in the news: Above and beyond the vox pop? Journalism, 18(4), 464–481. DOI: 10.1177/1464884915620206. Lau, R. R., & Pomper, G. M. (2001). Effects of negative campaigning on turnout in U.S.
Senate elections, 1988-1998. The Journal of Politics, 63(3), 804-819. DOI: 10.1111/0022-3816.00088.
Lau, R. R., Sigelman, L., & Brown Rovner, I. (2007). The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment. The Journal of Politics, 69(4), 1176-1209. DOI: 10.1111/j.1468-2508.2007.00618.x.
Lecheler, S. (2013). Be nice! How Incivility in Media and Politics May Influence Our Society. Retrieved from:
https://student.uva.nl/bcw/content/contentgroepen/commpulse/opiniestukken/2013/be- nice-how-incivility-in-media-and-politics-may-influence-our-society-by-sophie-lecheler.html.
Lefevere, J., De Swert, K., & Walgrave, S. (2012). Effects of Popular Exemplars in Television News. Communication Research, 39(1), 103–119. DOI:
10.1177/0093650210387124.
Levinsen, K., & Yndigegn, C. (2015). Political discussions with family and friends: exploring the impact of political distance. The Sociological Review, 63(2), 72–91. DOI:
10.1111/1467-954X.12263.
Lewis, H. (2018, August 29). How Britain’s political conversation turned toxic. New
Statesman. Retrieved from: https://www.newstatesman.com/politics/uk/2018/08/how-britain-political-conversation-turned-toxic.
MacKuen, M., Wolak, J., Keele, L., & Marcus, G. E. (2010). Civic engagements: Resolute partisanship or reflective deliberation. American Journal of Political Science, 54(2), 440-458. DOI: 10.1111/j.1540-5907.2010.00440.x.
Martin, P. S. (2004). Inside the Black Box of Negative Campaign Effects: Three Reasons Why Negative Campaigns Mobilize. Political Psychology, 25(4), 545-562. DOI:10.1111/j.1467-9221.2004.00386.x.
McClurg, S. D. (2006). Political Disagreement in Context: The Conditional Effect of Neighborhood Context, Disagreement and Political Talk on Electoral Participation. Political Behavior, 28(4), 349–366. DOI: 10.1007/s11109-006-9015-4.
Muddiman, A. R. (2013). The Instability of Incivility: How News Frames and Citizen
Perceptions Shape Conflict in American Politics (Unpublished doctoral dissertation). University of Texas at Austin, USA.
Mutz, D. C. (2002). The Consequences of Cross-Cutting Networks for Political Participation. American Journal of Political Science, 46(4), 838–855. DOI: 10.2307/3088437.
Mutz, D. C. (2006). Hearing the Other Side: Deliberative versus participatory democracy. New York: Cambridge University Press.
Mutz, D. C. (2007). Effects of “In-Your-Face” Television Discourse on Perceptions of a Legitimate Opposition. American Political Science Review, 101(4), 621–635. DOI: 10.1017/s000305540707044x.
Mutz, D. C., & Reeves, B. (2005). The New Videomalaise: Effects of televised incivility on political trust. American Political Science Review, 99(1), 1-16. DOI:
10.1017/s0003055405051452.
Nai, A. (2018). Fear and Loathing in Populist Campaigns? Comparing the communication style of populists and non-populists in elections worldwide. Journal of Political Marketing, 1-32. DOI: 10.1080/15377857.2018.1491439.
Papacharissi, Z. (2004). Democracy Online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259–283. DOI: 10.1177/1461444804041444.
Pe’er, G., Lakner, S., Müller, R., Passoni, G., Bontzorlos, V., Clough, D.,… Zinngrebe, Y. (2017). Is the CAP Fit for Purpose? An evidence-based fitness assessment. Retrieved from: http://extranet.greens-efa-service.eu/public/media/file/1/5401.
Petersen, M. B. (2010). Distinct Emotions, Distinct Domains: Anger, Anxiety and Perceptions of Intentionality. The Journal of Politics, 72(2), 357-365. DOI: 10.1017/s002238160999079x.
Phillips, T., & Smith, P. (2003). Everyday Incivility: Towards a Benchmark. The Sociological Review, 51(1), 85–108. DOI: 10.1111/1467-954X.00409.
Prior, M. (2010). You’ve Either Got It or You Don’t? The Stability of Political Interest over the Life Cycle. The Journal of Politics, 72(3), 747-766.
DOI:10.1017/s0022381610000149.
Rains, S. A., Kenski, K., Coe, K., & Harwood, J. (2017). Incivility and Political Identity on the Internet: Intergroup Factors as Predictors of Incivility in Discussions of News Online. Journal of Computer-Mediated Communication, 22(4), 163–178. DOI: 10.1111/jcc4.12191.
Reif, K., & Schmitt, H. (1980). Nine Second-Order National Elections – A Conceptual Framework for the Analysis of European Election Results. European Journal of Political Research, 8(1), 3–44. DOI: 10.1111/j.1475-6765.1980.tb00737.x.
Ritchie, S. M., Hudson, P., Bellocchi, A., Henderson, S., King, D., & Tobin, K. (2016). Evolution of Self-Reporting Methods for Identifying Discrete Emotions in Science Classrooms. Cultural Studies of Science Education, 11(3), 577-593. DOI:
10.1007/s11422-014-9607-y.
Rooduijn, M. (2014). The Mesmerising Message: The Diffusion of Populism in Public Debates in Western European Media. Political Studies, 62(4), 726–744. DOI: 10.1111/1467-9248.12074.
Rossini, P. G. C. (forthcoming). Disentangling Uncivil and Intolerant Discourse. In R. Boatright, T. Shaffer, S. Sobieraj & D. Young, (Eds.). A Crisis of Civility? Contemporary Research on Civility, Incivility, and Political Discourse. London: Routledge.
Rowe, I. (2015). Civility 2.0: a comparative analysis of incivility in online political discussion. Information, Communication & Society, 18(2), 121-138. DOI: 10.1080/1369118X.2014.940365.
Rozin, P, Haidt, J., & McCauley, C. R. (2016). Disgust. In L. F. Barrett, M. Lewis & J. M. Haviland-Jones (Eds.) Handbook of Emotions (4th Ed.) (pp. 815-834). New York: The Guilford Press.
Sapiro, V. (1999). Considering Political Civility Historically: A Case Study of the United States. Presented at the annual meeting of the International Society for Political Psychology in Amsterdam, The Netherlands, 1999. Retrieved from:
http://blogs.bu.edu/vsapiro/files/2018/06/Considering-Civility.pdf.
Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695-729. DOI: 10.1177/0539018405058216.
Schnall, S., Haidt, J. & Clore, G. L., & Jordan, A. H. (2008). Disgust as Embodied Moral Judgment. Personality and Social Psychology Bulletin, 34(8), 1096-1109. DOI: 10.1177/0146167208317771.
Schuck, A. R., Vliegenthart, R., & De Vreese, C. H. (2016). Who's afraid of conflict? The mobilizing effect of conflict framing in campaign news. British Journal of Political Science, 46(1), 177-194. DOI: 10.1017/s0007123413000525.
Schudson, M. (1997). Why Conversation is Not the Soul of Democracy. Critical Studies in Mass Communication, 14(4), 1–13. DOI: 10.1080/15295039709367020.
Sendelbah, A., Vehovar, V., Slavec, A., & Petrovčič, A. (2016). Investigating respondent multitasking in web surveys using paradata. Computers in Human Behavior, 55, 777-787. DOI: 10.1016/j.chb.2015.10.028.
Signorile, M. (2018, June 27). Fuck Civility. Huffington Post. Retrieved from:
https://www.huffingtonpost.com/entry/opinion-signorile-civility-trump_us_5b31ad0de4b0b5e692f0c7b8.
Sobieraj, S., & Berry, J. M. (2011). From Incivility to Outrage: Political Discourse in Blogs, Talk Radio, and Cable News. Political Communication, 28(1), 19-41. DOI:
Strohmeier, D., Barrett, M., Bora, C., Caravita, S. C. S., Donghi, E., Dragoti, E.,… Viejo, C. (2017). Young People’s Engagement with the European Union. Zeitschrift für
Psychologie, 225(4), 313-323. DOI: 10.1027/2151-2604/a000314.
Sullivan, D. G., & Masters, R. D. (1987). "Gut Reactions" and the Political Effects of the Media, PS, 20(4), 880-889. DOI: 10.2307/419251.
Sveningsson, M. (2014). “I don’t like it and I think it’s useless, people discussing politics on Facebook”: Young Swedes’ understandings of social media use for political
discussion. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 8(3). DOI: 10.5817/cp2014-3-8.
Sydnor, E. (2015). Fighting Words and Fiery Tone: The Interaction of Political Incivility and Psychological Conflict Orientation (Unpublished doctoral dissertation). University of Virginia, USA.
Torcal, M., & Maldonado, G. (2014). Revisiting the Dark Side of Political Deliberation: The Effects of Media and Political Discussion on Political Interest. Public Opinion Quarterly, 78(3), 679-706. DOI: 10.1093/poq/nfu035.
van Deth, J., & Martin, I. (2007). Political Involvement. In J. van Deth, J. R. Montero & A. Westholm (Eds.). Citizenship and involvement in European democracies: A
comparative analysis (pp. 303-333). London: Routledge.
Vandenbroek, L. M. (2012). Principled Abstention: A Theory of Emotions and Nonvoting in U.S. Presidential Elections (Unpublished doctoral dissertation). University of Texas at Austin, USA.
Verba, S., Schlozman, K. L., & Brady, H. E. (1995). Voice & Equality: Civic Voluntarism in American Politics. Cambridge, MA.: Harvard University Press.
Vliegenthart, R., & Boukes, M. (2018). On the Street and/or on Twitter? Digital Journalism, 6(7), 829-846. DOI: 10.1080/21670811.2018.1497449.
Wodak, R. (2013). ‘Anything Goes!’ – The Haiderization of Europe. In R. Wodak, M. Khosravinik & B. Mral (Eds.). Right-Wing Populism in Europe: Politics and Discourse (pp. 23–38). London: Bloomsbury Academic.
Zaller, J. (1992). The nature and origins of mass opinion. New York: Cambridge University Press.
Zillmann, D., & Brosius, H. (2000). Exemplification in Communication: The influence of case reports on the perception of issues. Mahwah, NJ: Lawrence Erlbaum Associates.
