Discrete emotions and Scottish independence: A content analysis of newspapers’ use of emotion in coverage of the 2014 Scottish independence referendum
Georgia Clyne
12744883 Master’s Thesis
Graduate School of Communication Master’s programme Communication Science
Supervisor: Dr Andreas Schuck
Abstract
This thesis investigates the use of emotions within newspaper coverage of the run up to the 2014 Scottish independence referendum in Scottish and UK-wide publications. Content analysis was conducted on articles (N = 377) published in seven newspapers in the four months leading up to the referendum date to identify their use of eight key emotions, using the NRC Word-Emotion Association Lexicon (Mohammad and Turney, 2010, 2013a). The use of emotion in relation to the presence of key political and economic figures, and key issues within articles was also tested. Based upon literature on the importance of the media in referenda (de Vreese and Semetko, 2004; de Vreese and Schuck, 2014; Schuck and de
Vreese, 2009), on discrete emotions (Druckman and McDermott, 2008; Kühne, 2012; Lecheler, Schuck and de Vreese, 2013; Lerner and Keltner, 2000; Nabi, 1999, 2002, 2003, 2010), and the role of emotions in referenda (Garry, 2014; Vasilopoulou and Wagner, 2017), this paper seeks to fill a gap in the literature on the presence of discrete emotions in
newspaper coverage of the 2014 Scottish independence referendum. Through the use of chi-square tests and independent samples t-tests, this paper found that anticipation, trust, anger and fear were the most used emotions in Scottish and UK-wide articles, which did not differ across time nor by publication.
Keywords: emotions, content analysis, Scottish independence, referendum, newspapers
Introduction
The media are, according to Ansolabehere, Behr and Iyengar (1991), a “crucial instrument of political campaigns” (p. 109), including in referendums (de Vreese and Semetko, 2004; de Vreese and Schuck, 2014; Schuck and de Vreese, 2009). As referenda often offer two alternatives about an issue citizens may not be well informed on (de Vreese and Schuck, 2014), the media may play an important role in these “contested contexts” (de Vreese and Schuck, 2014, p. 138), but how?
One way is through the use of emotion, for example in news stories (Schemer, 2014), including through framing (Lecheler, Schuck and de Vreese, 2013; Nabi, 2003), and in text (Brader, 2005). Political campaigns offer a prime opportunity for the use of emotional “campaign speeches, advertisements, conventions…” (Glaser and Salovey, 1998, p. 156), designed to spark a reaction. Therefore, this paper seeks to identify the use of emotion in newspaper coverage of the 2014 Scottish independence referendum.
On September 18 2014, Scotland voted to remain within the United Kingdom (UK) with 55.3% (Morisi, 2016) of the vote. The notion of “autonomous policymaking” (McGarvey, 2015, p. 35) has been deliberated since the 1990s since the formation of the Scottish Parliament, but became a real possibility with the 2011 electoral success of the Scottish National Party (SNP), who “included a commitment to an independence referendum” (McGarvey, 2015, p. 36) in their manifesto. This gave them the right to hold the referendum (Mullen, 2014), granted by the signing of the Edinburgh Agreement in October 2012 (Lynch, 2015) by the Scottish and UK governments, which asked the Scottish people: “Should
Scotland be an independent country?” (Dekavalla, 2018, p. 1593).
Despite their 2011 majority, the SNP had to gather support for independence, not just their ability to govern (Mitchell, 2016). At the start of the campaign, most opposed independence
(Dekavalla, 2018), which fluctuated (Morisi, 2016) as the referendum date approached as some “opinion polls showed a narrow split between Yes and No” (Dekavalla, 2018, p. 1593), with a Yes success predicted at one point (McGarvey, 2015; Mitchell, 2016).
The official campaign included Yes Scotland, supporting independence and consisting of the SNP, “Scottish Green Party and the Scottish Socialist Party” (Dekavalla, 2016a, p. 796). Better Together, encompassing “Scottish Labour, Conservative and Liberal Democrat parties” (Dekavalla, 2016a, p 796), backed the union.
Regarding the role of the media, Clancy (2020) argued there was a “struggle over the meaning of the referendum and its representation” (p. 499) as different newspapers focused on different issues (Dekavalla, 2018), with only one supporting independence (Haggerty, 2014; Law, 2015; Walker, 2018). In addition, Mitchell (2016) argued that even newspapers who did not take a stance “often meant subtle – and sometimes not so subtle – opposition” (p. 4) to independence. Despite this, and the fact that newspapers had “declined in importance” (Mitchell, 2016, p. 4), both sides sought their support (Mitchell, 2016), arguably showing their perceived importance to campaigners.
Furthermore, in politics, “it is highly unlikely that any target of consideration is devoid of emotional content or influence” (Marcus, 2000, p. 232), therefore, this likely featured in newspaper coverage of the independence referendum. Scholars have studied discrete
emotions (Druckman and McDermott, 2008; Kühne, 2012; Lecheler et al., 2013; Lerner and Keltner, 2000; Nabi, 1999; 2002; 2003; 2010), of which “each specific emotion (a) is defined by a set of central dimensions and (b) directs cognition to address specific problems or opportunities” (Lerner and Keltner, 2000, p. 477). Thus, different discrete emotions can trigger different feelings and behaviours, and within text may “elicit an emotional response”
(Brader, 2005, p. 390). This is interesting in campaigns, as Nai (2018) said that “messages able to stir those emotions” (p. 7) may be more successful.
Regarding the SNP and the independence movement, “the press…remained wedded to cynicism” (Walker, 2018, p. 68), arguably evidencing their opposition, and Dekavalla (2016b) said that within newspapers the “narrative confirmed and potentially reinforced a pre-existing uncertainty about independence in the public sphere” (p. 56), which may have been reflected in the use of emotions. Furthermore, Blain and Hutchison (2016) said that some Scottish publications are “‘editionised’ Scottish versions of London-produced
newspapers” (p. 17), and Greenslade (2014) highlighted that some English newspapers tried to discourage the support of independence, therefore it is interesting to investigate whether this was reflected in the differences in emotion in Scottish and UK-wide newspapers.
Therefore, this thesis seeks to investigate the use of emotions in newspaper coverage of the 2014 Scottish independence referendum by conducting content analysis on articles from Scottish and UK-wide publications. Thus, the research questions this paper seeks to answer are:
Main RQ: Which emotions were most used in articles on Scottish independence in newspapers circulated in Scotland and England in the run up to the 2014 Scottish independence referendum and to what extent?
Sub-RQ: How did the use of the most used emotion differ over-time and by publication?
The social relevance of this topic lies with the possibility of a second independence
referendum, which the Scottish Parliament has requested (Brooks, 2019) based upon Brexit (Campbell, 2020) and the SNP winning a majority of Scottish seats in the 2019 UK General Election (Carrell, 2019). Should Scotland be granted another opportunity to let its citizens decide on its future within the UK, this will again be the focus of media attention, including
in newspapers. Therefore, readers may benefit from awareness of how publications used emotions in coverage in 2014 to appeal to them, which would allow them to critically think about the messages they read. Academically, this topic could be useful for comparison studies of the use of emotion in other referendums, for example the UK’s 2016 EU referendum, or the hypothetical second Scottish referendum. Although there have been studies on the role of emotions in referendum campaigns (Garry, 2014; Vasilopoulou and Wagner, 2017), and the 2014 Scottish independence referendum has been analysed (Clancy, 2020; Lynch, 2015; McGarvey, 2015; Mullen, 2014), including the media’s role in this (Buchanan, 2016; Blain and Hutchison, 2016; Dekavalla, 2016a, 2016b, 2018; Law, 2015; Morisi, 2016; Rosie, MacInnes, Petersoo, Condor and Kennedy, 2004; Walker, 2018), this thesis appears to fill a gap in the literature by analysing the use of emotions in newspaper coverage of this particular referendum.
Theoretical Framework
This section will introduce the theoretical framework which will be used to formulate the hypotheses and answer the research questions.
Media and Referenda
As this paper focuses on newspaper coverage of the Scottish independence referendum, the role of the media in referenda in general should be discussed. de Vreese and Schuck (2014) highlighted how referenda often require citizens to make electoral decisions on issues they know little about, which “may trigger diverse perceptions of the issue” (pp. 129-130), in which how the referendum is perceived differs. de Vreese and Schuck (2014) also argued that public opinion may change as a referendum campaign progresses, which arguably could be supported by the fact that although Yes Scotland did not win in 2014, support for
independence rose significantly to 44.7%, which Lynch (2015) argued reflected a “broad social movement” (p. 2).
As referendums tend to offer two alternatives, de Vreese and Schuck (2014) argued that the media cover both, with a “slightly more offensive stance” (p. 139) on the opposition. In the 2014 Scottish referendum, this can be supported by Blain and Hutchison (2016) who argued that pro-independence narratives could be found in columns in “the Herald, Daily Record and Scotsman” (p. 18), despite their support for the union. de Vreese and Semetko (2004) drew attention to factors which can influence decisions in referenda, including “…government approval, perceptions of the economy, issue-related contextual evaluations, political cynicism…” (p. 701), some of which are relevant in the Scottish referendum which can be supported by Dekavalla’s (2016a, 2018) findings on framing.
Dekavalla (2016a, 2018) identified what newspapers focused on, and found that outlets focused on different issues (2016a). Policy was the most used frame (2016a), encompassing the economy and others, including “defence, membership of the EU and North Atlantic Treaty Organization (NATO), public services, welfare, immigration, broadcasting” (2016a, p. 801). She also found frames on national identity, self-determination, divorce of the union and division within Scotland (2016a).
For Yes Scotland, “constructing the referendum as a decision on policy served to engage voters because it connected the referendum to things that were important to them in their daily lives” (Dekavalla, 2018, p. 1600). In contrast, the opposition focused on the economy, as polls found this would “sway people’s voting intentions, particularly in relation to the perceived risk of losing economic security” (p. 1600), supporting de Vreese and Semetko’s (2004) argument about the importance of “perceptions of the economy” (p. 701). Economic news may have focused on currency uncertainty, as the SNP sought to keep the pound,
whereas this was rejected by UK party leaders (Mitchell, 2016), which may have been reflected in the use of emotion. A preference for certainty may be supported by Dekavalla (2016b) who argued that newspapers tended to support more devolution, as this allowed for “more control within the guarantees provided by membership of the UK” (p. 55), which may also be reflected in the use of emotion. Additionally, Walker (2018) argued that coverage of Scotland was often negative “because to do otherwise would be to praise the SNP” (p. 67), thus it could be posited that SNP coverage tended to be unfavourable.
Despite a “notable” (Buchanan, 2016, p. 70) amount of online discussion about
independence, this paper is focused on newspapers. Only the now defunct Sunday Herald supported independence in 2014 (Haggerty, 2014; Law, 2015; Walker, 2018), whereas others opposed independence or did not take a stance. For example, The Herald (Herald View, 2014), The Guardian (Haggerty, 2014; The Guardian, 2014), The Independent (Haggerty, 2014; Independent, 2014), The Scotsman (Haggerty, 2014; The Newsroom, 2014) and Mirror (2014) backed the union. Although Haggerty (2014) said The Daily Record did not endorse either side, Law (2015) argued there existed a “clear pro-Union bias” (p. 3). Articles from these newspapers will be analysed in this project.
Emotions
Within these newspapers, this paper seeks to identify the use of emotion in coverage of the 2014 Scottish independence referendum. According to Marcus (2003), there exists a “difficulty of clearly categorizing emotion” (p. 187), making a definition conceptually challenging. However, Glaser and Salovey (1998) referred to emotions as “acute,
differentiated arousal” (p. 157), and Brader (2005) defined them as “underlying responses to the perceived relevance of external stimuli” (p. 390), in which emotions could be considered as the way you feel about something. These are “short-lived” (Nabi, 2010, p. 153; Schemer,
2014, p. 572), as an evoked emotion is not permanent, and are focused on something in particular (Kühne, 2012; Nabi, 2010), for example feeling happy about a job promotion. When emotion is evoked, this “arouses in the receiver an action tendency or motivation to act in accordance with the emotion-induced goal” (Nabi, 1999, p. 308), which is relevant in a political context as this goal may be a political outcome. These can be evoked through text (Brader, 2005), although different people may feel different emotions after reading the same message (Burget, Karasek and Smekal, 2011).
However, emotional stimuli must be processed before individuals react. Cognitive appraisal refers to perceptions of how something, for example an electoral contest (Schemer, 2014) impacts people (Nabi, 1999), before evoking different emotions (Kühne, 2012; Schemer, 2014). Appraisals give relevance to a stimulus “by linking it to the individual’s concerns” (Frijda, 1993, p. 358), thus sparking emotions, each of which can be appraised in a different way (Kühne, 2012; Smith and Ellsworth, 1985). For example, fear can stem from uncertainty and a perceived lack of control (Druckman and McDermott, 2008; Schemer, 2014), which can cause anxiety (Druckman and McDermott, 2008).
Furthermore, specific emotions can be further defined considering discrete emotions. According to Plutchik, these include “joy, sadness, anger, fear, trust, disgust, surprise, and anticipation” (as cited in Mohammad and Turney, 2010, p. 27), and can be evoked by “different situations and objects” (Kühne, 2012, p. 3). Each one, when evoked, is caused by differing perceptions of an issue or event, which alter the way people feel and then behave (Nabi, 2010).
For example, anger, an “approach emotion” (Nabi, 2002, p. 207) can cause “aversion” (Druckman and McDermott, 2008, p. 301), and encourage motivated reasoning (Weeks, 2015), “risk-seeking” (Druckman and McDermott, 2008, p. 317) and can lower the extent to
which risks are considered (Lerner and Keltner, 2000). Whereas an “avoidance emotion” (Nabi, 2002, p. 207), like fear, encourages people to consider risks (Lerner and Keltner, 2000) and be more wary (Druckman and McDermott, 2008), but can still “influence
attitudes” (Nabi, 2002, p. 213). Additionally, enthusiasm may lead to taking risks (Druckman and McDermott, 2008).
This thesis uses Mohammad and Turney’s (2010, 2013a) NRC Word-Emotion Association Lexicon (EmoLex), to identify word associations with Plutchik’s key emotions (as cited in Mohammad and Turney, 2010). Mohammad and Turney (2013b) justified using these as they are “…well-founded in psychological, physiological, and empirical research…” (p. 5), include positive and negative emotions, and are “a superset of the emotions proposed by some others” (p. 5). According to Mohammad and Turney (2010), “anger, fear, disgust and sadness” (p. 31) are negative, and so “anticipation, joy and trust” (p. 31) are positive.
Therefore, based on the above, and the newspapers’ lack of support for the SNP (Walker, 2018), it could be posited that when SNP politicians featured, more negative emotions were present than positive:
H1: Articles which referred to Alex Salmond, Nicola Sturgeon and the SNP were more likely to contain anger, fear and disgust, and sadness than trust and joy.
Considering Druckman and McDermott’s (2008), and Lerner and Keltner’s (2000) arguments about fear, the existence of currency uncertainty (Mitchell, 2016), and that The Sunday Herald was the only pro-independence publication (Haggerty, 2014; Law, 2015; Walker, 2018), the second hypothesis can be introduced:
H2: Economic articles in The Sunday Herald were less likely to be associated with fear than in the other six publications.
Regarding discrete emotions and referenda, Vasilopoulou and Wagner’s (2017) study on emotion and the 2016 EU referendum is particularly relevant. They posited that those who were angry were more likely to vote Leave, whilst those who felt fear were inclined to Remain, as the angry are “more confrontational and less risk-averse” (p. 386). Therefore, the same sentiment could be applied to the Scottish independence referendum, in which angry voters may have backed independence, and fearful voters may have supported the status quo. Additionally, Garry’s (2014) article on emotions and EU referendums argued that in such contexts, “it seems plausible to expect that emotions such as anger and anxiety would play an important role” (p. 248), thus it is possible these emotions were prominent within newspaper articles.
Furthermore, considering Vasilopoulou and Wagner (2017), and Garry’s (2014) studies alongside the argument that anger is a mobilising emotion (Lecheler et al., 2013), and Levy, Aslan and Bironzo’s (2016) finding that more relevant articles were published as the 2016 EU referendum day approached, it is possible that articles included more anger as both sides sought to mobilise voters. In addition, pro-union articles may have also included more fear, a “non-mobilizing” (Lecheler et al., 2013, p. 202) emotion, to encourage citizens to support the status quo. This may be supported by Schuck and de Vreese’s (2009) argument that citizens may mobilise to reject a proposal if this equates to “the prospect of unwanted change” (p. 55).
H3: Articles which supported Scottish independence were more likely to feature anger, and those supporting the union were likely to use fear.
H4: Over time, the mobilizing emotion of anger was used more often in both
independence and union articles, whereas fear became more popular only in the pro-union articles.
According to Glaser and Salovey (1998), aspects of political campaigning allow for the placement of emotional appeals, of which “affect-based persuasive messages” (Marcus, 2000, p. 232) are more successful, highlighting the importance of emotions in the political arena. For example, citizens feelings about “candidates and political messages” (Glaser and Salovey, 1998, p. 156) may help them make decisions if they cannot do so based solely on information. However, susceptibility to emotional appeals depends on “prior attitudes” (Schemer, 2014, p. 580), and the extent to which the circumstances relate to the “individual’s goals” (Schemer, 2014, p. 573).
Studying emotional political appeals is important as they can influence opinion formation by directing attention to information, thus influencing its perceived value (Brader, Marcus and Miller, 2011), or may encourage the use of “preconceptions” (González‐Bailón, Banchs and Kaltenbrunner, 2012, p. 124). Emotions may also influence political behaviour (Marcus, 2000), such as information-seeking due to “anxiety or anger” (González‐Bailón et al., 2012, p. 124), or voting intention and political interest (Brader, 2005). Messages which contain such appeals (Nabi, 1999), are intended to “capture attention, influence attitudes and affect behaviour” (Nabi, 2003, p. 224).
However, emotions can be a “hindrance to rational decision making” (Kühne, 2012, p. 1), potentially leading to opinions and behaviour based on stimuli reactions and not
consideration. This can be supported by Nabi (2002) who said that citizens may seek to resolve “problematic” (p. 206) circumstances with actions fuelled by the evoked emotion. For example, voting for a party who claims they can resolve an issue which the individual feels strongly about.
Finally, location of the newspaper perhaps influenced the use of emotions in coverage of the 2014 Scottish independence referendum. Scottish editions of UK-wide newspapers have “quite different editorial copy” (Rosie et al., 2004, p. 440), in which emotions used may different depending on whether the newspaper was Scottish or UK-wide. This can be
supported by Greenslade (2014) who said that “the London-based English newspapers pulled out all the editorial stops to urge Scotland's population to reject independence” (para. 1). Therefore, it is possible that UK-wide newspapers were more likely to use negative emotions than positive emotions.
H5: UK-wide newspapers were more likely to predominantly use anger, disgust, fear and sadness when covering Scottish independence and the referendum than Scottish newspapers.
Method
To test the hypotheses and answer the research questions, quantitative content analysis was used. This is the “systematic and replicable examination of symbols of communication” (Riffe, Lacy, Watson and Fico, 2014, p. 19) based upon theory (Riffe et al., 2014), which allows for hypotheses testing (White and Marsh, 2006), and the obtainment of meaning as symbols are given “numerical values” (Riffe et al, 2014, p. 19). Prior (2014) said it can be used to analyse and collect data, whilst Prasad (2008) highlighted its relevance for election content in newspapers, thus is appropriate for this paper. Other benefits of content analysis include that it can be replicated, and tested for reliability and validity (White and Marsh, 2006).
Sample
Newspapers
Newspapers were analysed due to their political importance, as voters use them for political information (Druckman, 2005), thus they can be considered as a learning source (Jerit,
Barabas and Bolsen, 2006). In the context of the 2014 independence referendum, Haggerty (2014) argued that although less influential than TV/radio and social media, 35% of those surveyed used newspapers for decision-influencing information (para. 4), arguably showing their influence.
Specific Newspapers
Seven newspapers were chosen to compare the use of emotion by location, including four Scottish newspapers and three sister/equivalent UK-wide publications. These were all accessible via Nexis Uni for the duration of this study’s time-frame.
Scottish newspapers included, The Sunday Herald and The Herald, who differed on their stance on independence, the more right Scotsman (Dekavalla, 2016a), and tabloid, The Daily Record, so that analysis was not limited to broadsheets.
The UK-wide sister paper of The Daily Record, The Mirror, was analysed, as were
broadsheets The Independent and The Guardian, as UK-wide equivalents for The Scotsman and The Herald.
Time-frame
Articles which referred to Scottish independence or the 2014 referendum, directly or
indirectly were gathered from Nexis Uni using a keyword search for “Scottish independence referendum” AND/OR “Scottish independence” AND/OR “indyref” AND/OR
“independence” between May 25 2014 and September 19 2014, allowing for over-time analysis. May 25 was selected as the start-date as this was the official launch of Yes Scotland (BBC News, 2014b), and the end-date was the day after the referendum. This time-frame was appropriate as the six months before the referendum date was a “period of exceptionally heightened political discourse” (Law, 2015, p. 3).
Constructed Week Sampling
Before gathering articles for analysis, two constructed weeks were formed to represent the selected time-frame. This is a “stratified random sampling (SRS) technique” (Luke, Caburnay and Cohen, 2011, p. 78) in which random dates for each day of the week within the time-frame are chosen (Luke et al., 2011; Riffe, Aust and Lacy, 1993), and all articles published on each day are included. These are beneficial as they allow for “systematic variation due to day of week” (Riffe et al., 1993, p. 133) in newspaper coverage. Although one constructed week was sufficient for a six-month period, preciseness increased with two (Hester and Dougall, 2007; Riffe et al., 1993).
Although this paper uses a four-month time-frame, two constructed weeks were used to ensure enough material was gathered, so two random Sundays, Mondays etc. were chosen. For feasibility reasons, the number of articles to be analysed was reduced by ordering them by date and selecting every second article for the final sample. However, due to the limited number of relevant articles in The Sunday Herald and The Mirror from the original sample (before selecting every second article) all were included from these newspapers. Overall, the final sample yielded 377 articles: The Sunday Herald (40), The Herald (87), The Daily Record (53), The Scotsman (83), The Mirror (21), The Independent (41), and The Guardian (52).
Codebook
A codebook was created (see Appendix A) which detailed how each article was to be coded, including administrative variables and information on coding issues, actors and emotions.
As emotion was coded manually, only the headlines and lead paragraphs were analysed for feasibility reasons. This was justifiable as often the “most important information” (Thomson, White and Kitley, 2008, p. 212) is presented at the start of an article, and Strapparava and
Mihalcea (2007) said headlines are written “to ‘provoke’ emotions” (p. 71)and capture attention. In addition, Herman and Serafis (2019), and Mohammad (2012) conducted research on emotion and headlines, therefore only analysing these and lead paragraphs was deemed appropriate as this section “dominates the text” (Thomson et al., 2008, p. 219).
Within headlines and lead paragraphs certain words were coded, including nouns, verbs, and adjectives (Lexico, n.d.). Pronouns, prepositions, and conjunctions (Lexico, n.d.), determiners (Nordquist, 2018), and names, place names, dates and numbers were not coded.
Therefore, the sampling unit is newspapers and the unit of analysis is articles within these. The headlines and lead paragraphs of these articles are the context unit, and the coding unit is the emotion words within these.
Coding
After coding administrative variables, issues, some of which were based upon those
highlighted by Dekavalla (2016a, 2018), figures, and emotions were coded. Issues, and key political (BBC News, 2014a; Carrell, 2013) and economic (BBC News, 2014a) figures were coded as being mentioned (1 = Yes) or not mentioned (0 = No).
Emotion Coding
This paper used the NRC EmoLex (Mohammad and Turney, 2010, 2013a) to code emotion. This was created to allow users “to identify emotional tone of larger units of text”
(Mohammad and Turney, 2010, p. 33), including newspaper and online content, and can be downloaded from Mohammad’s website (Mohammad, n.d.).
Firstly, the coder determined whether any of the codable words in each article’s headline and lead paragraph featured in the NRC EmoLex (Mohammad and Turney, 2010, 2013a). If they did, the coder then checked whether the word was associated with any of the key emotions.
Within the lexicon, 1 represents an association, 0 represents no association (Mohammad and Turney, 2013b). The coders then marked whether the headline and lead paragraph contained words which were associated with (at least one) key emotion (1 = Yes, 0 = No). Following this, the codebook contained a variable for each of the eight emotions, in which the same codes were used (1 = Yes, 0 = No) to mark which were present. The most used emotion was also coded for.
Intercoder Reliability
To test the reliability of the variables in the codebook, a second coder coded 36 articles, just short of the 10% of the total sample as recommended by De Swert (2012). This was because testing had begun before it was realised that all of The Sunday Herald and The Mirror articles from the original sample should be included.
De Swert (2012) recommended a Krippendorff’s Alpha value of .67 as sufficient, although .60 is acceptable in some circumstances. After thorough training, most variables were sufficiently reliable (see Appendix B), although some were not. After consulting and re-training the second coder, some variables were altered for clarity which improved reliability.
However, some variables were still included in the analysis despite falling below .60 after re-training. Due to the nature of the emotion variables, subjectivity was involved when deciding which emotions to code a word as when it was associated with more than one, so lower agreement could be expected. Low Krippendorff’s Alpha values can also be explained by low occurrences of some variables in the sub-sample which was recoded, such as joy (.53) and surprise (.58). Therefore, any disagreement may have a larger statistical effect than if there were many occurrences. The inclusion of below sufficient variables can further be justified as percentage agreement was high. However, it is fully acknowledged that intercoder reliability was low for some variables, therefore results should be carefully interpreted.
Analysis
The content analysis data was then analysed by running frequencies, chi-square tests and independent samples t-tests using SPSS to answer the research questions and hypotheses. To test the main RQ, dummy variables were recoded from the most used variable. For H2, the publication variable was recoded into a dummy variable with The Sunday Herald = 1, and the other newspapers = 0. For H3, a new variable was created by combining the independence support and union support variables, so that if independence was supported in an article = 1, if the union was supported = 0.
Results
Main RQ: Which emotions were most used in articles on Scottish independence in newspapers circulated in Scotland and England in the run up to the 2014 Scottish independence referendum?
To answer the main research question, the means in Table 2 (see Appendix C) show that anticipation followed by trust, anger and fear were the four most used emotions in UK-wide and Scottish articles. An independent samples t-test was run to test the difference between the most used in Scottish (M = 3.84, SD = 2.78) and UK-wide (M = 3.64, SD = 2.69) articles, which was not significantly different, t (375) = -.66, p = .510, 95% CI [-.81, .40], d = .07, thus there was no significant relationship between the most used emotion and country the newspaper was circulated in.
Independent samples t-tests were also run to determine whether there was a difference in the use of each emotion in articles circulated in Scottish and UK-wide newspapers, not limited to the most used emotion (see Table 3 in Appendix C). Six of the eight discrete emotions were not statistically significant. However, the use of anticipation in Scottish (M = .63, SD = .48)
and UK-wide articles (M = .74, SD = .44) was significantly different with weak effect, t (232.85) = 2.00, p = .047, 95% CI [.00, .20], d = .24, and showed that anticipation was more often used in wide articles. In addition, sadness in Scottish (M = .20, SD = .40) and UK-wide articles (M = .30, SD = .46) was significantly different with weak effect, t (190.19) = 2.03, p = .044, 95% CI [.00, .20], d = .23, and was used more often in UK-wide articles.
Sub-RQ: How did the use of the most used emotion differ over-time and by publication?
A chi-square test was run to determine whether there was an association with the most used emotion and the ascending date variable to identify over-time changes. As over 20% of cells had counts of less than five, assumptions for chi-square were not met and thus the Likelihood Ratio was used which showed there was not a statistically significant association between most used emotion over-time, X2 (104, N = 377) = 114.91, p = .219.
The mean values in Table 4 (see Appendix C) show that, anticipation, trust, anger and fear also make up the four most used emotions for all publications (anticipation was the most used in all publication except for The Mirror). However, a chi-square test was run to determine whether there was a statistical association between the most used emotion and publication. The Likelihood Ratio was used as more than 20% of cells had counts of less than five, thus violating assumptions for the chi-square statistic. The Likelihood Ratio showed that there was not a statistically significant association between the most used emotion and
publication, X2 (48, N = 377) = 54.72, p = .235.
Therefore, there were no statistical differences between the most used emotion over-time, nor by publication.
H1: Articles which referred to Alex Salmond, Nicola Sturgeon and the SNP were more likely to contain anger, fear, disgust, and sadness than trust and joy.
To identify whether there was a relationship between these emotions and the presence of Salmond, Sturgeon and the SNP, chi-square tests were run for each figure separately. The mean results in Table 5 (see Appendix C) indicate how often each emotion was present in articles which mentioned them.
Insignificant results were given in all but two instances (see Table 5 in Appendix C). Firstly, there is a significant, weak association between articles which mention Salmond and anger, X2 (1, N = 377) = 6.49, p = .011, Phi = .13. The differences between the observed and
expected counts show that anger was present in articles referring to Salmond more often than was expected. In addition, there was a significant, weak association between articles which mention the SNP and sadness, X2 (1, N = 377) = 4.35, p = .037, Phi = .11. The differences between the observed and expected counts show that sadness was present in articles referring to the SNP more often than was expected.
According to the mean presence of each emotion in articles referring to these three actors, trust appears the most. However, this does not mean that these articles supported Salmond, Sturgeon or the SNP, it merely shows that trust was present in those which mentioned them. Although trust is the most common, anger, fear and sadness were also very prominent within each set of articles, in which case H1 has been partly supported.
H2: Economic articles in The Sunday Herald were less likely to be associated with fear than in the other six publications.
To determine whether the use of fear in economic articles was different for The Sunday Herald compared to the other newspapers, an independent samples t-test was run. This
showed that the use of fear in The Sunday Herald’s economic articles (M = .40, SD = .55) and in the other newspapers (M = .48, SD = .50) was not significantly different, t (84) = .350, p = .727, 95% CI [-.38, .54], d = .15.
The SPSS file was split to identify the mean use of fear in economic articles for each
newspaper. Table 6 (see Appendix C) shows that fear was used less in The Sunday Herald (N = 5, M = .40, SD = .55) compared to the other newspapers. Interestingly, Scottish newspapers were less likely to use fear in economic articles than UK-wide newspapers.
However, it should be noted that the t-test was not statistically significant, and that the number of cases in which fear was present in economic articles was very low, for example there were five in The Sunday Herald. This is a hindrance for statistical testing as results may have been significant if there were a higher number of cases. Therefore, there is no formal support for H2.
H3: Articles which supported Scottish independence were more likely to feature anger, and those supporting the Union were likely to use fear.
An independent samples t-test was run to test whether there was a difference in the use of anger and fear between pro-independence, and pro-union articles. Firstly, the use of anger in pro-independence articles (M = .20, SD = .40) compared to pro-union articles (M = .38, SD = .49) was significantly different with weak effect. Anger was more common in pro-union than pro-independence articles, t (141.71) = 2.48, p = .014, 95% CI [.04, .32], d = .45.
The presence of fear within pro-independence articles (M = .32, SD = .47) was not
significantly different compared to pro-union articles (M = .47, SD = .50), t (132.17) = 1.88, p = .062, 95% CI [-.01, .31], d = .31.
Thus, H3 has not been supported as there was no significant difference in the use of fear between union and independence articles, and anger was more common in those pro-union, in contrast to what was hypothesised.
H4: Over time, the mobilising emotion of anger was used more often in both
independence and union articles, whereas fear became more popular only in the pro-union articles.
Firstly, Figure 1 (see Appendix C) shows that with fluctuations, the number of relevant articles published increased as the referendum approached. To test whether there was a relationship between ascending date and anger in pro-independence and pro-union articles, and fear in pro-union articles, two chi-square tests were run looking at pro-independence and pro-union articles separately.
For pro-independence articles compared with both anger and fear, more than 20% of cells had counts of less than five, therefore assumptions for chi-square were not met. Therefore, the Likelihood Ratio was used which showed that there was not a statistically significant association between ascending date and the use of anger X2 (11, N = 60) = 11.67, p = .389, nor between ascending date and fear X2 (11, N = 60) = 17.01, p = .108.
For pro-union articles, more than 20% of cells had counts of less than five for anger and fear, so the Likelihood ratio was also used here. These showed no statistically significant
association between ascending date and anger X2 (12, N = 92) = 16.90, p = .154, nor between ascending date and fear X2 (12, N = 92) = 20.11, p = .065. Therefore, as the associations were insignificant, it cannot be said that there was a relationship between time and the use of anger and fear, thus H4 has not been supported.
H5: UK-wide newspapers were more likely to predominantly use anger, disgust, fear and sadness when covering Scottish independence and the referendum than Scottish newspapers.
Independent samples t-tests were run to determine whether there was a difference in the extent to which anger, disgust, fear and sadness were the most used emotions in UK-wide and Scottish newspapers.
The difference between anger as the most used emotion in UK-wide (M = .11, SD = .31) and Scottish newspapers (M = .13, SD = .34) was not significantly different, t (375) = -.75, p = .454, 95% CI [-.10, .05], d = .06. Disgust as the most used emotion in UK-wide newspapers (M = .01, SD = .09) as compared to Scottish newspapers (M = .02, SD = .12) was also not significantly different, t (375) = -.50, p = .617, 95% CI [-.03, .02], d = .09. UK-wide newspapers’ use of fear as the most used emotion (M = .10, SD = .30) and Scottish
newspaper’s use of fear as the most used emotion (M = .11, SD = .32) was not significantly different, t (375) = -.50, p = .616, 95% CI [-.09, .05], d = .03. Finally, the difference between sadness as the most used emotion in UK-wide (M = .04, SD = .19) and Scottish newspapers (M = .02, SD = .12) was not significantly different, t (157.76) = 1.05, p = .294, 95% CI [-.02, .06], d = .13.
Therefore, H5 cannot be supported as there was not a significant difference between UK-wide newspapers and Scottish newspapers in the extent to which anger, disgust, fear and sadness were the most used emotions.
Discussion and Conclusion
This paper sought to investigate the use of emotion in coverage of the 2014 Scottish
independence referendum in Scottish and UK-wide newspapers, and can now identify which were the most used in the run up to the referendum date.
Within Scottish and UK-wide articles, and each individual newspaper, anticipation, trust, anger and fear were most used. The prominence of anticipation is understandable as the referendum had not yet been held and fluctuating polls (Morisi, 2016) showed no obvious winner. Trust may be explained by either side promoting their own benefits, or may have reflected the benefits of further devolution as an alternative to independence which was supported by some newspapers (Dekavalla, 2016b). Anger may have been used to encourage mobilisation (Lecheler et al., 2013), and it is plausible that fear was used to encourage support for the status quo (Schuck and de Vreese, 2009; Vasilopoulou and Wagner, 2017). The prominence of anger and fear therefore support Garry’s (2014) argument that in
referenda, “anger and anxiety would play an important role” (p. 248), if one considers fear as a synonym of anxiety. Additionally, significant differences were found between the use of anticipation and sadness in Scottish and wide articles, with both more prominent in UK-wide articles. These may have included more anticipation because of fluctuating polls (Morisi, 2016) and because even though only Scots could vote, UK-wide newspapers did not want the union to split (Greenslade, 2014), which may also explain why UK-wide articles included more sadness. To answer the sub-research question, there were no statistical differences between the most used emotion over-time nor by publication, however the popularity of anticipation could be because newspapers were speculating on what would happen on September 18 2014.
Additionally, this paper found a significant association between articles mentioning then-First Minister Alex Salmond and the use of anger, and between the SNP and sadness. This may lend support to Walker (2018), who argued the press were unfavourable towards the SNP, thus it is unsurprising that significant results were found with negative emotions (Mohammad and Turney, 2010). Anger may have been used in articles mentioning Salmond to encourage
voters to reject independence if they felt “aversion” (Druckman and McDermott, 2008, p. 301) towards one of its drivers, and sadness could have been used when referring to the SNP as they were the main driver of independence, and the narrative may have been that they were trying to break up the union. However, this should be interpreted with caution. This paper only tested emotions’ presence and associations with issues and figures, thus it cannot be said that anger was directed at Salmond, or sadness about the SNP.
Greenslade’s (2014) argument that UK editorials attempted to warn voters about
independence could be informally supported as, despite no statistically significant difference, fear was least common in economic articles in the only pro-independence newspaper
(Haggerty, 2014; Law, 2015; Walker, 2018), and less common in Scottish economic articles than UK-wide ones. This may support Mitchell’s (2016) argument that an independent Scotland’s currency was uncertain, as fear may have been used to encourage voters to consider the economic risks (Lerner and Keltner, 2000), and be cautious (Druckman and McDermott, 2008) of independence. This may also be indicative of Dekavalla’s (2018) finding which showed that the pro-union side focused on the economy, “particularly in relation to the perceived risk of losing economic security” (p. 1600). However, this should be carefully interpreted as the hypothesis (H2) itself held no formal statistical support.
The lack of statistically significant differences in the use of fear between pro-independence and pro-union articles, could be explained by each side highlighting why the other was problematic for Scotland’s interests. Although it was hypothesised that pro-union articles were more likely to use fear as this may have encouraged the electorate to reject “unwanted change” (Schuck and de Vreese, 2009, p. 55), they included more anger, a mobilising emotion (Lecheler et al., 2013), than pro-independence articles. Therefore, the aim may have
been for voters to reject independence from a point of anger, not fear. This does not support Vasilopoulou and Wagner’s (2017) findings from the 2016 EU referendum, in which angry voters rejected the status quo. Despite there being an overall increase in the number of articles over-time, mirroring Levy et al.’s (2016) findings from the 2016 EU referendum, there was no statistical support for the increasing use of anger and fear. This could be due to the long duration of the campaign, in which both sides may have consistently used these emotions.
Finally, as three out of four Scottish newspapers also opposed independence, their lack of difference in the use of anger, disgust, fear and sadness as the most used emotion may reflect their shared stance on the matter. Therefore, if Greenslade’s (2014) argument that UK
editorials sought to discourage Scots from voting for independence was operationalised using emotion, Scottish publications may have done the same.
Furthermore, although the method used in this paper was suitable, it did have limitations. Firstly, by only analysing headlines and lead paragraphs, it is likely that codable issues, figures and emotions were missed in the rest of the article which would have offered a richer understanding of each publication’s use of emotion in the 2014 referendum context.
Secondly, some variables had low intercoder reliability, so tests including these should be carefully interpreted. This may be explained by the importance of context for selecting which emotion from the NRC EmoLex (Mohammad and Turney, 2010, 2013a) was appropriate (if multiple emotions were associated with that word), in which coder subjectivity may have played a role. Low intercoder reliability may also be explained by the few instances of some variables in the sub-sample analysed by the second coder. Due to the word limit of this thesis, the research questions specifically focus on the most used emotion. However, testing the differences between all emotions, for all newspapers over-time would have allowed for more
detailed analysis of each publication’s use of emotion. In addition, there were instances where an actor (for example Nicola Sturgeon), emotion (such as surprise) or issue (such as economic issues) was scarcely present in the overall sample. Therefore, some insignificant results may have been significant had there been more cases. Finally, this paper only coded emotions’ presence and associations, not what they were directed at. Therefore, despite finding some significant results, the true nature of the relationship between the emotions and other variables cannot be determined.
However, this paper may be used as the basis for further research. A similar study could expand the codebook to account for what emotions were directed at for a more concrete understanding of their relationship with the key issues and actors involved in the debate. Full articles may be analysed as opposed to just headlines and lead paragraphs, allowing for a more in-depth analysis of emotions within publications. This paper could also be used in a comparison study of newspapers’ use of emotions covering other Scottish or UK elections, the 2016 EU Referendum, or other independence referendums, such as in Catalonia or Quebec. Finally, the newspaper content analysis data could be linked (see de Vreese et al., 2017) with British Election Study Internet Panel (n.d.) data, which includes a question on newspaper readership and, in some waves, questions on voting intention and choice in the Scottish independence referendum. Therefore, conclusions could be drawn about the effects of newspapers’ use of emotion on opinion and voting behaviour in the Scottish independence referendum.
Overall, this paper has identified that in the run up to the 2014 Scottish independence referendum, anticipation, trust, anger and fear were the most used emotions in Scottish and UK-wide articles, which did not significantly differ over-time nor by publication. This appears to fill a gap in the literature about the ways in which discrete emotions were used by these newspapers in coverage of this particular referendum. Awareness of the emotional
sentiment within newspaper articles could be useful for readers in future election coverage, as it may be less likely that opinions and behaviour are fuelled by the evoked emotion (Nabi, 2002) if readers are aware they are present. Additionally, when linked with public opinion data, this paper could be used in a media effects study in which researchers may be able to determine how these four emotions influenced voters.
References
Ansolabehere, S., Behr, R., & Iyengar, S. (1991). Mass media and Elections: An Overview. American Politics Quarterly, 19(1), 109-139.
https://doi.org/10.1177/1532673X9101900107
BBC News. (2014a, September 19). Scottish independence: Key figures give their reactions. BBC News. Retrieved from https://www.bbc.com/news/uk-scotland-29272198
BBC News (2014b, May 25). Scottish independence: Supporters launch 'yes' campaign. BBC News. Retrieved from https://www.bbc.com/news/av/uk-scotland-18205601/scottish-independence-supporters-launch-yes-campaign
Blain, N., and Hutchison, D. (2016). The Media Landscape in Scotland. In: Blain, N.,
Hutchison, D., & Hassan, G. (Eds.), Scotland’s Referendum and the Media: National and International Perspectives (pp. 16-25). Edinburgh: Edinburgh University Press. Brader, T. (2005). Striking a Responsive Chord: How Political Ads Motivate and Persuade
Voters by Appealing to Emotions. American Journal of Political Science, 49(2), 388– 405. https://doi.org/10.1111/j.0092-5853.2005.00130.x
Brader, T., Marcus, G. E., & Miller, K. L. (2011). Emotion and public opinion. Edwards, G. C., Jacobs, L., & Shapiro, R. Y. (Eds.), The Oxford Handbook of American Public Opinion and the Media (pp. 384–401). Oxford: Oxford University Press.
British Election Study. (n.d.). Panel study data. Retrieved from
https://www.britishelectionstudy.com/data-objects/panel-study-data/
Brooks, L. (2019, December 19). Sturgeon demands independence referendum powers be devolved. The Guardian. Retrieved from
https://www.theguardian.com/politics/2019/dec/19/nicola-sturgeon-requests-independence-referendum-powers.
Buchanan, M. (2016). ‘Liked’, ‘Shared’, Re-tweeted: The Referendum Campaign on Social Media. In: Blain, N., Hutchison, D., & Hassan, G. (Eds.), Scotland’s Referendum and the Media: National and International Perspectives (pp. 70-82). Edinburgh:
Edinburgh University Press.
Burget, R., Karasek, J., & Smekal, Z. (2011). Recognition of Emotions in Czech Newspaper Headlines. Radioengineering, 20(1), 39–47. Retrieved from
https://doaj.org/article/b3fd0447e94b4ad8af7eb45af73dd8aa
Campbell, G. (2020, February 3). Scottish independence: How Brexit has changed the debate over EU membership. BBC News. Retrieved from https://www.bbc.com/news/uk-scotland-scotland-politics-51357050
Carrell, S. (2013, April 23). Scottish independence: the essential guide. The Guardian. Retrieved from https://www.theguardian.com/politics/scottish-independence-essential-guide#106
Carrell, S. (2019, December 13). Sturgeon demands Scottish independence referendum powers after SNP landslide. The Guardian. Retrieved from
https://www.theguardian.com/politics/2019/dec/13/nicola-sturgeon-to-demand-powers-for-scottish-independence-referendum
Clancy, L. (2020). ‘Queen of Scots’: The monarch’s body and national identities in the 2014 Scottish Independence Referendum. European Journal of Cultural Studies, 23(3), 495–512. https://doi.org/10.1177/1367549420902795
Dekavalla, M. (2016a). Framing referendum campaigns: the 2014 Scottish independence referendum in the press. Media, Culture & Society, 38(6), 793-810.
https://doi.org/10.1177/0163443715620929
Dekavalla, M. (2016b). The Scottish Press Account: Narratives of the Independence Referendum and its Aftermath. In: Blain, N., Hutchison, D., & Hassan, G. (Eds.), Scotland’s Referendum and the Media: National and International Perspectives (pp. 46-59). Edinburgh: Edinburgh University Press.
Dekavalla, M. (2018). Issue and game frames in the news: Frame-building factors in television coverage of the 2014 Scottish independence
referendum. Journalism, 19(11), 1588-1607. https://doi.org/10.1177/1464884916674231
De Swert, K. (2012). Calculating inter-coder reliability in media content analysis using Krippendorff’s Alpha. Center for Politics and Communication, 1-15.
de Vreese, C. H., Boukes, M., Schuck, A., Vliegenthart, R., Bos, L., & Lelkes, Y. (2017). Linking survey and media content data: Opportunities, considerations, and pitfalls. Communication Methods and Measures, 11(4), 221-244.
de Vreese, C. H., & Schuck, A. R .T. (2014). Political communication in referendums. In: Reinemann, C. (Ed.), Political Communication (pp. 129-150). Berlin/Boston: Walter de Gruyter. https://doi.org/10.1515/9783110238174
de Vreese, C. H., & Semetko, H. A. (2004). News matters: Influences on the vote in the Danish 2000 euro referendum campaign. European Journal of Political
Druckman, J. (2005). Media Matter: How Newspapers and Television News Cover Campaigns and Influence Voters. Political Communication, 22(4), 463–481. https://doi.org/10.1080/10584600500311394
Druckman, J., & McDermott, R. (2008). Emotion and the Framing of Risky Choice. Political Behavior, 30(3), 297–321. https://doi.org/10.1007/s11109-008-9056-y
Frijda, N. (1993). The place of appraisal in emotion. Cognition and Emotion, 7(3-4), 357– 387. https://doi.org/10.1080/02699939308409193
Garry, J. (2014). Emotions and voting in EU referendums. European Union Politics, 15(2), 235–254. https://doi.org/10.1177/1465116513514780
Glaser, J., & Salovey, P. (1998). Affect in electoral politics. Personality and Social Psychology Review: an Official Journal of the Society for Personality and Social Psychology, Inc, 2(3), 156–172. https://doi.org/10.1207/s15327957pspr0203_1
González‐Bailón, S., Banchs, R., & Kaltenbrunner, A. (2012). Emotions, Public Opinion, and U.S. Presidential Approval Rates: A 5‐Year Analysis of Online Political
Discussions. Human Communication Research, 38(2), 121–143. https://doi.org/10.1111/j.1468-2958.2011.01423.x
Greenslade, R. (2014, September 14). Scottish referendum - what England's Sunday papers say.... The Guardian. Retrieved from
https://www.theguardian.com/media/greenslade/2014/sep/14/scottish-independence-national-newspapers
The Guardian. (2014, September 12). The Guardian view on the Scottish referendum: Britain deserves another chance. The Guardian. Retrieved from
https://www.theguardian.com/commentisfree/2014/sep/12/guardian-view-scottish-independence
Haggerty, A. (2014, October 17). Social media more influential information source than newspapers in Scottish independence referendum, YouGov finds. The Drum. Retrieved from http://www.thedrum. com/news/2014/10/17/social-mediamore-influential-information-sourcenewspapers-scottish-independence
Herald View. (2014, September 16). The Herald's view: we back staying within UK, but only if there's more far-reaching further devolution. The Herald. Retrieved from
https://www.heraldscotland.com/opinion/13180138.the-heralds-view-we-back-staying-within-uk-but-only-if-theres-more-far-reaching-further-devolution/
Herman, T., & Serafis, D. (2019). Emotions, Argumentation, and Argumentativity: Insights from an Analysis of Newspaper Headlines in the Context of the Greek
Crisis. Informal Logic, 39(4), 373–400.
Hester, J. B, & Dougall, E. (2007). The Efficiency of Constructed Week Sampling for Content Analysis of Online News. Journalism & Mass Communication
Quarterly, 84(4), 811–824. https://doi.org/10.1177/107769900708400410
Independent. (2014, September 18). Scottish Independence: Both Britain and Scotland would be diminished by a Yes vote. Independent. Retrieved from
https://www.independent.co.uk/voices/editorials/scottish-independence-both-britain-and-scotland-would-be-diminished-by-a-yes-vote-9740454.html
Jerit, J., Barabas, J., & Bolsen, T. (2006). Citizens, Knowledge, and the Information Environment. American Journal of Political Science, 50(2), 266–282. https://doi.org/10.1111/j.1540-5907.2006.00183.x
Kühne, R. (2012). Media-induced affects and opinion formation: How related and unrelated affects influence political opinions. Living Reviews in Democracy, 3, 1–20.
Law, A. (2015). Mediating the Scottish independence debate. Media education journal, (56), 3-7.
Lecheler, S., Schuck, A. R. T., de Vreese, C. H. (2013). Dealing with feelings: positive and negative discrete emotions as mediators of news framing effects. Communications: The European Journal of Communication Research, 38(2), 189–209.
https://doi.org/10.1515/commun-2013-0011
Lerner, J. S, & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition and Emotion, 14(4), 473–493. https://doi.org/10.1080/026999300402763
Levy, D. A. L., Aslan, B., & Bironzo, D. (2016). UK press coverage of the EU referendum. Oxford: Reuters Institute for the Study of Journalism, University of Oxford. Retrieved from
https://ora.ox.ac.uk/objects/uuid:8a0aac1f-8805-4ce4-96a8-207c1479c0c6/download_file?safe_filename=UK%2BPress%2BCoverage%2Bof%2 Bthe%2BEU%2BReferendum.pdf&file_format=application%2Fpdf&type_of_work= Report
Lexico. (n.d.). Word classes (or parts of speech). Lexico. Retrieved from https://www.lexico.com/grammar/word-classes-or-parts-of-speech
Luke, D., Caburnay, C., & Cohen, E. (2011). How Much Is Enough? New Recommendations for Using Constructed Week Sampling in Newspaper Content Analysis of Health Stories. Communication Methods and Measures, 5(1), 76–91.
https://doi.org/10.1080/19312458.2010.547823
Lynch, P. (2015). Bottom-up versus Top-down Campaigning at the Scottish Independence Referendum 2014. Revue Française de Civilisation Britannique, 20(XX-2) 1-12. https://doi.org/10.4000/rfcb.405
Marcus, G. E. (2000). Emotions in Politics. Annual Review of Political Science, 3(1), 221– 250. https://doi.org/10.1146/annurev.polisci.3.1.221
Marcus, G. E. (2003). The psychology of emotion and politics. In: Sears, D. O., Huddy, L., & Jervis, R. (Eds.), The Oxford Handbook of Political Psychology (pp. 182-221).
Oxford: Oxford University Press.
McGarvey, N. (2015). The 2014 Scottish independence referendum and its aftermath. Social Alternatives, 34(3), 34-40.
Mirror. (2014, September 18). Scottish referendum: Don't rip apart the most successful family of nations in the world. Mirror. Retrieved from
https://www.mirror.co.uk/news/uk-news/scottish-referendum-dont-rip-apart-4278694 Mitchell, J. (2016). The Unexpected Campaign. In: Blain, N., Hutchison, D., & Hassan, G.
(Eds.), Scotland’s Referendum and the Media: National and International Perspectives (pp. 3-15). Edinburgh: Edinburgh University Press.
Mohammad, S. M. (2012, June). Portable features for classifying emotional text. In Proceedings of the 2012 Conference of the North American Chapter of the
Association for Computational Linguistics: Human Language Technologies (pp. 587-591). Association for Computational Linguistics.
Mohammad, S. M. (n.d.). NRC Word-Emotion Association Lexicon. Retrieved from https://www.saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm
Mohammad, S. M., & Turney, P. D. (2010, June). Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon. In Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and
generation of emotion in text (pp. 26-34). Association for Computational Linguistics. Mohammad, S. M., & Turney, P. D. (2013a). Crowdsourcing a Word–Emotion Association
Mohammad, S. M. and Turney, P. D. (2013b). NRC Emotion Lexicon. Technical report, NRC Technical Report. Retrieved from
http://www.saifmohammad.com/WebDocs/NRCemotionlexicon.pdf
Morisi, D. (2016). Voting under uncertainty: the effect of information in the Scottish independence referendum. Journal of Elections, Public Opinion and Parties, 26(3), 354-372. https://doi.org/10.1080/17457289.2016.1178648
Mullen, T. (2014). The Scottish independence referendum 2014. Journal of Law and Society, 41(4), 627-640. https://doi.org/10.1111/j.1467-6478.2014.00688.x Nabi, R. L. (1999). A Cognitive‐Functional Model for the Effects of Discrete Negative
Emotions on Information Processing, Attitude Change, and Recall. Communication Theory, 9(3), 292–320. https://doi.org/10.1111/j.1468-2885.1999.tb00172.x
Nabi, R. L. (2002). Anger, Fear, Uncertainty, and Attitudes: A Test of the Cognitive-Functional Model. Communication Monographs, 69(3), 204–216.
https://doi.org/10.1080/03637750216541
Nabi, R. L. (2003). Exploring the Framing Effects of Emotion: Do Discrete Emotions Differentially Influence Information Accessibility, Information Seeking, and Policy Preference? Communication Research, 30(2), 224–247.
https://doi.org/10.1177/0093650202250881
Nabi, R. L. (2010). The Case for Emphasizing Discrete Emotions in Communication Research. Communication Monographs, 77(2), 153–159.
https://doi.org/10.1080/03637751003790444
Nai, A. (2018). Fear and Loathing in Populist Campaigns? Comparing the Communication Style of Populists and Non-populists in Elections Worldwide. Journal of Political Marketing, 1–32. https://doi.org/10.1080/15377857.2018.1491439
The Newsroom. (2014, September 12). Scotland’s decision: The Scotsman’s verdict. Scotsman. Retrieved from
https://www.scotsman.com/news/opinion/columnists/scotlands-decision-scotsmans-verdict-1526529
Nordquist, R. (2018, August 31). Definition and Examples of Determiners in English. ThoughtCo. Retrieved from https://www.thoughtco.com/determiner-in-grammar-1690442
Prasad, D. B. (2008). Content analysis: A method of Social Science Research. In D.K. Lal Das (Ed.), Research Methods for Social Work (pp.174-193). New Delhi: Rawat Publications.
Prior, L. (2014). Content Analysis. In: Leavy, P. (Ed.), The Oxford Handbook of Qualitative Research (pp. 359-379). New York: Oxford University Press.
https://doi.org/10.1093/oxfordhb/9780199811755.013.008
Riffe, D., Aust, C. F., & Lacy, S. R. (1993). The Effectiveness of Random, Consecutive Day and Constructed Week Sampling in Newspaper Content Analysis. Journalism
Quarterly, 70(1), 133-139. https://doi.org/10.1177/107769909307000115
Riffe, D., Lacy, S., Watson, B. R., & Fico, F. (2014). Analyzing Media Messages: Using Quantitative Content Analysis in Research (4th ed.). London: Routledge.
Rosie, M., Macinnes, J., Petersoo, P., Condor, S., Kennedy, J. (2004). Nation Speaking Unto Nation? Newspapers and National Identity in the Devolved UK. The Sociological Review, 52(4), 437–458. https://doi.org/10.1111/j.1467-954X.2004.00490.x Schemer, C. (2014). Emotional effects in political communication. In: Reinemann, C.
(Ed), Political Communication (pp. 569-590). Berlin/Boston: Walter de Gruyter. https://doi.org/10.1515/9783110238174
Schuck, A. R. T., & de Vreese, C. H. (2009). Reversed Mobilization in Referendum Campaigns: How Positive News Framing Can Mobilize the Skeptics. The International Journal of Press/Politics, 14(1), 40–66.
https://doi.org/10.1177/1940161208326926
Smith, C. A., & Ellsworth, P. C. (1985). Patterns of Cognitive Appraisal in Emotion. Journal of Personality and Social Psychology, 48(4), 813–838. https://doi.org/10.1037/0022-3514.48.4.813
Strapparava, C., & Mihalcea, R. (2007, June). Semeval-2007 task 14: Affective text. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) (pp. 70-74).
Thomson, E. A., White, P. R. R., & Kitley, P. (2008). “OBJECTIVITY” AND “HARD NEWS” REPORTING ACROSS CULTURES: Comparing the news report in
English, French, Japanese and Indonesian journalism. Journalism Studies: Language and Journalism, 9(2), 212–228. https://doi.org/10.1080/14616700701848261
Vasilopoulou, S., & Wagner, M. (2017). Fear, anger and enthusiasm about the European Union: Effects of emotional reactions on public preferences towards European integration. European Union Politics, 18(3), 382–405.
https://doi.org/10.1177/1465116517698048
Walker, R. (2018). Whatever Happened to the Independence Press?. Scottish Affairs, 27(1), 63-70.
Weeks, B. E. (2015). Emotions, Partisanship, and Misperceptions: How Anger and Anxiety Moderate the Effect of Partisan Bias on Susceptibility to Political Misinformation. Journal of Communication, 65(4), 699–719. https://doi.org/10.1111/jcom.12164 White, M. D., & Marsh, E. E. (2006). Content Analysis: A Flexible Methodology. Library
Appendix A
Codebook for Content Analysis of Emotions in Newspaper Coverage
Coder Instructions
1. Please read all the instructions thoroughly and follow them exactly. Examples will be provided for clarity on what to code.
2. Please follow this codebook in order.
3. The administrative variables refer to information on the article as a whole.
4. After coding the administrative variables, sections 1 to 3 should be coded from the context units, the headlines and the lead paragraphs.
Administrative Information
AV1 Coder
Who coded the article?
1 = Georgia Clyne
2 = Other coder
AV2 Month Published
During which month was the article published?
1 = May
2 = June
3 = July
4 = August
AV3 Specific date 1 = May 27 2 = May 31 3 = June 5 4 = June 13 5 = June 16 6 = July 6 7 = July 7 8 = July 30 9 = August 1 10 = August 9 11 = August 20 12 = September 7 13 = September 9 14 = September 11 AV4 Publication
Which newspaper did the article feature in? The daily newspaper appears first, followed by the Sunday editions. Whether the newspaper is Scottish or UK-wide/English is stated within the brackets.
On Nexis Uni, most newspapers include their Sunday versions as the same newspaper. However, The Guardian and The Observer, and The Herald and The Sunday Herald are considered as four separate newspapers. For ease of coding, The Guardian and The Observer will be considered as one.
The Herald and the Sunday Herald are considered as separate newspapers, due to them endorsing opposing sides of the campaign.
1 = The Herald (Scottish)
2 = The Sunday Herald (Scottish)
3 = The Guardian/The Observer (UK-wide)
4 = Daily Record/Sunday Mail (Scottish)
5 = Daily Mirror/Sunday Mirror (UK-wide)
6 = The Scotsman (Scottish)
7 = The Independent (UK-wide)
AV5 Location of Newspaper
Is the publication Scottish or UK-wide? Please see AV4 above which lists location in brackets to determine where the newspaper is circulated. This should then be coded as follows:
1 = Scottish
2 = UK-wide
Only the article’s headline and lead paragraph (the first paragraph in the article) should be coded, as this is the context unit. This means that when identifying the main issues the article refers to, the figures (political or other) who are mentioned, and the emotions used, the coder should only refer to the headline and lead paragraph.
Please note, often the lead paragraph is a single sentence.
1. Coding of Issues Surrounding Independence and the Referendum
The coder should determine which broad issue the article is referring to in relation to Scottish independence and the referendum. This can be determined by the content of the headline and the lead paragraph. If multiple issues are raised, please code all of them. Some of the issues here are based upon those which Dekavalla (2016, 2018) determined the referendum was framed as in newspapers and on TV coverage.
V1.1 Does the article refer to economic policies/effects of Scottish independence?
Examples: The headline/lead paragraph refers to the economy in general, economic
costs/consequences, jobs, pensions, salaries, specific industries (for example oil, gas, fishing) etc.
0 = No
1 = Yes
V1.2 Does the article refer to other policies?
Examples: The headline/lead paragraph refers to “membership of the European Union (EU)/North Atlantic Treaty Organization (NATO); borders and defence; public services; and welfare” (Dekavalla, 2016, p. 808). Police, national health, immigration, housing etc. may also be included.
