Fear, Hope and Automation:
How Discrete Emotions Mediate Attitudes Towards Automated Journalism
William Brøns Petersen Student ID: 12846961
Master’s Thesis
Graduate School of Communication Master’s programme Communication Science
Supervisor: Andreas R. T. Schuck
May 29, 2020 Word count: 7,069
Abstract
The increasing presence of automated journalism in newsrooms has lately boosted interest from journalists, who write about this emerging technology in both a positive and negative light. Specifically, automation is framed as a risk for journalists’ job security or an
opportunity for improved and more efficient reporting. This study explores the effect of such news frames and the mediating role of discrete emotions on audiences’ attitudes towards the ongoing automation of journalism. With an online experimental design (N = 169), fear and hope are tested as possible mediators of the effects of risk and opportunity framing. Results show that while hope is elicited by the opportunity frame and mediates the effect on attitudes towards further automation in newsrooms in a positive direction, fear was not elicited by the risk frame nor did it have a significant mediating effect. The findings contribute to the limited knowledge of the mediating effect of hope in general and extends our understanding of discrete emotions as mediators to the context of framing emerging technology.
Introduction
Automated journalism is becoming more and more prevalent in newsrooms as algorithms and AI technology increasingly produce stories for media companies across the world. The Associated Press, one of the frontrunners in the field, uses so-called natural language generation “to turn data into insightful, human-sounding narrative” (Automated Insights, n.d., The Solution section, para. 1) in statistics-heavy areas such as finance and sports journalism, and in recent years, automated processes have also been implemented at news outlets like The Washington Post, The New York Times, Los Angeles Times, Forbes, and Bloomberg News (Delcker, 2019; Graefe, 2016). This tendency has recently garnered the attention of reporters and scholars alike, as they increasingly focus on the many implications of what one newspaper headline called “The Rise of the Robot Reporter” (Peiser, 2019).
When reporters write about the ongoing automation of journalism and its
consequences for news workers, the coverage usually highlights both the good and the bad aspects (e.g., Delcker, 2019), and content analyses of such writings have shown that that by undertaking journalistic tasks, automated journalism is often framed as a potential risk to the future of journalistic jobs or an opportunity for reporters to spend more time on quality reporting (Carlson, 2015; van Dalen, 2012). Seemingly, however, no one has engaged with the effect of such coverage on audiences’ attitudes towards the topic, even though “the increasing availability of automated news will impact journalism and the general public at both the individual (micro) and organizational (macro) level” (Graefe, 2016, p. 33). Exactly this is the intention of this study.
While some research has found that the type and valence of news frames have
significant effects on opinions (Schuck & de Vreese, 2006), other studies have concluded that specific frames also elicit certain discrete emotions and that some of those emotions can have
a mediating effect on opinions towards a range of topics (e.g., Gross, 2008; Kühne &
Schemer, 2015; Lecheler, Bos, & Vliegenthart, 2015; Lecheler, Schuck, & de Vreese, 2013). However, the role of emotions as mediators has not been researched in the context of risk and opportunity framing of new technologies. This is somewhat surprising, given that both this type of framing and the topic of new technology is suitable for such an exploration. Given the recent focus on the importance of discrete emotions in the framing effect process, this study zooms in on the role of two specific emotions likely elicited by risk and opportunity framing – namely fear and hope – and their mediating effect on attitudes towards automated
journalism by asking the central research question: To what extent are news readers impacted
by discrete emotions in reporting about automated journalism?
In the following, concepts and definitions of news framing effect, discrete emotion theory, and automated journalism are synthesised into a theoretical framework that can substantiate and explain how coverage of automated journalism impacts audiences as mediated by emotions.
Theoretical Framework The Effect of News Frames
Frames are a way of focusing on specific parts of reality by selecting, omitting and/or highlighting them in communication, which ultimately may or may not influence cognition (Entman, 1993) and subsequently opinions and behaviours of those who perceive the frame (Chong & Druckman, 2007). This is also true for news frames (e.g., de Vreese, 2005). To Entman (1993), frames are characterised by “the presence or absence of certain key words, stock phrases, stereotyped images, sources of information, and sentences that provide thematically reinforcing clusters of facts or judgments” (p. 52).
Central to understanding the relationship between frames, emotions and automated journalism is framing effect. Broadly speaking, literature usually distinguishes between an external and an internal understanding of what constitutes a frame: Frames in communication is the specific form and emphasis – or use of salience – employed when information is
communicated in texts, speeches and other forms of messages, while frames in thought describes whether and how someone perceives and evaluates something as important (Druckman, 2001; see also Scheufele, 1999). Framing effect, then, is the process where frames in communication impact frames in thought (Druckman, 2001) or the result of news frames interacting with audiences’ “prior knowledge and predispositions” (de Vreese, 2005, p. 52).
Research on the valence of frames and their effect on public opinion is a central component of framing theory. De Vreese and Boomgaarden (2003) argued that frames often carry an inherent valence, which indicate what is to be considered positive and negative, either implicitly or explicitly, and found that valenced frames impact opinion in the same direction as the frame indicate. This has been found to be true for risk and opportunity frames (Schuck & de Vreese, 2006) and is relevant for the topic of automated journalism coverage, given that risk and opportunity frames are often seen in media coverage of technological developments such as biotechnology (Marks, Kalaitzandonakes, Wilkins, & Zakharova, 2007), nanotechnology (Allan, Anderson, & Petersen, 2010), or the usage of social media to document sexual assault (Pennington & Birthisel, 2016). Thus, the first set of hypotheses are:
H1a: Framing automated journalism as a risk will have a negative effect on attitudes
towards automated journalism.
H1b: Framing automated journalism as an opportunity will have a positive effect on
In a working paper, Kühne (2014) argues that frames not only have cognitive effects but can also elicit emotional reactions, which also affect attitudes, and recently, papers have found that emotions seem to have a mediating effect on frames’ effect on opinion formation (e.g., Lecheler et al., 2015; Lecheler et al., 2013). Thus, this paper also examines literature on emotions and their role in influencing opinions about automated journalism.
Emotions Theory and Frames
An often-used definition of emotions describes them as “internal, mental states representing evaluative, valenced reactions to events, agents, or objects that vary in intensity (Ortony, Clore, & Collins, 1988). They are generally short-lived, intense, and directed at some external stimuli” (Nabi, 1999, p. 295). In this sense, emotions differ from other “affective phenomena” such as moods, attitudes or preferences (Scherer, 2005). Similarly, Izard (1977) distinguished between emotions as “traits” or “states”, the former comparable to Scherer’s (2005) “affective phenomena” and the latter to Nabi’s (1999) definition of
emotions.
Two major perspectives exist when categorising emotions: Proponents of the dimensional approach classify emotions in accordance with overarching, polar categories such as valence or arousal, while advocates of discrete emotion theory argue that particular thought patterns and specific action tendencies are the defining features of an emotion (Nabi, 2010). Some scholars argue that the discrete emotions approach more accurately captures the nuances and complexities in describing the processes and predicting the outcomes elicited by specific emotions (e.g., Nabi, 2010; Scherer, 2005). For instance, different emotions like anger and fear or anxiety would be placed in the same dimension, that is, all having negative
valence, even though they differ in other relevant ways (Holm, 2012; Lerner & Keltner, 2000; Scherer, 2005).
Central to discrete emotion research, and thus to this study, is cognitive appraisal theory. In general, appraisal theorists agree that emotions “reflect appraisals of features of the environment that are significant for the organism’s well-being” (Moors, Ellsworth, Scherer, & Frijda, 2013, p. 119). For instance, Roseman (1991) interpreted appraisal theory to mean that “it is not events per se that determine emotional responses, but evaluations and interpretations of events” (p. 162, emphasis in original) and proposed that emotions are determined by patterns of appraisal that are unique to each emotion. Appraisals of events guide how we react to them, known as “action tendencies” (Lazarus, 1991). Most appraisal theories agree on a number of dimensions or variables, which determine the elicited emotion: goal relevance, goal congruence, certainty, agency and control (Moors et al., 2013).
Notably, some scholars have argued that appraisal theory and its dimensions are central for understanding how and what role emotions play in framing effects (Gross, 2008; Kühne, 2014). Gross (2008) points out that “if frames alter the information and considerations subjects have at hand, cognitive appraisal models would predict that emotional outputs should differ” (p. 172). That is, a specific discrete emotion will be elicited if there is congruence between what a frame highlights and the emotion’s appraisal dimensions (Kühne, 2014). This means that if a risk or an opportunity frame is in accordance with the appraisals of specific discrete emotions, those emotions will be elicited. In the following, the topic of automated journalism is reviewed in order to gauge how it is framed in the media and which emotions are likely to be elicited by these frames.
Framing Automated Journalism and Elicited Emotions
In the bourgeoning literature of “software-generated news” (Linden, 2017), multiple differing definitions of automated processes in journalism have been devised and used
synonymously, which has resulted in conceptual ambiguity (Coddington, 2015): Terms in the field range from “robot journalism” (van Dalen, 2012) over “computational journalism” (Coddington, 2015) to “automated journalism” (Montal & Reich, 2017). Given this study’s focus on automated news output rather than journalistic collection methods or analytical tools, this paper conceptualises automated journalism as “algorithmic processes that convert data into narrative news texts with limited to no human intervention beyond the initial
programming” (Carlson, 2015, p. 417).
Still in its nascent stage (Carlson, 2015), research on automated journalism mostly focuses on the changing role of journalists, journalism and news production: Carlson (2015) sheds light on journalistic labour, news compositional forms and journalistic authority, while Wu, Tandoc, and Salmon (2019) focus on changing journalistic role perceptions caused by automation. There have also been more niche studies on automated journalism, e.g., in the context of its audience perception and perceived credibility (Wölker & Powell, 2018) and algorithmic authorship (Montal & Reich, 2017). In general, not much literature can be found on the actual content of automated journalism coverage, much less on the effect of automated journalism coverage in the news. However, given the increasing prevalence of automation in newsrooms (e.g., Carlson, 2015; Wu et al., 2019), its likely disruptive effect on journalism and the public (Graefe, 2016) and a seemingly growing salience of the topic in both journalistic and academic writings, it is important to gauge how writings about automated processes influence audiences’ perceptions of journalism and its relation to emerging technology. Being a substantial source of information, the news provides a significant and
relevant area to study how coverage of automated journalism impacts emotions and attitudes towards the subject.
In the few studies of writings about automated journalism that do exist, one common theme emerges: This technological advancement is either seen as a potential threat to
journalists or a helpful tool. Two studies focusing on the coverage of automated journalism in news articles and blog posts found that when it comes to journalistic labour, automation is often either framed pessimistically as something that will have a negative impact on journalistic labour or optimistically as a technological progress that will aid and free up journalists’ time so they can do more in-depth reporting (Carlson, 2015; van Dalen, 2012; see also Graefe, 2016). In other words, automated journalism is either framed as a risk or an opportunity. This raises the question of which emotions are likely elicited by such coverage.
In general, anger and fear are prominently featured in framing effect studies (e.g., Gross, 2008; Lecheler et al., 2013; Nabi, 2003). But also disgust and pity (e.g., Aarøe, 2011; Gross & D’Ambrosio, 2004), enthusiasm (e.g., Druckman & McDermott, 2008; Holm, 2012) and sympathy (e.g., Gross, 2008) have been given some attention, while only limited focus has been devoted to hope and sadness (cf. Lecheler et al., 2015). However, none of these studies have focused on risk and opportunity frames.
Theoretically, a risk frame should elicit the discrete emotion of fear: In Roseman’s (1991; Roseman, Antoniou, & Jose, 1996) model over appraisal determinants of emotions, the uncertainty of a specific outcome is a central aspect of fear (see also Smith & Ellsworth, 1985), just as it is appraised as being motive-inconsistent, circumstance-caused and with low control potential. The conceptualisation of fear is sometimes hazy in the literature. Lazarus (1991) distinguishes between fright and anxiety, which mainly differ in how imminent and concrete the threat is: Compared to fright, anxiety is a reaction to a threat that is more
existential and symbolic in nature. While the threat can be hard to define when experiencing anxiety, it can be objectified “by reference to sources of external and concrete danger, such as loss of a job” (Lazarus, 1991, p. 237). Therefore, this paper adopts a conceptualisation of fear as anxiety.
The appraisal dimensions of fear and anxiety match the framing of automated
journalism as a risk: Carlson (2015) points to uncertainty over the future of journalistic labour as a central element and uses words such as “fear” and “anxiety” to describe the coverage, while van Dalen (2012) writes that “automated content creation is seen as serious competition and a threat [emphasis added] for the job security of journalists performing basic routine tasks” (p. 653).
In other contexts, opportunity framing has indicated an outcome as potentially positive (Schuck & de Vreese, 2006), making it likely to result in feelings of hope for a number of reasons: While hope is elicited when something is appraised as being motive-consistent and caused by circumstances, uncertainty is fundamental in distinguishing it from a positive emotion like happiness (Roseman, 1991; Roseman et al., 1996). Despite the common
conceptualisation as such, hope should not be regarded a purely positive emotion: In essence, hope is characterised by a yearning for something better than one’s current and suboptimal or negative condition (Lazarus, 1991, 1999) and is elicited when there is focus on the possibility of a “positive future outcome” (Bruininks & Malle, 2005, p. 338). For instance, Chadwick (2015) found that in the context of climate change, hope could be predicted with “the
appraisal of future expectation” (p. 606). However, there is always the underlying knowledge that the outcome might be negative or unfavourable, which for instance distinguishes hope from optimism (Lazarus, 1999). Similar to anxiety, hope can also be directed at something specific, even though what we hope for “is often both complex and diffuse at the same time”
(Lazarus, 1991, p. 285). Other frames similar to opportunity frames have successfully been shown to elicit hope – for instance when framing immigration in terms of emancipation or assimilation (Lecheler et al., 2015) or in gain-positive framing of climate change (Bilandzic, Kalch, & Soentgen, 2017).
Framing automated journalism as an opportunity matches the appraisal dimensions of hope: One of the main arguments found in the coverage is that the automation in the future might help journalists in the news production process, create more jobs (Carlson, 2015) or free up time for journalists to do more in-depth, creative and contextualising – that is, “better” – journalism (van Dalen, 2012). Often, the optimistic interpretation of automation in the newsroom is offered as a counterargument to worries of layoffs (Carlson, 2015), which speaks to the appraisal of hope as originating from something negative.
Previous research has shown that news frames can impact opinion (e.g., Iyengar, 1991) and elicit emotional responses (e.g., Aarøe, 2011; Gross & Brewer, 2007), valenced frames elicit emotions that have the same valence (e.g., Lecheler et al., 2013), and risk and opportunity frames share similarities with appraisal dimensions of fear and hope. Therefore, it can be hypothesised that:
H2a: Automated journalism framed as a risk will lead to an increase in the feeling of
fear.
H2b: Automated journalism framed as an opportunity will lead to an increase in the
feeling of hope.
The Mediating Role of Emotions in Political Communication
A number of studies have shown that some emotions to some extent can mediate the framing effects on opinion towards different topics (e.g., Gross, 2008; Holm, 2012; Kühne &
Schemer, 2015; Lecheler et al., 2015; Lecheler et al., 2013; Major, 2011), thus, in the following, literature about fear and hope and their effect on opinion is reviewed.
On the one hand, the action tendency of fear and anxiety is to in some way to try and avoid or escape a threat, even though the threat can be difficult to pinpoint (Lazarus, 1991), and Nabi (1999) has suggested that fear “may lead to less careful message processing” (p. 300). On the other hand, Tomkins (1963) called fear “an overly compelling persuader” (p. 10) and within political communication, several studies have pointed to the fact that fear and anxiety are positively correlated with more careful information processing (Garry, 2014; Marcus & MacKuen, 1993), which is likely to have an impact on opinion formation (Brader, 2005; Vasilopoulou & Wagner, 2017) but less so with behaviour (Valentino, Brader,
Groenendyk, Gregorowicz, & Hutchings, 2011; see also Groenendyk, 2011 for an overview). Thus, a logical conclusion could be that fear/anxiety also mediates framing effects on opinion. However, in a number of studies, fear has not been found to mediate frames’ effect on opinion (Gross, 2008; Lecheler et al., 2013).
This discrepancy suggests that some nuance needs to be teased out. Firstly, while much of the political communication literature above employs affective intelligence theory (see Marcus, Neuman, & MacKuen, 2000) as the theoretical framework to explain the role of emotions, it does not mean the results are not comparable. However, it is important to note that “anxiety” in some of these studies are actually composite measurements of several discrete negative emotions (e.g., Marcus & MacKuen, 1993), which might have a stronger persuasive effect than anxiety alone. Secondly, Nabi’s (1999) interpretation of fear seems to be closer to Lazarus’ (1991) concept of fright, rather than anxiety. Finally, in studies
measuring the mediating effect of fear, the dependent variable, opinion, was directed towards rather contentious topics, namely crime (Gross, 2008) and the EU (Lecheler et al., 2013).
Arguably, automated journalism is a less polarising topic, especially since it is a more niche subject relatively less familiar to the majority of people, and thus, audiences might be easier to persuade. Given that, the hypothesis for the mediating effect of fear is:
H3a: The discrete emotion of fear mediates the effect of the risk frame on attitudes
towards automated journalism, so that increased feelings of fear will lead to a decrease in support for automated journalism.
Not much research has been done on the persuasiveness of hope (Nabi, 2002b), but the literature still provides some theoretical and empirical evidence for the effect of hope on opinions and behaviours. It has been suggested that the action tendency of hope is approach towards what is desired (Lazarus, 1991), however hope “is often just an outlook about what may happen rather than a clear mobilization for action, though when it involves a call for action there should be substantial activation” (Lazarus, 1999, p. 664).
The most comprehensive insight into the effect of hope can be observed in literature on climate change communication: Hope triggered by a message has a positive effect on interest in the topic and perceived effectiveness of the message (Chadwick, 2015) as well as attitudes (Nabi, Gustafson, & Jensen, 2018). However, hope seems not to have a direct effect (Chadwick, 2015; Nabi et al., 2018) or can even have a negative effect (Bilandzic et al., 2017) on behaviour and behavioural intention when it comes to mitigating climate change effects. This somewhat mirrors the action tendencies of fear, which might not be so surprising given their several overlapping appraisals (e.g., Roseman et al., 1996). Notably, in some of the literature on emotions in political communication, hope is one of several positive emotions that form a composite measurement of “enthusiasm”, which at least to some extent is likely to lead to political involvement and participation, (e.g., Marcus & MacKuen, 1993; Valentino et al., 2011). However, it is not clear from previous research what specific role hope plays.
Unsurprisingly, given that hope in itself is rarely studied compared to fear, even fewer studies focus on the mediating effect of hope on opinions. But some studies have shown that hope successfully mediates the effect of certain frames on opinion towards immigration (Lecheler, et al., 2015) and climate policy (Nabi et al., 2018). Given the similarities between hope and fear and the seemingly persuasive nature of hope on opinions in other fields, the mediation hypothesis for hope is:
H3b: The discrete emotion of hope mediates the effect of the opportunity frame on
attitudes towards automated journalism, so that increased feelings of hope will lead to an increase in support for automated journalism.
Method
An online experiment was conducted to test the mediating role of two discrete emotions (fear and hope) in the effect of risk and opportunity framing on attitudes towards further automated journalism in the newsroom. In total, 192 complete responses were collected. After data cleaning and removing respondents who did not read the stimulus material1 (n = 7) or qualified as outliers2 (n = 12), 173 responses remained. Finally, speeders were identified as respondents who completed the survey in less than 30% of the time of the median response (Mdn = 16.78 minutes) and were also removed (n = 4). Ultimately, the final data set consisted of N = 169 responses with a mean completion time of 20.22 minutes (SD = 10.28). The respondents were randomly assigned to one of three conditions: (a) A news article about automated journalism framed as an opportunity (n = 63); (b) a news article about
1 As indicated by respondents in an open text box following the stimulus material. Mostly, unread stimulus
material was due to the fact that respondents took the survey per phone and found the text illegible.
2 Outliers are defined in SPSS as values above or below 1.5 times the interquartile range (IQR) added or
automated journalism framed as a risk (n = 53); (c) a control condition containing only factual information (n = 53). The survey was pre-tested.3
Participants
Snowball sampling was used to find survey respondents via social media (specifically, Facebook, LinkedIn, Twitter, Instagram and Reddit). First, by sharing a link to the survey, and second, by asking respondents to further share the link. A similar approach has been successfully utilised before (Wölker & Powell, 2018). Although non-probability sampling is frowned upon when it comes to generalisability, random assignment to the different
conditions in the experiment ensures that potential systematic bias is not more present in one of the conditions (Reinard, 2006). Respondents were offered the chance to win a 20-euro gift card to Amazon.com as compensation for participating in the survey. 113 (66.9%) of the respondents identified as female, 54 (32.0%) as male, and two people identified as something else (1.2%) with the age of respondents ranging from 18 to 79 (M = 30.57, SD = 11.76).
Participants were successfully randomised into the three conditions, a randomisation check showed: There were no between-group differences for gender, χ2(4, N = 169) = 5.76, p = .218; education, χ2(8, N = 169) = 8.83, p = .357; or age, F(2, 165) = 0.11, p = .893.
Procedure
Each respondent participated in an online survey where their political orientation, affinity for technology, scepticism towards media and automation, and personality traits were measured. Second, respondents received one of three stimuli. The average reading time was
3 13 individuals pretested the frame strength of the stimulus material, gave emotional responses, and completed
several scale measurements. Results and feedback provided guidance for the optimal version of the stimuli, which emotion to measure, and which scales to discard or adapt to the final version of the questionnaire.
1.43 minutes (SD = 1.17). Respondents then answered a post-test questionnaire measuring emotional reactions to the stimuli as well as opinions and behaviours related to automated journalism. Finally, a manipulation check was completed and demographic variables like age, gender and level of education were recorded (see Appendix A for the full questionnaire).
Stimulus Material
Risk and opportunity frames were more or less implicitly pointed to in literature about the content of journalistic writings about automated journalism (Carlson, 2015; van Dalen, 2012; see also Graefe, 2016), and the stimulus material was based on an article about the possible benefits and disadvantages of automation entering newsrooms (Delcker, 2019). This was done in order to increase the external validity of the experiment. The risk and opportunity frames about automated journalism were operationalised similarly to how Schuck & de Vreese (2006) operationalised risk and opportunity frames about the enlargement of the EU. Thus, it included five dimensions, namely the usage of emotional expressions (i.e., not based on an argument), valenced evaluation, valenced quotes, focus on automation as a future benefit or loss for journalists, and rational argumentation.
In the opportunity frame, the advancement of automatic processes was overall
portrayed as an opportunity for journalists both with and without the underlying argument that it could lead to better reporting, it was portrayed positively in a quote by a journalists’ union and overall evaluated as a potential positive future benefit for journalists. The opportunity frame had positive adjectives (e.g., “optimistic”) and the word “opportunity” several times throughout. Finally, the word “hope” was incorporated into the quote, since the frame reflects some of the appraisal dimensions of hope. Similarly, the risk frame portrayed automatic processes as a potential threat or risk to journalists with and without the argument that
journalists risk getting fired due to automation. Automated processes were portrayed
negatively in the quote and in general portrayed as a potential negative future loss. Negative adjectives (e.g., “pessimistic”) and “risk” was used throughout. The word “fear” was a part of the quote, since the risk frame share similarities with the appraisal dimensions of fear.
Because the frame differed in the underlying argumentation (i.e., automation framed as a risk of job loss versus an opportunity for better reporting), the strength of the two frames were pretested in a pilot study. Specifically, different versions of two paragraphs which contained the rational argumentation for automation as a risk or an opportunity were tested. The two paragraphs used in the final experiment had almost no difference in terms of frame strength or how convincing, understandable, accessible or persuasive they were. Finally, the stimulus material was formatted to look like an online article (the risk and opportunity frames as well as the control condition can be seen in Appendix B).
Measures
Emotions. In the experiment, nine discrete emotions were measured as potential mediating variables: fear, hope, happiness, sadness, anger, enthusiasm, contentment,
excitement and worry. They were chosen based on the likely elicited emotions of the risk and opportunity frames, i.e. fear and hope, or because they are otherwise typically measured in literature about emotion effects (e.g., Gross, 2008; Lecheler et al., 2015; Lecheler et al., 2013; Major, 2011) in order to ensure that other emotions could be controlled for. Finally, excitement was included as a discrete emotion in order to give respondents sufficient positive emotions to choose from. In the pre-test, guilt, shame and disgust were also measured, but they were rarely elicited by the stimulus material. In addition to that, appraisal theory does not support that these three emotions would indeed play a role in the current context (e.g.,
Roseman et al., 1996), and they were therefore not included in the final experiment. Thus, eight emotions decidedly positive or negative were measured as well as hope, which to some extent is considered positive, but often based in negative circumstances (Lazarus, 1991).
Respondents were asked to rate how much they felt each emotion on a scale from 1 (Not at all) to 7 (Very much) after reading the stimulus (“Thinking back to the news article you read, to what extent did the article make you feel one or more of the following
emotions?” Hopeful: M = 3.19, SD = 1.61; afraid: M = 2.79, SD = 1.61; happy: M = 2.89, SD = 1.49; sad: M = 2.75, SD = 1.70; angry: M = 2.38, SD = 1.50; enthusiastic: M = 3.25, SD = 1.63; content: M = 3.18, SD = 1.50; excited: M = 3.24, SD = 1.67; worried: M = 3.50, SD = 1.73).
Attitude. The dependent variable, attitude towards further automated journalism in the newsroom, was measured using a five-item semantic differential scale (“In general, what do you think about the prospects of further automation in newsrooms? “I think it is…””
unfavourable/favourable; wrong/right; negative/positive; bad/good; foolish/wise) previously
used for measuring attitudes towards a proposed legislative plan (Nabi, 2002a). Higher scores indicated greater support for further automation, M = 3.99, SD = 1.34, α = .94.
Other measures. Some additional variables were measured in the questionnaire but ultimately were not used in the final analysis (see scale reliability tests in Appendix C).4
4 Possible moderators: political orientation (van Spanje & Azrout, 2019); media scepticism (adapted from Tsfati
& Cappella, 2003); affinity for technology (Edison & Geissler, 2003); general scepticism towards automation (consisting of only one item); trait innovativeness (Pallister & Foxall, 1998); a measure of trait anxiety (International Personality Item Pool, n.d.). Note, that given non-native English-speaking respondents, the scale was slightly adapted by changing the word “blue” to “sad” to avoid confusion over the meaning. Alternative dependent variables: attitude towards current automation in journalism; further automated journalism attitude (based on Boomgaarden, Schuck, Elenbaas, & de Vreese, 2011); emerging technology attitude (adapted from Ajani & Stork, 2013); risk perception (adapted from Wilson, Zwickle, & Walpole, 2019); general automation scepticism (adapted from Wike & Stokes, 2018); behavioural intention; behavioural actions.
Manipulation Check
Towards the end of the survey, a scale of three semantic differentials was created as a manipulation check measure. Respondents were asked to rate the article they read on a 7-point scale between three pairs of adjectives (as a risk/as an opportunity; as something
negative/as something positive and as a disadvantage/as an advantage), M = 4.48, SD = 1.67,
α = .95. The manipulation check showed successful manipulation, where readers of the opportunity frame evaluated the material to be in the more positive end of the spectrum (M = 5.52, SD = 1.15) and respondents who read the risk frame rated the article as more negative (M = 2.87, SD = 1.24). Finally, the control condition was rated between the two frame conditions (M = 4.86, SD = 1.36) (F(2, 166) = 68.02, p < .001).
Analysis
This paper studies the framing effect of risk and opportunity frames on opinion
towards automated journalism and the mediating effect of specific discrete emotions. Preacher and Hayes (2008) argue that when testing multiple mediation hypotheses, several simple mediation models are inferior to using one multiple mediation model. Furthermore, the researchers argue that bootstrapping is “the most powerful and reasonable method of obtaining confidence limits for specific indirect effects under most conditions” (p. 886). Therefore, the mediating effect of fear and hope is measured with the PROCESS macro for SPSS, developed by Andrew F. Hayes.5 The same method and tool have successfully been applied in other studies with comparative research design (Lecheler et al., 2015; Lecheler et al., 2013).
Results Main Effects on Attitude
An analysis of variance (ANOVA) showed no significant difference between the effect of the stimuli and the control condition on attitudes towards automated journalism at a 95% confidence interval (CI). No additional differences between the stimuli and the control condition were found at a significance level of .10. Finally, there were no significant
difference between the opportunity frame as opposed to the risk frame at either significance level. Thus, hypotheses H1a and H1b cannot be supported.
Main Effects on Emotions
A significant effect was found between the different frames with regards to hope (F(2, 166) = 3.10, p = .048), and a post hoc test with a Bonferroni correction showed that the opportunity frame was significantly more likely to elicit hope compared to the risk frame (b = .73, SE = .30, p = .042), as expected. The ANOVA showed no significant results for eliciting fear. Thus, this study has found no support for hypothesis H2a, however, there is support for hypothesis H2b when comparing the opportunity condition to the risk condition, but not when comparing it to the control condition.
Mediating Effects
Using the PROCESS macro’s model 4 with the number of bootstrap samples set at 5,000, no significant mediating effect was found for fear or hope on attitudes towards automated journalism at a 95% CI. However, at a 90% CI, which was tested given the directional nature of the hypothesis predicting hope to increase in response to opportunity framing, hope was seen to have a small indirect effect on attitude, when comparing the
opportunity frame to the risk frame, b = .10, SE = .09, 90% CI [.002, .270]. At this CI, the opportunity frame indeed had a significantly more positive effect on elicited hope compared to the risk frame (b = .73, SE = .30, p = .014), while the effect of hope on attitude was just above non-significant (b = .14, SE = .08, p = .104). No mediating effect was found for fear at the 90% CI. Thus, there is some (weak) support for hypothesis H3b but not for H3a and only when applying a 90% CI, therefore it has to be stressed clearly that one needs to remain cautious about this result.
The mediation analysis also showed that while the opportunity frame had a significant positive effect on enthusiasm (b = .51, SE = .30, p = .091), and enthusiasm had a significant effect on attitude towards automated journalism (b = .21, SE = .09, p = .025), the emotion did not mediate the effect, b = .11, SE = .08, 90% CI [-.003, .263]. Despite running the PROCESS macro several times, the lower limit remained below 0. When comparing the two frames, the opportunity frame also had a significant effect on happiness (b = .54, SE = .28, p = .051) and excitement (b = .66, SE = .31, p = .035). Worry had a significant negative effect on attitude (b = -.17, SE = .07, p = .016), while contentment had a significant positive effect (b = .18, SE = .08, p = .021). Figure 1 displays the individual effects of the multiple mediation (tables with individual direct and indirect effects can be found in Appendix D).6
6 The data was also examined for moderated mediation using the PROCESS macro’s model 7 at 95% CI with
5,000 bootstrap samples. While some variables had an effect on the path between frames and the mediators hope and fear, the moderated mediation showed no indirect effect on attitudes towards automated journalism.
Figure 1. Individual Direct and Indirect Effects of the Mediation Analysis. A multiple
mediation model for opportunity framing versus risk framing’s (X) effect on attitudes towards automated journalism (Y) as mediated by nine emotions (M1-9). The model illustrates the direct effect of X on Y (path c’) as well as direct effect paths between X, M1-9 and Y (paths a and b) (Preacher & Hayes, 2008). Hope (marked with bold) had a significant indirect effect on attitude, despite only having a significant direct effect on path a1. Though there were significant effects on paths a7 and b7, the indirect mediating effect of enthusiasm on attitudes was not significant.
Discussion and Conclusion
This study set out to explore the role of discrete emotions in reporting about automated journalism. Overall, the results are in line with previous research showing that some discrete emotions mediate framing effects (e.g., Holm, 2012; Kühne & Schemer, 2015; Lecheler et al., 2013; Major, 2011). More specifically, the findings demonstrate that (a) hope plays a relevant role in the framing effect process, given that framing automated journalism as an opportunity significantly increases the feeling of hope in respondents and that hope seems to mediate the effect of the opportunity frame on attitudes in the expected direction; (b) fear was not an essential emotion in the specific context of this study, since no significant effect was found for risk framing on fear, nor did fear mediate the effect of the risk frame; (c) together, these findings illustrate that the way media frame automated journalism impacts audiences’ attitudes towards the topic, albeit only indirectly and to a modest degree.
While some scholars have claimed that hope has been relatively understudied as a persuader (Nabi, 2002b), newer research into especially climate change communication has shown that hope does have a persuasive effect (Chadwick, 2015) and can mediate framing effects (Bilandzic et al., 2017; Nabi et al., 2018). Thus, this study contributes to the increasing focus on hope in the framing effect process by extending it to the context of framing (new) technology. By showing that the opportunity frame elicited hope, this study has supported the argument that appraisals of a particular frame can elicit specific emotions (Gross, 2008; Kühne, 2014). Despite only having a weak effect, the results also show that hope can change attitudes towards a subject by acting as a mediator. Ultimately, this study has shown that hope should be considered another potential persuasive emotion outside topics “traditionally” linked to hope, such as climate change communication.
While the strongest effect was found on hope, it is important to note that the
opportunity frame was also significantly better at eliciting three out of four positive emotions, namely happiness, enthusiasm and excitement. This is in line with previous research showing that positive and negative frames often elicit multiple emotional reactions with the same valence (e.g., Lecheler et al., 2013), indicating that appraisal of frame valence plays an
important role for eliciting emotions. It is also worth noting that enthusiasm, contentment, and worry had significant effects on attitudes towards further automation in journalism, which supports the argument that discrete emotions are significant variables in the framing effect process and substantiates the relevance of treating emotions as discrete entities in research (Nabi, 2010). Further research could potentially benefit from focusing on these emotions and their effect on attitude. Finally, since enthusiasm has mediated framing effects in other contexts (e.g., Holm, 2012; Lecheler et al., 2015), the somewhat ambiguous results in this study suggests that enthusiasm also has the potential to work as a mediator in the context of framing technology and could thus also serve as a prospective research path.
It is difficult to determine the specific reason for the non-significant effect of the risk frame on eliciting fear. However, emotion theory offers a possible explanation: Objects or circumstances have to be regarded as meaningful, relevant or significant to the one appraising it before an emotional reaction will occur (e.g., Lazarus, 1991; Moors et al., 2013; Scherer, 2005; Smith & Ellsworth, 1985). More specifically, Nabi (1999) writes that “to experience fear, receivers must perceive a message to suggest a threat to well-being to which they or
someone with whom they empathize [emphasis added] may be susceptible” (p. 307). In other
words, it could be argued that participants who received the risk frame either did not perceive automation of journalism as a relevant threat to themselves or journalists and/or did not empathise sufficiently with journalists and their situation. This seems plausible, especially
given that automated journalism is a rather new technological development (Carlson, 2015) with a limited distribution in some areas of the world (Wölker & Powell, 2018), indicating that the extent of its usage is probably not widely known to the general public.
While the initial non-significant main effect of the frames tested in this study on attitudes might also partly explain the non-mediating effect of fear in this study, the relationship between fear and information processing depth is potentially another critical point. Though contingent on other factors as well, Nabi (1999, 2002a) posits that avoidance emotions such as fear lead to less careful information processing compared to approach-based emotions, which could explain why fear was not persuasive in the context of this study. However, other research suggests that fear leads to more careful information processing (e.g., Garry, 2014; Marcus & MacKuen, 1993). The specific function of information processing depth cannot be extracted from the present research and further research should therefore delve into it given its potential role in framing effect process.
The present research has shown that discrete emotions play different roles in the framing effect process depending on how media covers the ongoing automation in the
newsroom. Notably, emotions had different effects on audiences’ attitude towards automated journalism if it was framed as an opportunity or as a risk. Arguably, this contributes to the understanding of the role of emotions in the framing of emerging technologies in a broader sense, given that new technology is often framed as risks and opportunities (see e.g., Allan et al., 2010; Marks et al., 2007; Pennington & Birthisel, 2016). Importantly, the findings reflect a comparison between the opportunity and the risk frames (i.e., comparing two opposite alternatives with each other), not when comparing the two frames to the neutral control condition. This sort of comparison has been done before (e.g., Schuck & de Vreese, 2006), however, it is important that the results be interpreted in that context – that is, it does not
seem to matter if media cover automated journalism as an opportunity as opposed to neutral or no coverage, but it significantly differed from framing it as a risk. Further research should focus on other areas of emerging technologies to which the general population is maybe more knowledgeable and has formed opinions about, for instance gene splicing or social media data tracking, in order to gauge if the effect of fear and hope is similar to that of the coverage of automated journalism.
Some limitations of this study need to be addressed as well. Firstly, while the external validity of the stimulus material is high, it is not completely so, given that journalistic
coverage of automated journalism often presents both the positive and the negative aspects of automation in the newsroom in the same article. Because actual news coverage is more nuanced than the stimulus material presented in this study, news readers will most likely experience not just one emotion and the effects on attitudes towards automated journalism will be more complex than the ones shown here. However, a stringent division between the risk and opportunity framing was necessary in order to gauge the effect of the two types of framing.
Secondly, there are some issues of generalisability with the sampling method used in this study, despite the random assignment to conditions. Convenience sampling is a common approach to survey experiments and research has shown that in the case of framing effect experiments, the results of convenience samples were comparable to those of representative samples (Mullinix, Leeper, Druckman, & Freese, 2015). However, in the case of convenience sampling, generalisability is not necessarily obtained, because “its relationship to the
population of interest is unknown and typically unknowable” (Mullinix et al., 2015, p. 123), and, when possible, representative sampling is therefore recommended for further research.
Finally, a number of additional variables were measured in the survey, however, only a few were used in the final analysis. There are several reasons for that. First of all, the attitude measurement used in the study was chosen because it was the most reliable (α = .94) and the measurement that most often was impacted by frames and emotions when analysed. Additionally, several possible moderators were measured but not included in this study, given their relatively minor effect on the overall mediation model. However, they did seem to play some role as moderators, indicating that these variables could provide a fuller picture of the framing effect process, and future studies could possibly benefit from focusing on moderated mediation as well.
In conclusion, this study has shown that news readers are impacted by the discrete emotion of hope in reporting about automated journalism, albeit only to a modest degree, especially when the coverage is framed as an opportunity. At the same time, fear does not seem to play major role in the framing effect process – at least in the context of framing automated journalism as a risk. Broadly speaking, the results are in line with similar research, which has shown that discrete emotion theory is useful for understanding the framing effect process (e.g., Lecheler et al., 2015) and has extended that knowledge by showing that this is also the case in the context of the framing of new and emerging technologies and their increasing use in journalism.
References
Aarøe, L. (2011). Investigating frame strength: The case of episodic and thematic frames. Political Communication, 28(2), 207-226.
Ajani, T., & Stork, E. (2013). Creating a semantic differential scale for measuring users’ perceptions and attitudes toward emerging technologies. 2013 Proceedings of the
Conference for Information Systems Applied Research, 1-7. Retrieved from
http://proc.conisar.org/index.html
Allan, S., Anderson, A., & Petersen, A. (2010). Framing risk: Nanotechnologies in the news.
Journal of Risk Research, 13(1), 29-44. doi:10.1080/13669870903135847
Automated Insights. (n.d.). Associated Press. Retrieved May 12, 2020 from https://automatedinsights.com/customer-stories/associated-press/
Bilandzic, H., Kalch, A., & Soentgen, J. (2017). Effects of goal framing and emotions on perceived threat and willingness to sacrifice for climate change. Science
Communication, 39(4), 466-491. doi:10.1177/1075547017718553
Boomgaarden, H. G., Schuck, A. R. T., Elenbaas, M., & de Vreese, C. H. (2011). Mapping EU attitudes: Conceptual and empirical dimensions of Euroscepticism and EU support. European Union Politics, 12(2), 241–266. doi:10.1177/1465116510395411 Brader, T. (2005). Striking a responsive chord: How political ads motivate and persuade
voters by appealing to emotions. American Journal of Political Science, 49(2), 388-405. doi:10.2307/3647684
Bruininks, P., & Malle, B. F. (2005). Distinguishing hope from optimism and related affective states. Motivation and Emotion, 29(4), 327-355. doi:10.1007/s11031-006-9010-4 Carlson, M. (2015). The robotic reporter: Automated journalism and the redefinition of labor,
compositional forms, and journalistic authority. Digital Journalism, 3(3), 416-431. doi:10.1080/21670811.2014.976412
Chadwick, A. E. (2015). Toward a theory of persuasive hope: Effects of cognitive appraisals, hope appeals, and hope in the context of climate change. Health
Communication, 30(6), 598-611. doi:10.1080/10410236.2014.916777
Chong, D., & Druckman, J. N. (2007). Framing theory. Annual Review of Political Science,
10(1), 103-126. doi:10.1146/annurev.polisci.10.072805.103054
Coddington, M. (2015). Clarifying journalism’s quantitative turn: A typology for evaluating data journalism, computational journalism, and computer-assisted reporting. Digital
Journalism, 3(3), 331-348. doi:10.1080/21670811.2014.976400
de Vreese, C. H. (2005). News framing: Theory and typology. Information Design Journal +
Document Design, 13(1), 51-62. doi:10.1075/idjdd.13.1.06vre
de Vreese, C. H., & Boomgaarden, H. (2003). Valenced news frames and public support for the EU. Communications, 28(4), 361-381. doi:10.1515/comm.2003.024
Delcker, J. (2019, March 13). This story was not written by a robot. Politico. Retrieved from https://www.politico.eu
Druckman, J. N. (2001). The implications of framing effects for citizen competence. Political
Behavior, 23(3), 225-256. doi:10.1023/A:1015006907312
Druckman, J. N., & McDermott, R. (2008). Emotion and the framing of risky choice. Political Behavior, 30(3), 297-321. doi:10.1007/s11109-008-9056-y Edison, S. W., & Geissler, G. L. (2003). Measuring attitudes towards general technology:
Antecedents, hypotheses and scale development. Journal of Targeting, Measurement
and Analysis for Marketing, 12(2), 137-156. doi:10.1057/palgrave.jt.5740104
Entman, R. M. (1993). Framing: Toward clarification of a fractured paradigm. Journal of
Garry, J. (2014). Emotions and voting in EU referendums. European Union Politics, 15(2), 235-254. doi:10.1177/1465116513514780
Graefe, A. (2016). Guide to automated journalism. doi:10.7916/D80G3XDJ
Groenendyk, E. (2011). Current emotion research in political science: How emotions help democracy overcome its collective action problem. Emotion Review, 3(4), 455-463. doi:10.1177/1754073911410746
Gross, K. (2008). Framing persuasive appeals: Episodic and thematic framing, emotional response, and policy opinion. Political Psychology, 29(2), 169-192.
doi:10.1111/j.1467-9221.2008.00622.x
Gross, K., & Brewer, P. R. (2007). Sore losers: News frames, policy debates, and emotions.
Harvard International Journal of Press/Politics, 12(1), 122-133.
doi:10.1177/1081180X06297231
Gross, K., & D’Ambrosio, L. (2004). Framing emotional response. Political
Psychology, 25(1), 1-29. doi:10.1111/j.1467-9221.2004.00354.x
Holm, E. M. (2012). Emotions as mediators of framing effects (PhD dissertation, Aarhus University, Denmark). Retrieved from https://politica.dk/politicas-phd-serie/else-marie-holm/
International Personality Item Pool. (n.d.). The items in the 15 preliminary IPIP
scales measuring constructs similar to those in the Jackson Personality Inventory (JPI-R). Retrieved April 3, 2020 from https://ipip.ori.org/newJPI-RKey.htm
Iyengar, S. (1991). Is anyone responsible? How television frames political issues. Chicago, IL: The University of Chicago Press.
Kühne, R. (2014). Political news, emotions, and opinion formation: Toward a model of
emotional framing effects (NCCR Working Paper No. 68). Retrieved from Semantic
Scholar website: https://www.semanticscholar.org/paper/Political-news%2C-
emotions%2C-and-opinion-formation%3A-a-Kühne/2ffc18cd869bb66f4abdd9f2e809c5e35a64f153
Kühne, R., & Schemer, C. (2015). The emotional effects of news frames on information processing and opinion formation. Communication Research, 42(3), 387-407. doi:10.1177/0093650213514599
Lazarus, R. S. (1991). Emotion and adaptation. New York, NY: Oxford University Press. Lazarus, R. S. (1999). Hope: An emotion and a vital coping resource against despair. Social
Research, 66(2), 653-678. Retrieved from https://www.jstor.org
Lecheler, S., Bos, L., & Vliegenthart, R. (2015). The mediating role of emotions: News framing effects on opinions about immigration. Journalism & Mass Communication
Quarterly, 92(4), 812-838. doi:10.1177/1077699015596338
Lecheler, S., Schuck, A. R. T., & de Vreese, C. H. (2013). Dealing with feelings: Positive and negative discrete emotions as mediators of news framing effects. Communications,
38(2), 189-209. doi:10.1515/commun-2013-0011
Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition and Emotion, 14(4), 473-493. doi:10.1080/026999300402763
Linden, C.-G. (2017). Decades of automation in the newsroom: Why are there still so many jobs in journalism? Digital Journalism, 5(2), 123-140.
Major, L. H. (2011). The mediating role of emotions in the relationship between frames and attribution of responsibility for health problems. Journalism & Mass Communication
Quarterly, 88(3), 502-522. doi:10.1177/107769901108800303
Marcus, G. E., & MacKuen, M. B. (1993). Anxiety, enthusiasm, and the vote: The emotional underpinnings of learning and involvement during presidential campaigns. The
American Political Science Review, 87(3), 672-685. doi:10.2307/2938743
Marcus, G. E., Neuman, W. R., & MacKuen, M. (2000). Affective intelligence and political
judgment. Chicago, IL: University of Chicago Press.
Marks, L. A., Kalaitzandonakes, N., Wilkins, L., & Zakharova, L. (2007). Mass media framing of biotechnology news. Public Understanding of Science, 16(2), 183-203. doi:10.1177/0963662506065054
Montal, T., & Reich, Z. (2017). I, robot. You, journalist. Who is the author? Authorship, bylines and full disclosure in automated journalism. Digital Journalism, 5(7), 829-849. doi:10.1080/21670811.2016.1209083
Moors, A., Ellsworth, P. C., Scherer, K. R., & Frijda, N. H. (2013). Appraisal theories of emotion: State of the art and future development. Emotion Review, 5(2), 119-124. doi:10.1177/1754073912468165
Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Freese, J. (2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2(2), 109-138. doi:10.1017/XPS.2015.19
Nabi, R. L. (1999). A cognitive-functional model for the effects of discrete negative emotions on information processing, attitude change, and recall. Communication Theory, 9(3), 292-320. doi:10.1111/j.1468-2885.1999.tb00172.x
Nabi, R. L. (2002a). Anger, fear, uncertainty, and attitudes: A test of the cognitive-functional model. Communication Monographs, 69(3), 204-216. doi:10.1080/03637750216541 Nabi, R. L. (2002b). Discrete emotions and persuasion. In J. P. Dillard & M. Pfau (Eds.), The
persuasion handbook: Developments in theory and practice (pp. 289-308). Thousand
Oaks, CA: Sage Publications.
Nabi, R. L. (2003). Exploring the framing effects of emotion: Do discrete emotions differentially influence information accessibility, information seeking, and policy preference? Communication Research, 30(2), 224-247.
doi:10.1177/0093650202250881
Nabi, R. L. (2010). The case for emphasizing discrete emotions in communication research.
Communication Monographs, 77(2), 153-159. doi:10.1080/03637751003790444
Nabi, R. L., Gustafson, A., & Jensen, R. (2018). Framing climate change: Exploring the role of emotion in generating advocacy behavior. Science Communication, 40(4), 442-468. doi:10.1177/1075547018776019
Pallister, J. G., & Foxall, G. R. (1998). Psychometric properties of the Hurt–Joseph–Cook scales for the measurement of innovativeness. Technovation, 18(11), 663-675. doi:10.1016/S0166-4972(98)00070-4
Peiser, J. (2019, February 5). The rise of the robot reporter. The New York Times. Retrieved from https://www.nytimes.com
Pennington, R., & Birthisel, J. (2016). When new media make news: Framing technology and sexual assault in the Steubenville rape case. New Media & Society, 18(11), 2435-2451. doi:10.1177/1461444815612407
Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research
Methods, 40(3), 879-891. doi:10.3758/BRM.40.3.879
Reinard, J. C. (2006). Communication research statistics. Thousand Oaks, CA: Sage Publications.
Roseman, I. J. (1991). Appraisal determinants of discrete emotions. Cognition and
Emotion, 5(3), 161-200. doi:10.1080/02699939108411034
Roseman, I. J., Antoniou, A. A., & Jose, P. E. (1996). Appraisal determinants of emotions: Constructing a more accurate and comprehensive theory. Cognition and
Emotion, 10(3), 241-278. doi:10.1080/026999396380240
Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science
Information, 44(4), 695-729. doi:10.1177/0539018405058216
Scheufele, D. A. (1999). Framing as a theory of media effects. Journal of Communication,
49(1), 103-122. doi:10.1111/j.1460-2466.1999.tb02784.x
Schuck, A. R. T., & de Vreese, C. H. (2006). Between risk and opportunity: News framing and its effects on public support for EU enlargement. European Journal of
Communication, 21(1), 5-32. doi:10.1177/0267323106060987
Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of
Personality and Social Psychology, 48(4), 813-838. doi:10.1037/0022-3514.48.4.813
Tomkins, S. S. (1963). Affect, imagery, consciousness: Volume II. The negative affects. New York, NY: Springer Publishing Company.
Tsfati, Y., & Cappella, J. N. (2003). Do people watch what they do not trust? Exploring the association between news media skepticism and exposure. Communication Research,
Valentino, N. A., Brader, T., Groenendyk, E. W., Gregorowicz, K., & Hutchings, V. L. (2011). Election night’s alright for fighting: The role of emotions in political participation. The Journal of Politics, 73(1), 156-170.
doi:10.1017/S0022381610000939
van Dalen, A. (2012). The algorithms behind the headlines: How machine-written news redefines the core skills of human journalists. Journalism Practice, 6(5-6), 648-658. doi:10.1080/17512786.2012.667268
van Spanje, J. H. P., & Azrout, R. (2019). Bringing background back in: A Dutch new party and the revival of socio-economic background voting. Comparative European
Politics. Advance online publication. doi:10.1057/s41295-019-00189-y
Vasilopoulou, S., & Wagner, M. (2017). Fear, anger and enthusiasm about the European Union: Effects of emotional reactions on public preferences towards European integration. European Union Politics, 18(3), 382-405.
doi:10.1177/1465116517698048
Wike, R., & Stokes, B. (2018). In advanced and emerging economies alike, worries about job
automation. Retrieved from
https://www.pewresearch.org/global/wp- content/uploads/sites/2/2018/09/Pew-Research-Center_In-Advanced-and-Emerging-Economies-Alike-Worries-about-Job-Automation_2018-09-13.pdf
Wilson, R. S., Zwickle, A., & Walpole, H. (2019). Developing a broadly applicable measure of risk perception. Risk Analysis: An International Journal, 39(4), 777-791.
doi:10.1111/risa.13207
Wölker, A., & Powell, T. E. (2018). Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism. Journalism. Advance online publication. doi:10.1177/1464884918757072
Wu, S., Tandoc, E. C., Jr., & Salmon, C. T. (2019). Journalism reconfigured: Assessing human–machine relations and the autonomous power of automation in news production. Journalism Studies, 20(10), 1440-1457.
Appendix A Survey Questions and Items.
Page 1: General Introduction and Informed Consent Dear Sir or Madam,
I would like to invite you to participate in a study for the Amsterdam School of
Communication Research (ASCoR) at the University of Amsterdam (The Netherlands). The study we are asking you to participate in is about getting insight into your views on
journalism and current affairs. This will take ca. 15 minutes. Everyone at and above the age of 18 can participate in this project.
As this research is being carried out under the responsibility of the ASCoR, University of Amsterdam, we can guarantee that:
1. Your anonymity will be safeguarded, we will not collect personal information such as names, addresses, IP-addresses, photos, videos etc. Fully anonymized research data can be shared with other researchers.
2. You can refuse to participate in the research or cut short your participation without having to give a reason for doing so. You also have up to 7 days after participating to withdraw your permission to allow your answers or data to be used in the research. 3. Participating in the research will not entail your being subjected to any appreciable
For more information about the research and the invitation to participate, you can send an email to the project leader Andreas Schuck (a.r.t.schuck@uva.nl). Should you have any complaints or comments about the course of the research and the procedures it involves as a consequence of your participation in this research, you can contact the designated member of the Ethics Committee representing ASCoR: ascor-secr-fmg@uva.nl. Any complaints or comments will be treated in the strictest confidence.
We hope that we have provided you with sufficient information. We would like to thank you in advance for your assistance with this research.
–––————
I hereby declare that I have been informed in a clear manner about the nature and method of the research.
I agree, fully and voluntarily, to participate in this research study. With this, I retain the right to withdraw my consent, without having to give a reason for doing so. I am aware that I may halt my participation in the experiment at any time.
If my research results are used in scientific publications or are made public in another way, this will be done such a way that my anonymity is completely safeguarded.
I have read and understood the above text and I: • Agree to participate in the research study • Do not wish to participate in the research study
Page 2: Pre-Test Information
In the following, we would like to ask you a number of questions related to different topics. Please give your honest opinion, and bear in mind that there are no right or wrong answers.
Please be aware that you cannot go back and change your answers in the survey as you continue through it.
Page 3: Political Orientation
In political matters, people talk about “the left” and “the right”. What is your position? Please indicate your views using any number on a scale from 0 to 10, where 0 means “left” and 10 means “right”. Which number best describes your position?
Note. Set up as a scale: (0) Left – (10) Right; numbers 0 and 10 are labelled with a number
and text, while the remaining positions are only labelled with numbers.
Page 4: Media Scepticism
To what extent do you agree or disagree with the following statements about news media in general: “In general, news media…”
• … cannot be trusted” (reverse coded) • … are accurate”
• … are fair”
• … tell the whole story”
• … help society solve its problems”
• … care more about being first to report a story over accuracy in reporting” (reverse
Note. Set up as a randomised matrix; answers ranging from “(1) Strongly disagree” to “(7)
Strongly agree”.
Page 5: Affinity for Technology
Next, we would like to know about your relationship with technology (technology should be interpreted in the broadest and most general sense and includes – but is not limited to – such things as DVDs, smartphones, fibre optics, GPS, CAT scans, genetic engineering and the internet). How well do the following statements apply to you on a scale from 1 to 7?
• Technology is my friend
• I enjoy learning new computer programs and hearing about new technologies • People expect me to know about technology and I don’t want to let them down • If I am given an assignment that requires that I learn to use a new program or how to
use a machine, I usually succeed
• I relate well to technology and machines • I am comfortable learning new technology
• I know how to deal with technological malfunctions or problems • Solving a technological problem seems like a fun challenge • I find most technology easy to learn
• I feel as up to date on technology as my peers
Note. Randomised matrix; (1) Strongly disagree – (7) Strongly agree.
According to Cambridge Dictionary, automation is “the use of machines and computers that can operate without needing human control”. To what extent do you agree with the following statement about automation:
• In general, automation is a good thing
Note. (1) Strongly disagree – (7) Strongly agree.
Page 7: Trait: Innovativeness
On a scale from 1 to 7, to what extent do you think the following statements describe your personality in general?
• I am generally cautious about accepting new ideas (reverse coded)
• I am suspicious of new inventions and new ways of thinking (reverse coded)
• I rarely trust new ideas until I can see whether the vast majority of people around me accept them (reverse coded)
• I am aware that I am usually one of the last people in my group to accept something new (reverse coded)
• I am reluctant about adopting new ways of doing things until I see them working for people around me (reverse coded)
• I find it stimulating to be original in my thinking and behaviour
• I tend to feel that the old way of living and doing things is the best way (reverse
coded)
• I am challenged by ambiguities and unsolved problems
• I must see other people using new innovations before I will consider them (reverse
coded)
Note. Randomised matrix; (1) Strongly disagree – (7) Strongly agree.
Page 8: Trait: Anxiety
On a scale from 1 to 7, to what extent do you think the following statements describe your personality in general?
• I get stressed out easily • I worry about things • I get upset easily
• I have frequent mood swings • I often feel sad
• I am relaxed most of the time (reverse coded) • I am not easily bothered by things (reverse coded) • I rarely get irritated (reverse coded)
• I seldom feel sad (reverse coded)
• I am not easily frustrated (reverse coded)
Note. Randomised matrix; (1) Very inaccurate – (7) Very accurate.
Page 9: Introduction to Stimulus Material
On the next page you will see a short news article that has recently been published in the news media.
The article will remain on the screen for a minimum of 10 seconds before you can proceed, but you may take as much time as you like to read it. When you have finished, scroll down and click the arrow to proceed.
We will ask you to answer some questions about the news article.
Please note: Read the news article carefully; you will not be able to access it again later.
Page 10: Stimulus Material
Note. Participants are randomly assigned to 1 of 3 conditions: (a) Opportunity frame
condition; (b) risk frame condition; (c) control condition.
Page 11: Open Question
We are interested in what you were thinking about when you read the news article. Please write down your reactions below. Please list as many thoughts, feelings and considerations as you might have. Do not worry about punctuation, spelling, grammar, or the use of complete sentences.
• [Text box]
Note. Open-ended response.
Page 12: Emotional Response #1
Thinking back to the news article you read, to what extent did the article make you feel one or more of the following emotions?
• Happy • Hopeful • Sad • Angry
