Do I do what I feel?
The effects of emotional framing on political behavior Raoul C. Wijsma
Abstract
In news framing effect studies, a growing amount of researchers acknowledges the mobilizing role that emotions play (Gross, 2008) on public opinion. However, their role as mediators on behavior has not been addressed much. In this online experiment (N = 123), the role of a specific kind of emotions is addressed. Namely, moral
emotions connect well to behavioral intentions. In total, seven moral emotions were issued (anger, compassion, disgust, moral outrage, pride, sadness and shame), as well as two cognitive mediators (belief content, and belief importance). The effects of a moral emotion in-group, and a moral emotion out-group frame on people’s political behavior via moral emotions are addressed in this study. A direct effect of exposure to the frames on political behavior was found. Contrary to the hypotheses, results
showed that participants in the out-group were more likely to engage into political behavior, compared to the in-group. Despite the fact that the frames elicited two of the moral emotions (shame and anger), they did not mediate the effect on political
behavior, nor did the cognitive mediators.
Introduction
The role of the media is very important in how individuals understand the world around them (Blumler & Kavanagh, 1999). Media shape news content by using certain frames, which suggest that some aspects of a political issue are more important than others (Scheufele, 1999). The effects of those news media frames are proved to influence people’s reactions and behavior (Entman, 1993). Therefore, recent framing studies tried to provide a more nuanced image of how news frames cause effects (Schuck, & Feinholdt, in press). For example, by studying their underlying processes (i.e. their role as mediators). According to Gross (2008), most of these studies merely focused on cognitive reactions (Slothuus, 2008) as mediators. However, despite the fact there is little doubt that emotions affect the political choices people make (Mutz, 2007), the role of emotions as mediators of framing effects is often overlooked in the field of political communication (Schuck, & Feinholdt, in press). A small (but
growing) number of studies (Gross & Brewer, 2007; Gross & d’Ambrosio, 2004; Gross, 2008; Kühne & Schemer, 2013) did study this relationship. However, most of them looked at mediation effects on people’s political opinions. According to Scherer (2005) emotions are also capable of driving people’s behavior. In political contexts, the effects of emotions should not be underestimated. Thompson (1973) states that people’s moral values, and standards regarding a political issue or event are important in their motives for political behavior. Emotions include that moral aspect (Haidt 2003). In his research, Haidt (2003) stresses the connection between morality and emotions: morally relevant emotions are mobilizing because of their altruistic character. If a certain frame is able to evoke emotions (Lecheler, Schuck, & de Vreese, 2013), and emotions and political behavior are connected with morals and values (Baron, 2003); moral emotions (as an underlying process) should be able to
influence the relationship between frame exposure and political behavior because of their mobilizing function (Skitka, & Wisneski, 2011).
By means of an experiment, I will study this relationship in news frames that portray a so-called in-group and out-group. According to Judd & Downing (1995) citizens routinely make judgments about groups and issues/policies/events in the political arena. In- and out-groups (Tajfel, 1982) are interesting in this context because of their different viewpoint towards an issue. For example, when in-groups grow large, the moral rules that confirm their (superior) status are turned into a moral authority that is “incompatible with tolerance for difference” (Brewer, M. B., 1999, p. 435. This leads to negative, emotional evaluations of the out-group. Therefore, I expect that people exposed to news frames with emotional language (that portray the political situation of in- and out-groups), will have different judgment and behavior regarding those groups. I designed an in- and out-group moral emotion frame by means of a news article, in which the increase of gay violence and its implications in respectively the Netherlands, and Russia is described. The Netherlands is considered the in-group, since Dutch citizens were participants in this experiment. In this study, political behavior is seen in a non-electoral context. Thus, I mainly try to expand the knowledge we have about the workings of morality and emotions on (relatively) unaddressed political behavior by stressing this connection in a value based context.
In this paper, I discuss several aspects of framing theory, its application on this study’s design, and political behavior (e.g. Gross, 2008; Iyengar, & Simon 1993; Kühne & Schemer, 2013; Lecheler, Schuck, & de Vreese, 2013; Nabi, 2003, 2010). I then outline moral emotions literature, their role as mediators of framing effects and the experimental design flowing from the main question. Finally, the results will be described and discussed.
Theory Frames and Framing Effects
De Vreese (2005) states that media are “a cornerstone institution in our democracies” (p. 51). The way in which a certain version of reality in a news message is presented via the media, influences how people receive and interpret that message. In short, this is what the concept of framing is about. In this study, Entman’s (1993) definition of framing is used: “To frame is to select some aspects of a perceived reality and make them more salient in a communicating text in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment
recommendation” (p. 52). Framing is considered a communication process in which frame building, frame setting and the consequences of the media frame are important. A frame shapes thought, and behavioral output (Druckman, 2001). Thus, news frames are about the presentation of issues (de Vreese, 2005) and their effects.
Different types of frames have been studied in political communication. In news framing, generally a distinction between generic news frames and issue-specific news frames is made. Generic news frames are applicable to a wide range of topics (Lecheler et al., 2013; Lecheler, & de Vreese, 2012; de Vreese, 2005), and are
opposed to issue-specific frames (Shah, Watts, Domke, & Fan 2002) that focus on just a single topic. The most important generic news frame in political discourse is the conflict frame (de Vreese, 2012). Indeed, the presence of a conflict is important for a news message to be picked up in the news (Harcup, & O’Neill, 2010). Furthermore, human impact, and economic frames (Neuman, Just & Crigler, 1992), thematic and episodic frames often appear in news coverage (Iyengar, & Simon, 1993).
Moreover, news frames often hold a certain valence. That is to say, in political discourse an issue is often framed in terms of opposition or support. This leads to a
certain judgment and/or behavior towards that issue (Chong & Druckman, 2007). For example, morality is one of those issues many researchers in political communication have focused on (e.g. Brewer, 2002; Brewer, 2003; Domke, Shah, & Wackman, 2000; Pan, Meng, & Zhou, 2010).
Thus, the way a frame is designed explains the effects of a news frame (Bizer, Larsen, & Petty, 2010). The strength of framing effects is dependent on the individual and the context (Schuck, & Feinholdt, in press). A main goal in this experiment is to look at the underlying processes that cause framing effects. In this study, I expect to find an indirect effect on the relationship between frame exposure and political behavior via moral emotions. Contrary to the framing studies described above, this study issues the in- and out-group distinction to explain emotional framing effects. This distinction made because of its connection with morality (Brewer, M.B., 1999). By doing so, new insights on framing of gay topics are provided. Earlier work (e.g. Brewer, 2002, 2003) framed gay topics in terms of morality and equality, in which the main point was to test differences in pro- and opponents on equality issues. I argue that these studies are now dated, or do not apply to the (legal) situation in the Netherlands. Instead, I will focus on the emergence of gay violence (as something morally wrong) against in- and out-groups, in which the in- and out-group distinction in the frames influences Dutch citizens in their experience and evaluations.
In- and out-group Framing
According to Judd and Downing (1995), the political arena is the perfect setting to study judgment and behavior of different groups with regard to their positions on certain political issues and events. Position taking often results in a conflict between proponents and opponents of an issue. For the media, this is ideal with regard to news
selection (de Vreese, 2005). A distinction between “us” and “them” encourages group feeling and thinking (Walters, & Murphy, 2008, p. 81). This distinction has been much reflected in news framing studies (Bizer, Larsen, & Petty, 2010; Corkalo & Kamenov, 2003; Lee & Ottati, 2010).
This way of thinking is derived from the classic intergroup theory of Tajfel (1982), who states that a group can be defined on the basis of internal and external criteria like group identification, and emotional investment. A requirement for group behavior is that a group can only exist when others recognize it as a group. Social identity is the key to in-group formation. It refers to that part of the individual’s self that is derived from his knowledge of his group membership, combined with the emotional significance and value of that membership (Tajfel, 1982).
The in-, out-group distinction can be found as well with regard to national identity (Brewer, M. B., 1999). For people to feel part of a national community, national practices that frame our everyday lives are needed (Anderson, 1991, & Brubaker, 1996). That way people can imagine they share the same experiences through communicative symbols like language, cultural values, morals, and beliefs (Sindic, 2011). Based on these rules, people tend to emphasize the favorable
characteristics of their own in-group (Tajfel, 1982). According to Rivenburgh (2000) media favor national identity by framing news in terms of shared national
membership.
In the Netherlands, approval of homosexuality is 81 % (van den Akker, van der Ploeg, & Scheepers, 2013). This general tendency towards tolerance, and favorable attitudes is reflected in the public opinion and morality of the Netherlands (van de Meeredonk, Eisinga, & Felling, 2003). In politics, homosexuality has a moral dimension (Green, 2005), whether or not in terms of support or opposition. People’s
opinions derive from several (inter) personal characteristics and backgrounds like nationality, education, religion, and age (van den Akker et al. 2013). Framing homosexuality in terms of in- and out-groups could therefore be interesting. The moral aspects, on which these groups thrive, are important with regard to their effect on people’s reactions. In this case: political behavior. Therefore, I choose for a distinction between news frames about gay violence in the Netherlands and Russia. The Netherlands is the in-group and Russia is the out-group. Given the fact that common moral rules and standards result in in-group favoritism, and that media encourage national identity by presenting news likewise, I expect that the Dutch participants in this experiment will have more favorable feelings towards the in-group frame (support the in-group), compared to the out-group frame. Additionally, in terms of behavior, the moral emotions that these frames elicit, will lead to the emergence of non-electoral forms of behavior.
Political Behavior
With regard to the role of emotions in news framing, most studies focused on (public) opinion as the dependent variable (Gross, 2008; Lecheler, et al. 2013; Nabi, 2003; Nelson, 2012). Despite the fact that the importance of emotions on decision-making (Kühne, & Schemer, 2013) is increasingly addressed in empirical studies, their effect on political behavior has not been studied much. Groenendyk (2011) states that there is one important reason why people are willing to politically participate: their
experience of emotions. This is in line with Haidt’s (2003) statement that moral emotions would respond to (moral) behavior. Participation in people is expressed when their social identity refers to a feeling of group identity. It is one of the strongest determinants of political behavior (Groenendyk, 2011), and earlier explained by
means of Tajfel’s (1982) intergroup theory. Therefore, I expect that this study’s news frames, in terms of national in- and out-groups will encourage political behavior via experiencing moral emotions. For example, Borah (2013) looked at framing effects on behavior via emotions. He found that emotions mediate the interaction between incivility and news frames (on gay marriage) in increasing willingness to engage into online political participation (e.g. signing a petition). This strengthens my hypothesis that emotions mediate behavioral intentions. In this study, there will be a focus on non-electoral forms of behavior. In other words, the emergence of political behavior in people with regard to non-electoral settings is studied. According to Dalton (2008), and Martin (2012) these forms of political participation expand the possibilities of citizenship. They distinguished three types of non-electoral behavior: direct action, protest, and Internet activism. No types of specific moral behavior are addressed, since the question what exactly is morally relevant behavior is dependent on the individual (Baron, 2003).
Emotions as Mediators of Framing effects
In scientific research, the cognitive effects of news framing have been studied a lot (Gross, 2008). Slothuus (2008) argues a dual process model of issue framing effects. He states that framing effects are mediated by belief importance, in which already accessible considerations concerning an issue are made more important than others. Furthermore, belief content is a mediator as well. This means that frames provide new information/considerations to an individual that, not only alters the (importance of) existing beliefs, but also the beliefs themselves (de Vreese, Boomgaarden & Semetko, 2011).
mediators of framing effects as well (Gross, 2008; Kühne & Schemer, 2013). For example, Lecheler et al. (2013) found that anger and enthusiasm mediated the relationship between positive/negative news frames and policy opinions. The following definition will explain what emotions are: “Internal, mental states that represent evaluative, valenced reactions to events, agents or objects that vary in intensity” (Nabi, 2003, p. 226). It is generally accepted that emotions are referred to as neural states triggered by events that have relevance for a person's well being. According to Damasio (2001), reasoning requires emotions, because emotions play a role in the decisions we make, and the behavior we exhibit. In other words, it has a cognitive aspect. In (political) news framing, emotions like anger, sadness (Kühne, 2014) and fear (Solloway, Slater, Chung & Goodall, 2013) are often issued in frames, because they are likely to appear in people as a reaction to political issues.
Along this line of reasoning there are two major schools of studying emotions with regard to framing effects. The affective intelligence theory of Marcus, McKuen and Neuman (2011) is not taken into account in this study, and will therefore not be discussed. In this study, the theory of cognitive appraisal will be issued to understand the workings of emotions. According to this theory, a person first appraises a situation in order for an emotion to be elicited. The appraisal, a cognitive interpretation implicit in a person’ mind, signifies the importance for one’s goals and precedes the emotion (Brader, & Marcus, 2013; Clore, & Ortony, 2008). For example, when a participant assigned to an in- or out-group reads an article with an emotional frame, he appraises the content of the text and relates it to his goals/standards/values. His emotional response is triggered by cognitive appraisals of the emotions in the frame article, leading to behavioral intentions. In that sense, emotions mediate the relationship between the frames and behavior. This study expands this knowledge by testing the
role of a specific kind of emotions: moral emotions. By doing so, a better understanding of their workings in political communication will be established.
Studying Moral Emotions
According to Markus (2000) moral values and position taking are important in democratic societies. Morality is a key element in maintaining social relations with other people (Tangney et al. 2007), and a tool in deciding about how to deal with social, and political values, and issues in order to live a “moral, constructive live”. The experience of moral emotions is important because they provide the link between moral standards and behavior (Tangney et al. 2007, p. 346). In this study, gay
violence in society is addressed. The emotions I expect people to feel about this issue are morally relevant (Haidt, 2003) and important with regard to people’s evaluation and exhibition of political behavior that flows from exposure to an in- or out-group frame. By studying this relationship new insights will be provided about emotions and their connection to morality with regard to movement and public opinion on current issues. Moreover, this study could prove that moral emotions are the key element in people’s evaluation of politics. The experience of moral emotions is stressed in two ways. The story in the frame addresses moral questions and is therefore interesting for moral evaluation. Namely, the in- and out-group frames establish the contradiction that leads to moral evaluation once a certain point of view in the story is taken.
Furthermore, when the cognitive appraisals of certain moral emotions in the frame are converted to emotional response, they establish behavioral tendencies (Haidt, 2003). The latter defines moral emotions as: "those emotions that are linked to the interests or welfare either of society as a whole or at least of persons other than the judge or agent” (p. 2). In other words: emotions that go beyond the interest of the self. Their
altruistic character makes moral emotions specifically interesting with regard to value loaded political contexts. Even though they have largely been neglected in framing effect research, quite a lot of psychological research on the moral emotion itself has been done (e.g. Eisenberg, 2000; Haidt, 2003; Lewis, 2008; Hoffman, 2008; Tangney et al., 2007). When applied to frames, certain moral emotions seem to fit this context better than others. In this study, I choose seven moral emotions that will be reflected in the in- and out-group frames. This way, in- and out-group moral emotion frames will be created. In table 1 the emotions and their effects are listed.
-- TABLE 1 ABOUT HERE -- Hypotheses
In sum, framing effect studies review the effects of certain news frames on thoughts, and opinions (Druckman, 2001). However, frames do not only influence people’s opinions (Gross, 2008), but can also lead to engaging into (actual) political behavior (Groenendyk, 2011; Borah, 2013). The role of emotions in political contexts is
relatively unaddressed (Gross, 2008), but growing in importance (Kühne, & Schemer, 2013). Recently, studies have started to switch their focus on how news frames cause effects (Schuck, & Feinhold, in press). For example, by looking at underlying
principles like cognitive mediators (Slothuus, 2008), or the role of emotions as mediators (Schuck, & Feinhold, in press). Their effect on actual behavior has
(empirically) not been addressed yet. I follow Haidt (2003), who states that a specific kind of emotions, namely moral emotions, are especially interesting because of their altruistic character (which leads to action tendencies). I expect that moral emotion frames that emphasize a distinction between groups will encourage group
political behavior favoring one of the groups. Having this taken into consideration, this study is the first to address the role of moral emotions as mediators of framing effects on political behavior. The above-mentioned text assumes that emotions indirectly affect this relationship. In order to find mediation effects, I also expect a direct effect of exposure to a moral emotion frame on political behavior.
H1: Participants exposed to an in-group moral emotion frame about gay violence will have a higher intention to engage in political behavior than participants exposed to an out-group moral emotion frame about gay violence.
Both pride and compassion are thought to result in pro-social motives. Pride is strong when it comes to group membership (Giles, & Giles, 2013). It occurs when somebody feels responsible for a socially valued outcome, or for being a socially valued person (Mascolo, & Fisher, 1995). It allows for action. Compassion, as an overarching principle for empathy and sympathy (Haidt, 2003), is especially strong when one’s kin or the one’s that are part of the group are harmed. Shame, in the Western world (Haidt, 2003) a painful, public emotion (Tangney et al., 2007), is elicited when someone notices that something is wrong with his core self, mostly because he could not live up to a social norm or did something wrong. Disgust, initially thought of as just a bodily reaction (Rozin, Haidt, & McCauley, 2008; Rozin, Lowery, Imada, & Haidt, 1999), is elicited when a situation is considered immoral. Therefore, when his in-group has been (immorally) treated, I expect a Dutch citizen to be more concerned for his in-group members, than for his out-group members. A moral obligation of the in-group supporting the out-group is also recognized in the literature, but merely takes place when (earlier) wrongdoings of the in-group harmed the out-group (Rees,
Alpress, & Brown, 2013). Because of their specific elicitors, these emotions are expected to be experienced more in the in-group than the out-group.
H2a: Participants exposed to an in-group frame about gay violence will have a stronger experience of pride, compassion, shame, and disgust than participants exposed to an out-group frame.
Sadness, according to Eisenberg (2000) a non-moral emotion, is (mainly) taken into account because of the results in the pilot study. Especially towards the out-group sadness was felt. It is related to feelings of sympathy when another person is in distress. In other words, empathic sadness is felt. It thus requires perspective taking and is other-oriented (Eisenberg, 2000). Sadness is elicited when situationally induced, but does often not lead to behavior. Therefore, I hypothesize that it will be elicited more in the out-group.
H2b: Participants exposed to an out-group frame about gay violence will have a stronger experience of sadness than participants exposed to an in-group frame.
Anger is a very influential “approach” emotion. If a person experiences anger, then it will often lead to retributive action tendencies through approach behavior (Nabi, 2003). Although it is often considered to result in immoral, primary reactions to defend one’s own well being, anger as well as moral outrage are often expressed when people feel the urge to stand up for “what’s right” when a moral rule is violated, demanding justice for themselves or others. Especially anger is considered one of the most powerful political emotions in the spectrum (Tangney et al., 2007). Therefore, I
expect that both emotions will lead to similar levels of experience in both framing conditions.
H2c: Participants exposed to an in-group or out-group frame about gay violence will have similar experience of moral outrage and anger.
Despite the fact that I expect that the frames will elicit all emotions, I hypothesize that sadness, shame, and disgust are no mediators on behavior because their effects are based on avoidance, hiding and withdrawal, (Eisenberg, 2000; Haidt, 2003).
H3a: The moral emotions, pride, compassion, moral outrage as well as anger mediate the effect of exposure to a moral emotion frame on political behavior. However, shame, disgust and sadness do not.
According to Slothuus (2008), both belief content, and importance serve as mediators. However, since they were tested on opinion, I hypothesize that emotions are stronger mediators because they better relate to action tendencies needed for political behavior.
H3b: Emotions as mediators better explain framing effects on political behavior than cognitive mediators.
Method
In this study, I conducted an online experiment to test the framing effects of in- and out-group frames on political behavior with emotions as mediators manipulated in the frames. The context of this study’s experiment was chosen because the Netherlands increasingly deals with gay violence (Segerink, 2013). An important reason why the experimental method is addressed is because it allows the experimenter to manipulate
the moral aspect of emotions in the conditions and test their specific effects. By doing so, it is possible to contribute to a new insight in the emotion literature. Namely, connecting the morality of emotions to behavior that is elicited because of people’s values when judging political events. The experimental method is the best method in this case, since it studies causal relationships (Bernard, 2000). Although the research situation might be controlled for (or artificial), at the end it is possible to at least explain the causal relationship you controlled for. In other words, the results have epistemological power. Moreover, the design of the experiment can diminish
artificiality (e.g. the layout of the stimulus material). A disadvantage is that the effects of an experiment are difficult to generalize to the population, since the composition of the sample is often not a good reflection of the general population.
Design
A between-subjects online experiment was carried out in which participants were randomly assigned to three experimental groups. A moral emotion in-group news frame, a moral emotion out-group news frame and a control group were designed. In the form of a constructed news article on a Dutch news website, the article showed a text in which the increase and its implications of gay violence in the Netherlands or Russia were described. In both conditions, the seven earlier explained emotions were manipulated in the article. The control group received a short news flash article about a general report on gay violence without the manipulated emotions. Thus, “frame” is the independent variable. The initial choice for these specific moral emotions (moral outrage, pride, shame, disgust, compassion) was based on the emotions that were detected in real coverage about this topic, the results of the pilot study (anger, sadness) and the literature. The effect of these emotions as mediators was tested on
the dependent variable political behavior. To avoid bias or opinions about existing newspapers/websites, the layout of a Dutch news website was chosen without showing the websites’ name.
Sample
A total of 123 participants between 19 and 69 years old (50,9 % female, 96,7 % Dutch nationality), with a mean age of 36,37 (SD = 13.54) took part in this experiment. Participants were gathered via e-mail, Facebook (both personally and via groups), via the experimenters’ personal network, and via the Internet on forums. The data were collected between December 7th 2014 and December 28th 2014. An amount of 86 responses could not be taken into account, because these surveys were not completed (initial amount of started surveys, N = 209). The sample was merely left oriented (66,7 %) and highly educated (82,1 % HBO or WO). In total, 87 % of the sample was heterosexual, 6.5 % gay or lesbian, 5.7 % bisexual, and .8 % did not want to answer the question. Eventually, 43 people were assigned to the in-group frame, 39 people to the out-group frame and 41 people to the control group.
Procedure
The participants were exposed to one news article containing the in- or out-group moral emotion frame or the control group. Prior to the start of the survey, the respondents gave their consent to participate in the study. At the end of the survey they were debriefed about the real purpose of this experiment.
Before exposure to the articles, participants were asked to rate certain statements which measured their need for affect, and several statements regarding their attitude towards homosexuality on 7 points Likert-scales. After having read the
articles, the participants received a post-test questionnaire in which manipulation checks, two emotion measures (both implicit and explicit) and behavioral measures were included. Furthermore, control variables like age, gender, educational level, and participants’ own sexual orientation etc. were included.
Pilot Study
Prior to the real experiment, a pilot study was conducted on November 30th 2014. A randomly assigned sample (N = 15) read an initial version of the two framing texts or a control group article. The text described the implications of the Russian anti-gay law for Russia and the Netherlands. They filled out a questionnaire in which emotions were explicitly measured, a manipulation check was carried out, a first behavioral measure conducted and the credibility of the pilot stimulus was tested. In the real experiment, the text of the frames was changed into a description of the emergence and implications of gay violence in general. This was (partly) done in order to make the news report belief worthy for the Dutch situation (see Appendix C).
Stimulus Material
The articles reported about a newly published report by human rights’ organization Amnesty International, in which an increase of gay violence is described (see
Appendix A). The causes and implications of this development are reflected through a spokesperson of Amnesty International, a gay rights organization, a victim and an organization against racism on the Internet. In the in-group article the text focused on the situation in the Netherlands, whether the out-group article focused on the situation in Russia. The control group received a short news flash in which one could only read about the report being published. The texts in the main study were designed following
real news coverage about the topic (Lichfield, 2013; www.bbc.com, 2013).1 The words of the speakers in the stimulus articles contained emotional language (i.e. emotions). In order to keep the conditions (almost) identical, the names and titles that addressed the in- or out-group were adjusted (e.g. “a new report published by the Dutch/Russian branch of …”). Per condition, minor changes were made to the text to keep its course logical (see Appendix A for sample stimulus).
Manipulation Check
In order to check if the participants could recognize the in-, out-group distinction between the articles, a manipulation check was carried out using a one-way analysis of variance (ANOVA). It showed that people in the in-group (M = 6.33, SD = .97) could indicate their article described the Dutch situation F(2, 120) = 33.14, p = .000, and the out-group (M = 6.15, SD = 1.18) the Russian situation F(2, 120) = 69,56, p = .000. Therefore the manipulation has been successful.
Randomization Check
Randomization checks were executed in order to test if the participants were
randomly assigned to the conditions. A chi-square for gender χ2(2, N = 123) = 2.37, p = .31 and education χ2(10, N = 123) = 10.01, p = .44 revealed that the groups were randomly assigned. Also an ANOVA for age F(2,117) = .24, p = .79 showed that the differences in the condition can be attributed to the manipulations made. Therefore it can be concluded that randomization has been successful.
Measures
In this study, I measured emotional response using both implicit and explicit measures. The explicit measure asks people to identify the emotions they feel regarding a certain stimuli and report them accordingly (Quirin, Kazén, & Kuhl, 2009). It is a direct and traditional way of measuring emotions. Seven emotions (the mediator variables) were measured. The choice for these emotions was based on the emotions found in real news coverage and on moral emotions literature (Eisenberg, 2000; Haidt, 2003; Lewis, 2008; Hoffman, 2008; Tangney et al., 2007). Participants were asked to rate to which extent they felt one or several emotions after reading the articles on 7 point Likert scales from 1 (not at all) to 7 (very much).2
However, according to Evans, and Hackenbracht (2014) explicit measurement of emotions make people more aware of their feelings, it gives them the opportunity to cognitively process and verbalize their feelings, which can alter the original experience. Quirin et al. (2009) state that people easily mislabel their emotions. In other words: explicit measuring gives a filtered representation of what people really feel. Therefore, the Implicit Positive and Negative Affect Test (hereafter IPANAT) proposed by Quirin et al. (2009) is used. This test measures people’s emotions implicitly by asking them to rate which and to what extent emotions (as answer categories), according to them, match six defined neutral, artificial words (e.g. VIKES). The way participants rate the words is a reflection of their unconscious experience regarding exposure to a prior stimulus. People are unaware that their rating of the emotions is an implicit measure of their feelings after frame exposure. In this study, the emotions that will be used are the seven earlier explained moral emotions. The two emotion words are TALEP and SAFME, and answers are rated on 7 point Likert scales. Following Quirin et al. (2009) the scores on the emotions of both SAFME and TALEP were averaged to form one single implicit score per emotion.3
Cognitive mediators
The cognitive mediators belief content and belief importance, as theorized by
Slothuus (2008), were tested in order to prove which mediators were more important in predicting political behavior. Two scales with each four items, in which the most important statements from the articles were represented, were measured on 7 point Likert scales. The belief content scale (M = 4.90, SD = .94), with answers varying from 1 (do not agree at all) to 7 (completely agree) scale had items like “the economic crisis has roughened society, and increased anti-gay sentiment (this development should be stopped)”, “the media play an important role in negative coverage about homosexuals (and they should be careful in doing this)”, “negative verbal expressions about homosexuality on the Internet are unacceptable (and should be supervised)” and “politics prefers political and economical interests above their standpoints on homosexuality (which they should not)”. The same statements were issued to measure belief importance (M = 5.47, SD = 1.02) with, complemented with text placed in brackets. However, belief importance had answer categories varying from 1 (not important at all) to 7 (very important).
Political behavior
I measured political behavior (M = 3.41, SD = .91) using ten different behavioral items as suggested by Martin (2012) and Dalton (2008). In their studies, they described non-electoral forms of political behavior, tapping political talk (on-, off line), participating in direct action and boycott of products. I complemented these behaviors with boycott of people and talking to friends (see table 2 for all behaviors described). For the purposes of this study, the items were adjusted to the topic of gay violence. A factor analysis with varimax rotation was performed to test the variability
between the items. The items loaded on four factors. The analysis showed that the first factor explained 33,8 % of the variance, the second 14,1 %, the third 11,72 % and the fourth factor 10.70 %. Together they explained 70.29 % of the variance. However, the results showed that the behaviors did not follow the pattern as was suggested by theory (see Apendix C). That is to say, the distinction of the several behaviors was not traced back in the factor analysis. Therefore, I compromised them into one scale of political behavior. This scale was reasonably reliable (α = .75), and could therefore be taken into account for analysis.
-- TABLE 2 ABOUT HERE --
Strategy of analysis
A reliability analysis and factor analysis were performed for the behaviors of the general political behavior scale. The two different words of the IPANAT measure were composed to one implicit measure scale and reliability was checked. The direct effect of exposure to a moral emotion frame on political behavior was checked by means of a one-way analysis of variance (ANOVA), just as the direct effects of the moral emotion frame on the measures for explicit and implicit emotions. Mediation analyses by means of PROCESS were performed in order to test the mediating role of emotions and the cognitive mediators on behavior.
In research, many researchers have relied on the Baron and Kenny method (1986) to test mediation models (Hayes, 2009). Their so-called causal steps approach to mediation assumes that mediation occurs when the effect of certain paths in a mediation model are statistically significant. That is to say, the effect of the independent variable X on the mediating variable M (path a), and the effect of the
mediating variable M on the dependent variable Y (path b) should be statistically significant while the indirect effect of the independent variable X on the dependent variable Y should diminish compared to the direct effect between X and Y (c’ closer to zero) (Hayes, 2009; Lecheler et al., 2013). However, this traditional method has received a great amount of criticism because it would actually not test the intervening effect since it is inferred logically by the outcome of certain expectations and not based on the product of their constituent paths (Hayes, 2009). Therefore, Hayes (2009) developed the PROCESS: macro method. By focusing on bootstrapping, a “non-parametric sampling procedure and an estimation strategy that improves power of a model as it accurately gauges the empirical sampling distribution of the test statistic” (Lecheler, et al., 2013, p. 200), the most powerful effects for mediation can be found. Ninety-five percent confidence intervals can be established when testing for indirect effects; zero should not be included in the interval for a significant indirect effect.
Results
The hypotheses in this study will be tested in two steps. In the main analyses part, the main effect of the frames on political behavior (H1), and the direct effects of the frames on emotions (H2a–c) will be reviewed. In the mediation analyses part I test, if emotions mediate the relationship between exposure to the frames and political behavior (H3a), and if cognitive meditators or emotions are better mediators in explaining framing effects (H3b).
Main Effects on Behavior
In the first hypothesis (H1) I expected that participants exposed to an in-group frame would have a higher intention to engage into political behavior than participants
exposed to an out-group frame. An ANOVA (F(2,120) = 3.94, p = .022) revealed that people showed political behavior after exposure to the moral emotion frames. An LSD post hoc test revealed that participants were more likely to exhibit any form of political behavior in general when they were assigned to the out-group condition (M = 3.73, SD = .89, p = .007), compared to the in-group condition (M = 3.19, SD = .91) and the control group condition (M = 3.35, SD = .85, p = .061). This suggests that the effects of frames on political behavior were actually driven by out-group, and not by the in-group.
In sum, the results showed that the moral emotion had a mobilizing function. However, not the participants of the in-group had a higher intention to engage into political behavior, but the participants of the out-group. Thus, H1 was not supported.
Main Effects on Emotions
Hypotheses 2a,b and c measured the direct effects of the frames on emotions. Hypothesis 2a stated that people exposed to an in-group moral emotion frame,
compared to an out-group moral emotion frame would experience the emotions pride, compassion, shame and disgust more. An ANOVA of the explicit emotion measure revealed that the moral emotion frames did not elicit any of the seven emotions (all p values > .05). In table 3, the means, standard deviations, F tests and p values of all the emotion measures are displayed. The participants in the in-group did not show more compassion than those of the out-group and the control group, as was the case for pride. No significant levels for shame were found either: The in-group did not cause more shame then the out-group and control group. This was also the case for disgust. Because the frames did not elicit compassion, pride, shame, and disgust no direct effects were found for the explicit measure of emotions. The composite scores of the
implicit measure revealed similar non-effects. An ANOVA revealed that the in-group did not cause more compassion (p = .39), pride (p = .61) or disgust (p = .55) than the out-group or control group. However, a marginally significant effect of exposure to a frame on eliciting shame was found (p = .098).4 An LSD post hoc test showed that the participants in the out-group were more likely to experience feelings of shame than those in the in-group (p = .042) and the control group (p = .095). Therefore, H2a is not confirmed.
-- TABLE 3 ABOUT HERE --
Hypothesis 2b stated that participants in the out-group would feel more sadness after exposure to an out-group frame, compared to the participants exposed to an in-group frame. An ANOVA of the explicit measure showed that the frames did not elicit sadness (p = .80). Also concerning the implicit measure, an ANOVA revealed that no direct effect from exposure to a frame on eliciting sadness was found either (p = .21). The out-group did not cause more sadness than the in-group or control group (see table 3). Therefore, H2b is not confirmed.
Hypothesis 2c states that both the participants of the in- and out-group would have similar experiences of anger and moral outrage. However, the frames did not elicit anger and moral outrage (both p values > .05; see table 3). The in-group did not significantly cause more anger than the out-group or control group. The same was the case for moral outrage. Regarding the implicit measure, an ANOVA showed that the frames did not elicit feelings of moral outrage (p = .85). Participants in the in-group did not cause more feelings of moral outrage than those in the out-group or control group. However, a marginally significant effect of exposure to a frame on inducing anger was found (p = .078).5 Participants in the out-group held stronger feelings of
anger than those in the in-group (p = .046) and the control group (p = .052). Participants in the in-group did not feel more anger than in the control group. As a result, H2c is not confirmed.
Mediation Analysis – Emotion Model
-- FIGURE 1 ABOUT HERE --
In this part the indirect effects of exposure to the frames on political behavior through emotions is tested (H3a). Furthermore, a comparison between cognitive mediators and emotions, and their indirect effects on political behavior is made (H3b). Because the frames only evoked the emotions anger and shame, I chose to include only these two emotions in the mediation analysis (see figure 1). In this part, I will review whether or not these emotions mediated political behavior. A dummy variable for the conditions was made, in which the in-group was labeled as 1 and the out-group as 0. A
mediation analysis aided by PROCESS (Hayes, 2009) of the marginally significant, implicitly measured emotions shame and anger, revealed that the moral emotion frames predicted both shame (β = -.60, SE = .30, p = .048), and anger (β = -.69, SE = .33, p = .040). Neither shame (β = .11, SE = .08, p = .20), nor anger (β = -.08, SE = .07, p = .27) affected political behavior. Thus, also no mediation effects of shame (β = .06, SE = .05) (95 % bca CI: .22; .01), and anger (β = .06, SE = .05) (95 % bca CI: -.01; .22) were found. In other words: there was no indirect effect of exposure to a frame on political behavior through shame and anger. The total direct effect of exposure to the frames on behavior was significant (β = -.54, SE = .20, p = .009), as was the remaining total effect (β = -.53, SE = .21, p = .012).
In sum, the results showed that despite the fact that the frames evoked both anger and shame, they did not mediate this relationship. Given the fact that the frames did not elicit one of the other moral emotions, they cannot be mediators either.
Therefore, H3a is not confirmed.
Mediation Analysis – Emotion-Cognitive model
-- FIGURE 2 ABOUT HERE --
In this part, cognitive mediators and emotions as mediators are compared to test which ones explain an indirect effect on the relationship between frame exospore and political behavior better. Initially, two scales of four items were included for belief content and importance. However, the frames evoked only two items on those scales: “the economic crisis has roughened society, increased anti-gay sentiment and this development should be stopped” (belief importance) (M = 5.34, SD = 1.41, p =.004), and “the media play an important role in negative coverage about gays” (belief content) (M = 4.82, SD = 1.73, p = .001). Therefore only these items are taken into account.
A PROCESS analysis (see figure 2) revealed that the frames did not predict belief content (β = -.17, SE = .34, p = .620) and belief importance (β = .04, SE = .27, p = .895). Furthermore, both belief content (β = .04, SE = .07, p = .55) and belief
importance (β = .06, SE = .08, p = .479) did not affect political behavior. Just like the emotions, no indirect effects of exposure to a frame on political behavior via belief content (β = -.01, SE = .03) (95 % bca CI: -.13; .02), and belief importance (β = .00, SE = .03) (95 % bca CI: -.04; .08) were found. As already reported above, the total
direct effect of X on Y was significant, as was the remaining direct effect when
controlled for both the emotions and cognitive mediators (β = -.53, SE = .21, p = .014) In sum, since neither the emotions anger and shame, nor the cognitive
mediators belief content and belief importance had an indirect effect on the relationship exposure to frames on political behavior, neither emotions, nor the cognitive mediators were actual mediators. Therefore, H3b is not confirmed.
Discussion
This study follows earlier research on the role of emotions in news framing effect studies. In the present study, seven moral emotions were manipulated in an in-group and out-group moral emotion frame reporting about an increase of gay violence in the Netherlands and Russia. These frames were supposed to lead to engaging into
political behavior. The main results showed that a direct effect of exposure to a frame on political behavior indeed was found. In general, the out-group, compared to the in-group was more likely to exhibit political behavior, which was contrary to the
hypotheses. Furthermore, the moral emotion frames elicited only two of the seven emotions, namely anger and shame. None of the moral emotions served as mediators in this relationship. In addition, two proved cognitive mediators (belief
content/importance) as reviewed by Slothuus (2008) were tested, but did not serve as mediators either.
The main (direct) effects of this study are rather surprising. They suggest that the situation in the out-group was more important than the moral emotions. This is contrary to statements of Markus (2000) and Thompson (1973) who state that in political judgment and decision-making, people’s values, standards, and sense of morality are important. The Dutch participants in the out-group generally had a higher
intention to engage into behavior to support Russian homosexuals instead of their own nationals in the in-group. Thus, the in- and out-group distinction was important, only not the way as originally thought.
An explanation for this is three-fold. On the one hand, you could argue that fighting for the rights of Dutch homosexuals is simply not important enough
(anymore) to Dutch citizens, while for Russian homosexuals a lot can still be reached. This finding is surprising when you look at the intergroup theory by Tajfel (1982). He states that in-group members favor the in-group instead of the out-group. The national identity characteristics (Brewer, M. B., 1999; Ben-Ner et al., 2009) and psychological citizenship as theorized by Sindic (2011) support that countries can be in- and out-groups. Looking at the results, it might also be possible that gay people are not considered a part of the in-group, especially when the in-group is national. This suggests that the nation as a whole would be too diverse to be taken into account as one in-group.
On the other hand, the specific situation in the Netherlands could be an explanation. Since this country was the first in the world to legalize gay marriage (Buijs, Hekma, & Duyvendak, 2011) and quite some other liberal laws, the Netherlands tends to be proud of their “Dutch tradition of tolerance” (Buijs et al., 2011, p. 633). In other words, many Dutch people think that gay emancipation in the Netherlands is so successful; they might be blind for the fact that discrimination and violence still happens. This can be explained by means of “culturalization of
citizenship” (Mepschen, Duyvendak, & Tonkes, 2010, p. 964). In the Western world, citizenship is increasingly based on cultural values and morality. Free sexuality is an important factor in this process. Since the Dutch hold common, moral values about supporting gay rights, they would, as these researchers state, see these moral values as
basis for their existence. Not supporting gay rights would make you a “moral outcast” in the Netherlands. Thus, social desire for support is high. If violence does take place people would possibly close their eyes for it or simply think that this is not possible in a developed country like the Netherlands (which they are part of). It could be the main explanation for the results in this study, since it is rather anti-ethical to not see the problems facing your own countrymen when confronted with a news article reporting about just that. The findings actually confirm the results of Rees et al. (2013), namely that the in-group is willing to help the poorly treated out-group. However, this
tendency would then not derive from a way to right earlier wrongdoings of the in-group to the out-in-group, as these researchers state, but presumably derive from a feeling of “we” the Dutch have to help poorly treated out-group members. This still confirms the in-, out-group distinction, but the homosexual Russian out-group would become favorable to the Dutch in-group compared to other Russians in the name of transmission of Dutch values. This argument could be a starting point of new research regarding the matter.
A third explanation regards the fact that (Dutch) media reported a lot about gay violence (Buijs et al., 2011). Recently, because of its anti-gay incidents, Russia has been in the news a lot (http://www.huffingtonpost.com, 2014).6 Therefore, people might have already been influenced because of this amount of exposure. In that sense people were already “pre-treated” prior to the experiment (Druckman, & Leeper, 2012). This would also explain why the participants in the control group on the political behavior scale, indicated that they were more likely to engage into political behavior than the in-group. Even after reading only a short news flash about gay violence in general, unconsciously people in the control group might have connected gay violence automatically to gay violence in Russia. Along this line of reasoning,
another explanation could be that the in-group participants did not consider the situation described in the Dutch article belief worthy for the Netherlands.
However, this does not explain why the frames almost did not elicit the moral emotions and why their action tendencies (Haidt, 2003) were not translated in this specific situation. Only two emotions were elicited and they were no mediators. Thus, the connection between morality and consequential political behavior cannot be proven. Perhaps the moral aspect of emotions affects people more in decision-making and opinions, rather than their engagement into actual behavior. For this, their
appraisals might be too weak. Namely, in their studies about the mediating role of emotions on opinions, e.g. Kühne and Schemer (2013), Lecheler et al. (2013) did find significant results. An interpretation for my results derives from Brader and Markus (2013). They state that the action tendencies of emotions do not always lead to direct actions of individual citizens. Instead, they lead to supporting a policy or candidate that pursues these ideals. In that sense, this study lacked including behavioral items that measured this argument. It could be the explanation why no mediation effects were found.
It is also possible that the wrong emotions were chosen in this specific context. For example, in this study shame was elicited but it did not lead to behavior. In
addition, H2b stated that sadness would not lead to behavior because of its avoiding character. However, sadness was not even elicited, which was unexpected and contrary to the literature. Also, the results from the pilot study indicated otherwise. Pride, the moral emotion that would lead to pro-social behavior was not elicited either. According to Etxebarria, Ortiz, Apodaca, Pascual, and Conejero (2014) an explanation could be the fact that pride in something happens through external judgment. A feeling of pride for a group or action takes place when communicating
with others. The fact that people took part in the experiment individually could be of importance. Furthermore, pride might not be felt because the situation for gays reported in the news article was received stronger than the appraisals of pride leading towards a proud feeling for the in-group (as was hypothesized). Especially in case of the explicit measure this could be a valuable explanation. Although compassion has a “directly pro-social action tendency” (Haidt, 2003, p. 12), it is also thought to be strongest for one’s own kin (Batson & Shaw, 1991). In this case, this could be an explanation why compassion was not elicited. The fact that disgust was not elicited might be because the moral aspects of this emotion are overrated. As Rozin et al. (2008) already stated, disgust was foremost considered a bodily reaction and its working in moral situations is not always proven. According to Nabi (2002) disgust is more a dimension of anger. Bodily reactions do refer to so-called core disgust, but the moral aspect is closer to anger. This brings me to a final discussion about anger. Anger was elicited strongest by the moral emotion frames, although it did not lead to behavior. Anger has been characterized as the most potent political emotion (Brader, & Markus, 2013). Its risk-taking behavioral output is known; it only depends on the individual and situation how anger is elicited. The fact that anger is so often regarded in people and the other emotions to a lesser degree (or not) might depend on how the distinction between emotions is made. Not only by theorists but especially in
laypeople. For example, it could be that moral outrage was classified under the denominator anger, as well as disgust (Nabi, 2002), and sadness (Solloway et al. 2013), especially when explicitly measured. A last consideration why none of the emotions served as mediators is simply because the situation was not important enough to the participants to engage into political behavior. Which makes me wonder if results would have been different when heterosexuals were harmed. A suggestion
could be to study moral emotions with regard to heteronormativity: do they play a role in situations where a comparison is made between violence against homo- or
heterosexuals?
There are certain limitations to this study. For example, the IPANAT measure was bound to the answer categories (i.e. the several emotions) provided by the
experimenter. Other emotions that were not listed might have played a role as well. Although emotions were measured both implicitly and explicitly, other measures like word stem completion tasks (Roediger III, Weldon, Stadler, & Riegler, 1992) might have led to different results. In addition, measures for vocal expression (Juslin & Scherer, 2005) or facial expression (Kring & Sloan, 2007) could be important in implicitly measuring emotions after stimulus exposure, whether or not in combination with different stimulus material like pictures, moving images or recited texts.
Furthermore, it would be wise to look at certain factors, like religious status, attitude towards gay people, that moderate the relationship between the frames and behavior via moral emotions. Also in this study these moderators were taken into account. However, due to low reliability of the used scales they could not be analyzed. In this study, no hypotheses were outlined in which an effect of the moral emotions on political behavior for participants with specific characteristics was expected.
Therefore I chose not to include control variables in the analysis. The most interesting control variable to include would have been a participant’s own sexual orientation. However, the part of the sample that had a different sexual orientation than
heterosexual was so small (13 %), I assumed that nothing substantially could be contributed on the effects on emotional response and political behavior. Another point of criticism is that this study did not distinguish which behaviors were morally
individual I chose not to address this. However, a closer examination might have led to more valuable results. Furthermore, the results of the factor analysis did not follow the pattern as suggested in the theories of Dalton (2008) and Martin (2012).
Therefore, only one combined scale of political behavior was used. In future research, a better distinction of behaviors should be used. At the same time, behaviors that fit the context of this study better, and their connection with actual moral behavior should be regarded.
A closer look at which emotions would fit the situation better should be given. However, also then, other, unexpected results would not be taken into account. The fact that these measures relied on self-report could be a subjective limitation, since social desirability to be “gay friendly” is high in the Netherlands (Buijs et al., 2011). Apart from the manipulation and stimulus material limitations, another limitation is the size and composition of the sample population, which was only a reflection of a small part of the general population in the Netherlands. Therefore it remains difficult to generalize the results to the whole population.
Despite of its limitations, the (non) results of this study indicate that a lot of research is still needed when studying the relationship between emotions and behavior. This study was the first to address the connection of the morality of emotions on political behavior. A more carefully designed study, with a higher reliability than would have been possible for this thesis, could gain more insight.
References
Adolphs, R. (2010). Emotion. Current Biology, 20(13), 549–552.
Van den Akker, H., van der Ploeg, R., & Scheepers, P. (2013). Disapproval of
homosexuality: Comparative research on individual and national determinants of disapproval of homosexuality in 20 European countries. International Journal of Public Opinion research, 25(1), 64–86. doi:10.1093/ijpor/edr058 Anderson, B. (1991). Imagined communities: Reflections on the origin and spread of
Nationalism. London: Verso. 2nd ed.
Baron, J. (2003). Value analysis of political behavior – self interested : moralistic :: altruistic : moral. University of Pennsylvania law review, 151(3), 1135–1167. Baron, R., & Kenny, D. 1986. The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. Retrieved from http://ovidsp.tx.ovid.com/sp-3.14.0b/ovidweb.cgi?
Batson, C. D., & Shaw, L. L. (1991). Encouraging words concerning the evidence for altruism. Psychological Inquiry: An International Journal for the
Advancement of Psychological Theory, 2(2), 159–168. doi:10.1207/s15327965pli0202_17
Ben-Ner, A., McCall, B. P., Stephane, M., & Wang, H. (2009). Identity and in-group/out-group differentiation in work and giving behaviors: Experimental evidence. Journal of Economic Behavior and Organization, 72, 153–170. doi:10.1016/j.jebo.2009.05.007
Bernard, H. R. (2000). Research design: experiments and experimental thinking. In C. D. Laughton (Eds.), Social research methods: qualitative and quantitative approaches (pp. 103-141). Thousand Oakes, Califiornia: Sage Publications,
Inc.
Bizer, G. Y., Larsen, J. T., & Petty, R. E. (2010). Exploring the valence-framing effect: Negative framing enhances attitude strength. Political Psychology, 20(20), 1–
22. doi: 10.1111/j.1467-9221.2010.00795.x
Blumler, J. G., & Kavanaugh, D. (1999). The third age of political communication: Influences and features. Political Communication, 16(3), 209–230. doi: 10.1080/105846099198596
Borah, P. (2013). Interaction of incivility and news frames in the political
blogosphere: Consequences and psychological mechanisms. In J. Bishop, & A. M. G. Solo (Eds.), Handbook of Research on Political Activism in the Information Age (pp. 407–425). Hershey: IGI Global
Brader, T., & Markus, G. E. (2013). Emotion and political psychology. In L. Huddy, D. O Sears, & J. S. Levey, (Eds.), The Oxford Handbooks of Political
Psychology, (pp. 1–31). Oxford: University Press.
Brewer, M. B. (1999). The psychology of prejudice: Ingroup love or outgroup hate? Journal of Social Issues, 55(3), 429–444. doi: 10.1111/0022-4537.00126 Brewer, P. R. (2002). Framing, value words, and citizens’ explanations of their issue
opinions. Political Communication 19(3), 303–316. doi:10.1080/01957470290055510
Brewer, P. R. (2003). Values, political knowledge, and public opinion about gay rights: A framing based account. Public Opinion Quarterly, 67, 173–201. Brubaker R. (1996). Nationalism reframed: Nationhood and the national question in
the new Europe. New York: Cambridge University Press
Buijs, L. Hekma, G., & Duyvendak, J. W. (2011). ‘As long as they keep away from me’: The paradox of anti-gay violence in a gay friendly country. Sexualities,
14(6), 632–652. doi:10.1177/1363460711422304
Clore, G. L. & Ortony, A. (2008). Appraisal theories: How cognition shapes affect into emotion. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 628–642). New York: Guilford Press. Čorkalo, D., & Kamenov, Ž. (2003). National identity and social distance: Does
in-group loyalty lead to outin-group hostility? Review of Psychology, 10(2), 85–94. Dalton, R. J. (2008). Citizen norms and the expansion of political participation.
Political studies, 56, 76–98. doi:10.1111/j.1467-9248.2007.00718.x
Dalton, R. J. (2012). Citizen attitudes and political behavior. Comparative Political Studies, 33(6/7), 912–940. doi:10.1177/001041400003300609
Damasio, A. R. (2001). Emotion and the human brain. Annals of the New York Academy of Sciences, 935(1), 101–106.
doi:10.1111/j.1749-6632.2001.tb03475.x
Domke, D., Shah, D. V., & Wackman, D. B. (2000). Rights and morals, issues and candidate integrity: Insights into the role of the news media. Political Psychology, 21(4), 641–665. DOI: 10.1111/0162-895X.00211
Domke, D., Shah, D. V., & Wackman, D. B. (1998). “Moral referendums, forums, news media, and the process of candidate choice. Political Communication, 15(3), 301–321. doi:10.1080/105846098198911
Druckman, J. N. (2001). The implications of framing effects for citizen competence. Political Behavior, 23(3), 225–256 doi:10.1023/A:1015006907312
Druckman, J. N., & Leeper, T. J. (2012). Learning more from political
communication experiments: Pretreatment and its effects. American Journal of Political Science, 56(4), 875-896. doi:1/j.1540-5907.2012.00582.x
Psychology, 51, 665–695. doi:10.1146/annurev.psych.51.1.665
Entman, R. M. (1993). Framing: Toward clarification of a fractured paradigm. Journal of Communication, 43(4), 51–58.
doi:10.1111/j.1460-2466.1993.tb01304.x
Etxebarria, I., Ortiz, M. J., Apodaca, P., Pascual, A., & Conejero, S. (2014). Antecedents of moral pride: The harder the action, the greater the pride? Spanish Journal of Psychology, 17(52), 1–11. doi:10.1017/sjp.2014.56 EU LGBT survey: Poll on homophobia sparks concern. (2013, may 17). Retrieved
from http://www.bbc.com/news/world-europe-22563843
Evans, S., & Hackenbracht, J. (2014). Hooked on a feeling: Implicit measurement of emotion improves utility of concept testing. ALERT! Marketing Research Association, 54(4), 6–9.
Giles, H., & Giles, J. (2013). Ingroups and outgroups, In A. Kurylo (Eds.),
Inter/Cultural Communication, representation and construction of culture (pp. 141–162). Marymount Manhattan College: SAGE Publications.
Green, B. C. (2005). Homosexual signification: A moral construct in social contexts. Journal of Homosexuality 49(2), 119–134. doi:10.1300/J082v49n02_07 Groenendyk, E. (2011). Current emotional research in political science: how emotions
help democracies overcome its collective action problem. Emotion Review, 3(4), 455–464. doi:10.1177/1754073911410746
Gross, K. (2008). Framing persuasive appeals: Episodic, and thematic framing, emotional response, and policy opinion. Political Psychology, 29(2), 169– 192.
Gross, K., & Brewer, P. R. (2007). Sore losers: News frames, policy debates, and emotions. Harvard International Journal of Press/Politics, 12, 122–133. doi:
10.1177/1081180X06297231
Gross, K., & d’Ambrosio, L. (2004). Framing emotional response. Political Psychology, 25(1), 1–29. doi:10.1111/j.1467-9221.2004.00354.x
Haidt, J. (2003). The moral emotions. In R. J. Davidson, K. R. Scherer, & Goldsmith, H. H. (Eds.), Handbook of affective sciences (pp. 852–870). Oxford: Oxford University Press.
Harcupp, T., & O’Neill, D. (2010). What is news? Galtung and Ruge revisited. Journalism Studies, 2(2), 261–280. doi:10.1080/14616700120042114 Hayes, A. 2009. Beyond Baron and Kenny: Statistical mediation analysis in the new
millennium. Communication Monographs, 76, 408–420. doi:10.1080/03637750903310360.
Hoffman, M. L. (2008). Empathy and prosocial behavior. In M. Lewis, J.M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 440–455). New York: Guilford Press.
Iyengar, S., & Simon, A. (1993). News coverage on the gulf crisis and public opinion: A study of agenda-setting, priming and framing. Communication Research, 20(3), 365–383.
Judd, C. M., & Downing, J. W. (1995). Stereotypic accuracy in judgments of the political positions of groups and individuals. In M. Lodge, & K. M. McGraw (Eds.), Political judgment: Structure and process. Michigan: University of Michigan Press.
Juslin, P. N. and Scherer, K. R. (2005). Vocal expression of affect. In J. A. Harrigan, R. Rosenthal, & K. Scherer, (Eds.). The New Handbook of Methods in
Nonverbal Behavior Research (pp. 65–135). Oxford: University Press Kühne, R. (2014). Political news, emotions, and opinion formation: Toward a model
of emotional framing effects (Working Paper. No. 68). Amsterdam: the Netherlands, National Centre of Competence in Research (NCCR). Kühne, R. & Schemer, C. (2013). The emotional effects of news frames on
information processing and opinion formation. Communication Research, 20(10), 1–21. doi:10.1177/0093650213514599
Kring, A. M., & Sloan, D. M. (2007). The facial expression coding system (FACES): Development, validation, and utility. Psychological Assessment, 19(2), 210– 224. doi:10.1037/1040-3590.19.2.210
Lee, Y-T., & Ottati, V. (2010). Attitudes toward U.S. immigration policy: The roles of in-group-out-group bias, economic concern, and obedience to law. The Journal of Social Psychology, 142(5), 617–634.
doi:10.1080/00224540209603922
Lecheler, S. & de Vreese, C. H. (2012). News framing and public opinion: A mediation analysis of framing effects on political attitudes. Journalism & Mass Communication Quarterly, 89(2), 185–204.
doi:10.1177/1077699011430064
Lecheler, S., Schuck, A. R. T., & de Vreese, C. H. (2013). Dealing with feelings: Positive and negative discrete emotions as mediators of framing effects. Communications, 38(2), 189-209. doi:10.1515/commun-2013–0011 Lewis, M. (2008). Self-Conscious emotions: Embarrassment, pride, shame, and
guilt. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of Emotions, 3rd Ed. (pp. 742–756). New York: Guilford Press.
Lichfield, J. (2013, april 12). Rise in homophobic attacks mark passage of French gay of marriage law. The Independent. Retrieved from
