Sensationalism and Humanitarian
Content in Online Dutch Newspaper
News and User Comments
A study of the text of refugee (crisis) related news
articles and the comments posted to them on Facebook
Author: Esmeralda V. Bon
Student number: 10340386 Thesis type: Master’s thesis
School: Graduate School of Communication
Program: Research Master’s programme Communication Science
Supervisor: Dr. K. de Swert
Acknowledgements
It is a pleasure to thank those who have made this thesis possible. First of all, I owe my
deepest gratitude to Dr. De Swert, whose insightful comments, advice and critical remarks
have guided my writing and whose patience knows no bounds. Second, I am immeasurably
thankful to my parents. Their everlasting guidance and immense support have made me the
academic I am today and they reduced the laborious task that thesis writing can be to a
manageable, gratifying project. Third, I would like to thank my fellow students and friends
who have been ever so supportive in these final months and who have also been an
irreplaceable sounding board of knowledge about, amongst others, communication theory,
communication science, research methods and (im)migration: Marieke, Saba, Joanna,
Abstract
Europe is challenged by an ongoing migrant crisis. How the media presents this crisis
affects audience perception and agenda setting. This original study focuses on the use of
human interest vs. humanitarian frames by a quality and a popular Dutch newspaper, the
emotional neutrality of their news articles and the degree to which these feature issue
elaboration. The first research question enquires: To what extent do the
post-embedded-articles on refugees of Dutch newspapers on Facebook contain humanitarian and
sensationalist elements? Simultaneously, to provide insight into the potential effects of these
news portrayals, it is asked whether there are commonalities between the content of the
articles and the content of the top 100 comments they received. The studied newspapers are
NRC Handelsblad (quality) and De Telegraaf (popular). After a literature review and the
introduction of adapted scales, a content analysis of a sample of the Facebook post-embedded
news articles of these newspapers (N = 109), published between January 1st, 2015 and April
22nd, 2016, and the top 100 comments to them (N = 5792) is reported. It is found that the
news articles of the NRC feature more human interest framing, humanitarian framing, issue
elaboration and emotionality than those of the Telegraaf, but that these differences are due to
the types of news articles and the length of the articles written. The content of the comments
is uncorrelated to that of the news articles and they therefore seem to be largely independent.
Striking study-specific observations are made about the presence of elements of sensation in
quality refugee related news and the relation between the news articles’ and comment content.
Keywords
■ Humanitarian ■ Sensationalism ■ Framing ■ Refugee ■ Newspaper ■ Human Interest ■ Popular journalism ■ Quality journalism ■ Dutch ■ Content Analysis ■ Emotionality
Word count: 9.105
Introduction
At the time of writing, Europe is facing an ongoing refugee crisis and refugee (crisis)
related issues are high up on the public agenda (Kosho, 2016). The Dutch agenda is
influenced by the news media (Boomgaarden & Vliegenthart, 2007), and the descriptions of
these refugee (related) issues are subject to the frames they use, which affect subsequent
discourse, creating a certain understanding among readers and watchers about the events
covered and the information conveyed (Entman, 1993; Gitlin, 1980; Neuman et al., 1992). They may even create a ‘change in judgment’ in viewers and readers (Iyengar, 1987, p. 816).
Thusly the frames employed in refugee related news affect public opinion (e.g. Van
Gorp, 2005; Kosho, 2016). Seemingly unfortunately, a general move towards tabloidization
has been observed, which is characterized by sensationalism (Harcup & O’Neill, 2001;
Sparks, 2000). A frame that is commonly linked to sensationalism is the human interest frame
(e.g. Reinemann et al., 2011), which as defined by Semetko & Valkenburg (2000) brings a
face to an issue or presents it primarily emotionally. This frame seeks to entertain and to grab
attention by extensively narrating how people are affected by an issue (Bennett, 1995).
Another frame which focuses on people and how they are impacted is the humanitarian frame.
However, in contrast this frame focuses less on the trivial and more on aspects surrounding
development assistance (e.g. Druckman, 2001) and it can be used for eliciting compassion
(Moeller, 1999; Tester, 2001). These frames may also foster social action, especially on social
networking sites (Madianou, 2013). Thusly, the human interest and humanitarian frames are
expected to at least differ with regards to their inherent sensationalism and purposes.
Based on the role and use of the aforementioned frames and the observed increase of
sensation in the news, this project is concerned with how refugee (crisis) related news is
presented. The first research question posed is: To what extent do the post-embedded-articles
provide an answer to this question two Dutch newspapers are selected, one classified as
popular and the other as quality: the Telegraaf and the NRC Handelsblad respectively. The
selected post-embedded-articles (N = 109) and the top 100 comments to them (N = 5792) are
analyzed. The level of sensationalism present is measured by the introduction and use of
adapted frames and scales for measuring the presence of human interest framing,
humanitarian framing, issue elaboration and emotional neutrality. For the latter emotional
neutrality – emotionality scale, an algorithm based sentiment analysis is employed.
Thereafter, to tap into the potential effects of the use of these frames in this context, a
second research question is posed, namely: Are there commonalities between the content of
the articles and the content of the top 100 comments posted in reaction? The aforementioned frames and scales are to be slightly adapted1 before their application to the measurement of
the content of the comments. By providing an answer to these two questions this study
reflects on the role of sensationalism in relation to the valuation of different types of
journalism and introduces a differentiation between the human interest and humanitarian
frame that may bring theoretical specification in this study and those that follow.
Theory
New and bad journalism: tabloidization and sensationalism
The link between entertainment and news has been contested for almost a century,
with criticism being risen against yellow journalists in the 1920s and, for instance,
Habermasian critique in 1962 (Winch, 1997). Yet especially after the turn of the century a
new journalism has been observed, one that features traces of tabloidization (Harcup & O’Neill, 2001) and is fueled by capitalism (McManus, 1995; Örnebring & Jönsson, 2004).
1
It is, for instance, deemed problematic to code for the presence and type of human interest visuals in each of the top 100 comments to the news articles in the sample. The comment visuals do not seem to have the same purpose as the article visuals, as many of them represent ‘memes’ or other pictures with captions meant to convey a sarcastic message. This could, however, be an interesting topic for, and addition to, a future study (design).
With tabloidization Sparks (2000), in a much-cited definition, specifically refers to ‘serious
media’ starting to adopt tabloid news values, and these tabloidization elements are
“sensationalism, triviality, malice and plain, simple credulity (p. 1).He further characterizes the tabloid as follows:
“[the tabloid] devotes relatively little attention to politics, economics, and society and relatively much to diversions like sports, scandal and much popular entertainment; it
devotes relatively much attention to the personal and private lives of people, both
celebrities and ordinary people, and relatively little to political processes, economic
developments and social changes” (p. 10).
One may read this definition as a valenced judgment, describing the tabloid as providing
distractions from topics which are implicitly presented as more important to public life.
Tabloidization is as such seen as a negative phenomenon, a “crisis for democracy”, by elites and as constituting ‘bad journalism’ versus the ‘good journalism’ of the quality or broadsheet newspaper (Sparks, 2000, p. 10).
Yet some scholars underline that the distinction between quality and tabloid or popular
newspapers is, in fact, not as clear-cut or defined, which suggests a more nuanced view might
be in order. Winch (1997), for example, states that these ‘boundaries’ are “contextually contingent, local, and episodic, with the potential to become stable and widespread” (p. 10). This means that a tabloid may not by definition be a tabloid, and vice versa. Furthermore, they
also recognize the positive role that the tabloid may play. For example, Sparks (2000) also
describes the tabloid as being “a vital source from which the people can generate their own
meanings and thus empower themselves at the expense of the ‘power bloc’” (p. 25).
At the moment, regardless of the Western nation that the news media belong to, these
new news values seem to have been adopted throughout the Western world and they appear to
(e.g. Vasterman & Ruigrok, 2013). Thusly, due to the rise of tabloidization, it is expected that
serious matters are increasingly discussed and presented in trivial ways. Based on this
assumption, an issue elaboration scale is introduced in the analysis of the sensationalism
practiced by the NRC and the Telegraaf, the NRC being conventionally identified as a quality
newspaper and the Telegraaf as a popular newspaper (see, f.i., De Haan 2011 and Schafraad,
Wester & Scheepers, 2013) 2.
While these newspaper labels are used for ensuring that two very different newspapers
make up the sample, they should not be seen as definitional. As Meijer (2012) argues, quality
journalism may not actually be quality journalism, for there is a discrepancy between how
quality journalism is regarded and experienced and as such, the common-held view of ‘quality journalism’ may be flawed. Instead, the quality of quality journalism should be measured in terms of whether it meets what the audience values, which is “participation in the journalistic process” (Meijer, 2012, p. 766). One way in which the readers can participate is by posting comments on sites and web pages dedicated or run by the newspaper.
Human interest framing versus humanitarian framing
The news values of the tabloid – which as argued by scholars are on the rise – are
linked to the human interest frame. Semetko and Valkenburg (2000), for instance, specifically
note that especially those news outlets that feature sensationalist traits use human interest
frames and Reinemann et al. (2011) is just one study in which the use of human interest frames is specifically linked to the rise of this new journalism that creates ‘soft news’. The human interest frame introduces a ‘human face’ into the article and it brings “an emotional
2 The author of this thesis conducted a small-scale content analysis on the reporting of the MH17 crash in these
two newspapers, a year ago, focusing on the extent to which sensationalism was present. This study confirmed that the Telegraaf does contain more human interest framing in relation to some of the indicators of the conflict frame. This study not only provided inspiration for opposing the Telegraaf and NRC as newspapers in relation to sensationalism but it also showed that not all of the indicators of even the most adopted and accepted frames always apply to news items. Some cases may require different frames or different indicators.
angle to the presentation of an event, issue or problem” (Semetko & Valkenburg, 2000, p. 95-96). Semetko and Valkenburg (2000) also note that this frame is used for grabbing attention
and they provide five indicators for measuring the extent to which news features human
interest framing. The indicators that make up this frame can be found in the appendix.
Contrary to the human interest frame, the humanitarian frame – which also addresses
how people are affected – has not been as widely adopted and applied. The humanitarian
frame as presented by Druckman (2001) concerns text with a focus on “how increased assistance would ensure help for people who need it”3 (p. 1047). In turn, Mintz and Redd
(2003) in their paper about the use of framing in international relations describe an utterance
by George W. Bush that contains references to the suffering of children in Afghanistan, a call
for help and a reference to a humanitarian agency an instance in which a humanitarian frame
is used (see appendix for this example).
These instances show that a humanitarian frame should at least contain a reference to
development assistance and people in need. Since, to the best of the author’s current
knowledge, there is not an existing thoroughly tried and tested indicator-based humanitarian
frame and scale to date, a new humanitarian (effort) frame with a number of indicators is
formulated and tested in this study. This frame includes the aforementioned necessary
references and is further based on the well-accepted and widely adopted principles of
humanitarian efforts by the International Federation of Red Cross/Red Crescent and other
NGO’s, as developed through practice. These principles, given by Spielberg and Adams (2011) on pages 4 and 5, are listed in table 1.
3
Druckman (2001)’s example of a text using the humanitarian frame is: “In the next few weeks, the US Congress will likely accept one of two proposals that will alter the amount of federal assistance to the poor. One proposal is to increase assistance while the other is to decrease assistance. An increase in assistance to the poor would ensure help for many people who need it. A decrease in assistance would prevent people from receiving basic support. Do you think Congress should increase or decrease assistance to the poor?” (p. 1062)
Table 1: The humanitarian principles as used and drafted by the Red Cross and other NGO’s 1 Agencies and individuals should operate with respect to culture and custom
2 Humanitarian response should use local resources and capacities as much as possible 3 The beneficiaries should be encouraged to participate in the humanitarian activity 4 Emergency response should strive to reduce future vulnerabilities
5 Agencies should be accountable to both donors and beneficiaries
6 Agencies should portray victims as dignified human beings, not hopeless objects of external assistance
The relevance of a humanitarian frame to European immigration issues is evident in Helbling
(2014), who also finds that references to morality are at the core of this frame, for which
indicators will be added in this study. Since both frames concern how parties are affected by
an event, but one in a superficial and another more in-depth, it is suspected that these news
frames may be confused by the news audience, but also by scholars.
Ardèvol-abreu (2015), for example, addressing the coverage of humanitarian crises in
Africa, explains that the presence of humanitarian news in the media is due to the fact that
journalism seeks to entertain. This raises the question to what extent humanitarian news and
the humanitarian frame, in contrast to the human interest frame, overlap and it asks for
theoretical specificity4. Nickels (2007), in turn, introduces a ‘human dignity’ frame in his study of ‘asylum discourse’, one that concerns images of suffering but also references to human rights and he thereby obscures the distinction between the human interest and
humanitarian representations as defined in this study.
A way to distinguish between these two frames is by focusing on their different uses.
Snow and Benford (1988) have introduced three distinct purposes or ‘tasks’ that frames have,
namely the ‘diagnostic’, to address the problem and who is responsible, the ‘prognostic’, to focus on a solution and the party that should provide it, and the ‘motivational’, to urge an
audience to act. Whereas the humanitarian frame, as defined here, could be regarded to have
4 Because of this reason indicators which could create an overlap between the two frames were either adapted or
removed, giving rise to a ‘core’ or stricter humanitarian frame. See the method section for an example of a removed indicator.
one or multiple of these uses, the human interest frame is not likely to go beyond the
diagnostic. In other words, it will only refer to the problem and maybe attribute responsibility.
As such, taking the traditional view of quality journalism as reporting issues as being
of public concern, the humanitarian frame does indeed belong to the serious news media. This
is also the case in Meijer’s (2001) alternative view of quality journalism, who sees ‘quality’ journalism as “instilling a sense of social cohesion”, as “expressing a sense of citizenship”, as ‘contributing’ to the ‘empowerment of citizens by calling them to action, and this “by
establishing a basis for dialogue and collaboration rather than creation [of] a sense of
confusion or rivalry” (p. 201). It also fits her later view of journalistic participation by the audience as the basis of quality journalism (Meijer, 2012).
Newspapers and audience interaction on social networking sites
Concerning audience participation, newspapers, whether local, regional or national,
are increasingly turning towards social networks for getting into contact with users and
readers, whether actual or potential (Ju, Jeong & Chyi, 2014). In theory this may lead to
newspapers adopting a more personal stance and increase newspaper-user interaction, not
least through likes, shares and comments (e.g. Guirado Zamora, 2012).
However, studies suggest that, also in the Dutch case, this move does not necessarily
go hand in hand with increased engagement. For example, regarding the use of Facebook by
Dutch news media, Hille and Bakker (2013) have found that Facebook users ‘distribute’ news through liking and sharing rather than interacting with it. Others have concluded that the
provision of likes seems to be the most common form of engagement between the refugee
(related) post-embedded articles of the newspapers on their pages and Facebook users, at least
in Sweden and Spain (Guirado Zamora, 2012; Larsson, 20161). Unfortunately, although
(2016)1, reactions of a passive audience that consumes the news rather than interacts with it.
This suggestion is supported by Ju, Jeong and Chyi (2014), who have found that there is only a moderate correlation between the amount of ‘likes’ to post-embedded news articles and actual site visits. They note that for popular and quality newspapers only a few readers access
the news web site via Facebook compared to visitors from “other content sites” (p. 11).
Yet, when Facebook users do respond to the articles with written comments, the
comments may be of a better quality than those posted on the news sites, in terms of not
requiring moderation. For example, Hille and Bakker (2014), who like Larsson (20162) study
the link between human interest news articles shared on Facebook by the newspapers
themselves and the comments to them, conclude that there are less but higher quality
comments on Facebook to the news articles of Dutch newspapers than on the newspapers’
own news sites, likely due to the anonymity that the latter provides.
This impact of anonymity on the content of comments to news articles is, amongst
others, also pointed out by Reader (2012), Rowe (2013) and Santana (2014). Whereas Reader
(2012) argues that the quality of news site comments is higher if anonymity is not allowed
and Rowe (2013) states that Facebook comments reflect greater civility due to the lack of
anonymity, Santana (2014), who studied anonymous comments on comment boards to
immigration news, believes that online comments generally feature incivility.
This incivility can be removed by the newspapers that still perform a gatekeeping
function as they have the power to moderate the comments that are being posted (Hermida & Thurman, 2008). One may expect that those comments that fit the newspaper’s stance are most likely to be kept. After all, in their study on the framing of immigration and integration
Roggebrand and Vliegenthart (2007) have found that the media are selective and support
those voices that fit ‘hegemonic’ frames. Thusly newspaper-user interaction and the content
Framing effects, emotional neutrality and emotion
If the authors of these comments, the commenters, have read and respond to a news
article with a particular frame, the content of their comments is thought to be affected, for
frames can cause changes in understanding and perception (Entman, 1993; Gitlin, 1980;
Iyengar, 1987; Neuman et al., 1992), and comments reflect opinions and attitudes. Emotions
play a role in sensational, humanitarian content and in light of framing effects.
First, the use of emotions in relation to sensationalism again concerns the quality of
journalism, as the deviation of a newspaper article from emotional neutrality has been
regarded as characterizing less-than-quality news (Bird, 2009), which is about distracting
readers with the trivial, with scandals and other, tabloid news values mentioned earlier. It is
also related to the human interest frame, which concerns news reports that grab attention not
least by having an ‘emotional angle’ (Semetko & Valkenburg, 2000).
Second, concerning humanitarian news and emotions, Tester (2001) focuses on TV
news when referring to compassion fatigue as it occurs in response to the overexposure to
negative, humanitarian occurrences and how it affects viewers, stating that “compassion
fatigue is becoming so used to the spectacle of dreadful events, misery or suffering that we
stop noticing them. We are bored […] and we are left unmoved” (p. 13). Those static images
in the newspaper articles to be analyzed in this study could have a similar effect.
Third, regarding framing effects, recent studies acknowledge the role of emotions in
the reception of news frames. For example, Lecheler, Schuck & De Vreese (2013) indicate
that news frame effects are mediated by a number of positive and negative emotions, such as
enthusiasm and anger and state that “exposure to news frames that bear emotional relevance will invoke emotional reactions” (p. 202). This is also observed in Iguarta, Moral-Toranzo and Fernández (2011) and Lecheler, Bos and Vliegenthart (2015). These studies conclude that
Based on these findings one can imagine the level of emotional neutrality vs.
emotionality of a news article to play a role in how – and to what extent – the refugee related
news message is received. Thusly, an emotional neutrality – emotionality scale, based on the
application of an algorithm-based-sentiment analysis, is adopted in this study for measuring
the extent to which the news article and comment content is emotional. It is expected that
traces of the emotionality of the news article will be present in the user comments to the
Facebook post-embedded newspaper articles, although there will surely be discrepancies
between the article and comment content. After all, Druckman (2001), for example, notes that
the effects of news frames are mitigated by source credibility. Therefore the trust of a
commenter in the news source could affect the extent to which the emotionality overlaps.
Hypotheses
Based on the insights gathered from this literature review it is expected that the quality
newspaper focuses less on the trivial and on emotions and more on the substance of issues
with emotional neutrality, in contrast to the popular newspaper. Following these expectations,
a number of hypotheses have been formulated. The first set of hypotheses concerns the
sensationalist content of the news articles of the newspapers and is based on the popular vs.
quality newspaper distinction.
H1: A quality newspaper (NRC) features more issue elaboration than a popular newspaper (Telegraaf)
H2: A quality newspaper (NRC) employs less human interest framing than a popular newspaper (Telegraaf)
H3: A quality newspaper (NRC) employs more humanitarian framing than a popular newspaper (Telegraaf)
H4: A quality newspaper (NRC) features more emotionally neutral articles than a popular newspaper (Telegraaf)
Next, to investigate in a rather exploratory fashion whether one can speak of interaction
between the journalist and the commentators five more hypotheses will be tested. First, it will
be asked whether the between-newspaper differences of the scale and frame values are
reflected in the comments:
H5: The differences in the framing and scale values found between the two newspapers will be reflected in the comments to their articles
After gaining this preliminary insight into how the articles and comments may relate, the
correlations between the content of the articles and comments will be computed, to establish
to what extent there are commonalities between the content of the news articles and the
comments to them:
H6: The presence of markers of issue elaboration in the comments is affected by the presence of markers of issue elaboration in the news articles.
H7: The presence of human interest framing in the comments is affected by the presence of human interest framing in the news articles.
H8: The presence of humanitarian framing in the comments is affected by the presence of humanitarian framing in the news articles.
H9: The level of emotionality of the comment text is affected by the level of emotionality of the newspaper articles.
Method Pilot Study
In February and March, 2016, a pilot study was carried out to investigate whether
humanitarian and human interest frames are actually featured in the Facebook post-embedded
news articles by Dutch news providers. Only some news outlets posted about refugee related
issues frequently and while the NOS, the Dutch national public service broadcaster, was
initially taken as one of the news providers for this study, the digital news articles of its
online newspaper was subsequently chosen as the medium of interest. The online news article
selection criteria originally derived from the analysis of 20 purposively sampled Facebook
post-embedded NOS news articles were adapted to the online newspaper medium. See the
appendix for an overview of these selection criteria.
Out of the Dutch newspapers with a Facebook platform, the NRC is the quality
newspaper that posts refugee related news articles the most often and the Telegraaf is the
popular newspaper that meets this criterion. These newspapers are also conventionally used
for studying the differences between quality and popular newspapers (e.g. Schafraad, Wester
& Scheepers, 2013), which means that there are established expectations regarding their news
content. After selecting these newspapers the news article selection criteria were updated to
take into account medium-specific, content and formatting differences5.
Sample, data collection and codebook construction
The Facebook post-embedded online news articles of the Telegraaf and NRC, posted
between January 1st 2015 and April 22nd, 2016 by these newspapers on their own Facebook
pages, were sampled. The titles of these Facebook posts were accessed via the Facebook
graph API. If the titles contained one of the keywords6 the news article content was examined.
The remaining articles were numbered chronologically and from this sampling frame of 544
articles, 150 articles per newspaper were chosen by number generator assisted random
selection. Out of these, 57 were coded for the NRC and 52 for the Telegraaf7 (N = 109) as
5 Three NRC articles and three Telegraaf articles were examined to create these updated scale indicators. These
six articles were later subjected to the content analysis and included in the final sample.
6 See appendix for these keywords. In cases where the title was ambiguous – which happened to be quite
frequently, the social media posts were accessed and the articles were skimmed for the presence of keywords in the body of the article. For further information about this selection process, see the appendix.
7 Two extra NRC articles were added to ensure that the amount of articles with comments in the sample was
exactly the same. Out of the 57 articles of the NRC 52 have comments and for the Telegraaf 52 articles with comments were coded respectively. Coding these news articles including the top 100 comments to them took about 40 minutes on average and thusly the coder spent almost the entire month of May on coding the sample.
well as the top 100 comments to each of these articles (N = 5792)8. See table 2 in the
appendix for more sample information.Meanwhile, the codebook used for the content
analysis of this study was subjected to several revisions, based on the pilot study and feedback
from the thesis supervisor, fellow students and news readers. It was also continuously updated
with examples and exceptions to ensure that the coding remained consistent.
Measures and Operationalization
The codebook has two main sections: one for analyzing the online newspaper article
content and the other for analyzing the top 100 comments to the article. At the beginning the
coder is asked to confirm that the article meets the selection criteria, to ensure its relevance to
the study. Thereafter, the coder is presented with a number of basic categorization variables,
which consist of the date, the newspaper (Telegraaf or NRC), the article ID, the type of news
article and the identification of the issue (theme)9. The newspaper variable makes it possible
to compare the results between the two newspapers, without necessarily splitting the dataset10.
Following these basic categorization variables the theory and pilot study based scale variables
for measuring issue elaboration, human interest framing and humanitarian framing are
introduced. Their application descriptions can be found in the codebook in the appendix.
The first set of these variables consists of the following issue elaboration indicators:
quantitative problem definition, qualitative problem definition, causes of the issue/crisis, past
8 Some Telegraaf articles received more than 2000 Facebook comments. It was impossible to code all of these.
Instead, only the top 100 comments would be coded, because 1) these reflect the most liked comments and 2) tis is the order in which comments are retrieved through the Facebook graph API. Since this study includes a sentiment analysis of the comments, the text of which needs to be accessed through the API, it is important that the first 100 comments are (more or less) the same on Facebook and in the comment list that the API tool returns. Readable comments are those comments with textual content. These can be read by the algorithm.
9 The difference in the amount of likes that the NRC and Telegraaf Facebook pages have received and the extent
of interaction on their pages is probable to affect the amount of likes, the amount of comments and the content of the comments that the posts that embed the articles receive. On June 20th, the NRC had 121.456 fans (users who like the page) and the Telegraaf had 308.871 fans. The amount of comments the articles in the sample have received are given in the appendix and this information – about the likes and comments – has not been taken into account for this study, but these factors could be interesting and worthwhile to investigate in future research.
10 While in reality this distinction between newspapers is also made to distinguish between the popular and
quality newspaper, one should not assume that the newspapers necessarily reflect these expected labels. As a result, this distinguishing variable simply denotes the newspaper from which the articles are being coded.
development of the issue/crisis, mention of consequences, local or international impact of the
issue, mention of (alleviating) plan/treatment, mention of solution for the issue/crisis,
attribution/mention of responsibility for an issue and attribution/mention of responsibility to
act. If present, these elements are added up to each other (present = +1, dominant = +2) and
the final number expresses how elaborate the refugee (crisis) related crisis is described in the
news article. This scale is identical for the newspaper articles and the comments.
The second set of the variables measure human interest framing and it is preceded by a
control question, which inquires whether there is the mention of an individual, party or parties
involved in a refugee (crisis) related crisis, development or issue. If so, to measure the
prominence of the human interest frame, these indicators were coded for their absence,
presence and dominance: a human example signifying the (refugee (crisis) related) issue, a
mention of how a party or parties are affected by the issue, expressions of feelings of dislike
and objection with regards to the issue, expressions of feelings of compassion, sympathy and
liking with regards to the issue, and human interest (audio-)visuals. These visuals are images
of: a disaster site, refugees as the affected party/individual, visible emotions, injured or dead
parties/individuals, and a protest. For every such visual present, one point is added. If present,
these elements are summed (present = +1, dominant = +2) and the final number expresses the
extent to which human interest framing is present in the article or comments. Some indicators
are based on Semetko and Valkenburg (2000)’s human interest frame (see appendix), and the
article and comment scales are not identical, for comment visuals were not taken into account.
Considering the nature of the case study, which taps into a refugee crisis, and the
human interest and humanitarian frame comparison, two extra human interest indicators were
added: ‘emergency rhetoric’ and ‘scandalization’. These were likewise coded for their
presentation of an issue as one of urgency, scandalization provides the explicit portrayal of a
situation or issue as being scandalous or outrageous.
The third set of variables consists of humanitarian frame indicators and concern
humanitarian references, humanitarian calls or humanitarian rhetoric. These references and
calls are: reference to/call for solidarity or empathy, reference to/call for respecting culture
and custom, reference to/call for the use of local resources and capacities for solving a refugee
(crisis) related issue and a reference to a humanitarian agency/agent. These indicators, largely
based on the Red Cross Code, were tested for relevance in the pilot study, coded for presence
(= +1)11 and summed to indicate the level of humanitarian framing present. While some
studies include victimization as a humanitarian frame indicator (e.g. Watson, 2009), this
indicator is excluded because it relates to the human interest frame12. Instead, recognizing the
prominence of morality as an indicator of the humanitarian frame in Helbling (2014) and its
expected inherent link to calls for assistance, two humanitarian rhetoric indicators were
added: reference to morality and reference to immorality. Collectively the humanitarian frame
indicators are identically applied to the analysis of the newspaper articles and the comments.
Finally, the emotional neutrality - emotionality of the articles and comments is
measured using sentiment analysis, a computational method for analyzing big data that
concerns the use of dictionaries for establishing to extent to which text features positive and
negative emotions (Mostafa, 2013). More specifically, the SentiStrength algorithm as
provided for Dutch text is employed, which returns two values, one for the presence of
positive sentiment (1 to 5 scale), and the other for the presence of negative sentiment (-1 to -5
scale). The highest absolute value (5) indicates that there is a very strong positive or negative
sentiment present. In other words, it represents an extreme level of positive emotionality (5)
11
The pilot study results indicated that it was unlikely that the humanitarian frame would be as present as the human interest frame and therefore only the presence (and not the dominance) of these indicators was coded.
12 The human interest frame already includes an indicator for coding references to how people are affected,
which is close in interpretation to victimization. For the sake of theoretical differentiation this indicator was not included for the humanitarian frame.
or negative emotionality (-5). A value that lies close to 0 indicates emotional neutrality. The
emotional neutrality – emotionality scale is created by adding the absolute positive and
negative values and by subtracting 1, which gives rise to a 9-point scale. See the appendix for
a description of how the article and comment text was preprocessed for the analysis.
Intercoder Reliability
To check the reliability of the variables and indicators, a second coder was asked to
code 30 articles from the final sample. This second coder, a social science research MSc
alumna, with the immigration specialization, was given the opportunity to ask questions after
reading the codebook and before she started with the coding13. To assess inter-coder reliability Krippendorff’s alpha was used.This test, conducted in SPSS with a macro, only consisted of the 36 items used for coding the content of the newspaper articles. Those
variables that received an insufficient reliability are marked grey in table 2 and were removed
from the applicable scales. The reference to respecting culture and custom indicator could not
be measured for the newspaper articles because it did not occur in the sample, but was kept in
for determining the level of humanitarian framing in the comments.
Table 2: The Krippendorff alpha values of the items tested
Basic categorization items Α
Control: Embedded article by news outlet about refugees 1
Newspaper: Telegraaf or NRC 1
Type of embedded article 0.54114
Presence of issue statement 1
Sensationalist items (& scales) Α Non-sensationalist items (& scales) Α Human Interest Frame Issue Elaboration
Control: Involved parties 0.548 Quantitative problem definition 0.534 Human example 0.541 Qualitative problem definition 0.368
How affected 0.167 Causes of the issue/crisis 0.528
Expression of dislike present 0.590 Past development of the issue/crisis 0.567 Expression of like present 0.563 Mention of consequences 0.666
Disaster visual 0.35615 Impact 0.356
13
Coding the articles took this coder approx.11 hours in total, spread over four days. Each article took 20-30 minutes to code. The comments were not coded for the reliability test. As she did not code the comments to the newspaper articles, it took her less time to code these thirty articles than the primary coder of this study.
14
This variable is kept in because it is expected that this variable scores low due to the rarity of certain values.
Affected party visual 0.515 Mention of plan/treatment 0.647 Visible emotion visual 0.35616 Mention of solution 0.505 Injured or dead party/indiv. Visual 1 Attribution of responsibility to act 0.660 Protest visual 0.841 Attrib. of responsibility for issue 0.682 Scandalization 0.636 Humanitarian Frame
Emergency rhetoric 0.517 Reference to morality 0.614
Reference to immorality 0.527 Reference to solidarity or empathy 0.764 Ref. to respecting culture and custom 0.636 Ref. to use of local resources & capacities 0.605 Ref. to a humanitarian agency/agent 0.710 Call for solidarity or empathy 0.786 Call to respect culture and custom - Call to use local resources & capacities 0.655
Data Analysis
To answer the research questions and to test the hypotheses, statistical tests were
performed and, as mentioned, a sentiment analysis was conducted. The statistical tests, which
were preceded by normality tests, when applicable17, concerned a multitude of independent
samples t-tests with bootstrapping, simple and multiple (logistic) linear regressions and
correlational analyses including Pearson’s r, Spearman’s rho and Fischer’s exact test. For the sentiment analysis a program was written in python and run in a virtual machine.18
Results
The extent of the framing and emotional neutrality in the articles, by newspaper
The first set of hypotheses, formulated to answer the first research question, concerns
the degree of issue elaboration, human interest framing, humanitarian framing and emotional
neutrality in the newspaper articles. It is hypothesized that the NRC, as the quality newspaper,
employs more issue elaboration, less human interest framing, more humanitarian framing and
greater emotional neutrality than the Telegraaf, the popular newspaper. To test these
17 See appendix for kurtosis and skew values. The emotionality and humanitarian framing of the NRC comments
may be non-normally distributed. It is expected that these values are due to the consistently lower amount of NRC comments in general and in the sample (to be discussed), creating less opportunity for the presence of frame elements and emotionality and thereby causing scale values to center around 0 and exceptions to be more prominent. The skew and kurtosis underline that some NRC comment content results are especially tentative.
hypotheses, independent samples t-tests were computed. An overview of the results of these
independent samples t-tests can be found in table 3.
Table 3: The framing and sentiment (values) compared by newspaper
Newspaper 95% CI for Mean
Difference NRC Telegraaf M SE n M SE N T df Issue elaboration 4.82 0.29 57 2.90 0.25 52 -2.70 -1.13 -4.98** 106 Human interest fr 1.75 0.18 56 1.11 0.13 46 -1.08 -0.21 -2.92** 97 Humanitarian fr 1.54 0.19 57 1.00 0.15 52 -1.02 -0.07 -2.27* 102 Emot. neutrality – emotionality 4.95 0.16 57 3.62 0.19 52 -1.78 -0.85 -5.33** 106 * p < 0.05, ** p < 0.01
First, the differences between the scores of the two newspapers on the issue elaboration scale
were compared. This is an 8 point scale with the qualitative-problem-definition and impact
indicators removed, as the inter-coder reliability test indicated that these may be too
dependent on interpretation. On average, the news articles of the NRC feature greater issue
elaboration than the news articles of the Telegraaf (p < 0.01). Thus, there is a difference in the
extent to which an article features issue elaboration, based on the newspaper and the null
hypothesis that there is no difference between the groups can be rejected. Hypothesis 1 that
the quality newspaper features greater issue elaboration can be confirmed.
Second, concerning the human interest frame, the indicators that received low
Krippendorff Alpha values (see table 2) were removed, creating an 8 point scale. As can be
read from the table, on average the news articles of the NRC feature more human interest
framing, than the news articles of the Telegraaf (p < 0.01). It can thus be said that the
newspaper that the article belongs to does affect the extent of human interest framing. The
null hypothesis that there is no difference between the groups can be rejected. Hypothesis 2
that states that the NRC employs less human framing has to be rejected. In fact, the opposite
Third, regarding the humanitarian frame19, measured on a 9 point scale, the news
articles of the NRC feature more humanitarian framing than those of the Telegraaf (p < 0.05).
Thus, the newspaper that a news article belongs to affects the extent of humanitarian framing
used. The null hypothesis that there is no difference between the groups can be rejected and
hypothesis 3 that the quality newspaper employs more humanitarian framing is confirmed.
Fourth, the differences between the scores of the two newspapers on the emotional
neutrality - emotionality 9 point scale were calculated. The results of these tests indicate that
the newspapers significantly differ on this variable. On average, the news articles of the NRC
feature greater emotionality than the news articles of the Telegraaf and the null hypotheses
that there are no differences between the groups can therefore be rejected. However,
hypothesis 4 that the NRC is more neutral and therefore features more emotional neutrality is
rejected. In fact, the opposite is true: the articles of the Telegraaf are more neutral than those
of the NRC.
Predictors of the articles’ framing and emotional neutrality/emotionality
These independent samples t-tests were followed by the computation of a number of
multiple linear regressions to determine the main predictor of the scores on these scales.
While the interaction between the news article and newspaper variables could be tested due to
collinearity issues20, the following model comparisons argue in favor of the news article type
being the primary predictor. As it turns out, rather than the newspaper, it is often the type of
news article – whether it is a regular news article or a different kind of news article, such as a
background report or an interview – and the article length that affect the framing and scale
scores. The newspaper only plays a role as a predictor for the extent of issue elaboration. In
19 For the humanitarian framing scale, no indicators were removed on the basis of the Krippendorff alpha values. 20
Due to this low amount of non-regular news articles in the Telegraaf it was not possible to calculate the extent of the interaction between the news article and the newspaper by means of inserting the interaction effect in a multiple linear regression (the interaction was not shown due to collinearity.
the appendix a table can be found which contains an overview of the multiple regression
results for the performance of the newspaper, news article and the rescaled article length21 as
predictors of the values on the four scales: the human interest frame, the humanitarian frame,
the issue elaboration scale and the emotional neutrality – emotionality scale. Below the best
fitting simple and multiple predictor models derived from these multiple regressions have
been included, in table 4.
Table 4: Best fitting models with news article, newspaper and rescaled article length as the predictors of the frame and scale values
Scale/Frame B SE b b* Sig. Adj. R2
Issue elaboration
Constant 1.63 0.60
Newspaper 0.15 0.04 0.39 0.000*
Article length 0.97 0.43 0.22 0.028* 0.28
F (2,106) = 22.16, p < 0.001
Human interest frame
Constant 2.33 0.19 News article -1.24 0.23 -0.48 0.000*** 0.22 F( 1,100) = 29.70, p < 0.001 Humanitarian frame Constant 0.73 0.15 Article length 0.10 0.02 0.47 0.000*** 0.21 F( 1,107) = 30.00, p < 0.001
Emotional neutrality – emotionality scale
Constant 4.77 0.33
News article -1.28 0.31 -0.39 0.000***
Article length 0.09 0.02 0.35 0.000*** 0.43
F( 2,106) = 43.06, p < 0.001 * p < 0.05, *** p < 0.001
For the human interest frame values the news article is the primary predictor. The type of
news article – regular or non-regular – predicts 22% of the variance in human interest
framing. Next, concerning the humanitarian frame values, article length is the primary
predictor. Whereas the variance in the humanitarian frame values is for 21% explained by
article length, the issue elaboration values are significantly predicted by the newspaper and
21 The original article length variable where 800 words gave a value of 800 and 1400 words a value of 1400 has
been divided by 100 to make sure that the coefficient values of the article length-variable are on a similar scale as the others.
article length variables, which together predict 28% of the variance. Lastly, concerning the
emotional neutrality – emotionality of the news articles, the variance in these values is
predicted for 43% by the type of news article and the article length.
From these results it becomes clear that, with the exception of the issue elaboration
values, rather than the newspaper the type and length of the news article predict the values
received on the framing and emotionality scales featured in this study.
The between newspaper differences for scale and frame values of the comments
The second set of hypotheses, formulated to answer the second research question,
concerns the relationship between the content of the newspaper articles and the content of the
comments posted in response to them. First, hypothesis 5 states that the news article content
and comment content similarly differ between newspapers. In other words, it is expected that
the comments to the NRC articles will also feature more human interest framing,
humanitarian framing, issue elaboration and emotionality than the comments to the Telegraaf
articles. To test this hypothesis a number of independent samples t-tests were computed and
the results are found in table 5.22
Table 5: The framing and sentiment (values) of the comments by newspaper
Newspaper 95% CI for Mean
Difference NRC Telegraaf M SE n M SE N T df Issue elaboration 2.00 0.24 52 4.46 0.24 52 1.78 3.14 7.21*** 102 Human interest fr 1.52 0.15 52 3.56 0.14 52 1.64 2.44 10.11*** 102 Humanitarian fr 0.73 0.14 52 3.88 0.24 52 2.59 3.72 11.11*** 82 Emot. neutrality – emotionality 2.15 0.09 52 2.37 0.06 52 0.01 0.44 2.08* 86 * p < 0.05, *** p < 0.001
22 To answer this research question and the ensuing hypotheses the 5 newspaper articles of the NRC that do not
include any comments were removed from the sample as well as the ‘comment’ versions of the article content indicators that received low Krippendorff alpha values. Here it has to be pointed out that there is a slight difference between the indicators of the human interest scales: the human interest framing scale of the news article content contains visual indicators and the comment scale does not and the humanitarian scale for the comments has more indicators with specific foci. These differences between the comment and article content scales are based on practicality and the (non)-occurrence of certain aspects.
From table 6 it can be read that the comments to the Telegraaf feature more human interest
framing, more humanitarian framing, greater issue elaboration (p < 0.001) and greater
emotionality (p < 0.05). Each of these findings is the opposite of what would be expected if
there is a relationship between the content of the articles and the content of the comments to
them. These results therefore go against hypothesis 5: while there are significant
between-newspaper differences on these scales, the NRC article comments feature in fact less issue
elaboration, less human interest framing, less humanitarian framing and less emotionality.
Content correlations between the articles and the comments
For each newspaper separately a Pearson product-moment correlation coefficient was
computed to assess the relationship between the frame and emotionality values of the
newspaper articles and the frame and emotionality values of the comments, as tested in
hypotheses 6 to 9. Those variables which were previously removed for being too dependent
on interpretation, such as the impact, ‘how-affected’ and qualitative-problem-definition
variables/indicators, are not included in these correlational tests and results.
Furthermore, due to the relative infrequency of the dominant presence of a number of
remaining frame elements the indicators were recoded from ordinal into dichotomous, binary
indicators for denoting the presence or absence of a certain element. Because of the fact that
the phi coefficient is the same as the Pearson correlation for dichotomous, binary variables,
the Pearson correlation coefficients were computed for determining whether there is a
relationship between the content of the newspaper articles and the comments on the basis of
the presence and absence of these indicators. These results can be found in the appendix.
When an element does not occur for one of the newspapers either in the articles or comments,
then no statistics have been computed and there are no results in the table.
Following the results, there are three elements or indicators whose occurrence in the
respecting-culture-and-custom and reference-to-using-local-resources in the Telegraaf and the
Telegraaf comments and the mention of a solution in the NRC and the NRC comments.
Before adopting these correlations as a fact, however, Fischer’s exact test needs to be
employed, since this test is more rigid and takes into account the possibility of a correlation
when cells in a factorial design brought about by correlating two dichotomous or binary
variables contain less than 5 cases. These results can be found in table 6.
Table 6: The Fischer exact test results of the three aforementioned item pairs
Item pair Fischer’s exact tests’s p-value Reference to respecting culture and custom, Telegraaf 0.079
Reference to the use of local resources, Telegraaf 0.021*
Mention of a solution, NRC 1.00023
* p < 0.05
These results indicate that there is only a correlation between a reference to the use of local
resources in the news articles and in the comments to these news articles in the Telegraaf. The
Pearson correlation coefficient for this indicator pair is 0.35, indicating a medium effect.24
This one element or indicator was subsequently subjected to Pearson correlational
analyses to find out whether there are any independent variables that, in the case of the
Telegraaf, are significantly correlated with it. It is found that for the independent variable of
the reference to the use of local resources, two of the other scale variables significantly
correlate. These variables are the reference to a humanitarian agency in the news article (r =
0.29, p = 0.04) and the reference to respecting culture and custom in the news article (r =
0.31, p = 0.03). These variables were added as predictors to the simple binary logistic
regression model of the reference-to-use-of-local-resources indicator pair, of which the results
are reported in table 7.
23
This result is due to the fact that there is no NRC article in which such a reference to a solution is present.
Table 7: The binary logistic regression results of the reference to local resources pair 25
B SE b 95% CI for Odds Ratio
Included Lower Odds Upper
Constant -1.72
[-3.33, -0.89]
1.91
Reference to use of local resources in the articles
1.62* [0.30, 3.19]
2.09 1.36 5.04 18.68
Note. R2 = .11 (Cox & Snell) .16 (Nagelkerke). Model X2 = 6.22, p = 0.013, * p < 0.05
These results of the simply binary logistic regression indicate that the odds of there being a
reference to the use of local resources in the comments is 5.04 times greater when such a
reference is present in the articles than when such a reference is absent. Adding the
aforementioned significantly correlated indicators to the model does not lead to any model
improvement: the model chi-square improvements are non-significant when adding these
predictors and interactions (see table 826).
Table 8: The Multiple Binary Logistic Linear Regression Models tested Model 1: Reference to the use of local resources
Model 2: Ref.to the use of local resources + Ref. to a humanitarian agency
Model 3: Ref. to the use of local resources + Ref. to a humanitarian agency + interaction Model 2, X2 (1) = 0.09, p = 0.76
Model 3, X2 (1) = 1.02, p = 0.31
Model 1: Reference to the use of local resources
Model 2: Ref. to the use of local resources + Ref. to respecting culture and custom Model 3: Ref. to the use of local resources + Ref. to respecting culture and custom + interaction
Model 2, X2 (1) = 0.37, p = 0.55 Model 3, X2 (1) = 1.52, p = 0.22
Thus, whilst based on the non-significant correlation results, hypotheses 6, 7 and 9 have to be
rejected, there is partial support for hypothesis 8: there is a humanitarian indicator that does
25
The Hosmer and Lemeshow goodness-of-fit statistic could not be calculated due to the categorical predictor.
26 Concerning the respecting-culture-and-custom-variable, although every cell is populated, the latter variable
does not have more than 5 cases in each cell, which can be problematic when calculating a logistic regression. While for completeness’ sake this variable is included as a potential predictor in table 9 and results are given, it is not considered any further.
predict the presence of the identical humanitarian indicator in the comments, namely that of
the reference to the use of local resources. The inclusion of extra predictors does not improve
the model and it can therefore be said that on the basis of the indicators in this study there
does not seem to be a model that better predicts the presence/absence of a reference to the use
of local resources than the use of local resources variable itself.
Conclusion and Discussion
Following the research questions that i) asked about the differences between a quality
and a popular newspaper on a set of sensationalism and humanitarian framing scales and ii)
the correspondence in content between these news articles and the comments they receive, the
main findings of this study are as follows. The articles of the quality newspaper NRC feature
more issue elaboration, human interest framing, humanitarian framing and emotionality than
those of the popular newspaper Telegraaf. These differences are mostly predicted by news
article type and/or article length and thusly the distinction between the quality and the popular
newspaper seems largely irrelevant for when sensationalism is concerned. Moreover, the
content of the comments to the Facebook post-embedded articles do not seem related to the
articles in terms of the measured indicators and scales, with the exception of the
reference-to-the-use-of-local-resources indicator, for the Telegraaf, suggesting that the comments are
either largely unaffected by, or independent from, the Facebook post-embedded news articles.
Thus, as expected, the NRC articles feature more issue elaboration and humanitarian
framing. The level of issue elaboration depends on newspaper type and article length, and the
extent of humanitarian framing depends on the type of article and article length. The fact that
the NRC features more issue elaboration was expected, as Scholten and Ruigrok (2006) have
found that the NRC is the Dutch newspaper that focuses the most on ‘issue news’27 and the
27
Scholten and Ruigrok (2006) focus on the coverage of politics in the Dutch news and these types of news should be seen in relation to politics. Whereas – what they refer to as – ‘issue news’ concerns the attention for
Telegraaf more on conflict. The fact that the NRC uses more humanitarian framing should be
interpreted with caution, for these indicators were rarely featured in the articles analyzed. In a
previous study of regional newspapers it was noted that, at least in asylum seeker (center)
coverage, a frame surrounding moral values and social prescriptions was ‘barely registered’
(d’Haenens & De Lange, 2001, p. 859). Although this is an older study, together with the thesis’ findings this could suggest that the humanitarian frame is uncommon to Dutch news.
Some of these findings were unexpected. First, it was concluded that the
traditionally-viewed quality newspaper NRC employs more human interest framing. Second, it was found
that the NRC’s news articles feature more emotionality than those of the Telegraaf. Of course,
an emotional story is not necessarily a sensational story, but the presence of emotions can
indicate sensationalism and therefore be linked to the popular newspaper (St. John III &
Johnson, 2012). Thusly, the NRC articles include two sensational aspects, which may indicate
that they depict refugee news in a more sensationalistic way than the Telegraaf.
However, to differentiate, the articles of the NRC provide a more elaborate description
of refugee (crisis) related issues, which makes it more likely for the human interest frame
elements to occur as well as the articles to deviate from emotional neutrality. The NRC
articles are also longer, they have 852 words on average, whereas the Telegraaf has 209 words
on average, and they feature more background reports or, in other words, non-regular news
articles. The type of news article – regular or non-regular – is found to largely affect the
presence of humanitarian, human interest and issue elaboration elements and emotional
neutrality, more so than the newspaper. While the NRC scores higher than the Telegraaf on
sensationalism, the NRC publishes more background articles on refugee issues (N = 19;
the substance of policies, conflict news focuses on support and criticism between politicians, horse race news focuses on the horse race and evaluative (‘evaluatie’) news concerns evaluative utterances regarding politicians or political parties.
29.7%), than the Telegraaf (N = 2; 3.7%)28. Thus the news article type should be considered
as a factor affecting the presence of sensationalist elements in the news article.
Concerning the relation between article and comment content, based on the results we
can cautiously conclude that these are largely unconnected. Only for the Telegraaf the
absence and presence of a reference to the use of local resources is the news article and in the
comments is correlated positively. It was unexpected that there is no such correlation between
the article content and comment content for the NRC, as Reader (2012) observed that when it is impossible to post comments anonymously, the quality of the comments “closely
approximate[s] the quality of the news media” (p. 565).
This scarcity of correlations between the news article content and the comments
suggests that there may be little to no framing effect. Explaining the absence of a
phenomenon tends to be more challenging than its presence. As Lecheler, Bos and
Vliegenthart (2015)’s findings suggests that refugee related news frames and emotions would
have an emotional effect, an absence of any effects could be indicative that the commenters
did not actually read the news article concerned. However, at the same time Van Gorp (2006)
has argued that an audience may simply be unaware of the emotionality of these news articles
and the media in general, which is why ‘media literacy’ should be improved (p. 242).
Another reason for missing correlations could be the absence of comments by i) those
concerned with privacy and the lack of anonymity on Facebook (Reader, 2012) or ii) those
who are already involved in commenting on the news site (Hille & Bakker, 2014). Meijer and
Groot Kormelink (2014) state that rather than “considerations of personal reputation, also in
connection to privacy, appear to lead to a certain self-restraint in digital social news use” (p.
12). Interacting with news on Facebook may not be the most desirable place for doing so.
28 Due to this low amount of non-regular news articles in the Telegraaf it was not possible to calculate the extent
of the interaction between the news article and the newspaper by means of inserting the interaction effect in a multiple linear regression (the interaction was not shown due to collinearity). The Telegraaf almost exclusively posts regular news articles , whereas for the NRC is only the case in half of the time. .
What is more, the fact that commenters question the credibility of the newspapers, as
observed by Druckman (2001) in his study of the effects of, amongst others, the humanitarian
frame, could present another explanation. Among the sampled comments were expressions of
dislike and objection directed towards the article, journalist and the newspaper, with some
commenters questioning the quality of the newspaper and its journalism.
Meanwhile, a limitation of this study is the discrepancy between the amount of regular
and non-regular news articles of the newspapers, which makes it difficult to determine
whether the differences between the newspapers’ articles are due to newspaper characteristics
and conventions or, more specifically, because of the type of news articles published. It is also
unclear whether the majority of Telegraaf articles are always regular news articles. Perhaps
this type of article is mostly used for refugee (crisis) related news and maybe the randomly
created sample just happens to include a disproportionate amount of regular news articles.
Another limitation is the rather dissimilar amount of comments posted to the
newspapers’ Facebook post-embedded articles (see appendix). The higher amount of comments to the Telegraaf translates into a greater chance that these comments feature the
scale elements. As such the scale scores of the Facebook comments to the Facebook
post-embedded-articles of the Telegraaf are predisposed to being higher than those of the NRC.
Furthermore, in the intercoder reliability test most indicators received an alpha value
between 0.5 and 0.6, which with leniency is seen as indicating weak but sufficiently
reasonable reliability29. The results should therefore be seen as tentative. Differences in
language skills and political knowledge, potential issues to reliability (Peter & Lauff, 2002),
are unlikely to have played a role. Krippendorff’s alpha was employed because it takes into
account how indicators are measured (Hayes & Krippendorff, 2007).
29 This while Krippendorff (2004)1 and Krippendorff (2004)2 argue that indicators above .667 are acceptable
only if the findings do not have great implications. Many of the codebook variables are nominal and
dichotomous, which means that the alpha value is lower and that one can be more lenient with the results. See appendix for a link to the reliability test datasets. If unavailable, contact the author at esmeraldabon@gmail.com.
Finally, the last limitation of this study concerns generalizability. This study only
included the online versions of two national newspapers and a study of different newspapers
would give dissimilar results. In fact, in the sampling stages of this study it became evident
that there are large differences regarding the social network use of the newspapers as well as
their coverage of the refugee crisis. Between-newspaper differences are therefore to be
expected and should be contextualized30. The study of the refugee (crisis) issue related
coverage of more newspapers would provide supplementary support for the results of this
study and could be used for truthfully testing the humanitarian frame and its indicators.
However, regardless of these limitations and even without the feedback from the
newspapers concerned, this study has touched on the link between sensationalism and
newspaper types, using human interest framing, humanitarian framing, issue elaboration and
emotional neutrality scales. It presents an elaborate and original picture of how refugee
(crisis) related issues are dealt with in the media in a varying sensational manner, and makes
the following, specific contributions. First, by including the human interest frame and by
contrasting it to the humanitarian frame this study responds to the criticism by Vliegenthart &
Van Zoonen (2011) on the missing reflection and adaptation of previous frame theorizations.
The findings of future studies on framing (effects) and immigration could help delineate the
human interest and humanitarian news frames further. They could contribute to the
specification of the frame criteria and provide further insight into the quality of these news
representations, for the sake of the audience, democracy and the refugees.
Second, like Burgers and De Graaf (2013), this study contributes to developing
journalism and political communication research by presenting an approach for the
identification of sensational elements in news text. The fact that a traditionally and
30 To contextualize the findings, the newspapers were contacted. The Telegraaf did not respond. Contact was
established with the NRC’s ombudsman, domestic and foreign news correspondents. They received a report of the results in early June, but did not respond within the time period allocated to this study, due to holidays and commitments, causing them to be out of office. This report (which is in Dutch) can be found in the appendix.
