University of Amsterdam
Graduate School of Communication
Master's programme Communication Science
Europe in the news: the effects of European Commission speeches on newspaper coverage and the absence of Dutch government involvement.
A longitudinal analysis 2007-2015 Master's Thesis Supervisor: Dr. Damian Trilling Submitted by: Erik de Vries Student number: 6056180 erik@devries.pm January 29, 2016
Acknowledgements
First of all I would like to thank my supervisor, Dr. Damian Trilling, for his encouraging support and ideas, as well as his very timely responses to any questions.
I would also like to thank my brother, Thijs de Vries, for his support and valuable technical advice. And finally, I would like to thank my family and friends, who have always been there to support me, and without whom I would probably not have made it to this point.
Abstract
A lot of research has been done in the field of media coverage concerning the European Union. However, most of this agenda building research focuses on specific topics and/or specific periods. In contrast, this study is concerned with a longitudinal analysis that
investigates the relationships between the topics present in European Commission speeches, Dutch government speeches on related topics and Dutch newspaper coverage on EU-related topics. By incorporating these agendas, the study also provides an overview of the relationships between different kinds of agenda building, most notably intermedia agenda building and agenda building on a national versus European level. Because the study
employs inductive topic modeling, no predefined topics are used. Rather, the topics are based on the most prominent topics that are present in the dataset, which means these topics are per definition relevant to the study. The results are obtained using vector autoregression analyses, enabling the assessment of over-time effects between the different agendas. Most importantly, the results indicate that the European Commission has substantial direct influence over Dutch newspaper coverage, especially quality newspaper coverage. But only on topics in which the European Commission is actively involved and which are both immediate and endogenous. In addition, the Dutch government turns out not to be much involved at all in EU-related topics, while the European Commission itself is most strongly influenced on topics that directly concern the functioning of the European Union and the European Commission itself. Keywords: EU, Netherlands, agenda building, topic modeling, VAR, time series, automated content analysis, political speeches, newspaper coverage
Note: All scripts and data discussed in the article are on request available from the author Introduction
The role of the European Union in the daily lives of citizens living in member states has been ever increasing. A recent example of this influence can be found in the Euro currency crisis and the more immediate example of the refugee crisis that started taking serious form in August 2015. Especially this last crisis shows how European common policy (Schengen) can impact the lives of citizens living in EU member states. At the same time, this crisis also shows the disagreement between and within EU member states regarding the future course of
action. In both cases disagreements are, through the electoral system, in the end based on the opinions of individual citizens. And as these citizens base their opinions primarily on the information that is provided by news media (Koopmans, 2007), the above examples illustrate the relevancy of the relationship of the European Union and its institutions on the one hand, and national news media on the other.
This relationship has already often been investigated in numerous different contexts, varying in scope from one EU member state to the entirety of the European Union, and investigating different topics such as the amount of coverage, the subjects that are being covered and the way in which these subjects are being covered, but also the (non-)existence of a European public sphere in different kinds of media, such as on television, in newspapers and on the Internet. However, most of this research focuses on shorter time periods, or on specific events, such as elections. The current study, in contrast, aims to conduct a longitudinal
analysis (March 2007 – October 2015) of the relationships between the European political agenda, the Dutch political agenda and the Dutch media agenda regarding EU-related topics. In addition to that, the current study does not focus on the general representation of EU topics in news media, or on specifically preselected topics. Instead, it utilizes inductive topic
modeling to assess the direct relationship between a concrete European agenda, represented by European Commission speeches, with the media agenda as represented in national
newspapers and the political agenda of the Dutch government, as represented in speeches by government members. Besides that, the inclusion of four Dutch national newspapers
(Volkskrant, NRC Handelsblad, Telegraaf, Algemeen Dagblad), enables the assessment of intermedia agenda building regarding EU-related topics. In this context, the former two newspapers can be considered quality newspapers, while the latter two can be considered popular newspapers. This results in the following research question:
RQ: What topical relationships exist between the European political agenda, the Dutch political agenda regarding Europe and the Dutch media agenda regarding Europe?
Theoretical framework
As illustrated above, the media are a key actor when it comes to informing the public about the EU. Because of the importance of what the media discuss, the media agenda, and more
broadly the theory of agenda building is central to this study. The media agenda is constructed based on numerous processes, most notably the impact of (unexpected) real-world events, the news selection process and the influence of external actors, such as politicians (Berganza, 2009; Brandenburg, 2002; Shoemaker & Reese, 1996, in: Denham, 2010), while similar processes influence the political agenda (Van Aelst & Walgrave, 2011). The aim of the current study is however not to fully assess the agenda building dynamics of either the media or the political agenda, but rather to specifically analyze the relationships between and within them. Therefore it is acknowledged that there are multiple factors influencing agenda building processes, but it is not the goal of this study to incorporate all those factors. However, without taking into account some major real-world events, such as the Euro currency crisis and the refugee crisis of autumn 2015, it would not be possible to adequately interpret the results of this study, and therefore these kinds of events will be used to provide context to the results. Media agenda building
Besides the distinction between different agendas, such as the political and the media agenda, it is also possible to distinguish two different types of agendas within the political agenda, namely the substantial and symbolic political agenda. While the former is concerned with the allocation of resources and actual political action, the latter is concerned with politicians' viewpoints and arguments as they can be found in for example publications and parliamentary questions, and thus with political ideas (Van Aelst & Vliegenthart, 2014; Walgrave & Van Aelst, 2006). As this study uses speeches by national and European actors as a primary data source, and speeches definitely do not directly represent actual policy measures, the focus will be on the symbolic political agenda, rather than the substantial one.
The relationship between the political and media agenda is however of prime concern, and based on previous research and results, there are a couple of ways to analyze this
relationship. Basically, three approaches can be distinguished, namely those assuming a causal relationship from the political agenda towards the media agenda, those assuming the inverse relationship, from the media agenda towards the political agenda, and those assuming a reciprocal relationship between the two agendas. Which of these assumptions is most viable depends on the specific context of the relationship, as will be discussed in detail below (Denham, 2010).
same agenda are also possible, as is the case with intermedia agenda building (Denham, 2010). In this case, media outlets influence the agenda of each other, resulting in one of the outlets initiating the coverage on a particular topic, while the other outlets follow. This results in a reciprocal process, by which media outlets reinforce each other. The concept of
intermedia agenda building is of particular importance to the current study, because multiple newspapers are analyzed. It also provides additional insight compared to regular agenda building between the political and media agenda by also taking into account the effects between media.
Just like there are multiple ways to examine the relationship between the political and the media agenda, studies show results with various effect sizes regarding the impact of one agenda on the other (Walgrave & Van Aelst, 2006). Similarly, and often dependent on the research design, results differ with regards to the direction of a causal relationship. Some studies find that the political agenda dominates the media agenda (Brandenburg, 2002; Walgrave & Van Aelst, 2006), while others find the media agenda dominates the political agenda (Van Aelst & Vliegenthart, 2014 ; Van Noije, Kleinnijenhuis & Oegema, 2008). It is noteworthy that studies finding the political agenda dominating were conducted in an election context, while those finding the media agenda dominating were conducted during
non-election periods.
Related to the idea of the political agenda dominating the media agenda is the concept of information subsidies. This concept is based on the assumption that public relations
officials supply ready-made pieces of information to journalists in the form of press conferences and press releases, with the aim of influencing what topics are covered, and in what way (Kiousis, Mitrook, Wu & Seltzer, 2006; Parmelee, 2013). The concept of
information subsidies can especially be applied to the case of the European Commission. Over time the communicative aims of the Commission have changed radically, and are currently in a phase of more transparent communication towards the media and the public, which indicates a slow adoption of a more mediatized form of communication. This has also been noticed by journalists, who are observing an increasing influence of public relations on the Commission's communication (Meyer, 2009). At the same time, other studies indicate that European institutions in general do not make much of an attempt at influencing the media agenda at all. And efforts that are undertaken, are in majority aimed at EU correspondents, rather than journalists in general (Statham, 2008). Regardless of the extent to which the
European Commission undertakes such attempts, it is however clear that these attempts are aimed both at influencing what the media cover, and how they cover it (Brüggemann, 2005). And, as Statham (2008, p. 419) formulates it, in the end “… the media tend to follow the political system over Europe; they represent much more than lead political debate...”.
Trenz (2004) also illustrates that a significant amount of attention is paid to news within a European context in national quality newspapers in various countries. Unfortunately, Trenz (2004) does not compare the coverage in quality newspapers directly with popular newspapers, as included in the current study. A quick search through the LexisNexis
newspaper database however shows that there is far more news covered in a European context in quality newspapers (NRC, Volkskrant) than there is in popular newspapers (Algemeen Dagblad, Telegraaf). Combining these two pieces of information leads to the assumption that because quality newspapers devote more of their news-carrying capacity to coverage within a European context, it is also easier for European actors to get in and influence this coverage than it is to influence the much more limited coverage of popular newspapers. The other way around, it is unlikely that the political agenda of the European Commission is influenced by the national newspaper media agenda of just one country, as it needs to take into account the media agendas of all 28 member states. Combined with the aforementioned concept of symbolic political agendas, which are concerned with political ideas and opinions rather than actual policy implications, this results in the following hypotheses (see also figure 1):
H1a: The symbolic political agenda of the European Commission will influence the Dutch national media agenda.
H1b: The national media agenda will not influence the symbolic political agenda of the European Commission.
H1c: The symbolic political agenda of the European Commission will have a stronger influence on the Dutch quality newspaper agenda than it will have on the Dutch popular newspaper agenda.
Political agenda building
Similar hypotheses can be formulated regarding the relationship between the European Commission's political agenda and that of the Dutch government. It should be noted that little research has been conducted into the relationship between the symbolic political agenda of the
European Commission and that of member state politicians or political institutions. This enables the current study to provide additional insight into this relationship. The main conclusion that can be drawn from studies investigating substantive political agendas is that the process whereby member state actors influence the political agenda of the European Union is highly complex, which is mostly caused by the interaction between actors from different member states (Börzel, 2002; Maes & Verdun, 2005, Princen, 2007, Schlesinger, 1999). In this regard, a rough distinction can be made between three different types of member states, namely those that actively push the development of European policies, those that actively try to prevent the development of European policies and those that do neither, and depending on the topic either form an alliance with those who are trying to implement policy measures, or those that are trying to block them (Börzel, 2002).
Various studies indicate that The Netherlands can in general be considered a member state that actively promotes the development of European policy (Börzel, 2002, Maes & Verdun, 2005). However, these results are more than 10 years old, and more importantly from before the economic crisis that started in 2008. Since then, populist and anti-Europe parties have been rising in popularity, with in The Netherlands the example of the PVV (Partij voor de Vrijheid). Because of this it can be argued that the active promoting of European policy has diminished in more recent years. Regardless, the general argument can be made that highly industrialized and regulated member states, such as The Netherlands, will strive to get policies more unified at the European level (Börzel, 2002; Princen, 2007). And whether it is to
promote or obstruct European policy development, it can be concluded that the Dutch
government at least attempts to have significant influence over the agenda building process of the European Commission.
The studies mentioned before however emphasize the importance of coalitions and alliances for EU policy development. Thus it can be argued that even if The Netherlands plays a role in building the European Commission agenda be it substantially or symbolically, it will be relatively small. And if it does, it does not do so by the means of speeches, as these are seldom directly aimed at the European Commission. It is more likely that these speeches, with a national audience in mind, are aimed at explaining and/or promoting European policy measures. This leads to the following hypotheses (see also figure 1):
Dutch national symbolic political agenda.
H2b: The national symbolic political agenda will not influence the symbolic political agenda of the European Commission.
National reciprocity
While the political agenda can be dominant over the media agenda, studies also reveal a reversed relationship, at least for Western European countries. First of all members of
parliament consider the media to have some to very much influence over the political agenda (Van Aelst & Walgrave, 2011). This is closely related to mass media offering access to mass audiences that will keep MPs responsible for not responding adequately to topics raised in the media (Van Noije et al., 2008). The reliance of politicians on media coverage is also
illustrated by the backing of roughly one out of every three Dutch parliamentary questions by references to one or more media outlets. Related to that, 80% of the topics is already present in media coverage before the question is asked (Van Aelst & Vliegenthart, 2014). And while politicians often use media coverage as a source for their questions, previous media coverage also makes these questions more newsworthy (Van Santen, Helfer & Van Aelst, 2015). All of this illustrates the reciprocal nature of the relationship between Dutch politics and the media. But where these studies only take into account the relationships between the national political agenda and the media agenda, the current study also incorporates the direct influence of the European political agenda on the media agenda, as well as possible intermedia agenda building. Both of these might mediate the reciprocal relation indicated above.
Reciprocity can also be expected because of the importance of domestication and news values for news production. It should however first be noted that the EU in general gets very little coverage in national media. And when the EU is covered, this is mostly related to specific high-profile events, such as EU summits or the European Parliament elections (Boomgaarden, Vliegenthart, De Vreese & Schuck, 2010; De Vreese, Banducci, Semetko & Boomgaarden, 2006; Gleissner & De Vreese, 2005; Kaitatzi-Whitlock, 2007; Machill, Beiler & Fischer, 2006; Van Noije, 2010). In addition to that, some of these studies also point out the importance of a national context for EU news coverage (Kaitatzi-Whitlock, 2007; Koopmans, 2007; Machill et al., 2006). This national context is most often found in the presence of national political actors in EU news coverage (De Vreese et al., 2006; Koopmans, 2007). Combining the lack of interest in EU news in general with the relative newsworthiness of elite
people, such as government representatives, it is reasonable to assume that when EU topics are being covered, there is a high chance this coverage is related to elite people. Koopmans (2007) illustrates this for specifically the Dutch case, by showing that EU-related media coverage is relatively often (compared to purely national coverage) related to either European level or national level executive/government actors. Because of the reliance on government actors, and because of the argument made above, regarding national politicians following instead of leading the media agenda, it is safe to assume a reciprocal relationship exists between the Dutch symbolic political agenda regarding the EU, and the media agenda regarding the EU.
This reciprocal relationship is however dependent on the type of newspaper (quality or popular) concerned. As Van Aelst and Vliegenthart (2014) illustrate for the Dutch case, the popular newspaper Telegraaf dominates as a source for parliamentary questions since 2007, followed at some distance by the quality newspapers Volksrant and NRC Handelsblad. This illustrates that popular newspapers have the upper hand in influencing the political agenda. Regarding the opposite influence, from the political to the media agenda, results are less clear, but still suggest a small edge for the Telegraaf. Based on this, the following hypotheses can be formulated (see also figure 1):
H3a: The symbolic national political agenda and the media agenda regarding the EU will influence each other reciprocally
H3b: Dutch popular newspapers will both influence and be influenced stronger by the symbolic national political agenda than Dutch quality newspapers
Intermedia agenda building
As already briefly mentioned above, intermedia agenda building is concerned with the relationships between the agendas of different media outlets. Multiple studies find that these agendas are often closely related, both during and outside of election times, and within and between different types of media (Boyle, 2001; Denham, 2010; Lim, 2011, 2012; Vliegenthart & Walgrave, 2008). Regarding specifically newspapers both Lim (2011) and Vliegenthart and Walgrave (2008) find that, compared to other media types, there occurs relatively little
intermedia agenda building between newspapers. When distinguishing between following, upgrading or ignoring coverage from a competing newspaper, journalists most often just
ignore coverage by other newspapers. And while topics may indicate intermedia agenda building adequately when assuming competing newspapers follow in their coverage, this is no longer substantively true when assuming they augment the topic with new information or a different viewpoint (Lim, 2011). This illustrates the difference between agenda building on topic and frame level. Because the current study is assessing agenda building on an aggregate level, it is obvious that the focus is on observing if agenda building occurs, rather than how it occurs. As the studies mentioned above do not differentiate between popular and quality newspapers, there is no theoretical reason to assume that the agenda building effect will be stronger in one way than the other. The results of this study can however provide additional insight into the possible direction of intermedia agenda building. Based on the above, the following hypothesis is formulated (see also figure 1):
H4: Popular and quality newspaper media agendas will influence each other with regards to EU news coverage
Issue dependence
Lastly, agenda building, be it of the political agenda or the media agenda, is highly dependent on the type of issue which is transferred from one agenda to the other. For example,
politicians might have “preferred” issues, on which agenda building effects of the media are stronger (Denham, 2010; Walgrave & Van Aelst, 2006). In general the media tend to have less agenda building power with regards to domestic issues than with regards to European-level issues (Van Aelst & Vliegenthart, 2014; Van Noije et al., 2008). Besides the effect sizes, also the direction of the agenda building relationship between the political and the media agenda can vary depending on the issue. Soroka (2002) concludes that the effect of the media and/or political agenda on the public agenda is dependent on the extent to which the public
experiences the problem directly. A similar distinction can be made for the relation between the political and the media agenda, and the relation between media agendas. This distinction, between endogenous and exogenous issues depends on the cause of issues lying either within or outside of the sphere of influence of official institutions, such as political, financial and science institutions (Vliegenthart & Walgrave, 2008). A specific case can also be made for EU-related issues, as the European Commission has to take into account the agendas of 28 member states. Because of this, influence on the political agenda of the European
Commission is most likely to occur when an issue is present in multiple agendas and/or multiple member states, and thus for issues that are shared between numerous member states, rather than issues concerning one or only a few member states.
While specific issues are analyzed in the studies mentioned above, the current study utilizes inductive topic modeling. This means that while the other studies analyze a limited set of predefined issues, this study uses the actual data to determine which are the most
frequently occurring issues, providing a set of issues that is always relevant and specific to the dataset. Because of this, an additional research question has been formulated, as well as a hypothesis regarding issues and their influence on the political agenda of the European Commission.
RQ1: In what way do the agenda building dynamics between the European symbolic political agenda, the national symbolic political agenda and the media agenda differ based on the topic concerned?
H5: The influence of the national symbolic political agenda and media agenda on the symbolic political agenda of the European Commission depends on the topic, with more influence when topics are shared between more member states.
Figure 1: Theoretical model (excluding topic influence)
To summarize, the current study does not focus on one specific aspect of agenda building, as do the studies discussed above, but rather incorporates different concepts in a way not found in any of the studies mentioned above. These concepts are intermedia agenda building, symbolic political agenda building on two levels, European and Dutch, and media
European Commission Popular newspapers National government Quality newspapers H1 H1 H2 H3 H3 H4 H4
agenda building. Incorporating specifically the political agenda on two different levels enables a more thorough analysis of agenda building dynamics in the case of EU-related topics, while incorporating intermedia agenda building makes it possible to take into account possible indirect relationships. Figure 1 illustrates the incorporation of all the concepts in a theoretical model.
Method Data collection and processing
An automated content analysis has been conducted for the period ranging from March 2007 until October 2015. The data sources that have been used are speeches by members of the European Commission, speeches by members of the Dutch government (both ministers and state secretaries), and newspaper coverage on EU-related topics in the four national daily newspapers with the largest circulation. These four newspapers can also be divided in two popular newspapers (Algemeen Dagblad, Telegraaf) and two quality newspapers (NRC Handelsblad, Volkskrant). Sample size in number of speeches/articles and when applicable circulation numbers are presented in table 1.
Table 1: Sample sizes and 2014 circulation numbers (Wikipedia, n.d.) European Commission Dutch Government Algemeen Dagblad Telegraaf NRC Handelsblad Volkskrant Sample size 7,427 1,608 5,571 10,789 16,684 13,947 Circulation - - 420,000 520,000 200,000 250,000
European Commission speeches have been collected from the official press release database by filtering specifically for speeches (http://europa.eu/rapid/search.htm). Dutch speeches have been collected in a similar fashion, from the official government website (https://www.rijksoverheid.nl/ministeries/). Because of the structure of this website, speeches were collected per ministry, and as only speeches concerning EU-topics were of importance, the search string “EU OR Europa OR Europees OR Europese” has been used to filter out relevant results. The search string used is relatively simple and broad, because the search engine did not support combining multiple keywords using quotation marks. To collect newspaper articles the LexisNexis database has been used in combination with the following search string: “"de EU" OR "Europese Commissie" OR "Europese Unie"”.
Because the dataset consists of items in different languages, and sometimes even different languages within a single item, all non-Dutch items had to be translated to Dutch before a topic model could be generated from the data. The choice was made to translate to Dutch instead of English because a vast majority of the dataset, represented by the newspaper articles and most Dutch government speeches, was already in Dutch, while only the European Commission speeches were mainly in English. Translating has been done through a self-written Google Translate python script. It operates by letting Google Translate automatically detect the language of portions of text, so that also texts with multiple languages in them are accurately translated. Balk et al. (2012) show in an exploratory study that Google Translate delivers acceptable results when translating medical academic articles from Romance languages to English. Interpretation of translated texts is still significantly different/worse than for English texts. This however concerns the substantive interpretation of the articles, rather than deriving the main topics from these articles. And as the current study is only concerned with the latter, Google Translate, even with translating mistakes, is considered to be accurate enough to infer the main topics from the translated text. In addition, there is relatively little research into the accuracy of specifically Google Translate, and studies that do investigate this, do so in very specific exploratory contexts.
Because no standard software is available for scraping and processing texts from both of the websites and LexisNexis, custom scripts in the Python coding language were written for each source separately. This has resulted in two collections of scripts, one for getting the data, and one for cleaning and processing it. Additional scripts were also written to
compensate for an unforeseen problem, namely that some of the Dutch speeches were
provided in pdf files, rather than as html web pages. All these scripts and the resulting data are on request available from the author. Please refer to the title page for contact information.
Cleaning of the newspaper articles involved removing metadata from the text body, as well as datelines, reporting locations and common opening lines (for example: “Amsterdam, 11 juli 2013. Van onze verslaggever,”). Besides, as publication dates were provided in multiple different formats, they were converted to a single consistent format. For the European Commission and Dutch speeches, similar procedures were followed, removing metadata, document titles and when present contact information. Following this, all non-alphanumerical characters were removed from the data, and the data was lowercased.
both using the nltk python package (Bird, Klein, & Loper, 2009). Finally, extremely short texts were dropped from the dataset. In the case of newspaper articles this meant removing all items that had a length (in words) shorter than one standard deviation from the mean (< ~150 words), while for the European Commission speeches this was 1.5 standard deviation (<~300 words), and for the Dutch speeches 1.4 standard deviation (<~170 words) was used.
Topic Modeling
Using the parsed and translated data, a topic model has been constructed using the gensim python package (Rehurek & Sojka, 2010). Latent Dirichlet Allocation (LDA) has been used as the specific method to construct this model. Because LDA is an unsupervised
machine-learning algorithm, it is based on the assumption that each text contains a number of different topics, in different distributions (Blei, 2012; Blei, Ng & Jordan, 2003; Boumans & Trilling, 2016). These topics consist of a number of words, with the most important words being determined based on a combination of the number of occurrences and how specific the word is to the topic. To generate the topics, a choice first has to be made regarding the total number of topics that can be present in the model. The generated topics can then be ranked in terms of coherence, using the Umass value. This measure, developed by Mimno, Wallach, Talley, Leenders and McCallum (2011), indicates to what extent words within a topic are
semantically coherent with each other, and lower absolute values of this measure indicate more coherence.
To identify the optimal number of topics for the current dataset, first some exploratory models were tested with the maximum number of topics ranging from 40 to 150. This was done only for the European Commission speeches, as the large number of newspaper articles was collected using relatively generic search terms. Because of this, including the newspaper data could have caused skewing of the topic model with topics that would be not at all or only slightly related to the EU and the European Commission. The Dutch speeches were not included because of their small sample size, and because no effect from the Dutch speeches to the European Commission speeches was hypothesized. For each model, the top 20 topics based on their Umass value where examined to investigate what kind of topics developed, and how consistent they were. While increasing the number of topics by definition leads to more coherent topics, it also allows for more specific and immediate topics, as there is more room within the model to distinguish between these topics. Using a lower number of topics,
conversely, decreases the space for specific topics, and results in topics that are more general in nature, but also less coherent.
Based on a combination of the Umass values and the extent to which the topics could be substantively interpreted, the choice was made to run a final topic model on the European Commission speeches with a maximum number of 70 topics, and a total of 50 passes over the data. By monitoring the convergence parameters for each pass, it was established that after approximately 40 passes 28% of the articles converged, but that this number did not increase substantially with additional passes. Thus it can be concluded that the maximum level of convergence was reached after 50 passes. The resulting topic model has then been applied to the entire dataset to determine the topic distribution for each text. After that, a total score for each topic per source and day was computed by summing up the scores on each topic. The different sources are European Commission speeches, Dutch government speeches, popular newspaper articles and quality newspaper articles. For days in which there were no speeches or relevant newspaper coverage, zero has been used as topic score for all topics on that day, based on the assumption that when no data was present, the topics were also not discussed that day. In addition to that, all Sundays have been removed from the dataset, as only the Telegraaf newspaper was published on Sunday during a part of the research period, and in general no speeches were held during the weekends. Also, the summer and Christmas recess periods for the European Commission were removed from the dataset, as almost no speeches were held during these periods. The size of these periods was 5 weeks for the summer recess, from mid-July until the end of August, and 2 weeks for the Christmas recess.
Analysis
In order to test the hypotheses and research question formulated earlier, a time series analysis is the most viable approach. As the expectation is that the relationships between the different agendas might run in multiple ways, a vector auto-regression (VAR) analysis would be most suitable (Vliegenthart, 2014). In such a model, all variables serve both as independent and dependent variables in different regression equations. This means that estimations of the independent variable are based on both previous lags of the independent variable itself and previous lags of the other variables in the model (Vliegenthart, 2014).
In the context of the current study, a VAR model should be tested for each topic separately, with the topic scores of each source (European Commission, Dutch government,
popular newspapers, quality newspapers) as a separate time series, resulting in four time series per topic in total. In order to be able to conduct a VAR, the data should be stationary, in that it does not exhibit an over-time trend (Vliegenthart, 2014). This can be tested using the Dickey-Fuller test, which indicates stationarity when the test results are significant. If the data does contain a trend, the first difference can be used instead of the original time series. Following the stationarity test, a maximum number of lags for the model needs to be defined. In principle, this needs to be based on theory, but various model fit-statistics, such as AIC and BIC can be used to aid in the selection of the maximum number of lags. Because a large part of the dataset consists of media coverage, and media attention is often both immediate and short-lived, effects might be expected within several days. Contrary, as political speeches are more infrequent, effects might take somewhat longer to occur. To take both into consideration, a maximum lag length of 7 has been chosen to test model fit of all models. AIC has then been used to select the actual number of lags for the specific model, mostly because AIC and FPE (final prediction error) were always in agreement with regards to the best fitting model, while the other statistics (HQIC, SBIC) did often not agree with each other or the AIC.
Coefficients produced by a VAR model have a substantial risk of high levels of collinearity due to the multiple lags used in the model (Vliegenthart, 2014). This means that while both a European Commission speech and newspaper coverage could cause a Dutch government speech on the subject, the European Commission speech and newspaper coverage can also be highly correlated. Because of this, one should exercise caution when interpreting these coefficients, and relationships between variables are better interpreted using Granger-causality tests (Vliegenthart, 2014). Granger-Granger-causality indicates to what extent the current value of a variable is not only predicted by its own past values, but also by the past values of another variable. Thus Granger-causality does not imply true causality, it simply indicates the additional explanatory power of a variable.
Results Topics
The topic model described above produces a lot of topics that can be substantively interpreted. A selection of topics has therefore been made from the top 20 topics ranked by Umass topic coherence. This selection is based on both the Umass values of the topics and their
topics could have been included in the analyses as well, but were left out because of the limited space to discuss them. An overview of all the top 20 most coherent topics and the 20 most defining words of each topic can be found in the appendix. In general, the topics discussed here are not very coherent, with Umass values ranging from -150 to -228. Other studies using the same method of topic modeling yielded coherence results in a single-digit negative range (Stevens, Kegelmeyer, Andrzejewski & Buttler, 2012). However, an argument can be made that Umass values should in this case be interpreted relatively, instead of
absolutely, because they are used to get the most coherent topics from the dataset. The actual value of the measures simply indicates the coherence achieved, which is the maximum possible level of coherence with this dataset, also when looking also at the low convergence rate of the model. Besides, another study finds negative values that are three to four times larger than in this study (Küntzler, 2015), and the authors of the measure note explicitly that “with respects to human judgements… coherence scores do not always indicate a better representation of distributional information.” (Stevens et al., 2012, p. 960). So these values should not be used as the sole indicator for topic coherence. In table 2, the six topics that have been selected are presented based on their top defining words, their Umass coherence and a short interpretation of the topic. The values in brackets next to the individual words indicate the position of these words in the top 20 of most defining words for the topic.
Table 2: Topics in European Commission speeches
Most defining words (rank in top 20) Umass Topic interpretation rusland(1); economisch(5); samenwerk(8); betrek(11);
energie(15)
-150.13 Economic relations with Russia/EU sanctions europes(1); verdrag(2); parlement(5); lissabon(6); burger(7); -171.06 Lisbon Treaty/EU
constitution handel(1); invester(2); markt(5); wereld(6); bedrijv(10);
onderhandel(12); vs(14); amerikan(19)
-190.15 International trade/EU-US trade relations energie(1); hernieuw(4); ontwikkel(9); vervoer(13);
duurzam(14) -202.75 Renewable energy sources
president(1); belangrijk(6); land(8); premier(9); bezoek(10); blij(11); samenwerk(12); dank(14); goed(15); steun(16)
-211.54 Relationship between EU and member states economisch(1); groei(2); eurozon(3); hervorm(5); crisis(7);
financiel(8); structurel(13); fiscal(16); monetair(17)
-227.31 Economic crisis/Euro crisis/Structural reforms
Some additional explanation is necessary regarding the inclusion of the topic about the relationship between the EU and its member states. This topic was included because while it is clearly a substantive topic, it is not about any specific topic, but rather about the EU dealing with itself. It is interesting to see that such a “non-issue” is so prominently present in the speeches held by the European Commission.
Agenda relationships
When looking at the results, it should first of all be noted that for all models and all variables, the results of the Augmented Dickey-Fuller tests were significant, indicating that all data was stationary. Thus for all analyses the original variables have been used, and no first differences have been calculated. The Granger-causality tests, as presented in table 3, and the amount of explained variance displayed in table 4 indicate that H1a can be confirmed. Significant Granger-causality is found for quality newspaper coverage in the EU-Russia topic, the Lisbon Treaty topic, the renewable energy topic and the economic crisis topic. Similarly, European Commission speeches significantly Granger-caused popular newspaper coverage in the renewable energy topic, the EU relations topic and the economic crisis topic. The most notable difference between the results regarding popular and quality newspapers is that coverage on the EU relations topic is only significantly Granger-caused by European Commission speeches for popular newspapers, and not for quality newspapers.
Hypothesis 1b cannot be confirmed so easily. Although no influence of Dutch national newspapers was expected on the European Commission speeches, this turned out not to be true for all topics. While the European Commission speeches were not Granger-caused by newspaper coverage in the EU-Russia topic, the EU relations topic and the economic crisis topic, Granger-causality was present in the other topics. Most notably Lisbon Treaty speeches were significantly Granger-caused by both popular and quality newspaper coverage.
International trade speeches were only Granger-caused by popular newspaper coverage, while Granger-causality regarding renewable energy speeches was only found with quality
newspaper coverage. Lastly, quality newspaper coverage is Granger-caused by European Commission speeches for two more topics than is the case for popular newspapers, when excluding the aforementioned EU relations topic. This clearly provides support for H1c.
In general, the European Commission speeches do not Granger-cause Dutch government speeches, with the notable exception of the EU-Russia topic, where
Granger-causality is significant. This indicates that H2a can only be confirmed for this topic, and has to be rejected for the others. The other way around, regarding the influence of Dutch
government speeches on European Commission speeches, a similar pattern emerged. Overall, no significant Granger-causality was found, with the exception of the Lisbon Treaty topic. For this topic, contrary to the other topics, H2b cannot be confirmed.
The reciprocal relationship between the Dutch government speeches and newspaper coverage in general was not as strong as expected. Only in the economic crisis topic did quality newspaper coverage Granger-cause Dutch government speeches, while on the same topic only popular newspaper coverage was Granger-caused by Dutch government speeches. Regarding the other topics, however, no such relationships can be found. This is with the exception of again the Lisbon Treaty subject, where Dutch government speeches strongly Granger-cause both popular and quality newspaper coverage. But in general there is no clear reciprocal relationship between Dutch government speeches and newspaper coverage, leading to a rejection of H3a. And as there is not much influence of newspapers on Dutch government speeches at all, H3b needs to be rejected as well. This is with the note that Dutch government speeches do indeed Granger-cause popular newspaper coverage more (two topics) than they do cause quality newspaper coverage (one topic).
Turning to hypothesis 4, regarding intermedia agenda building, a strong relationship is found for all the topics. Looking at the EU-Russia topic, for example, both popular and quality newspaper coverage Granger-causes each other very significantly. But this does only illustrate that the relationship exists, not who is following who. When looking at the
coefficients of the VAR models, which can, along with the standard errors be found in the appendix, it becomes clear that in general both past quality and popular newspaper coverage is suitable to predict the current levels of their own coverage. But besides that, quality
newspaper coverage is also well predicted by popular newspaper coverage for the EU-Russia topic in lags 1, 2 and 3. Despite the risk of collinearity, it definitely seems like popular newspaper coverage is causing quality newspaper coverage much more than the other way around. A similar result is found when looking at the coefficients for the Lisbon Treaty model, where popular newspaper coverage in lag 1 significantly, and with a large coefficient causes quality newspaper coverage. The same goes for the international trade and renewable energy topics, with the only difference being that lag 6 of popular newspaper coverage significantly predicts quality newspaper coverage.
Table 3: Granger-causality tests Topic European Commission Dutch government Popular newspapers Quality newspapers EU-Russia relations European Commission 9.71 3.04 7.86 Dutch government 17.95* 9.84 7.53 Popular newspapers 7.36 9.63 55.40*** Quality newspapers 23.03** 12.54 93.12*** Lisbon Treaty European Commission 21.60** 18.83** 30.23*** Dutch government 5.15 3.58 4.10 Popular newspapers 8.67 23.66** 123.51*** Quality newspapers 20.23** 24.89** 25.60** International trade European Commission 4.18 18.06** 6.26 Dutch government 8.13 5.71 4.59 Popular newspapers 6.52 4.40 14.99* Quality newspapers 11.79 8.25 16.49* Renewable energy European Commission 4.32 4.53 16.81* Dutch government 10.26 5.93 3.58 Popular newspapers 12.90* 3.21 32.99*** Quality newspapers 12.71* 10.60 32.22*** EU – Member state relations European Commission 0.63 2.14 1.81 Dutch government 0.60 4.66 5.96 Popular newspapers 7.99* 4.29 22.33*** Quality newspapers 0.23 0.92 12.13** Economic crisis European Commission 7.75 6.33 1.61 Dutch government 4.31 8.06 16.33* Popular newspapers 44.72*** 13.47* 114.21*** Quality newspapers 20.05** 5.91 9.52
Note: ***=p < 0.001; **=p < 0.01; *=p < 0.05. All values rounded to the second decimal. N = 2294 (all observations, when VAR is conducted, an effective sample size of 2288
observations is achieved). Reading example: Dutch government speeches on the EU-Russia topic are Granger-caused by the EC. Quality newspaper coverage on the same topic is Granger-caused by the EC.
The only topic for which the reverse is true is the economic crisis, where quality newspaper coverage in lag 1 significantly causes popular newspaper coverage. While again caution is necessary when interpreting these coefficients, the same conclusion can also be
drawn from the Granger-causality test, as also there popular newspaper coverage is
significantly Granger-caused by quality newspaper coverage, but the other way around, no significant Granger-causality is found. Because of the results above hypothesis 4 is
confirmed. All the hypotheses tested above are also visualized in figure 2. This figure shows the found results in the context of the theoretical model, where dotted lines illustrate
significant Granger-causality for less than three topics, and the solid lines indicate significant Granger-causality for three or more topics.
Figure 2: Findings
Topic influence
The results show that there are definitely differences in the relations between the symbolic political agenda of the European Commission, the symbolic political agenda of the Dutch government, the Dutch popular newspaper agenda and the Dutch quality newspaper agenda when considering the different topics. In general, none of the actors have much influence on the symbolic political agenda of the European Commission, with the exception of the Lisbon Treaty topic. On this topic both the Dutch government agenda and the popular and quality newspaper agenda are quite influential. Regarding the influence of the European Commission agenda on the other agendas, most influence is visible on the quality newspaper agenda. In almost all cases there is significant Granger-causality, but the relationship is stronger for the EU-Russia, Lisbon Treaty and economic crisis topics, and is absent for the international trade and EU relations topics. Lastly, the European Commission is not so successful with the EU relations topic, which only significantly Granger-causes popular newspaper coverage, but also to a limited extent. The symbolic political agenda of the Dutch government does not influence
European Commission Popular newspapers National government Quality newspapers Legend: = Relation >=3 topics = Relation < 3 topics H1 H1 H2 H3 H3 H4 H4
or is influenced much by any other agendas. The notable exception is again the Lisbon Treaty topic, where the Dutch government agenda Granger-causes all other agendas. Inversely, the Dutch government agenda itself is also once significantly Granger-caused by another agenda, and that is the quality newspaper agenda in the economic crisis topic. It should be noted that quality newspaper coverage itself on the same topic is in turn Granger-caused by the
European Commission.
Both newspaper agendas are in terms of influence not really bound to any specific topic, with exception of the relation between EU-related topics and quality newspaper
coverage already discussed above. Thus to answer RQ1, agenda building dynamics definitely differ for the influence on and by the European Commission when taking into account
different types of topics, and the same goes to a lesser extent also for the Dutch government. The only relation that is not influenced by the type of topic is the intermedia agenda building. Hypothesis 5 also sees partial confirmation based on the results above, as the Dutch
government and newspaper agendas are indeed far more influential on the Lisbon Treaty topic than on any of the other topics.
Finally, and not specifically related to any hypotheses or research questions, some attention should be paid to table 4, which reports the amount of explained variance in the various models. In general, the amount of explained variance for the European Commission is in all models very low (ranging from 0.01 to 0.08), indicating that the other agendas do not have much predicting power when it comes to the topics in European Commission speeches. The same goes for Dutch government speeches, with values ranging from 0.01 to 0.02, with the notable exception of the Lisbon Treaty topic, with an explained variance of 0.14. When looking at the coefficients for this model, the result is explained by the Dutch government agenda influencing itself, rather than other agendas having influence on the Dutch
government agenda. Regarding the popular and quality newspaper agendas, notably more variance is explained for some of the topics, while not for others. The explained variance also comes from these agendas influencing themselves, but as discussed above this is not the only source of explained variance. In general, the topics EU-Russia, Lisbon Treaty and economic crisis were much better explained than the other three topics (international trade, renewable energy and EU relations). While EU relations can be considered a non-issue as it is
represented in this particular topic, the only conclusion for the other two topics can be that there are external factors largely influencing the newspaper coverage on these topics.
Table 4: Adjusted R² values for all topics European Commission Dutch government Popular newspapers Quality newspapers EU-Russia relations 0.02 0.02 0.24 0.27 Lisbon Treaty 0.08 0.14 0.14 0.24 International trade 0.01 0.01 0.02 0.04 Renewable energy 0.02 0.02 0.05 0.05
EU – Member state relations 0.02 0.01 0.03 0.05
Economic crisis 0.01 0.02 0.13 0.23
Conclusion
When considering the results presented above, it should be taken into account that Google Translate has played a vital role in obtaining usable data. As already indicated before, not much investigation has previously been done into the accuracy of Google Translate for topic modeling. Because it was also not the goal of the current study to do so, the translations have been assumed to be accurate enough to derive the main topics from them. However,
inaccuracies in the translation of the European Commission speeches, which were used as the basis for the topic model, might have influenced the results in unforeseen ways. Besides taking this into account when interpreting the results, more research should definitely be conducted to assess the accuracy and reliability of Google Translate for topic modeling.
When looking at the results, none of the relationships, regardless of the topic, are particularly reciprocal, except for the intermedia agenda building. The only examples of (indirect) reciprocity are in the Lisbon Treaty topic, where quality newspaper coverage Granger-causes European Commission speeches, but also the other way around, and the economic crisis topic, where quality newspaper coverage Granger-causes Dutch government speeches, while the government speeches in turn cause popular newspaper coverage. This is also the only case in which quality newspaper coverage causes popular newspaper coverage, but not the other way around.
In general, however, the European Commission speeches influence in particular quality newspaper coverage. These results support earlier findings by Brandenburg (2002) and Walgrave and Van Aelst (2006), but augment them by proving them also outside a specific election context. Because of the longitudinal nature of the analyses conducted here, it is however not possible to explicitly differentiate between election (European Parliament
2009/2014, Dutch national 2010/2012) and non-election times included in the time-frame. The same data could however be used to assess the difference between election and non-election times. But most important of all, these results show that the European Commission can directly influence quality newspaper coverage on EU-related topics. This is of particular interest when taking into account the conclusions drawn by Trenz (2004), which show that in those same quality newspapers a Europeanized public sphere is emerging, which makes it particularly worthwhile for the European Commission to influence coverage in these newspapers.
Dutch non-effects
While European Commission speeches have notable influence on quality newspaper
coverage, this is not so much true for the influence on Dutch government speeches. As argued in the theoretical framework, the rise in anti-EU sentiment in recent years might explain the diminished attention paid by the Dutch government to these topics, because of the negative connotation associated with them. More in general however, it indicates that the Dutch government does not promote the topics that are considered important by the European Commission in its speeches. With one exception, and that is the EU-Russia topic. However, here it might not be so much the case that the Dutch government deliberately follows the line of the European Commission. Rather, The Netherlands has been clearly involved in the dispute between the EU and Russia through the shooting down of Malaysia Airlines flight MH-17 in Ukraine. This flight had a large number of Dutch citizens aboard, and allegedly the flight was shot down over Ukraine with a Russian missile-launching system. Thus in general it cannot be said that the Dutch government is in any way influenced in its discussion of European topics by the European Commission.
Because of this, there is also no indirect relationship by which the European
Commission influences Dutch government speeches, which in turn would make a topic more newsworthy for newspapers because of the added national perspective. This augments the conclusion drawn above, by not only showing that the European Commission is able to influence newspaper coverage directly, but also showing that there is no mediation/support from the Dutch government. As the Dutch government does also not follow topics present in popular and quality newspapers, with the exception of quality newspapers for the economic crisis topic, there is not much reciprocity going on between the Dutch government and the
national newspapers. This is inconsistent with the findings by Van Aelst and Walgrave (2011) and Van Aelst and Vliegenthart (2014), who find a strong reciprocal relationship. One
possible explanation for this might be that these studies consider members of parliament, instead of members of the government, but it might also be that this relationship is
specifically absent on EU-related topics. Regardless, the conclusion can be drawn that the Dutch government does not influence or is influenced by national newspapers on EU-related topics. As the sample of Dutch government speeches on EU-related topics is very small compared to the other sources, even though the used search string was broad, the most viable reason for this lack of influence is that the Dutch government in general does not make much statements regarding EU-related topics. This might be explained by the aforementioned rise of populist and anti-EU political parties, and a general rise in anti-EU sentiment. So even despite the possible positive effects of a national context, which makes it easier for newspapers to domesticate and incorporate EU-related topics into their coverage, this result shows that the European Commission should try to influence the media directly, rather than indirectly through national governments.
From popular to quality news
Regarding intermedia agenda building, some remarkable results were obtained as well. Because while quality newspaper coverage is often influenced by European Commission speeches, this coverage does not in turn cause popular newspaper coverage. Rather, popular newspaper coverage on some of the topics also causes quality newspaper coverage, with the exception of the topic concerning the economic crisis. This result supports the assumption by Lim (2011) that journalists do not just follow coverage, but also upgrade it. The implication would then be that popular newspapers bring the news first, but that quality newspapers expand on it. In addition, newspaper coverage also causes itself based on past coverage within the group of quality or popular newspapers. This does however not per definition say that individual newspapers follow only their own past reporting, it might just as well be true that the Volkskrant and NRC Handelsblad Granger-cause each others coverage, just like the Algemeen Dagblad and the Telegraaf might do so. Further research using the same dataset can be conducted to investigate this. Regardless, these results show that there is also significant intermedia agenda building going on between the clusters of quality and popular newspapers. It is also interesting that the results indicate a clear influence from popular to quality
newspapers regarding EU-related topics, but not the other way around, as previous studies do not indicate a clear direction for intermedia agenda building.
Topics matter
Finally, topics do matter. This is supported by both the effects between the agendas and the amount of explained variance in the models. Regarding the explained variance, some topics are better explained within the model than others, and only for newspaper coverage. The topics that are well-explained are the EU-Russia relationship, coverage regarding the Lisbon Treaty and coverage regarding the economic crisis. Something these topics all have in
common is that they concern rather immediate topics that, although they can be continuing for some time, always contain new and unexpected events. They are also topics on which the European Commission has acted quickly and decisively. But more importantly, all well-explained topics fit well in the endogenous category of the endogenous-exogenous topic categories distinguished by Vliegenthart and Walgrave (2008). This is because all of them concern official institutions, in the Lisbon Treaty and EU-Russia topic most prominently the EU itself, but in the economic crisis topic the EU itself and financial institutions. For these topics the European Commission also significantly Granger-causes quality newspaper coverage, and popular newspaper coverage on the economic crisis topic.
The less well-explained topics, like international trade and renewable energy are more long-term, and do not involve quick and decisive actions by the European Commission. This leads to the assumption that external factors are the main cause for news coverage on these topics. Thus, while the European Commission is able to influence in particular quality newspaper coverage, it is mainly able to do so with regards to topics that are immediate and endogenous. So the European Commission can get into the news, but only when words are combined with action, and both are related to official institutions.
The European Commission does in general not react much at all to any of the topics, with the exception of the Lisbon Treaty topic, where it does so strongly. This is most likely because this topic is more or less about the European Union, and the European Commission itself, and thus the Commission has to some extent been forced to react to this topic. The Lisbon Treaty topic also showed some interesting national-level results. Because contrary to all the other topics, the Dutch government is Granger-causing all the other agendas. Most likely this is related to the specific EU-oriented nature of the topic, which leads both the
European Commission and the Dutch media to closely monitor the Dutch government. For the European Commission this can be explained by it wanting to know the opinions of member state governments, and reacting to them, while the national media are susceptible for a similar reason, namely to report the Dutch government position on the topic to the public. This informative role of the media is also supported by the fact that both quality and popular newspaper coverage are equally Granger-caused by the Dutch government speeches.
Overall, the results discussed above lead to a complex answer to the original question of this study, namely in what way the symbolic political agenda of the European Commission, the symbolic political agenda of the Dutch government and the popular and quality Dutch newspaper agendas influence each other. For some topics, there is hardly any relationship between the agendas, and very little explained variance, leading to the conclusion that these topics are strongly influenced by external factors. For the topics where some variance is explained, mainly newspaper coverage can be adequately explained. This leads to the overall conclusion that while the European Commission agenda and the Dutch government agenda provide explanations for media coverage, the same is not true the other way around. For a visual representation of these relationships, see also figure 2 in the results section.
Finally, the results most of all show that topics matter, with topics on which the European Commission is active also additionally explaining newspaper coverage besides the existing intermedia effects and the effects of self-reinforcement. They also show that the Dutch government is in general not very involved in EU-related topics in general. But to answer the research question: The Dutch political agenda influences and is influenced only marginally by all other agendas, while the European Commission and newspaper agendas only influence each other on topics that are immediate and endogenous, and on which the European Commission is actively taking action. Or translated to more practical implications, the European Commission can get into particularly quality newspaper coverage with topics on which it actually takes action, while the Dutch government in general does not say much about EU-related topics at all. At the same time, the European Commission itself is
susceptible to both media coverage and speeches by the Dutch government when it comes to topics that mainly concern the functioning of the EU and the European Commission itself.
But perhaps the most notable finding of the current study is that one of the topics (the EU relations topic) is primarily concerned with the EU talking about itself and its member states. While this “feel-good” topic involves words like “important”, “happy”, “cooperation”,
“thanks”, “good” and “support”, its variance is hardly explained in the VAR model. This shows that talking about yourself is not the way to get media attention. And even if it would get media attention, the topic would not have any substantive meaning. In general, it shows that in its speeches, the European Commission devotes ample space to formalities, space that might be better used to discuss substantive topics.
References
Balk, E. M., Chung, M., Hadar, N., Patel, K., Yu, W. W., Trikalinos, T. A., & Chang, L. K. W. (2012). Accuracy of data extraction of non-English language trials with Google Translate.
Berganza, R. (2009). Framing the European Union and building the media agenda: The 2004 European Parliamentary elections in the Spanish daily press. Journal of Political Marketing, 8(1), 59-69.
Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python. " O'Reilly Media, Inc.".
Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. the Journal of
machine Learning research, 3, 993-1022.
Boomgaarden, H. G., Vliegenthart, R., de Vreese, C. H., & Schuck, A. R. (2010). News on the move: Exogenous events and news coverage of the European Union. Journal of European Public Policy, 17(4), 506-526.
Börzel, T. (2002). Member state responses to Europeanization. Journal of common market studies, 40, 193-214.
Boumans, J. W., & Trilling, D. (2016). Taking Stock of the Toolkit: An overview of relevant automated content analysis approaches and techniques for digital journalism scholars. Digital Journalism, 4(1), 8-23.
Boyle, T. P. (2001). Intermedia agenda setting in the 1996 presidential election. Journalism & Mass Communication Quarterly, 78(1), 26-44.
Brandenburg, H. (2002). Who follows whom? The impact of parties on media agenda formation in the 1997 British general election campaign. The Harvard International Journal of Press/Politics, 7(3), 34-54.
of Information Policy. Javnost-The Public, 12(2), 57-73.
Denham, B. E. (2010). Toward conceptual consistency in studies of agenda-building processes: A scholarly review. The Review of Communication, 10(4), 306-323.
De Vreese, C. H., Banducci, S. A., Semetko, H. A., & Boomgaarden, H. G. (2006). The news coverage of the 2004 European Parliamentary election campaign in 25 countries. European Union Politics, 7(4), 477-504.
Gleissner, M., & De Vreese, C. H. (2005). News about the EU Constitution Journalistic challenges and media portrayal of the European Union Constitution. Journalism, 6(2), 221-242.
Kaitatzi-Whitlock, S. (2007). The Missing European Public Sphere and the Absence of Imagined European Citizenship: Democratic Deficit as a Function of a Common European Media Deficit. European Societies, 9(5), 685-704.
Kiousis, S., Mitrook, M., Wu, X., & Seltzer, T. (2006). First-and second-level agenda-building and agenda-setting effects: Exploring the linkages among candidate news releases, media coverage, and public opinion during the 2002 Florida gubernatorial election. Journal of Public Relations Research, 18(3), 265-285.
Koopmans, R. (2007). Who inhabits the European public sphere? Winners and losers, supporters and opponents in Europeanised political debates. European Journal of Political Research, 46(2), 183-210.
Küntzler, T. (2015). Who influences whom? The agenda-setting power of Twitter and newspapers during the European election campaign 2014. Master's Thesis Communication Science, University of Amsterdam.
Lim, J. (2011). Intermedia agenda setting and news discourse: A strategic responses model for a competitor's breaking stories. Journalism Practice, 5(2), 227-244.
Lim, J. (2012). Competitors’ news coverage as a source within economic and symbolic power relations. First Monday, 17(11).
Machill, M., Beiler, M., & Fischer, C. (2006). Europe-Topics in Europe's Media The Debate about the European Public Sphere: A Meta-Analysis of Media Content Analyses. European Journal of Communication, 21(1), 57-88.
Maes, I., & Verdun, A. C. (2005). Small States and the Creation of EMU: Belgium and the Netherlands, Pace-setters and Gate-keepers. Journal of Common Market Studies, 43(2), 327-348.
Meyer, C. O. (2009). Does European Union politics become mediatized? The case of the European Commission. Journal of European Public Policy, 16(7), 1047-1064. Mimno, D., Wallach, H. M., Talley, E., Leenders, M., & McCallum, A. (2011). Optimizing
semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 262-272). Association for
Computational Linguistics.
Parmelee, J. H. (2013). The agenda-building function of political tweets. New Media & Society, 1461444813487955.
Princen, S. (2007). Agenda-setting in the European Union: a theoretical exploration and agenda for research. Journal of European Public Policy, 14(1), 21-38.
Rehurek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. Schlesinger, P. (1999). Changing spaces of political communication: The case of the European
Union. Political communication, 16(3), 263-279.
Soroka, S. N. (2002). Issue Attributes and Agenda‐Setting by Media, the Public, and Policymakers in Canada. International Journal of Public Opinion Research, 14(3), 264-285.
Statham, P. (2008). Making Europe news How journalists view their role and media performance. Journalism, 9(4), 398-422.
Stevens, K., Kegelmeyer, P., Andrzejewski, D., & Buttler, D. (2012). Exploring topic coherence over many models and many topics. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and
Computational Natural Language Learning (pp. 952-961). Association for Computational Linguistics.
Trenz, H. J. (2004). Media coverage on european governance exploring the european public sphere in national quality newspapers. European Journal of Communication, 19(3), 291-319.
Van Aelst, P., & Vliegenthart, R. (2014). Studying the tango: An analysis of parliamentary questions and press coverage in the Netherlands. Journalism Studies, 15(4), 392-410. Van Aelst, P., & Walgrave, S. (2011). Minimal or massive? The political agenda–setting
power of the mass media according to different methods. The International Journal of Press/Politics, 1940161211406727.
