Citizen science for public policies
Exploring the integration of citizen science in the National Institute for Public Health and the Environment
Author Michelle Oldenburger
Student number 10440550
Project Master thesis political science
Specialisation Contested knowledge and alternative facts. Reason and power in politics and governance
Supervisor Mw. Dr. A.M.C. Loeber Second reader Dhr. Prof. Dr. J. Grin
Table of contents
1. Introduction
1.1 Citizen science: a participatory approach of knowledge production p.4 1.2 Citizen science in public research institutes p.4 1.3 Citizen science in the National Institute for Public health and the Environment p.6
1.4 Research question p.7
1.5 Research relevance p.9
1.6 Reader’s guide p.10
2. Theoretical framework
2.1 Working towards a theoretical framework p.11 2.2 Definitions and typologies p.12
2.2.1 Citizen participation as a ladder p.14 2.3 Understanding the rise of citizen science p.15 2.3.1 From mode 1 to mode 2 science p.16 2.3.2 The ‘death’ of expertise p.17
2.4 Promises p.19 2.5 Pitfalls p.21
2.6 Citizen science innovations p.22 2.7 Theoretical framework p. 23 3. Methodology
3.1 Research design p.25 3.2 Research methods p. 26 4. Analysis
4.1 Exploring RIVMs citizen science motivations p. 31 4.1.1 Tackling complex issues together p. 31 4.1.2 Abandoning the ‘ivory tower’ p.34 4.1.3 Technical advances p.35
4.1.4 Conclusion p.36 4.2 Citizen science in practice p. 37
4.2.1 Citizen science as a two-way street p.37 4.2.2 Conclusion p.40
4.3 The challenges of integrating citizen science in standing procedures of knowledge production p.41
4.3.1 Changing customs p.41
4.3.2 Calibrating research methods p. 43 4.3.3 Communication p.43
4.3.4 Conclusion p.44
4.4 Recommendations for the future p.45
4.4.1 Stimulating scientists to experiment p.45 4.4.2 Modern communication p.46
4.4.3 Conclusion p.47 5. Conclusion
1. Introduction
1.1 Citizen science: a participatory approach of knowledge production Over the last two decades there has been a burgeoning interest in the concept of citizen science amongst academics and practitioners (Irwin, 2001; Stilgoe, 2009; Prainsack, 2014; Lewenstein, 2016; Den Broeder, 2017a; Eitzel et al., 2017; Hecker et al. 2018). Citizen science has been defined as ‘’a form of research collaboration involving members of the public in scientific research projects to address real-world problems’’ (Wiggins and Crowston, 2011: 1). Since the start of the twenty-first century, the amount of research projects that rely on citizen scientists has increased significantly (Cohn, 2008; Bonney et al., 2014). Citizen science differs from other types of public participation as citizens are actively engaged in scientific work instead of having a more passive role, like participating as a subject in a research study (Cohn, 2008: 3). Citizen scientists may contribute to research projects in various ways. They may for example assist scientists with the collection or analysis of data, as happened in the ‘iSPEX’ project where European citizens measured concentrations of fine dust in their neighbourhood through their smartphones (iSPEX.nl, 2015). Moreover, citizens can also help scientists to develop research questions that respond to public concerns and questions. The ‘Dutch National Research Agenda’ is one example where citizens were stimulated to share research topics that they deem relevant for the development of Dutch society (Wetenschapsagenda.nl, 2016). According to the official Dutch National Research Agenda website (ibid.) these questions ‘’reveal the complexity of the issues challenging Dutch society today, and provide a glimpse into the areas where Dutch scientific research plans to focus on in the coming years’’. Other examples of citizen science projects that have received international attention are ‘Galaxy Zoo’, a university-led project where citizens help to identify distance galaxies, and ‘PatientsLikeMe’, an online health platform created by people who suffer from rare diseases.
1.2 Citizen science in public research institutes
Anno 2018, several public research institutes have also expanded their knowledge production processes with different forms of citizen science, which
demonstrates the growing acknowledgment of public inclusion as a legitimate method to conduct research for public policies. However, this development is quite remarkable regarding the fact that public research institutes have long operated autonomously to safeguard the quality of their scientific data (RIVM, 2018a). Public research institutes have to comply with strict criteria to ensure outstanding research quality. These criteria entail for example accuracy, independency and verifiability (ibid.). Non-compliance with these criteria has resulted in fierce criticism on research reports in the past and has caused societies to question data produced by public research institutes (Soff, 2016; Guido, 2017; Seinen, 2017). Therefore, the admittance of non-professionally trained citizens in scientific research procedures could be seen as a radical decision.
Perhaps this development might be understood in the light of shifting scientific values. From the mid 20th century onwards, modernistic forms of knowledge production, which Nowotny et al. (2003: 186) refer to as ‘mode 1 science’, are being replaced by reflexive forms of knowledge production, which have been labelled as ‘mode 2 science’ (ibid.). Whereas mode 1 science is described as academic and discipline based, mode 2 science is contextual and transdisciplinary (Nowotny et al., 2003: 187). Citizen science can be seen as an example of mode 2 science because stakeholders from different public en private domains collaborate to find answers to real-world challenges. The exponential growth of citizen science projects over the last two decades suggests the rise of mode 2 science at the expense of the formerly acclaimed mode 1 science.
Nonetheless, the emergence of citizen science has also raised fundamental questions. Critics fear that the involvement of non-professionally trained citizens in research projects may have harmful consequences for the quality of scientific data and the independency of public research institutes (Stilgoe, 2009; Kosmala et al., 2016). Hence, the question arises why any renowned scientific institute would voluntarily choose to involve citizens in their research procedures regarding the potential pitfalls of citizen science that may interfere with research quality standards.
1.3 Citizen science in the National Institute for Public health and the Environment
In order to understand what drives public research institutes to invest in citizen science as an innovative strategy to conduct research for public policies, this thesis will analyse one institute that appears to have made the deliberate decision to integrate citizen science in their standing procedures of knowledge production. This organisation is the National Institute for Public Health and the Environment (RIVM), situated in the Netherlands. RIVM is concerned with the promotion of public health, consumer safety and environmental quality. RIVM conducts research on these topics and subsequently produces advice for policy-makers, researchers, regulatory authorities and the general public (RIVM, 2018b). As such, RIVM occupies a unique position, operating in the middle of a society-policy-science triangle (figure 1). RIVM is regarded as a renowned public research institute on a national and international scale. According to the official RIVM website (RIVM, 2018a), the institutes owes it respected position to the scientific quality of research projects and the engagement of RIVM scientists. RIVM ensures its reputable position through a supervisory board that verifies the scientific quality of all RIVM research projects (ibid.). RIVMs supervisory board evaluates scientific research on the basis of five objective criteria, namely accuracy, reliability, verifiability, impartiality and independency. These criteria have been documented in RIVMs science code of conduct (RIVM, 2007).
The deliberate decision to invest in citizen science is illustrated by the inclusion of this research method in RIVMs 2015-2018 strategic programme (Strategischprogrammarivm.nl, 2018). RIVMs strategic programme supports the institute to answer tomorrow’s questions today. RIVM encourages its employees to explore topics that are not yet on the political or public agenda, but are likely to become influential within the near future (ibid.). Early research on these topics helps RIVM departments to anticipate on future trends and questions. Honoured research proposals are bundled and together constitute RIVMs strategic programme, which is released every four years. The inclusion of citizen science in RIVMs strategic programme has stimulated scientists to study (Den Broeder et al., 2016; Den Broeder, 2017a) and experiment with this research strategy (Samenmetenaanluchtkwaliteit.nl, 2018a).
Figure 1: RIVM in the middle of a society-policy-science triangle
Source: The Royal Netherlands Meteorological Institute, 2018. 1.4 Research question
Summarising the first half of the introduction, we found that RIVM, a renowned public research institute that strives to produce excellent scientific research, has recently chosen to explore citizen science as a novel method to conduct research. This thesis aims to get a better understanding of the reasons why RIVM chooses to cooperate with non-professionally trained citizens to develop knowledge for public policies despite the potential pitfalls of this participatory strategy, which could harm RIVMs scientific reputation. Closely related to the question why
Society
Policy Science
In consultation with and for the benefit of
society
Enhancing the efficiency and implementation of
policies
In coherence with national and European
research programmes Connecting with universities founded in science Multidisciplinary research approach
RIVM invests in citizen science is the questions how the organisation exercises citizen science in practice. Critics ask themselves whether citizens are merely used by scientists to collect more data or whether they are also granted the opportunity to develop and adapt research plans according to their needs (Iriwn, 2001; Haklay, 2012; Shirk et al., 2012). The why and how questions correlate, as it is likely that the motivational factors will dictate what citizen science looks like in practice. This thesis will therefore proceed to examine how RIVM scientists seek to integrate citizen science in their standing procedures of knowledge production. During the research process of this thesis, which initially focussed solely on the questions why and how RIVM seeks to integrate citizen science, interviews with RIVM employees revealed that asking these questions resulted in anecdotes about the difficulties of citizen-scientist partnerships. RIVM employees explained that even though citizen science has been included in RIVMs 2015-2018 strategic programme, they are struggling to implement citizen science in current practices of knowledge production. Perhaps this is not much of a surprise as RIVM has only recently started to experiment with the incorporation of citizen science, meaning that one could expect a trial and error process. Nonetheless, exploring this process and revealing the hurdles that employees experience when trying to integrate citizen science is a relevant endeavour as it helps to think about future recommendations. These recommendations could help the institute to enhance citizen science in the future. Or, in the words of RIVM scientist Den Broeder (2013), further research might help RIVM to realize the promises of citizen science. Furthermore, These insights might not only be useful for RIVM, but may also support other organisations that have invested in or want to experiment with citizen science. Considering the research objectives that have just been discussed, this thesis will concentrate on the following question:
For which reasons and how does RIVM seek to integrate citizen science in their standing procedures of knowledge production and what practical lessons can be learned from RIVMs citizen science trajectory?
The main research question will be explored through the following four sub-questions. First, why does RIVM want to expand their standing research procedures with citizen science? In other words, what is the added value of using non-professionally trained citizens as a source of expertise? Second, what does citizen science look like in practice? This question serves to understand how RIVM has collaborated with citizen scientists in different research projects. Furthermore, this thesis will examine the practical challenges that RIVM employees run into when carrying out citizen science projects. This leads to the third sub-question: What are the challenges of integrating citizen science in RIVMs standing knowledge production procedures according to RIVM insiders? Finally, the information retrieved from the third sub-question will be used to discuss how current challenges can be dealt with in order to contribute to the incorporation of citizen science in the future. The fourth and final sub-question will therefore be: which practical lessons can be drawn from the observed challenges?
1.5 Research relevance
Elaborating on the questions that have just been discussed is deemed relevant because citizen science is a relatively young phenomenon of which many aspects still require further exploration. In the past years, a significant amount of literature has been devoted to explaining what citizen science is (Irwin, 1995; Prainsack, 2014; Lewenstein, 2016; Eitzel et al., 2017). The potential advantages and challenges of using citizen science to produce knowledge have also been discussed in a number of articles (Rosner, 2013; Stilgoe, 2016; Den Broeder, 2017a). However, motivational questions regarding the use of citizen science have largely been left unanswered. This thesis aims to contribute to the growing insights on citizen science by critically analysing why public research institutes would choose to incorporate citizen science in their standing research procedures and how they have formalised collaborations with citizens.
Furthermore, the amount of research on the integration of citizen science into existing processes of knowledge production is limited. Little is known about the integration of citizen science into the existing science-policy nexus. The question remains how citizen science can be integrated into standing knowledge
production processes for policy arrangements in such a way that it helps citizens to answer questions effectively and, at the same time, contributes to the quality of data produced by public research institutes. According to RIVM scientist Den Broeder (2013), who is actively involved in citizen science projects in the field on public health, ‘’the engagement of citizens in public health research certainly holds a promise, both for knowledge development and as a contribution to health promotion. However, there is still much to be done to realise this potential.’’ She mentions the need for evaluation and new procedures to enhance citizen science in the future (ibid.). Hecker et al. (2018: 4) affirm this notion and state that ‘’inter- and transdisciplinary discussions and critical analyses are needed to use the current momentum to evaluate, demonstrate, and build on the advances that have been made in the past few years’’. This thesis strives to provide public research institutes with new insights and tools to further develop this research strategy by inquiring into the current hurdles and exploring potential solutions.
1.6 Reader’s guide
Before proceeding to the literature review the structure of this thesis will be discussed. In chapter two several theories regarding citizen science will be discussed. The chosen theories will guide the analysis of this rapport as they help to interpret research data and serve as a basis to think about future recommendations for the development of citizen science. Based on the literature review, a theoretical framework has been developed consisting of a number of sensitizing concepts that will be used to structure the analysis. The third chapter will elaborate on the research design of this thesis as well as the methods that have been used to collect data. Furthermore, the decision to focus on RIVMs citizen science trajectory will be explained. Next, the analysis unfolds in chapter four, which sets out to answers the above-mentioned sub-questions. After the discussion of research results, chapter five proceeds to present the conclusion of this thesis. This conclusion summarises the main findings of this rapports and compares these to previous theoretical findings.
2. Literature review
2.1 Working towards a theoretical framework
Considering the main research question, which has been formulated in the previous chapter, the following literature review will discuss a number of citizen science theories that shed light on different aspect of the research question. First, several definitions and typologies of citizen science will be discussed, because they provide an overview of the potential purposes and applications of citizen science. As such, these theories may give an idea of the reasons why RIVM has deliberately chosen to incorporate citizen science and how the organisations may collaborate with citizens in practice. Next, the growing popularity of citizen science will be analysed. The previous discussed change from mode 1 to mode 2 science will be elaborated on in more detail and a number of societal developments that have potentially given participatory sciences a boost will also be reviewed. Furthermore, the advantages of citizen science will be discussed. Again, these theories may help to understand RIVMs motivations to invest in citizen science. The last two sections of the literature review are devoted to describing potential pitfalls of citizen science as well as the innovations that academics have proposed in order to enhance citizen science in the future. Describing the perceived disadvantages of citizen science is relevant, because these insights might provide clues about the challenges that RIVM employees run into when trying to get citizen science accepted as a valid research strategy. Finally, exploring citizen science innovations helps to think about future recommendations for RIVM to enhance the use of citizen science.
The discussion of these theories will result in a theoretical framework that will be presented at the end of the literature review. This framework provides a systematic overview of the insights that have been distilled from the literature review. These insights, or sensitizing concepts, will guide the analysis as they suggest directions along which to look for information and subsequently provide guidance in approaching empirical data (Blumer, 1954: 7).
2.2 Definitions and typologies
As mentioned earlier, this literature review will start off with an overview of different citizen science definitions and typologies. The presented overview does not aim to be all-encompassing, because describing all different definitions and typologies of citizen science would take up an entire thesis (for an extensive overview of citizen science terminologies see Eitzel et al., 2017). Instead, this review sets out to give an insight in the variety of definitions and typologies and the dynamic nature of citizen science.
Some citizen science definitions are rather specific and focus on one activity or stage of the scientific research process. For example, Bonney et al. (2009: 977) refer to citizen science as an activity where citizens solely assist scientist by collecting data. According to them ‘’citizen science enlists the public in collecting large quantities of data across an array of habitats and locations over long spans of time.’’ In 2013, the European Commission applied a more generic definition and described citizen science as ‘’the general public engagement in scientific research activities when citizens actively contribute to science either with their intellectual efforts or surrounding knowledge or with their tools and resources’’. This definition leaves room for citizens to be involved in numerous other aspects of the research cycle as well. It suggests that citizens may for example also be involved in the research development stage in which research questions and methods are developed. The varieties of those two definitions suggest that citizen science is a dynamic concept that has changed over time in depth and scope. Riesch, (2014: 631) explains that as the concept has traveled across different disciplines, institutions and countries, the ideals and goals have shifted and now produce a large variety of citizen science projects that are loosely connected by an emphasis on combining participatory research with public engagement. Eitzel et al. (2017: 5) confirm this idea by stating that the activities that fall under the guise of citizen science are nuanced and in a state of flux. However, ‘’it is generally agreed that citizen science refers to the inclusion of members of the public in some aspect of scientific research’’ (ibid.).
Just as there are many different definitions of citizen science, the amounts of citizen science classifications are also plenty. Wiggings and Crowston (2011: 2) have classified citizen science projects according to their aim. They
distinguish between five different types of projects. In ‘action’ projects, citizens and scientists work together to address local issues. ‘Conservation’ projects focus on managing natural resources. ‘Investigation’ projects are set up to answer scientific questions. In ‘virtual’ projects, activities are carried out remotely. ‘Education’ projects aim at improving citizens’ knowledge.
Shirk et al. (2012: 4) distinguish between the degree of participation, which they define as ‘’the extent to which individuals are involved in the process of scientific research’’ and the quality of participation, or ‘’the extent to which a project’s goals and activities align with, respond to, and are relevant to the needs and interests of public participants’’. Shirk and his colleagues have used these two elements to produce a framework for deliberate designs, consisting out of five dimensions. This framework (table 1) describes how public participants interact with scientist through public participation in scientific research. Their framework follows a pattern that King et al (2016: 4) refer to as ‘’for the people, with the people and by the people’’.
Table 1: Framework for deliberate design
Public action Members of the public…
Contract Ask scientists to conduct a scientific
investigation and report on results
Contribute Are asked by scientists to collect and
contribute data and/or samples
Collaborate Assist scientists in developing a study and
collecting and analysing data for shared research goals
Co-create Develop a study and work with input from
scientists to address a question of interest or an issue of concern
Colleagues Independently conduct research that advances
Furhtermore, Haklay (2012: 13) classifies citizen science projects by levels of volunteer engagement, which resembles the ‘degree of participation’ as discussed by Shirk et al. As illustrated in table 2, Haklay (ibid.) distinguishes between four levels of volunteer engagement.
Table 2: Volunteer engagement levels
Level Volunteers…
1. Crowd sourcing 2. Distributed intelligence
Serve as sensors or provide computing power Learn basic skills before collecting or
interpreting data 3. Participatory science
4.Extreme citizen science
Co-decide about research questions and the types of data that have to be collected Collaborate with scientist on all aspects of scientific research
2.2.1 Citizen participation as a ladder
Considering the above-mentioned typologies, there is one schematic feature that stands out. Shirk et al. (2012) as well as Haklay (2012) both use a classification method that resembles Arnstein’s ladder of citizen participation (figure 2), which she introduced in 1969. This hierarchical scheme illustrates eight stages of citizen participation, starting with non-participation at the bottom of the ladder, moving up through various stages of tokenism and finally arriving at several steps that reflect extensive citizen power.
Besides Shirk et al. and Haklay, other academics (Roy et al., 2012; King et al., 2016) have also followed Arnstein’s example and applied similar vertical schemes to illustrate various degrees of citizen participation. However, Lawrence (2006: 282) suggests that the normative assumptions of these typologies (e.g., that more power is transformative and less power is exploitative) do not necessarily reflect individuals’ experiences. Saldivar-Tanaka and Krasny (2004) as well as Cornwall (2008) argue that it should not be
assumed that individuals or communities will automatically benefit from a greater degree of control over a given research process or agenda
Figure 2: Arnstein’s ladder of citizen participation
Source: Arnstein, 1969: 217.
2.3 Understanding the rise of citizen science
The previous paragraphs have revealed that the dissemination of citizen science has produced a large variety of definitions and typologies. But what has caused the spread of citizen science? This literature review will proceed to discuss a number of developments that may have fostered the use of citizen science as an innovative strategy to produce knowledge.
First, it is important to note that citizen science is not entirely new. Although Lewenstein (2016: 1) refers to citizen science as ‘’one of the most dramatic developments in science communications in the last generation’’, the involvement of citizens in science is a phenomenon that dates back before the start of the twentieth century. At that time so-called ‘gentleman scientists’, such as Antoni van Leeuwenhoek and Benjamin Franklin, contributed to novel
scientific discoveries and inventions (Silvertown, 2009). From the twentieth century onwards, the nature of scientific research changed as scientifically trained researchers working on behalf of universities and governmental institutions started to dominate the production of knowledge (Prainsack, 2014: 148). Conducting research became the prime domain of professionally trained experts. Consequently, science became an institutionalised affair. Recently, however, this practice has started to change. As a result of digital media, open-source databases, the Internet and portable sensors, non-professionally trained citizens have gained new opportunities to participate in scientific activities (ibid.). Even though some of the largest citizen science projects, such as the Audubon Society Christmas Bird Count, have been operating for several decades, the number of projects has grown dramatically over the past decade due to technical revolutions (Bonney et al., 2014). Besides technical advances, a number of socio-cultural developments have also been related to the rise of citizen science. As a result, non-professionally trained citizens have found their way back into scientific procedures. The participation of lay people in scientific research projects could therefore be seen as a revival of citizen science instead of a completely new phenomenon. The following paragraphs will proceed to elaborate on a number of technical and societal developments that may have fostered the return of citizen science.
2.3.1 From mode 1 to mode 2 science
First of all, citizen science can be understood as an example of what Nowotny and her colleagues (2003: 186) have labelled ‘mode 2 science’. Mode 2 science refers to a new way of knowledge production that began to emerge in the mid-twentieth century and differentiates from traditional mode 1 science. Mode 1 science is characterized as academic, investigator-initiated and discipline-based (ibid.). Mode 1 research results from scientific questions and does not regard the social applicability of its finding. This way of producing knowledge was common practice in the nineteenth and twentieth century, when scientific experts dominated the production of knowledge. On the contrary, mode 2 science is defined as contextual, meaning 'research carried out in a context of application, arising from the very work of problem solving and not governed by the
paradigms of traditional disciplines of knowledge" (Gibbons et al., 1996: 15). Moreover, mode 2 science is trans-disciplinary as researchers are not necessarily derived from pre-existing disciplines (Nowotny et al., 2003: 187). Instead, knowledge can be derived from the expertise of different individuals or groups of people. This has allowed many new types of knowledge organisations, such as think tanks and activist groups, to join the ‘research game’. Subsequently, mode 2 science differs from mode 1 science as knowledge may be produced in a great variety of places, meaning research can be done in places other than laboratories (ibid.). Furthermore, mode 2 science is described as highly reflexive as ‘’it has become a dialogic process, an intense (and perhaps endless) ‘conversation’ between research actors an research subjects’’ (Nowotny et al., 2003: 187). In recent years, the notion of mode 2 science as best practice has spread quickly (Floud et al., 2013). Procedures of autonomous scientific research are being replaced with a ‘’science in society’’ approach (Felt et al., 2013: 8). Citizen science can be seen as one practical approach to engaging citizen in the production of scientific knowledge.
2.3.2 The ‘death’ of expertise
Second, the decreasing authority of experts may provide a valuable clue as to why public research institutes have recently embraced the notion of active public participation. The prevalence of expert-based knowledge has recently been publicly contested (Hoppe, 2004; Polletta, 2006; Mance, 2016). According to the founders of the Technobility Webinar Series, experts are currently suffering from a debilitating handicap as ‘’they're nearly always right (in their field of study) and they know it - but no one else really cares’’ (De Jager & Co and Interthink Consulting, 2014).
A number of developments might help to explain this phenomenon. First, recent globalisation processes and technologic revolutions have made contemporary problems increasingly more complex (Abbott and Snidal, 2009; Hale and Held, 2011). The complexity of novel issues challenges the capacity of even highly trained experts to deal with them effectively. Experts are struggling to develop advice for action on wicked problems such as climate change or health inequalities (Den Broeder, 2017a: 11). Due to scientific uncertainties as
well as dissensus amongst experts, traditional evidence-based methods of producing knowledge are failing to deliver. Furthermore, citizens’ trust in scientific rationality has decayed after two World Wars and recent ecological disasters (Hoppe, 2005: 766).
Second, people are now exposed to more information than ever before due to technologic revolutions, such as the Internet, and increasing access to higher levels of education (Nichols, 2017). These revolutions have caused citizens to become more critical and outspoken towards knowledge-claims made by experts (Hoppe, 2004: 768). The rise of Internet has provided citizens with the means to conduct their own investigations, which may lead them to reject theories produced by experts (ibid.) In his book ‘The Death of Expertise’, Nichols argues that ‘’these societal gains, however, have also helped fuel a surge in narcissistic and misguided intellectual egalitarianism that has crippled informed debates on any number of issues’’ (OUP.com, 2017). The increasing availability of information has caused people to become more critical towards solutions created by experts. Especially the rise of the Internet has given people the opportunity to collect information about almost everything. Nichols states that this has caused average citizens to believe that they are on an equal intellectual footing with doctors and diplomats with only a quick trip through Wikipedia (ibid.).
Third, the traditional prevalence of a technical discourse in research has caused public frustration over a ‘democratic deficit’. The act of developing public policies without consulting the communities that will be affected by these policies is being rejected (Hoppe, 2004; Polletta, 2006). Hoppe (2004: 766) explains that ‘putting social ideas in scientific and rational arguments is more often seen as exclusive, suppressive, technocratic and undemocratic’’. Instead, concepts such as reflexivity and discourses are being praised (ibid). Moreover, the continuous fragmentation and individualisation of society has made citizens question the rational and universal explanations of social phenomena (Hoppe, 2004: 768). Building on this argument, Polletta (2006: 2) states that ‘’grand narratives of progress, faith and rationality that once held the status of common sense have lost force and been replaced by a babble of competing moral values
and authorities. We now trust local stories that give insight, but do not claim to tell the truth’’.
As a result of the developments mentioned above, the authority of expert-based knowledge has decreased significantly over the past decades. Recent controversies such those surrounding the HPV vaccination and Brexit demonstrate the feeble position of experts. Consequently, experts that have long worked independently on the production of knowledge for policies now find themselves in the midst of conflict. Therefore, experts operating in different domains grasp the need to re-evaluate the way in which knowledge is produced and science and society are geared to one another (Irwin, 2001: 3). According to Heiss en Matthes (2017: 22) scientific actors must actively involve societies in the process of knowledge production in order to ensure the external legitimacy of institutionalized scientific research. Citizen science is seen as a potential solution to bridge the gap between experts and citizens and to produce knowledge that is more socially robust (Den Broeder, 2017b).
2.4 Promises
The previous paragraphs have illustrated that citizen science made a comeback that has like been supported by a changing scientific landscape in which investigator-based and academic-driven research is condemned while collaborations and contextual knowledge are highly valued. Citizen science foresees in the need for collaborative and contextual research in a number of ways. First, citizen science contributes to better knowledge as lay people can complement scientific data with their local or traditional knowhow (Irwin, 1995). Furthermore, collaborations generate understanding of and trust in research procedures, which allow public research institutes to produce knowledge that is accepted by the general public (Nowotny et al., 2001). Similarly, collaborating with citizens allows scientists to develop more effective policies as the inclusion of lay people helps researchers to understand the interests of communities, which allows them to produce knowledge that responds to real-world problems (Den Broeder, 2017b). For example, the previously discussed citizen science project ‘The Dutch National Research Agenda’ helps researchers to understand which topics citizens are interested in
and require further examination. Furthermore, citizen science increases research capacity. Increased research capacity refers to the need for larger quantities of data, which scientist may for example accomplish by collaborating with other parties (Rosner, 2013: 71). Projects that are relatively labour-intensive are therefore ideally suited for a citizen science strategy. (ibid.). Finally, citizen science yields benefits for citizens in a number of ways. Den Broeder (2017: 35) created a list of ten main participant benefits. These benefits can be categorized into three groups as illustrated by table 3. Table 3 demonstrates that collaborations between citizens and scientists help citizens to better understand scientific procedures. Also, individuals and communities are empowered by the act of participating in research projects, for example because individual members of a community work together to gather scientific data, which stimulates community building. Finally, citizen science helps to diminish the perceived distance between scientists and citizens as collaborations may demonstrate that conducting research is not necessarily reserved solely for trained experts.
Table 3: categories of citizen science participant benefits
Benefit category Examples
1. Understanding of scientific processes Enhanced scientific knowledge, improved access to science information, increases in scientific thinking, improved ability to interpret scientific information
2. Empowering participants Strengthened connections between people, nature and place, improved participants self-efficacy, increased community-building, social learning and trust, citizen science learn how to take actions to influence research policies 3. Bridging the expert-citizen gap Corrected interpretations of science as too
complex, changes in attitudes, norms and values towards science, citizen scientists gain access to broader networks
2.5 Pitfalls
Although previous information has demonstrated that citizen science is celebrated by several academics such as Rosner (2013) and Den Broeder (2017a), the advantages of citizen science have not been undisputed. Whereas protagonists mention better knowledge as an advantage, critics argue that the involvement of lay people may harm the quality of research data (Stilgoe, 2016: Kosmala et al., 2016). Stilgoe (2016: 1) mentions that ‘’behind the scenes fears remain that giving citizens the key to the ivory tower is a recipe for bad science.’’ Scientists are often sceptical of the ability of non-professionally trained volunteers to produce accurate data (Kosmala et al., 2016). Moreover, critical scientists fear that the opinions and interests of citizens could interfere with scientific evidence (Weeda, 2018). Recent frictions in the Dutch Health Council board illustrate this fear (NRC.nl, 2018). In March, 2018, the Dutch Health Council decided to collaborate with patient representatives to get a better understanding of the chronic fatigue syndrome: an illness that is surrounded by many uncertainties. After the research report had been published, one scientist decided to resign from the council’s board, because in his opinion the council’s report had become a compromise between the opinions of patients and the results of scientifically conducted research. The scientific quality of the results could therefore no longer be trusted. According to the resigned scientist, the Health Council should refrain from making such compromises in order to satisfy patients, because in the end patients are more likely to benefit from accurate, reliable findings.
Finally, some critics doubt the potency of citizen science. Public research institutes who have been criticised for their research procedures may for example use citizen science as a greenwashing technique (Irwin, 2001: 16). Institutes might want to boost their public reputation by saying that they invest in partnerships with the public, while in essence research procedures are left unaltered. Moreover, the typologies provided by Shirk et al. (2012) and Haklay (2012) suggest that in many types of citizen science projects, citizens are merely used by knowledge producing organisations to gather more data. This is for example the case when citizens contribute to research projects via ‘crowdsourcing’, which means citizens are used as sensors or provide computing
power (Haklay, 2012: 13). If this is the case, part of the reported benefits of citizen science for participants (see table 3), such as the improved ability to analysis scientific, could be frustrated as citizens are not involved deeply enough to develop such skills.
2.6 Innovations in citizen science
This last section of the literature review will elaborate on innovations in the field of citizen science, because this thesis also aims to provide several recommendations to enhance citizen science within RIVM in the future. The innovations that other academics have discussed might be useful for the improvement of RIVMs citizen science trajectory.
Hecker et al. (2018: 6) have created a list of seven recommendations for the development of citizen science in the future. These recommendations have been divided intro three subfields, namely science, policy and society. The science domain entails three recommendations, namely demonstrating the scientific benefits of citizen science, branching out across disciplines, and fostering active networking and new formats for collaboration. In the policy field, the opportunities for policy monitoring and development should be actively embraced and the funding of citizen science should align with other scientific approaches. Hecker et al. (ibid.) conclude their list with recommendations in regard to society. First they mention the need for scientists to engage with society in various participatory formats, ranging from contributory to co-creative projects. Second, excellent citizen science communication should be fostered in order to motivate citizens to join collaborative research projects. In regard to this final recommendation, Hecker et al. (2018: 9) came forth with another set of advices, focussed specifically on communication in citizen science projects. First, they argue that modern forms of communication, such as social media networks and platforms, as well as specific communication devices like websites and mobile apps should be used to attract wider participation. Nonetheless, traditional mass media should not be neglected as newspapers and television still reaches a relatively large and diverse audience. Second, active communication is crucial for the success of citizen science projects and needs to be developed at the outset. Face-to-face meetings
are seen as an important element of good communication and provide the opportunity to jointly celebrate success. Finally, scientist should incorporate participant feedback moments throughout the project to ensure the continuous improvement of data.
2.7 Theoretical framework
The inquiry into existing citizen science theories has produced interesting clues as to why RIVM might seek to integrate citizen science in standing procedures of knowledge production. These clues include, amongst other things, the decreasing authority of experts and the changing nature of scientific research. Next, the discussed typologies help to understand how the institute might formalize citizen science. Furthermore, the pitfalls of citizen science provide insights into the potential challenges that RIVM employees might experience when trying to incorporate citizen science in research projects. The fear of declining research quality as a result of non-professionally trained citizens’ incapability to conduct scientific research could hinder a successful incorporation of citizen science within RIVM. Finally, the innovations as proposed by Hecker et al. (2018) form a solid basis to think about future recommendations for the enhancement of citizen science within RIVM.
Building on these findings, a theoretical framework (table 4) has been developed which presents the four sub-questions of this thesis followed by relevant concepts that may partially provide an answer to these questions. The concepts are indicated as sensitizing concepts, which provide directions along which to look for information during the analysis of this thesis.
Table 4: theoretical framework
Sub-questions Sensitizing concepts
Why does RIVM want to expand their standing
research procedures with citizen science? 1. Technical advances
2. Changing scientific landscape - From mode 1 to mode science 3. Decreasing authority of experts
- Increasing complexity of issues - Technological empowerment of
citizens
- Democratic deficit of technocratic discourse
4. Possibility to produce better knowledge
5. Increasing efficiency of policies 6. Increasing legitimacy of RIVMs
research methods
7. Increasing research capacity What does citizen science look like in practice? 1. Hierarchic levels of citizen
participation from which organisations get to choose
2. Goal of research project determines collaboration with citizen scientists What are the challenges of integrating citizen
science in RIVMs standing knowledge production procedures according to RIVM insiders?
1. Decreasing quality of research 2. Decreasing independency
- Avoiding conflict of interest 3. Refraining from sloganizing Which practical lessons can be drawn from the
observed challenges? 1. Demonstrating scientific benefits of citizen science 2. Branching out across different
disciplines
3. Fostering active networking and new formats for collaboration
4. Fostering policy monitoring and development
5. Aligning citizen science funding with other scientific approaches
6. Collaborating in various formats 7. Investing in excellent communication
3. Methodology
Before proceeding to the analysis, this chapter will elaborate on the research design and explains why RIVMs citizen science trajectory will be studied. The second half of the methodology discusses the methods that will be used to collect data for the analysis.
3.1 Research design
In order to gain preliminary insights in each of the sub-questions, this thesis started off with an exploratory research approach. According to Winston (2018: 1) exploratory research starts with a general question and uses preliminary research, such as a focus group or literature review, as a tool to identify sensitizing concepts that could be used as a focus for future research. In the introductory chapter, this research raised the general question why any renowned scientific institute would voluntarily choose to involve non-professionally trained citizens in their research procedures, regarding the potential pitfalls of citizen science that may interfere with the strict quality criteria that research projects have to comply with. Thereafter, a literature review has provided several clues as to why public research institutes would want to integrate citizen science in their standing procedures of knowledge production for public policies and how they might do so. Likewise, the literature review has provided insights in the potential challenges of integrating citizen science in modernistic research procedure and elaborated on future recommendations to incorporate citizen science successfully. These clues and insights – or sensitizing concepts - have been summarized in a theoretical framework (table 4).
Now that the sensitizing concepts have been found, this thesis will proceed to conduct an in-depth case study to explore whether these leads indeed provide answers to the questions. A case study design has been chosen because thorough examination of a case is not only helpful to verify existing expectations, it may also provide detailed new insights that may complement the current knowledge on citizen science (Flyvbjerg: 2006: 13). In this research, the in-depth case study exists of a single case, namely RIVM, which is an example of public research institutes within the Dutch agglomerate of knowledge producing
organisations. Although many other public research institutes exist in the Netherlands, of which some have also started to experiment with citizen science (see Netherlands Environmental Assessment Agency or The Royal Netherlands Meteorological Institute), RIVMs citizen science trajectory has been documented most thoroughly. RIVM employees have for example conducted research on citizen science, which resulted in a thesis (Den Broeder, 2017a) as well as a number of articles on citizen science (Den Broeder, 2013; Den Broeder et al., 2016; Den Broeder, 2017b). Moreover, RIVM created an online platform on their official website where citizens can gather information on the meaning, implementation and benefits of citizen science (Samenmetenaanluchtkwaliteit.nl, 2018a). Here they can also find an overview of RIVMs citizen science projects. Lastly, RIVM has developed a YouTube channel titled ‘Samen Milieu Meten’ (in English: ‘Measuring our Environment Together’) where employees post video’s of citizen science projects. Judging from these initiatives, and a lack of similar initiatives in other Dutch public research institutes, RIVM can be seen as a frontrunner in the field of citizen science within the Dutch knowledge agglomerate. Therefore, RIVM can be referred to as a paradigmatic case. According to Flyvbjerg (2006: 16) a paradigmatic case ‘’operates as a reference point and may function as a focus for the founding of schools of thought’’. RIVMs rich and detailed records of citizen science may help to develop insights that could support other public research institutes which have started to experiment with citizen science or want to do so in the future. The challenges that RIVM employees experience when trying to integrate citizen science in standing procedures of knowledge production and the recommendations that flow from them could for example help other institutes think about how they want to organise citizen science.
3.2 Research methods
The following section discusses how data will be collected within the chosen case study. This thesis is based on a qualitative research method and combines interviews with a literature study. Concerning the interviews, three RIVM employees have been interviewed who are responsible for the integration of citizen science in their respective domains. Respondent 1 operates in the field of
public health and specialises in community based public health assessments. Besides publishing a thesis on how to engage communities in citizen science projects (Den Broeder, 2017a), respondent 1 has also been involved in several health related citizen science projects, including ‘Gezond Slotermeer’ (in English: ‘Healthy Slotermeer’) (RIVM, 2015). Respondent 2 works as an air quality scientist at RIVMs environmental monitoring department. Under her lead RIVM has established several research collaborations with citizens in the field of environmental monitoring, including the iSPEX project. Respondent 3 operates in the environmental domain as a social scientist specialised in risk and society and contributed to citizen science research (Den Broeder et al., 2017c). The variety of the respondent’s roles within RIVM and their distinctive network within the organisation enriches this research with different experiences and insights. The interviews have been conducted on the basis of semi-structured question lists, which have been tailored to each respondent according to their expertise (see appendix 1 till 3). The interview questions have been formulated based on the sensitizing concept, which are presented in the theoretical framework (table 4). After conducting three interviews with different RIVM insiders, it became clear that respondents were largely responding to questions with similar answers. When the third interview did not provide any new information but confirmed previous findings it was decided not to conduct other interviews.
To complement the interview data, a literature study has been conducted using the scoping review method. A literature search on official RIVM websites, Google, Google scholar, the University of Amsterdam’s online library ‘Catalogue Plus’ and YouTube has yielded a total of fifteen articles, which will be used during the analysis. Table 5 provides an overview of the scoping review method. The first column entails the sensitizing concepts that have been derived from the previous literature review. These concepts have been ranked according to the sub-question to which they belong. The second column of the table gives an overview of specific search terms tat have been used to find relevant information online. These search terms are based one the sensitizing concepts. Finally, the third column presents the studies that have been gathered. Some studies are mentioned multiple times because they have been used to answer more than one
sub-question. The scoping review process has been iterative, meaning that throughout the analysis new or relevant studies have been added to the table while other studies that appeared to be less relevant have been removed. Table 5 presents the final list of documents that have been used during this research. Table 5: Literature study on the basis of scoping review method
Sensitizing concepts Online search terms Studies Technical advances
Changing scientific landscape - From mode 1 to mode
science
Decreasing authority of experts - Increasing complexity of issues - Technological empowerment of citizens - Democratic deficit of technocratic discourse Possibility to produce better knowledge
Increasing efficiency of policies Increasing legitimacy of RIVMs research methods
Increasing research capacity
- Citizen science - Participatory research
strategies
- Why citizen science - Citizen science
benefits
- Pros and cons citizen science
- Death of expertise - Expert dilemma - From mode 1 to mode
2 science Irwin (2001) RIVM (2009) Haklay (2012) Shirk et al. (2012) RIVM (2014)
Den Broeder (2017a) MAN.RIVM.nl (2018) RIVM (2018a) Samenmetenaan- Luchtkwaliteit (2018b) Samenmetenaan- Luchtkwaliteit (2018c) Levels of citizen participation
Goal of research project
- Citizen science - Citizen science
definitions - Citizen science
typologies
- Citizen science levels - Citizen science tasks
Roter et al. (1997) De Veer et al. (2004) Samenmetenaan- Luchtkwaliteit (2018b)
Haklay (2012) Shirk et al. (2012) Luchtkwaliteit.nl (2018c) Waag.org (2018) YouTube.com (Channel: ‘Samen Milieu Meten’) Decreasing quality of research
Decreasing independency - Avoiding conflict of
interest Refraining from sloganizing
- Citizen science - Citizen science pros
and cons - Citizen science
disadvantages - Citizen science risks
Den Broeder (2017a) Waag.org (2018)
Demonstrating scientific benefits of citizen science
Branching out across different disciplines
Fostering active networking and new formats for collaboration Fostering policy monitoring and development
Aligning citizen science funding with other scientific approaches Collaborating in various formats Investing in excellent
communication
- Citizen science - Citizen science
innovations
- Citizen science lessons - Citizen science toolkit - Citizen science
improvements
Den Broeder (2017a) Hecker et al. (2018) RIVM (2018c)
4. Analysis
The following chapter sets out to answer the four sub-questions of this research. The structure of the analysis is as follows. First, RIVMs motivations to invest in citizen science will be examined. Previous chapters of this thesis have revealed that RIVM research must comply with strict quality standards. The disadvantages of citizen science may interfere with these standards. Hence, one might wonder what makes citizen science valuable enough to integrate it in standing procedures of knowledge production.
Second, the organisation of citizen science projects will be analysed to reveal what citizen science looks like in practice. The why and how questions are closely related, because analysing how scientists and citizen cooperate makes it possible to determine whether RIVMs interests correspond with their actions. Considering the fact that a significant amount of critical citizens (Soff, 2016; Guido, 2017; Seinen, 2017) as well as academics (Irwin, 2001; Shirk et al., 2012, Haklay, 2012) suspect organisations to use citizen science as a greenwashing technique or to exploit citizens for their own research goals, this thesis will invest whether this suspicion is valid or not in the case of RIVM. A discrepancy between RIVMs stated interests of investing in citizen science and the actual execution of citizen science could for example prove that RIVM indeed uses this participatory research strategy as a slogan to boost their public reputation.
Third, the challenges of incorporating citizen science in RIVMs standing procedures of knowledge production will be analysed. As mentioned earlier (see paragraph 1.4) interviews with RIVM researchers have revealed that asking questions about how RIVM scientists and citizens collaborate resulted in stories about the difficulties of such partnerships. RIVM employees explained that they are struggling to implement citizen science in current practices of knowledge production. Considering the fact that a limited amount of research is currently available on these struggles and how to deal with them, this thesis aims to explore the challenges of integrating citizen science in standing procedures of knowledge production followed by a number of future recommendations for the integration of citizen science. These recommendations may help employees to better understand how citizen science can be integrated into RIVMs current research arrangements in such a way that it helps citizens to answer questions
effectively and, at the same time, contributes to the quality of data produced by public research institutes.
4.1 Exploring RIVMs citizen science motivations
When introducing the topic of citizen science to friends, peers or academics, there is one question that comes up time and time again: why would experts, who are judged on the scientific quality of their research projects, want to collaborate with non-professionally trained citizens? Dutch public research institutes, including RIVM, have long operate autonomously in order to maintain the high quality of scientific data that they are expected to deliver (RIVM, 2018a). So what might explain RIVMs decision to change it research procedures? The following paragraphs will elaborate on each development that is partly responsible for RIVMs decision to invest in citizen science as an innovative research method.
4.1.1 Tackling complex issues together
First, citizen science has been introduced in a number of research projects because RIVM grasps the increasing complexity of current issues. According to RIVMs strategic vision (2014: 3) innovation and collaboration are extremely important in a world that is changing rapidly and is increasingly difficult to predict. RIVM scientist Den Broeder (2017a) affirms that scientists are nowadays confronted with increasingly complex problems, such as the obesity epidemic and air pollution. This insight aligns with the discussed theories produced by Abbott and Snidal (2009) and Hale and Held (2011), which suggest that expert-based knowledge production is currently failing to deliver due to the increasing complexity of societal challenges. Den Broeder (ibid.) argues that complex issues can indeed only be addressed properly through a ‘Whole of Society’ approach; meaning other stakeholders, including citizens, should also be involved in the production of knowledge. A whole of society approach moves away from a one-dimensional ‘technocratic’ research strategy and, instead, aims to bring different actors together that engage in jointly exploring solutions to shared problems (Kickbusch and Gleicher, 2011: 34).
The whole of society approach reflects the idea that combining the skills and knowledge of different actors creates new possibilities to combat intricate challenges. As discussed in the literature review, Irwin (1995) explains that citizen science may for example contribute to better knowledge as lay people can complement scientific knowledge with their local or traditional knowhow. The Ammoniac Monitoring Network in Natural Reserves (in Dutch: Meetnet Ammoniak in Natuurgebieden) is an example of an RIVM environmental project where lay people successfully contribute to the mapping of ammoniac with their local knowhow. In this specific RIVM project forest rangers are responsible for the collection of ammoniac data in natural reserves (MAN.RIVM.nl, 2018). Measurements are taken by forest rangers at locations that they deem relevant. RVM scientist have decided to collaborate with forest rangers, because they know the dynamics of the natural reserves better than anyone else. By now, the measurements are of such high quality that they are directly used for public policies in regard to the preservation of natural reserves (Volten, H., interview, May 07, 2018).
Second, citizen science does not only help to gather data that is richer and more diverse nature, it also helps to collect more data (Volten, H., interview, May 07, 2018). With the help of volunteers, scientists can distract data from more places. RIVM projects such as ‘The Fireworks Experiment’ (Samenmetenaanluchtkwaliteit.nl, 2018b) illustrate how citizens may help to collect data from a large variety of places. During this particular project, citizens in various Dutch cities were measuring fine dust with small portable sensors during the New Years festivities. This allowed RIVM scientist to map potential increases in fine dust concentration during New Year’s Eve. The contributions of volunteers helped RIVM scientist to collect more fine dust measurements and also made it possible to gather data from different places throughout the Netherlands.
Third, the involvement of citizens in research projects enables scientists to gather data from geographical places or communities that would have been more difficult to monitor without the help of volunteers (Den Broeder, L., interview, May 02, 2018). In her thesis on citizen science, Den Broeder (2017a: 52) provided an example of one citizen science project where key figures from a
disadvantaged neighbourhood in Amsterdam were trained to interview other community members in order to get a better insight of neighbourhood health assets, which could be used to develop local policy. According to Den Broeder (ibid.), the residents in this particular neighbourhood were ‘hard to reach’. Therefore local residents were trained to interview community members, because it was expected that residents would share more information with people who they already know from their community then they would when talking to strangers from RIVM.
Lastly, interviews with respondents 1 (Den Broeder, L., interview, 02 May, 2018) and respondent 2 (Volten, H., interview, May 07, 2018) have suggested that RIVM has grasped the added value of involving citizens in the preliminary stages of research designs, because it helps scientists to understand what it is exactly that citizens worry about and what they would like to see investigated. For example, respondent 1 (Den Broeder, L., interview, May 02, 2018) shared a story about a community that was awaiting the development of a new railway, which would run through their city. The community was unpleasantly surprised by this development, which led scientists to investigate the noise pollution that the railroad would produce. Scientist hoped to find that the noise pollution would not exceed legal standards in an attempt to reassure the community. Although the results of this research were positive, residents remained angry. The municipality asked RIVM scientists to come and talk with community members about the effects of the railway. When RIVM started an inquiry into the worries of community members it appeared that residents were not so much bothered by potential noise pollution, but worried instead about the large amount of blind citizens living in their neighbourhood who would not be able to cross the railroads safely. This information made it possible for scientists to conduct research that anticipated on the interests of the community, which in turn helped to develop effective and legitimate local policies. Respondent 2 (Volten, H., interview, May 07, 2018) shared another anecdote where collaborations between scientist and a harbour community revealed that citizens were not interested in scientists to measure high frequency sounds in their neighbourhood, which is a common practice when conducting research on noise pollution, but wanted to know more about particular low frequency sounds,
because they were perceived as much more irritating. Residents from this community would lie awake at night due to ship generators that produce a low humming sound. As there are no legal standards on these types of low-frequency sounds, scientists had not previously considered taking these sounds into account during their research.
4.1.2 Abandoning the ‘ivory tower’
Second, RIVM decided to invest in citizen science because the institute has recently came to realize that its conventional method of knowledge production is starting to lose legitimacy (Den Broeder, L., interview, 02 May, 2018; Volten, H., interview, 07 May, 2018; Devilee, J., interview, 23 May, 2018). Previously, RIVM scientists refrained from collaborating with communities that would be affected by their research results (Volten, H., interview, May 07, 2018), which sometimes resulted in fierce public critique. This was for example the case with the controversy concerning HPV vaccinations. In 2009, RIVM suggested that all girls should be vaccinated against cervical cancer at the age of twelve (RIVM, 2009). Different groups of people, including doctors and parents, publicly opposed RIVMs decision, which caused a significant amount of public distress. According to critics, the effects of this vaccination had not been investigated thoroughly enough, leaving many questions to be answered about the effectivity of these vaccinations and the potential side effects that may appear later in life (for an overview of public critique see Zorgnu.avrotros.nl, 2016). Instead, collaborating with citizens allows scientist to get a more detailed understanding of the questions that communities are dealing with, which they can then take into account when developing knowledge for public policies. Such strategies are likely to stimulate the development of public policies that are perceived as more acceptable by the general public (Volten, H., interview, May 07, 2018). To illustrate this suggestion, respondent 2 (ibid) explained how unpopular advices, like prohibiting cars to enter certain areas because the air quality in this place is relatively bad, can be justified when residents from that particular area have been involved in the monitoring of the air quality.
After the negative HPV vaccination experience, respondents understood that RIVMs standing procedures of knowledge production were no longer
