Combatting Online Misinformation Regarding Vaccinations.
The Influence of a Warning Tool on Information Choice and Information Attitude.
Leonie Westerbeek, 11883472 Master’s Thesis
University of Amsterdam Graduate School of Communication
Research Master’s Program Communication Science
Supervisor: Dr. Hanneke Hendriks
Abstract
An increasing number of parents is refraining from vaccinating their children. This causes a low immunization coverage, which can result in disease outbreaks. A potential cause of this problem is related to online misinformation regarding vaccination. This study tests whether a warning tool with the appearance of a traffic light can influence parents’
information choice and information attitude. Parents’ with varying vaccination attitudes and decision stages were included. An online experiment was conducted with 197 participants during which the warning tool was tested and participants’ information choices and attitudes were measured. People were asked to choose three links on a Google page for more
information and were asked to evaluate this information after reading it. The results showed that people in the warning tool condition selected a higher number of articles marked as reliable than people in the control condition. This effect was stronger for parents who still had to make a vaccination decision for their child than for parents who had already decided. The findings are promising for future practical implication of the tool. Future research is urged to examine the warning tool even further, as the current study serves as a very first exploratory experiment.
Currently in the Netherlands, a lot of debate is going on around vaccinations (Rusman, 2018). In 1957, the RIVM has created a vaccination program in which children receive certain vaccinations at certain ages (RIVM, 2019). Whether or not to follow this vaccination program is the choice of the parents. However, in order for the program to induce group immunity, a high percentage of children has to be vaccinated (i.e. the immunization coverage) (RIVM, 2018). The required immunization coverage for group immunity differs per disease. For the measles, for instance, at least 95% of all children have to be vaccinated to reach group
immunity (RIVM, 2018). Over the past few years, however, the immunization coverage in the Netherlands has been gradually decreasing (Van Lier et al., 2019), meaning that an increasing number of parents is choosing not to vaccinate their children. This creates a public health hazard, as a decreasing immunization coverage can result in disease outbreaks (RIVM, 2018).
A potential cause of this process may be related to online misinformation. Today, anyone can place anything on the Internet for everyone to find and read. This means that lay-persons can also spread information on a large scale, resulting in uncertainty regarding the reliability1 of online information (Flanagin & Metzger, 2000; Meppelink, Smit, Fransen & Diviani, 2019). Alarmingly however, an increasing number of individuals is using the internet to seek health information (Cotten & Gupta, 2004). Because a lot of unreliable information regarding vaccinations is spread on the Internet, the risk of online misinformation about this topic is high (Kata, 2010). Online misinformation regarding vaccinations can have harmful consequences, as it can influence parents towards choosing not to vaccinate based on unreliable information. It is therefore important to examine ways of helping individuals to distinguish between reliable and unreliable information. This study aims at doing so.
Previous studies have looked at possible solutions for online misinformation (e.g. Diviani & Meppelink, 2017; Nyhan, Reifler, Richey & Freed, 2014), but a single effective
solution remains absent. This study argues that a possible method for helping people to distinguish reliable from unreliable information is by using a warning tool, as it has been applied successfully in other contexts (Hwang & Jeong, 2016; Ludolph, Allam & Schulz, 2016). The goal of the current study is to test such a warning tool which will have the appearance of a traffic light. The color of the traffic light (green vs red) will be indicative of the reliability of the information. This study will test if the tool is effective in guiding people towards choosing reliable information over unreliable information. The current study
contributes to previous literature on combatting online misinformation regarding vaccinations. To test the warning tool, it will be assessed if the tool can guide people towards
choosing reliable information and if the tool has an effect on people’s attitude towards the information they read. Furthermore, it will be assessed if the tool is equally effective for all parents or if differences can be found between individuals with varying attitudes towards vaccination and between individuals who have already made a vaccination decision for their child versus individuals who still have to decide. This study aims at answering the following research question:
What is the effect of a warning tool in the context of online vaccination information on information choice and information attitude for several groups of the population?
Theoretical Framework Online Misinformation
In today’s digital society, the Internet is widely used and plays an important role in our daily lives. In 2018, 86% of the Dutch population used the internet on a daily basis (CBS, 2019). The Internet has developed into an interactive medium in which users contribute to the content (O’reilly, 2009). As a consequence, anyone can place information on the Internet regardless of their qualifications (Flanagin & Metzger, 2000). This results in a massive amount of misinformation being present on the Internet (Shao, Ciampaglia, Flammini &
Menczer, 2016). When seeking health information online, individuals can experience
difficulties in distinguishing between reliable and unreliable information. As a consequence, if individuals do not succeed in recognizing reliable and unreliable information, they might trust misinformation. In the context of online vaccination information, trusting unreliable
information can seriously harm the immunization coverage.
In regard to the immunization coverage, in the past mainly religious parents were deciding to refrain from vaccinating their children (Ruijs, 2013). More recently, however, a lot of non-religious parents are also refraining from having their children vaccinated
(Vermeulen, 2015). They believe that vaccinations can have harmful effects on their children or that their children will benefit from going through certain diseases instead of being
protected against them (Vermeulen, 2015). A lot of individuals are spreading their ideas online on blogs or anti-vaccination websites. Since individuals now get conflicting messages on vaccination, it is difficult for those seeking online vaccination information to determine which information to trust. These anti-vaccination websites often contain misinformation, such as a suggested link between autism and vaccinations which, in reality, has been proven wrong (Godlee, Smith & Marcovitch, 2011). Previous research has shown that exposure to anti-vaccination information leads to a lower intention to vaccinate (Jolley & Douglas, 2014), thereby stressing the importance of helping people to filter out unreliable information. Preventing Online Misinformation
Online misinformation regarding vaccinations can have severe consequences for public health. If individuals who are searching for online information about vaccinations trust misinformation, this can negatively influence their decision to vaccinate their children, thereby endangering the immunization coverage (Kata, 2010). It is, thus, of great importance to find ways of combatting misinformation. This has been examined in previous studies numerous times, but an effective solution has not yet been discovered.
Existing literature for instance suggests using storytelling to combat anti-vaccination misinformation (Shelby & Ernst, 2013). The spreading of anti-vaccination misinformation often relies on the power of storytelling and the authors suggest using this powerful form of conveying information to combat the misinformation. For instance by creating story-based pro-vaccination blogs combined with scientific evidence (Shelby & Ernst, 2013). This solution is, however, not supported with empirical evidence. Another solution posed in previous research is training experts in how to defuse the misinformation (Smith &
MacDonald, 2017). Unfortunately this solution is also not supported with empirical evidence. This can partly be explained by the fact that trying to defuse misinformation is very likely to be ineffective, as people are often unable to update their memories after a correction
(Pluviano, Watt & Della Sala, 2017). After being told that earlier distributed information was incorrect, people are often unable to disregard the incorrect information and still tend to fall back on it (Pluviano et al., 2017).
What these previously proposed solutions have in common, is that they try to counter misinformation after exposure. However, research has shown that countering vaccination misinformation after exposure has taken place is most often ineffective and can even backfire (Pluviano et al., 2017). Nyhan et al. (2014) also confirm that trying to correct misinformation regarding vaccinations is very likely to be counterproductive. Therefore, a more preventive method against misinformation seems necessary. Warning people about the unreliability of information before they choose to read it is expected to be more effective. Research has shown that presenting a warning at the time of initial exposure to misinformation, rather than afterwards, can successfully reduce the impact of the misinformation (Lewandowsky, Ecker, Seifert, Swarz & Cook, 2012). In this way, the misinformation no longer has to be countered or corrected after exposure.
Warning Tools
One preventive method for combatting misinformation is the use of warning tools. Warning tools have been used for several subjects and often yield successful results (Lewandowsky et al., 2012). Sponsorship disclosure is an example of an effective warning tool used in the persuasive communication field. A simple sponsorship disclosure, stating “this is a sponsored post” has been shown to reduce persuasion by the post (Hwang & Jeong, 2016). Moreover, a study by Ecker, Lewandowsky & Tang (2010) tested the effects of explicitly warning people that they might be misled before exposure to misinformation in a news story. Once again, results showed a significant decrease in influence of the
misinformation (Ecker et al., 2010).
Furthermore, the literature suggests that warning tools which are simple, are most effective in reducing effects of misinformation (Hwang & Jeong, 2016; Ludolph, Allam & Schulz, 2016). In the context of vaccination misinformation, a simple warning about
vaccination misinformation, with a limited amount of easily understandable text, turned out to be more effective than a more complicated warning (Ludolph et al., 2016). Furthermore, warning tools can influence the information choices and perceived information quality of participants (Ludolph et al., 2016). In the case of the traffic light, this could cause participants to hold more positive attitudes towards information marked as reliable and more negative attitudes towards information marked as unreliable.
In the present study, the warning tool would have the appearance of a traffic light. Using a traffic light as a warning tool has been proven successful in warning individuals about the credibility of information (Idris & Jackson, 2011). This type of warning tool could have a similar positive effect in the context of vaccination misinformation. In the study by Ludolph et al. (2016) textual warning information was presented to the participants. An even more simple warning could be even more effective in reducing the effects of misinformation by
guiding individuals towards a reliable information choice and by influencing their attitude towards the information. A simple visual warning with the appearance of a traffic light could be very suitable for this.
Presenting a warning tool with the appearance of a traffic light to participants before they have to choose which webpage with vaccination information they want to read, can help them to identify reliable and unreliable information. This can guide them in making reliable information choices and increases awareness about the reliability of the information they choose to read. Based on the literature it can be expected that presenting the warning tool to participants will influence their information choice and will influence their attitude towards the information they read. The following hypotheses are therefore posed:
H1a: Participants in the warning tool condition will choose to read information marked as reliable as opposed to information marked as unreliable more often than participants in the control condition.
H1b: Participants in the warning tool condition will hold more positive attitudes towards information marked as reliable and more negative attitudes towards
information marked as unreliable compared to participants in the control condition. Pre-Existing Attitude Towards Vaccination
It can be expected that the effect of the warning tool on participants’ information choice as well as their attitude towards the information will depend on pre-existing attitudes towards vaccination. During information seeking, individuals often seek evidence that is congruent with pre-existing beliefs; a phenomenon known as confirmation bias (Nickerson, 1998). Confirmation bias entails that individuals actively seek or interpret information in line with what they already believe or expect (Klayman, 1995).
Research has shown that when it comes to online health information seeking regarding vaccinations, parents tend to select belief-consistent information (Meppelink, Smit, Fransen &
Diviani, 2019). Furthermore, they hold a more positive attitude towards information confirming their beliefs than towards information opposing their beliefs (Meppelink et al., 2019). This process of confirmation bias is rather powerful and might cause individuals to ignore the warning tool.
People with a negative attitude towards vaccination show a strong lack of confidence in authorities that deliver messages (Masaryk & Hatoková, 2017). This group of individuals has rather strong pre-existing beliefs and are extremely skeptical towards authorities claiming anything else (Masaryk & Hatoková, 2017). This has two consequences. Firstly, as the pre-existing beliefs of this group are rather strong, confirmation bias is especially likely to occur. It can be expected that participants are so set on selecting belief-consistent information that they will not take the warning tool into account while doing so. Secondly, their lack of confidence in authorities makes them less likely to trust a warning tool informing them about reliability. Seeing a traffic light which tells them that certain information is reliable will not easily convince them and it can therefore be expected that their attitude towards the
information is influenced less strongly by the warning tool.
Based on the literature, the effects of the warning tool are expected to occur less strongly for participants with a negative attitude towards vaccination. Their lack of confidence in authorities and their search for belief-consistent information makes them more likely to ignore the warning tool. As a consequence, their information choice as well as their attitude towards the information will be influenced less strongly than those of participants with a more positive attitude towards vaccination. This results in the following hypotheses:
H2a: The more positive participants’ attitude towards vaccination, the stronger the effect of the warning tool on the information choice of participants is.
H2b: The more positive participants’ attitude towards vaccination, the stronger the effect of the warning tool on the information attitudes of participants is.
Decision Stage
Another personal characteristic very relevant to this study is whether individuals have already made a vaccination decision for their child. The effects of the warning tool might differ for these groups. For the parents who have already decided whether or not their child will be vaccinated, cognitive dissonance is likely to occur during the online information seeking process. Cognitive dissonance entails having to deal with cognitions that are conflicting (McMaster & Lee, 1991). When a person encounters two (or more) conflicting cognitions they reach a state of dissonance, this evokes uncomfortable feelings motivating the individual to reduce the dissonant state (McMaster & Lee, 1991).
If parents who have already decided to either vaccinate their child or refrain from vaccination encounter information which is conflicting their decision, an uncomfortable state of dissonance will occur. Research has shown that in order to prevent being in a state of dissonance, individuals tend to select information that is consonant with their previous actions (Adams, 1961). Furthermore, if individuals read vaccination information that is not in line with their previous behavior, they will negatively evaluate the information in order to prevent being in a state of dissonance (Wilson, Mills, Norman & Tomlinson, 2005). It can be expected that parents who have already made a vaccination decision want to prevent reaching a state of dissonance and are likely to ignore the warning tool while doing so. It is possible that they would rather choose information with a red traffic light and alter their attitude towards that information than to choose incongruent information causing them to reach a state of dissonance.
For the warning tool tested in this study, the aforementioned findings about cognitive dissonance would suggest a different effect of the tool for people who have already made a vaccination decision versus people who still have to decide. For the former, it can be expected that participants will try to prevent reaching a state of dissonance, even if that means ignoring
the presented warning tool. Furthermore, participants will most likely try to reduce their state of dissonance by holding more negative attitudes towards information that is not in line with their decision, regardless of the warning tool. It is expected that the information choice and information attitude of this group of individuals are influenced less strongly by the warning tool. This results in the final two hypotheses assessed in this study:
H3a: The effect of the warning tool on the information choice of participants is stronger for participants who still have to make a vaccination decision than for participants who have already decided.
H3b: The effect of the warning tool on the information attitudes of participants is stronger for participants who still have to make a vaccination decision than for participants who have already decided.
All of the proposed relationships have been visualized in a conceptual model (see Figure 1).
Figure 1. Conceptual Model
Methods Participants and Design
The online experiment was conducted among parents who are either expecting a child or have a child of nine years or younger. The age of nine was set as a maximum, as children receive vaccinations until this age according to the national program (RIVM, 2019).
several parts of the Netherlands have agreed to cooperate in distributing the questionnaire among their clients. Furthermore, two primary schools and an association with five childcare facilities have assisted by sharing the questionnaire with parents. Lastly, the questionnaire has been shared on a Facebook page for pregnant women.
In total, 394 participants started the questionnaire, 191 of these participants completed the online experiment. Responses of individuals who indicated not having any children (n = 4) and individuals who looked at the information on the webpages for less than 10 seconds (n = 8) were excluded. This resulted in a final sample of 179 participants. Table 1 provides insight into the characteristics of the participants.The sample included mainly female
participants. Slightly over one third still had to make a vaccination decision for their child, the others had already made this decision. The sample had a mean age of 33.35. The majority of the participants, 83.8%, vaccinated or was planning to vaccinate their child according to the Dutch national immunization program. Participants indicated searching for a bit of
information online when making their vaccination decision and, overall, had a fairly high e-health literacy.
This study used a design with 2 experimental between-subjects conditions (warning tool: absent vs present) and 2 moderators. One moderator used 2 groups (decision stage: before vs after), the other moderator (vaccination attitude) was used as a continuous variable. Participants were randomly assigned to one of two experimental conditions, seeing either a regular Google page or a Google page with the warning tool present. The two moderators, vaccination attitude and decision stage, were measured in the questionnaire. The study has two dependent variables. Firstly, the information choice of the participants, which was
measured in the questionnaire by having participants choose three links on a Google page and coding how many of these articles were marked as reliable. Secondly, participants’ attitude
towards the information they read was also measured in the questionnaire after reading each page of information.
Procedure
The experiment was conducted online via Qualtrics. Participants received a link to the survey through one of the previously mentioned channels. When opening the link, participants first received an invitation to participate with an explanation of what they could expect during the study. Participants were then asked to give their informed consent. The questionnaire then started with the vaccination hesitancy scale to assess the pre-existing vaccination attitude of the participants. Afterwards, participants saw a regular Google page or a Google page with the warning tool depending on their experimental condition. The participants were asked three times to choose a link from a Google page leading to webpages with information about
vaccinations. After reading each webpage with information, participants answered questions about their attitude towards the information they just read.
Afterwards, participants answered questions about their children, if their children were vaccinated and how they came to this decision. Participants’ age and level of education were also asked. Lastly, participants answered questions to assess their e-health literacy. The questionnaire ended with a debriefing, disclosing that the Google page had been manipulated to show these specific links. It was also disclosed that for the purpose of this experiment, the traffic light did not represent the actual reliability of the information. Participants were told that they are always advised to rely on the RIVM for reliable information. Lastly, participants were thanked for their participation and the study ended. The entire questionnaire can be found in Appendix B.
Materials
The stimulus material of this study was a fake Google page with ten links related to childhood vaccinations. The Google page was created in such a way that the title of each link disclosed if the information was positive, negative or neutral about vaccination. The page displayed four links which were positive about vaccination, four links which were negative and two neutral links. In order to present links that were actually perceived positive, negative and neutral, a set of links was pretested. The pretest was conducted among 29 participants (22 women, 7 men). All participants had children and their mean age was 43.34 (SD = 6.43). The pretest was conducted through an online survey during which participants were exposed to 20 links in a random order and were asked questions about the valence, reliability and their familiarity with the source of each link. The links were gathered by Googling several positive and negative search terms about vaccinations such as “importance of childhood vaccinations” and “danger childhood vaccinations”. A total of eight positive, eight negative and four neutral links was presented. Based on the results of the pretest, the four most positively valued, four
most negatively valued and two most neutral valued links, which scored neutral on reliability and low on familiarity were chosen for the stimulus material. The neutral reliability score was necessary, as the articles would be marked as (un)reliable at random and participants were not supposed to become suspicious. Furthermore, the low familiarity score ensured the least pre-existing attitudes towards the links and information presented.
During the experiment, participants could choose three links to click on. When participants clicked the links, the information of the corresponding webpage would appear, allowing them to look at the provided information. Screenshots of the corresponding websites were used for this. The links and corresponding webpages were identical for the two
conditions. In the control condition, participants were simply exposed to a regular Google page with links about childhood vaccination. For the experimental condition, the warning tool had been added to the Google page. The warning tool consists of a red or green traffic light placed in front of each link, indicative of the reliability of the information. It was explained to participants that reliable in this case means that the information is based on truth and is trustworthy (Adams, 2010). In reality, the reliability was randomly chosen and spread equally across the links with varying valence. This resulted in two reliable positive links, two
unreliable positive links, two reliable negative, two unreliable negative, one reliable neutral and one unreliable neutral link. This was disclosed to the participants during the debriefing. The stimulus material for both conditions can be found in Appendix A.
Measures
Vaccination attitude.
The independent variable ‘vaccination attitude’ has been measured with the
Vaccination Confidence Scale (Gilkey, Magnus, Reiter, McRee, Dempsey & Brewer, 2014). The scale consisted of eight items, α = .89, M = 5.51, SD = 1.20. The items included for instance “Vaccines are safe” and “Children receive too many vaccines” (see Appendix B). All
items were measured on a 7-point Likert scale (1 = completely disagree, 7 = completely agree). The variable was used as a continuous variable during analyses.
Decision stage.
The independent variable ‘decision stage’ was also measured in the questionnaire. A distinction was made between participants who still have to decide if they vaccinate their child and participants who have already made this decision. This was operationalized by asking for the age of participants’ youngest child. If the child was not born yet or was younger than six weeks old, the participant still had to make a vaccination decision, as children receive their first vaccine at six weeks according to the national vaccination program. If the
participant’s youngest child was older than six weeks, a vaccination decision had already been made.
Information choice.
The dependent variable ‘information choice’ was a score calculated based on
participants’ three information choices. For each chosen link which was marked as reliable, a point was added to their score. This resulted in an information choice score varying between 0 and 3. Where 0 indicated choosing three links marked as unreliable and 3 indicated choosing three links marked as reliable. This resulted in an ordinal dependent variable.
Information attitude.
The second dependent variable, ‘information attitude’, was measured with six items based on a scale by Hong & Lee (2007) and a scale by Elliott (1979), α = .93, M = 4.41, SD = 1.01. All items were measured on a 7-point Likert scale (e.g. 1 = bad, 7 = good). The items were “good - bad”, “impressive - unimpressive”, “informative - uninformative”, “interesting - uninteresting”, “believable - unbelievable”, “useful - useless”. These questions were asked three times, immediately after reading each webpage with information. A mean score of these items was used to determine the participants’ attitude towards the information.
Control variables.
Lastly, control variables were also measured in the questionnaire and used as covariates during analyses. Level of education and age of the participant were assessed. Furthermore, participants were asked if their children had been vaccinated, if religion played a role in this decision, with whom this decision was made and if they searched for information online before deciding. Finally, participants’ e-health literacy was assessed using the eHEALS measure (Norman & Skinner, 2006). This measure consisted of eight items measured on a five point scale (1 = completely disagree, 5 = completely agree), α = .89, M = 4.04, SD = 0.52. The scale included items such as “I know which health information can be found on the Internet” and “I know where I can find health information on the internet” (see Appendix B).
Results
The data was analyzed using several analyses in SPSS. The variables age, gender, level of education, role of religion, online vaccination information seeking and e-health literacy were included in every analysis as control variables in order to control for their potential interference.
Effects on the Information Choice
According to Hypothesis 1a, participants in the warning tool condition would choose to read more links marked as reliable (as opposed to unreliable) than participants in the control condition. The dependent variable, the number of reliable links chosen, is ordinal. Therefore, in order to test this hypothesis, an ordinal logistic regression was performed. The results showed a significant main effect of the warning tool condition on the reliability of participants’ information choice. The odds of people in the warning tool condition choosing reliable links was 2.74 (95% CI = 1.54 - 4.88) times that of people in the control condition, Wald χ2(1) = 11.81, p = .001, also displayed in Table 3. Frequencies of the information choice scores in each condition can be found in Table 2. This indicates that participants who
were exposed to the warning tool were more likely to choose a webpage that was marked as reliable by the warning tool than when no such warning was present, confirming Hypothesis 1a.
Hypothesis 2a predicted a moderating role of vaccination attitude for the direct effect of the warning tool on the information choice. It was expected that the more positive
participants’ pre-existing attitudes towards vaccinations were, the stronger the effect of the warning tool on the information choice would be. In order to test this hypothesis, an interaction effect between condition and vaccination attitude was added into the ordinal regression analysis. The results showed no significant interaction effect between the warning tool and participants’ vaccination attitude, Wald χ2(1) = 0.01, p = .910, as can be seen in Table 3. These results indicate that a more positive or negative vaccination attitude did not affect the strength of the effect of the warning tool on the information choice. Hypothesis 2a was, thus, not confirmed.
Hypothesis 3a predicted another moderator for the main effect of the warning tool on the information choice: decision stage. It stated that the effect of the warning tool on the information choice of participants would be stronger for participants who still have to make a vaccination decision for their child than for participants who have already decided. In order to test this hypothesis, another ordinal logistic regression with an interaction effect between the condition and the decision stage was performed. The analysis showed a significant interaction effect between the warning tool condition and the decision stage of participants, as is shown
in Table 3. The effect of the warning tool turned out to be stronger for participants who still had to make a vaccination decision for their child as compared to participants who had already decided. The odds of people in the warning tool condition who still had to make a vaccination decision were 5.65 (95% CI = 2.18 – 14.64) times that of people in the warning tool condition who had already made this decision, Wald χ2(1) = 12.68, p = .001.
Effects on the Information Attitude
The other three hypotheses in this study concerned the dependent variable information attitude and were tested with a linear regression analysis. Parameter estimates of these
analyses can be found in Table 4. The dependent measure, information attitude, is the mean score of the information attitude measured after viewing each webpage. A higher information choice score was expected to result in a higher information attitude score. In other words, it was expected that participants choosing a higher number of reliable articles would hold a more positive attitude towards those articles. In order to test this in SPSS, an interaction effect between condition and information choice on information attitude had to be analyzed, as it was expected that a higher information choice score results in a higher information attitude score, but only for participants in the warning tool condition. This interaction effect would, thus, test if the warning tool condition causes participants to have a more negative attitude
towards articles marked as unreliable and a more positive attitude towards articles marked as reliable. As mentioned, this could be tested with a simple linear regression analysis.
Hypothesis 1b stated that participants in the warning tool condition would hold more positive attitudes towards reliable information and more negative attitudes towards unreliable information compared to participants in the control condition. In order to test this hypothesis, a linear regression with an interaction between the condition and the number of reliable webpages chosen as the independent variables and participants’ attitude towards the information presented as the dependent variable was performed. The results showed no significant regression equation, F(3, 175) = .315, p = .814, R2 = .005. This means that marking links as (un)reliable with the warning tool had no significant effect on participants’ attitude towards the information (β = -.064, p = .736). Hypothesis 1b was, therefore, not confirmed.
According to Hypothesis 2b, it was expected that the more positive participants’ pre-existing attitudes towards vaccinations were, the stronger the effect of the warning tool on the information attitudes would be. In order to test this hypothesis, another linear regression was performed in which vaccination attitude was added as a moderator of the effect on
information attitude. The results showed no significant regression equation, F(5, 173) = .502, p = .774, R2 = .014. No significant moderating effect of vaccination attitude was found (β = .379, p = .353), meaning that Hypothesis 2b was not confirmed.
Hypothesis 3b predicted that the effect of the warning tool on the information attitudes of participants would be stronger for participants who still have to make a vaccination
decision for their child than for participants who have already decided. Once again, a linear regression analysis was performed with decision stage added as a moderator of the effect on information attitude. The results showed no significant regression equation, F(7, 171) = .394,
p = .905, R2 = .016. No significant moderating effect of decision stage was found (β = -.054, p = .629), thereby not confirming Hypothesis 3b.
Conclusion and Discussion
The goal of this study was to investigate whether a warning tool for online misinformation regarding vaccinations influences participants’ information choice and
information attitude. These effects were studied for individuals with varying attitudes towards vaccinations and people at varying decision stages, in order to discover potential differences between these groups.
Findings and Implications
The first important finding was an effect of the warning tool on participants’
information choice. The warning tool caused participants to choose more articles marked as reliable (as opposed to unreliable) compared to the control condition. This means that based on the results, the warning tool can effectively guide individuals in choosing reliable
information about vaccinations. These findings supported Hypothesis 1a and were in line with the literature which states that warning tools have been used effectively in numerous fields (Lewandowsky et al., 2012) and that more simple warning tools are most effective in influencing individuals’ information choices (Hwang & Jeong, 2016; Ludolph, Allam & Schulz, 2016). This study confirms the effectiveness of a simple warning tool in the context of online vaccination information and is therefore an addition to the existing base of literature. It
shows that a traffic light can successfully guide individuals towards choosing reliable information about vaccinations. These findings are also very relevant in practice, as future implementation of the warning tool in real life can contribute to protecting the immunization coverage.
The second finding of this study was that the effect of the warning tool on
participants’ information choice did not differ between people with different attitudes towards vaccinations, in contrast with Hypothesis 2a. It was expected that a more positive vaccination attitude would result in a stronger effect of the warning tool on information choice. The literature suggested a lack of confidence in authorities that deliver messages among people with a negative attitude towards vaccinations, resulting in skepticism towards anything they say (Masaryk & Hatoková, 2017). The findings of our study are not in line with this. The skepticism towards authorities did not result in a weaker effect of the warning tool.
In practice, this is an interesting and desirable outcome, as the aim of this study is creating a warning tool that is effective for people with varying attitudes towards vaccination and especially individuals who hold more negative beliefs towards vaccinations need to be protected against online misinformation as much as possible. A possible explanation for the absence of a moderating effect of vaccination attitude is that the source of the warning tool was not clearly the RIVM or another authority. This might not evoke the expected skepticism among individuals with a negative vaccination attitude. Furthermore, an equal amount of positive and negative links were marked reliable and unreliable. This might have caused the individuals to consider the tool more trustworthy. In real life, perhaps a higher number of pro-vaccination links would be marked reliable and links which are negative towards pro-vaccination might be unreliable more often. This could enhance feelings of skepticism among individuals with a negative vaccination attitude, as more links in line with their pre-existing beliefs are then claimed to be unreliable.
Moreover, this study found that the effect of the warning tool on the information choice was stronger for participants who still had to make a vaccination decision than for participants who had already decided. This is in line with Hypothesis 3a and also in line with the literature. The literature suggested that individuals who have already made a decision do not want to read any information that is not in line with their choice and might therefore ignore the warning tool in order to prevent reaching a state of dissonance (McMaster & Lee, 1991). The difference between these decision stages has not yet been assessed in this manner in previous research. The current study therefore provides valuable knowledge for filling this gap in the literature. Furthermore, this is a finding with important practical implications, considering that in practice the warning tool is especially important for individuals who still have to make a vaccination decision. These individuals are searching for information to help them make a decision and it is of great importance that they rely on reliable information. The fact that the effect of the warning tool is the strongest for this group of the population is therefore a desirable outcome for future implementation of the tool.
The effect of the warning tool on participants’ attitude towards the information on the webpage has also been examined. This study found that there was no effect of the warning tool on the information attitude and that there were also no moderating roles of pre-existing vaccination attitude and decision stage. These findings were in contrast with hypotheses 1b, 2b and 3b. It was expected that participants in the warning tool condition would have a more negative attitude if the information was marked unreliable and a more positive attitude if it was marked reliable, as previous research had shown an influence of a warning tool on people’s attitude towards the information (Ludolph et al., 2016). Once again, it was expected that skepticism towards authorities would cause people with a negative vaccination attitude to be less influenced by the warning tool (Masaryk, Hatoková, 2017). Furthermore, it was expected that participants who had already made a vaccination decision would be likely to
encounter cognitive dissonance during exposure to information not in line with their decision and would therefore ignore the warning tool while forming attitudes towards the information (Adams, 1961).
However, the current study was not in line with this literature. The warning tool could successfully guide participants towards a reliable information choice, but once this choice had been made, the tool did not affect their attitude towards the information. A possible
explanation for this could be that the warning tool was only visible on the Google page while making their information choice. When reading the information, the warning tool was not visible anymore and participants were no longer reminded of the (un)reliability of the presented information. Displaying the warning tool on the webpage itself could perhaps contribute to an effect on participants’ attitudes towards the presented information.
The above presented findings are a valuable addition to the existing literature. It shows the effectiveness of the warning tool for online vaccination information and a difference in the effect between people at varying decision stages which had not yet been examined before. Furthermore, this study has major practical implications. The decreasing immunization coverage in the Netherlands is an increasing problem (Van Lier et al., 2019) and the current study has shown that a warning tool can help people in making reliable information choices. The findings of this study show that the warning tool causes people to choose reliable information more often and suggests great potential for implementing the warning tool in practice in the future.
Limitations & Future Research
A limitation related to the sample of this study has to do with the number of participants with a negative attitude towards vaccination. Even though the immunization coverage in the Netherlands is decreasing (Van Lier et al., 2019), a large majority of the Dutch population still has a fairly positive attitude towards vaccination. Individuals with a
more negative attitude towards vaccination are difficult to reach and often not very willing to cooperate in research about vaccinations. As a result, the sample of this study contained mainly individuals with relatively positive attitudes towards vaccinations and only 22 participants with a vaccination attitude score below 4 on a 7-point scale. This might have influenced the results regarding the moderating role of vaccination attitude, as the small proportion of people with a negative vaccination attitude could possibly impede statistical significance for this moderating effect. For future research it would be advisable to target people with a negative attitude towards vaccinations even more. A bigger proportion of the sample with a negative vaccination attitude is important for testing the tool more elaborately, as people with a negative vaccination attitude are an important part of the target audience of this warning tool once it is implemented in practice.
Furthermore, a limitation regarding the survey arose during the process of data collection. During the online experiment, individuals had to choose a link to a webpage with information about vaccinations three times. The decision to have three choices was made in order to have enough variance to get meaningful results. However, answers to an open
question in the survey indicated that many participants found it too much work or found it too repetitive to choose three links and view three webpages. This lead to a rather low completion rate. A total of 394 individuals started the online experiment and only 191 individuals fully completed it. Many participants left the online experiment at the second or third moment of choosing a link. Future studies are advised to look into ways of making the experiment more user friendly in order to increase the completion rates. Finding a method for increasing the completion rate without losing the validity of having multiple moments of choosing a link will contribute to a larger final sample and more valuable results.
Moreover, the current study tested the warning tool with an equal amount of reliable and unreliable positive, negative and neutral links to webpages. The reliability was randomly
assigned to the links and did not represent actual reliability. This is a very good starting point, as it is necessary to start by assessing if the tool can influence individuals at al. The warning tool turned out to be effective in this way. As a next step, it would be good to test the warning tool with the real life reliability of the webpages implemented. Meaning that links only receive a green traffic light if they are truly considered reliable based on certain criteria. A measure such as DISCERN, which assesses the reliability and quality of online health
information, could be used for this (Khazaal, Chatton, Zullina & Khan, 2011). This will result in a greater external validity and the results will be even more representative for what will happen if the warning tool is implemented in real life.
A final discussion point of this study concerns the term reliability. Reliability is a somewhat questionable term, as it is hard to assess who decides which information is
considered reliable or unreliable. In the current study, the term reliable was used and defined as evidence-based and trustworthy (Adams, 2010). In the future, however, a shift in
terminology towards the word evidence-based can be expected. This is a more objective term which is easier to assess. All in all, it is important to realize the difficulties that come with reliability and future research should continue to evaluate and evolve this term.
Conclusion
In short, this study has some very interesting and relevant findings. The most
important finding is the effectiveness of the warning tool in increasing the amount of reliable webpages chosen by participants. This effect is especially strong for individuals who still have to make a vaccination decision for their child, compared to individuals who have already decided. The practical implications of these findings are that implementing the warning tool in practice can have great potential in combatting online misinformation regarding
vaccinations. As the current study was a first exploratory study, it is important to increase our knowledge on this topic with future research. In the end this will hopefully lead to successful
implementation of the warning tool in society, ultimately resulting in diminished effects of unreliable online information regarding vaccination and contributing to increasing the immunization coverage.
References
Adams, J. S. (1961). Reduction of cognitive dissonance by seeking consonant information. The Journal of Abnormal and Social Psychology, 62(1), 74-78. doi:10.1037/h0047029 Adams, S. A. (2010). Revisiting the online health information reliability debate in the wake of
“web 2.0”: An inter-disciplinary literature and website review. International Journal of Medical Informatics, 79(6), 391-400. doi:10.1016/j.ijmedinf.2010.01.006
Cotten, S. R., & Gupta, S. S. (2004). Characteristics of online and offline health information seekers and factors that discriminate between them. Social Science & Medicine, 59(9), 1795-1806. doi:10.1016/j.socscimed.2004.02.020
Diviani, N., & Meppelink, C. S. (2017). The impact of recommendations and warnings on the quality evaluation of health websites: An online experiment. Computers in Human Behavior, 71, 122-129. doi:10.1016/j.chb.2017.01.057
Ecker, U. K., Lewandowsky, S., & Tang, D. T. (2010). Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & Cognition, 38(8), 1087-1100. doi:10.3758/mc.38.8.1087
Elliott, G. C. (1979). Some effects of deception and level of self-monitoring on planning and reacting to a self-presentation. Journal of Personality and Social Psychology, 37(8), 1282-1292. doi:10.1037/0022-3514.37.8.1282
Flanagin, A. J., & Metzger, M. J. (2000). Perceptions of internet information credibility. Journalism & Mass Communication Quarterly, 77(3), 515-540.
doi:10.1177/107769900007700304
Gilkey, M. B., Magnus, B. E., Reiter, P. L., McRee, A., Dempsey, A. F., & Brewer, N. T. (2014). The vaccination confidence scale: A brief measure of parents’ vaccination beliefs. Vaccine, 32(47), 6259-6265. doi:10.1016/j.vaccine.2014.09.007
Godlee, F., Smith, J., & Marcovitch, H. (2011). Wakefield's article linking MMR vaccine and autism was fraudulent. British Medical Journal, 342(7788), 64-66.
doi:10.1136/bmj.c7452
Hong, J., & Lee, A. Y. (2008). Be fit and be strong: Mastering self-regulation through regulatory fit. Journal of Consumer Research, 34(5), 682-695. doi:10.1086/521902 Hwang, Y., & Jeong, S. (2016). “This is a sponsored blog post, but all opinions are my own”:
The effects of sponsorship disclosure on responses to sponsored blog posts. Computers in Human Behavior, 62, 528-535. doi:10.1016/j.chb.2016.04.026
Idris, N. H., Jackson, M. J., & Abrahart, R. J. (2011, January). Colour coded traffic light labelling: A visual quality indicator to communicate credibility in map mash-up applications. Paper presented at International Conference on Humanities, Social Sciences, Science & Technology, Manchester.
Internet; toegang, gebruik en faciliteiten. (2019). Retrieved from
https://opendata.cbs.nl/statline/#/CBS/nl/dataset/83429NED/table?dl=16DA3 Jolley, D., & Douglas, K. M. (2014). The effects of anti-vaccine conspiracy theories on
vaccination intentions. PLoS ONE, 9(2). doi:10.1371/journal.pone.0089177 Kata, A. (2010). A postmodern pandora's box: Anti-vaccination misinformation on the
Internet. Vaccine, 28(7), 1709-1716. doi:10.1016/j.vaccine.2009.12.022
Khazaal, Y., Chatton, A., Zullino, D., & Khan, R. (2011). HON label and DISCERN as content quality indicators of health-related websites. Psychiatric Quarterly, 83(1), 15-27. doi:10.1007/s11126-011-9179-x
Klayman, J. (1995). Varieties of confirmation bias. Psychology of Learning and Motivation, 32, 385-418. doi:10.1016/s0079-7421(08)60315-1
Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012).
Psychological Science in the Public Interest, 13(3), 106-131. doi:10.1177/1529100612451018
Ludolph, R., Allam, A., & Schulz, P. J. (2016). Manipulating Google’s knowledge graph box to counter biased information processing during an online search on vaccination: Application of a technological debiasing strategy. Journal of Medical Internet Research, 18(6), e137. doi:10.2196/jmir.5430
Masaryk, R., & Hatoková, M. (2016). Qualitative inquiry into reasons why vaccination messages fail. Journal of Health Psychology, 22(14), 1880-1888.
doi:10.1177/1359105316656770
McMaster, C., & Lee, C. (1991). Cognitive dissonance in tobacco smokers. Addictive Behaviors, 16(5), 349-353. doi:10.1016/0306-4603(91)90028-g
Meppelink, C. S., Smit, E. G., Fransen, M. L., & Diviani, N. (2019). “I was right about vaccination”: Confirmation bias and health literacy in online health information seeking. Journal of Health Communication, 24(2), 129-140.
doi:10.1080/10810730.2019.1583701
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. doi:10.1037//1089-2680.2.2.175 Norman, C. D., & Skinner, H. A. (2006). eHEALS: The eHealth literacy scale. Journal of
Medical Internet Research, 8(4), e27. doi:10.2196/jmir.8.4.e27
Nyhan, B., Reifler, J., Richey, S., & Freed, G. L. (2014). Effective messages in vaccine promotion: A randomized trial. Pediatrics, 133(4), e835-e842. doi:10.1542/peds.2013-2365
O’Reilly, T. (2009). What is Web 2.0. O’Reilly Media.
Pluviano, S., Watt, C., & Della Sala, S. (2017). Misinformation lingers in memory: Failure of three pro-vaccination strategies. PLOS ONE, 12(7). doi:10.1371/journal.pone.0181640
RIVM (2018). De pijlers van het Rijksvaccinatieprogramma. Retrieved March 15, 2019, from https://www.rivm.nl/pijlers-van-rijksvaccinatieprogramma
RIVM (2019). Vaccinatieschema. Retrieved March 15, 2019, from https://rijksvaccinatieprogramma.nl/vaccinaties/vaccinatieschema
Ruijs, W. L. (2013). Acceptatie van vaccinatie in de reformatorische gezindte. Retrieved April 12, 2019, from https://www.rivm.nl/acceptatie-van-vaccinatie-in-reformatorische-gezindte
Rusman, F. (2018, November 27). 'De discussie over vaccinaties is asymmetrisch, dat frustreert me mateloos'. NRC. Retrieved from https://www.nrc.nl/
Shao, C., Ciampaglia, G. L., Flammini, A., & Menczer, F. (2016). Hoaxy: A platform for tracking online misinformation. Proceedings of the 25th International Conference Companion on World Wide Web - WWW '16 Companion, 745-750.
doi:10.1145/2872518.2890098
Shelby, A., & Ernst, K. (2013). Story and science: How providers and parents can utilize storytelling to combat anti-vaccine misinformation. Human Vaccines &
Immunotherapeutics, 9(8), 1795-1801. doi:10.4161/hv.24828
Smith, I. M., & MacDonald, N. E. (2017). Countering evidence denial and the promotion of pseudoscience in autism spectrum disorder. Autism Research, 10(8), 1334-1337. doi:10.1002/aur.1810
van Lier, E. A., Geraedts, J. L. E., Oomen, P. J., Giesbers, H., van Vliet, J. A., Drijfhout, I. H., … de Melker, H. E. (2018). Vaccinatiegraad en jaarverslag Rijksvaccinatieprogramma Nederland 2017. [Immunisation coverage and annual report National Immunisation programme in the Netherlands 2017]. (No. 2018-0008). Bilthoven, The Netherlands: Rijksinstituut voor Volksgezondheid en Milieu. doi: 10.21945/RIVM-2018-0008
Vermeulen, M. (2015, March 28). Het brein werkt als een advocaat. Het negeert bewijs dat niet van pas komt. De Volkskrant. Retrieved from https://www.volkskrant.nl/ Wilson, K., Mills, E. J., Norman, G., & Tomlinson, G. (2005). Changing attitudes towards
polio vaccination: A randomized trial of an evidence-based presentation versus a presentation from a polio survivor. Vaccine, 23(23), 3010-3015.
Appendix A: Stimulus Material
Appendix B: Questionnaire Welkom!
Ik nodig u graag uit om deel te nemen aan een onderzoek dat uitgevoerd wordt onder de Graduate School of Communication, onderdeel van de Universiteit van Amsterdam.
De titel van de studie waaraan ik u vraag deel te nemen is ‘Vaccinatie informatie’. In de online vragenlijst zult u vragen beantwoorden over vaccinaties en online informatie bekijken en lezen. Alleen ouders die in verwachting zijn van een kindje of die een kind van maximaal 9 jaar oud hebben mogen deelnemen aan dit onderzoek. Het doel van dit onderzoek is om inzicht te krijgen in de mening van ouders over vaccinaties en de manier waarop zij online informatie zoeken. Het onderzoek zal ongeveer 15 minuten duren.
Dit onderzoek wordt uitgevoerd onder de verantwoordelijkheid van ASCoR, Universiteit van Amsterdam. Hierdoor kunnen wij garanderen dat:
1. Uw anonimiteit bewaakt zal worden en dat uw persoonlijke informatie niet doorgespeeld zal worden naar derden onder wat voor omstandigheden dan ook, tenzij u hier expliciet toestemming voor heeft gegeven.
2. U kunt weigeren deel te nemen aan dit onderzoek of uw deelname kunt afbreken zonder daar een reden voor te hoeven geven. Bovendien heeft u tot 7 dagen na deelname om uw toestemming om uw data voor dit onderzoek te gebruiken in te trekken.
3. U tijdens uw deelname niet blootgesteld zal worden aan enig risico of ongemak, dat de onderzoekers u niet bewust zullen misleiden en dat u niet blootgesteld zal worden aan enig expliciet aanstootgevend materiaal.
Voor meer informatie over dit onderzoek en de uitnodiging om deel te nemen bent u welkom om ten alle tijden contact op te nemen met de onderzoeksleider Leonie Westerbeek via leonie.westerbeek@live.nl, 0623757904, post-adres: Amsterdam School of Communication Research (ASCoR) Postbus 15791 1001 NG Amsterdam.
Indien u klachten of opmerkingen heeft over het onderzoek en eventuele gevolgen van uw deelname in dit onderzoek kunt u contact opnemen met het aangewezen lid van de Ethische Commissie die ASCoR vertegenwoordigd, via ASCoR Secretariaat, Ethics Committee, University of Amsterdam, Postbus 15793, 1001 NG Amsterdam; 020-5253680, ascor-secr-fmg@uva.nl. Alle klachten en opmerkingen worden uiterst vertrouwelijk behandeld.
We hopen dat we u van genoeg informatie voorzien hebben. We willen dit graag als kans aangrijpen om u bij voorbaat hartelijk te bedanken voor uw hulp bij dit onderzoek, dit wordt zeer gewaardeerd.
Met vriendelijke groet, Leonie Westerbeek
Hierbij verklaar ik dat ik duidelijk geïnformeerd ben over de aard en methode van dit onderzoek, zoals beschreven in de hiervoor weergeven uitnodiging.
Ik ga, volledig en vrijwillig, akkoord met deelname aan dit onderzoek. Hiermee behoud ik het recht om mijn deelname in te trekken, zonder hier een reden voor te geven. Ik ben me er
bewust van dat ik mijn deelname aan dit onderzoek op ieder moment mag stoppen.
Als mijn onderzoeksresultaten gebruikt worden in wetenschappelijke publicaties of op enige andere manier publiek gemaakt worden, wordt dit gedaan op een manier waarbij mijn
anonimiteit volledig beschermd blijft. Mijn persoonlijke data zullen niet doorgespeeld worden naar derden zonder mijn expliciete toestemming.
Als ik meer informatie over dit onderzoek wil ontvangen, nu of in de toekomst, kan ik contact opnemen met Leonie Westerbeek via leonie.westerbeek@live.nl. Als ik klachten of
opmerkingen heb over dit onderzoek kan ik contact opnemen met het aangewezen lid van de Ethische Commissie die ASCoR vertegenwoordigd, via ASCoR Secretariaat, Ethics
Committee, University of Amsterdam, Postbus 15793, 1001 NG Amsterdam; 020-5253680, ascor-secr-fmg@uva.nl.
Ik begrijp de tekst die hierboven weergeven wordt en ik ga akkoord met deelname aan het onderzoek.
We zijn benieuwd naar uw mening over vaccinaties. U gaat nu een aantal vragen
beantwoorden over vaccinaties. Er zijn hierin geen goede of foute antwoorden, probeer het antwoord te kiezen dat het dichtst bij uw mening ligt.
Voor de volgende stellingen over vaccinaties geeft u op een schaal van 1 tot 7 aan in hoeverre u het met elke stelling eens bent. Hierbij betekent 1: Helemaal mee oneens en 7: Helemaal mee eens.
Vaccinaties zijn noodzakelijk om de gezondheid van kinderen te beschermen.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Vaccinaties zijn goed in het voorkomen van ziekten.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Vaccinaties zijn veilig.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Als ik mijn kind niet vaccineer kan hij of zij een ziekte krijgen zoals de mazelen. Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Kinderen krijgen te veel vaccinaties.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Als ik mijn kind vaccineer kan hij of zij ernstige bijwerkingen krijgen.
Artsen op het consultatiebureau hebben het beste met mijn kind voor.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Ik heb een goede relatie met de arts op het consultatiebureau.
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens
We zijn benieuwd naar de manier waarop u online informatie zoekt over vaccinaties. U krijgt straks een Google pagina te zien van een zoekopdracht over vaccinaties. We willen u vragen een link uit te kiezen die u normaal gesproken aan zou klikken wanneer u op zoek zou zijn naar informatie over vaccinaties. Hierna klikt u op het pijltje om verder te gaan, dan zal de informatie die bij de link hoort gepresenteerd worden. Neemt u de tijd om deze informatie te lezen. Wanneer u klaar bent klikt u onderaan de pagina op het pijltje en vragen we naar uw mening over de informatie. In totaal zult u drie links kiezen om meer informatie over te lezen. (Het stoplicht naast de link geeft aan hoe betrouwbaar de informatie is. Hierbij betekent een groen stoplicht betrouwbaar en een rood stoplicht minder betrouwbaar. Met betrouwbaar bedoelen wij dat de informatie op waarheid berust en te vertrouwen is.)
U mag nu 1 link kiezen die u aan zou klikken tijdens het zoeken naar informatie over vaccinaties. Wanneer u een link gekozen heeft klikt u op het pijltje om verder te gaan.
U mag zelf weten hoe lang u deze informatie wilt bekijken/lezen. Wanneer u klaar bent klikt u op het pijltje om verder te gaan.
*Informatie die bij de gekozen link hoort*
De volgende standpunten gaan over de informatie die u zojuist gelezen heeft. Selecteer het antwoord dat uw mening het beste weergeeft. In hoeverre bent u het eens met de volgende standpunten?
De informatie die ik zojuist gelezen heb vind ik…
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens
Geloofwaardig 1 2 3 4 5 6 7
Nauwkeurig 1 2 3 4 5 6 7
Betrouwbaar 1 2 3 4 5 6 7
Eenzijdig 1 2 3 4 5 6 7
Compleet 1 2 3 4 5 6 7
De informatie die ik zojuist gelezen heb vind ik…
Slecht ○ ○ ○ ○ ○ ○ ○ Goed
Niet informatief ○ ○ ○ ○ ○ ○ ○ Informatief
Oninteressant ○ ○ ○ ○ ○ ○ ○ Interessant
Ongeloofwaardig ○ ○ ○ ○ ○ ○ ○ Geloofwaardig
Nutteloos ○ ○ ○ ○ ○ ○ ○ Nuttig
Wat vindt u van de informatie die u zojuist gelezen heeft?
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Deze informatie is relevant voor mijn situatie 1 2 3 4 5 6 7
Deze informatie is belangrijk voor mijn vaccinatie keuze 1 2 3 4 5 6 7 Ik heb iets aan deze informatie 1 2 3 4 5 6 7 Deze informatie is nuttig om te gebruiken 1 2 3 4 5 6 7
Waarom heeft u deze link aangeklikt? U mag hier meerdere antwoorden selecteren. ○ Omdat de informatie aansluit bij mijn mening over vaccinaties
○ Omdat de informatie niet aansluit bij mijn mening over vaccinaties
○ Omdat de informatie mij juist leek en ik benieuwd was wat er gezegd werd ○ Omdat de informatie mij onjuist leek en ik benieuwd was wat er gezegd werd ○ Omdat ik deze informatie nog niet eerder gehoord had
○ Anders, namelijk…
U mag nu nogmaals 1 link kiezen die u aan zou klikken tijdens het zoeken naar informatie over vaccinaties. Wanneer u een link gekozen heeft klikt u op het pijltje om verder te gaan.
U mag zelf weten hoe lang u deze informatie wilt bekijken/lezen. Wanneer u klaar bent klikt u op het pijltje om verder te gaan.
*Informatie die bij de gekozen link hoort*
De volgende standpunten gaan over de informatie die u zojuist gelezen heeft. Selecteer het antwoord dat uw mening het beste weergeeft. In hoeverre bent u het eens met de volgende standpunten?
De informatie die ik zojuist gelezen heb vind ik…
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens
Geloofwaardig 1 2 3 4 5 6 7
Nauwkeurig 1 2 3 4 5 6 7
Betrouwbaar 1 2 3 4 5 6 7
Eenzijdig 1 2 3 4 5 6 7
De informatie die ik zojuist gelezen heb vind ik…
Slecht ○ ○ ○ ○ ○ ○ ○ Goed
Niet indrukwekkend ○ ○ ○ ○ ○ ○ ○ Indrukwekkend
Niet informatief ○ ○ ○ ○ ○ ○ ○ Informatief
Oninteressant ○ ○ ○ ○ ○ ○ ○ Interessant
Ongeloofwaardig ○ ○ ○ ○ ○ ○ ○ Geloofwaardig
Nutteloos ○ ○ ○ ○ ○ ○ ○ Nuttig
Wat vindt u van de informatie die u zojuist gelezen heeft?
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Deze informatie is relevant voor mijn situatie 1 2 3 4 5 6 7
Deze informatie is belangrijk voor mijn vaccinatie keuze 1 2 3 4 5 6 7 Ik heb iets aan deze informatie 1 2 3 4 5 6 7 Deze informatie is nuttig om te gebruiken 1 2 3 4 5 6 7
Waarom heeft u deze link aangeklikt? U mag hier meerdere antwoorden selecteren. ○ Omdat de informatie aansluit bij mijn mening over vaccinaties
○ Omdat de informatie niet aansluit bij mijn mening over vaccinaties
○ Omdat de informatie mij juist leek en ik benieuwd was wat er gezegd werd ○ Omdat de informatie mij onjuist leek en ik benieuwd was wat er gezegd werd ○ Omdat ik deze informatie nog niet eerder gehoord had
○ Anders, namelijk…
U mag nu voor de laatste keer 1 link kiezen die u aan zou klikken tijdens het zoeken naar informatie over vaccinaties. Wanneer u een link gekozen heeft klikt u op het pijltje om verder te gaan.
U mag zelf weten hoe lang u deze informatie wilt bekijken/lezen. Wanneer u klaar bent klikt u op het pijltje om verder te gaan.
*Informatie die bij de gekozen link hoort*
De volgende standpunten gaan over de informatie die u zojuist gelezen heeft. Selecteer het antwoord dat uw mening het beste weergeeft. In hoeverre bent u het eens met de volgende standpunten?
De informatie die ik zojuist gelezen heb vind ik…
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens
Geloofwaardig 1 2 3 4 5 6 7
Nauwkeurig 1 2 3 4 5 6 7
Eenzijdig 1 2 3 4 5 6 7
Compleet 1 2 3 4 5 6 7
De informatie die ik zojuist gelezen heb vind ik…
Slecht ○ ○ ○ ○ ○ ○ ○ Goed
Niet indrukwekkend ○ ○ ○ ○ ○ ○ ○ Indrukwekkend
Niet informatief ○ ○ ○ ○ ○ ○ ○ Informatief
Oninteressant ○ ○ ○ ○ ○ ○ ○ Interessant
Ongeloofwaardig ○ ○ ○ ○ ○ ○ ○ Geloofwaardig
Nutteloos ○ ○ ○ ○ ○ ○ ○ Nuttig
Wat vindt u van de informatie die u zojuist gelezen heeft?
Helemaal mee oneens 1 2 3 4 5 6 7 Helemaal mee eens Deze informatie is relevant voor mijn situatie 1 2 3 4 5 6 7
Deze informatie is belangrijk voor mijn vaccinatie keuze 1 2 3 4 5 6 7 Ik heb iets aan deze informatie 1 2 3 4 5 6 7 Deze informatie is nuttig om te gebruiken 1 2 3 4 5 6 7
Waarom heeft u deze link aangeklikt? U mag hier meerdere antwoorden selecteren. ○ Omdat de informatie aansluit bij mijn mening over vaccinaties
○ Omdat de informatie niet aansluit bij mijn mening over vaccinaties
○ Omdat de informatie mij juist leek en ik benieuwd was wat er gezegd werd ○ Omdat de informatie mij onjuist leek en ik benieuwd was wat er gezegd werd ○ Omdat ik deze informatie nog niet eerder gehoord had
○ Anders, namelijk…
U bent bijna bij het einde van de vragenlijst! Er volgt nu een aantal vragen over vaccinaties en over uzelf. Onthoud dat er geen goede of foute antwoorden zijn, probeer eerlijk te
antwoorden.
Bent u of is uw partner in verwachting? o Ja
o Nee
Heeft u al kinderen? o Ja, ik heb één kind o Ja, ik heb twee kinderen o Ja, ik heb drie kinderen o Ja, ik heb vier kinderen
o Ja, ik heb meer dan vier kinderen
o Ik heb nog geen kinderen, ik ben nu wel in verwachting van een kindje o Nee, ik heb geen kinderen en ik ben ook niet in verwachting
Hoe oud is uw jongste kind? Als u of uw partner in verwachting is wordt dit beschouwd als uw jongste kind.
o Nog ongeboren (ik ben in verwachting) o Jonger dan 6 weken
o 7 weken – 1 jaar o 2 – 3 jaar o 4 – 5 jaar o 6 – 7 jaar o 8 – 9 jaar o 10 jaar of ouder
Wat is de leeftijd van uw oudere kind(eren)? U kunt hier meerdere antwoorden selecteren indien van toepassing.
o 7 weken – 1 jaar o 2 – 3 jaar o 4 – 5 jaar o 6 – 7 jaar o 8 – 9 jaar o 10 jaar of ouder
Bent u van plan uw kind te laten vaccineren volgens het Rijksvaccinatieprogramma? Als u meerdere kinderen heeft, houdt hierbij dan uw jongste kind in gedachten. Als u in
verwachting bent wordt dit beschouwd als uw jongste kind.
o Ja, ik ben van plan mijn kind volgens het Rijksvaccinatieprogramma te vaccineren. o Nee, ik ben van plan een aangepast vaccinatieschema te volgen voor mijn kind. o Nee, ik ben van plan mijn kind helemaal niet te laten vaccineren.
o Anders, namelijk…
Laat u uw oudere kind(eren) vaccineren volgens het Rijksvaccinatieprogramma? o Ja, ik laat mijn kind(eren) volgens het Rijksvaccinatieprogramma vaccineren. o Nee, ik laat mijn kind(eren) volgens een aangepast vaccinatieschema vaccineren. o Nee, ik laat mijn kind helemaal niet vaccineren.
o Anders, namelijk ...
Laat u uw kind vaccineren volgens het Rijksvaccinatieprogramma? Als u meerdere kinderen heeft, houdt hierbij dan uw jongste kind in gedachten.
o Ja, ik laat mijn kind volgens het Rijksvaccinatieprogramma vaccineren. o Nee, ik laat mijn kind volgens een aangepast vaccinatieschema vaccineren. o Nee, ik laat mijn kind helemaal niet vaccineren.
