- 1 - Abstract
Vaccination rates are dropping in some countries due to the anti-vaccination movement worldwide. To deal with this problem, scholars have been finding ways to develop persuasive strategies in combat with vaccine misinformation. Although the content of anti-vaccination messages constantly changes with healthcare-related news, they share the similar persuasive strategies. However, there are very few studies examine these attractive features with large volumes of data. There is also limited evidence that characteristics of vaccine misinformation being effective in promoting the persuasiveness of pro-vaccination messages.
Based on the context of Twitter, this study extends knowledge on the application of McGuire's communication/persuasion matrix by employing features of "successful" opponent campaigns (i.e. popular vaccine misinformation) on pro-vaccination message designing. The content analysis part of this study also shows the competences of computer-assisted automatic content analysis from big data retrieving to sentimental and textual analysis. The study finds that while popular vaccine misinformation on Twitter features in presence of pictures and
self-disclosure, those appealing characteristics are not necessarily effective in increasing people’s intention to like or share the post. Moreover, in combination of the health belief model (HBM), a following experiment shows that image cues in the media information would increase people’s perceived benefit of vaccination, while the presence of self-disclosure would make people more likely to get vaccinated. These findings add to a small but promising evidence of the
effectiveness of applying misinformation’s characteristics on health interventions and campaigns. Also, it reveals implications for designing pro-vaccination messages according to the audience’s literacy and social networks, since persuasive characteristics appeared to be more general for the highly-educated people participated in this study.
- 2 -
Key Words: vaccine misinformation, automated content analysis, pro-vaccination messages, perceived benefit, behavioural intention, self-disclosure, HPV vaccination, social media
Introduction
According to World Internet Users Statistics (2018), most people in North America and almost a half of the world’s populations are Internet users. Data from Pew Research Centre (PRC) suggests that a majority of US Internet users are looking for health-related information online (Mejova, 2018). As stated in PRC’s previous reports, 70% of them claimed that information online would influence their health-related decisions. However, much of the information they can found online is incorrect (Eysenbach, Powell, Kuss, & Sa, 2002). Being among the top threats to societies, misinformation has always been a public concern (World Economic Forum, 2013). Belief in misinformation would result in rejecting important correct information
(Lewandowsky & Oberauer, 2016). In this digital age, social media platforms are providing fertile ground for health-related misinformation’s spread to a large audience by peer-to-peer sharing, and thus posing threats to misinformed users’ medical decisions (Fernández-Luque & Bau, 2015; Ghenai & Mejova, 2017)
One of the topics of which much misinformation can be found online are vaccines (Kata, 2010). While vaccines prevented an increasing number of diseases, misinformation produced by online anti-vaccination movement, such as “vaccination causes autism”, remains a hindrance for advocating vaccination globally (Fernández-Luque & Bau, 2015; Kata, 2010). Vaccine
misinformation, it is often claimed, to be one of the chief culprits of vaccination hesitancy and the drop of vaccine rates in many regions (Dubé et al., 2013; Freed, Clark, Butchart, Singer, & Davis, 2010; Salathé & Khandelwal, 2011; Shelby & Ernst, 2013; Tafuri et al., 2014).
- 3 -
been playing an important role in resulting parents’ vaccination hesitancy for their children. In order to combat the influence of misinformation, scholars have been researching on methods to improve vaccination uptake. McGuire (1999, p.153) proposed an aid for persuasive health interventions from his communication/persuasion and input/output matrix. The matrix suggests that communicators could look into characteristics of the messages (e.g. type of appeal and argument structure), and further examine the persuasion outputs (e.g. liking, agreement, decision making). Some of the health communicators studied the characteristics of vaccine misinformation. Compared with correct information, vaccine misinformation is usually without scientific or statistical endorsement, but can still win more attention and shares from Internet users in this controversial debate (Faasse, Chatman, & Martin, 2016; Gesualdo, Zamperini, & Tozzi, 2018; Guidry, Carlyle, Messner, & Jin, 2015). Vaccine misinformation online is constantly changing with trends in public health and success of vaccination (Bean, 2011). Nonetheless, content characteristics of anti-vaccination websites indicated that this kind of information
usually shares similar persuasive techniques such as “vivid and compelling anecdotes or personal testimonials” (Moran, Lucas, Everhart, Morgan, & Prickett, 2016). On the other hand, some experimental health interventions have been launched to examine the persuasive output. Moran et al., (2016) did a pilot study of narrative messages in improving vaccine knowledge among people with low literacy. Commentaries were that applying “the narrative that accompanies the anti-vaccination messages” could be an alternative for communicating vaccine information to lay audiences (Goldstein, LeVasseur, & Purtle, 2017). The two consecutive studies conducted by Moran et al., is a first trial of finding persuasive features of vaccine misinformation and testing the effectiveness of their application on pro-vaccination messages. As a further step in this topic, researches about other appealing characteristics of online vaccine-related misinformation are
- 4 - warranted for better pro-vaccination communication.
The current research aims to examine some of the vaccine-related misinformation’s characteristics on Twitter in an automated-content analysis approach, as well as verifying their relationship with popularity of the tweets (i.e. number of likes, shares and comments). We decided to start with analyzing short messages online, while Twitter is a suitable source for its large volume of information and quantifiable popularity. Also, a following experiment testing to what extent characteristics of misinformation are effective in pro-vaccination messages will be conducted. By applying the influential features to advocate vaccination, we aim at testing the perceived benefit and behavioral intention towards vaccination after exposure to a designed message. We hereby propose the following research questions:
RQ1: What characteristics does vaccine misinformation on Twitter have, and do these characteristics make the tweet more likely to be liked, shared and commented?
RQ2: Will the presence of influential characteristics also work in increasing popularity of the message, people’s perceived vaccine benefit and intention to vaccinate?
Study 1
On the basis of analysing existing studies of online vaccine misinformation, this study introduces several prominent characteristics to provide better directions for a content analysis. Considering the special context of the social media platform Twitter, we first provide a brief introduction on user’s behaviour of like, share and comment.
Like, Share and Comment Behaviour on Twitter
Social media is continuously gaining its attention from mass communicators and researchers worldwide (Perrin, 2015). Activities such as sharing and liking in Twitter was regarded as a fast way to keep up with new events and share daily life with friends and
- 5 -
colleagues (Zhao & Rosson, 2009). Chen et al., (2015) mentioned that people with social needs like to share information on social media, even if the information is fake or false. Meanwhile, according to McGuire’s matrix (1999), the spread of vaccine (mis-)information online would increase people’s chance of exposure to these messages (i.e. persuasive output). To assess the popularity of a certain post, people often refer to its likes, shares and comments towards it (Alhabash, McAlister, Lou, & Hagerstrom, 2015). Therefore, in this study, we use the number of like, share (i.e. retweet) and comments to evaluate a vaccine-related tweets' performance.
Characteristics of Online Vaccine Misinformation
Sentiment. Preceding researchers have argued that information with negative sentiment could be related to the selective exposure of media users. Taylor (1991) has pointed out a phenomena named negativity bias: Even if with the same intensity, negative information would get more attention than that of a positive or neutral sentiment. Also, negative information weighs more heavily on the brain, especially those concerning possible health dangers (Ito, Larsen, Smith, & Cacioppo, 1998; Siegrist & Cvetkovich, 2001).
With the development of automated content analysis, sentiment analysis is a prevalent method used among researches of vaccine misinformation. Yet researchers lack in agreement on whether messages with negative, positive or neutral emotion are more popular (i.e. be liked, shared or received other interaction) online. In 2011 while the outburst of H1N1 (swine flu) pandemic has raised wide concern globally, several researchers applied sentiment analysis to vaccine-related tweets, exhibiting general sentimental disparities and their real-time proportions (Salathé & Khandelwal, 2011; Signorini, Segre, & Polgreen, 2011). However, they did not mention impact of sentiments on readers’ interactions.
- 6 -
found that videos holding an explicit negative tone were liked more by viewers, compared with positive or ambiguous ones. In addition, after being exposed to a majority of negative
information about HPV vaccination, users would three times more likely to post negative tweets compared with those exposed to positive or neutral tweets (Dunn, Leask, Zhou, Mandl, & Coiera, 2015). Somehow by contrast, as a more recent study indicated, though prevailing in
anti-vaccination messages, more negative words in vaccine-related websites did not significantly result in more reactions correspondingly (Xu & Guo, 2018). The ambivalence on the role of textual sentiment to some extent provided a hint for a close examination, hence we proposed a hypothesis as:
H1: Vaccine misinformation on Twitter is (a) more likely to have negative sentiment, and (b) more likely to be liked, shared and commented upon if the tweet has negative sentiment
compared with positive and neutral tweets.
Presence of Statistics. The presence of statistics is often compared with narrative stories or bottom-line information in prior studies of vaccine misinformation in previous researches. A quantitative empirical study of vaccine-related information on Pinterest suggested that among 800 posts, anti-vaccination information used significantly more narrative stories than statistic representations (Guidry et al., 2015). Another study on news articles indicated that vaccine-related information that is shared at least once usually contain statistics, but not necessarily indicate it will be shared more times compared with articles containing bottom-line information (Broniatowski, Hilyard, & Dredze, 2016). The authors have drawn upon the fuzzy-trace theory (FTT), which indicates that people comprehend information from two different processes: the precise data, and the bottom-line meanings (Reyna, 2008). To verify the role of statistics in vaccine misinformation, the second hypothesis was thus proposed as:
- 7 -
H2: Vaccine misinformation on Twitter is (a) less likely to contain statistics, and (b) less likely to be liked, shared and commented upon if it contains statistics compared with others who does not contain statistics.
Presence of Pictures. Pictures are widely acknowledged to contribute to health education (Chen & Dredze, 2018; Houts, Doak, Doak, & Loscalzo, 2006; McGuire, 1999). It has been revealed in a previous experiment that pictorial appeals, together with the participants’ imagery ability, jointly have a significant impact on participants’ vividness perception, recall, and attitude to HPV vaccines (Yang & Guo, 2015). Kata (2010) stated that pictures with “vaccine victims” and needles are common in anti-vaccination webpages. Similar features of these pictures has been concluded by a content analysis on Twitter, indicating that vaccine-related tweets with pictures are twice as likely to be shared as non-image tweets, and many often have embedded texts, as well as include images of people and syringes (Chen & Dredze, 2018). However, the authors did not distinguish between correct information and misinformation. To verify whether picture presence is prevalent in vaccine misinformation on Twitter, and whether pictures would bring higher popularity, we proposed the hypothesis as:
H3: Vaccine misinformation on Twitter is (a) more likely to contain pictures, and (b) more likely to be liked, shared and commented upon if it contains pictures compared with others who does not contain pictures.
Self-disclosure. In Kata’s analysis (2010), 88% of the analysed websites contained personal testimonies (i.e. stories of harmed children/personal experiences). There are few articles focusing on “personal issues” of vaccine misinformation, but research in other fields might provide hints for the importance of personal experiences in vaccine misinformation. Siminoff, Traino and Gordon (2011) analysed the comments under videos related to tissue donation, and found a
- 8 -
positive relation between self-disclosure in the message and people’s consent to donation. The interpersonal liking and trust was likely to be facilitated by “revealing personal information” (Collins & Miller, 1994). Therefore, we could infer that vaccine misinformation may had also employed this strategy to increase its credibility, and the following liking and trust into the information. Based on prior reviews, we thus reach another hypotheses that:
H4: Vaccine misinformation on Twitter is (a) more likely to contain self-disclosure, and (b) more likely to be liked, shared and commented upon if it contains self-disclosure compared with others who does not contain self-disclosure.
Self-disclosure is in the same category with imageries in Kata’s suggestion (2010): They both belong to the characteristics emotive appeals. It is possible that there might be a joint impact of these two features on the popularity of vaccine misinformation. Therefore, we have an additional hypothesis that:
H5: Vaccine misinformation with pictures will be more likely to be liked, shared and commented upon if it contains disclosure compared with others who does not contain self-disclosure.
Method Sampling
With twitterscraper tool (Taspinar, 2018), original tweets posted between September 1, 2015 and September 1, 2018 with specific hashtags were scraped. A set of 25 anti-vaccine hashtags was applied in the search string. These hashtags were identified by the study of Dredze et al., (2017), all of which are widely used in tweets with anti-vaccination sentiments (see Appendix I). A total of 118,984 English tweets were collected in the first stage. Since this research has a strong tendency into 2016 US elections, there were some hashtags that were not
- 9 -
directly related (e.g. #hearus, #wakeupamerica), tweets not mentioning any of vaccine-related words nor obvious vaccine-vaccine-related hashtags (e.g. #vaxfax, #vaccineinjury) were deleted from the dataset. The dictionary of vaccine-related English words and abbreviations was
retrieved from the official website of Centers for Disease Control and Prevention (CDC) (2018). The final sample contained 6,418 tweets and was analysed in this study.
Analysis
The automated content analysis was conducted with Python 3.5.2.
Sentiment of tweets. In this study, the sentiment analysis module TextBlob designed and updated by Loria et al., (2014) was applied to classify tweets into three categories: negative, neutral and positive by their sentiments. TextBlob is a simplified text processing tool developed on Natural Language Tool Kit (NLTK), which is a widely used platform in natural language processing with python (Giatsoglou et al., 2017). Prior studies has already been applying this package in tweets’ sentiment analysis, for its ability in detecting emotional differences in short sentences (Kumar Singh, Kumar Gupta, & Mohan Singh, 2017). In TextBlob the sentiment polarity is calculated into parameters varying from -1 to 1, where -1 indicates strong negative sentiment and 1 stands for strong positive emotion. To identify the sentiment status of each tweet, tweets with negative scores were regarded as negative, while those with positive scores were positive and with score zero as neutral.
Presence of images and data. By matching specific strings in our sample, tweets with pictures or statistics were filtered out. Regular expressions, as a time-saving approach in
computer sciences for searching specific texts with their “descriptions” (Goyvaerts & Levithan, 2012) was applied in both searches. We search the presence of default format of images’ links in twitter (i.e. pic.twitter/) for tweets with picture illustrations. To avoid disruptions from numbers
- 10 -
in the links, usernames or hashtags, these elements were all removed before detecting the presence of data in each tweet’s content.
Self-disclosure. In this study, self-disclosure was detected by two separate approaches. First, a simple literal search for singular first-person pronouns (i.e. I/me/my/mine/myself) was searched for first-person expressions in tweets. This approach has been applied in previous researches to indicate whether there was self-enclosure in health blogs (Holtgraves, 1990; Rains, 2014) It should be noted that multiple first-person words do not necessarily means
self-disclosure, since the speakers may also refer to a third party when they mention “we” (Stiles, 1979). Second, other signs of self-disclosure in social psychology, namely nonverbal openness and emotional openness were also examined (Montgomery, 1982; Bak, Kim & Oh, 2012). Nonverbal openness includes facial expressions, gestures, voice tones, etc. Since Twitter is limited within texts and images, we adapted the way of Bak et al., in detecting nonverbal openness with emoticons in tweets, all of which were retrieved from Wikipedia.org. For emotional openness, we looked for common words for expressing feelings in these tweets, the list of which were retrieved from Psychpage (“List of Feeling Words,” 2018). Each tweet that mentioned either of the two criteria above was concluded as tweets with self-disclosure.
Number of like, share and comments. Throughout this paper, the term share refers to retweet (i.e. sharing posts within Twitter). And the like and comments refer to the literal meaning as presented in Twitter. However, these original data was rarely normally distributed and thus posed difficulties for statistical analysis. A large amount of tweets received no like, comment or share, while some tweets exceeded 1,000 in these indexes. A common way to reduce this
variability is to make a log transformation (Feng et al., 2014). In this study, likes, comments and shares were transformed in the following way and put into statistical analysis:
- 11 - x1=lg(x0+1)
The original data, for example, number of likes of each tweet, varied from 0 to 1,765; the disparity was sharply reduced after the transformation (i.e. from 0 to around 3.25).
A random sample of 200 twitter pictures was also inspected from this anti-vaccination tweet dataset. Apart from unrelated pictures such as health-unrelated commercial advertisements, the general observation was in coherence with prior studies, suggesting that these images usually have embedded texts, people (e.g. child(ren)) or syringes (Chen & Dredze, 2018).
Result Table 1:
ANOVA results of Like, Share and Comments’ Dependence on Message Characteristics
Independent Variable Value Number of Tweets Dependent Variable
Like Share Comment
M SD M SD M SD Sentiment Negative 892 1.07 0.32 0.69 0.23 0.07 0.08 Neutral 3013 2.42 0.36 1.94 0.34 0.28 0.16 Positive 2423 0.86 0.24 0.73 0.25 0.13 0.11 Presence of Image(s) No 1041 1.99 0.34 1.73 0.33 0.26 0.16 Yes 5377 4.61* 0.45 3.01* 0.40 0.38* 0.18 Presence of data No 2390 1.85 0.33 1.49 0.31 0.19 0.14 Yes 4028 3.35* 0.41 2.69* 0.39 0.43* 0.20 Self- No 1174 2.14 0.35 1.81 0.34 0.23 0.15
- 12 - disclosure
Yes 5244 3.31* 0.40 2.35* 0.37 0.44* 0.20 Note: To be in consistency of the original popularity of tweets, mean values were calculated
from original data instead of reversely transforming from the data for statistical analysis. *p < .05, between group comparison
Table 1 provides an overview of data analysis of this part. The application of negative sentiments in anti-vaccination tweets did not result in more likes, shares and comments. Also, there are actually more tweets with positive or neutral sentiments. In general, there are no
significant differences in users’ like, share or comment activities with the variance of sentimental categories. H1 is thus fully rejected.
In addition to the tweets’ significant tendency of using pictures and statistics, people are more likely to share, like and comment on a tweet when it have either of these tendencies. The findings of statistical disclosure in tweets are not consistent with our hypothesis in that vaccine misinformation also likes to employ data information. Therefore, H2 is not supported, while H3 is supported.
By examining self-disclosure in tweet contents, we found that expressing emotions, disclosing feelings or referring to oneself in a tweet would make that tweet more likely to be liked, shared and commented. Therefore, H4 is supported. We also conduct a multi-factor between-subjects ANOVA analysis of presence of images and self-disclosure in the tweets, and found the joint impact of them upon number of shares approached marginal significance, F (1,6414) = 5.37, p = .07. However, tweets with images did not significantly have more likes F
(1,6414) = 2.57, p = .28, or comments F (1,6414) = .41, p = .72, when they contain
- 13 - partly supported.
Conclusion
In light of the content analysis results above, vaccine misinformation on Twitter features in pictures, statistics and self-disclosure, and these features to different extents increased their popularity (i.e. like, share and comments). We found that whether the sentiment is negative, positive or neutral does not necessarily influence the number of shares of a certain tweet. Presence of images, data and self-disclosure in the tweet content promoted a significant rise in the number of like, share and comments. In addition, a combination of self-disclosure and picture disclosure is more effective in increasing shares than plain pictures without
self-disclosure. The results indeed indicate a positive impact of statistics on like, share and comments of tweets, but according to the research by Broniatowski et al., (2016), they defined information with precise details such as statistics as “verbatim”. Nonetheless, the concept of “statistics” is so broad that it is difficult to define – it may be the date, the number of deaths, the cure rate or even the addresses. Since we only regard “presence of numbers” as the criteria of statistics disclosure in the content analysis above, it is not convincing enough whether “verbatim” misinformation is more prevalent and popular than others. Hence we abandoned this variable in the following research.
Study 2
The characteristics of online vaccine misinformation would shed light on offline practices – as McGuire’s (1999) suggested for the application of his matrix, we could diagnose how the appealing characteristics works with examining the persuasion outputs. Scholars have invoked that debunking misinformation is not enough and even counterproductive for promoting vaccines (Gesualdo et al., 2018), while presenting the pro-vaccination message with features
- 14 -
accompanying misinformation to the lay-audience would possibly get the public listen to it (Goldstein et al., 2017). These all indicated that pro-vaccination information might benefit from the appealing characteristics of misinformation to make it more acceptable by the audiences. Therefore, to further verify the performance of these appealing features, we conducted a following experiment on whether these features will work on pro-vaccination messages. Liking and Proselytizing of the Pro-vaccination Messages
Given the result of Study 1, misinformation messages containing certain characteristics usually increase the likelihood of users’ liking, sharing and commenting behaviour on Twitter. In evaluating the persuasive outputs of an intervention (especially here on social media), one could refer to people’s liking and proselytizing (i.e. sharing ideas to others) behaviour after being exposed to it. In the Twitter context, liking could be interpreted as its literal meaning, while comments and shares would reflect people’s proselytization of this message. To be consistent
with our first study, we thus propose:
H6: In pro-vaccination tweets, presence of (a) images and/or (b) self-disclosure will make the post more popular (i.e. with more like, share and comments), compared with those who do not have these characteristics.
Persuasive Strategies in Media information
Scholars have argued that health communicators will gain more from researching pro-vaccination information’s actual influence on people’ pro-vaccination behaviour, compared with only focusing on messages online (Betsch et al., 2012). Apart from this online context in Twitter, examining whether the message would actually result in audiences’ cognitive and attitudinal responses to vaccination is also necessary. To address the research gap, we planned to take the further steps in verifying people’s reactions after exposed to designed pro-vaccination message.
- 15 -
The health belief model (HBM) presents some criteria for evaluating people’s reaction to health interventions: Cues of action (e.g. media information) would lead to the output of people’s perceived benefit and the likelihood of action (e.g. vaccinate) (Briones et al., 2012; Rosenstock, Strecher, & Becker, 1988). Our study seeks to examine people’s perceived benefit and
behavioural intention to vaccination after being exposed to specifically designed pro-vaccine messages (i.e. with the appealing features of misinformation like pictures and self-disclosure). In the first part, we conducted a content analysis, revealing some tentative strategies we may use in developing pro-vaccination messages. As a cue of action, people’s perceived benefit and
behavioural intention would be directly or intermediately influenced by media information they are exposed to (Rosenstock et al., 1988). In combination of McGuire’s input/output matrix (1999), we could propose that the persuasive strategies (i.e. information characteristics in this study) could alter the outputs of people’s perceived vaccine benefit and vaccination intention. Perceived Benefit of Vaccination
Perceived vaccination benefit is the most commonly reported facilitators of vaccination behaviour (Rambout, Tashkandi, Hopkins, & Tricco, 2014). Besides, it is also considered as associated with both vaccination intent and behaviour (Juraskova, Bari, O’Brien, & McCaffery, 2011). Avoidance of an epidemic of preventable diseases or minimizing the risk of contracting such dangerous illnesses could both be considered as perceived benefit (Song, 2014).
Specifically in the case of HPV vaccines, perceived benefit is the belief that the HPV vaccine will reduce the likelihood or severity of HPV infection or cervical cancer (Brewer & Fazekas, 2007).
Researchers have found that misleading information about vaccines online will lead to significant misperceptions of vaccines, especially increased perceived risks and reduce
- 16 -
vaccination intention, for example among high-school students (Kortum, Edwards, & Richards-Kortum, 2008), and pregnant women (Fabry, Gagneur, & Pasquier, 2011). The number of pregnant women willing to take vaccination went through a sharp decrease when they are
exposed to online vaccine misinformation and their risk perception of vaccination also increased. Previous studies on effects of vaccine-related information mainly emphasize people’s perceived risk of vaccinating, attitude towards vaccination and behaviour intention (Nan & Madden, 2012). In contrast, perceived benefit (i.e. perceived effectiveness in some studies such as Brewer & Fazekas, 2007) is the other side of the coin compared with perceived risk: people will more likely to think vaccination useful when they overweigh the benefit to risk, and to refuse this idea when they overweigh risk to benefit (Song, 2014). Compared with anti-vaccination
information’s nature in addressing risks of vaccination, pro-vaccination information could in turn focus on the benefits.
Meagre scholar attention has been paid to study the relationship between features of
information and perceived benefit of vaccination. Moran et al., (2016) conducted an experiment for HPV educational materials among population with low health literacy, while narrative materials (i.e. narrative films) resulted in participants’ higher knowledge gains of the material, compared with non-narrative films. This could provide hint for the current study that by
manipulating specific features of the message delivered to the audience, they would learn more about the health-related issue. In Nan and Madden’s study (2012), though also discussing HPV vaccines, pro-vaccination blogs did not significantly rise the participants’ perceived vaccine safety nor positive attitudes to the vaccine, in comparison with the reverse effects of anti-vaccination blogs.
- 17 -
On the other hand, positive behavioural intention is also regarded by many studies as the ultimate goal of e-health communications (Ahern, Kreslake, & Phalen, 2006). It is summarized that higher perceived benefit of vaccination will increase people’s corresponding behavioural intention in early reviews (Brewer & Fazekas, 2007). However, recent researches have proved several failures in this positive relationship of perceived benefit and behavioural intentions. Nan and Madden’s study on HPV vaccination (2012) also revealed that negative information towards vaccines would lead to decrease of vaccination intention whether it was free of charge or at the current price, but in sharp contrast, positive information did not increase people’s behavioural intentions though it could increase people’s perceived efficacy of vaccines. There are also existing studies reporting strategies that did not work in promoting people’s health-related behaviour intention. In an online experiment about information credibility, whether the health-related information is credible or not did not significantly influence people’s behavioural intention to use sunscreen and consume raw milk (Hu & Sundar, 2010). Designed
pro-vaccination information was also proved in vain according to the experiment by Pluviano, Watt and Della Sala (2017), while they manipulated three strategies (i.e. myths and facts correction, visual correction and fear correction) towards their pro-vaccination messages, trying to increase people’s behavioural intention.
In Study 2, we thus proposed hypotheses as:
H7: A pro-vaccination post with an image will lead to (a) higher perceived vaccine benefit and (b) higher vaccination intention than a post without any images.
H8: A pro-vaccination post with self-disclosure will lead to (a) higher perceived vaccine benefit and (b) higher vaccination intention vaccination than a post without a disclosure.
- 18 -
presence of images. Therefore, we raise another hypothesis as:
H9: A pro-vaccination post with pictures will lead to (a) higher perceived vaccine benefit and (b) higher vaccination intention than a pro-vaccination post without a self-disclosure, and this effect will be stronger for posts with disclosure compared to those without self-disclosure.
Experimental Design
A 2 between-subjects (picture disclosure: present vs. absent) x 2 between-subjects (self-disclosure: present vs. absent) design was applied to this study. Four groups in total were tested, and participants were randomly allocated to one group upon their participation (see Table 2). Table 2:
Groups under Four Conditions
DV: Perceived Vaccine Safety & Attitude towards Vaccination
Picture Disclosure
Self-disclosure
Absent Present
Absent Group 1 Group 3
Present Group 2 Group 4
In this study, we made corresponding stimuli about Human papillomavirus (HPV) vaccines in the format of Twitter posts. Being among the most prevalent viruses, HPV can cause human infections and fatal cancers afterwards, while HPV vaccines can prevent the most
common types of infection. According to CDC, although the best time to receive HPV
vaccination is between 11 and 12 years old, it is also recommended for girls and women age 13 through 26 years of age who have not yet been vaccinated or completed the vaccine series (CDC, 2016). Adult women aged between 18 and 26 would be suitable for this study, since they have
- 19 -
full capacity for act and also lay in the age limit of HPV vaccination.
Together with the experiment stimuli, we developed a questionnaire that contained 21 questions pertaining participants' existing attitudes to HPV vaccination, perceived benefit of getting vaccinated, intention of vaccination behaviour and some demographic questions (see Appendix II).
Participants
Prior to conducting this experiment, ethical clearance was obtained from Amsterdam School of Communication Research (ASCoR). The suggested HPV vaccination age is between 9 and 26 in the US according to CDC, therefore English-speaking adult women would be suitable populations. Participants were then recruited via e-mail advertisements and link-sharing
requesting them to fill in an online questionnaire. A comprehensive of 120 participants were planned to be recruited in the final sample, and randomly assigned to four different experimental groups (i.e. each group will equally have 30 participants allocated).
Manipulation of the Independent Variables
Presence of image: as one of the independent variables, this variable takes two levels: present/absent. Experimental conditions were thus manipulated as textual posts plus an image, in contrast with the other groups exposed to plain texts. Embedded text and people were used in this image considering both the feature of pictures in our sample and previous researches (Chen & Dredze, 2018), namely embedding extra texts and showing images of people. The detailed stimuli are presented in Appendix III. For the purpose of verifying whether the manipulations were successful, a manipulation check was conducted after the experiment. Manipulation check was applied in every condition with the same questions and choices for participants to decide whether they think the posts they just saw is with image(s) (e.g. “There was image disclosure in
- 20 -
the post I just saw.” with a seven-point Likert scale from “Strongly agree” to “Strongly disagree”). A pre-test of 15 samples revealed that the manipulation is successful, participants distinguished well between the stimuli with and without pictures, F (1, 14) = 180.15, p=.001.
Self-disclosure: the variable of self-disclosure also takes two levels: present versus absent. According to prior studies, emotive openness and non-verbal openness are two representative part of self-disclosure (Montgomery, 1982; Bak, Kim & Oh, 2012). To address the presentation of “first-person”, relative pronouns were also examined in the content analysis part of this study. Therefore, the experiment manipulation was an elaborated text with the first-person perspective, textual disclosures of the speakers’ feeling, and adequate emoticons. For accurate manipulation, we did a pre-test (along with testing the presence of pictures) to check the effectiveness of this self-disclosure. They were asked whether the tweet they just saw looked like someone telling their own story (e.g. “The post I just saw was someone revealing personal feelings, information and beliefs of herself.” with a seven-point Likert scale from “Strongly agree” to “Strongly disagree”). Also, the 15 participants successfully distinguished the presence of self-disclosure, F(1, 14) = 75.63, p =.001.
With these between-subject designed manipulations mentioned above, we thus generated with four different web view “Twitter posts”. We adapted a fact-checked (i.e. reliable) health news story about the success of HPV vaccination program in Australia in reducing prevalence of HPV into tweet-like text within 280 characters as the tweet without self-disclosure (Healthline, 2018),and integrated several tweets in the search string “I HPV” into a new tweet as the tweet with self-disclosure. The image attached to the tweet is adapted from a pro-vaccination picture from the World Health Originzation (WHO), featuring embedded text, people and also needles. In order to eliminate the influence of unnecessary variables, name of the user was set as the same.
- 21 -
Number of like, share and comments of all the posts are also set as the average number of popularity indicated in Study 1 (i.e. 4 likes, 3 shares and no comments) to ensure the heuristic popularity is the same.
Measurements of the Dependent Variables
Like, share and comment. In the experiment participants were exposed to fake posts, therefore they could make no actual response or interactions. Instead, we apply a five-point Likert scale to measure their intention to like, share and comment upon the post (e.g. “After reading the post, I would like to click on ‘like’ of this post” with scale from strongly agree to strongly disagree).
Perceived vaccine benefit. The perceived vaccine benefit was measured through adopting an established scale from Allen et al., (2009). Since the stimuli only focused on preventive benefits of HPV vaccination, the item was employed as “I think HPV vaccination can effectively prevent cervical cancer” in this study. A seven-point Likert Scale was used as parameters of agreement of this statement.
Behavioural intention. The measurement of behavioural intention is adapted from Nan and Madden’s design (2012), simulating two scenarios for participants that they are provided with HPV vaccines free of charge and charged with a market price. In both pricing conditions, participants were asked how likely they are to take HPV vaccination (1) soon, (2) today and (3) in the future with a seven-point scale. The score behavioural intention was calculated by the mean of these three questions. Scales measuring behavioural intention in the questionnaire were found to be highly reliable. Cronbach’s alpha for the 6 behavioural intention items was .93. Covariate
- 22 -
the covariate for the following reason. In previous experiments of specially designed pro-vaccination messages, pre-existing attitude is important when health communicators expose people to persuasive messages of vaccines. Pro-vaccination messages debunking the
misinformation of vaccines’ relationship with autism indeed reduced misperceptions, yet it decreased the intention to vaccinate among participants holding least favourable attitude to vaccines (Mathew, Mohan, & Kumar, 2014). However, it is still meaningful for trying out persuasive strategies in formulating pro-vaccination messages. Results from search engines would lead to changes of attitudes towards vaccination, whether their content is consistent with people’s original attitude or not (Lev-On & Hardin, 2008). Hence, before the presentation of the experiment stimuli, participants’ pre-existing attitude should be examined.
Similar to the test of perceived vaccine safety, the study measured participant’s attitude to vaccination through seven-level Likert scales adapted from existing studies (Askelson et al., 2010). Participants were asked whether they agree with vaccinating ‘‘is necessary,’’ ‘‘is a good idea,’’ and ‘‘is beneficial.’’. Attitude towards HPV vaccination was calculated by the mean score of these three questions. The Cronbach’s alpha (alpha = .90) also proved the reliability of attitude-measuring scales.
Results
There were a total of 122 participants having filled in the online questionnaire. Our sample was generally highly-educated, with the minimum education level of high-school degree obtained. 36.9% (n=45) of them have bachelor or equivalent degree and 46.7% (n=57) have a master’s education, while 1.6% (n=2) declared they have completed their doctoral degree. Besides, more than a half of the participants indicate their mother language as Chinese (51.6%, n=63), followed by Dutch (18.9%, n=23) and English (14.8%, n=18). Most participants were
- 23 -
resided in China (34.4%, n=42) and the Netherlands (32.0%, n=39), followed by the United Kingdom (9.8%, n=12) and the United States (6.6%, n=8).
Prior to analysing the co-variation effect of pre-existing attitude, a correlation analysis was first conducted between attitudes, intention to like, share and comment, perceived benefit and behavioural intention. There was no significant association between participants’ attitude to HPV vaccination and their intention to like, r(121) = .11, p = .247, or leave any comments on. r(121) =.14, p = .129, the pro-vaccination post. On the other hand, pre-existing attitude had
significant impacts on intention to share the tweet, r(121) =.20, p = .033, perceived benefit, r(121) =.37, p=.001, and behavioural intention, r(72) =.28, p=.019. Hence, attitude is regarded as
a co-variate only in analysing influence of picture and self-disclosure on participants’ intention to share, perceived benefit and behavioural intention.
To evaluate the main effects of picture and self-disclosure and their interactive effects, a multiple analysis of variance (MANOVA) was conducted. Table 3 presents the results.
24 Table 3:
MANOVA Results
Dependent Variables
Presence of Pictures Presence of Self-disclosure Joint impact of Pictures and Self-disclosure F df p partial η2 F df p partial η2 F df p partial η2 Like the Tweet 3.74 4,116 0.055 0.031 0.77 4,116 0.383 0.006 2.26 4,116 0.135 0.019 Share the Tweeta 1.80 4,116 0.182 0.015 4.14 4,116 0.044* 0.034 0.66 4,116 0.419 0.006 Comment on the Tweet 0.08 4,116 0.783 0.001 2.15 4,116 0.145 0.018 0.71 4,116 0.401 0.006 Perceived Benefitb 8.49 4,116 0.004* 0.068 1.86 4,116 0.176 0.016 0.47 4,116 0.493 0.004 Behavioral Intentionc 0.14 4,67 0.710 0.002 4.48 4,67 0.038* 0.063 4.73 4,67 0.033* 0.066
Covariates appearing in the model are evaluated at the following values: a: Attitude = 5.245179; b: Attitude = 5.245179; c: Attitude = 4.708333. *p<0.05 between group
25 Evaluating the Tweets’ Popularity
Hypothesis 6 suggested that if the designed pro-vaccination tweet contains picture and/or self-disclosure, it would be more likely to be liked, shared and commented upon by participants. The only significant association was that people were more likely to share tweets without self-disclosure (M = 3.37, SD = .24) compared with tweets with self-self-disclosure (M = 2.68, SD = .23), F(4,116) = 4.14, p = .044, and was on the contrary of our hypothesis. The influence of picture
presence on intention to like the tweet was only at the margin of significance, F(4,116) = 3.74, p = .055. In terms of joint impact on popularity, the combination of picture and self-disclosure did not make significant differences either in intention to like, share or leave comment. Therefore, H6 is rejected.
Main Effect of Picture Presence
In H7, we proposed that pro-vaccination post with an image will lead to higher perceived vaccine benefit and positive attitude towards vaccination than a post without any images. After being exposed to tweets with picture presence, participants’ perceived vaccine benefit (M = 5.97, SD = .14) was significantly higher than those exposed to tweets without picture (M = 5.41, SD
= .13). Nonetheless, picture disclosure in tweets did not significantly vary participants’
behavioural intention, F(4,67) = .14, p = .710. Only H7(a) is supported that presence of picture would effectively increase people’s perceived vaccine benefit, and H7(b) is rejected.
Main Effect of Self-disclosure
We also proposed H8 that presence of self-disclosure in tweets would have a positive impact on people’s perceived benefit and vaccine intention, in comparison with the tweets without self-disclosure. H8(a) is rejected since no significant differences in perceived vaccine benefit were observed between tweets with (M = 4.99, SD = 1.12) and without self-disclosure (M
26
= 5.33, SD = 1.33), F(4,116) = 1.86, p = .176. On the other hand, H8(b) is supported, as tweets with self-disclosure increased the behavioural intention among participants (M = 5.20, SD = .25) compared with tweets without self-disclosure (M = 4.48, SD = .23), F(4,67) = 4.48, p = .038. Synergistic Effects of the Picture and Self-disclosure’s Combination
H9 stated a joint impact of picture presence and self-disclosure in the pro-vaccination tweet. Figure 1 and Figure 2 illustrates the interactive impacts of the two independent variables. Comparing self-disclosing tweets with and without pictures, perceived benefit did not
significantly change. Therefore, H9(a) could not be supported. Significant differences were observed when we focus on participants’ intention to vaccinate, F(4,67)= 4.73, p=.033. When there was no picture presence in a tweet, participants would show higher intention to vaccinate if the tweet contain self-disclosure (M=5.50, SD=.35) compared with those without self-disclosure (M=4.05, SD=.36), but no similar findings were found when there was picture presence. H9(b) is also rejected, since no evidence could prove the joint impact of presence of both picture and self- disclosure. In sum, the interaction hypothesis as H9 is not supported.
Figure 1: Estimated Marginal Means of Perceived Benefit 1 2 3 4 5 6 7
Self-disclosure Present Self-disclosure Absent Picture Present Picture Absent
27
Figure 2: Estimated Marginal Means of Behavioural Intention
Conclusion and Discussion
This study aimed at verifying prevalent characteristics of vaccine misinformation on Twitter, and test their persuasiveness (i.e. impact on popularity, perceived vaccine benefit and vaccinate intention) by an experiment study. Evidences presented in the content analysis
highlighted that presence of pictures and self-disclosure in anti-vaccination tweets could increase their number of like, share and comments. In consistency with previous studies, presence of pictures would increase the popularity of vaccine misinformation retrieved in this study (Broniatowski et al., 2016; Yang & Guo, 2015). Along these lines, manual observation of the pictures’ containing “embedded text and people” was consistent with prior scholarly findings (Chen & Dredze, 2018; Kata, 2010)
By a large-scale content analysis of tweets (N=6418), we successfully employed automatic content analysis to boost our analysis. Meanwhile, the current study is an attempt to verify the suggestion of Kata (2010) and Moran et al., (2016) that personal testimonials are
1 2 3 4 5 6 7
Self-disclosure Present Self-disclosure Absent Picture Present Picture Absent
28
prevalent among anti-vaccination information and might a be persuasive strategy when we apply them into designing pro-vaccination messages. However, the results of this study are in the opposition of the negativity bias: vaccine misinformation online did not contain more negative sentiment, and negative sentiment did not bring about more popularity. In fact, sentiment
embedded in textual contents was proved to be ineffective to increase anti-vaccine information’s popularity in this study. This is coherent with the most recent studies, as Xu & Guo (2018) suggested in their study on vaccine-related websites that anti-vaccination information’s popularity was not related to its emotion words usage.
Meanwhile, our findings suggest vaccine misinformation is considerably rich in statistics. Numbers in tweets would increase instead of decrease their popularity, in contrary to previous works (e.g. Guidry et al., 2015). Nonetheless, limitations were that in this automated content analysis, all numeric information was regarded as “statistics”, regardless of their attributes (e.g. telephone numbers which obviously could not be regarded as statistics). In addition to the academic contributions, this study also provides some practical implications for health communicators such as the application of illustrations in promoting vaccination, and further adopting other persuasive characteristics of misinformation to effectively disseminate health-related messages.
To verify the effectiveness of persuasive strategies analysed in Study 1, we combined the frame of the health belief model and McGuire’s input/output matrix in the experiment (McGuire, 1999; Rosenstock et al., 1988). Although characteristics such as picture and self-disclosure presence were prevalent in vaccine misinformation and made the tweets more popular (i.e. with more likes, shares and comments), this popularity was not significant in the experimental stimuli. In the meantime, some characteristics only achieve part of the persuasive outputs as expected –
29
presence of pictures could increase people’s perceived vaccine benefit, and presence of self-disclosure was encouraging for people to get vaccinated. Interestingly, participants were unwilling to share the pro-vaccination tweet with self-disclosure, which is in contrast to the findings in the content analysis. Apart from the difference of experiment setting and real social networking, the ambivalent performance of self-disclosure might be explained by people’s perceived credibility of the message, since there could also be mechanisms in credibility’s boosting persuasiveness (Nan & Madden, 2012). Consensus have not yet been reached by current scholars whether (perceived) credibility would increase or decrease vaccine information’s persuasiveness, as there are also literatures showing weak or reverse
persuasiveness of “authorities” (e.g. Hu & Sundar, 2010; Kata, 2010). As an important part of “input” in McGuire’s matrix, people’s (perceived) credibility would also be a topic for future discussions on health interventions.
When communicating health-related messages to general audiences, scholars have pointed out that health literacy plays an important role in people’s understandings (Biasio et al., 2018). Participants recruited in this study were generally well-educated, with the lowest
completed degree of high-school or equivalent education. We could infer that our participants have adequate health literacy, which calls for additional research that account for this variable. Also, the factor of health literacy provides an alternative explanation for picture disclosure’s failure in increasing participants’ intention to vaccinate. Preliminary work on designing
simplified health messages indicated that people with high health literacy perceived the textual information well, regardless of the illustrations (Meppelink, Smit, Buurman, & van Weert, 2015). It could be thus explained that the behavioural intention of participants in this study remained at a similar level whether there was a picture in the tweet or not, and the illustration could even be
30
counterproductive when self-disclosure is present. Although the output of employing vaccine misinformation’s persuasive strategies is not much to our expectance, future studies should be aimed at testing these characteristics to a broader audience and getting more insight to how people of different education level perceive pro-vaccination messages. Also, researchers have commented that health communication on vaccines is sometimes limited in an “echo-chamber” (i.e. echo-chamber effect), where people’s ideas and beliefs are amplified and reinforced by their online activities in a homogenous community (Colleoni, Rozza, & Arvidsson, 2014; Faasse et al., 2016; Gesualdo et al., 2018). For tailored pro-vaccination interventions, future researches should also consider people’s relationship (i.e. social network) where they talk about the vaccination topic.
References
Ahern, D. K., Kreslake, J. M., & Phalen, J. M. (2006). What is eHealth (6): Perspectives on the evolution of eHealth research. Journal of Medical Internet Research.
https://doi.org/10.2196/jmir.8.1.e4
Alhabash, S., McAlister, A. R., Lou, C., & Hagerstrom, A. (2015). From Clicks to Behaviors: The Mediating Effect of Intentions to Like, Share, and Comment on the Relationship Between Message Evaluations and Offline Behavioral Intentions. Journal of Interactive Advertising. https://doi.org/10.1080/15252019.2015.1071677
Allen, J. D., Mohllajee, A. P., Shelton, R. C., Othus, M. K. D., Fontenot, H. B., & Hanna, R. (2009). Stage of adoption of the human papillomavirus vaccine among college women. Preventive Medicine. https://doi.org/10.1016/j.ypmed.2008.12.005
Askelson, N. M., Campo, S., Lowe, J. B., Smith, S., Dennis, L. K., & Andsager, J. (2010). Using the theory of planned behavior to predict mothers’ intentions to vaccinate their daughters
31
against HPV. Journal of School Nursing. https://doi.org/10.1109/ECCE.2017.8096848 Bean, S. J. (2011). Emerging and continuing trends in vaccine opposition website content.
Vaccine. https://doi.org/10.1016/j.vaccine.2011.01.003
Betsch, C., Brewer, N. T., Brocard, P., Davies, P., Gaissmaier, W., Haase, N., … Stryk, M. (2012). Opportunities and challenges of Web 2.0 for vaccination decisions. Vaccine.
https://doi.org/10.1016/j.vaccine.2012.02.025
Brewer, N. T., & Fazekas, K. I. (2007). Predictors of HPV vaccine acceptability: A theory-informed, systematic review. Preventive Medicine.
https://doi.org/10.1016/j.ypmed.2007.05.013
Briones, R., Nan, X., Madden, K., & Waks, L. (2012). When Vaccines Go Viral: An Analysis of HPV Vaccine Coverage on YouTube. Health Communication.
https://doi.org/10.1080/10410236.2011.610258
Broniatowski, D. A., Hilyard, K. M., & Dredze, M. (2016). Effective vaccine communication during the disneyland measles outbreak. Vaccine.
https://doi.org/10.1016/j.vaccine.2016.04.044
CDC. (2016). HPV Vaccine Recommendations | Human Papillomavirus | CDC. Retrieved November 21, 2018, from
https://www.cdc.gov/vaccines/vpd/hpv/hcp/recommendations.html
Chen, T., & Dredze, M. (2018). Vaccine images on twitter: Analysis of what images are shared. Journal of Medical Internet Research. https://doi.org/10.2196/jmir.8221
Chen, X., Sin, S. C. J., Theng, Y. L., & Lee, C. S. (2015). Why Students Share Misinformation on Social Media: Motivation, Gender, and Study-level Differences. Journal of Academic Librarianship. https://doi.org/10.1016/j.acalib.2015.07.003
32
Colleoni, E., Rozza, A., & Arvidsson, A. (2014). Echo Chamber or Public Sphere? Predicting Political Orientation and Measuring Political Homophily in Twitter Using Big Data. Journal of Communication. https://doi.org/10.1111/jcom.12084
Collins, N. L., & Miller, L. C. (1994). Self-disclosure and liking: A meta-analytic review. Psychological Bulletin. https://doi.org/10.1037/0033-2909.116.3.457
Dredze, M., Wood-Doughty, Z., Quinn, S. C., & Broniatowski, D. A. (2017). Vaccine opponents’ use of Twitter during the 2016 US presidential election: Implications for practice and policy. Vaccine. https://doi.org/10.1016/j.vaccine.2017.06.066
Dubé, E., Laberge, C., Guay, M., Bramadat, P., Roy, R., & Bettinger, J. (2013). Vaccine hesitancy: An overview. Human Vaccines and Immunotherapeutics.
https://doi.org/10.4161/hv.24657
Dunn, A. G., Leask, J., Zhou, X., Mandl, K. D., & Coiera, E. (2015). Associations between exposure to and expression of negative opinions about human papillomavirus vaccines on social media: An observational study. Journal of Medical Internet Research.
https://doi.org/10.2196/jmir.4343
Eysenbach, G., Powell, J., Kuss, O., & Sa, E. R. (2002). Empirical studies assessing the quality of health information for consumers on the World Wide Web: A systematic review. Journal of the American Medical Association. https://doi.org/10.1001/jama.287.20.2691
Faasse, K., Chatman, C. J., & Martin, L. R. (2016). A comparison of language use in pro- and anti-vaccination comments in response to a high profile Facebook post. Vaccine.
https://doi.org/10.1016/j.vaccine.2016.09.029
Fabry, P., Gagneur, A., & Pasquier, J. C. (2011). Determinants of A (H1N1) vaccination: Cross-sectional study in a population of pregnant women in Quebec. Vaccine.
33 https://doi.org/10.1016/j.vaccine.2010.12.109
Feng, C., Wang, H., Lu, N., Chen, T., He, H., Lu, Y., & Tu, X. M. (2014). Log-transformation and its implications for data analysis. Shanghai Archives of Psychiatry.
https://doi.org/10.3969/j.issn.1002-0829.2014.02.009
Fernández-Luque, L., & Bau, T. (2015). Health and social media: Perfect storm of information. Healthcare Informatics Research. https://doi.org/10.4258/hir.2015.21.2.67
Freed, G. L., Clark, S. J., Butchart, A. T., Singer, D. C., & Davis, M. M. (2010). Parental Vaccine Safety Concerns in 2009. PEDIATRICS. https://doi.org/10.1542/peds.2009-1962
Gesualdo, F., Zamperini, N., & Tozzi, A. E. (2018). To talk better about vaccines, we should talk less about vaccines. Vaccine. https://doi.org/10.1016/j.vaccine.2018.07.025
Ghenai, A., & Mejova, Y. (2017). Catching Zika Fever: Application of Crowdsourcing and Machine Learning for Tracking Health Misinformation on Twitter. In Proceedings - 2017 IEEE International Conference on Healthcare Informatics, ICHI 2017.
https://doi.org/10.1109/ICHI.2017.58
Giatsoglou, M., Vozalis, M. G., Diamantaras, K., Vakali, A., Sarigiannidis, G., & Chatzisavvas, K. C. (2017). Sentiment analysis leveraging emotions and word embeddings. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2016.10.043
Goldstein, N. D., LeVasseur, M. T., & Purtle, J. (2017). Is this thing on? Getting the public to listen to the pro-vaccine message. Vaccine. https://doi.org/10.1016/j.vaccine.2015.12.070 Guidry, J. P. D., Carlyle, K., Messner, M., & Jin, Y. (2015). On pins and needles: How vaccines
are portrayed on Pinterest. Vaccine. https://doi.org/10.1016/j.vaccine.2015.08.064
Healthline. (2018). HPV Vaccination Australia Program Success. Retrieved November 14, 2018, from
https://www.healthline.com/health-news/australia-using-hpv-vaccine-to-eliminate-34
cervical-cancer#Misconceptions-about-the-vaccine-
Houts, P. S., Doak, C. C., Doak, L. G., & Loscalzo, M. J. (2006). The role of pictures in
improving health communication: A review of research on attention, comprehension, recall, and adherence. Patient Education and Counseling.
https://doi.org/10.1016/j.pec.2005.05.004
Hu, Y., & Sundar, S. S. (2010). Effects of online health sources on credibility and behavioral intentions. Communication Research. https://doi.org/10.1177/0093650209351512 Ito, T. A., Larsen, J. T., Smith, N. K., & Cacioppo, J. T. (1998). Negative information weighs
more heavily on the brain: the negativity bias in evaluative categorizations. Journal of Personality and Social Psychology. https://doi.org/10.1037/0022-3514.75.4.887
Juraskova, I., Bari, R. A., O’Brien, M. T., & McCaffery, K. J. (2011). HPV Vaccine Promotion: Does Referring to Both Cervical Cancer and Genital Warts Affect Intended and Actual Vaccination Behavior? Women’s Health Issues. https://doi.org/10.1016/j.whi.2010.08.004 Kata, A. (2010). A postmodern Pandora’s box: Anti-vaccination misinformation on the Internet.
Vaccine. https://doi.org/10.1016/j.vaccine.2009.12.022
Kortum, P., Edwards, C., & Richards-Kortum, R. (2008). The impact of inaccurate Internet health information in a secondary school learning environment. Journal of Medical Internet Research. https://doi.org/10.2196/jmir.986
Kumar Singh, A., Kumar Gupta, D., & Mohan Singh, R. (2017). Sentiment Analysis of Twitter User Data on Punjab Legislative Assembly Election, 2017. International Journal of Modern Education and Computer Science. https://doi.org/10.5815/ijmecs.2017.09.07
Lev-On, A., & Hardin, R. (2008). Internet-based collaborations and their political significance. Journal of Information Technology and Politics.
35 https://doi.org/10.1080/19331680802076074
Lewandowsky, S., & Oberauer, K. (2016). Motivated Rejection of Science. Current Directions in Psychological Science. https://doi.org/10.1177/0963721416654436
List of Feeling Words. (2018). Retrieved November 1, 2018, from http://www.psychpage.com/learning/library/assess/feelings.html
Mathew, J. L., Mohan, P., & Kumar, R. (2014). Effective messages in vaccine promotion: A randomised trial. Indian Pediatrics. https://doi.org/10.1007/s13312-014-0426-8
McGuire, W. (1999). Constructing social psychology: Creative and critical aspects. Cambridge University Press.
Meppelink, C. S., Smit, E. G., Buurman, B. M., & van Weert, J. C. M. (2015). Should We Be Afraid of Simple Messages? The Effects of Text Difficulty and Illustrations in People With Low or High Health Literacy. Health Communication.
https://doi.org/10.1080/10410236.2015.1037425
Moran, M. B., Frank, L. B., Chatterjee, J. S., Murphy, S. T., & Baezconde-Garbanati, L. (2016). A pilot test of the acceptability and efficacy of narrative and non-narrative health education materials in a low health literacy population. Journal of Communication in Healthcare. https://doi.org/10.1080/17538068.2015.1126995
Moran, M. B., Lucas, M., Everhart, K., Morgan, A., & Prickett, E. (2016). What makes anti-vaccine websites persuasive? A content analysis of techniques used by anti-anti-vaccine websites to engender anti-vaccine sentiment. Journal of Communication in Healthcare. https://doi.org/10.1080/17538068.2016.1235531
Nan, X., & Madden, K. (2012). HPV Vaccine Information in the Blogosphere: How Positive and Negative Blogs Influence Vaccine-Related Risk Perceptions, Attitudes, and Behavioral
36
Intentions. Health Communication. https://doi.org/10.1080/10410236.2012.661348 Perrin, A. (2015). Social media usage: 2005-2015. Pew Research Center.
https://doi.org/202.419.4372
Pluviano, S., Watt, C., & Della Sala, S. (2017). Misinformation lingers in memory: Failure of three pro-vaccination strategies. PLoS ONE. https://doi.org/10.1371/journal.pone.0181640 Rambout, L., Tashkandi, M., Hopkins, L., & Tricco, A. C. (2014). Self-reported barriers and
facilitators to preventive human papillomavirus vaccination among adolescent girls and young women: A systematic review. Preventive Medicine.
https://doi.org/10.1016/j.ypmed.2013.10.009
Reyna, V. F. (2008). A theory of medical decision making and health: Fuzzy trace theory. Medical Decision Making. https://doi.org/10.1177/0272989X08327066
Rosenstock, I. M., Strecher, V. J., & Becker, M. H. (1988). Social Learning Theory and the Health Belief Model. Health Education & Behavior.
https://doi.org/10.1177/109019818801500203
Salathé, M., & Khandelwal, S. (2011). Assessing vaccination sentiments with online social media: Implications for infectious disease dynamics and control. PLoS Computational Biology. https://doi.org/10.1371/journal.pcbi.1002199
Shelby, A., & Ernst, K. (2013). Story and science: How providers and parents can utilize storytelling to combat anti-vaccine misinformation. Human Vaccines and
Immunotherapeutics. https://doi.org/10.4161/hv.24828
Siegrist, M., & Cvetkovich, G. (2001). Better negative than positive? Evidence of a bias for negative information about possible health dangers. Risk Analysis.
37
Signorini, A., Segre, A. M., & Polgreen, P. M. (2011). The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS ONE. https://doi.org/10.1371/journal.pone.0019467
Siminoff, L. A., Traino, H. M., & Gordon, N. H. (2011). An exploratory study of relational, persuasive, and nonverbal communication in requests for tissue donation. Journal of Health Communication. https://doi.org/10.1080/10810730.2011.561908
Song, G. (2014). Understanding public perceptions of benefits and risks of childhood vaccinations in the United States. Risk Analysis. https://doi.org/10.1111/risa.12114 Tafuri, S., Gallone, M. S., Cappelli, M. G., Martinelli, D., Prato, R., & Germinario, C. (2014).
Addressing the anti-vaccination movement and the role of HCWs. Vaccine. https://doi.org/10.1016/j.vaccine.2013.11.006
Taylor, S. E. (1991). Asymmetrical effects of positive and negative events: The mobilization-minimization hypothesis. Psychological Bulletin.
https://doi.org/10.1037/0033-2909.110.1.67
World Economic Forum. (2013). Digital Wildfires in a Hyperconnected World. Global Risks. https://doi.org/DOI: 10.1017/S2045381715000131
World Internet Users Statistics and 2018 World Population Stats. (2018). Retrieved October 29, 2018, from https://www.internetworldstats.com/stats.htm
Xu, Z., & Guo, H. (2018). Using Text Mining to Compare Online Pro- and Anti-Vaccine Headlines: Word Usage, Sentiments, and Online Popularity. Communication Studies. https://doi.org/10.1080/10510974.2017.1414068
Yang, F., & Guo, S. (2015). The moderating effect of imagery ability on perceived vividness: the case of HPV vaccine advertising in China. Chinese Journal of Communication.
38 https://doi.org/10.1080/17544750.2015.1008531
Zhao, D., & Rosson, M. B. (2009). How and why people Twitter: The Role that Micro-blogging Plays in Informal Communication at Work. In Proceedings of the ACM 2009 international conference on Supporting group work. https://doi.org/10.1145/1531674.1531710
Appendix I: Hashtags Used to Scrape Anti-vaccination Tweets (Dredze et al., 2017) #autism #autismwarriorny #autismawareness #2many2soon #b1less #vaccineinjury #sids #cdcrally #cdcfraud #cdctruth #cdcwhistleblower #mpvr #healthfreedom #sb277 #sb277referendum #sb792 #hr2232 #hearthiswell
39 #vaxfax #vaxxed #vaxxedthemovie #wakeupamerica #hearus #breakabillion #cdcvax
Appendix II: Questionnaire for Study 2, designed in and exported from Quatrics Survey Flow
Block: Fact Sheet and Informed Consent (2 Questions) Standard: Filter Questions (2 Questions)
Standard: Cover Story (1 Question) Standard: Attitude (2 Questions) Standard: Cover Story 2 (1 Question)
BlockRandomizer: 1 - Evenly Present Elements Block: Group 1 (3 Questions)
Block: Group 2 (3 Questions) Block: Group 3 (3 Questions) Block: Group 4 (3 Questions)
Standard: Like, Share and Comment (2 Questions) Standard: Perceived Benefit (2 Questions)
Standard: Behavioural Intention (5 Questions) Standard: Manipulation Check (3 Questions) Standard: Demographics (3 Questions) Page Break
40 Start of Block: Fact Sheet and Informed Consent FS Dear participant,
With this letter, I would like to invite you to participate in a research study to be conducted under the auspices of the Graduate School of Communication, a part of the University of
Amsterdam. The title of the study for which I am requesting your cooperation is ‘HPV Vaccine Information’. In this online survey, a screenshot of a vaccine-related tweet will be displayed. You will be asked to report your feelings after reading the tweet. In addition, some demographic information will be asked. Only female students under the age of 26 years are eligible to
participate in this study. The goal of this research is to gain insight into several types of vaccine-related information.
The study will take about 5 minutes. Three winners will be selected at random from all participants, and each winner may choose one of three prizes valued at €10: cash of the equal value, a gift certificate for Bol.com or a gift certificate for H&M.
As this research is being carried out under the responsibility of the ASCoR, University of Amsterdam, we can guarantee that:
1) Your anonymity will be safeguarded, and that your personal information will not be passed on to third parties under any conditions, unless you first give your express permission for this. 2) You can refuse to participate in the research or cut short your participation without having to give a reason for doing so. You also have up to 7 days after participating to withdraw your permission to allow your answers or data to be used in the research.
3) Participating in the research will not entail your being subjected to any appreciable risk or discomfort, the researchers will not deliberately mislead you, and you will not be exposed to any explicitly offensive material.
4) If you want to receive a report on the findings of the study, please send an e-mail to Anqi Shao anqi.shao@student.uva.nl. For more information about the research and the invitation to participate, you are welcome to contact the project leader Anqi Shao at any time via
anqi.shao@student.uva.nl or dr. Corine Meppelink (c.s.meppelink@uva.nl).
Should you have any complaints or comments about the course of the research and the procedures it involves as a consequence of your participation in this research, you can contact the designated member of the Ethics Committee representing ASCoR, at the following address: ASCoR Secretariat, Ethics Committee, University of Amsterdam, Postbus 15793, 1001 NG Amsterdam; 020-525 3680; ascor-secr-fmg@uva.nl. Any complaints or comments will be treated in the strictest confidence.
We hope that we have provided you with sufficient information. We would like to take this opportunity to thank you in advance for your assistance with this research, which we greatly appreciate.
Kind regards, Anqi Shao
Cellphone: +31 0626197097
