Know your followers: How to evoke positive social media sentiments

(1)

Know your followers: How to evoke positive

social media sentiments

Measuring and fostering sentiments for brands by analyzing Facebook posts and comments from the beverages industry

Eva Wilke

Faculty of Economics and Business Master Thesis Marketing Intelligence 16th June, 2019 Korreweg 214 9715 AM Groningen +4915785994252 e.wilke@student.rug.nl S3161013 Dr. Hans Risselada University of Groningen

(2)

Abstract

Which factors play a role in determining the sentiments of Facebook comments for brands? Five studies investigate this question by applying sentiment analysis and topic modelling to five datasets from brands of the beverages industry, namely Starbucks, Coca-Cola,

Budweiser, Guinness and Lipton. This paper finds that post sentiments positively influence the comment sentiments. Additionally, previous comment sentiments have a positive effect on the subsequent comment sentiments, which is in line with the idea of social adaption.

Moreover, positive post reactions such as “haha” and “love” are not related to the positivity of the comment sentiments. Lastly, this paper finds that social media posts and comments which deal with the product of the brand do not positively influence the comment sentiments. The comment sentiment response to certain topics are brand-specific.

Keywords: sentiment analysis, topic modelling, social media, Facebook comments

Acknowledgments

(3)

1.0 Introduction

“Social media thus creates tremendous opportunities as well as challenges. Smart firms grow with them and as a result build capability that are conducive to the new way of doing business, while traditional firms continue to try to out-compete the social side, hoping it to go

away. But in the age of social technologies and the social economy, social media is here to stay.” - Arora and Predmore (2013)

As stated in the opening quote, social media is an instrument which builds valuable capabilities for the firm. Arora and Predmore (2013) state four reasons why social media is important to a firm as a strategic tool: Firstly, social media is a more efficient and less costly two-way street to connect to the customer and secondly, it encourages loyalty and engages those who are important to the firm. Thirdly, it converts the two-way conversations into data which depicts sentiments and emotions about topics of the present and future of the company. Lastly, the authors highlight that social media creates strategic value. This paper specifically builds up on the third reason why social media is important: Social media data which gives valuable insights into the customer sentiments evolving around the company.

(5)

purchase and detect buying patterns as well as finding online topics (Reisenbichler and Reutterer, 2018).

Kaplan and Haenlein (2010) define social media as a „group of Internet based applications […] that allow the creation and exchange of user-generated content”. However, there is more to social media than only user-generated content. Social media platforms also allow brand-generated content by two-way communication between brands and the individual. On brand pages, posts are shared by brands to which consumers react and comment. This form of communication is important to firms, because it gives them valuable insights on how receptive consumers are to certain topics. Knowing whether consumers react more positively to posts about the brand or to posts about the product helps the brand to increase overall consumer sentiments on their social media sites.

Previous research argues that generating well-received content on social media can improve the relationship between the brand and the consumer and enhance consumer loyalty. Cvijikj and Michahelles (2011) state that sentiment analysis helps to comprehend the user-generated social media comments which facilitates to understand how a brand or product is perceived by the consumer. Therefore, understanding and analyzing brand and user-generated content, can help to build an emotional attachment to the brand. This emotional attachment manifests consumer loyalty and brand advocacy (Turri, Smith and Kemp 2013). This paper studies brand and user-generated social media content and therefore sets the ground stone for fostering consumer loyalty and brand advocacy. This paper studies how the topic and sentiment of a brand’s social media post influence the sentiments of the comments to that post. Moreover, this paper analyzes how emotional reactions to posts, comment topics and previous comment sentiments influence this relationship. Topic modelling and sentiment analysis is applied to posts and comments of 5 brands of the beverages industry.

(6)

2017). Moreover, the influence of other consumers’ comments on the positivity of following comments has been researched, as well as (to a very limited extent) how emotional reactions to posts are related to sentiments (Tian, Galery, Dulcinati, Molimpakis and Chao Sun, 2017). However, so far, no previous work has precisely researched the relationship between post sentiments and comment sentiments and neither the relationship between post topics and comment sentiments. With the increasing importance of social media in strategic marketing, it is important for firms to know how post sentiments and topics are related to the comment sentiments. By knowing what influences the comment sentiments of consumers, brand’s followers can be better understood and a better connection to the customer can be built. Questions remain unanswered, such as: How does a brand-generated post topic affect the sentiment of its comments? How does the sentiment of the post itself affect the sentiment of the comments? And do emotional reactions to posts affect the sentiments among post comments? Do other individuals’ comments influence others? All of these questions will be answered in this paper.

In the past years, customers have increasingly used social media platforms such as Facebook to share their customer experience and opinions about brands (Krebs, Lubascher, Moers, Schaap and Spanakis, 2017). The experience of a customer with a certain product can entail satisfaction or dissatisfaction. This (dis)satisfaction is often voiced on social media sites. Moreover, posts can entail other emotional reactions to certain features of the brand as for instance a change in slogan. The reactions to these changes are extremely valuable and insightful for companies. Additionally, those reactions are most of the time loaded with personal feelings. It is expected that those reactions in form of comments can be triggered by certain post sentiments or post topics.

(7)

Additionally, post reactions can not be taken as a measure for the sentiments found in the comments. Positive post reactions are not positively related to the comment sentiments. In the topic modelling part of the paper it is found that post and comment topics related to the product itself do not have higher sentiment scores than posts and comments for other topics.

This paper is unique in the sense that it combines both sentiment analysis and topic modelling. It provides an example of how to apply these techniques to social media analytics and thereby builds up upon already existing literature and simultaneously gives a solid base for further analysis. With the insights of the combined techniques of analysis, this paper particularly sheds light on those factors which influence sentiments of Facebook comments.

With this knowledge, certain kind of social media sentiments can be evoked which in the end can foster emotional attachment to the brand. Emotional attachment is related to brand loyalty (Thomson, Macinnis and Park, 2005). Knowing how to develop emotional attachment to a brand is an extremely valuable skill for marketing managers. It helps to build the brand and to foster brand loyalty. This paper shows how social media brand content creation can foster positive sentiments, increase emotional attachment to a brand and as a result develop brand loyalty. With the help of this knowledge, social media content creation can be optimized.

2.0 Theoretical Framework

As shown in figure 1, this paper entails four hypotheses and one explorative study. Each of those hypotheses test a relationship between the dependent variable, namely comment

(8)

emotional reaction to posts and the comment topic, whereas the fourth study inspects the relationship between the previous comment sentiment and the following comment sentiment. Lastly, the fifth study explores the effect of the comment topic on the comment sentiment. In the following, each variable is defined and put into theoretical context and the hypotheses of this paper are introduced.

Figure 1: Conceptual Model

Comment Sentiment

(9)

followers and not on comments by the brands itself. The reason for this is that this paper researches how certain posts are perceived by customers and makes the comments to posts by the brand itself irrelevant.

Post Sentiment

This paper analyzes those posts which are brand-generated. Previous research has shown that brand pages with negative overall valence of posts are perceived lower by consumers and lead to lower purchase intentions than brand pages with more positive posts (Barcelos, Dantas and Sénécal, 2018). Moreover, Gallan, Jarvis, Brown and Bitner (2012) find that higher levels of positivity lead to a higher perception of the quality of the service provider and the experience with the service. Hence, approaching the consumer with positivity, in this case a positive post, leads to a more positive perception. This leads to the assumption that more positive sentiments in a post are perceived better and hence generate more positive sentiments among its comments than negative sentiments in a post.

H1: The more positive the sentiment of the post itself, the more positive are the sentiments

among the comments to that post.

Sentiment of previous comments

(10)

Additionally, Piacentini & Szmigin (2018) state that conformity is evoked whenever there is the pressure of compliance with a person or group existing.

It is assumed that the urge to receive approval and adapt to the audience is applicable to the context of social media. In social media, consumers also express their opinion on products and those opinions in form of comments are public to everyone. Whenever consumers see a post, they automatically see comments which have been written by others beforehand. In line with the idea of conformity, this paper argues that consumers are influenced by comments by others and adjust their comments in order to match previous comments. Thus, this paper argues that it is most likely that positive comments are followed by positive subsequent comments.

H2: Positive previous post comments positively affect the sentiment of the subsequent

comments

Emotional reaction to post

Broad (1954) studies the connection between emotions and sentiments. The author explains that a person associates several emotions with an object through experience. Hence, the dispositions about this object have something to do with the associated emotions. Anytime, the disposition of this object is evoked, the emotional dispositions are brought up. Broad (1954) defines this as the forming of a sentiment. This means that sentiments are formed of multiple emotions and are therefore interlinked. Hence, this paper argues that sentiments are in line with emotions.

(11)

statistically test or prove it. This paper agrees with the argument that the emotional reaction buttons on Facebook can potentially mirror the user’s sentiment. Yet, in order to test this relationship, this paper analyzes whether the emotional reactions to posts are in line with the overall sentiments among the post comments. With this insight, companies could conclude comment sentiments by just looking at the overall emotional reaction to posts. Even if those followers using emotional reactions and those followers commenting are not the same type of consumer, this study shows whether the emotional reactions are indicative of the sentiments found in the comments.

Since “love” and “haha” are more positive reactions than “sad” and “angry”, this paper argues that the former reactions indicate that the post comments are more positive, whereas the latter indicate that the post comments are more negative. Since the “wow” reaction is ambiguous and can be used for both positive and negative reactions, it is not taken into consideration.

H3a: Emotional reactions such as “love” and “haha” lead to more positive comment

sentiments.

H3b: Emotional reactions such as “sad” and “angry” lead to more negative comment sentiments.

Post Topic

(12)

“entertainment” and “other”. Their findings show that people are most engaged when posts are about entertainment and most likes are given to posts which deal with events or the products itself.

Since both authors found that posts about the product itself earn the most positive reactions, this paper argues that sentiments found among comments to posts which are related to the product itself are more positive than for other topics.

H4: The sentiments of comments to posts related to the product itself are more positive than for other topics.

Comment topic. Since it has been hypothesized before that post topics about the product

itself lead to positive comment sentiments, it could be assumed that comment topics which are related to the product are also higher in comment sentiment score than other comment topics. However, since there is fundamental theory missing to form a hypothesis, the study of the effect of the comment topic on the comment sentiment is of explorative nature. This means that this study explores which kind of topics customers deal with in the comments and to which sentiments those topics lead.

There is no analysis on the relationship between the post topic and comment topic, because those variables are of subjective nature which makes a quantitative statistical test biased. Hence, those two variables are analyzed on their own.

3.0 Data

(13)

industry to study the relationship between posts and comments, since users are very active on these sites. Beverage products are part of our everyday life: We drink coffee or tea for breakfast, a coke for lunch and an after-work beer at night. On Facebook brand pages, consumers can either send a direct message to the brand or comment on posts. Since direct messages are not public, most consumers prefer to comment on brand posts because they want to state their opinion publicly. Lima, Irigaray and Lourenco (2019) state that consumers are primarily motivated to engage in virtual brand communities (such as Facebook brand pages), because they want to benefit from the interaction with the members and the brand itself as well as the consumption of the exchanged information. Thus, sharing their thoughts via a comment to a post results in a much higher added value than sending a direct message to the company. Therefore, companies need to be careful with their post content and should not trigger certain negative comments to their posts with certain topics or post sentiments. This paper sheds light on which kind of posts yield the highest comment sentiment scores and are therefore best received by followers.

(14)

Five distinct brands from the beverages industry are used for analysis, namely Starbucks, Coca-Cola, Budweiser, Lipton and Guinness. First of all, those brands were chosen because they have a huge follower base on their Facebook page and they are all big players in the beverages industry. This means that there is a lot of social media content, i.e. Facebook posts and comments which can be analyzed, increasing the generalizability of the findings.

Brand Amount of Posts Amount of Comments Starbucks 38 48.746 Coca-Cola 48 27.466 Budweiser 77 48.996 Lipton 107 1.746 Guinness 210 16.557 Total 470 143.511

Table 1: Overview Dataset

Overall, the dataset comprises 470 Facebook posts and about 140.000 comments and is limited to the time range from January 2018 until March 2019. Since the data is limited to a certain time frame, the amount of posts and comments vary per brand (Table 1). Except for Starbucks, all brands have national Facebook pages. To keep the analysis focused on a

specific market, only the US-American market was chosen for the Facebook pages with national options. For Starbucks, the international page was chosen which includes the US-American market. This choice was made to ensure an English-speaking community, facilitate further analysis and extend generalizability, because the English-speaking community is the biggest for all five of those brands.

Post data. There are in total 470 Facebook posts, of which 210 are published by

(15)

of the brands except for Starbucks and Coca-Cola (Diff=0.96, p=0.02). This is extremely interesting to see, because Coca-Cola and Starbucks are the brands with the most negative comment sentiments but are extremely different in their post sentiments. Coca-Cola’s post sentiments are much more negative than those for Starbucks. An overview of post sentiments for all brands can be found in figure 2.

Comment data. When taking a first look at the comment data for the brands, it becomes

apparent that the sentiments differ widely (Figure 3). In fact, there is a highly significant difference between the means of the comment sentiments between the brands (F=202.4,

p=0.00). The Tukey post hoc comparison test shows that the differences in means are highly

significant for all pairs of brands (p=0.00) except for those mean differences among Budweiser, Guinness and Lipton. The mean plot in figure 3 (panel b) visualizes these findings: The mean sentiment of post comments for those three beverages companies are close to each other (Budweiser: M= 1.61, Guinness: M=1.56, Lipton: M=1.50). The mean of the comment sentiments for Coke and Starbucks are significantly lower (Coke: M=1.18, Starbucks: M=1.22). This signals that consumers on average show more negative sentiments for Coke’s and Starbuck’s posts than for Budweiser, Guinness and Lipton. Even though the sentiments for Starbucks and Coke are lower than for the rest, their sentiment scores are still positive on average.

(16)

Interesting to see in the boxplot of figure 3 (panel a) is that the medians of the comment sentiments are very similar to each other compared to the means shown in the mean plot. This might be the case because there are more extreme values or more rather negative value for Coke and Starbucks than for the other brands. Moreover, the difference is caused by the fact that there are many mid-ranged sentiment scores of about 2.0. This reveals that most comments are rather positive in sentiments than negative.

The variances in comment sentiments is lowest for Guinness and highest for Budweiser (Guinness: Var=2.49, Budweiser: Var=3.53). A high variance in comment sentiments shows that consumers express very diverse feelings implied by opinion.

Budweiser shows a high number of negative outliers which seems to be inconsistent with the fact that Budweiser’s average sentiment score is the highest. However, this can be explained by the fact that Budweiser also has the most comments and therefore, a few outliers do not weigh strongly. Removing outliers from the dataset is not appropriate in this case because those outliers can be the most interesting data points in the dataset. They indicate when consumers feel very strongly which gives valuable insights.

(17)

4.0 Overview of Studies

This paper operationalizes the effect on comment sentiments in five studies (Figure 4). Study 1 until 3 use sentiment analysis, whereas the two last studies combine sentiment analysis and topic modelling.

Figure 4: Overview of Studies

(18)

Study 1: Effect of post sentiment on comment sentiment

Study 1 investigatesthe relationship between post sentiments and comment sentiments. In order to test this relationship, this paper applies sentiment analysis to the existing dataset. Sentiment analysis is used to “determine the perception of a specific topic, product or person by people and the strength of the sentiments in terms of their positive and negative nature” (Lyu and Kim, 2016).

(19)

µ Comment level µ Post level Budweiser 1.61 1.24 Coca-Cola 1.35 0.86 Guinness 1.56 1.57 Lipton 1.50 1.42 Starbucks 1.22 0.94

Table 2: Mean comparison of comment sentiments

An ANOVA analysis shows that there are still significant differences between the sentiment means of the five different groups on the post level (F(4,460)=17,89, p=0.00) . All differences are highly significant (p=0.00), except for the mean differences between Starbucks-Budweiser, Starbucks-Coke and Lipton-Guinness. The biggest difference in mean sentiments of 0.71 is between Guinness and Coke.

With the sentiment scale of the posts and the average sentiment scale of post comments, a regression analysis is run which depicts whether more positive posts lead to more positive comments or not. The linear additive model equation is shown in 1.0. The Shapiro-Wilk test shows that the post level comment data is normally distributed for each brand (p>0.05), which is an assumption for conducting a regression analysis.

!_" = $_"+ &_"'_"+ (_" (+. -) / = 0122345 634572345 8 = 01465945

: = ;165 634572345 3 = <765=>?94@3 53>2

(20)

7 = A>94<

In order to test whether it is allowed to pool the observations of all brands for the regression analysis, a Chow-Test is run. The Chow-Test compares the residuals and degrees of freedom to test whether there is a significant difference between the pooled and unpooled versions. The results show that pooling is not allowed (F=5.72, Fcrit=1.93). Hence, the regression analysis is run per brand.

Results. The regression analyses for each of the five brands show that there is a

significant, positive relationship between the post sentiment and comment sentiment for Guinness and Lipton (Table 3). In fact, one unit increase in post sentiment leads to 0.09 higher sentiment scores for Guinness and to 0.28 higher sentiment scores in the comments for Lipton.

Estimate p-value R2 Budweiser 0.02 0.79 0.00 Coca-Cola 0.07 0.45 0.03 Guinness 0.09 0.03 0.03 Lipton 0.28 0.02 0.07 Starbucks 0.05 0.74 0.01

Table 3: Results regression analysis of hypothesis 1 (Equation 1.0)

The R2_{’s in this case range from 3% explained variance to 7% explained variance for}

Lipton’s model. Since the aim of this model is to give an indication of positive or negative relationship between the variable and does not need to give precise predictions of how much exactly the dependent variable is affected, a low R2_{is acceptable. However, it is important to}

acknowledge that the low R2 _{might be caused by an omitted variable problem which might lead}

(21)

The Shapiro-Wilk test shows for the regression analyses of all brands that only for Starbucks and Coke the residuals are normally distributed (p>0.05). Hence, non-normality can not be assumed for Budweiser, Guinness and Lipton. Plotting the residuals of the significant regression analyses (Lipton and Guinness) shows that their distribution is in fact not clearly normal (Figure 6). The problem of the non-normality distribution is assumed to be linked to the fact that there are outliers in the dataset which are not removed due to their importance. Those outliers reflect extreme sentiment responses by consumers which are relevant to take into account for the analysis. As can be seen in Figure 6, the plot of the residuals shows that they seem to be generally normally distributed except for several extreme values.

Removing the outliers in order to achieve normality of the regression analysis is not an option. Thus, a different solution is needed in order to test hypothesis 1 for Budweiser, Guinness and Lipton. Since the normality assumption for Coke and Starbucks was met and the two regression analyses did not show significant results, only the three remaining brands are further investigated. This paper considers two options to yield statistical valid results as an alternative to regression analysis: Bootstrapping and quantile regression.

Bootstrapping resamples the dataset with replacement to yield an estimate and thereby foregoes the issue of outliers. The results of the bootstrapping method show a positive relationship between post sentiments and comment sentiments for Budweiser, Guinness and Lipton (ß=0.001, ß=0.03 and ß=0.07 respectively).

(22)

As an alternative option to bootstrapping, a quantile regression is run for Lipton and Guinness. The quantile regression is similar to the linear regression but estimates the effect of the independent variable on the dependent variable of a specific quantile. By taking quantiles into account, the problem of outliers is solved. Due to statistical reasons, the quantile regression for Budweiser is not taken into consideration; There are too few data points available per percentile, because Budweiser’s data set only incorporates 77 posts.

For Guinness, the 10th_{percentile of comment sentiment equals a sentiment score of 1.00,}

whereas the 90th_{percentile equals a sentiment score of 2.31. The quantile regression shows that}

the 10th_{percentile comment sentiment increases by 0.09 for one unit increase in post sentiment,}

whereas the 90th_{percentile comment sentiment increases by 0.17 for one unit increase in post}

sentiment. The intercepts are relatively similar to each other, ranging from 0.83 to 2.00. Note that there is an increase of almost 100%. This shows that there is not only a general positive influence on the comment sentiment, but it significantly increases the higher the post sentiment gets.

For Lipton, the 10th_{percentile of comment equals a sentiment score of 0.24, whereas}

the 90th_{percentile equals a sentiment score of 2.15. When looking at the results of the quantile}

regression it becomes apparent that the intercepts vary extremely. The intercepts range from -1.05 to 2.0. This means that when the post sentiment score is kept neutral or at 0, the predicted value of the comment sentiment score is -1.05 for the 10th_{percentile and 2.0 for the 90}th

percentile. Due to the high variance in intercepts, the estimates need to be interpreted with respect to the intercepts. For the 10th_{percentile, the estimate equals 0.59 and for the 90}th

percentile, the estimate equals 0.08. Even though the estimate for the 90th_{percentile is much}

lower than for the 10th_{percentile, taking into account the intercepts leads to the result that the}

comment sentiment scores for the 10th_{percentile are much lower for a neutral post sentiment}

(23)

Discussion. The analyses show a positive relationship between the post sentiment and

the comment sentiment for Guinness, Lipton and Budweiser: the more positive the post sentiment, the more positive are the comment sentiments in general. The reason why the result is insignificant for the other brands might be of statistical nature. There might just not be sufficient data points available for a statistically significant result.

The following study explores whether there are other factors besides post sentiments which influence comment sentiments, namely previous comment sentiments.

Study 2: Previous comment sentiments

For Hypothesis 2, it is being analyzed whether the sentiment of previous comments have an influence on the sentiment of the following comments (Equation 2.0). Previous comment sentiments are defined as the first five comments found below a post. The reason why the first five posts are chosen is that Facebook displays only a limited amount of comments when users click on a post. Those posts are as a default setting the most relevant comments to a post. Those most relevant comments are most of the time about five comments. It is possible to choose to see all comments, but it is not necessary for posting another comment under a post. Thus, most users do not even read all of the comments, but only the most relevant five comments. Thus, they can also only be influenced by the first most relevant ones which is equivalent to the first five comments of the dataset.

(24)

Before running a regression analysis, it needs to be checked whether it is allowed to pool the data of all brands. The Chow-Test shows that pooling is not allowed (F=6.32,

Fcrit=1.94). Hence, the regression analyses are conducted for each individual brand.

Results. The results show that there is a significant positive effect for Guinness and

Lipton (Table 4). Moreover, the regression residuals for Coca-Cola’s, Starbucks’ and Lipton’s analysis are normally distributed according to the Shapiro-Wilk Test (p>0.05). The residuals for the regression analysis for Guinness and Budweiser are not normally distributed (p<0.05). Hence, only the results of the analysis for Lipton are taken into consideration, because only those results are significant, and the residuals are normally distributed.

Estimate p-value R2 Budweiser 0.09 0.25 0.02 Coca-Cola -0.07 0.79 0.00 Guinness 0.32 0.02 0.03 Lipton 0.37 0.02 0.08 Starbucks 0.28 0.52 0.01

Table 4: Results regression analysis of hypothesis 2 (Equation 2.0)

Lipton’s analysis indicates that indeed, positive previous comment sentiments lead to more positive sentiments in the following comments. This means that an increase in sentiment score of one unit for the first five comments leads to an increase in the following comments of 0.37 for Lipton.

Since the residuals of Guinness’ and Budweiser’s regression analysis are not normally distributed, bootstrapping is applied to those brands. The results show a positive relationship for both Guinness and Budweiser (ß=0.03 and ß=0.02 respectively). Thus, hypothesis 2 is approved in the case of Lipton, Guinness and Lipton.

Discussion. The results are line with what has been hypothesized by already existing

(25)

Study 3: Effect of emotional reaction to post on comment sentiment

This study investigates hypothesis 3 on how emotional reactions to posts are in relation to the sentiment of the belonging comments (Equation 3.0 and 3.1). The emotional reactions are those buttons with which users can express their opinions about posts. This paper argues that “love” and “haha” lead to more positive comment sentiments than “sad” and “angry”. The results of this study test causation but do not eliminate correlation of the post reaction and comment sentiments.

First of all, in order to run the analysis, it needs to be assessed whether “love” and “haha” can be grouped as well as “sad” and “angry”. Each reaction variable consists of the amount of reactions to a post. In order to group those reactions, it is necessary that the reactions are correlated. Otherwise, the results of the analyses would be distorted. A Pearson’s correlation test shows that the number of negative reactions such as “sad” and “angry” are significantly correlated (p=0.00) and the positive reactions “haha” and “love” are also highly significantly correlated (p=0.00). Thus, the negative reaction variables are combined into one variable as well as the positive post reaction variables.

(26)

3 = <765=>?94@3 53>2 7 = A>94<

First, the comment sentiment scores are assigned to each post reaction group. Afterwards, a regression analysis is run to test whether there is a relationship between positive post reactions and positive post comments and negative post reactions and negative comment sentiments. Five Chow-Tests are run to see whether the regression analyses can be run for all brands combined or whether the analyses need to be run per brand (p>0.05). The results show that pooling is not allowed for either of the brand and post reactions. Hence, a regression analysis per brand and per post reaction group (positive or negative) is run.

Results. The results only show a significant relationship between “haha” and “love”

reactions and the comment sentiments for Starbucks (Table 5). Even though the results show a significant relationship, the effect equals 0.001. This means that one unit increase in positive post reactions lead to an increase of 0.001 in comment sentiments. This effect is almost equal to zero and hence there is a very small to no effect proven.

However, there is a negative relationship between the negative post reactions and the post comment sentiments for Guinness and Starbucks which borders at being significant (Table 6). In fact, one unit more negative post reaction leads to a decrease in comment sentiments by 0.09 for Guinness and to a decrease of 0.01 for Starbucks.

Estimate p-value R2 Budweiser 0.00 0.35 0.02 Coca-Cola 0.00 0.61 0.01 Guinness 0.00 0.71 0.00 Lipton 0.00 0.97 0.00 Starbucks 0.00 0.01 0.28

(27)

Estimate p-value R2 Budweiser 0.00 0.68 0.00 Coca-Cola 0.00 0.75 0.00 Guinness -0.09 0.06 0.02 Lipton 0.39 0.49 0.01 Starbucks -0.01 0.08 0.1

Table 6: Results negative post reactions (“sad” and “angry”) (Equation 3.1)

Discussion. Even though it seems very logical at first sight that a Facebook post with

(28)

Thus, the difference between the positive reactions to the post and the negative sentiments in the post comments might be due to the fact that the post reactions are more superficial, and consumers are rather not as involved as when they write a comment. When consumers see a positively connotated post, a “like” or a “haha” is easier given than writing a comment. When consumers write a comment, they are more involved in what they are writing and think actively about their content. Maybe they even get reminded of the criticism that has been piled up in their heads but has not been triggered yet. As for instance, the Starbucks post about CPR is a very social-related post which let people think beyond their everyday coffee or iced tea and gets them to think in a bigger picture. This resulted in the memory of the police incidence or discrimination incidence, which is as well on the more social, general level than product-related.

(29)

post comments are very likely to be negative as well, because the negativity of the post triggers negativity on the comment level.

Study 4: Effect of post topic on comment sentiment

Hypothesis 4 combines sentiment analysis with topic modelling. To determine the relationship between the post topic and the sentiments of the comments, it is necessary to depict the different post topics. This is done with the help of Latent Dirichlet allocation (LDA). LDA “is a generative probabilistic model of a corpus […] where each topic is characterized by a distribution over words” (Blei, Ng and Jordan, 2003). This approach is applied to the Facebook posts of each brand individually. This approach divides the word documents, in this case comments, into groups and gives a beta score which indicates which words are most associated with each topic. After looking at the beta scores and the words which are most associated with each topic, post topics are assigned to each topic group (i.e.., product-related, brand-related etc.). Next, the sentiment scores for each topic are measured which show which topic yields the most positive or negative comment sentiment scores. An ANOVA is run to test whether there are statistically significant differences between the topic groups per brand.

Applying LDA to the brand posts shows that there are always about three main topics covered in the posts. Those post topics differ among the five brands. Figure 7 shows an example of the different topics for the brands Lipton and Coca-Cola. The figures for the remaining brands can be found in Appendix 1.

(30)

Coca-Cola’s posts are divided into topics about “day of…” posts, call to action posts and meaningful posts. The “day of…” posts contain words like “thursdaythoughts”, “worldkindnessday” or “marchmadness”. The call to action posts feature words such as “watch” or a link to their website to invite followers to visit their website or watch a video. The meaningful posts are mainly about sharing a coke and giving the Coca-Cola product an emotional meaning. In those posts, words like “togetherisbeautiful”, “strong”, “future” and “enjoy” are most prominent.

Budweiser has three different kind of post topics, namely event-related posts, call to action posts and “A bud for…” posts. Call to action posts include challenges and win-a-prize posts, whereas “A bud for…” posts include posts which are specifically for a certain kind of customer group.

Guinness’ post topics, similarly, like Budweiser’s, also entail event-related topic. Moreover, Guinness’ posts are cause-related such as topics about funding and charity or product-related such as posts about beer flavors and limited editions.

Lastly, Starbuck’s post topics are divided into brand-related posts, meaningful posts and posts about store openings. The meaningful posts cover topics such as Valentine’s day or thanking Starbuck’s partners.

(31)

Results. An ANOVA test shows that there are no significant differences between the

sentiments of the topic groups for each brand (p>0.05). However, when looking at the differences, it becomes apparent that there are differences, even if not statistically significant (Table 7). Product-related posts are not the best-received topics in the case of Guinness and

(32)

Lipton (µ=1.63, µ=1.29 respectively). Instead, announcements and cause-related topics are better received by the followers of each brand (µ1.53, µ=1.65). It is important to note that since the differences in means are not statistically different, the latter topics have only slightly better sentiment scores than product-related topics. The sentiment scores for product-related topic comments are still high on the sentiment score scale.

It is surprising to see that those post topics which are defined as “meaningful” and relate to those posts which want to foster emotional brand attachment are received relatively bad. For both Coca-Cola and Lipton, those topic posts are the worst received (µ=0.72, µ1.24). In fact, the “meaningful” posts of Coca-Cola are not only the worst-received posts for the brand itself but among all brands in the dataset.

In the end, Hypothesis 4 can not be proven correct, which leads to the result that product-related posts do not have higher comment sentiment scores than other post topics.

Brand Post Topic Comment Sentiment score Post likes Budweiser 1. event-related 1.43 6960

2. call to action 1.00 1780

3. A Bud for… 0.89 6128

Coca-Cola 1. day of… 1.00 548

2. call to action 1.02 2870 3. meaningful 0.72 4240 Guinness 1. event-related 1.53 610 2. good cause 1.65 570 3. product-related 1.63 760 Lipton 1. product-related 1.29 83 2. announcements 1.53 65 3. meaningful 1.24 59 Starbucks 1. brand-related 0.84 4400 2. meaningful 0.96 5000 3. opening of stores 1.12 8200

Table 7: Comment Sentiments for Post topics

Discussion. The results show that those post topics which deal with cause-related topics

(33)

contrary, posts about already existing products might evoke criticism, because people had negative experiences.

As an additional remark, it is interesting to see that those posts which have a deeper message (hereby related to “meaningful” posts) and are shared in order to foster emotional attachment to the brand have relatively low comment sentiments.

Those include posts by Coca-Cola about the international women’s day and how to be strong for the future or Starbuck’s posts about favorite drinks on Valentine’s day and about how proud they are on their partners. Even though those brands would expect their followers to comment positively on those posts, this is not the case.

To take the discussion one step further, do people react (in form of likes) differently to posts than they do when commenting? Surprisingly, as shown in table 2, “meaningful” posts topics are much more popular in terms of likes than in terms of comment sentiments. As for instance, meaningful posts by Coca-Cola receive about 4240 post likes which is about 33% more than the most popular post in terms of comments. This is extremely contradictive, because even though “meaningful” Coca-Cola posts receive the worst comment sentiment scores of all brands, the post likes are the highest of all Coca-Cola posts. As for instance, Coca-Cola’s post about international women’s day triggered negative sentiments including gender equality comments like “It's funny you hear everything about international women's day, but on November 19 when it's international mens day you don't hear thing about it.” And comments completely unrelated to the post such as “Coca Cola sponsored festival to kill horses”. Yet, this post received a high number of likes.

(34)

to posts but have a higher involvement when commenting and tend to be more critical in those comments.

Study 5: Effect of Comment Topic on Comment Sentiment

(35)

Results. The results of the ANOVA analysis show that there are statistically significant

sentiment score differences for all comment groups (p=0.00). Table 8 shows the sentiment scores of those comment topics which are lowest and table 9 shows the sentiment scores for the comment topics which are highest. For Lipton, the biggest difference in comment sentiment

(36)

mean is between topic 3 (µ= 0.97) and topic 4(µ=1.77). Topic 3 comments are mainly about tea bags and the tea itself. Consumers say that their tea bags have either never broken, did break or they state that they would like to try a certain kind of tea. Those comments are in general rather neutral on average, because the mean sentiment score is close to zero. In topic 4, consumers mainly talk about what they love about the tea which makes this comment topic a very positive one.

For Coca-Cola, the Tukey post Hoc Test shows the biggest difference between comment topic 1 and 5 (p=0.00). Topic 1 (µ=1.37) mainly covers comments about the different flavors, especially vanilla and orange which consumer are very positive about. Topic 5 (µ=0.88) mainly entails comments about the availability of their products in supermarkets. People state that they can not find a specific Coke in their store. Hence, topic 5 has a relatively low average sentiment score.

Budweiser’s comment sentiment scores are lowest for comment topics relating to the brand, whereas they are highest for comment topics relating to the product as such. Guinness’ comments which relate to the product have the lowest sentiment score, whereas those comments related to experiences related to the brand yield the highest sentiment scores. Lastly, Starbucks shows the lowest sentiment score for comment topics related to the brand and the highest sentiment score for comment topics related to the product.

Brand Topic Comment Sentiment score Budweiser Brand-related 0.52

Coca-Cola Product-related 0.88

Guinness Product-related 0.37

Lipton Product-related 0.97

(37)

Brand Topic Comment Sentiment score Budweiser Product-related 1.6 Coca-Cola Brand-related 1.37 Guinness Experience-related 2.23 Lipton Brand-related 1.77 Starbucks Product-related 1.52 Table 9: Highest sentiment scores among comment topics

Discussion. The results show that comments related to the product itself are not better

received by followers and do not have higher sentiment scores than other comment topics. This is not surprising, because hypothesis 4 had similar findings: Post topics about the product itself do not have higher sentiment scores than for other topics.

Moreover, the comment sentiment scores differ per brand and per comment topic. As for instance, brand-related topics are best received by Lipton’s follower base but worst received by Budweiser’s follower base.

5.0 General Discussion

The findings of this paper show which factors influence the sentiments of Facebook comments in five studies. Study 1 examined the relationship between post sentiment and comment sentiment. In study 2, the influence of previous comment sentiments on the following comment sentiments was measured and in study 3, the relationship between post reactions and comment sentiments was analyzed. The influence of post and comment topic on the comment sentiment was investigated in study 4 and 5.

(38)

positive posts are in fact more positively received by consumers than negatively valanced posts. Consumers react more positively to positivity.

Hypothesis 2 is also approved, since the results show that positive previous comments lead to positive subsequent comments. This confirms ideas of social conformity: People adjust to the opinions by others to fit in with society. When consumers comment to a brand’s post, they are indeed being influenced by what has been commented by other users beforehand.

Hypothesis 3a can not be approved, because study 3 shows that positive post reactions can not be taken as an indication of the comment sentiments which contradicts already existing literature in this field (Tian, Galery, Dulcinati, Molimpakis and Chao Sun, 2017). Hypothesis 3b is only very limitedly confirmed: Only for one brand negative post reactions indicate negative comments.

Concerning study 4 and post topics, it is important to note that meaningful posts are mostly worst-received, whereas cause-related and announcements are better received than other post topics. However, the post and comment topics evoke comment sentiments which can only hardly be generalized. As for instance, product-related topics evoke different sentiments for Coca-Cola than for Starbucks.

(39)

6.0 Scientific Contribution

This paper contributes to the academic world, because it gives an example on how a qualitative data analysis technique can be turned into an insightful quantitative statistical analysis. Sentiment analysis and topic clustering are rather qualitative ways to analyze big texts which makes statistical hypothesis testing difficult. However, this paper exemplifies how to turn those techniques into variables which can be used for regression analysis.

From a scientific point of view, this paper fills a gap in literature about social media comment sentiments. A combined analysis of sentiments and topic modelling on comment sentiments has not been operationalized to date. It sets a milestone for this kind of social media analysis and serves as a foundation for further research.

7.0 Managerial Contribution

The author of this paper advices social media managers to carefully consider the overall sentiment of their post: Highlighting the positive sides in their posts leads to more positive reactions in form of comment sentiments. As for instance instead of stating that “We are sorry to announce that we ran out of stock for this product”, it is wiser to phrase it as “We are happy to announce that our product will be back in stock soon”.

Since consumers are being influenced by the most relevant comments to a Facebook post, for a brand it is important to keep an eye on the first five comments to their posts in order to respond appropriately and prevent an overall negative valence in the first comments being shown.

(40)

Facebook posts. Hence, this paper advices social media managers to take a careful look at the comments below a post to dig deeper and find the roots of criticism – they are most of the times somewhere completely different than expected.

Moreover, it is advisable that marketing managers should avoid to post about meaningful topics in order to avoid negative emotional laden comment reactions. Instead, cause-related or announcements are much better received by followers.

With these insights, marketing managers can foster positive sentiment scores and thereby build a strong relationship to their customers. With this relationship, brand loyalty and advocacy can be built in the long run which is an essential asset for the firm.

7.0 Limitations and Further Research

A limitation to the studies is that there are only limited explanatory variables involved in the conceptual model which leads to only a limited explained variance in the regression models. As for instance, each individual customer is described with a combination of random words and numbers due to privacy concerns. Hence, no follow-up on demographic variables is possible and those factors stay undiscovered.

Another limitation to this paper is related to LDA. Defining topics from the LDA results is a subjective procedure. The output solely gives a beta which indicates the strength of association to the topic of individual words but does not define those topics. Hence, it is possible that different individuals besides the author of this paper define the topics differently.

(41)

(42)

References

Arora, Poonam and Carolyn E. Predmore (2013), “Social Media as a Strategic Tool: Going Beyond the Obvious,” Social Media in Strategic Management Advanced Series in

Management, 115–27.

Barcelos, Renato Hübner, Danilo C. Dantas, and Sylvain Sénécal (2018), “Watch Your Tone: How a Brands Tone of Voice on Social Media Influences Consumer Responses,” Journal of

Interactive Marketing, 41, 60–80.

Blei, David M., Ng, Andrew Y. and Michael I. Jordan (2003), “Latent Dirichlet Allocation”, Journal of Machine Learning Research, (3), 993-1022.

Broad, C. D. (1954), “Emotion and Sentiment,” The Journal of Aesthetics and Art Criticism, 13 (2), 203.

Cvijikj, Irena Pletikosa and Florian Michahelles (2011), “Understanding social media marketing,” Proceedings of the 15th International Academic MindTrek Conference on

Envisioning Future Media Environments - MindTrek 11.

Gallan, Andrew S., Cheryl Burke Jarvis, Stephen W. Brown, and Mary Jo Bitner (2012), “Customer positivity and participation in services: an empirical test in a health care context,” Journal of the Academy of Marketing Science, 41 (3), 338–56.

(43)

Kaplan, Andreas M. and Michael Haenlein (2010), “Users of the world, unite! The challenges and opportunities of Social Media,” Business Horizons, 53 (1), 59–68.

Krebs, Florian, Bruno Lubascher, Tobias Moers, Pieter Schaap, and Gerasimos Spanakis (2018), “Social Emotion Mining Techniques for Facebook Posts Reaction

Prediction,” Proceedings of the 10th International Conference on Agents and Artificial

Intelligence.

Liu, Bing (2017), Sentiment analysis: mining opinions, sentiments, and emotions, New York: Cambridge University Press.

Moe, Wendy W., and David A. Schweidel. "Online product opinions: Incidence, evaluation, and evolution." Marketing Science 31.3 (2012): 372-386

Reisenbichler, Martin and Thomas Reutterer (2018), “Topic modeling in marketing: recent advances and research opportunities,” Journal of Business Economics, 89 (3), 327–56.

Satapathy, Ranjan, Erik Cambria and Amir Hussain (2019), Sentiment analysis in the

bio-medical domain: techniques, tools, and applications, S.l.: SPRINGER.

(44)

Schweidel, David A. and Wendy W. Moe (2014), “Listening in on Social Media: A Joint Model of Sentiment and Venue Format Choice,” Journal of Marketing Research, 51 (4), 387– 402.

Shen, Bin and Kimberly Bissell (2013), “Social Media, Social Me: A Content Analysis of Beauty Companies’ Use of Facebook in Marketing and Branding,” Journal of Promotion

Management, 19 (5), 629–51.

Tian, Ye, Thiago Galery, Giulio Dulcinati, Emilia Molimpakis, and Chao Sun (2017), “Facebook sentiment: Reactions and Emojis,” Proceedings of the Fifth International

Workshop on Natural Language Processing for Social Media.

Tetlock, Philip E., Linda Skitka, and Richard Boettger (1989), “Social and Cognitive

Strategies for Coping with Accountability: Conformity, Complexity, and Bolstering,” Journal of Personality and Social Psychology, 57 (October), 632–40.

Thomson, Matthew, Deborah J. Macinnis, and C. Whan Park (2005), “The Ties That Bind: Measuring the Strength of Consumers’ Emotional Attachments to Brands,” Journal of

Consumer Psychology, 15 (1), 77–91.

Turri, Anna M., Karen H. Smith and Elyria Kemp (2013) “Developing Affective Brand Commitment Through Social Media” Journal of Electronic Commerce Research. 14 (3), 201-214.

(45)

(46)

Appendices

(47)

(48)

(49)

Appendix 3

R-Script is sorted by Hypotheses: rm(list=ls())

setwd("/Users/evawilke/Desktop/Master/Marketing Intelligence/Thesis") #Loading Posts from 5 different beverage companies

Posts_Starbucks <- read.csv("Comment_Files_Eva/Posts/Posts_Starbucks.csv", header = TRUE, sep='\t')

Posts_Coke <- read.csv("Comment_Files_Eva/Posts/Posts_CocaCola.csv", header = TRUE, sep='\t')

Posts_Bud <- read.csv("Comment_Files_Eva/Posts/Posts_Budweiser.csv", header = TRUE, sep='\t')

Posts_Lip <- read.csv("Comment_Files_Eva/Posts/Posts_Lipton.csv", header = TRUE, sep='\t')

Posts_Gui <- read.csv("Comment_Files_Eva/Posts/Posts_Guinness.csv", header = TRUE, sep='\t')

#Loading Comments from 5 different beverage companies Comments_Starbucks <-

read.csv("Comment_Files_Eva/Comments/Comments_Starbucks.csv", header = TRUE, sep='\t')

Comments_Bud <- read.csv("Comment_Files_Eva/Comments/Comments_Budweiser.csv", header = TRUE, sep='\t')

Comments_Coke <- read.csv("Comment_Files_Eva/Comments/Comments_CocaCola.csv", header = TRUE, sep='\t')

Comments_Lip <- read.csv("Comment_Files_Eva/Comments/Comments_Lipton.csv", header = TRUE, sep='\t')

(50)

HYPOTHESIS1:

#Loading Posts from 5 different beverage companies