The effect of three different types of digital content marketing posts on consumer response : a sentiment and qualitative analysis of Facebook responses

(1)

Floor Banga (10003975) University of Amsterdam Faculty Economics and Business Thesis MSc Business Administration Specialization: Digital Business Supervisor: dr. A. Nayak Date of submission: August 18, 2017 Word count (excluding qualitative quotations): 14382

The effect of three different types of digital

content marketing posts on consumer

response

(2)

Abstract

Purpose Nowadays, due to technological and social changes of the internet, consumers take on a

more central role in the marketing activities, co-creating value of the marketing activities. This forces marketers to identify and meet consumers’ needs, by offering something valuable in order to receive value in return. Digital content marketing does this, by offering ads in the form of a freebie rather than a traditional ad. Digital content is usually characterized by subjective characteristics like esthetics. This research aims to categorize based on characteristics of purpose (entertaining, informative or promoting). It aims to test if ‘freebie’ content (entertaining, informative) receives a better response sentiment than traditional ads (promoting) as hypothesized based on existing literature. Furthermore, it explores which other variables influence the effect of content marketing on consumer response as this yet remains unknown.

Methodology 12000 consumer responses to content posts on Facebook were analyzed through

sentiment analysis. An interrater reliability test was used to sort these into the three content types. Through an ANOVA, an observed effect between content types was determined, that was further analyzed trough qualitative analysis of the responses.

Originality/value This led, next to new ways to categorize content, to a contradicting result which

states that in the scope of Facebook, traditional (promoting) type of content can be more beneficial than informative content. Also, it indicated the probable variables that influence the effect of digital content marketing on Facebook, and thus opens the black box of digital content characteristics and variables.

(3)

Statement of originality

This document is written by Floor Banga who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(4)

1. Introduction

While digital content marketing is relatively new and saw a huge growth in the last five or so years, content marketing itself is not new at all. The first clear example of content marketing dates to 1895, namely John Deere’s Furrow magazine. This magazine debuted over 120 years ago and nowadays still obtains around two million readers globally (de Bakker, 2013). The idea behind this magazine, was to create a way to help small business farmers to do better at farming. Instead of saying “we have this new beautiful innovative tractor” as normal advertisement would do, the company of John Deere decided to offer the consumer something of value by every month delivering useful content. Another striking example, but from current times, is Red Bull Media House, originally only selling beverages, but now operating as a media company (Pullizi, 2015). Nowadays, companies have to deal with a customer-centric marketing environment. Through new technologies consumers have their devices with them 24/7 and they are in control, they can choose to ignore or avoid advertisements completely. Therefore, marketers must find new ways to attract consumers’ attention. Digital content marketing does so through the aim of building long term relationships with consumers by offering them valuable experiences outside of the products and services (Pullizi, 2015). While examples of John Deere and Red Bull that turned in to actual publishers, are on the more extreme side, through digitalization of the world all companies are in some sense (potential) media companies and digital content marketing is one of the fastest growing areas of internet marketing. The most successful brands have proven to be content hero’s. To be a content hero, means to offer helpful, exceptional, relevant and enjoyable content that binds consumers, so that the next time they need something, the content hero’s brand is the first one on their mind (Pullizi, 2015).

(7)

1.1 Literature gap

Because the rapid rise of digital content marketing is still relatively new, a lot of unknowns remain. Rakic & Rakic (2014) state that because there are infinite possibilities in creating content, the environment in which content is created, moves very rapidly and due the lack of talented content creators, it is difficult for companies to create valuable content. The result is a quantity over quality situation. Which may lead to a digital marketing myopia. The term “marketing myopia” is coined by Theodore Levitt (1960). When a firm has a marketing myopia, it means that a company is focusing too much on selling a product rather than fulfilling a customer’s needs and creating value. Rakic & Rakic (2014) argue that this is what happens when a firm delivers content that does not meet the consumer’s needs. Their suggestion is that it should be researched whether this myopia exists, what the causes and consequences are and how it can be solved. After all, if a content marketing strategy fails to deliver value due to a myopia, then the marketing effort becomes useless (Rakic & Rakic, 2014). In line with this gap, is the need identified both by Koiso-Kantilla (2004) and Rowely (2008). They argue that there is not enough understanding on the characteristics of digital content that are of most importance to digital content marketing and how they create value or meet consumer’s needs. When more is known about which characteristics possibly create value, a more specific approach can be used in content creation, diminishing the likeliness of marketing myopia. Koiso-Kantilla (2004) proposes further research on that by executing real case studies and Rowley (2008) puts emphasis on the urge of developing a taxonomy for digital content marketing including most important characteristics. Because these characteristics, that make content valuable, are rather ambiguous (useful, original, amusing etc.), research on how content can be objectively identified into different types of content is needed. Once these different types are developed, it can be researched how these different types impact consumer response. By evaluating the impact of different types, a beginning of a taxonomy of digital content can be made. Also identifying the

(8)

different types and their effect on consumer response is a first step towards understanding the value consumers get from digital content. Therefore, the following research question will be central to this research:

RQ: How do different characteristic based types of digital content used in digital

content marketing affect consumer response sentiment?

This research aims to find an answer to this question in the scope of Facebook by analyzing responses to content posts by brands through a sentiment and qualitative analysis. The aim of this research is to identify by which characteristics digital content as part of a digital content marketing strategy can be categorized into different types. Next to the identification of these characteristics, it also means to achieve a better understanding on how these characteristics influence the customers response sentiment. However, as there is still little known about the characteristics of digital content marketing and how they work, one of the prior goals is to gather new insights to serve as a basis for future research. It does so by observing real life situations following the philosophy of Eysenck (1975 pp. 9):

“Sometimes we simply have to keep our eyes open and look carefully at individual cases-

not in the hope of proving anything, but rather in the hope of learning something”

Firstly, the research will go into a comprehensive literature review to construct a conceptual framework for the research. Thereafter, the appropriate methodology will be explained, this will continue into the results of the first part of the first part of the research and the observed effect. This observed effect will be used as a starting point for the qualitative analysis. Finally, the research will conclude and aim to give an answer to the research question in the discussion.

(9)

2. Literature review

2.1 Defining digital content marketing

To fully grasp what digital content marketing is all about, it is necessary to state a clear definition. To do this, it must first carefully be specified what is meant with “digital content” in this research. Both Koiso-Kantilla (2004) and Rowely (2008) define digital content as follows:

“Digital content is conceptualized as bit-based objects distributed through electronic channels”.

It is important to keep in mind that digital content has an informative nature, it can be seen as an ‘information good’ or ‘electronic information product’ (Rowely, 2008). Because of this nature, the boundary between being an information product or a marketing communication can be a gray area. Companies (almost) always include information with their products, to make sure the customer can gain an optimum benefit from the product, i.e. a user’s manual for a coffee machine or the nutritional facts of cookies.

(Digital) content becomes a marketing activity however, when the information extends beyond the technical details about the product itself. In that case the information serves as an extra or as a ‘freebie’, to serve as an attractor for customers (Rowely, 2008). In order to do this the customers’ needs must be identified, anticipated and satisfied just as in traditional marketing. By combining the definition of digital content and its informative nature with the definition of traditional marketing, Rowely (2008) developed a comprehensive definition of digital content marketing:

(10)

Digital content marketing is the management process responsible for identifying,

anticipating, and satisfying customer requirements profitably in the context of digital content, or bit-based objects distributed through electronic channels.

A good illustration to this definition is an explanation by James O’Brian (2012):

"The idea central to [digital] content marketing is that a brand must give something valuable to get something valuable in return. Instead of the commercial, be the show. Instead of the banner ad, be the feature story. The value returned is often that people associate good things with — and return to engage with — the brand"

This idea is supported by Pullizi (2012), as he states the notion that in this day and age storytelling should be at the center of all marketing. All non- media corporations need to be involved in tactics originally used by traditional media publishers and should behave as media companies to thrive in the digital marketing arena (Pullizi, 2012). The only difference between a non-media company using digital content marketing and a real media company is the fact that a media company obtains its profit directly from selling the information product it offers and a non-media company offers its information product for free as a way to attract and retain customers for their primary products or services (Pullizi, 2012). In other words, digital content marketers use the product originally only produced by media companies as a way of marketing. Examples of these information products or digital content are: articles, blogs, social media posts, videos, information guides, images and so on (Pullizi, 2012) (Rakic & Rakic, 2014).

(11)

2.2 Why is digital content marketing important and why now?

A switch in focus from companies’ general marketing activities towards digital content marketing has become more and more evident (Pullizi, 2012). This is also supported by a quick analysis on Google Trends; it shows a rapid growth in the number of times “content marketing” is Googled starting from 2011 until present time (Figure 1).

Figure 1: Numbers represent search interest relative to the highest point on the chart for the given

region and time. A value of 100 is the peak popularity for the term. A value of 50 means that the term is half as popular. Likewise, a score of 0 means the term was less than 1% as popular as the peak (Google Trends, 2017).

The causes of this switch of focus can be traced back to two major changes in how the modern-day internet works, how people use it and why they use it. Firstly, digital content is closely related to Search Engine Optimization or SEO, because changes in the algorithms of main search engines like google, forced websites and companies to create complete digital content rather than just using keywords to stay in the top results of a search engine (Pullizi, 2012). This research however, will focus on digital content marketing within social media rather than on the scope of the whole web, and therefore will not go any further into the details of SEO. The second cause of the switch towards digital content marketing, lies in the rise of social media platforms. This however, is of great interest for this study and will be further discussed in the next section.

(12)

2.2.1 The rise of Social Media Platforms: consumers as co-creators of brands

One of the main causes for companies’ need to participate in digital content marketing, lies in the rise of social media platforms. Social media platforms revolve for a large part around content creation and content sharing. Social media allows users to create their own content (user generated content) and/or to share and react to existing content created by other people or companies. Current most famous and relevant examples are Facebook, Twitter and Instagram. New media like these replaced the traditional media outlets and their marketing potential. This means consumers don’t encounter traditional advertising as they used to, by streaming or downloading tv-shows and videos, they avoid television advertising. In addition, consumers have become very skillful in using the internet without encountering traditional online ads due to new technologies like adblockers (Ong Yi Lin & Yazdanifard, 2014). Furthermore, the growing relevance of social media to consumers’ daily lives has increased the marketing potential of the social platforms and has caused a shift in marketing towards a service-dominant logic. In this logic, the customer stands at the center of marketing theory, which means that the value of a product or service is co-created by the customer, instead of only by the outputting party (Hutter & Hautz, 2013). Next to that the brand value is co-created as well, through social interactions between the consumers and other stakeholders. With this new logic, brands have become complex social phenomena in which the consumer has a dominant role, which means a change in traditional marketing towards a more social form of marketing (Hutter & Hautz, 2013).

(13)

The way marketing and branding affect the process of decision making of a consumer between buying or not buying a product can be understood through the Hierarchy of Effects (HOE) model (fig. 2), derived from the field of advertising and communication (Hutter & Hautz, 2013).

The HOE model consists of three stages, in the first stage the consumer attains awareness and knowledge about the brand or product (cognitive stage), in short brand awareness. In the second stage, he or she develops positive or negative feelings about this product or brand (affective stage), in short word of mouth (WOM) or willingness to share with others (Hutter & Hautz, 2013). In the last stage, the decision between buying/using or rejecting/avoiding the brand or product is made (conative stage), in short purchase intention. However, these three stages are influenced by the shift towards content marketing on social media platforms; the social media marketing activities Figure 2: The Hierarchy of Effects model,

depicting the three stages a consumer runs through before making a purchase decision (Hutter & Hautz, 2013).

(14)

of a company affect the cognitive, affective and conative stage the consumer goes through. This happens through two independent variables. The first variable is, Brand Page Commitment (BPC) or the active and psychological involvement of a consumer with the social media activities of a brand. BPC has a positive effect on brand awareness, WOM and purchase intention (Hutter & Hautz, 2013). However, the second variable, annoyance, occurs when social media content turns into something intrusive and annoying that turns the consumer away and has a negative effect on commitment of the brands social media page and the WOM. Potential drawbacks of Social Media are spread of negative WOM and content causing annoyance, for example by information overload (Hutter & Hautz, 2013). Marketing managers should thus aim to keep the BPC variable as high and the annoyance variable as low as possible to promote engagement and prevent negative WOM. In their research, Hutter et al. (2013) have found significant proof that by influencing the stages of the HOE model, social media (Facebook), influences the economic outcome of brands. However, economic outcomes are not the only desirable outcomes. Returns from social media investments should not always be measured in dollars but also in consumer investments. Consumer investments are for example time spent on the page and WOM (Hoffman & Fodor, 2010). Moreover, WOM and digital content marketing on social media have become inseparable forms of marketing. The next section will go deeper into Facebook, one of the most important social media platforms with regards to digital content.

(15)

2.3 A Social Content Giant: Facebook

2.3.1 Basics of Facebook

With a market share of 39,36% Facebook is the largest within the social media segment (Kallas, 2017). On this platform, users can befriend other users, follow brand pages, create content, share content, like content and leave comments to content. Because Facebook evolves around content creation and sharing, content marketing on Facebook plays a large role and is a very attractive way for companies to brand and market their products and interact with their customers. The most important components of Facebook with regards to content marketing will be briefly discussed.

The news feed appears on every user’s homepage. Here, information that is relevant to the user is highlighted through Facebook’s algorithms, for example photos, status updates, links and posts posted by friends appear here. Next to that, also posts by brands the user follows and recommended posts will appear on the newsfeed. The newsfeed is the place where content becomes visible through word of mouth. The word of mouth on Facebook is build up from frequency of appearances on the newsfeed, number of comments, tags, likes and shares (Hoffman & Fodor, 2010).

All content posted on Facebook has a comment section, where users can easily react to the content. When a user reacts to a content post, the friend’s users will see this in their news feeds.

Tagging gives users the opportunity to easily link a friend to a post, this is a very common

occurrence in the comment section of content posts. Users simply write the name of a friend, upon which the friend receives a notification that links back to the content and the accompanying comment. Tagging is an easier way of sharing content without actually adding anything to your own page, and thus is very popular. Because of this, large parts of the comments section are made up of ‘empty’ comments containing only name tags. While these types of comments are valuable

(16)

as WOM, they more closely resemble sharing and liking as they usually do not contain qualitative information that comments usually do contain.

The like button enables users to quickly show their approval of a content post. It is a quick nod in the form of a thumbs up, that shows that the user liked that post. Once a user likes a piece of content, this content will appear in the news feed of the user’s friends. Next to that, it will also indicate the number of people that liked the content. Not only posts can receive likes, also pages from brands, famous people etc. can be liked. The number of likes to for example a company’s page gives a quick but strong indication about the popularity of the company. Also, the number of likes to the posts of a company relative to the total likes of the company’s page gives an easy indication of the engagement of the company.

Another but more active way to share content to friends’ newsfeeds, is by using the share function. This function does not only share to the newsfeeds of friends, but also permanently posts the content to the user’s personal profile.

With regards to digital content marketing this means that there are three ways in which content from a brand becomes visible for a Facebook user. Firstly, the content appears in the newsfeed because the user himself liked or followed the brand page. Secondly, content might appear because a friend of the Facebook user liked, shared or commented on the content. Lastly, brands can pay to make their content more visible, in that case the content will appear for relevant users as a sponsored post.

For companies, with regards to marketing purposes, this visibility or digital WOM is of high importance. Research shows that Facebook users are 30% more likely to recall the brand if users see that one of their friends has liked or commented on a brands post (Ang, 2011). Engaging Facebook users through making them comment to posts is even more desirable. Research (n = 112.000) has shown that people are 65% more likely to buy from a brand and 55% more likely to

(17)

recommend the brand to a friend if they engage in online positive conversation (Tsai, 2009). However, negative WOM has an extremely high effect, as 90% of consumers do not purchase products or services of a company involved in negative word of mouth (Woerndl, Papagiannidis, Bourlakis & Li, 2008). Thus, the aim of digital content marketers is to create content that encourages people to like and share and to positively engage in online conversations by commenting to the content posts. To be able to understand how content can encourage people to engage, the next section will discuss the psychology behind Facebook and what makes people willing to like, share and comment.

2.3.2 Valuable Content: what makes people comment, like and share

As becomes clear from the previous section, there is great marketing potential in the positive engagement of consumers with brand generated content within the Facebook platform. This section will explore the reasons to why consumers choose to engage with a brand on Facebook.

In a research by Syncapse (Kalehoff, 2013) 2080 Facebook users that were following a brand’s Facebook page were asked why they became a follower of the brand on Facebook. The main answers were: to support the brand, to get a coupon/discount, to receive updates and to participate in contests. These findings are summarized in Table 1.

49% To support the brand I like 27% To share my interests / lifestyle with

others

42% To get a coupon or discount 21% To research brands when I was looking

for specific products/services

41% To receive regular updates from

brands I like

20% Seeing my friends already liked the brand

35% To participate in contests 18% A brand advertisement led me to fan the

brand

31% To share my personal good experiences 15% Someone recommended me to fan the

brand

(18)

From these results, Kalehoff (2013) concluded that there are two main reasons people follow a brand’s Facebook page. These are either practical, because they get something transactional in return (e.g. discounts, contests) or they are of emotional nature (e.g. to support the brand). Kalehoff (2013) argues that there are “good” and “bad” fans, the good ones being the fans that follow a page due to emotional motivations rather than transactional ones. Brands should try to avoid being the target of deal hunters that form a lower-quality fan membership that consists of low-engagement followers with low purchase intentions. Instead brands should aim to use content that serves as relational and emotional motivators that attracts high-quality, loyal fans (Kalehoff, 2013).

Research by the Consumer Insights Group of the New York times (Brett, 2011) sheds some light on how to achieve this loyal fanbase. According to the research fans are willing to share content because they want to bring valuable, enlightening and entertaining content to people they care about. They want to define themselves, show causes they believe in and grow and maintain relationships with fellow users. Furthermore, sharing brings users a sense of self-fulfillment and they state that, not only do they connect to friends and the content more, but they also feel more connected to the world. In addition, research by IPSOS (Wiltfong, 2013) among 12,420 global content ‘sharers’ on Facebook, shows that people mainly share content because they seek to share interesting things (61%), to share important things (43%) and to share funny things (43%). In addition, research has found that the main reasons for users to engage on the internet are entertainment and interactivity (Raney, Arpan & Pashupati, 2003).

However, in contrast to why people like, there are also some indicators to why people choose to not like a brand or content of that brand. In another research (n = 626), Facebook users were asked why they were cautious to liking a brand page. Major findings were that they did not want to be bombarded with advertisements (54%) and/or that they did not see the benefit (23%) in it (ExactTarget, 2011). This research indicates that users do not want to see content in the form of

(19)

a traditional ad and they want content to offer them some sort of benefit. This is in line with one of the main advices given in a report by Google (Lecinski, 2014) towards marketers: “Have something

interesting, relevant and/or engaging to say. “Links to products [e.g. traditional ads] are no longer enough, brands should provide experiences that are as informative and entertaining as possible”

(Lecinski, 2014). This is also backed by a notion from a model known as the Persuasion Knowledge Model. According to this model, when a person has conscious awareness of a persuasion attempt (as in traditional ads), this person will use coping tactics that redefines their attitude towards the advertisement and brand. To overcome this, marketers must us communication that is not directly recognizable as an ad (Raney, Arpan, Pashupati & Brill, 2003).

When it comes to motivations of why people comment to content it can be said based on a research among 1200 respondents, that they do so simply “when they have something to say” (Burke, Kraut & Marlow, 2011). However, commenting requires more time and effort than simply clicking the like or share button. This explains why in general content receives significantly more likes and shares than comments. On the other hand, it also suggests that the people who do comment, are highly engaged compared to the mere ‘likers’ (Shen & Bissels, 2013) and that commenting is an action with powerful emotional drivers (Burke, Kraut & Marlow, 2011). This means that people who comment positively are highly positively engaged but the negative commenters are likely to have a high negative brand engagement.

Marketing takeaways

Based on these findings from multiple researches, some pointers were already constructed as takeaways for today’s marketers. The Consumer Insights Group of the New York times (Brett, 2011) indicates four valuable lessons:

1. Post content that appeals to the motivations of people to connect with each other, not

(20)

engagement are driven by building and maintaining relationships between each other. It is important to create a relationship between the consumer and the brand, however, it is even more important to create content in which stimulants are embedded that trigger the consumer to interact with their peers, as this stimulates the sharing and engagement of the content which in itself improves visibility.

2. Post content that appeals to the consumers’ sense of humor: From the research (Brett,

2011) it appeared that humor was one of the most important drivers to make people share content with their peers. This is because, as became clear from the study, the sharing part of content that is humorous is essential to enjoying the content. Next to that it was found that there is substantial social reward or credit in being the person that shares the funny content with others (Brett, 2011).

3. Keep it simple: In digital content marketing, this has become even more important than it

already was in traditional marketing. First, the content needs to be straightforward so that it quickly appeals to the consumer and that they can understand it (Brett, 2011). Secondly, as content can be rapidly and widely spread, it is more prone to distortion. To prevent content from being misinterpreted and possibly leading to negative response, complexity should be avoided (Brett, 2011).

4. Trust is essential. According to Brett (2011) if consumers don’t trust the content’s source

or doubt the accuracy or motivation behind the content they will not (positively) bring this into the lives of the people they care about and are thus less likely to share or engage with the content post.

(21)

5. Content must make sense contextually. In extension of the fourth point, is a suggestion

by Rowely (2008) that the content a brand publishes, must be in context with the brand itself or with the circumstances of the brand. Content must make sense contextually, when in the eyes of the consumer it doesn’t, it is likely that they will doubt the accuracy or trustworthiness of the content and brand in question (Rowely, 2008).

Adding to that based on the findings of IPSOS (Wiltfong, 2013), Raney et al. (2003) and Lecinski (2014) that

6. Content should be as entertaining or informative as possible. This is based on the

findings from Raney et al. (2003) that people tend to seek entertainment and interactivity online, the findings from IPSOS (Wiltfong, 2013) that people want to share and engage with interesting and funny things on social media and the idea that people get more emotional attachment when they enjoy the content and are triggered to share this with people they care about. Combining this with the number one reason people tend to not like or engage with a brand on social media (the fear of being bombarded with advertisement) and the aim of marketers to avoid deal-hunters associated with promoting content, leads to the suggestion that content should be as entertaining or informative as possible rather than a traditional ad type of style that is more promoting of nature, which is backed by the argument of Lecinski (2014).

Categorizing content based on objective characteristics

Points one to three are rather ambiguous because they are influenced by subjective values like esthetics, humor and emotions. So, instead of categorizing content based on subjective characteristics, based on the findings summarized in point six, categories of content can be identified that could serve as a more objective way to characterize digital content. Following the

(22)

lines of the literature, these categories are then entertaining content, informative content and

promoting content. The next section will incorporate these categories, summarize all the findings

of the literature review and visually display them in one conceptual framework, that will serve as the basis of the research.

2.4 Conceptual framework, hypothesis and propositions

Combining the findings of the literature review leads to a conceptual framework in which digital content on Facebook can either result in in BPC or Annoyance (Hutter et al. 2013). BPC than has a positive impact on WOM in the form of liking, sharing and positive commenting. Annoyance however can result in negative word of mouth in the form of negative comments. Next, WOM has an effect on visibility, which then has a positive impact on brand awareness. Furthermore, positive brand awareness has a positive effect on purchase intentions that lead to higher economic outcomes. However, value is not only determined by economic outcomes, but could also consist of other consumer investments like very actively promoting the brand (positive word of mouth). These findings are summarized in figure 3.

(23)

Figure 3: Conceptual framework showing the place of H1 in the mechanism derived from existing

literature. The framework combines earlier mentioned findings from: a= Hutter et al. (2013),

(24)

Through the mechanism of BPC, WOM, visibility and brand awareness, the existing literature concludes that digital content can create higher economic outcomes and consumer investments. However, digital content is still treated as a black box, as to date it has been characterized in subjective ways. Therefore, it remains unknown how differentiations in digital content marketing affect the WOM, or in other words, how different types of content influence the response sentiment (negative or positive WOM).

This research breaks up the digital content construct into three types: entertaining, informative and promoting type of content. The focus will be on the effect of the type of content on the word of mouth, more specifically on the negative and positive comments of the consumer. Based on the literature findings it is expected that entertaining and informative content have a better effect on response sentiment, because they make use of the ‘freebie-effect’. Assuming that the findings of the literature review that positive WOM has a positive effect on brand awareness and value hold as a fact, knowing more on the impact of the content types on WOM also means knowing more about the impact on value.

From these literature findings, the hypothesis that is central to this research is as follows:

H1: Entertaining and informative content will have a higher positive and a lower negative

consumer response sentiment score than promoting content.

In addition to this hypothesis, based on the literature the following suggestions are proposed: Proposition 1: Out-of-context content will cause negative consumer responses

Proposition 2: Promoting content will attract low-engagement ‘deal-hunters’ rather than

(25)

3. Methodology

In this section, the used methodology will be discussed in detail. Firstly, there will be elaborated on the way the method was designed and why this is suitable. Thereafter the separate steps of the research method will be discussed in more detail.

3.1 Research approach

Fisher (2009) states that successful social media programs are successful because they revolve around people, because of that they are highly people- and consumer-centric. Thus, a collection of just figures (quantitative information) is not enough to tell the complete story. Fisher (2009) identified attributes that still need more measurement. These are the sentiment attribute and the qualitative attribute. Based on the notion of Fisher (2009) the research was build up from a quantitative part and a qualitative part. More specifically the quantitative part is based on sentiment analysis and the qualitative part on case studies as suggested by Koiso-Kantilla (2004). Important questions to ask are: what is going on? What happened? This part is covered by a sentiment analysis on comments of the Facebook pages of ten different companies. After that, what really matters then are the opinions, voices and experiences that people are sharing that will be explored in the qualitative part. Therefore the focus of the research is on the Facebook responses in the form of comments as this is the part of word of mouth that contains qualitative information about whether and why it is negative or positive word of mouth, that cannot be observed in shares and likes.

To build up a sentiment database for different types of content this research used Facebook as a data source as it is one of the most important social networks with regards to digital content and it offers free and easy accessible data. After selection of the Facebook posts and sentiment analysis, a qualitative analysis was performed on the comments in the shape of clustered case studies.

(26)

3.2 Digital content marketing posts: selecting, cleaning and sorting of the data

3.2.1 Data collection

As mentioned above, Facebook was used as the main source of data. This data consisted of consumer responses to digital marketing content posted by a company. Because the research aims to understand consumer responses to digital marketing content, only content generated by the company as part of a marketing activity was considered, thus user generated content was not included in the research. By making use of the Facebook API, the 100 first responses to a post were loaded into the statistical computing software R and saved into a simple text database.

Ten companies were selected to provide with the digital marketing content. This is because ten companies are a large enough number to have a variety in the sort of company and the products or service they offer but at the same time small enough to be suitable for the time limitations of this research. The companies were selected on a couple of criteria; first, they had to practice some sort of digital content marketing campaign on Facebook. Secondly, their Facebook page should be in English so that the content marketing posts and the majority of the consumer responses would be in English. This was required for the used sentiment analysis software. Finally, the company should have enough online engagement that provides enough response to be analyzed to each post. Based on these criteria the following companies were selected to serve as a data source:

(27)

Company Market

Kate Spade New York Fashion

Nasty Gal Fashion

Coca Cola Food and Beverages

Pepsi Food and Beverages

Ben & Jerry’s Food and Beverages

Nike Sports

Ikea Home and Furniture

Samsung Technology

Microsoft Technology

Amazon Technology/consumer goods

Table 2: Companies and their respective markets

After the selection of these companies, 20 posts were collected from each company, the criteria being that there should be at least 100 comments to each post and that it was part of digital content marketing. Next to the comments, the post including the posted image or video and the number of shares and likes were stored in a database. This resulted in a collection of about 20,000 Facebook comments belonging to 200 posts of 10 different companies.

3.2.2 Sorting the data: three types of content and inter-rater reliability

To be able to say something about the relation between digital content and the consumer responses, the characteristics of the content must be considered. Based on the literature study, content could be divided into three types: entertaining, informative and promoting content. To objectively sort the data into three types of content, an interrater reliability test was conducted. Firstly, for each of these three types an initial definition was constructed based on literature and definitions from the

(28)

Webster Dictionary (2017). These definitions were carefully explained and presented to a group of three individual raters.

After each rater was fully confident they carefully understood the definitions, he or she was asked to rate each piece of content to identify if according to him or her it belonged to the informative, entertaining or promoting type. The relevant question asked was: in what way does this piece of content market the company? After completion of the test, the interrater reliability was calculated, to measure the “agreement among raters”, in this case the agreement amongst raters whether a piece of content fits within the proposed type. To eliminate the agreement based on chance, Fleiss’ Kappa was calculated (McHugh, 2012). In this research, the notion of Fleiss (1971) that kappa with a value above 0.41 shows moderate agreement and kappa with a value above 0.61 shows substantial agreement was maintained. After calculating the interrater reliability for the initial definitions, it showed that there was a substantial agreement, however there was still some room for improvement with regards to the agreement amongst raters (κ = 0.63). This was because, according to the raters, some posts showed overlap (i.e. they could be equally identified as promoting as well as entertaining). To overcome this, the initial definitions were adjusted and posts that according to the raters didn’t seem to belong to any specific type were deleted. A more substantial rate of agreement (κ = 0.85) was then attained with the following final definitions and categorization:

Entertaining content: This type of content is engaging and the main goal is to provide

amusement or enjoyment to the consumer. Characteristics of entertaining content are that it’s amusing, fun, enjoyable, inspiring and/or engaging. Entertaining content be a ‘freebie’, a piece of entertainment or ‘the show’.

(29)

Examples: online episodes, music playlists, jokes/humorous images/memes,

inspirational quotes

Informative content: A piece of content can be identified as informative content when it’s

clear that the main goal is to provide useful information and to impart knowledge. This type of content has an educational or instructional nature and provides the consumer with new knowledge.

Examples: infographics, blog posts, interviews, instructional videos, recipes

Promoting content: Promoting content has as a main goal to present to achieve buyer

acceptance. This can be achieved through plain advertising publicity or offering discounting. This type of content, is closest related to traditional ads when compared to informative and entertaining content.

Examples: announcement of promo’s/sale, announcements of (new) products,

competitions

3.2.3 Cleaning the data

A major downside of using Facebook as a data source is that in the comments section there is a substantial buildup of unusable data. Examples of this could be spam, links, emoji uni-codes, empty inputs and tagging of friends’ names. To execute a reliable sentiment analysis and qualitative analysis on the comments, it was needed to clean up the data. Facebook has an algorithm that sorts all the comments in a way that the comments with full sentences are shown first versus the “empty” comments with tags of names and emojis that are shown lower in the comments. This was a useful fact fort his research, because this made selecting the first 100 comments reliable. Usually a post

(30)

had less than 100 useful comments, and the useful comments were selected for sure because, due to the algorithm, they were at the top of the comment section. The comments that fell out of the range were in general empty useless comments. Because of this, only a minimum of useful data got lost, rather than when the useful comments were spread out along the whole comment section. However, in the comments that contained full sentences, there was still some useless data left, for example comments that weren’t written in English or meaningless spam messages, that needed to be deleted. Eliminating all this unusable data resulted in a final database of a little over 12.000 useable comments.

3.3 Analysis

3.3.1 Sentiment analysis

The quantitative analysis consisted of a sentiment analysis of 12.000 comments, using Monkeylearn software. This software gives sentiment scores to words and then calculates whether a piece of text is negative, neutral or positive. Because each post consisted of a collection of responses and thus a collection of negative, neutral and positive labels, the average sentiment label had to be calculated.

Of each post, the mean percentage of negative and positive responses was calculated by dividing the number of negative/positive responses by the number of collected responses for each post. By doing this, it becomes clearer how negative or positive a posts response is rather than a nominal score of a ‘negative’, ‘neutral’ or ‘positive’ label. After this, for each type (entertaining, informative & promoting) the averages of the percental scores of negativity and positivity of the posts were calculated for the three different types of content. Along with these averages, the variance and standard deviations were calculated.

(31)

The sentiment analysis software by default divides the sentiment into three levels rather than two (negative, neutral, positive), which made it harder to spot effect. This was solved by using the SIM, or Social Influence Marketing score. This summary score is the net sentiment score that distinguishes between bad and good conversations and not only includes negative and positive but also neutral sentiment. To make a distinction between bad and good response, neutral sentiment can be added to the positive sentiment. The distinction is then made between negative vs. neutral to positive sentiment (Ang, 2011). The SIM score is calculated by simply adding positive with neutral comments minus negative ones, divided by the total number of comments (Ang, 2011). Lastly, the average number of shares and likes of the posts of each type of content was included purely to serve as background information in Table 2. Not every brand’s Facebook page has the same number of likers, so to adjust for this, for each post the percentage of likes and shares was calculated by dividing the post’s number of likes and shares by the total page likes of the accompanying brand’s Facebook page.

To test H1 and find if there was an effect at play, but also to check whether this observed effect was significant, a one-way ANOVA with three groups, each representing a content type, was executed on the SIM scores of the content posts. The data was checked to meet the assumptions of equality of variances and normality of distribution. While equality of variances was determined in a Levene’s test with a p >.05 (p = .37), the assumption of a normal distribution was not met, as indicated by the Kolmogorov-Smirnov goodness-of-fit test (p <.001). To overcome this, the negative skewed dataset could be successfully transformed with a log transformation after adjusting for the negative score outcomes by adding a constant to each data point (y = log (x + c), with constant c = 1). This resulted in a dataset that met the assumptions required for an ANOVA as it

(32)

was normally distributed (Kolmogorov-Smirnov, p = .085) and maintained equality of variances (Levene’s test, p = .595).

After the one-way ANOVA that signified an effect, a post-hoc test was conducted to find out between which groups the effect was apparent and significant. Because the group sizes were not equal, the Tukey-Kramer was not appropriate as this could lead to being too conservative in rejecting the H0, therefore the appropriate test selected was Scheffe’s method (Whitlock, 2009).

The results of the ANOVA served as a base for part two of the research. After determining the observed effect, interpretation of the observed effect was formed into two follow-up questions. Thereafter, these questions were further examined in part two of the research that consisted of a qualitative analysis.

3.3.2 Qualitative analysis: case studies

Part 2 of the research consists of a qualitative analysis of the comments and their context to identify where the observed effects for each type could possibly come from. For this qualitative analysis the guidelines from the book The SAGE Handbook of Qualitative Research (Denzin & Lincoln, 2011) were considered by coding parts of text, summarizing these codes into frequency tables and finally using the frequency tables as a base to form solid statements. Because the data consisted of many comments, that would require weeks of analyzing, for each type of content 250 comments were randomly selected. These, in total 750 comments were then labeled into detailed codes, that were at the end of the coding categorized into more overarching code labels that can be found in the frequency tables.

The observed effect in the first part of the research caused some questions about the cause of the effect that needed answering. Finding answers was done by analyzing the three cases of each content type that identify differences between the types and causes that are type specific. From this

(33)

analysis, it became clear that effects on consumer response were partly caused by differences between type of content, but there were also observations that were more general. These observations were clustered in three additional case studies.

(34)

4. Results

This section represents the results of the complete analysis of the Facebook reactions, sorted into the three different types: entertaining, informative and promoting content. Section 4.1 will contain the results and the descriptive statistics of the sentiment analysis. Section 4.2 will further explore these by considering the results of the qualitative analysis.

4.1 Results: Sentiment Analysis

4.1.1 Descriptives, ANOVA and Post-hoc

Descriptives

The first sorting of the content types based on the inter-rater reliability test, resulted in a sample of 76 pieces of entertaining content, 29 pieces of informative content and 54 promoting pieces of content. As can be seen in Table 2, on average both entertaining and promoting type of content resulted in being labeled positive when it comes to the sentiment of the consumer responses. In contrast, informative type of content scores lower and is labeled as neutral in consumer response sentiment. With the lowest average percentage of negative comments per post (24,98 %) and the highest percentage of positive comments per post (43,33%) the promoting type ranks the highest in positive effect on consumer response. This is summarized by the SIM score of 50,04, which is higher than that of the type second in line. This second in line is the entertaining type of content, with an average negative response percentage of 30,20 % and a positive response percentage of 40,60% and a SIM score of 39,60. Lastly, the type with the lowest consumer response sentiment is informative content with a negative response of 36,4% and a positive response of 35,8%. These scores cancel each other out, therefor labeling this type of content as neutral. This is summarized in the lowest SIM score of 27,2.

(35)

Entertaining (n= 76) Informative (n=29) Promoting (n=54)

Average sentiment Positive Neutral Positive

Average percentage negative comments 30.2 36.4 24.98 Variance Negative comments 0.02327 0.02790 0.019359 Average percentage positive comments 40.6 35.8 43.33 Variance positive comments 0.018754 0.034498 0.0192476

Average SIM score (mean)

39.54 27.27 50.04

Standard Deviation 30.31 32.81 27.83

Average likes % 0.070175416 0.024278495 0.0629223

Average shares % 0.012647678 0.008209838 0.00819267

Table 3: Descriptives of the overall sentiment analysis divided into the three content types.

When comparing the variances, it becomes clear that the spread of entertaining type (negative; 2,327; positive; 1,875) and promoting type (negative; 1,935; positive; 1,924) are relatively close to each other whereas informative type of content has a variance almost twice as high for positive response (3,445) and a little less, but still higher for negative response (2,790).

ANOVA and Post-hoc

While the above mentioned descriptives show a trend, they do not provide any information on whether this trend is significant or not. However, from the results of the one-way ANOVA it can be reported that there is a significant effect of content type on SIM score at the p<.05 level for the three groups [F (2, 158) = 5.64, p = 0.004]. Based on both the p- and F-value, H0 of the ANOVA

(36)

can be rejected, concluding that there is at least one group’s mean significantly different from the others.

To understand between which groups this effect holds, a post-hoc test was needed. As mentioned in the method, for this, the Scheffe test was selected. The Post hoc comparisons using the Scheffe test indicated that the mean SIM score for the promoting type (M = 50,04) is significantly different (p = .005) from the informative type (M = 27,27). However, the mean SIM score of the entertaining type (M = 39,54) did not significantly differ from the promoting nor the informative type (fig. 4).

Figure 4: Visualization of the observed effect, between types informative and promoting content

there is a significant difference in SIM score.

This means that in general promoting type of content will receive a more positive sentiment than informative type of content. Furthermore, when choosing between posting entertaining or promoting content it should not matter which one is picked.

(37)

While there is a significant effect between groups, it should be noted that the effect size is limited. From the eta squared value (R2 = .067) of the ANOVA it can be indicated that the type variable explains little of the variability in the SIM scores between groups.

4.1.2 Results: preliminary propositions and follow-up research questions

To be able to continue with part two of the research, it is first necessary to preliminary conclude on the results from the first part, as these serve as a foundation for the second part.

Firstly, based on the results of the sentiment scores per content type the following observed effect can be summarized as:

Observed effect: In general, promoting type of content will receive a significantly more

positive sentiment than informative type of content. For entertaining content, there are no significant differences in the response sentiment scores compared to the informative and promoting types.

This observation says something about the response sentiment score, however there can be different causes for a negative or positive score for each type of content. Therefore, instead of just observing whether the response is negative or positive, the second part of the research will go into the type of responses that each type of content obtained. To do so, the following sub questions can be formed:

Sub question 1a: What sort of comments are the general responses to each content

(38)

Sub question 1b: How can the low(er) consumer response sentiment of informative

content be qualitatively explained compared to the entertaining and promoting type?

From the ANOVA, it became clear that there was a significant effect caused by the content type variable, however the effect size was small. An R2 value close to zero indicates that most of the variability is within the groups, rather than between the groups. This suggests that there are more variables involved that go beyond the content type variable (Whitlock, 2009). Based on this, the qualitative part of the research will explore which these could be, using the following question as guidance:

Sub question 2: Which influences can be suggested future additions to the

conceptual framework to increase explanatory power of the model?

The next section will show the results of the qualitative analysis of the consumer responses, taking in account the above-mentioned sub questions.

4.2 Results: qualitative analysis of case studies

In this section, the results of the sentiment analysis will be complemented by a qualitative analysis that goes deeper into the responses. Section 4.2.1 sums up the main drivers for positive and negative responses for each type of content based on their frequency tables. Next to that, it also compares the frequencies of the drivers of each type of content with each other. Based on this comparison, two types of comments were apparent. Firstly, there are types that are specific for the type-of-content variable that will be discussed in section 4.2.1. Secondly, there are causes that go beyond

(39)

this variable and hold for each of the three types of content, which will be elaborated in section 4.2.2.

4.2.1 Responses to entertaining, promoting and informative content: a qualitative comparison

Entertaining content

From analyzing the responses to entertaining content posts, it becomes apparent that the positive sentiment scores were originating from people expressing something positive about the content post itself, actively engaging with the content, showing to have a positive brand image and responding by tagging and joking with friends. The negative sentiment scores were mainly caused by controversy around the content, complaints about the brand, controversy around the product and the brand. These results can be found in table 4 that summarizes the complete frequencies of the code labels of the responses to entertaining content.

Label Frequency Positive about content post 58

Actively engaging with content 32 Positive brand image 25 Joking with friends 24 Controversy content 20 Product/brand complaint 13 Active boycott 13 Controversy product 13 Annoyance content 10 Controversy brand 9 Positive product 8 Service issues 7 Question for the brand 4

Table 4: Frequencies of code labels for the responses to entertaining content posts (n=250).

Promoting content

For promoting content, the positive sentiment scores were built up of comments mainly consisting of the customer’s statement that he or she wanted to have or buy the product being promoted and usually thereby tagging their friends. Next to this “want to have” wish-listing, positivity about the

(40)

product, brand image and the content post itself were main drivers for positive sentiment scores. Even though promoting content has the highest positive response sentiment score, there were still some negative responses to be found. In general, these negative sentiment scores came from a product or brand complaint, controversy around the product, content or brand and service issues (table 5).

Label Frequency

Want to have product 95 Positive about product 35 Positive brand image 20 Positive about content post 19 Product/brand complaint 18 Controversy product 16 Controversy content 13 Question for the brand 11 Partaking in competition 7

Service issues 6 Controversy brand 3 Active boycott 3

Table 5: Frequencies of code labels for the responses to promoting content posts (n=250).

Informative content

Lastly, for informative content, that scored the lowest in consumer response sentiment, the main negative scores were caused by discussion-like statements in the comments. Next to that controversy around the brand, product and brand complaints and services issues were main drivers for the negative sentiment scores. Positive sentiment scores were mainly build up by responses consisting of positivity about the content post itself, a positive brand image or positivity about the product (table 6).

(41)

Label Frequency Discussion 50

Positive about content post 43 Controversy brand 35 Positive brand image 25 Product/brand complain 24 Question for the brand 22 Service issues 13 Positive about product 12 Controversy product 12 Want to have product 10 Active boycott 9 Engagement with content 5 Controversy content 3

Table 6: Frequencies of code labels for the responses to informative content posts (n = 250).

Comparing these results, leads to distinguishing between two different types of causes. Firstly, causes that are specific to the type of content variable (table 7a) and secondly, those that go beyond this variable and are valid for each of the three types of content posts (table 7b). These are summarized in table 8.

Variable specific response

sentiment score

Entertaining Joking with friends ₊ actively engaging

with content

+

Promoting Wish listing, ₊

Informative Discussion, _-

Variable non-specific response

sentiment score

Controversy about product,

brand or content -

Service issues _-

Questions _+/-

Brand image _+/-

Complaints & boycotts _- Positivity about product,

brand or content +

Table 7a: The responses that are specific for the

different content types

Table 7b: The responses that hold for

(42)

Table 8: summary of the main causes of negative and positive sentiment scores for entertaining,

promoting and informative content. The type specific causes are shown underlined.

From table 7a it becomes clear that each type of content is linked to a unique type of response. Those will be further explained in section 4.2.1.1 until 4.2.1.3. Thereafter, the more general type of responses as shown in table 7b, will be further explored and examples will be given trough outtakes of the comments.

4.2.2 Further analysis: explanation and examples of the codes

4.2.2.1 Entertaining content

The main driver behind positive sentiment scores for entertaining content, that was specific for this type of content, is that users respond in a way in which they actively engage with the content in a positive manner, that is, they interact with the content for example by answering questions posed in the content. This kind of response is unique for the entertaining content type because of the opportunities entertaining content offers marketers to make use of engaging triggers. These triggers that stimulate people to engage and respond are embedded within entertaining content per definition. For example, marketers can pose questions, like small quizzes, users favorite activities,

Negative sentiment scores can be explained by:

Positive sentiment scores can be explained by:

Entertaining content Controversy about product, brand or content, service issues, questions, complaints & boycotts

Joking with friends, actively engaging with content, Positivity about brand, product or content, positive brand image

Promoting content Controversy about product, brand or content, service issues, questions, complaints & boycotts

Wish-listing Positivity about brand, product or content, positive brand image

Informative content Discussion, Controversy about product, brand or content, service issues, questions, complaints & boycotts

Positivity about brand, product or content, positive brand image

(43)

foods, their plans for the day and so on. Next to that, other triggers could be jokes, puzzles or inspiring quotes, as long as it aims to fulfill the enjoyment of the users.

One example of a trigger like this comes from Amazon, that posted an image of a jar of jellybeans and challenged the audience to guess the number of jelly beans, with no other goal than to entertain (e.g. there was no price or reward involved). Another example is an April’s fool joke posted by Ikea, that announced new but ‘too good to be true’ products and services like in-store massages and bars. Examples of these ‘interacting’ responses can be found in box 1 and box 2.

“Hmmm...I'm gonna take a guess of 4,467 beans. I've never been good at these guessing games.lol.”

“I have better idea, can you guess how many I want? If you said, "a handful" you are correct.” “Please put them all in my stomach, and then there will be ZERO in the jar!”

“I spent about 30 seconds thinking and came up with 2,000. Am I close?”

Box 1: Examples of interacting responses from users that tried to solve a riddle by Amazon, purely

for fun

“Hahaha! I fell for the massages! Another great reason I love IKEA....funny joke.”

“Please make this real! If there was a bar and massage room at IKEA, I would NEVER LEAVE!!!” “This was great customer focus group work. Now you know that we all want a bar in our IKEA.” “That's the April Fools spirit! Great job. I was so excited.”

“They say to think outside the box...you sure did, IKEA!! But all in all, these are very reasonable and doable ideas!!! I think you have evolved to that next level...come on IKEA, make it so!! These jokes were AWESOME!!!”

Box 2: Examples of users interacting with IKEA after reading their entertaining content post

containing an April fools’ joke

Lastly, there was another driver to positive sentiment which only appeared for a particular form of entertaining content. Namely, the ‘meme-type’ of entertaining content. This type of content post,

The effect of three different types of digital content marketing posts on consumer response : a sentiment and qualitative analysis of Facebook responses