Master Thesis
Thesis title:
The Influence of Affective Content in Expert and Consumer Reviews on the Box Office Performance of Hollywood Motion Pictures.
Jurrit Veltman University of Amsterdam, Faculty Economics and Business
10492194
Supervisor: Dr. F. B. Situmeang Second supervisor: Dr. U. Konus
Date: 26-6-2014 Msc. in Business Studies –
Abstract
Purpose – The purpose of this thesis is to contribute to the marketing literature and practice by examining the relationship between affective content in expert and consumer reviews of Hollywood motion pictures and the performance of those cultural products, measured by the worldwide box office revenues. Also the influence of consensus among reviews on this performance is measured.
Design / methodology / approach – A set of hypotheses are developed, based on earlier research done in the field, i.a Eliashberg & Shugan (1997), Gemser, van Oostrum & Leenders (2006) and Ludwig et al. (2013). The dataset is obtained from websites as Metacritic and The-Numbers, and with use of the LIWC program developed by Pennebaker et al. (2012), the affective content is subtracted from the reviews. The dataset contains 2572 movies ( N=2572), with a total of 72.045 written expert reviews and 145.426 written consumer reviews. The hypotheses are tested by performing correlation and regression analyses.
Findings – Results suggest that affective content in consumer reviews is positively related to the performance of motion pictures, and that affective content in expert reviews is not
significantly related to this performance. With regard to consensus among written reviews, it can be stated that a high level of consensus in both consumer and expert reviews have a positive influence on the performance of movies. Other variables (movie characteristics) don’t work as a moderator on the relationship between expert and consumer reviews and the
performance of movies.
Research implications – This study is one of the first attempts to research the relation between reviews and performances based on qualitative data. This study differences from previous research in the area by using textual data mining tools, where previous research studies numerical data (scores as a signal of the product quality).
Practical implications – This study helps marketing managers and producers of motion pictures to better manage expert and consumer reviews as an influencer on the performance of motion pictures. It is important for managers to know how their product performance is affected by positive and negative affective content in expert and consumer evaluations.
Originality / value – The results of this study contribute to the continuous discussion about the impact of expert and consumer reviews on the box office revenues of entertainment products. Besides that, it highlights the importance of consensus among expert and consumer reviews and the influence on product performances.
Keywords: Motion pictures, Worldwide Box Office Revenues, Expert reviews, Consumer
Table of content
Abstract ... 2
Introduction ... 5
Expert and consumer reviews ... 5
Cultural Industries ... 6
Aspects of the film industry ... 7
Research gap ... 9
Outline thesis ... 10
Literature review ... 11
Current knowledge in relationship between reviews and performance ... 11
Affect ... 12
Consensus between experts and consumers ... 14
Calculation of the affect score ... 16
Hypotheses ... 17
Affect ... 18
Consensus ... 20
Genre and age restriction ... 21
Methods ... 23 Research outline ... 23 Data collection ... 23 Sample ... 24 Measurement development ... 25 Statistics... 27 Quality criteria ... 28
Analysis and results ... 29
Correlation matrix and descriptive statistics ... 29
Test of the hypotheses ... 32
Discussion ... 34 Implications ... 35 Limitations... 36 Future research ... 37 Conclusion ... 38 Bibliography ... 41 Appendices ... 51
Introduction
Expert and consumer reviews
‘’Let’s be clear from the outset: The Wolf of Wall Street is not a “scathing indictment of
capitalism run amok” or a “cautionary fable for our time” or any of the comparable high-minded plaudits that are likely to be thrown its way. Yes, Martin Scorsese’s new feature is undeniably topical: the story of a rogue Wall Street trader, Jordan Belfort, who made himself and his partners fabulously wealthy at the expense of the broader American public and got off—even after multiple fraud convictions—nearly scot-free. But the film displays almost no interest whatsoever in Belfort’s victims, and it is extravagantly incurious regarding the mechanisms by which he took their money. If this is a message movie, it’s one that features a message suitable for a cue card.
None of which, incidentally, is intended as an indictment. The Wolf of Wall Street is a magnificent black comedy, fast, funny, and remarkably filthy. Like a Bad Santa, Scorsese has offered up for the holidays a truly wicked display of cinematic showmanship—one that also happens to be among his best pictures of the last 20 years.’’
This review is written by Christopher Orr, a professional reviewer for the Atlantic, about the latest collaboration between Martin Scorsese and Leonardo di Caprio. These first two paragraphs are just the start of a 10-paragraph long homage to the product called the Wolf of Wall Street. To show this reviewer his enthusiasm about the picture, read his closing
paragraph.
‘’The Wolf of Wall Street is not a subtle movie, or a thoughtful movie, or a
particularly innovative movie. But for those susceptible to its vulgar charms, Scorsese’s latest is a great—no, a fucking great—movie.’’
Compare this review with another one: ‘’Enjoyable time… I read some of the reviews
below about exploiting our country..blah blah blah. Good funny movie. Lots of drugs, sex, hookers, etc. Just a heads up.. Been a while since I've seen that much nudity too! But who cares!…’’. This review is done by a consumer who watched the movie, written under the
nickname Msepos. You can see a lot of differences between reviews written by professional critics and normal consumers, to start with the length of the reviews. You notice that Christopher has experience in writing reviews. He compares the movie with other movies, knows which techniques are used, he uses more ‘’difficult sentences’’. The consumer review is just a small story about what you can expect from the movie, nothing more, nothing less. He or she gives much more his own opinion about the product.
But what does the affective content in reviews mean? Do they have an influence on consumer choices, or in a broader perspective, do they influence the performance of movies, measured by the box office revenues? And what can be the influence of consensus in positive and negative content in these reviews?
Cultural Industries
As the world shifts to a service economy (Shugan, 1994), we find enormous growth in the entertainment industries (Eliashberg & Shugan, 1997), including movies, theatre, music and so on. Because of this growth, the cultural market has become bigger and more dynamic. Cultural industries are defined as those industries ‘’which produce tangible or intangible artistic and creative outputs, and which have a potential for wealth creation and income generation through the exploitation of cultural assets and production of knowledge based goods and services, which both can be traditional and contemporary’’ (UNESCO). New technologies are changes this market dramatically, and that is why it presents several new research challenges.
One segment of the cultural industries is the motion picture industry. Movies can be seen as experience goods. ‘’Experience goods are products or services where the quality and utility for a consumer can only be determined upon consumption’’ (Nelson, 1970). This assumption is backed up by Situmeang, Leenders and Wijnberg (2013), where they state that ‘’ the quality of a product is always the quality as perceived by observers, and sales
performances as well as product reviews provide as indicators of how this quality is
perceived’’. Only after watching a movie, consumers can evaluate the product and generate thoughts and feelings about the product. So because the quality of experience goods is very difficult to ascertain before the consumption-stage, there are different sources where consumers can gain information about the qualities of experience goods. According to Reinstein and Snyder (2005), such sources can be advertising (branding) of products and information from peers. According to Duan et al. (2008), ‘’one of the most influential resources of information transmission (especially for experience goods) is word-of-mouth’’. WOM is ‘’the informal communication between two private parties concerning the
evaluations of goods and services (Dichter, 1996; Fornell & Bookstein, 1982; Singh, 1988; Westbrook, 1987) and could be positive, negative, and even neutral (Anderson, 1998). The definition for WOM according to Davis and Khazanchi (2008) is ‘’when people have the ability to communicate and share experiences with each other online’’. Examples of these WOM resources are expert reviews (written by professionals), user reviews (written by consumers). Chen et al. (2004) even state that the amount of WOM is a key driver related to the performance of experience goods.
Aspects of the film industry
There are several interesting factors in the film industry, like the marketing aspect of the advertising expenditure, but also the role of well-known movie stars and the influence of film critics and their reviews. All these factors are in most of the studies linked with the box office
revenue of movies (Eliashberg & Shugan, 1997; Shrum, 1991; Gemser, van Oostrum & Leenders, 2006; Basuroy, Chatterjee & Ravid, 2003), which can be seen as the commercial success of the product and is a very important topic in the movie industry. According to De Vany and Walls (1999), the movie business is very risky. This statement is backed up by Menger (1999). He states that artistic markets are puzzling ones. According to Menger, the attractiveness of artistic occupations is high but has to be balanced against the risk of failure of an unsuccessful professionalization that turns ideally non-routine jobs into ordinary undertakings. Earnings distributions are extremely skewed.
De Vany and Walls also state that ‘’the movie business is a profoundly uncertain business. The probability distributions of movie box-office revenues and profits are characterized by heavily tails and infinite variance’’. The movie business is also risky for consumers, because they pay for the product before they know that they are going to receive in return (Hennig-Thurau, Walsh and Wruck, 2001).
Elberse and Eliashberg (2003) have studied the role of advertising expenditures in product performances. They state that variables such as movie attributes and advertising expenditure are usually assumed to influence audiences directly, are mostly an indirect influence, namely through their impact on exhibitors screen allocations. De Vany and Walls (1999) conclude the same about star power. They studied the link between the addition of a star in a movie picture can reduce the outcome uncertainties (read; predict box office
performance) of entertainment products and concluded that the audience make a movie a hit an no amount of ‘’star power’’ or marketing hype can alter that. In other words, the real star is the movie itself.
Actual research in the field of predicting the performance of motion pictures is done by Sochay (1994). He state that ‘’each film has a dual nature, in that it is both an artistic
statement and a commercial product’’. Sharda and Delen (2006) have also done research in this field, and in their study the use of neural networks in predicting the financial performance of a movie at the box office before is theatrical release is explored. Results show that the neural network employed in this study can predict the success category of a motion picture before is theatrical release with pinpoint accuracy of 36,9%. The authors indicate that their model should be improved in the future. It becomes clear that there is done a lot of research to (prediction of) box office performances of motion pictures. For instance, Litman and Ahn (1998) found that 25% of total revenue of a motion picture comes from the first two week of receipts. In the field of box office revenues, there is also an interesting role for film critics and their reviews.
Research gap
Many research is already done about the impact of film reviews on the box office performance of art house and mainstream motion pictures (Eliashberg & Shugan, 1997; Shrum, 1991; Gemser, van Oostrum & Leenders, 2006, Basuroy, Chatterjee & Ravid, 2003). Most of the research is done by comparing the scores (quantitative data) of reviews with the performance. But less research is done about the texts of the reviews itself. Research by Korfiatis et al. (2010) suggests that the qualitative content of reviews have a serious impact. It was found that the word length and readability scores were indicators for a review ‘’to be considered very helpful by consumers’’. It is very interesting to study the expressed emotions in those reviews, and its influence on performance. Expressed emotions in text
communications can be positive or negative, and is called affect. Research of Isen (2001) states that it is widely accepted that affect plays a role in cognitive processes as decision making an evaluating. Affect is related with thoughts and feelings. So the research question for this thesis is going to be:
‘’What is the impact of the positive and negative affective content in expert- and consumer reviews on the box office performance of Hollywood motion pictures? ‘’
Fig 1: Model of the research question of the thesis
The first aim of this thesis is to determine whether affective content in expert reviews and consumer reviews have an impact on the box office performances of Hollywood motion pictures. This is done by analyzing textual content using the LIWC program, and comparing it with cumulative box office revenues. As mentioned before, the study is about experience goods and will therefore focus on the creative industries, in particular the movie business industry. The second aim of this paper is to explore the effects of (the lack of) consensus in the reviews written by experts and consumers as a determinant in the relationship between written reviews and movie performances
Outline thesis
In the introduction the cultural industry has been described, as also the reason why the setting of this study will be about the motion picture industry. The research question of this thesis is given, as also the associated conceptual figure. The following sections consists of (i.) the literature review on which this thesis has been written, (ii.) the hypotheses and why these hypotheses has been defined, (iii.) the research methods, (iv.) results and findings of the research, (v.) the conclusion and discussion of the thesis and (vi.) limitations and future research.
Literature review
First of all, we start with the relation between critical reviews and the performance of movies. Situmeang et al. (2013) state that expert and consumer reviews transmit information about products. ‘’When reliable information is scarce, especially if the quality of the product is difficult to evaluate before consumption, both types of reviews can have a strong impact on consumer behavior’’ (Basuroy et al., (2003) and Caves (2000). According to Eliashberg and Shugan (1997), critics and their reviews pervade many industries and are particularly
important in the entertainment industry. Few marketing scholars, however, have considered the relationship between the market performance of entertainment services and the role of critics, with regard to analyzing the textual content of the reviews. With the market performance of entertainment services in the movie industry can be read the box office performances of motion pictures. The term box office is frequently used, especially in the context of the film industry, as a synonym for the amount of money a particular production (movie) receives during its theatre release. So it is the commercial success of movie pictures (Zufryden, 2000).
Current knowledge in relationship between reviews and performance
Many researchers already have done studies looking at the relationship between online reviews and the performance of movie products (Litman, 1983; Reddy, Swaminathan and Motley, 1998; Elberse and Eliashberg, 2003; Reinstein and Snyder, 2005; Boatwright, Basuroy and Kamakura, 2007). Eliashberg and Shugan (1997) state that critical reviews correlate with late and cumulative box office receipts, but do not have a significant correlation with early box office receipts. Other findings are that positive reviews have no influence in early weeks, but they do in later weeks and overall performance. Negative reviews have the same effect on the box office performance of movies. Another conclusion the authors make is that the total written about a movie is not predictable for success or failure, but a predictor for
the first week performance. So their conclusion is that film critics and the reviews can be seen as predictors of the market performance of entertainment services, and not as influencers. This findings were based by using the numerical data of critical reviews.
Gemser, van Oostrum and Leenders (2006) have done some more recent research in this topic, and made a distinction between art house film and mainstream motion pictures. Using the Dutch film industry as their empirical setting, the authors have studied the effects of reviews on the opening weekend and on the cumulative box office revenues. Their research show that the number and size of film reviews in Dutch papers directly influence the behavior of art-movie-going public in their film choice. In this case, the reviews work as a influencer. They called this the ‘’influence effect’’ of product reviews. The number and size of film reviews of mainstream movies, on the other hand, only predict the movie performance. So in this case, reviews work as a predictor. This phenomenon is called the ‘’prediction effect’’ of reviews. Both studies (Eliashberg & Shugan versus Gemser et al.) have in common, is that they use the scores of reviews in relationship with the performance of movies. What is not done yet, is test this relationship by focusing on the actual words in the reviews. These words can contain positive of negative emotion, which can influence the consumers. A synonym for this emotion is affect.
Affect
Former research on affect has shown that it has a significant influence on consumer’ choices and their judgments (Loewenstein and Lerner, 2003), on decision making by consumers (Shiv and Fedorikhin, 1999) and evaluation of products (Lench et al., 2011). This theory is backed up by Miller (2005). He stated that evaluations done by experts can be the input for
consumers to come up with attitudes about a product themselves. Feldman and Lynch (1988) mention that ‘’the relative weight of heuristic inferences depends on two context-dependent facets: their relative accessibility and their diagnosticity compared with alternative inputs’’.
In the case of online reviews, sheer volumes of these reviews lead consumers to heuristically process information in many cases. According to Jones et al. (2004), this has a decisive influence on purchase decisions of consumers. As mentioned before, existing research has focused on the diagnosticity of accessible, quantifiable customer review information cues, such as volume (Duan et al, 2008), reviewer identity information, by what you have to think about names and locations of the reviewer (Forman et al, 2008) and quality ratings (Chevalier and Mayzlin, 2006). But research is also done on product-related aspects, for instance price (Yong, 2006) and product popularity (Zhu and Zhang, 2010). But according to Yong (2006), ‘’empirical investigations into the influence of numerical cues on sales often provide mixed or inconclusive results, which suggest some doubts about their diagnosticity and predictive ability. So according to Yong, these information cues cannot be used as explorations of the box office performances (sales) of motion pictures. And besides that, Delarocas et al. (2007) indicated that numerical ratings are positively related to box office revenues, with no relations to the volume of these reviews, whereas Duan, Gu and Whinston (2008) and Yong (2006) find that review volume, with no relations to ratings, drives sales. So results from former research give tremendous opposite results. Potential explanations for these mixed findings might be methodological shortcomings or the inability of numerical data to do justice to the very fine, nuanced nature of verbatim reviews (Cao et al. 2011, Pavlou and Dimoka, 2006).
According to Tirunillai and Tellis (2012), ‘’making use of recent advances in text analytics to systematically analyze large volumes of collections of customer review verbatim scripts and taking a dynamic perspective, which is more reflective of the rapid, continual changes in user-generated content’’. Research by Huffaker et al. (2011) on textual
communications suggests that content and style elements in reviews are relevant inputs that help determine relative diagnosticity and accessibility. At a word level, ‘’content words are generally nouns, regular verbs, and many adjectives and adverbs. They convey the content of
a communication’’ (Tausczik and Pennebaker, 2010). So no content can be communicated without style words. If we go a bit further, affective content words reveal the intent of a text message (Bird et al., 2002). So affective content conveys emotions such as sadness, anger, happiness. According to Cohen et al. (2008), ‘’affect in and of itself is not a linguistic
property by refers to an internal feeling state’’, ‘’that is consciously accessible as the simplest raw feelings evident in moods and emotions’’ (Russel, 2003). And according to Ortony et. (1987), the use of word cues is probably the most effective way to make affect accessible. And at an individual level, ‘’affective content words should particularly likely to influence consumers whose motivation to engage in detailed cognitive processing is low and those with limited access to processing resources’’ ( Ludwig et al., 2013; Andrade, 2005). They believe that because they can be distracted or under time pressure, so instead of using a central route of processing information, those consumers use the peripheral route of processing
information.
The last piece of conceptual foundations with regard to affect is about accessibility of reviews. Zajonc’s (1980) research indicated that affective cues are more accessible than factual or descriptive information. So in text-documents, reviews with affective content will be more easily noticed than reviews with a description of the product itself. Pham et al. (2001) also demonstrate that affective cues are noticed sooner than cognitive assessments. More information about how affective content will be used in this thesis will be explained in the
hypotheses part (page 17). Research from Ho-Dac et al. (2013) suggests on the other hand
that higher product sales will lead to increasingly positive reviews.
Consensus between experts and consumers
Another gap in the research about the relation between film critics and the performance of movies is which type of film critics are involved. Reviews can be written down by experts, but also by consumers. So it is interesting to research the (lack of) consensus within consumer
reviews and expert reviews. According to von der Gracht (2012), consensus can be seen as a certain level of agreement. Most studies focus nearly on the exclusively on ‘’quantitative surrogates’’ of review contents (Mudambi and Schuff, 2010). But according to Cohen et al. (2008), affective cues provided in texts (for instance, ‘’I love this book) might influence respondents’ attitudes. According to Boor (1990) and West and Broniarczyk (1998), ‘’in most social groups reviews of the same object will vary among group members due to
individual differences, in taste or perspective’’. When you can find a lack of consensus among written reviews, it means that there is a high variability between the reviews written about the same object. Until recently, there isn’t done much research about the variability of opinions on the effects of reviews, but exceptions are Sun (2012) and Das and Chen (2007), whom did research in the financial sector. They found a strong relation between the variability of opinions of stockholders and negative movements in stock prices. Sun (2012) did research in book reviews and this study suggest ‘’that interaction of the average consumers reviews of books and the standard deviation of these reviews has a significant effect on the demand’’, where products which received low ratings benefited from high variability and products with high ratings suffered from variability. With regard to this thesis, if consensus is low over expert and consumer reviews, on an aggregate level, it was probably very difficult for the expert and consumer reviewers as a whole to determine the overall quality of the movie products. Additionally, Neuwirth, Frederick and Mayo (2007) and state that ‘’ a lack of consensus decreased the chance that consumer will expect to see their opinions reinforced by other consumers, which may affect the likelihood that they will express their opinions, or at least, express them strongly’’.
So it is very interesting to investigate the level of consensus within expert and consumer reviews. You can assume that when most of the expert reviewers are relatively
positive about a new movie product, you will have a good chance that performance of the motion picture is also positive. The same suggestion accounts for the consumer reviews.
But the main focus in this thesis will be on the relationship between movie reviews, both consumer and expert, and the box office performance of motion pictures. As mentioned before, the reason why reviews are written is to help consumers decide if they want to purchase the product, so they reduce uncertainty by providing information about the product (Kirmani and Rao, 2000). The focus will be on the emotions which are written in critical reviews, and the research area will be Hollywood motion pictures, whereby most of them are blockbusters. According to Collins, Hand and Snell (2002), a blockbuster is a term in the motion picture business, which is used to indicate movies which have big star appearances and whereby is expected that is will receive big box office revenues.
Calculation of the affect score
Customer reviews have become one of the most frequently accessed online information sources, as consumers appear to be weary of traditional, marketer-dominated information channels (Godes and Mayzlin, 2004). So textual online reviews are more often used as information sources than traditional information sources. Ludwig et al. (2013) jumped in to this statement, and conclude also that customers increasingly rely on other consumer’s reviews to make purchase decisions. They have come with new insights by studying the content of reviews and examine their influence. To do this, the authors studied to content of the texts to extract changes in affective content and linguistic style properties. For this thesis, the first part is interesting and important. With a dynamic panel model, see figure 2., they were able to study this influence. A part of this model is going to be used to study the data in this thesis.
Fig 2: Dynamic panel model created by Ludwig et al. (2013), for the analysis of qualitative data
This model is set up by the authors Ludwig, de Ruyter, Friedman, Bruggen, Wetzels and Pfann (2013). How the aggregation works and what part of the model is used in this thesis, is explained in another part of this thesis, methods, where the empirical study is explained (page 22).
Hypotheses
The movie industry is a typical example of a business where consumers make use of expert and consumer evaluations of industry products. Expert reviews are written on a weekly basis in newspapers and magazines, discussing the latest outcomes of Hollywood motion pictures, evaluating the movie products with stars (most of the time they give one star (very bad product) to 5 stars (excellent product) and writing down their judgments and feelings about the movie. Furthermore, for consumers there are a lot of websites (Example:
rottentomatoes.com / metacritic.com) where they can find evaluations of movie products and also can write their own reviews of movies. More and more consumers are using the internet to provide their opinion. O’Reilly (2005) even states that one of the core characteristics of the internet is interaction between users. So a lot of information is exchanged between consumers on the internet. People are now able to create their own content on the internet, and created content by users of the internet is a relatively new characteristic of the internet (Shao, 2009).
Affect
As stated before, according to Duan et al. (2008), ‘’one of the most influential resources of information transmission (especially for experience goods) is word-of-mouth’’. WOM is ‘’the informal communication between two private parties concerning the evaluations of goods and services (Dichter, 1996; Fornell & Bookstein, 1982; Singh, 1988; Westbrook, 1987) and could be positive, negative and even neutral (Anderson, 1998). So it is clear to state that people are influenced by opinions and expressed emotions of other people. When the user generated content characteristic was added to the internet, more people were writing
evaluations of products on the internet, so the reach of WOM is tremendously increased in the last decade. Consumers can now communicate their experiences with and evaluations of products (Chen, Fey and Wang, 2011). These evaluations can be positive or negative.
Affect are mental responses to the consumer environment, in this case the critical reviews, and can be described as emotions, moods and attitudes (Peter & Olson, 2001). So affect is the feeling consumers have when they are exposed to the environment. Affective content words reveal the intent of a text (Bird, Franklin and Howard, 2002). According to Lench et al. (2011), affect drives evaluation and decision making.
This evidence can also be concluded from previous studies, like the Lau-Gesk and Meyers-Levy study in 2009, which suggests that ‘’exposure to affective cues influences evaluations and/or judgments of attitude objects’’, where they used experiments with brands and comprehensive products. Their result was that positive or negative affective cues lead to more positive, respectively negative evaluations and or judgments. So stated in other words, by just reading a text with an affective content, readers’ thoughts can already be influenced.
Research and theory by Baumeister et al. (2007) suggests that text with affective cues immediately elicit affective responses and those responses don’t require much processing resources. They also state that ‘’the transfer of affect from such diagnostic cues and the
corresponding automatic responses may be best understood according to a positive-negative continuum’’ (Russel, 2003). So critical reviews must be text analyzed to capture the positive and negative affective content of those reviews, and the changes in those content (from positive to negative emotions) must be compared to the performance of the associated products.
According to Petty et al. (2003), ‘’new product reviews with an extreme imbalance of positive and negative affective content is likely to initiate consumer wariness and corrections to the influence of these affective cues’’. This line of reasoning suggests that more positive content in movie reviews should result in a higher performance of those movies, while negative content should result in a lower performance of movies. This line of reasoning is reasonably justifiable, when we take in mind the next scenario. Warner Bros. releases a new motion picture which is picked up by professional reviewers and normal movie fans. When both the pooled reviews of those groups are on the positive side of the continuum, consumers will be triggered to go to the theatre and watch the movie. This results in a high box offices, e.g. high performances, of the movie. According to this theory, I formulate the following hypotheses:
H1a: There is a positive relationship between positive reviews written by professionals and the box office performance of Hollywood motion pictures.
H1b: There is a positive relationship between positive reviews written by consumers and the box office performance of Hollywood motion pictures.
H2a: There is a negative relationship between negative reviews written by professionals and the box office performance of Hollywood motion pictures.
H2b: There is a negative relationship between negative reviews written by consumers and the box office performance of Hollywood motion pictures.
Figure 3.1: Emotion can both have an positive or negative influence on the performance of movie products
Consensus
As already stated, reviews can be written down by experts, but also by consumers. So it is interesting to research the (lack of) consensus among consumer reviews and expert reviews. The expectation is that there will be a high level of conformity within professional and consumer reviews. This because given the fact that expert critics are professionals, they probably, as a sign of professionalism, take their peers’ opinion into account when they write their review (Das and Chen, 2007). This is the opposite view of the findings of Wijnberg and Gemser (2000), who stated that in high art industries reviewers want to be innovators. In other studies, results where reviewers strive for consensus have been observed (Hsu, (2006) and Shoemaker and Vos (2009). The expectation is that a high level of consensus amongst expert reviews will have a positive influence on performance of motion pictures. So the hypotheses will be:
H3a: The relationship of expert reviews with the performance of Hollywood motion pictures is
moderated by the degree of consensus among expert reviews of those motion products. The relationship is less positive if there is a lower degree of consensus.
H3b: The relationship of consumer reviews with the performance of Hollywood motion pictures is moderated by the degree of consensus among consumer reviews of those motion products. The relationship is less positive if there is a lower degree of consensus.
Figure 3.2: A high level of consensus within expert and consumer reviews
Genre and age restriction
The major purpose of these thesis is to test the relationship between textual content written by professional and consumer reviewers of motion pictures. Researching the
consensus between these written evaluations is just an addition to the thesis, an can perhaps give additional results. With that insight, this thesis is also going to test the influence of two other characteristics of movie products, namely
the genre of the movie and whether or not there is an age restriction attached to the movies. These characteristics will serve as dummy variables, to research whether the relationship
between written expert and consumer reviews Table 1: Movie genres
and movies performances will change adding these two variables.
The first variable will be ‘Genre’. Genre is the phrase for any category of literature or other kinds of art or entertainments, like music, could be spoken or written, audial or visual (movies) and is based on different sets of criteria. Genres are developed by agreements which can change as new genres are invented and the existence of old ones are terminated. Most of the time, entertainment products fit into multiple genres. For instance, The Wolf of Wall Street, the movie of the introduction, can be labeled as a ‘’comedy’’, but also as ‘’crime’’ and
Action Adventure Comedy
Crime Erotica Faction
Fantasy Historical Horror Mystery Paranoid Philosophical Political Realistic Romance Saga Satire Science fiction Slice of life Speculative Thriller
‘’realistic’’. In table 1 you can find a list with the possible genres a movie can be labeled with. The researcher expect that this variable hasn’t got any influence on the relationship between movie performances and written evaluations of those products.
Another variable which can have an influence on the box office performance of
Hollywood motion pictures is the ‘motion picture rating system’. This system is designed as a classification for movie products and checks if movies are suitable for consumers in terms of delicate issues like sex, violence, substance abuse and other types of adult content. Another term for these classification is ‘rating’. This system is designed as a helping tool for parents or guardians to decide whether a movie picture is suitable for children. The ratings used in the United States (and noted in that manner in the dataset) are G (general audience), PG (parental guidance suggested), PG-13 (parents strongly cautioned), R (restricted for children under the age of 17) and NC-17 (children are not admitted).
The researcher expect that this variable hasn’t got any influence on the relationship between movie performances and written evaluations of those products.
H4: The relationship between the performance of Hollywood motion pictures and expert and consumer written reviews is not going to be changed through the influence of the variables ‘’genre’’ and
‘’rating’’.
In the next section the research design (‘’Methods’’) will be explained. This section will be used to give the research outline, explain the steps taken with regard to data collection, state information about the sample size and measurement developments, and explain the necessary statistics and quality criteria.
Methods
Research outline
My empirical study focusses on measuring the effects of the content of film reviews on the box office performance of Hollywood blockbusters and other mainstream motion pictures. A blockbuster is a term in the motion picture business, which is used to indicate movies which have big star appearances and whereby is expected that it will receive big box office revenues. The main objective of the study is to test whether there is a relationship between expert and consumer reviews and the performance of movies, measured by the box office performance of those movies. Also the consensus between expert and consumer reviews is researched. To do this, this study builds on data which is transferred from qualitative into quantitative data. A part of the data is made available by the thesis supervisor, other parts are subtracted from the internet. Because the data is already available on the internet, costs can be reduced by
performing a quantitative research which is one of the several advantages of this type of research (Thomas, 2013). Findings from this research support and elaborate on findings from earlier research in the same field.
Data collection
The dependent variable in the study is the performance of motion pictures. Because ‘performance’ is a vague concept to work with, the decision is made to use the box office revenues of those motion pictures as an indicator for the performance. The films used in the research is decided by the data gathered by the thesis supervisor. He provided several excel sheets, and all the movies with matching reviews are used in this study, provided that box
www.the-numbers.com, a website where ‘’ data and movie business meet’’. This medium has got a large data network of the global box office revenues of motion pictures. Unfortunately, not for every movie in the dataset is box office information available, so these movies and the associated expert and consumer reviews are deleted from the study. This is called list wise deletion, where you only do analysis on those cases without any missing data in any variables.
An example of the excel sheet with all the movies (products) and total domestic box office revenues can be seen in the table 2.1 (see appendices). As you can see, before the movie titles you can see a column called ‘’ID’’. These ID numbers matches with other excel sheets with data describing professional and consumer reviews. Some ID numbers are missing from the list (for example, #1,#2, #3, and the numbers between 4 and 14). For these motion pictures, no box office data could be found so they are deleted from the list.
The online film reviews are my independent variables. The texts can be divided in four variables, consumers reviews which contain positive affective content, so positive emotion (ConsPos) and negative affective content, negative emotion (ConsNeg) and expert reviews with the same dimension (ExpPos / ExpNeg). The data is subtracted with a tool from Metacritic. Both groups, consumer reviews and expert reviews, are saved in separate excel sheets, so the focus in this study is easier kept. You see examples of those excel sheets in the appendices (table 2.2 and 2.3): You can see the first expert reviews are all corresponding with the ID_Movie #4, which stands for the movie ‘(500) Days of Summer’, see table 2.1 .
Sample
The dataset with motion pictures started with 6422 movies. In this dataset you find the biggest productions of Hollywood motion pictures like Avatar, Pirates of the Caribbean, The Dark Knight Rises, all movies from major publishing institutes like MGM, Fox, Warner Bros. Pictures, Columbia Pictures and so on. But also movies with much smaller production budgets, and from smaller publishers are in this dataset. For many of these movies, the total
box office revenue is never made public, so these movies are deleted from the dataset, as mentioned before. After deleting some movies, the dataset contained 2572 movies. This is a percentage of 41% of suitable movie products residual from the original dataset.
Matching with the movies found in dataset 1, the dataset with the consumer reviews started with 188.789 reviews. After the likewise deletion, still 145.426 reviews were suitable for data analysis. This is a percentage of 77% . In the case of the dataset with the expert reviews, the sample started with 136.907 expert reviews, and after the deletion the dataset was reduced until 72.045 reviews , a percentage of 53% . These reviews are from the major North American newspapers like the Wall Street Journal, Times, Variety and the New York Times, all newspapers with a major impact.
Table 3: Sample size used to test the hypotheses
Measurement development
To estimate the emotional content of the reviews texts, I’m going to conduct a content analysis of the review’s qualitative texts. According to Sing et. al (2011), content analysis is an increased used method for studying online posts. To qualitative data will be transferred into quantitative data. For this transformation, content analysis uses automated, systematic procedures, which ensure the reliability and objectivity of the data analysis (Chung, Pennebaker and Fiedler, 2007).
Movies Expert reviews Consumer reviews
Original dataset 6.422 136.907 188.789
Dataset after deletion 2.572 72.045 145.426
The online reviews are going to be automatically analyzed by using the LIWC (Linguistic Inquiry and Word Count) program developed by Pennebaker et. al (2007). According to them, LIWC dictionaries ‘’offer strong, reliable convergence between the dimension they extract and content ratings performed by human coders’’ (Ludwig et. al, 2013). So the program is using word counts for a given reviews and calculates the proportion of words with a predefined dictionary. For example, ‘’dumb’’ is written in the negative affective dictionary, and will be counted as 1 in the total amount of negative affective content words in the review. The method design is based on the study of Ludwig et. al (2013), where in this case the affective score is calculated similar to their model. The remaining part of the model given in the literature review (page 17), and used for this study is as follows:
Fig 4: The model used for this thesis
In the original model, ACit is representing the overall intensity of affective content in reviews for movie i in week t. The first part of the equation (before the + ) can be set on 0, because there aren’t any review titles in the gathered data. So this part is deleted in the adjusted model. The subscript B denotes the body of the review text, and the calculations for affective content of this body are the same as for the titles, so PAitb stands for the positive affective content words in the body of the texts, the NAitb for the negative affective content words and Nitb represent all the words used in the texts. This part isn’t divided by two anymore (original model), because the title score is not applicable. In simple content, per review the equation will give a score, based on the positive affective content in the review,
minus the negative affective content, divided by the total amount of words in the review. To know the total amount of words of each single review, the formula
‘’=SOMPRODUCT(LENGTE(SUBSTITUEREN(SPATIES.WISSEN(E2)," "," "))-LENGTE(SUBSTITUEREN(SUBSTITUEREN(SPATIES.WISSEN(E2 )," "," ")," ",""))--(LENGTE(SPATIES.WISSEN(E2))>0)) ‘’ was added in the excel sheet so the LIWC scores could be calculated. After all the single LIWC scores per review were available, the LIWC per movie product could be calculated, just as an average of the single scores.
Statistics
After the required data is attained the analysis of the results is the next stage in the research. The data is analysed by making use of statistical program SPSS 22. First descriptive statistics are presented, the correlation matrix is analysed in order to get an insight in the relationship between de dependent variable and independent variables. With regard to the independent variables, first a linear regression analysis will be performed.
When using SPSS, the linear regression analysis is an approach by which the relationship between a dependent variable, in this case the box office performance of
Hollywood Motion Pictures, and one of more explanatory variables can be researched. When the relationship is tested with one explanatory variable, this test is called a simple linear regression. In the case of this thesis, more than one explanatory variables are used in the research, so we make use of a multiple regression analysis. The independent variables are the textual contents of the consumer and expert reviews, so the LIWC scores, other independent variables are production budgets, movie ratings (scores) and word counts, and the dummy variables will be ‘’genre’’ and ‘’rating’’ of the movie products.
Quality criteria
This research has to meet certain criteria to be able to make meaningful statements. According to van Aken et al. (2012), these requirements are three quality criteria; controllability,
reliability and validity (van Aken et al., 2012). Controllability is whether the equal results would be found if other researchers would conduct the same study. ‘’Meeting the
controllability criteria leads the way to meet the validity and reliability criteria’’ (van Aken, 2012).
The reliability criteria show many similarities with controllability criteria.
Successfully meeting the reliability criteria means that when you should repeat the study you will come up with the same results in the future (van Aken, 2012). Saunders et al. (2009) define reliability as the extent to which data collection techniques or analysis procedures yield consisting findings. And that the consistency can only be determined through multiple
measurements. So it is an instrument’s ability to yield comparable results across similar situations. In the case of this thesis, it is about the fact if other researchers will be able to obtain the same results as I have found if they replicate the same analysis.
The last criterion that ensures the quality of the research is the validity criteria. The validity can be divided into internal validity, external validity and construct validity. Validity is about the quality of the conclusions that are drawn from the research (van Aken, 2012). Cronbach and Meehl (1955) define validity as the degree to which an instrument measures the construct it is intended to measure. With external validity, the results should be statistically generalizable, so the same results can be replicated in another sample. Also internal validity should be taken into account. It is about whether the conclusions that are made truly can be made, looking at the way the research was conducted and designed. The research design should rule out alternative potential explanations for the observed relationship.These are crucial criteria that have been taken in mind while conducting the research.
Analysis and results
Correlation matrix and descriptive statistics
Table 5 (page 31) presents the correlation matrix and descriptive statistics of the variables used in this research. This information gives an good overview of the main features of the variables (see table 5, overview of the variables, page 29). The results of the correlation analysis can be used as a preliminary evidence to support the basis hypotheses. On average, the movies in the sample received positive reviews from both consumers and experts (consumers µ = 6,67 out of 10, experts µ = 57,94 out of 100). Consumers use more words writing their evaluation of movies ( µ = 67,6) compared with experts ( µ = 24,4).
Variable name Description
logBoxO Sales data what is adjusted by inflation
logBudget Production budget what is adjusted by inflation
Worldwide Gross The worldwide total revenues of a movie product (in $)
Production Budget The production costs of a movie product (in $)
consEmo The average LIWC score of consumer reviews
expEmo The average LIWC score of expert reviews
VarOfposemo The variance of positive emotion in the consumer reviews
VariOfposemo1 The variance of positive emotion in the expert reviews
ConsRevCon_AvgOfScore The average of the consumer score of the combined reviews
ConsRevCon_AvgOfWord The average of the used words in consumer reviews
ConsRevCon_VarOfScore The variance of the consumer score of the combined reviews
ConsRevCon_VarOfWords The variance of the used words in consumer reviews
ExpRevCon_AvgOfScore The average of the expert score of the combined reviews
ExpRevCon_AvgOfWords The average of the used words in expert reviews
ExpRevCon_VarOfScore The variance of the expert score of the combined reviews
ExpRevCon_VarOfWords The variance of the used words in expert reviews
Action Genre classification of movie is Action; dummy variable
Adventure Genre classification of movie is Adventure; dummy variable
Comedy Genre classification of movie is Comedy; dummy variable
Drama Genre classification of movie is Drama; dummy variable
Romance Genre classification of movie is Romance; dummy variable
Sci-fi Genre classification of movie is Sci-Fi; dummy variable
Thriller Genre classification of movie is Thriller; dummy variable
Rated R Rating classification of movie is Rated R; dummy variable
The correlation analysis provides the information that positive consumer reviews have a positive and significant correlation with the worldwide box office revenues (performance) of motion pictures ( r = .554, p < .01) and also with the computed variable (logarithm) of the box office of motion pictures ( r = .361, p < .01). The computed variable (logBoxO) is the sales data which is adjusted by inflation. The same is done with the production budget of the movie products. The analysis also provides preliminary evidence that positive expert reviews should have a positive and significant correlation with the worldwide box office revenues of motion pictures ( r = .174, p <.01) and with the computed variable ( r = .213, p <.01).
Other information that could be found in the table is the significant correlation
between the aggregated consumer scores and the performance of motion pictures ( r = .045, p < .05) and the significant correlation between the aggregated expert scores and the
performance of motion pictures ( r = .175, p < .01). With regard to production budget, the correlation analysis provides a significant correlation between worldwide box office revenues and production budgets ( r = .709, p < .01), which can be seen as a logical outcome of the analysis. Also there is no correlation above r = .5 between the independent variables, therefore the independence of the constructs is verified.
Variable name Mean St. Dev. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 logBoxO 16.8138 1.42572 2 logBudget 17.2999 2.04416 .675** 3 Worldwide Gross 10841940 7.92 173855625. 196 .609** .470** 4 Production Budget 38491312 .38 41548475.9 11 .560** .726** .709** 5 consEmo 232.7543 448.09088 .361** .256** .554** .410** 6 expEmo 77.9607 79.39090 .213** .101** .174** .104** .302** 7 VarOfposemo 44.83 70.236 -.040* -.002 -.044* -.031 .016 .073** 8 VariOfposem o1 33.22 27.223 .043* -.012 .015 -.033 .049* .509** .106** 9 ConsRevCon _AvgOfScore 6.67 1.765 -.043* -.168** .045* -.114** .198** .208** .097** .093** 10 ConsRevCon _AvgOfWord 67.60 36.105 .033 .001 .087** .081** .081** -.040* -.217** -.163** -.087** 11 ConsRevCon _VarOfScore 8.58 5.152 -.046* .035 -.104** -.007 -.09** -.143** .016 -.024 -.494** -.090** 12 ConsRevCon _VarOfWords 7267.95 11102.732 .027 -.005 .045* .037 .043* .042* -.077** -.064** -.034 .764** -.053** 13 ExpRevCon_ AvgOfScore 57.94 16.001 .092** -.141** .175** -.008 .314** .418** -.006 .071** .492** .142** -.339** .109** 14 ExpRevCon_ AvgOfWords 24.40 5.162 .134** .067** .148** .108** .141** -.080** -.106** -.378** -.008 .283** -.109** .146** .189** 15 ExpRevCon_ VarOfScore 270.39 123.238 -.031 .016 .029 -.016 -.10** -.247** .005 -.063** .017 -.065** .049* -.084** .249** .002 16 ExpRevCon_ VarOfWords 129.64 55.373 .043** .025 .065** .063** .138** -.034 -.053** .108** -.075** .161** -.004 .067** .090** .573** -.051** Note: ** Correlation is significant at the 0.01 level (2-tailed ) | * Correlation is significant at the 0.05 level (2-tailed)
Test of the hypotheses
First a summary of the model. The sample size in this research was 2.572 motion pictures (N=2572). The model correlates with .743 with the independent variables. Also the model explains for 55,3% (.553 under R²) the dependent variable, Hollywood motion pictures.
The hypotheses are tested with a linear regression analysis. The results of the analysis are presented in table 6 (see next page). The most important outcome of this analysis is the relationship between consumer and expert reviews and the performance of motion pictures. The results of the analysis suggest that affective content in consumer reviews is positively related to the box office revenues of movies ( β = .141, p < .000), and that affective content in expert reviews is positive related to the box office revenues, but nog significant ( β = .034, p >.1). These results offer support for H1b and H2b, but reject H1a and H2a. To use the words of Eliashberg and Shugan (1997), the expert reviewers in this case can be seen as
‘’influencers’’ and consumer reviewers as ‘’predictors.
With regard to the second hypotheses, ‘’The relationship of expert (or consumer) reviews with the performance of Hollywood motion pictures is moderated by the degree of consensus among expert (or consumer) reviews of those motion products. The relationship is less positive if there is a lower degree of consensus’’, the results of the analysis suggest that a high level of variance in positive emotion in consumer reviews has a negative influence on the performance of movies (β = -.037, p < .01), so when converted these results mean that a high level of consensus in expert and consumer reviews have a positive influence on the
performance of movies. Results also show that a high level of variance in expert emotion has a positive influence on the performance of movies ( β = -.043, p < .01). These results offer support for H3a and H3b.
Table 6: Regression results
The third hypothesis was ‘’the relationship between the performance of Hollywood motion pictures and expert and consumer written reviews is not going to be changed through the influence of the variables ‘’genre’’ and ‘’rating’’.
To find results for this hypothesis, there must be some sort of consensus among those dummy variables and their relationships with the dependent variable. So if those variables score comparable, we can assume that the relationship between the performance of motion pictures and expert and consumer reviews is indeed moderated by the influence of these
Variable β Stand. error Beta Sig.
(Constant) 1.748 .480 .000 Drama -.527 .061 -.132 .000 Comedy -.008 .068 -.002 .904 Romance .156 .071 .033 .027 Action -.010 .078 .002 .893 Thriller .408 .069 .097 .000 Sci-Fi -.206 .098 -.031 .035 Adventure .006 .086 .001 .944 Rated R -.421 .060 -.105 .000 logBudget .838 .022 .598 .000 ConRevCon.AvgOfScore -.061 .022 -.052 .005 ConRevCon.VarOfScore -.004 .006 -.011 .485 ExpRevCon.AvgOfScore .025 .003 .199 .000 ExpRevCon.VarOfScore .001 .000 .041 .006 ConRevCon.AvgOfWords -.002 .001 -.033 .163 ConRevCon.VarOfWords 5.835E-6 .000 .033 .136 ExpRevCon.AvgOfWords .032 .008 .081 .000 ExpRevCon.VarOfWords -.002 .001 -.064 .000 consEmo .001 .000 .141 .000 expEmo .001 .001 .034 .103 VarOfposemo -.001 .000 -.037 .008 VarOfpesemo1 -.001 .001 -.043 .009
a. Dependent Variable: logBoxO
N 2572
R .743
R² .553
variables. The results of the analysis suggest that there is no compared relationship between ‘’genre’’(selection of the most selected genres on Metacritic) and the performance of motion pictures ( drama, β = -.132, p < .05| comedy, β = -.002, p > .05 | romance, β = .033, p < .05 | action, β = .002, p > .05 | thriller, β = .097, p < .05 | sci-fi, β = -.031, p < .05 | adventure, β = .001, p > .05).
So some of the movie genres have a negative influence on the box office performance (drama, comedy, sci-fi), other genres have a positive influence on the box office performance (romance, action, thriller, adventure) but most of them are not significant related to the performance of motion pictures.
The ‘’rating’’ variable has also got an negative influence on the performance ( β = -.105, p < .05). So we can assume that the relationship between expert and consumer reviews and the performance of motion pictures is not moderated by the genre of the motion pictures, and this results supports H4. It can be stated that most of the hypotheses are supported by the results of the analysis. Only the expectation that there was a positive relation between positive emotion in expert reviews and the performance of movies was not justified. This hypothesis was rejected by the results.
Discussion
With regard to the first hypotheses, these results were partially expected (consumers expectation is supported, experts expectation is rejected), but can be explained by viewing expert reviewers as real professionals, they write many different reviews about many different movies and have all the time an objective opinion about the product. Consumer reviewers are the fans of the products, most of the reviews are really positive about the products, they are the fans who will go to the theatres and are responsible for the box office revenues of the movies. A small part of the consumer reviews contains negative content, those consumers use
their evaluation as a medium to show their disappointment in and frustration about the product. In line of the language used by Eliashberg and Shugan (1997), expert reviews influence consumers by providing objective information about the products, consumer reviews can be seen as predictors for the box office performance of movies.
The second set of hypotheses were based on the consensus topic. Results show that there is a high level of agreement about the quality of a product, by both experts and by consumers, you can see the influence of this agreement back in the performance of movies. Whether the influence is positive or negative depends on the aggregated kind of affect these reviews contain. In the case of the thesis, for both the expert and consumer reviews the combined reviews were located on the positive side of the positive-negative continuum. So the overall content was relatively positive. If the combined reviews had been relative negative, the results probably would have been the opposite. So, if the affective content is relatively positive, the influence will be positive and if the affective content is relatively negative, the influence will be negative.
Implications
The outcomes found in this thesis have several implications for marketing research and marketing managers. First, this study is an attempt to gain new insights in the differences between experts and consumer reviews. In the past, most of the studies had as topic expert reviews, because these were more easily available. Consumers had few options to publish their evaluations of motion products before the internet developed. According to Zhu and Zhang (2010), there have been few empirical research papers which explore consumer reviews. Second, due to results in earlier research, there was an expectation to find a positive relation between positive emotional content in expert reviews and the performance of motion pictures. The analysis show contrary results, what means that conclusions in former research don’t hold up in this thesis.
Third, this study differences from previous studies in this area by using textual data mining tools and by focusing on the textual content of reviews. In most of the previous research, the dataset used in the analysis contains numerical data (consumer and expert scores). Textual data mining tools offer a new way of studying the impact of consumer and expert evaluations of motion pictures on the box office performance of movies. To do this kind of research, a very large dataset and a powerful calculation algorithm is required, but if succeeded, that is only beneficial to the outcomes of the study.
Fourth, this study and the results have considerable managerial implications, where the results can be used for profit-oriented improvements. It is important to know for movie
producers that the worldwide box office revenues of their products are influenced by the opinions of the consumers whom evaluating the quality of the product. It explains the importance of the management of these external influences, because for the success of the movie it is important the product evaluations are positive. It is the task of marketing managers in this business to ensure the positive content in consumer reviews by all means.
Another important implication is that producers and managers in the movie business shouldn’t take the opinion of expert reviewers too much into account, because results
suggested that the affective content in their reviews is not significant related to the box office revenues of motion pictures.
Limitations
At the same time, this study also contains several limitations. First of all, this study is limited to the motion picture industry, and solely used movie products produced in Hollywood. Besides the fact that the relationship between movie performances and expert- and consumer reviews in other film industries can be researched ( think of ‘’Bollywood’’ or smaller art-house movies) by using textual data mining tools, more research is needed to confirm the results of this research. The construct of research design can be applied by analyzing other
experience goods where there can be an influence of consumer and expert reviews on the performance (sales) of those products. You can think of any product where consumers consult internet sites before they decide to purchase the product.
Second, the data sources limits the range of and the possibility to repeat the study. If Metacritic also provides box office revenue data in their database, probably a much larger percentage database would remain to research in this thesis. Now, two different datasets had to be combined (the reviews were extracted from Metacritics, the box office data was extracted from The-Numbers), where there was less overlap than expected. Because of this lack of overlap, much data had to be deleted before several analyses could be performed. Because consumers and experts have the ability to change, remove, and add reviews to the website Metacritics, the possibility to repeat this study is limited. The same accounts for The-Numbers’ website. Worldwide gross revenues are constantly subject to modification due to addition of new box office data. This is another infraction in the possibility to repeat this study.
Future research
This study points the way for further research with the use of data text mining models in other industries were experience goods are produced, and where there can be an influence of
consumer and expert reviews on the quality of the products. Many reviews written on the internet are for example about electronics or other technologic devices, in this industries the same research design can be applied. This is beneficial for the reliability of this research.
Future studies in the same research area is also necessary, where data is used that actually reflects the behavior of the consumers who have written the reviews. By using data from Metacritics, you can’t be sure that the consumer reviews are truly based on experiences with the product, due to the high level of anonymity among consumer’s identifications.
Also, future studies that extend the approach used in this study in the motion picture industry, may also have to include more industry specific signals that may influence the performance of motion pictures. For instance, in analyzing signals in the movie industry, it is important to control for star power ( Ravid, 1999). Other signals are used in this study, consumer and expert scores, the word count of the reviews, genres and rating classifications, but there are many more movie industry characteristics which could be taken into account.
Finally, this study points the way toward further research on the effects of consensus among expert and consumer reviews. Consensus can be conceptualized and measured in different ways than is done in this study. Also the effects of consensus among expert reviews on consumer evaluations can be researched, as well as the effect of variance in reviews as a moderator on other signals of quality of experience goods.
Conclusion
The research area of this thesis was the impact of film reviews on the box office performance of Hollywood motion pictures. The movie business ‘’is a profoundly uncertain business’’ (De Vany and Walls, 1999). There is many research done in this field, where most of the research is done by comparing the scores of written expert reviews and consumer evaluations with the performance of those movies (Eliashberg & Shugan, 1997; Shrum, 1991; Gemser, van Oostrum & Leenders, 2006, Basuroy, Chatterjee & Ravid, 2003). But less research is done with regard to analyses of the qualitative content of reviews and their relationship with the performance of movies. Situmeang et al. (2013) state that expert and consumer reviews transmit information about products, and this is necessary because the ‘’quality of the products is difficult to evaluate before consumption, so both types of reviews can have an strong impact on consumer behavior’’. Motion pictures are real experience goods
reviews of those experience goods, and the influence on the performance of movies. Isen (2001) states that it is very widely accepted that affect plays a role in cognitive processes as decision making. So consumers who search for information about movie products, are probably influenced by the affective content in reviews they read. To investigate this statement, the research question for this thesis was:
‘’What is the impact of the positive and negative affective content in expert- and consumer reviews on the box office performance of Hollywood motion pictures? ‘’
The first aim for this thesis was to determine whether affective content in expert reviews and consumer reviews have an impact on the box office performance of Hollywood motion pictures. Affect is a mental response to the environment and can be described as emotions, moods and attitudes (Peter & Olson, 2001). Based on the literature review,
hypotheses were framed, where the expectation was (i.) that there was a positive relationship between positive reviews written by professionals (or consumers) and the box office
performance of movies, and (ii.) there is a negative relationship between negative reviews written by experts (or consumers) and the box office performance of movies. The results suggested that affective content in consumer reviews is positively related to the performance of motion pictures, and that affective content in expert reviews is positive, but not significant related with that performance.
The second aim of this paper was to explore the effects of (the lack of) consensus in the reviews written by experts and consumers as a determinant in the relationship between written reviews and movie performances. The expectation is that there will be a high level of conformity within professional and consumer critical reviews. This because given the fact that expert critics are professionals and are used to take their peers’ opinion into account when they write their review (Das and Chen, 2007). The same expectation accounts for consumers.
