1
Georgios-Vasileios Araitzoglou - 11385359
Master Thesis - MSc Business Economics
Managerial Economics & Strategy
Number of ECTS: 15
14
thof July 2017
2
Statement of Originality
This document is written by Georgios Vasileios Araitzoglou who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in
creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Acknowledgment
I would like to thank my family for their lifetime support, as well as my mentor Michail Amourgianos and my Brothers (Nikos, Timos, Ntinos, Marios, Menios, Dimitris, Thanos, Kwstas, Vag, Giannos)
3 Abstract
I provide evidence that the additional information of Rating and Price have a significant effect on people’s predictions of the actual next rating from a sample of properties in Amsterdam from the platform of Airbnb. Specifically, supported by the survey that I conducted, people that belong to the control conditions with either rating or price as additional information made predictions closer to the actual next rating of a random guest. Also, prediction errors along with their confidence intervals showed significant differences in thirteen out of twenty-one selected properties. With respect to the prediction errors of each participant per each property, panel data regressions were used and for treatment and control conditions regarding “rating”, the additional information of the number of reviews has a significant effect but by increasing the prediction errors. On the other hand, for the treatment and control conditions regarding “price”, only price has a significant effect in decreasing the people’s prediction errors, while reviews, current average rating, location and demographics don’t affect significantly the prediction errors. Finally, properties with more reviews increase the predictions of the participants. However, a generalized result cannot be provided as the prediction errors are higher for the properties with many reviews. Therefore, predictions of participants are independent from the number of the reviews of each property.
4
Contents
1.Introduction ... 5
2. Literature Review ... 11
2.1 Reviews and Ratings over Sales ... 11
2.2 Reviews &Ratings Biases (Behavioral-Statistical) ... 12
2.3 Word-Of-Mouth Online Consumer Reviews... 13
2.4 Determinants of reviews and ratings ... 14
2.5 Reviews/Rating and Trust ... 15
3. Methodology ... 16
3.1Hypotheses ... 19
4 Data gathering and variable definitions ... 23
4.1Survey mechanism ... 23
4.2 Survey questions ... 23
5. Results ... 24
5.1 Data and summary statistics ... 24
5.2 Prediction Errors ... 29 5.3 Regressions ... 38 5.3.1.Hypothesis 1 ... 38 5.3.2.Hypothesis 2 ... 42 5.3.3.Hypothesis 3 ... 45 6. Discussion ... 49 7. Conclusion ... 53 8. Appendix ... 55 8.1. Appendix A ... 55 8.2 Appendix B ... 58 8.3 Appendix C ... 59 8.4 Appendix D ... 60 8.5 Appendix E ... 61 8.6 Appendix F ... 62 9. References ... 65
5 1.Introduction
In a period where peer-to-peer accommodation platforms have gained increasing popularity among consumers, ratings and reviews constitute the necessary stepping stone for understanding a widely form named as “user generated content”. Also, in a highly sophisticated world with consumers more prepared to analyze online information, the impact of ratings and reviews is gaining high significance. As a result, the impact of them has acquired extended global adhesion as more and more industries have already started to focus on how they can analyze and interpret ratings and reviews, let alone the significant profits that the firms can acquire by increasing the awareness with them.
The first approach for understanding the “user generated content” is derived from reviews. Reviews are considered to be written comments from direct or indirect consumers emphasizing their opinions to a product or service. The review system was first introduced by Epinions.com and Amazon.com and these reviews were based on writing opinions from both users and non-users, something that alerted the first contradictions towards the biases of these personal comments. Some critics point out that positive reviews are sometimes written by the businesses or individuals being reviewed, while negative reviews may be written by competitors, or anyone with a grudge against the business being reviewed (Qiang et al. 2009). While the beginning of the review system had introduced high consideration with regard to the actual objectivity that it could provide to the broad online market, the last decade a different approach made the reviews an important intermediate in the purchasing decisions of consumers. More specifically, many companies that are active on the Web, allow consumers to write a review only if they have experienced the product/service at least once. Therefore, random and subjective reviews were eliminated by giving emphasis to the actual experiences of consumers that used the product. However, many academics have argued that the biases in the comments still exist because group polarization often occurs, and the result is very positive comments, very
6
negative comments, and little in between, meaning that those who would have been in the middle are either silent or pulled to one extreme or the other (Yardi and Boyd, 2010).
Another approach is the online content of ratings on online platforms where they were created as a quick indicator that measures the evaluation of a product or service and comes in line with the comment of the review. In other words, ratings are used as a supportive element and most of the times as a supplementary of the reviews. Ratings taking the form of numbers via half or entire integers and often a star symbol is used for representing either the 1-10 or 1-5 rating scale. Similar to reviews, critics emphasize that even if it’s a quicker way of displaying the user-generated content, many statistical biases exist which makes them valueless (Dellarocas 2000). Academics are perpendicular against the use of ratings as many consumers are using them as a substitute of reviews without actually investigating whether or not a high rating contains opinion biases or overrating effects.
Having a general overview in the user generated content from ratings and reviews is critical to examine today’s pioneers of this sector. Nowadays, plenty of platforms exist that are specified to provide experienced reviews in different markets. Yelp and TripAdvisor are two of them and their main focus is to provide reviews from their large social community. Thus, the interaction of consumers with reviews and ratings had shaped out new platforms that target to take advantage of it by positioning themselves as intermediates between companies that provide the actual product or service and the end-users/consumers. Another interesting market that it’s in the lights of the publicity the current years is the “Sharing Economy” market. Sharing Economy or Collaborative Consumption; consumers gain access to goods and services by paying for the experience of temporarily accessing them, highlighting that no ownership is transferred in these transactions, (Bardhi and Eckhardt, 2012). Companies like Airbnb and Uber are some examples of this blooming market where the ratings and reviews are considered really important, not only because they can increase the awareness
7
of a specific property or a car driver when having high ratings and good reviews, but also because they can successfully moderate any “trust concerns”.
The aim of this essay is to examine the next rating predictions of 124 individuals from a sample of properties in Amsterdam from the platform of Airbnb, taking into consideration different information content for each prediction. It is not possible to directly measure the predictions of people without specifically asking them to do so. Therefore, a survey on Qualtrics was created with participants distributed to one of the four conditions that are specified to different information sets over different properties. In the two conditions; “Control Rating” and “Treatment Rating”, participants made predictions for the same ten properties, but under different information sets. More specifically, the participants in the “Control Rating” condition had all the information of that property available including; current average rating, price, number the last three reviews of the properties, location as well as a general description. On the other hand, in “Treatment Rating” condition, participants had the same information except the current average rating. The rest of the participants were randomly assigned to either “Control Price” condition or “Treatment Price” condition, but in this case, the focus is whether or not the information of price made the predictions of the participants more or less accurate towards the next rating from a random guest on Airbnb. Additionally, one more property with either low or high reviews was displayed to all of the participants, in order to explore whether the difference in the quantity of reviews significantly affects the predictions of the participants. The motivation behind this essay stems from whether someone actually cares about the experiences of a random renter in the platform of Airbnb. There are many reasons to care for someone else’s experiences. First, as mentioned before, platforms like Airbnb were created for the purpose of enabling people to rent short-term accommodations all over the world from just owners and not specifically professional renters. Thus, one of the most detrimental factors is whether or not perspective renters trust the experiences (the reviews) and the ratings of people that have lived in that property. As a result, the question of
8
predicting the next actual rating covers inside also a “trust element” of whether the available information that is displayed on a property’s page can be informative and can minimize any “trust concerns”. Second, and related to the above, experiences that are displayed online comprise a social cost (Gunawardena et al. 1997). Reviews on online accommodation platforms such as Airbnb incorporate the experiences of previous renters. Therefore, the social cost of these either positive or negative experiences is revealed and shared across the individuals that are searching for that property. By asking them to predict the next rating of a chosen property from Airbnb, participants make a general evaluation not only from the actual tangible services (house supplies/information) and intangible services (distance to center, price, name of host etc.), but also by taking into consideration the social costs from the previous renters. Thus, participants that predict closer to the actual rating of a random renter can balance and analyze the information of the property better as well as they can incarnate more efficient the previous renters’ experiences. Third, when people think of experiences, what usually follows is to represent them with situations from familiar faces, such as family or friends (Boothby et al. 2014). For that reason, the predictions of the next actual rating can incorporate more interpersonal elements than just predicting a rating from a random guest. Consequently, making predictions in the content of this paper could also be interpreted as trying to understand the perspective experiences of familiar-related individuals.
Another highly related motivational factor of this research is whether people predict accurately upcoming ratings. The information that is acquired from this analysis indicates that various companies could initiate research towards acquiring upcoming predictions from a random sample of the general public. Specifically, by using surveys and other-related gathering data tools, companies like Airbnb could better understand in a short-term period whether new properties have the potential to become highly successful for the period that they will be listed online. Hence, decisions regarding the pricing as well as the display of a property could become more efficient, especially if this is analysis is produced by
9
a sophisticated algorithm which minimizes the time and the resources spent. Overall, this paper covers a small but significant aspect of how information and predictions from actual consumers can enhance the “user-experiences” inside platforms that based on “user generated content”, by giving emphasis on the fact that predictions can be considered as objective when the content is strictly focused on ratings and reviews from online accommodation platforms.
The structure of this essay is as follows. Section 2 discusses the literature review of reviews and ratings along with every supportive element that is detrimental for the interaction of the reviews and ratings in online platforms. Results are presented from other papers, where behavioral, statistical as well as other empirical approaches are analyzed and their results towards the reviews and ratings provide clear guidance throughout this paper.
Section 3 outlines the Methodology as well as the creation of the hypotheses.
Initially, an in-depth analysis conducted and several heuristics used for collecting twenty-one properties of interest from the platform of Airbnb. Then, an analytical way of the steps that were materialized is explained with given emphasis on the “inflated rating” effect that also exists for the properties in Amsterdam. Finally, hypotheses were formulated and a motivational reason for the content of them is provided.
Then, the method of data gathering is discussed (Section 4) along with the variables definition and interpretation. Initially, the survey method is discussed by analyzing the differences between the different information conditions. After elaborating on the different questions that were asked to the participants including properties ratings predictions and demographics, variables categorization is provided. More specifically, the questions are analyzed with statistical and numerical interpretations and the purpose of these questions is emphasized.
Section 5 presents the results of the survey including basic descriptive statistics
10
of each individual and per each property. The section concludes with panel data regressions and emphasizes on the prediction errors of participants taking into consideration different specifications.
Section 6 provides a discussion of the paper as well as some limitation of the
procedures and the collection of the properties.
11
2. Literature Review
2.1 Reviews and Ratings over Sales
Academics have recognized the importance of online reviews and ratings and have already produced a number of important results in this area. The predominant literature on reviews focuses on the correlation between consumer reviews on sales (Chevalier and Mayzlin 2006, Clemos et al. 2006) as online reviews can positively influence future sales of a product or service. Basuroy et al. 2003, was one of the first academics that revealed a correlation in the revenues of weekly box office movies with either positive or negative review and found that negative reviews can hurt only short-term (first week) the movie’s run. Chevalier and Mayzlin (2006) focused on the book market (Amazon.com and Barnes&Noble) and one of the interesting findings in their study is that negative reviews have a much greater effect on book sales than positive reviews. In the same market, Zhu and Zhang 2006 found that online reviews are more influential for less popular online games. This finding suggests that the information role of reviews becomes more prominent in highly competitive markets and marketers can benefit more by allocating resources to managing online consumer’s reviews. Empirical results from Hu et al. 2013 show that the rating effect on sales is mostly indirect, and only through sentiments. An interesting implication of their study is that ratings can play a significant role in the early stage decision, but the final evaluation and choice comes from the sentiments. On a different note, Hu, Liu, 2008 also gathered data from Amazon.com but studied how consumers use qualitative and quantitative aspects of online reviews to make purchase decisions. By using the Transaction Cost Economics (TCE); theory that specifies on asset specificity, uncertainty and transaction frequency, (Williamson 1979) they show that a higher level of uncertainty implies a higher transaction cost because of the information asymmetry and thus will result in lower sales.
12
2.2 Reviews &Ratings Biases (Behavioral-Statistical)
Understanding how online reviews are affected by previous reviewers is crucially to gauge the extent of their independence or bias, especially given the significant effect that online reviews have on sales. Thus, a highly interesting topic which is in the lights of the publicity from academics nowadays refers to the statistical and behavioral biases of online reviews and ratings. One of the major concerns about the online reviews and ratings is whether the accumulated information about a product or service truly reflects the unbiased opinions of reviewers. Research by Askalidis et al. 2017 focus on how retailers can reduce the impact of Social Influence Bias and Selection Bias that refer to the tendency that one’s opinion is influenced by the opinions expressed in other reviews and when a submitting review is not representative of the entire purchasing population, respectively. They found that by splitting verified buyers in either intrinsically or extrinsically motivated groups1 can help reduce people’s tendency to be affected by previously posted reviews. Ma et al. 2013 gathered data from Yelp.com and they indicate that reviewer characteristics (i.e. experience, geographic mobility, gender) and review characteristics (review length and time interval since last review) significantly moderate the extent to which consumers’ online ratings are biased by earlier ratings. More specifically, they support the theoretical prediction that reviewers with more experience, higher geographic mobility and female raters rely less on prior reviews and thus the selection bias is less volatile to this group of participants. An interesting study conducted by Fradkin et al. 2015 and the data came from Airbnb. They showed that public reviews are informative and typically correspond with private and anonymous ratings. However, bias in reviews still occurs due to sorting, strategic reciprocity and socially induced reciprocity. By eliminating these three biases, they concluded that an additional 28% of negative experiences would have been documented in the reviews section on the Airbnb’s website.
1
Extrinsically motivated group; refers to reviews written by buyers invite via email invitations. Intrinsically motivated group; refers to reviews written by buyers without email invitations.
13
Several empirical papers have analyzed the rating distributions that arise on major review platforms and the majority of them arrived at a similar finding. The rating distributions tend to be overwhelmingly positive or differently that exists the overrating effect (Hu et al. 2009, Zervas et al. 2015, Fradkin et al. 2015). This result can be explained statistically by the J-shaped distribution that tends to be extremely asymmetric. Work by Hu et al. 2007 showed that purchasing bias2 and under-reporting-bias3 can explain the J-shaped distribution of online product reviews. Similar to Hu, Zervas et al. 2015 revealed the same overrating effect for both Airbnb and TripAdvisor properties. Also, he made a cross-listed comparison between properties that are both displayed on the same platforms and he found that 14% more properties rated 4.5 stars or above on Airbnb. One explanation for this finding is that bilateral reputation mechanisms systems create strategic considerations in feedback giving, which in turn cause underreporting of negative reviews due to fear of retaliation. Therefore, the examination of rating for products or services that belong to the “overrating group” should deal with caution as they might include upward and marginal sample of opinions.
2.3 Word-Of-Mouth Online Consumer Reviews
Online word-of-mouth communication in the form of product reviews is a major information source for consumers and marketers about product quality. However, online word of mouth activity differs from those in the real world in many aspects. Traditional (offline) word-of-mouth plays a major role in customer’s buying decision (Richins 1988). Online product reviews have become a major information source for consumers due to the fast spread of WOM communication through the Internet. Services do not have “try before you buy” or “return in case quality is below expectations” features. Therefore, WOM is particularly important in services due to the heterogeneity of service quality, the higher associated risk
2
Purchasing-bias refers to the small target group of consumers with favorable disposition towards a product purchase and thus they are more likely to write a product review.
3
Under-reporting-bias states that consumers with either positive or negative reviews are more likely to report their reviews than consumers with moderate reviews.
14
and the intangible nature of services (Bansal and Voyer et al. 2000). The positive effects of E-WOM are not only restricted to consumers that can have greater access to more information but also to marketers and managers that can implement more specific strategies (Henning-Thurau 2004).Indeed, the marketing literature suggests that consumers do pay attention to online product reviews and act upon them to make purchasing decision (Chatterjee 2001). Chen and Xie 2005 examined when and how firms adapt their strategies to online product reviews and they showed that they are two types of companies. Companies that encourage their consumers to read and write product reviews while other firms that they specialize on gathering, synthesizing and disseminating online product reviews.
2.4 Determinants of reviews and ratings
Although it is interesting to evaluate how and for what reasons the online reviews and ratings can affect the sales of a product or a service, as well as how the E-WOM can be the stepping stone for this occasion, the ultimate result is to understand how consumers choose to write a review or rate a product or service. Consumer’s intrinsic characteristics, such as cultural background or country of origin, demographic, and personally traits, act as fundamental factors influencing their ratings or reviews behavior (Gao et al. 2017).Apart from that, Ma et al. 2013 states that review characteristics (length and time between reviews) could also be critical determinants of consumers decision of choosing a review. Hong and Li 2007, found that compared to consumers from collectivistic cultures, consumers from individualistic cultures are more likely to give a rating that deviates from the average of prior consumers’ ratings. In a closely related area as culture characteristic, gender can also play a determinant role of reviews and ratings. Although non-significant gender differences derived from Mattilas study in 2000, she found that women are more likely to rate a product or service lower. Finally, consumer personality is another vital factor that affects the level of consumer satisfaction and evaluation (Gountas 2007). Among the Big Five Personality
15
Traits4, Jani and Han 2014 have indicated that extraversion agreeableness and neuroticism significantly affect customer satisfaction and thus reviews and ratings can be influenced by individual’s personality traits.
2.5 Reviews/Rating and Trust
Inarguably, consumers are relying more on online search strategies, by using blogs, forums or review sites when making product decisions (Xiang & Gretzel, 2010). However, a lot of uncertainty entails in the online transactions. Therefore, the advent of reviews and ratings over products and services has maintained a high-value relationship between consumers and businesses. While trust can be influenced by the broader context such as the industry itself or by firm level web site design features, it is often the actions of the frontline employee and the firm itself which has the most impact on building trust (Grayson et al.. 2008). Since reviews are posted most of the times by people who have purchased the product/service in question, even subjective and emotional reviews (Park, et al. 2007) provide important and useful information when they are either negative or positive. Other things being equal, reviews that are more persuasive have a great positive effect on consumers’ purchasing intention. Sirdeshmukh, Singh & Sabol 2002, define consumer trust as the expectation that a firm is dependable and will deliver on its promises.
4
16
3. Methodology
The goal of this research is to estimate whether or not people predict closer to the next actual rating of a random guest under different information sets. The source of data comes from Airbnb, an online marketplace and hospitality service that enables people to lease or rent short-term lodging including vacation rentals, apartment rentals, homestays, hostel beds or hotel rooms5. Airbnb categorizes properties into three main categories; Entire House, Private Room and Shared room. The difference between them is that only in the entire house category perspective guests rent the entire property and thus enjoy complete privacy. Due to the fact that private or shared room properties entail an interpersonal element with the host, it is unavoidable to use properties only from the Entire House Category because communication can affect the people’s review and rating behavior (Hu et al. 2008).
The “Entire House property” is the leading category on Airbnb and has in total more than 3296 properties in Amsterdam alone, the city that this paper focuses on. Also, this category consists of individual scenes to entire apartments, penthouses and villas. For having the properties of interest, no particular search filter used, only to ensure that all properties belong to the entire house category and in total 306 have appeared on Airbnb. This sample is around the 10% of the whole population in the entire house category. Two robustness steps materialized in order to ensure that these properties consist of a representative sample. The first one assures that on Airbnb the search results display a random sample of properties every time you start searching and when the properties exceed the number of 3066. Additionally, the second robustness step came by analyzing the average price of the properties chosen and an average price of 128.96€ can clearly support the three lower average price that is given7
on the platform of Airbnb for the Entire House category in Amsterdam.
5https://www.airbnb.com/ 6
The maximum number of properties that are displayed in one random search is 306
7
17
As a next step, it is important to leave outside possible outliers for eliminating properties either with really low or high price. Only 13 properties exceed the high value of the interquartile range and thus the new sample is converted to 293 entire house properties. Figure 1 displays the basic steps for finding the properties that have been displayed on the survey. Initially, the properties sample consists of 58 new properties (properties with 0-2 reviews or properties with “new” label on them) or differently 19.7%. The new properties on Airbnb are specified to properties with maximum two reviews. This is really important because the rating of a property is displayed only when a property has minimum three reviews. This change was materialized in 2014 as it seemed inappropriate or untruthful for a property to display maximum rating (5 out of 5) from only one previous renter. Moreover, an interesting result that also existed in the sample is the overrating “phenomenon” that was first observed on Airbnb from Zervas, Proserpio and Byers (2015).
Figure 1: Basic steps for finding the properties of focus (displayed on the survey)
More specifically, 126 properties out of 293 have the maximum rating, 5 out of 5 or differently 43%, Figure 2. On the other hand, 109 properties exist with ratings that are between 3.5 and 4.5 and only one property has a rating under 3.5. This
18
interesting rating behavior of renters is in line with previous literature and for the platform of Airbnb can be explained by taking into consideration the following reasons. First and most important, Airbnb uses simultaneous review systems in which reviews are hidden until both parties submit a review (“simultaneous reveal”). With this system, the retaliation power is reduced but because of the uncertainty that is hindered due to the waiting time, the ratings may accumulate overrating elements. (Dellarocas and Wood 2008, Bolton et al. 2013).
Figure 2: Rating distribution. The dotted line shows the distribution mean
Second, herding behavior which states that prior ratings subtly bias the evaluations of subsequent reviews (Salganik et al. 2006, Muchnick et al. 2013) can also explain the “rating inflated” effect that is observed on Airbnb ratings. Third, it is highly likely that some consumers are ex-ante more likely to be satisfied when participating on a sharing economy platform (Hamari et al. 2015), and this situation can be described as the self-selection effect (Li and Hitt 2008). Finally, strategic manipulation of reviews can be used by firms who seek to artificially inflate online reputations (Mayzln et al. 2004, Luca and Zervas 2016).
1 1.5 2 2.5 3 3.5 4 4.5 5 Σειρά2 0.00% 0.00% 0.00% 0.00% 0.34% 1.37% 7.17% 28.67% 43.00% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Per ce n tage Average Rating
Rating distribution
4.719
The next heuristic is to rule out properties that are new (0-2 reviews) and properties with more than 12 reviews, (Step 2, Figure 1) and thus 127 properties remaining for the final selection. The choice to focus on properties with maximum 12 reviews can be explained by the fact that with lower reviews perspective renters inside Airbnb feel more uncertain in contrast to properties with many reviews and thus longer period displayed online. Park et al. 2008, support the small review sample, as they showed that with a large volume of prior reviews there is a greater possibility for subsequent reviewers to make poor purchase decisions. Godes et al. 2012 refer to this as the Information overload. The last step includes restriction of properties that don’t offer a variation for the predictions on the survey. Therefore, properties that have the maximum rating (5 out of 5) were dismissed for the final selection. This heuristic can be explained by the zero variation that these properties offer and thus can be characterized as being highly inflated and therefore valueless. Consequently, sixty-nine properties (Step 4, Figure 1) between 3 and 12 reviews fulfilled the requirements (heuristics) for the final selection.
3.1Hypotheses
The aim of the hypotheses is to investigate the impact of average rating, price and the number of reviews on the prediction of the next rating in a sample of properties. The context for our inquiry is properties from Airbnb. There are several reasons for choosing such a research context. First, average rating plays a major role in every online accommodation platform (Zervas et al. 2015). Also, average ratings consist one of the many critical determinants of starting a transaction with a perspective product or service (Sidali et al. 2009). Second, online reviews are a form of user-generated content that is increasingly becoming an important source of information to consumers in their search, evaluation, and choice of products (Hu et al. 2014). Consequently, the number of reviews has been considered a critical factor in influencing participation on online accommodation platforms such as Airbnb because they depict the number of
20
previous renters and thus participants increase their “trust attitude” when face properties with many reviews (Gretzel and Yoo 2008). Third, given that price may play an important role in consumer’s purchasing decisions and product satisfaction (Oliver and Swan 1989, Jiang and Rosenbloom 2004, Hermann et al.2007), it is interesting to evaluate how the lack of price from properties will affect the participant’s prediction. In addition, supporting element is the study by Dodds et al. 1991 and Grewal 1995 which states that when customers faced with quality uncertainty, are more likely to use price as a signal of quality. Thus, price may have an indirect effect which can be ultimately reflected on participants predictions.
Based on the analysis above, we derive the following hypotheses:
H1: Participants predict better (closer to the actual next rating) under rating
information (current average rating) than under no rating information.
For the Hypothesis 1, ten random properties were chosen from our sample and sixty participants joined either the control or the treatment condition. From now on and for the rest of the paper the participants that joined the condition with the information of rating will refer to as “Control Rating”. On the other hand, participants that joined the condition with no information of rating will refer to as “Treatment Rating”. Also, in both conditions the participants received some basic information including; the number and the last three reviews of that property, the price and a general information on the property including a short description and a map showing the approximate location of the listing (roughly within a few hundred meters) in Amsterdam.
A highly likely consequence of hypothesis 1 is that participants in the “Control Rating” condition to predict closer to the actual next rating as the information that was included in their predictions exceed the information in the “Treatment Rating” condition. Several factors can positively support the Hypothesis 1 such as herding behavior; the average current rating works as an anchor for the participants and thus are more likely to follow the previous renter’s ratings (Lee
21
2012). Additionally, incomplete information can play a significant role in the prediction of the next rating (Mudambi and Schuff 2010). Therefore, in theory, participants that don’t have the rating information (treatment rating) are more likely to make less accurate predictions of the next rating and this argument will be analyzed in the following sections of this paper.
A representative example of one property that belongs to the hypothesis 1 and has been displayed on the survey can be seen in Appendix A.
H2: Participants predict better (closer to the next actual rating) under price
information (price of the property) than under no price information.
Similar to H1, sixty-four participants joined either the condition with the Price as additional information or not. Again, from now on and for the rest of the paper participants that joined the condition with the price will refer to as “Control Price” and the rest of them will refer to as “Treatment Price “condition. The only difference is that here these properties are different from the ones in the H1. Again, participants have to predict the next guest’s rating based on the same information as in H1, expect from the average rating that now is displayed in both the groups because the variable of interest is the next prediction based on price information or not.
To examine whether a higher number of reviews of a property creates closer to the actual next rating predictions instead of a low review, Hypothesis 3 was created, focusing on one property with both low and high number of reviews. More specifically, all the participants have to predict the next rating from a standard initial review point and to predict the next rating again from the same property but now with five additional reviews. For this property, additional information factor such as the current average rating is not included. This approach makes the predictions of the participants more accurate towards the difference between the numbers of reviews, eliminating additional information such as the current average rating that may have a direct effect on the predictions.
22 Specifically, H3 is formulated as:
H3: Participants predict better (closer to the actual next rating) with more reviews
After formulating the hypothesis including the collection of properties that were displayed on the survey, it is critical to state that for every one of the chosen property, tracking procedure was materialized in order to find the next actual rating of each one of these. However, only for the Hypothesis 3, additional five reviews had to be tracked for the selective property for testing whether or not the increase of the reviews makes the predictions more or less accurate. In Appendix B, the current average ratings of each property when the survey was distributed are analyzed alongside with their actual next ratings. Additionally, in Appendix C, the writing comments of reviews for the hypothesis 3 are provided for the purpose to understand whether or not the five additional reviews created variability in the predictions of the participants. In other words, if the five additional reviews from the starting point of seven are completely different for the ones that were displayed to participants when they made the prediction of the property with the seven reviews, the difference between the seven and twelve reviews can be explained by the difference experiences of the actual renters. However, from the Appendix C, there is not a difference in the opinions of the renters throughout their experience of that property and this fact can make valid the upcoming results concerning prediction means, errors and regressions.
23
4 Data gathering and variable definitions
4.1Survey mechanism
It is not possible to directly measure the predictions of people without specifically asking them to do so. No other study attempts to quantify and explain what type of information makes the predictions of the people more or less accurate. Two primary types of information were gathered via the survey on Qualtrics8. First, the prediction of a next guest’s rating for twenty-one selected properties under different information sets. Additional questions were included and emphasized in basic demographics elements such as age and gender. Note that the survey is anonymous, so all the demographic and the predictions of the participants are gathered via the survey.
4.2 Survey questions
Every participant was asked to predict the next guest’s rating of eleven properties between 3 and 12 reviews on a rating scale of 1 to 5, with half integers included. In other words, 9 point scale exists with participants able to make a prediction choosing one of the nine available options, with 1 being the lowest and 5 being the highest rating prediction. More precisely, all the participants answered the same question that is “Predict the next guest’s rating of the following property “. Moreover, the survey entails also questions concerning demographics. In the part of the analysis, gender and age types of questions are transformed into binary form by assigning a given variable a value of 1 if the respondent is male and 0 if the respondent is female. Regarding age, for the age group of “under 18” a value of 1 is assigned, between “18-24” a value of 2, for the age group of “25-34” a value of 3 and the age group of “34-44” and “45-54” values of 4 and 5 respectively.
8
24
5. Results
5.1 Data and summary statistics
Between 5th and 15th of May, one hundred twenty-four participants’ were surveyed and 1736 individual responses obtained regarding predictions under different information sets and demographics. Their average age was in the range of 18-24 and 53.2% of them were male. Also, the gender differences across controls and treatments conditions are analyzed, emphasizing that the randomization of the survey was efficient as the participants have small gender differences across the treatments and controls conditions. An analysis of the actual frequencies of the Age and Gender can be seen in Appendix D.
Figure 3 displays the overall frequencies and the percentages from the 1-9 rating scale choice of the predictions. 1488 predictions9 were analyzed and as can be seen, 1.005 predictions are accumulative distributed towards the rating range of 4 to 5. As a result, 67.54% of the total predictions entail inflated ratings. The distribution of the predictions is line with the Figure 2 (Rating distribution from properties on Airbnb) but here the maximum rating is less chosen from the 124 participants. However, a direct comparison cannot be materialized as Figure 3 emphasizes on the observations of the predictions from the survey whereas Figure 2 displays the actual rating of properties on Airbnb taking into consideration the average rating of all the properties that consist the initial sample of this paper.
9
25
Figure 3: Frequencies of predictions from 1488 responses
Table 1 contains reported descriptive statistics analyzed on SPSS software. The following Table specifies to the means of 60 individuals (30 in “Control rating” and 30 in “Treatment rating” condition) for H1; participants make better predictions (closer to the next actual) under rating information than under no rating information. An important measure to analyze is the mean difference in the prediction of each property (each review) between the “Control rating” and “Treatment rating” condition. In all of the cases, participants that joined the “Control Rating” condition made higher predictions. Another interesting result from Table 1 comes from analyzing the Standard deviation. Even though it’s not possible to compare the standard deviation between different properties, as the participants made estimations with different information sets, the standard deviation for nine out of ten properties in the treatment condition is higher than in the control condition. As a result, the estimations of the participants in the treatment condition are more spread out and can be explained by the more variability in their estimations due to the lack of the rating information.
1 1.5 2 2.5 3 3.5 4 4.5 5 Σειρά2 0.87% 1.75% 2.82% 3.76% 6.52% 16.60% 25.27% 23.59% 18.68% 0% 5% 10% 15% 20% 25% 30% Per ce n tages Ratings
26
Similar results can be analyzed from Table 2 but now regarding Hypothesis 2, where participants entered either the “Control Price” condition with the information of price or the “Treatment Price” with no information of price for each property. More precisely, the mean prediction for the 32 individuals that joined the “control price” condition is higher for every property than the “treatment price” condition. Regarding the standard deviation, six out of ten properties have higher standard deviation in the treatment condition and this difference can be explained again by the more spread in the predictions of the participants in the treatment condition. N Mean Std. Deviation Contr_rev3 (Treat_rev3) 32 (32) 4.29 (3.92) 0.55 (0.67) Contr_rev4 (Treat_rev4) 32 (32) 3.85 (3.43) 1.03 (0.60) Contr_rev5 (Treat_rev5) 32 (32) 4.60 (4.28) 0.48 (0.55) Contr_rev6 (Treat_rev6) 32 (32) 4.26 (3.90) 0.62 (0.62) Contr_rev7 (Treat_rev7) 32 (32) 4.32 (3.73) 0.64 (0.79) Contr_rev8 (Treat_rev8) 32 (32) 4.57 (4.37) 0.62 (0.59) Contr_rev9 (Treat_rev9) 32 (32) 4.39 (4.18) 0.54 (0.68) Contr_rev10 (Treat_rev10) 32 (32) 4.45 (4.34) 0.52 (0.57) Contr_rev11 (Treat_rev11) 32 (32) 4.01 ( 3.46) 0.83 (0.79) Contr_rev12 (Treat_rev12) 32 (32) 4.15 (3.43) 0.64 (0.70)
Table 2: Descriptive statistics for Hypothesis 2 Notes: The Control is for all the participants that joined the survey with Price information. Treatment is for all the participants that made a prediction without the information of price per property and their values are displayed within the parenthesis.
The differences in the mean for every property for both hypothesis 1 and 2 can be seen in the Figures 4 and 5. The line chart of figure 4 displays the differences
N Mean Std. Deviation Contr_rev3 (Treat_rev3) 30 (30) 3.33 (3.30) 0.68 (0.90) Contr_rev4 (Treat_rev4) 30 (30) 3.88 (3.86) 0.72 (0.98) Contr_rev5 (Treat_rev5) 30 (30) 4.20 (3.83) 0.65 (1.12) Contr_rev6 (Treat_rev6) 30 (30) 4.16 (3.96) 0.78 (1.03) Contr_rev7 (Treat_rev7) 30 (30) 2.96 (2.75) 0.86 (0.81) Contr_rev8 (Treat_rev8) 30 (30) 4.20 (3.91) 0.74 (0.94) Contr_rev9 (Treat_rev9) 30 (30) 4.45 (3.76) 0.60 (1.06) Contr_rev10 (Treat_rev10) 30 (30) 4.41 (4.03) 0.54 (0.96) Contr_rev11 (Treat_rev11) 30 (30) 4.46 ( 4.00) 0.68 (1.02) Contr_rev12 (Treat_rev12) 30 (30) 4.66 (4.01) 0.44 (1.07)
Table 1: Descriptive statistics for Hypothesis 1 Notes: The Control is for all the participants that joined the survey with Rating information. Treatment is for all the participants that made a prediction without the information of Rating per property and their values are displayed within the parenthesis
27
in the mean for the control and treatment condition with regard to rating. The steep decrease in the mean predictions for the property with 7 reviews can be explained by the negative impression that this specific property had in both conditions in the participants’ predictions. Also, the red line in Figure 4 displays the actual next ratings for each of the selected properties from online users on Airbnb. Thus, a line closer to the actual next rating line emphasizes the fact that the participants under the control condition made estimations closer to the actual ratings.
Figure 5 displays the mean differences for the “Control Price” and “Treatment Price” conditions for each property. The difference in these ten properties is that they are different from the ones in hypothesis 1. As the Table 2 showed, the mean predictions for the 32 individuals that joined the information of price are higher than the 32 individuals that joined the treatment condition with no information of price. From Figure 5, the only property with a small difference in the prediction of “control price” and “treatment price” conditions is the one with the ten reviews. For this property, the lack of information of price didn’t create high prediction differences as it’s clearly the case for all the other properties.
Figure 4: Analysis of mean for “Control Rating” and “Treatment Rating” conditions, Note: The red line shows the actual next rating for the ten selected properties of hypothesis 1.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 3 4 5 6 7 8 9 10 11 12 A ve rag e r ating p re d ic tion Number of reviews
Mean differences for Hypothesis 1
Control Rating Treatment Rating Actual_Next_Rating
28
Again, the red line depicts the actual next rating of the ten selected properties from the actual users on Airbnb. The predictions of the participants that joined the “control price” condition are closer to the red line, emphasizing that the additional information of price affected the accuracy of the participant’s predictions.
Figure 5: Analysis of mean for “Control Price” and “Treatment Price” conditions, Note: The red line
shows the actual next rating of the ten selected properties of hypothesis 2.
Regarding Hypothesis 3; participants predict better (closer to the actual next rating) with more reviews, Figure 6 is created and it unravels the difference in the mean for the same property but under different reviews. More specifically, 124 participants made a mean prediction of 3.61 for the property with 7 reviews and 3.87 for the same property but with five additional reviews. The red points show the actual next ratings for the two mentioned reviews. In other words, the 8th review was 4 and the 13th review of the same property was 4.5.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 3 4 5 6 7 8 9 10 11 12 A ver ag e rati n g p red ict ion Number of reviews
Mean differences for Hypothesis 2
Control Price Treatment Price Actual_Next_Rating
29
Figure 6: Analysis of mean from all the participants for the same property under 7 and 12 reviews, Note: The dotted red points show the actual next rating for the two different reviews.
5.2 Prediction Errors
Having in total 1240 predictions for the hypothesis 1 and 2 for a total of 20 properties, both the accuracy of the predictions, as well as whether a prediction is “good” can be analyzed. Table 3 displays the “control rating” and “treatment rating” overall differences by taking as a good prediction a rating from participants that range between plus or minus 0.5 from the actual next rating. For example, a prediction of 3.5 out of 5 for a property that its next actual rating was 4 is count as a good prediction but also a prediction of rating of 4.5 out of 5 for the same property makes the prediction good.
Moreover, from Table 3 there is also the disclosure of the percentage of participants that made exactly the same prediction as the rating of an actual renter on Airbnb for each of the properties. By comparing the two conditions over the “good prediction” columns of Table 3, higher percentages come from 8 out of the 10 properties for the control treatment and this difference can again be explained by the asymmetry of information between the conditions. Observing the number of accurate predictions of Table 3, it’s difficult to have a generalized result because predictions entail subjectivity (Gao et al. 2017). However, in five out of ten properties the participants in the treatment condition made more
1 1.5 2 2.5 3 3.5 4 4.5 5
7 reviews Actual_next_rating 12 reviews Actual_next_rating
M
e
an
R
ating
Same property Under 7 & 12 reviews
30
accurate predictions than participants in the control condition. This interesting result as described above is difficult to be interpreted in depth but it’s also important to state that the other information that was available for the treatment condition (Reviews, Price, and General Information), positively affected the participants to make good predictions and this can be explained by the number of the accurate predictions.
Table 3: This Table displays the % of the good predictions for each property between the two conditions and the % of the accurate predictions by comparing the actual next ratings from the Airbnb platform. Notes: The individual predictions are good when the actual next rating of a property minus the prediction of each individual is ±0.5.
Regarding hypothesis 2, Table 4 was created for analyzing the percentage points of good prediction but also to uncover the number of accurate predictions for both “Control Price” and “Treatment Price” conditions. Again, in eight out of ten properties participants in the control condition made more “good” predictions than in the treatment condition. This can be seen by the higher percentages for all the properties of the “good prediction” columns except from the properties with 10 and 11 reviews.
Properties “Control Rating” (n=30) “Treatment Rating” (n=30) How good the
prediction is? Difference |0.5|
Percentage of accurate
predictions
How good the prediction is? Difference |0.5| Percentage of accurate predictions Review 3 43.3% 16.6% 63.3% 23.3% Review 4 53.3% 26.6% 40% 13.3% Review 5 83.3% 23.3% 70% 26.6% Review 6 80% 33.3% 73.3% 30% Review 7 26.7% 6.6% 26.6% 6.6% Review 8 83.3% 33.3% 70% 40% Review 9 70% 40% 40% 10% Review 10 70% 23.3% 56.6% 16.6% Review 11 76.6% 40% 56.6% 16.6% Review 12 96.6% 23.3% 73.3% 33.3%
31
Properties “Control Price” (n=32) “Treatment Price” (n=32) How good the
prediction is? Difference |0.5|
Percentage of accurate
predictions
How good the prediction is? Difference |0.5| Percentage of accurate predictions Review 3 84.3% 28.1% 59.3% 28.1% Review 4 53.1% 6.25% 28.1% 3.1% Review 5 78.1% 50% 50% 25% Review 6 84.3% 21.8% 71.8% 25% Review 7 75% 9.3% 50% 18.7% Review 8 71.8% 56.2% 62.5% 28.1% Review 9 87.5% 46.8% 84.3% 43.7% Review 10 59.3% 37.5% 68.7% 21.8% Review 11 59.3% 18.7% 71.8% 31.2% Review 12 81.1% 6.25% 65.6% 18.7%
Table 4: This Table displays the % of the good predictions for each property between the two conditions and the % of the accurate predictions by comparing the actual next ratings from the Airbnb platform. Notes: The individual predictions are good when the actual next rating of a property minus the prediction of each individual is ±0.5.
After analyzing whether the predictions are “good” and more or less accurate, it’s interested to specify to the actual prediction errors that can assist the procedure of determining that the two control conditions (“control rating” and “control price”) outperform and thus have smaller prediction errors than the two treatment conditions (“treatment rating” and “treatment price”). Two different ways of measuring the prediction errors will be analyzed, and the first measure is prediction error over a property. For Table 5, the prediction error is estimated by subtracting the actual next rating with the prediction of each individual. The mean prediction error per property is derived by taking the sum of the prediction errors and divided by the number of participants in each condition. More specifically, it can be seen that in all the properties (10 out of 10) the prediction errors are higher for the “treatment rating” than the “control rating” condition. These differences between the prediction errors emphasize that the participants in the control condition outperformed in their predictions the treatment condition with regard to making predictions closer to the actual next rating and thus having lower errors. However, important is to evaluate whether or not these differences are significant. Therefore, the confidence interval of each condition under different reviews is provided also in Table 5. The confidence interval can be used
32
without the p-value because the standard deviation can be determined per its review/property. Also, the sample of hypothesis 1 is 60 (30 participants in control and 30 in treatment condition). Thus, the central limit theorem can be applied and the z-values are efficient. The confidence interval is estimated by taking per each review; the mean prediction error, the sample’s standard deviation10
as well as an assumption of a confidence level of 95% (z-value=1.96). In other words, the confidence intervals of Table 5 are derived by using the following general formula twice (one for the control and one for the treatment condition) and twenty confidence intervals are calculated.
𝜒̅
1…..10± 1.96 (
𝜎=1…10)√ 𝑛=30
)
Where 𝑥̅ is the mean prediction error for the total 10 properties. σ, is the standard deviation and 1.96 is the z-value of the confidence level of 95%.
Table 5: Mean Prediction Errors per each review/property along with their confidence intervals levels Notes: The prediction errors are derived by taking the difference between the actual next rating and the prediction of the next rating per each individual and the sum of them was divided by the total number of participants in either control or treatment condition. Also, the confidence intervals are estimated by using the z-value along with the mean and the standard deviation.
Properties “Control Rating” (n=30) “Treatment Rating” (n=30) Mean Prediction Error Confidence Intervals Mean Prediction Error Confidence Intervals Review 3 0.666 (0.416,0.916) 0.7 (0.379,1,02) Review 4* 0.616 (0.386,0.846) 1.133 (0.779,1.48) Review 5 0.3 (0.069,0.53) 0.666 (0.26,1.06) Review 6 0.333 (0.049,0.61) 0.533 (0.163,0.903) Review 7 1.03 (0.72,1.34) 1.25 (0.96,1.54) Review 8* 0.21 (-0.05,0.47) 0.58 (0.242,0.923) Review 9* 0.55 (0.34,0.76) 1.233 (0.853,1.613) Review 10* 0.583 (0.392,0.772) 0.966 (0.625,1.3) Review 11* 0.533 (0.293,0.773) 1 (0.64,1.359) Review 12* -0.16 (-0.32,0) 0.483 (0.102,0.853)
From Table 5, six out of ten properties/reviews have differences in the prediction errors that are significant. These properties can be seen from the red star that is
10
33
next to each review. To decide whether or not a difference is significant, confidence intervals used as a determinant. More specifically, a mean prediction error that is included in the other’s condition confidence interval has not significant difference. For example, the Review 4 for the Control condition has a mean prediction error of 0.616 and this value is not included in the confidence interval of the same review but in the treatment condition, which is (0.7709, 1.48). Thus, the difference in the mean prediction error between the control and the treatment is significant. Similar, significant is the difference in the mean prediction error for the Reviews 8, 9, 10, 11 and 12. As a result, for these six properties participants have significant mean prediction errors differences across condition while for the rest four, even if the mean of the errors are different and higher in the treatment condition, significant differences cannot be supported by the confidence intervals method.
The other way of measuring the prediction errors (individualistic prediction errors) is to analyze the prediction error per each individual for each property. Thus the prediction errors are estimated by taking the difference of the actual next rating and subtract it from the individual’s prediction of each property. Then, the sum of the prediction errors per each individual is divided by the total number of properties that each individual made a prediction, in this case, 10. Individuals with prediction errors close to 0 can be interpreted as predictions that were closer to the actual next rating. Figure 7 displays two “potential” histograms; one for “Treatment Rating” condition and for the “Control Rating” condition. The x-axis shows the prediction errors under intervals while the y-axis measures the percentages of these intervals for either the control or treatment condition. As can be seen, participants in the control condition have higher prediction errors in the intervals of 0.2 to 0.8, but the individualistic prediction errors are higher for the treatment condition in the more “extreme” intervals such as 1 to 1.2 and 3 to 3.2. As a result, even though in some intervals the errors between the conditions are slightly different, overall the participants that entered the treatment have higher prediction errors. Additionally, the two lines show the potential histograms per each condition. The red line (treatment condition) is less normal distributed
34
with more extreme values such as in the interval of 3-3.2 whereas the black line (control condition) is a not optimal normal distributed histogram, but it entails less extreme values.
The same procedure conducted for the hypothesis 2. Table 6 displays the prediction errors for all the participants that joined either the “Control Price” or “Treatment Price” condition and specifying to property’s prediction errors. Also the confidence intervals per each review (property) are provided. From Table 6, it can be seen that for all the properties the prediction errors are higher for the treatment condition. This means that participants in the “control price” condition made prediction closer to the actual next rating and thus their prediction errors regarding each property (each review) are lower. In order to evaluate whether these differences are significant, confidence intervals were created. Similar to the above analysis, the confidence intervals were created by using the mean
Figure 7: Individualistic prediction errors. Notes: The prediction errors are separated via different intervals and the prediction errors of each participant per each property are divided by the total predictions they made (10 properties) in order to reveal the percentages of their prediction errors. Also, the red and the black lines depict the two histograms for the treatment and the control condition, respectively. 0% 5% 10% 15% 20% 25% 30% 35% Per ce n tage o f o b ser vation s Prediction Errors
Individualistic Pred. Errors for Hypothesis 1
Treatment_Rating Control_Rating
35
prediction error per each review, the standard deviation11 and the confidence level of 95% (z-value=1.96) in which the central limit theorem can be applied as the sample consists of 32 individuals per condition. The confidence intervals of Table 6 are derived by using the following general formula twice (one for the control and one for the treatment condition) and twenty confidence intervals are calculated,
𝜒̅
1…..10± 1.96 (
𝜎=1…10√ 𝑛=32
)
Where 𝑥̅ is the mean prediction error for the total 10 properties. σ, is the standard deviation and 1.96 is the z-value of the confidence level of 95%.
Table 6: Mean Prediction Errors per each review/property along with their confidence intervals levels Notes: The prediction errors numbers derived by taking the difference between the actual next rating and the prediction of the next rating per each individual and the sum of them was divided by the total number of participants in either control or treatment condition. Also, the confidence intervals are estimated by using the z-value along with the mean and the standard deviation.
From Table 6, six properties have differences in the prediction errors that are significant. These properties can be seen from the red star that is next to each review. To decide whether or not a difference is significant, confidence intervals used as a determinant. More specifically, a mean prediction that is included in the other’s condition confidence interval has not a significant difference. From
11
The standard deviations for the “Control Price” and “Treatment Price” can be seen in Appendix F
Properties “Control Price” (n=32) “Treatment Price” (n=32) Mean Prediction Error Confidence Error Mean Prediction Error Confidence Error Review 3* 0.203 (0.013,0.393) 0.57 (0.348,0.807) Review 4* 0.64 (0.28,1) 1.06 (0.852,1.27) Review 5* 0.39 (0.22,0.56) 0.71 (0.528,0.907) Review 6* 0.234 (0.014,0.45) 0.593 (0.373,0.813) Review 7* 0.171 (-0.048,0.391) 0.765 (0.495,1.035) Review 8 0.421 (0.2,0.64) 0.625 (0.415,0.835) Review 9 0.109 (-0.081,0.299) 0.31 (0.072,0.552) Review 10 0.546 (0.36,0.72) 0.65 (0.45,0.85) Review 11* -0.01 (-0.305,0.27) 0.53 (0.26,0.8) Review 12 0.343 (0.12,0.56) 0.67 (0.43,0.91)
36
Table 6 properties with 3, 4, 5, 6, 7 and 11 reviews have significant mean prediction errors differences. For the rest, even if the mean of the errors are different and higher in the treatment condition, significant differences cannot be supported by the confidence intervals.
The individualistic prediction errors can be seen from the Figure 8. The prediction errors, in this case, are estimated as before and by subtracting the actual next rating with each individual prediction of each property. The sum of the prediction errors per participants is divided by the total number of predictions that each individual made, in this case, ten. Figure 8 displays two “potential histograms”; one for “Treatment Price” condition and the other for the “Control Price” condition. Analyzing the errors intervals, participants in the control condition have higher prediction errors in the intervals of 0.6 to 1 but the individualistic prediction errors are higher for the treatment condition in the more “extreme” intervals such as 1 to 1.2, 1.4-1.8. As a result, even though in some prediction error intervals the errors are higher in the control condition, overall the participants that entered the treatment condition have higher prediction errors. Additionally, the two lines show the potential histograms per each condition. The red line (treatment condition) is less normal distributed with more extreme values whereas the black line (control condition) is close to a normally distributed histogram and entails less extreme values.
37
For Hypothesis 3, only the prediction errors regarding different reviews for the one chosen property are analyzed and from Table 7, prediction errors are higher under 12 reviews than 7 reviews. This means that participants made less accurate prediction when the property had more reviews but the difference is rather small. Again the prediction errors were derived by taking the actual next rating and subtract it with all the individuals’ predictions for the two reviews (7 & 12). The sum of the prediction errors is divided by the total number of participants, in this case, 124 and this outcome is displayed in Table 7. Also, this difference is significant based on the confidence intervals12, as the mean prediction error for the 7 reviews is not included in the confidence interval of the 12 reviews.
12
The analysis of the confidence interval is based on the same procedure of hypothesis 1 and 2. Figure 7: Individualistic prediction errors. Notes: The prediction errors are separated via different intervals and the prediction errors of each participant per each review are divided by the total predictions they made (10 properties) in order to reveal the percentages of their prediction errors. Also, the red and the black lines depict the two histograms for the treatment and the control condition, respectively.
0% 10% 20% 30% 40% 50% 60% Per ce n tages o f o b ser vation s Prediction Erros
Individualistic Pred. Errors for Hypothesis 2
control_Price Treatment_Price
