AN OFFER YOU CAN’T REFUSE: INVESTIGATING ACCURACY OF
WILLINGNESS TO PAY FROM A DIRECT AND INDIRECT METHOD
AND AN EXPLORATION ON PAYMENT CONTEXT
Master thesis, Msc Marketing, track Management & track Intelligence
1
ABSTRACT
2
CONTENT
1. Introduction ... 3
2. Literature review ... 5
2.1 Roots of the (hypothetical) bias ... 5
2.2 Direct approach of measuring WTP: Open-Ended questioning ... 6
2.3 Indirect Methods to Measure WTP: Choice-Based Conjoint Analysis ... 7
2.4 Subscription priced vs. traditional priced ... 9
3. Methods and data collection ... 11
3.1 Data collection ... 11
3.1.1 Participant and stimulus ... 11
3.1.2 Experimental design ... 11
3.1.3 Incremental willingness to pay ... 11
3.1.4 Open ended ... 11
3.1.5 Choice Based Conjoint Analysis ... 12
3.1.6 Experimental Procedure ... 14
3.2 Data analysis ... 14
3.2.1 Dealing with incomplete responses ... 14
3.2.2 First view on the different datasets: missing values, outliers and abnormalities .... 14
3.2.3 Comparing survey demographics ... 16
3.2.4 Comparing OE subscription and traditional ... 16
3.2.5 Analysis of CBCA ... 17
3.2.6 Comparing OE and CBCA ... 17
4. Results ... 18
4.1 Comparing survey demographics ... 18
4.2 Comparing OE subscription and traditional ... 19
4.3 Analysis of CBCA ... 19
4.4 Comparing OE and CBCA ... 21
5. Conclusion and discussion ... 22
6. References ... 26
3
1. INTRODUCTION
“Everything in life has its price” wrote author Paulo Coelho (1993, p. 14) in his book the alchemist, although he was referring to a deeper level in life, every product and service you can purchase somehow has some price attached to it. But how do these prices come to be, from a tomato to a computer, what is a reasonable price? Aim too high and the tomato will rot on the supermarket shelves and the computer will become obsolete, never having left the storage as new and improved versions enter the market. Aim too low and your profits might dwindle to the point whether you can’t sustain your business and consumers might even presume your low prices as a sign of low quality. One can sense, just from this example that it is important to determine how much a consumer is willing to pay in determining pricing. Wertenbroch and Skierra (2002) define the Willingness to pay (WTP) as the top price a potential buyer would be willing to pay for a certain amount of a good or service. They reveal that the earliest attempt of capturing some form of WTP dates back to 1797. The challenge is to establish an incentive-compatible format in which you can entice the buyer to truthfully reveal his WTP or reservation price.
Literature elaborates on the importance of determining valid WTP estimates among potential consumers. Schmidt and Bijmolt (2020) even call it “the cornerstone of marketing strategy” (p. 1). As presented by Breidert, Hahsler and Reutterer (2006), without valid estimates an optimal pricing strategy cannot be developed. A second argument given is that WTP estimates can also be used to forecast how the market will behave when prices are altered which also enables the modelling of demand functions. They finally also mention the significance of WTP in relation to brand equity, where for example a brand can be valued in terms of monetary added value. And then we haven’t even discussed the importance of determining the introductory price in relation to innovation, a poorly established introductory price can ultimately mean the failure of an innovation (Ingenbleek, Frambach & Verhallen, 2013). Pricing tactics can also be based upon WTP estimates; Hofstetter, Miller, Krohmer and Zhang (2020) name nonlinear pricing, one-to-one pricing and targeted promotions as some of these potential pricing tactics.
4 when financial consequences are imposed upon the participants’ decisions which makes incentives align (Schmidt & Bijmolt, 2020). Feasibility of the real context, however, is often an issue. Taking feasibility into account, ideally, we would thus still yield the same insights with the hypothetical context.
Schmidt and Bijmolt (2020) recently found that indirect methods actually overestimate the willingness to pay significantly more than direct methods. This means that the hypothetical bias found in the WTP is larger when using indirect methods than using direct methods. This surprising finding is contrary to popular belief and could have significant impact if results are replicable beyond their meta-analyses. Replicability would entail that with a direct method which is ultimately associated with lower cost and effort you could get more accurate results than an indirect method. This will ultimately benefit researchers and managers in their search for WTP, as Steiner and Hendus (2012) find that 76% of firms they surveyed use a direct approach and none with incentives aligned methods. We also find indicative signals in Voelckner (2006) that they don’t find clear evidence on whether directly stated WTP or through conjoint analysis elicited WTP yield the higher WTP. Additionally, Schmidt and Bijmolt (2020) find a greater hypothetical bias for higher valued products.
In this research paper we will primarily compare the results of a WTP elicited from an indirect and a direct method in a hypothetical setting to establish whether Schmidt and Bijmolt (2020) findings are replicable beyond the meta-analyses. Without establishing the share of hypothetical bias but assuming an inflated WTP, this would entail that the hypothetical WTP found by the direct method would be lower than the WTP found in the indirect hypothetical method.
5 subscription WTP with another method but also with another payment type. If indeed a difference is found based on context, this would give pricing managers a first piece of information in their consideration what type of payment to apply. Table 1 gives a first overview of both the methods and the context we will compare.
TABLE 1
Overview of Methods and Contexts Applied and Compared
Context
Subscription priced goods Traditional priced goods
Type of measurement
Direct method Open-ended questioning Open-ended questioning Indirect method Choice-based conjoint
The remainder of this paper is organized as follows. More details on the methods will follow in section 2 which provides a theoretical background on the bias associated with hypothetical WTP, benefits and challenges of the –in Table 1 introduced- direct and indirect approach and explorations on a subscription and traditional context. With the remaining research questions from section 2 we will elaborate on the method and data collection in section 3. Section 4 then provides results which are then discussed in section 5: the conclusion and discussion.
2. LITERATURE REVIEW
2.1 Roots of the Hypothetical BiasThe difference in WTP found between the real and hypothetical context is the hypothetical bias (Harrison & Rutström, 2008). The financial consequences in a real context make sure that incentives are aligned, and a real context would for that matter be preferred. In practice however, the real context is often not feasible when taking into account potential prototypes, privacy concerns, legal restrictions and costs associated with establishing the real context to elicit WTP estimates (Hofstetter et al., 2020).
6 find this more so for contingent valuation (Johannesson et al., 1998) which is the valuation of non-market resources such as the air quality. List and Gallet (2001, p. 243) propose that “the average person seems to exaggerate his or her actual WTP across a broad spectrum of goods with vastly different experimental protocol”.
Literature has come up with some factors that influence the hypothetical bias we find, such as the type of goods. Schmidt and Bijmolt (2020) show that the type of private good has an impact on the hypothetical bias, they formulate it as such that products demanding a strong search effort increase the hypothetical bias. Hofstetter et al. (2020) show that for lower valued goods the hypothetical bias is smaller and for higher appreciated goods the bias tends to be higher. Reasoning might be that the higher appreciated durable goods are simply less purchased and therefore price is less remembered (Estelami, Lehmann, & Holden, 2001). Hofstetter et al. (2020) make the distinction that novel or harder to obtain reference priced goods also may yield a higher bias. Lack of familiarity with the goods increases the likelihood of susceptibility to question format induced biases.
While literature often establishes that a hypothetical bias exists, and some factors are recognised to play a role, a more satisfactory overview of what would influence the size and direction of this bias remains to be seen. When researching characteristics and motives influencing the hypothetical bias of innovative products, Hofstetter (2013, p. 1051) even has to conclude that “biases can come from unlikely sources that run counter to intuition”. As he ends with a call out for replication Schmidt and Bijmolt (2020) were unable to do so. The hypothetical bias thus remains a volatile concept and without certainty we must sometimes make assumptions until we are able to extent our knowledge base.
2.2 Direct Approach of Measuring WTP: Open-Ended Questioning
7 On the one hand, the OE approach succeeds in eliciting the relationship of WTP to preceding preferences and with regards to factors, such as income, that are revealing of the ability to pay (Bateman, Willis & Garrod, 1994). Miller, Hofstetter, Krohmer and Zhang (2011) find indication that with the generated hypothetical bias; the open-ended (OE) format can still lead to the correct demand curves and pricing decisions. Hofstetter et al. (2020) also show that even though the WTP tends to be inflated in the OE-format, through de-biasing statistically, managerially valid predictions of consumers’ WTP can be made. This rigorous debiasing method is a first attempt and based on the assumption that consumers tend to overstate their WTP in a hypothetical setting. For data collection and analysis the OE approach is conceptually clear and effortless to implement, resulting in quick results while also being the economical option (Jedidi & Jagpal, 2009). Hofstetter et al. (2020) point out that with advances in digital technology, nowadays the ability to pursue massive online data collection speaks to the advantages of the direct approach even more.
On the other hand, the OE approach does not fully acknowledge possible differences in reference frames (Jordan & Islam, 2006). Asking how important price is relative to other attributes, for instance, will be less meaningful if one individual believes the range to be €8 to 12 while the other individual believes it to be €12 to 18. Respondents in theory can have an undetected distribution of reference ranges for the attributes. Kalish and Nelson (1991), also find that directly questioning the WTP does not live up to the level of robustness of respondents’ involvement associated with ranks or ratings. Meaning that the likelihood of errors in measurement by the respondent or researcher increases and in turn its associated effect on estimated product evaluations is more noticeable. In the specific case of contingent valuation, a hypothetical open-ended elicitation of the WTP cannot even work. Asking someone what they would pay for a non-market resource would yield one cent for a rational decision maker (Voelckner, 2006), because why would you pay for the environment which you can make use of all the same without paying. In such a case a hypothetical setting is the only context that could work.
2.3 Indirect Methods to Measure WTP: Choice-Based Conjoint Analysis
8 (Simon, 2018; Schmidt & Bijmolt, 2020). CBCA is primarily used to establish the preference effect of innovations and improving existing achievements (Gustafsson, Herrmann & Huber, 2000). In particular, it is designed to “determine trade-offs among product features or attributes, including price” (Wertenbroch & Skiera, 2002, p. 229). Additionally, it is also used in the field of pricing policies, advertising and distributions. Through utility estimates one can acquire the hypothetical WTP of not just one static product but of different versions through choices made by the respondent on different levels of attribute combinations. Taken apart this method also comes with both benefits and pitfalls.
On the one hand, CBCA has been used as it is supposed to closely mimic the actual shopping experience and thus the most accurate WTP (Breidert et al., 2006). It can give us extensive insights into how different levels of attributes of a product or service are appreciated by the (potential) consumer. It gives us more information on the WTP for different product attributes (Hofstetter et al., 2020). It is also argued to be cognitively easier whether a certain price for a product is acceptable rather than having to assign a price value (Brown, Champ, Bishop, & McCollum, 1996). Murphy et al. (2005) state that this method is associated with less hypothetical bias and CBCA even with hypothetical bias can still lead to correct demand curves (Miller et al., 2011).
9 utility is not influenced by its price. By treating it as an attribute, utilities are estimated for the coherently shown price.
Second, practical problems are presented in the violation of the additive-compensatory model. By mixing the different levels of attributes with one another, unrealistic good or bad deals can be presented. This could then impair the comparison with other profiles. Practical problems also can be found in the form of the price effect, range effect and the number-of-levels effect. When the number of attributes grows; the importance of price lowers artificially causing the price effect. The range in levels of the attribute in turn can also influence the importance of price, the range effect translates itself in a high importance for the price attribute with a large range of levels as well as the other way around. Even stronger than the range effect is the number-of-levels effect (Verlegh, Schifferstein & Wittink, 2002) which can also artificially increase the price’s importance by, as its name suggests, the number of levels.
Finally, it is important to note that the preference structure is usually estimated at an aggregate level. It can be used at an individual level as well, but this requires a large amount of data points, which are typically not available. Adding Bayesian estimation techniques, however, make individual level estimates possible (Breidert et al., 2006).
Reflecting on both methods we find that in the past we should expect to find the hypothetical bias to be smaller in CBCA -elicited WTP than a OE-elicited WTP. However, most recent findings from Schmidt and Bijmolt (2020) suggests OE-elicited WTP to hold a smaller hypothetical bias. As status quo we expect the hypothetical bias to be caused by an inflated WTP, following this expectation we formulate the following hypothesis below:
Hypothesis 1: The hypothetical WTP for OE is lower than for CBCA
2.4 Subscription Priced vs. Traditional Priced
10 for different products such as flowers and meal boxes up to razors, watches and underwear (Bischof, Boettger & Rudolph, 2020). Similarly, a phone service post-paid subscription plan has become the norm whereas prepaid is more or less a dying breed.
Schmidt and Bijmolt (2020) find that hypothetical bias is larger for higher valued products, meaning that the hypothetical WTP found for higher valued product is found to be relatively larger than the hypothetical WTP for lower valued goods. We wonder if, in that sense or even more general, the perception of value of subscription and traditional priced goods may yield different hypothetical WTPs.
Following this explorative direction of value perception, we present the concept of “subscription fatigue” (Warrillow, 2015); in which people grow tired of seeing amounts withdrawn from their creditcard which, although perhaps individually of a small amount, together grow to be quite a sum. Once they are in a subscription oversaturated state there’s a strong correlation between feeling fatigued and the desire to reduce subscription services (Loucks, 2020). This desire to reduce subscription services might decrease perceived value of a good that is sold in a subscription setting. Additionally, one could argue that the cognitive load of having to commit oneself to a subscription and having to estimate one’s financial capabilities for an extended time in a similar sense to subscription fatigue makes people less willing to commit to what could be described as a reasonable price. Similarly, this argument could be used in favour of a relatively higher WTP, if one simply does not take on the cognitive load of thinking ahead, they could overestimate what would be a reasonable price to pay every month.
As this discussed area is as of yet poorly investigated, we then in an explorative manner formulate the following non-directional hypothesis:
11
3. METHODS AND DATA COLLECTION
3.1 Data Collection3.1.1 Participant and stimulus. Data was collected online; for the direct method we used
surveys spread via Qualtrics. Data from the indirect method was partly collected by Qualtrics and partly via conjoint.ly. In total 30.000 customers of telcom provider Vodafone were approached, 10.000 for each survey. Participation was motivated by the statement that it would help VodafoneZiggo to improve its services. The stimulus used was access to the 5G network which at the time of data collection was still very upcoming as the majority of phones was not “5G-ready” and it was expected that the full extent of the network’s potential would only be available about 2 years later.
3.1.2 Experimental design. Each survey started with a question whether the participant was
familiar with the added value of 5G, after which a small explanation of 5G and its benefits and capabilities followed. To ensure that the explanation was read, we asked two questions referring to the explanation. As 5G could be considered an innovation we followed with a question to determine what segment of technology adoption (Rogers, 1995) to allocate the respondent to. Segments ranked from most innovative to least innovative are then innovators, early adopters, early majority, late majority and finally laggards. Before questions regarding 5G were presented, each participant was asked to imagine that they had a phone which was “5G-ready” in order to place all participants in a position that 5G would have been a plausible purchase. We ended each survey with demographics to ensure respondents in all three surveys were comparable. Additionally, we were able to attach 3 variables related to the respondent’s personal phone plan with Vodafone to all responses.
3.1.3 Incremental willingness to pay. In order to be able to compare findings from the direct
format subscription priced survey with both the traditional priced survey and the CBCA survey we have used incremental willingness to pay for 5G as compared to 4G. For the OE question, in both the subscription and traditional survey, we asked the respondent how much more they would be willing to pay for 5G. In the conjoint analysis we were able to calculate this through estimates provided by the model.
3.1.4 Open ended. For the OE questioning format, we started off with prior mentioned
12 TABLE 1
Incremental WTP Questions in Direct Formats
Survey Questioning Incremental WTP
Subscription “Imagine your current phone is able to use the 5G network if this is not currently the case already. How much, with your current phone plan would you be willing to pay extra each month to be able to use 5G?” Traditional “Imagine your current phone is able to use the 5G network if this is not
currently the case already. How much, with your current phone plan, would you be willing to pay now in order to turn 5G on for a duration of 2 years?”
By having kept all questions prior and post to the incremental WTP question consistent in both scenario’s we expected no differential influence to occur beforehand. As we have collected data of their current phone plan we were able to check that both groups are comparable on that aspect as well. By having stated that their phone could be used to access the 5G network even if this was at the time of questioning not the case we have taken away this uncertainty. It was only at the incremental WTP question where we have changed the payment tactic and nothing else. Rationally, both the traditional payment and subscription offered the same as the prior mentioned a 2-year durability and the subscription as well. Therefore, we were able to compare them without a rational value difference.
3.1.5 Choice Based Conjoint Analysis. Within the conjoint category we applied the
13 attributes and corresponding levels. Apart from the discussed attributes, products were told to be identical on all other elements (Mugera, Burton & Downsborough, 2017).
5G Within Vodafone was offered as an add-on for €2,-/ month for smaller plans and standard included in larger plans. For the price attribute we stayed close to the actual range of what Vodafone offered but created equal gaps in order to ensure calculations were possible later on. By staying close to the offered range, we hoped to avoid the ad hoc nature of the price range effect (Miller et al., 2011). All attributes and levels were prior researched by Vodafone. By staying close to the attributes and levels that Vodafone was offering we ensured that they could be considered realistic as people are currently willing to purchase the given options. It was mentioned in the survey that some of the options proposed through the conjoint study might be less realistic options, as we didn’t exclude options. Given that these days not having a phone plan is unrealistic for most consumers; it was stressed that the none-option would simply mean that the respondent would rather go back out to the market to find another phone plan. Having the none-option would otherwise be quite an unrealistic option. It could then in turn give unrealistic outcomes in the chosen phone plans. The story setting of the CBCA group was that they were ready to pick a new phone plan in a full postpaid setting in an online shop “right there”.
TABLE 3
Attributes and levels used in CBCA analysis
Attributes Levels Number of attribute levels
Minutes/ texts 0, 150, unlimited 3
Data 2, 4, 6, 8, 10 5
Network 4G, 5G 2
Price/ month for 2 years 15; 17,50; 20; 22,50; 25; 27,50 6
14 Consumer response can also be affected number of attributes and levels presented, as well as overall use of language and design of the survey (Carlsson, 2010). It was noted that the possible relative unfamiliarity with 5G may have made the outcomes more prone to anomalies. We tried to adhere as much to the in-shop setting of the used provider Vodafone and explain the 5G attribute at the start of the survey. In an attempt to lower the effect of a hypothetical bias (Sichtmann et al., 2011), respondents were reminded of their personal budget.
3.1.6 Experimental procedure. All questionnaires were structured according to a description,
the WTP task and background questions on demographics and phone plan information of each respondent is attached. The OE procedure received a description of the product and the CBCA procedure received a description of the different attributes. Similar to Miller et al. (2011) we ensure that the WTP abstraction method was clear to the respondents.
3.2 Data Analysis
3.2.1 Dealing with incomplete responses. As we export all datasets from Qualtrics and
Conjoint.ly we exclude all respondents who have not finished the survey. There are multiple approaches on how to deal with missing data, our surveys, however, all are structured with ‘forced responses’, entailing that a question cannot be skipped. So missing data is rooted only in surveys that were not finished. By excluding those responses who did not finish the survey we must be mindful not to introduce systematic bias (Kitchenham & Pfleeger, 2003). Investigating the responses, we find no specific pattern in where respondents have left the survey and thus find little reason to believe that systematic bias is introduced. Another method of dealing with missing data; is imputing. This could be a viable method with few missing datapoints, in this case it is inappropriate since we are dealing with respondent drop-out numbers of around 20%.
3.2.2 First view on the different datasets: missing values, outliers and abnormalities. Before
15 missing values in the dataset as omitting these observations would take a large chunk of valuable information gained from the actual survey. Using state of the art methods to impute the missing values is not necessary since we are mostly interested in the distribution of answers for each survey. In this sense the missing values take part in the distribution and show us information whether the amount of missing values is similar in all groups. Our only non-categorical variable in this dataset is age, which is continuous. Through a boxplot we find a few extreme outliers which can be removed as common sense for instance denotes that an age of zero cannot be the true age of a respondent.
The second dataset includes the WTP for the two open-ended surveys. As the open format of this survey question allowed for it to be left without a numerical answer; missing values and unusable character answers occur. Since the number of missing values and unusable answers are small; we delete these missing values. In boxplots of the WTP, shown in Figure A10 in Appendix A, in both the subscription format and the traditional format we see quite a few outliers. As a large number of respondents stated that they would be willing to pay nothing more for 5G a lot of weight in the distribution is here in both survey formats. If we remake the boxplots without the zero-values we only take into account those who would be willing to pay more and the weight shifts up which result in fewer outliers. There are different approaches on how to deal with outliers, since we are comparing methods and are interested in all results yielded, we can argue to keep them in. Common sense tells us, however, that some of the higher stated amounts are too far from reasonable. In order to be thorough and consider all options we make different versions of the dataset with outliers kept in or out and with zero values included or dropped. The latter distinction is made as on the one hand all respondents are weighed when calculating the incremental willingness to pay through the conjoint survey but we also see that in similar approaches exclusion of small or in our case non-positive values is also done (Lamiraud, Oxoby & Donaldson, 2016). Note that all outliers are based on the dataset with dropped zeros as including the zeros would shift the weight in a manner that leaves all reasonable values to become outliers as well. In method comparison we can then establish its effect and strengthen the argument whether to include or omit the outliers an our comparison.
16
3.2.3 Comparing survey demographics. As we are comparing methods, we want to ensure that
the group of respondents in all three surveys do not significantly differ from one another. If a significant difference is found between the groups; any results yielded from comparing the methods could be explained not just by the different methods but also by the difference in the groups. We compare the groups on demographic-related questions but also on a few questions we deemed relevant to the specific product of which we are eliciting the WTP. Very demographic related background variables are age, gender, level of education and living situation. Variables we want to compare which we deem relevant in relation to the incremental willingness to pay for 5G are the ‘level of innovativeness’ in the sense of how quickly one adopts something new compared to others and whether one prior to the survey was aware of the difference between the 4G and the 5G network. Additional variables attached are the respondent’s current price plan in descriptive format and the data- and voice allowance each month.
After a first visual comparison of the different variables, statistical analysis was done on the variables. Age is a continuous variable but did not meet normality assumptions, which is visually displayed in Figure A11-A13 in Appendix A and tested through the Shapiro-Wilk test for which significance entails we cannot assume a normal distribution (w=0.97-0.98, p<0.001). Taking into account nonnormality and the need to be compare between three groups, we used the Kruskal Wallis test. Before proceeding, we tested for homogeneity of variances, this is a prerequisite in order to run the Kruskall Wallis test. Through the Bartlett test (𝑥2 (df. = 2) =
1.07; p > 0.10) we found that this assumption had been met. The other categorical variables are compared through a Pearson chi-squared test.
3.2.4 Comparing OE subscription and traditional. In order to be able to compare values found
17
3.2.5 Analysis of CBCA. In order to yield the incremental WTP through CBCA we first
analysed the data through a Multinomial logit (MNL) model, which is a preferred model to analyse choice data (McFadden, 1974). This model is based upon the assumption that coefficient β is static across respondents. Choosing alternative 𝒊 from a choice set with 𝑱 alternatives is represented by the MNL model in terms of choice probabilities 𝒑:
𝑝(𝑖|𝐽) = "#$ ('!)
∑#"$%"#$ ('") (1)
𝑉𝑖 refers to the systematic utility for product 𝑖 which is the sum of part-worth utilities:
𝑉𝑖=∑𝐾𝑘=1𝛽𝑘 𝑥𝑖𝑘 (2)
With 𝑘 being the number of attributes, 𝑥 a dummy indicating the specific attribute level of product 𝑖 and 𝛽 the part-worth utility for attribute 𝑘.
Our choice data consists of zeros and (minus) one’s, a one implying choice of the specific variable whereas zero is the counterpart of not being chosen and minus one’s are effect coded to imply the reference level. As each row is an observation belonging to a category, rather than each row containing all the choices of on respondent; our format is considered long. We then created a multinomial logit model which produces estimates by maximum likelihood. We made an initial model (Ml1) with price as a partworth variable. Additionally, we made a second model where we included price as a linear variable. If a significant difference between the two was found it would entail that we lose a significant amount of explanatory power by including price as a linear variable rather than a part-worth price variable. Incremental WTP was calculated through dividing the estimate of the network 5G variable by the estimate of the price variable as shown in the formula below.
𝑊𝑇𝑃/012345 67 = 8&'()*+, ./
80+!1' (3)
3.2.6 Comparing OE and CBCA. Since both our incremental WTP elicited from the OE survey
18
4. RESULTS
4.1 Comparing Survey Demographics
Before we can answer both hypothesis 1 and 2 we need to compare survey demographics. We start by visually comparing the groups on all relevant variables. Age as a continuous variable is compared through a boxplot. All other variables are categorical variables shown as factors in R. The categorical variables in turn can be visually compared through bar charts. Visually the three groups look very similar on all variables as can be seen in throughout Figure A1-A9 in Appendix A. In Table 4 below outcomes are shown of comparing the variables between the three survey groups.
Running the Kruskall Wallis returns insignificant results, which entails that we cannot reject the H0, there is no significant difference in age between the three groups. All other background variables are categorical, here we apply the Pearson chi-squared test, although data allowance could be considered marginally significant, all other tests yield insignificant results and thus for the categorical variables we also cannot reject H0 and conclude that there is no significant difference between the distributions.
Table 4
Comparing Demographics Variables Among the Three Datasets
Variable Form Test 𝒙𝟐 df p-value
Age continuous Kruskal Wallis 3.19 2 0.20
Gender categorical Pearson 3.32 4 0.51
Education 6.95 14 0.94 Living situation 7.92 10 0.64 Innovativeness 7.73 8 0.46 5G knowledge 2.81 4 0.59 Data allowance 38.38 26 0.06. Priceplan 37.83 34 0.30 Voice allowance 10.12 8 0.26
19
4.2 Comparing OE Subscription and Traditional
In Table 5 results are shown of comparison through the Mann Whitney U test, in a search for an answer to our second hypothesis. Looking at the mean values we can see the true effect of both the zeros and outliers on the mean value. The mean subscription increases by about 4 times when zeros are removed. When outliers are also removed the mean takes on about half the size, we see a similar trend when inspecting the mean of the traditional context. We find no significant difference when we use the test on the original dataset. When outliers are removed the p-value grows smaller but does not reach significance or only marginal if you would accept
𝛼 = 0.10, which in practices is less common. When the zero values are dropped, however, both datasets yield a highly significant test statistic, meaning that values found from the subscription survey differ significantly from the traditional survey. We can thus state that, if respondents are willing to pay more; they are willing to pay a significantly higher amount in the subscription context than in the traditional context.
TABLE 5
Comparing Elicited WTP Found in the Subscription and Traditional Format
Dataset Mean Subscription N Subscription Mean Traditional N Traditional W P-value Original 1.54 379 0.46 361 66970 0.52 Original: outliers removed 0.60 361 0.31 354 60436 0.09. Nonzero 6.85 85 1.58 104 5438 0.00*** Nonzero: outliers removed 3.22 67 1.14 97 8838 0.00***
Notes: Significance levels are as follows: . p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001
4.3 Analysis of CBCA
20 TABLE 6
Multinomial Logit Models from Choice Based Conjoint
Multinomial Logit Model Parameters Log likelihood P-value
Ml0: chance 0 -4325.24
Ml1: price partworth 13 -3394.50
Ml2: price linear 9 -3396.40 0.42
Notes: Significance levels are as follows: . p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001
The incremental WTP for the 5G network is calculated from estimates shown in Table 7 and through formula 3 which yields an incremental WTP of €4,09 per month.
TABLE 7
Estimates of Ml2
Coefficients Ml2 Estimate Std. Error z-value Pr(>|z|)
None Option 1.80 0.14 -12.45 0.00 *** Voice: 0 minutes -1.23 0.06 -21.45 0.00 *** Voice: 150 minutes 0.32 0.04 8.09 0.00 *** Voice: unlimited 0.90 0.04 23.06 0.00 *** Data: 2GB -0.90 0.07 -12.68 0.00 *** Data: 4GB -0.11 0.06 -1.91 0.06 . Data: 6GB 0.18 0.05 3.34 0.00*** Data: 8GB 0.30 0.05 5.57 0.00*** Data: 10GB 0.53 0.05 10.31 0.00*** Network 4G 0.60 0.06 10.49 0.00*** Network 5G 0.60 0.06 10.62 0.00*** Price -0.15 0.01 -21.15 0.00***
21
4.4 Comparing OE and CBCA
Finally, in Table 8 we find the outcomes of the Mann Whitney U test conducted on all four created versions of the OE subscription survey to answer hypothesis 1. The version where we exclude zero results and outliers is only marginally significant. All other comparisons yield statistically significant results meaning that the values from both methods are considered not to be the same. Directions are mixed, when we look at the nonzero mean of OE we find this to be significantly higher than the CBCA mean. For both original datasets the mean of OE is smaller than the mean CBCA. The comparisons with the original versions are in line with our hypothesis.
TABLE 8
Difference Between Incremental WTP Elicited From the Open-Ended Surveys and CBCA
Dataset Mean OE N Subscription Mean CBCA N CBCA W P-value Original 1.54 379 4.09 260 12220 0.00*** Original: outliers removed 0.60 361 4.09 260 7540 0.00*** Nonzero 6.85 85 4.09 260 12220 0.03** Nonzero: outliers removed 3.22 67 4.09 260 7540 0.08.
22
5. CONCLUSION AND DISCUSSION
In this study we investigate whether the hypothetical WTP for OE is lower than CBCA based on a controversial finding of Schmidt and Bijmolt (2020) and we explore the hypothetical WTP in a subscription and traditional context. Literature has praised the indirect method of CBCA for closely mimicking actual shopping experience and giving supposedly the most accurate WTP (Breidert et al., 2006). When comparing it to a direct method such as OE it is also supposedly cognitively easier to determine whether a certain price for a product is acceptable than having to assign a price value (Brown et al., 1996). The major downside of CBCA and other indirect methods is the amount of effort and inherently costs associated with the data collection and proficiency in analysis (Hofstetter et al., 2020). In practice Steiner and Hendus (2012) find that these costs lead 76% of the firms they asked to resort to a direct approach. As mentioned, the feasibility of incentive aligned approaches is low and Steiner and Hendus (2012) are also unable to find incentive aligned approaches in practice. Given this gap between what is preferred in theory and what is actually done in practice the finding of Schmidt and Bijmolt (2020) deserves further research. They found that indirect methods actually overestimate the real willingness to pay significantly more than direct methods. This entails that the hypothetical bias found in indirect methods is larger when using indirect methods than using direct methods. If replicable this would justify the usage of an easier and cheaper direct method over an indirect method as a source of decision making. Results show mixed directions so we cannot fully accept nor reject hypothesis 1.
Additionally, we took a first step in exploring possible differences in WTP found in a subscription context and a traditional context. This was based on the notion that higher valued goods are shown to be related to a higher hypothetical bias (Schmidt & Bijmolt, 2020), which stimulated the question whether context would influence the value perception and thus yield a different WTP due to difference in bias. This is relevant to investigate as payment methods have developed over the past years and where a traditional one-time payment used to be the standard now a subscription is a very common option. If a difference is found with context as the only varying factor this could be related to a difference in value perception and would be an avenue of further research. Results show that in the circumstance when people are willing to pay more, they are willing to pay a significantly higher amount in the subscription context than in the traditional context, in line with hypothesis 2.
23 partiality has its roots in the different datasets we created for the OE-elicited WTP. As there are different methods and perceptions on how to deal with outliers, we decided early on to make separate datasets based on the treatment of outliers. Common sense could denote some outlier values as unlikely to be realistic, however, they could also contain valuable information. Since we are comparing methods and are interested in how well values are captured, we decide to analyze both scenarios of inclusion and seclusion of outliers. Additionally, we find that the OE-method yields a substantial amount of ‘zero’ answers, meaning that people are not willing to pay more for use of the 5G network. For similar reasoning we also separate the dataset based on the inclusion and seclusion of zero-values. Through comparison with CBCA we find a significant difference that applies to all OE-datasets except where both the zeros and outliers are removed, we refrain from further interpreting the marginally significant difference. When comparing the nonzero dataset, we find that the OE-elicited WTP is significantly higher (P<0.01) than the CBCA-elicited WTP. This finding is in contrast with our hypothesis. We can, however, see that when we take the weight of the zeros away; the outliers shift the weight upward to a large extent. On the one hand we said these outliers to with common sense could be interpreted as unrealistic so we might downplay this finding. However, captured by the method and possibly interpreted as more realistic by others also pursuing WTP findings from an OE-method we cannot definitively disregard this. We find most compelling significant difference on comparison of the original dataset and that same dataset with the outliers removed and the CBCA-elicited WTP. Here both OE-mean values are significantly lower (P<0.001) than the mean of CBCA. This is in in line with our hypothesis and the findings of Schmidt and Bijmolt (2020). It is a first indication of replicability. In favor of this interpretation as opposed to the exclusion of zeros is the fact that the OE-method captured the zeros and that this as plain method comparison thus could be argued to be included. In the CBCA elicitation people were given a choice to select none of the options presented to them, so even here people had the choice to refrain from 5G. However, this would then entail that they would not only be willing to pay the offered amounts for 5G but also for the amount of data and minutes/ texts. Perhaps this is then a matter where the incremental WTP from the CBCA gives too much utility to an attribute level. This in turn would still not speak in favor of this method.
24 the subscription context. This is not confirmed in the original context and only a marginally significant difference is found when outliers are removed, which again, we will refrain from further interpretation. When zeros are removed, we find highly significant differences between both contexts. If respondents are willing to pay more; they are willing to pay a significantly higher amount in the subscription context than in the traditional context. We think the most distinctive significant difference is the one where both zeros and outliers are removed. Even without the outliers that seem to have a large effect especially within the subscription context the value of the subscription context remains significantly higher, it takes on a value of almost triple the traditional context. Now the question remains whether the difference is purely a function of hypothetical bias; increasing the mean WTP found in the subscription setting or whether the real WTP is actually increased in this context.
25 combination with the limitation that we cannot determine the share of hypothetical bias we cannot be sure what this difference entails.
26
6. REFERENCES
Bateman, I.,Willis, K. & Garrod, G. 1994. Consistency between contingent valuation estimates: a comparison of two studies of UK national parks, Regional Studies, 28: 457-474. Bischof, S. F., Boettger, T. M., & Rudolph, T. 2020. Curated subscription commerce: a theoretical conceptualization. Journal of Retailing and Consumer Services, 54: 101822. Breidert, C., Hahsler, M., & Reutterer, T. 2006. A review of methods for measuring willingness-to-pay. Innovative Marketing, 2(4): 8-32.
Brown, T. C., Champ, P. A., Bishop, R. C., & McCollum, D. W. 1996. Which response format reveals the truth about donations to a public good?. Land Economics, 72: 152-166.
Carlsson, F. 2010. Design of stated preference surveys: is there more to learn from behavioral economics?. Environmental and Resource Economics, 46(2): 167-177.
Coelho, P. 1993. The alchemist: a story about following your dream. New York: Harper-San-Fransisco.
Davidovici-Nora, M. 2013. Innovation in business models in the video game industry: free-to-play or the gaming experience as a service. The Computer Games Journal, 2(3): 22-51. Diamond, P. A., & Hausman, J. A. 1994. Contingent valuation: is some number better than no number?. Journal of economic perspectives, 8(4): 45-64.
Donaldson, C., Thomas, R., & Torgerson, D. J. 1997. Validity of open-ended and payment scale approaches to eliciting willingness to pay. Applied Economics, 29(1): 79-84.
Estelami, H., Lehmann, D. R., & Holden, A. C. 2001. Macro-economic determinants of consumer price knowledge: a meta-analysis of four decades of research. International Journal
27 Green, P. E., Krieger, A. M., & Wind, Y. 2001. Thirty years of conjoint analysis: reflections and prospects. Interfaces, 31(3_supplement): 56-73.
Grunert, K. G., Juhl, H. J., Esbjerg, L., Jensen, B. B., Bech-Larsen, T., Brunsø, K., & Madsen, C. Ø. 2009. Comparing methods for measuring consumer willingness to pay for a basic and an improved ready made soup product. Food Quality and Preference, 20(8): 607-619.
Gustafsson A., Herrmann A., Huber F. 2000. Conjoint analysis as an instrument of market research practice. In A. Gustafson, A. Herrmann, & F.Huber (Eds.), Conjoint Measurement:
Methods and Application: 5–45. Berlin: Springer.
Harrison, G. W., & Rutström, E. E. 2008. Experimental evidence on the existence of hypothetical bias in value elicitation methods. In V. L. Smith (Eds.), Handbook of
experimental economics results, 1: 752-767. New York: Elsevier Science.
Hofstetter, R., Miller, K. M., Krohmer, H., & Zhang, Z. J. 2020. A de-biased direct question approach to measuring consumers' willingness to pay. International Journal of Research in
Marketing, pp. https://doi.org/10.1016/j.ijresmar.2020.04.006
Huber, J., & Zwerina, K. 1996. The importance of utility balance in efficient choice designs. Journal of Marketing research, 33(3): 307-317.
Ingenbleek, P. T., Frambach, R. T., & Verhallen, T. M. 2013. Best practices for new product pricing: impact on market performance and price level under different conditions. Journal of
Product Innovation Management, 30(3): 560-573.
Johannesson, M., Liljas, B., & Johansson, P.O. 1998. An experimental comparison of dichotomous choice contingent valuation questions and real purchase decisions. Applied
Economics, 30: 643-647
28 Kalish, S., & Nelson, P. 1991. A comparison of ranking, rating and reservation price measurement in conjoint analysis. Marketing Letters, 2(4): 327-335.
Kitchenham, B., & Pfleeger, S. L. 2003. Principles of survey research part 6: data analysis. ACM SIGSOFT Software Engineering Notes, 28(2): 24-27.
Lamiraud, K., Oxoby, R., & Donaldson, C. 2016. Incremental willingness to pay: a theoretical and empirical exposition. Theory and Decision, 80(1): 101-123.
List, J. A., & Gallet, C. A. 2001. What experimental protocol influence disparities between actual and hypothetical stated values?. Environmental and resource economics, 20(3): 241-254.
Little, J., & Berrens, R. 2004. Explaining disparities between actual and hypothetical stated values: further investigation using meta-analysis. Economics Bulletin, 3(6): 1-13.
Loucks, J. 2020. Digital media trends survey, Deloitte Insights Digital Media Trends, 14: 1-28.
Louviere, J. J., & Islam, T. 2006. A comparison of importance weights and willingness-to-pay measures derived from choice-based conjoint, constant sum scales and best–worst scaling. Journal of Business Research, 61(9): 903-911.
Louviere, J. J., & Woodworth, G. 1983. Design and analysis of simulated consumer choice or allocation experiments: an approach based on aggregate data. Journal of marketing
research, 20(4): 350-367.
McFadden, D. 1974. Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Eds.), Frontiers of Econometrics, 105–142, New York: Academic Press
29 Mugera, A., Burton, M., & Downsborough, E. 2017. Consumer preference and willingness to pay for a local label attribute in Western Australian fresh and processed food products. Journal
of Food Products Marketing, 23(4): 452-472.
Murphy, J. J., Allen, P. G., Stevens, T. H., & Weatherhead, D. 2005. A meta-analysis of hypothetical bias in stated preference valuation. Environmental and Resource
Economics, 30(3): 313-325.
Neill, H. R., Cummings, R. G., Ganderton, P. T., Harrison, G. W., & McGuckin, T. 1994. Hypothetical surveys and real economic commitments. Land Economics, 70(2): 145. Pierce, T. 2013. Generating revenue from various business models. In T. Pierce (Eds.),
Appreneur: 27-30. Berkeley, CA: Apress
Rogers, E.M. 1995. Diffusion of innovations, (4th ed.), New York: Free Press.
Schmidt, J., & Bijmolt, T. H. 2020. Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias. Journal of the Academy of Marketing
Science, 48(3): 499-518.
Sichtmann, C., Wilken, R., & Diamantopoulos, A. 2011. Estimating willingness‐to‐pay with choice‐based conjoint analysis–can consumer characteristics explain variations in accuracy?. British Journal of Management, 22(4): 628-645.
30 Verlegh, P. W., Schifferstein, H. N., & Wittink, D. R. 2002. Range and number-of-levels effects in derived and stated measures of attribute importance. Marketing Letters, 13(1): 41-52.
Voelckner, F. 2006. An empirical comparison of methods for measuring consumers’ willingness to pay. Marketing Letters, 17(2): 137-149.
Warrillow, J. 2015. The automatic customer: creating a subscription business in any
industry. London: Penguin.
31
7. APPENDIX A
FIGURE A1
Boxplot of Age Distribution Among Three Datasets
FIGURE A2
Plot of Gender Distribution Among Three Datasets
FIGURE A2
32 FIGURE A3
Plot of Living Situation Distribution Among Three Datasets
FIGURE A4
33 FIGURE A5
Plot of 5G Knowledge Distribution Among Three Datasets
FIGURE A6
Plot of Data Allowance Distribution Among Three Datasets
34
Plot of Priceplan Distribution Among Three Datasets
FIGURE A8
35 FIGURE A10
Boxplot of Incremental WTP Traditional and Subscription
FIGURE A11
36 FIGURE A12
Age Distribution of Conjoint Dataset in Normality Plot
FIGURE A13
