REDUCING HYPOTHETICAL BIAS IN CHOICE-BASED CONJOINT ANALYSIS:

REDUCING HYPOTHETICAL BIAS

IN CHOICE-BASED CONJOINT ANALYSIS:

Investigating Realization Probability in Incentive Alignment

Master thesis, MSc Marketing, specialization Marketing Intelligence University of Groningen, Faculty of Economics and Business

Author Ji Xiao

REDUCING HYPOTHETICAL BIAS

IN CHOICE-BASED CONJOINT ANALYSIS:

Investigating Realization Probability in Incentive Alignment

ACKNOWLEDGEMENT

First and foremost, I would like to express my appreciations to my supervisor Dr. Felix Eggers for being an incredible mentor since my Pre-Master. Without his rigorous and patience, my thesis cannot be built up from the sketch to a four-engine jumbo jet (Airbus A380 to be exact)! I am also grateful to Junpeng Chen, Yu Luo, Xuyang Zhang, Chenming Peng, and Rob Valkenburg for their great help during my thesis writing. I would especially express the gratitude to my family, particularly my mom. Without their unconditional supports, I cannot reach this point in my life. Lastly, I need to admit my reluctance to say goodbye to my student career. However, when I look at the path in front of me, I simply need to do as my motto indicates:

“life is about courage and going into the unknown!”

ABSTRACT

Incentive alignment is frequently used in CBC studies to effectively reduce hypothetical bias and improve the estimate accuracy of the relative partworths. However, it is constrained by budget and prototype availability in many cases. In the current study, we illustrate that realization probability technique adds flexibility to incentive-aligned mechanism and shows a substantial performance to weaken hypothetical bias than hypothetical and “cheap-talk” mechanisms. Price and dual-response format are determined as two key identifiers to measure the effect of realization probability on arousing consumers’ motivation to answer preference questions realistically. Within realization probability, we discover a linear effect between 10% and 50% of the probability range. Furthermore, this study argues “cheap-

talk” design as an alternative of incentive alignment or realization probability under certain circumstances.

Keywords: Conjoint analysis, realization probability, incentive alignment, hypothetical bias,

Table of Contents

1. Introduction ... 5

2. Literature Review... 7

2.1. Conjoint analysis and its choice-based format ...7

2.2. Hypothetical bias ...8

2.3. Incentive-aligned CBC analysis as a solution of hypothetical bias reduction ...8

2.4. Realization probability and its impact on hypothetical bias reduction ... 11

2.5. “Cheap-talk” design in hypothetical CBC as an alternative to diminish hypothetical bias 14 2.6. Conceptual Framework ... 15 3. Experimental Design ... 15 3.1. Procedure ... 15 3.2. Conjoint Design ... 17 3.3. Model Specification ... 18 4. Results ... 20 4.1. Data Description ... 20 4.2. Conjoint Models ... 21

The Base Model ... 21

The Moderation Model ... 22

The Full Model Simplification ... 23

4.3. Moderation Analysis ... 23

Internal Validity and Scale Effect ... 23

Interpretation of Results and Discussion of Hypotheses... 25

5. Discussion ... 29 5.1. Findings ... 29 5.2. Implementations ... 30 5.3. Limitations ... 31 Bibliography ... 32 Appendices ... 35

Appendix I. Conjoint design ... 35

Appendix II. Estimates of base model and extended model in partworth form ... 36

1. Introduction

Before launching a new product, marketers usually hold great interests in knowing the product valuation among consumers in order to evaluate the competitiveness of the new product in the market. Besides, knowing consumers’ sensitivity to price can help marketers to form the pricing tactic of the product when it launches (Jedidi and Zhang, 2002). Often, there is a challenge to gain these insights as the historical data is usually missing due to the nature of a new product, e.g., an innovation. In such case, a popular solution is using conjoint analysis to gather insights of hypothetical product stimuli being evaluated by consumers.

In brief, conjoint analysis is an experimental method to understand consumer preferences for products. In this method, products are recognized as attribute bundles, in which the utility (consumer preference) of a product is the total of the partworth utilities of the attribute levels (experimental factors). One issue of using this method to examine consumer preference is that conjoint analyses are susceptible to hypothetical biases produced by the discrepancy between the hypothetical product valuation and the actual valuation of consumers (e.g., Frykblom, 2000; Wertenbroch and Skiera, 2002; Murphy et al., 2005). To prevent this problem in conjoint analysis and particularly in choice-based conjoint (hereafter, CBC) analysis, the incentive-aligned mechanism has been introduced. In comparison with the hypothetical setting, incentive alignment produces fewer hypothetical biases in conjoint analysis by making the choices consequential and, thus, is more reliable (e.g., Ding et al., 2005; Ding, 2007; Miller et al., 2011).

relevant studies, there are two contradictory conclusions. Dong et al. (2010) indicate the effectiveness of two realization probabilities on consumers’ preference decisions do not differ much when the experiment is incentive-aligned and the probabilities are located in a valid range. Similarly, Carson et al. (2014) also conclude that there is no significant difference between various probabilities. On the other hand, Cummings et al. (1998) and Cao and Zhang (2017) claim that the larger the realization probability is, the higher the consumers’ cognitive efforts are putted on choices in an incentive-aligned conjoint setting. These contrary empirical findings leave a research gap for this study to replicate incentive-aligned CBC study with different realization probabilities in more detail. In order to help on explaining the conflict, this study aims to contribute more evidence to the following primary research question:

How does realization probability in incentive-aligned CBC experiments reduce the hypothetical bias?

Another disadvantage of incentive alignment is that at least one prototype of the product is needed as the reward to give away (e.g., Wertenbroch and Skiera, 2002; Miller et al., 2011). When the product is unavailable as an incentive, one possible solution can be using the “cheap-talk” design proposed by Cummings and Taylor (1999). The “cheap-“cheap-talk” implementation is found significantly positive to lower hypothetical bias, in comparison with the hypothetical setting. However, it may only apply adequately with inexperienced respondents (List, 2001). The “cheap-talk” approach is a less focused topic in current academic research and more evidence is needed to prove its sufficiency in practice. Thus, it is meaningful for this research to reproduce this design to examine whether it can be an ideal alternative if incentive alignment cannot be applied. With such an intention, we try to answer the secondary research question:

How does the “cheap-talk” mechanism contributes to hypothetical bias decrement in CBC experiments?

probabilities to ensure the cost is minimized and predictive ability is still powerful. By answering the secondary research question, we desire to propose the “cheap-talk” design as another applicable mechanism to cut down the bias effect for preference research, especially in the case when incentive-aligned CBC is not feasible due to, for example, the studying product is highly priced, complex, or lacked a prototype.

2. Literature Review

2.1. Conjoint analysis and its choice-based format

Conjoint analysis is originally designed as a solution for key academic and industry problems (Hauser and Rao, 2002). In 1971, Green and Rao introduced this methodology to solve marketing and product development problems. In marketing and product development segments, conjoint analysis is often used to conjecture consumer multidimensional perceptions and consumer preferences, in relation to a (new) product (Carroll, Arabie, and Chaturvedi, 2002). Generally, there are different techniques to elicit preferences in conjoint analysis. The most popular conjoint analysis procedures ask consumers to either choose, rank or rate alternative products determined by specific attributes. Then, these procedures use collected data to locate the set of partworths that reflect respondents’ overall preferences the most (Green and Srinivasan, 1978).

to limit hypothetical bias due to diminished uncertainty and improved accuracy (Murphy et al., 2005).

2.2. Hypothetical bias

In many cases, CBC studies are implemented in a hypothetical setting. This type of CBC setting builds imaginary inconsequential situations, which experimenters are not required to offer any substantial benefits to the participant (Carson and Groves, 2007). Therefore, hypothetical CBC carries out benefits like riskless and low-cost (Cao and Zhang, 2017). When putting these benefits into actual practice, the hypothetical condition is more preferred for innovations without an existing prototype and high-priced or complex products (Ding et al., 2005).

On the other hand, one main downside of the hypothetical setting is that it creates a gap between the respondent’s stated hypothetical preferences and revealed preferences in the marketplace, and this gap can be referred as hypothetical bias (e.g., Diamond and Hausman, 1994; Murphy et al., 2005; Miller et al., 2011). When a CBC experiment lacks incentive-compatibility, its participants may have fewer budget constraints of the included price attribute. Also, participants may hold more willingness to comply with social desires and norms when the object is a public good (Cao and Zhang, 2017). As an outcome, participants tend to spend less cognitive efforts on their decision-making, and hence, their stated preferences become less reliable as they often exceed the true preferences (e.g., Neill et al., 1994; Harrison and Rutström, 2002; Lusk and Schroeder, 2004). In the real-life situation, hypothetical bias, which is created by the stated preference (hypothetical) methods for eliciting consumers’ product preference or willingness-to-pay (hereafter, WTP), may mislead managers to overprice products to induce potential economic losses (Wertenbroch and Skiera, 2002).

2.3. Incentive-aligned CBC analysis as a solution of hypothetical bias

reduction

Table 2.1 summarizes multiple recent choice experiments, which have used incentive-aligned conjoint. In this table, it is notable that these conjoint experiments use different elicitation mechanisms which either directly or indirectly answer preference or WTP questions. Accordingly, one of the listed experiments uses BDM, an incentive-aligned method of conjoint analysis proposed by Becker et al. (1964). This mechanism allows respondents to adopt the product if a randomly generated price is less or equal than their directly stated product valuation. Six out of eight listed studies adopt Ding’s (2005) incentive-aligned CBC mechanism (hereafter, ICBC) in their experiments. As an extension of BDM, ICBC uses the preference-inferred WTP than the directly stated WTP within the BDM approach. The preference-inferred preference approach is a circuitous way to predict a respondent’s valuation for any variation of the product being examined, based on his or her responses of preference-related questions. The study of Dong et al. (2010) offer and apply a so-called RankOrder approach to solve the problem of ICBC in terms of product availability and uneasy understandable procedure of the conjoint experiment. In RankOrder setting, the lottery picked winners receive the top-ranked product that they identified after their conjoint responses. When the top-ranked product is not available, an alternative from the preference list can be provided as a replacement.

All of the studies from Table 2.1 compare either BDM, ICBC or RankOrder mechanism to the hypothetical setting, and the hit rate ratio in the last column compares the hit rate (the level of correct preference prediction of the mechanism) of the chosen incentive-aligned conjoint with the hypothetical conjoint. Mathematically, the hit rate ratio equals the hit rate of incentive-aligned conjoint divide by the hit rate of hypothetical conjoint. According to the calculated ratio of all listed studies, the hit rate improves in incentive-aligned condition, and therefore, it is more reliable in terms of predicting consumer preference than the hypothetical setting. Hence, incentive-aligned conjoint produces less hypothetical bias than its hypothetical counterpart.

Table 2.1. Preference/WTP tests of incentive-aligned conjoint Author Year Type of

Elicitation

Research Context Incentive Alignment Hit Rate Ratio Wetenbroch

& Bernd

2002 BDM Ballpoint pens A bag of M&Ms vs. No

M&Ms; The chosen pen

1.10

Ding et al. 2005 ICBC Chinese dinner specials $10 for the chosen Chinese meal

1.85

Ding et al. 2005 ICBC Snack combos $3 for the selected snack combo

1.38

Ding 2007 ICBC iPod Nano packages The selected iPod Nano package

2.12

Ding 2007 ICBC iPod Shuffle packages The selected iPod Shuffle package

1.62

Miller et al. 2011 ICBC Cleaning products for high-tech equipment MP3 player; The cleaning product 1.59 Wlömert & Eggers

2016 ICBC Music streaming services Free subscription of the music streaming service

1.11

According to previously introduced incentive-aligned mechanisms, price (as an attribute) plays a crucial role in conjoint experiments to estimate consumers’ preferences, especially when they are incentive aligned. In a hypothetical setting, respondents may be less price-sensitive as they have no obligation to buy. Due to the insensitivity to price, respondents may not fully understand the true product valuation and make intuitional choices, and therefore, the bias increases (Cao and Zhang, 2017). However, when a reward is involved and the price becomes “real”, the situation alters. According to Dodds et al. (1991), the real price is an external clue to stimuli buyers’ perceptions and purchase intentions of a product. It has a positive effect on perceived quality but a negative impact on perceived value. By using price as an attribute to explore consumers’ product valuation in their incentive-aligned CBC studies, Miller et al. (2011), Eggers et al. (2016), and Cao and Zhang (2017) confirm the finding from Dodds and his colleagues addressing that respondents become more price-conscious and are willing to spend costly efforts into defining the product valuation when incentive is aligned with their decisions on product valuation. When respondents are obligated to purchase a product with their perceptions of the product value but the marked price does not match their stated price, they become more realistically price-sensitive to choose an alternative to make sure the stated product valuation does not exceed their buying budget (Wertenbroch and Skiera, 2002). Such purchasing behavior allows the stated product valuation becoming more realistic, and from the research perspective, it effectively deducts hypothetical bias.

In order to examine the effect of price on product preference in those two CBC settings, this study initially uses product price as an essential attribute and consumers’ product preference choices as the utility to replicate the CBC study in both hypothetical and incentive-aligned settings. Accordingly, the proposed hypothetical is:

2.4. Realization probability and its impact on hypothetical bias

reduction

When digging further into the implementation of incentive-aligned CBC analysis for predicting product preference, it is found popular to use the “realization probability” concept (e.g., Becker et al., 1964; Ding, 2007; Cao and Zhang, 2017) to conduct an incentive-aligned CBC study. In definition, realization probability refers to the probability that the respondent’s efforts of doing the experiment could be recognized for a reward (Cao and Zhang, 2017). Therefore, the “realization probability” approach allows providing fewer rewards when full incentive alignment is not a feasible approach.

In conjoint analysis, it is known that incentive alignment holds a positive effect on hypothetical bias reduction, especially when comparing with the hypothetical manner (e.g., Ding et al., 2005, Ding, 2007). However, with respect to the impact on realization probability, conclusions from different works of literature vary. Table 2.2 indicates relevant incentive-aligned conjoint tests with different realization probabilities applied. In this table, we present two contradictory findings in terms of the effect of the magnitude of realization probability on hypothetical bias reduction.

Table 2.2. Incentive-aligned conjoint with various realization probabilities Author Year Research

Context Alignment Incentive Realization Probability (RP)

Incentive

Value Expected Value Significant Difference of RPs Voelckner 2006 Doughnuts vs. Mugs Doughnuts or Mugs 10% vs. 100% Doughnuts: €0.55 Mug: €0.73 Doughnuts: €0.055 (10%) vs. €0.55(100%) Mug: €0.073 (10%) vs. €0.73 No

Dong et al. 2010 Digital picture frames vs. T-shirts Digital picture frames or T-shirts Digital picture frames: 2% vs. 4% T-shirts: 100% Digital picture frames: $200 T-shirts: $18

Digital picture frames: $4 (2%) vs. $8 (4%) T-shirts: $18 No Carson et al. 2014 Vote on a public good $10 for the Cal Ripken Jr. ticket 20% vs. 50% vs. 80% $10 $2 (20%) vs. $5 (50%) vs. $8 (80%) No Cummings & Taylor

1998 Policy vote $10 for participation 25% vs. 50% vs. 75% $10 $2.5 (25%) vs. $5 (50%) vs. $7.5 (75%) Yes Cao & Zhang 2017 Mobile soccer game platform The player package 3.33% vs. 50% $16, 20, 24, 28, 32 $24 as an example: $0.8 (3.33%) vs. $12 (50%) Yes

with choosing the whole study group being the winner (100%). In another research conducted by Dong et al. (2010), their conclusion firstly addresses that different priced incentives ($200 for the digital picture frame and $18 for the customized T-shirt) with different realization probabilities (2% or 4% for the digital picture frame and 100% for the customized T-shirt) have no difference in terms of the effect on people’s effort making. When the incentive is digital picture frame in specific, they indicate that different realization probabilities between 0 and 1 do not stimulate consumers to put more (or less) cognitive efforts into their answers when the experiment is incentive-aligned and the realization probabilities are in a reasonable scope. In their study, they notice the hit rates would remain the same if the probability to receive the digital picture frame changed from 4% to 2%. In a public vote study (Carson et al., 2014), researchers ask participants to vote for a project and probabilistically choose a group of people who vote for passing to reward prize. As a result, they find that, as long as the probabilities are away from zero, they have no significant difference in decreasing hypothetical bias.

On the contrary, in a similar study of Cummings and Taylor (1998), when the vote passes, the chosen voters from different probabilistic groups need to donate partially of their incentives to the project. In this case, voters from a group with higher realization probability are more careful to express “yes” to pass the referendum. In another study, Cao and Zhang (2017) state that the predictive power improves if incentive alignment involves and a higher realization probability can boost this improvement (in their case, the realization probability of 50% contributes more on estimation than 3.33%).

incentives’ EV for an individual respondent may be similar ($4/$8 for the digital picture frame and $18 for the T-shirt), because the respondent might realize that the probability to be rewarded the expensive digital picture frame is low and the value that each individual receives the digital frame is on the same level as receiving the T-shirt. Therefore, this respondent is likely to put equal or similar efforts for making decisions in these two incentive settings, and consequentially, the predictive ability of the two conjoint experiments using different incentives are similar. In order to avoid such EV effect caused by the reverse setting between the incentive value and realization probability (e.g., EV ($100 * 10%) = EV ($20 * 50%)), this study provides only one incentive but multiple realization probabilities to ensure the focus of this study is solely on the influence of realization probability on consumers’ product preferences.

To sum up, the finding from Cummings and Taylor (1998) and Cao and Zhang (2017) show that there is a linear effect of realization probability on attributes to utility. However, this assumption cannot be drawn from the research of Cummings and Taylor (1998), Dong et al. (2010) and Carson et al. (2014) as they present the opposite conclusion. Such an argument leaves this study to conduct a similar conjoint experiment to confirm either side of this debatable topic with further insights. Derived by this motive, the following hypothesis is formed:

Hypothesis 2a (H2a): With one incentive aligned and when realization probabilities increase, respondents are more attentive to the product offerings, hence are more responsive to the choice alternatives, thus increasing the effect of realization probability on hypothetical bias reduction.

are less sensitive to the difference among the probabilities. In the case of Dong et al. (2010), respondents may judge both 2% and 4% as low chance to be rewarded. Therefore, they perceive no significant difference of them, and hence performing similarly towards the choices. In the other case (Cao and Zhang, 2017), subjects may identify 3.33% as a rare opportunity to receive the prize but 50% belongs to the “medium-chance-to-win” group. Hence, subjects of the two probabilistic experiments perform different cognitive efforts of the choice making. According to this representativeness heuristic, potentially, there is a cluster effect among different realization probabilities, in relation to their influence on reducing hypothetical biases in conjoint analysis. For example, in a CBC experiment, when two chosen probabilities are not in a large distance (e.g., 10% vs. 20%), respondents may put the same level of efforts to make decisions as they perceive similar opportunities to gain the incentive. On the other hands, when two probabilities are quite different from each other, saying 10% and 50%, respondents may perform dissimilarly due to the distant chance to be rewarded. To sum up, this study tries to shed more lights on this assumption by proposing the following hypotheses:

Hypothesis 2b (H2b): With one incentive aligned and when realization probabilities are close to each other, respondents are similarly sensitive to the product offerings, and hence are equally responsive to the choices, thus not changing the effect of realization probability on hypothetical bias reduction.

2.5. “Cheap-talk” design in hypothetical CBC as an alternative to

diminish hypothetical bias

remarks of the “cheap-talk” design. The authors of the “cheap-talk” method address that the cheap-talk script need to be concretely to include enough stimuluses to ensure subjects hold sufficient awareness of the bias. In a later application, List (2001) argues the “cheap-talk” design effectively decreases hypothetical bias only for inexperienced respondents of conjoint studies. This study holds a curiosity on whether a standard “cheap-talk” design is sufficient enough to weaken the hypothetical bias with a particular research context, books. Accordingly, a hypothesis relates to the “cheap-talk” concept structures as follows:

Hypothesis 3 (H3): When applying “cheap-talk” mechanism, respondents place more awareness on hypothetical bias, hence are more responsive to product choices than in the hypothetical condition, thus reducing hypothetical bias.

2.6. Conceptual Framework

3. Experimental Design

3.1. Procedure

10%, 20%, 50%, 60%. We create these realization probability groups to observe the effect of small and moderate realization probabilities and the difference among the four, as the reflection of proposed H2a and H2b.

Generally, all three brief CBC conditions include two parts: introductory survey and conjoint task. In our study, we choose academic books as the research context and target university students as the experiment objects. Therefore, the introductory survey contains the experimental instruction and three essential questions in terms of the object’s field of study, current year of study, as well as a recent academic book the participant desires to purchase. The later CBC experiment firstly offers a test scenario by including the elicited information from the introductory survey, together with other necessary messages in relation to the CBC setting. By providing the introductory survey and the test scenario afterwards, we try to simulate the real-life situation by taking the object’s perception of familiarity and knowledge of the research context into account. We assume that keeping these two sections can make objects feel more engaged to the CBC experiment since they should be familiar with and knowledgeable to the scenario proposed book.

3.2. Conjoint Design

Our CBC experiment includes specific attributes of an academic book, such as book price, format and the inclusion of online supplements. Among the two given book formats (printed book and eBook), several attributes are developed for each format. The printed book sector includes information like book condition, shipping time and the existence of a combo which provide a standard e-version of the book upon purchase. In the eBook condition, four attributes are selected with information about the inclusion of interactive features, notes and highlights function, read-aloud function, and the accessibility on different electronic devices. The attribute levels of all attributes vary between 2 and 4, and each level is designed to be exclusive from each other mutually. A full list of attributes and levels can be found in Table 3.1.

Table 3.1. Attributes and attribute levels of the CBC experiment

Attribute Level 1 Level 2 Level 3 Level 4

M ai n A tt ri b u te

s Book format _Price Printed book _€10 eBook _{€20 €30 €40}

Online supplements No Yes (Access to

online resources, tests, and quizzes) A tt ri b u te s of p ri n te d b ook

Combo No Yes (With

additional standard eBook)

Condition New Used, shows

sign of wear

Shipping time 1 day 3 days 5 days

A tt ri b u te s of e B ook

Interactive features No Yes, videos in

the eBook Yes, quizzes in the eBook Yes, homework/assignment questions in the eBook

Notes and highlights No Yes, allow

adding highlights and taking notes Yes, allow adding highlights and taking notes, seeing highlights and notes from other readers

Yes, allow adding highlights and taking notes, seeing highlights and notes from the teacher

Read-aloud function No Yes

Accessibility On a computer On a smartphone, tablet, or e-reader

On all devices

“No-choice” option No, none of the alternatives is preferred

Yes, one product alternative is preferred

3.3. Model Specification

The random utility (hereafter, RUT) model is used in this CBC study to predict students’ book preference. RUT is the theoretical framework of choice behavior. It specifies that choices are based on overall utilities of alternatives, and good is the combination of attributes. The mathematical algorithm of the model is as followed:

𝑈𝑖𝑗 = 𝑉𝑖𝑗+ 𝜀𝑖𝑗

This equation explains the overall utility U of consumer i for product j is a latent construct that contains a systematic utility component (rational utility) V and the stochastic utility component (error term) 𝜀 (Manski, 1977). From the consumer perspective, the RUT theory emphasizes that people implicitly compare the utility of each option in the choice set and choose the alternative with maximum utility.

In order to estimate the systematic utility of choice options, we can apply multinomial logit (MNL) model (e.g., McFadden, 1974; Islam et al., 2007; Eggers et al., 2016). According to the MNL model, the chosen option of the respondent i can be any alternative from choice set J, and the probability to choose the alternative m from J can be displayed as:

𝑝𝑟𝑜𝑏_𝑖(𝑚|𝐽) = exp⁡(𝑉𝑖𝑚) ∑_𝑗𝐽exp⁡(𝑉_𝑖𝑗)

This model allows us to predict choice probabilities of respondents by modeling the utility of different book propositions.

In our CBC analysis, we reveal the impact of book attributes and their levels on the systematic utility component V by assuming the overall systematic utility V is a linear combination of the partworth utilities of the book format f and other attributes A:

𝑉_𝑖𝑗 = 𝛽_𝑓𝑓_𝑗+ 𝛽_𝐴𝐴_𝑗

When the book format is printed book (𝑓_{𝑝𝑟𝑖𝑛𝑡}), the overall systematic utility V is constructed as:

𝑉_𝑖𝑗 = 𝛽_𝑓𝑓_{𝑝𝑟𝑖𝑛𝑡,𝑗}+ 𝛽_{𝑝𝑟𝑖𝑐𝑒}𝑃𝑟𝑖𝑐𝑒_𝑗+⁡𝛽_{𝑠𝑢𝑝𝑠}𝑆𝑢𝑝𝑠_𝑗+ 𝛽_{𝑐𝑜𝑚𝑏}𝐶𝑜𝑚𝑏_𝑗+ 𝛽_{𝑐𝑜𝑛𝑑}𝐶𝑜𝑛𝑑_𝑗⁡+ 𝛽_{𝑡𝑖𝑚𝑒}𝑇𝑖𝑚𝑒_𝑗

When the book format is eBook (𝑓_{𝑒𝐵𝑜𝑜𝑘}), the overall systematic utility V is generated as: 𝑉𝑖𝑗 = 𝛽𝑓𝑓𝑒𝐵𝑜𝑜𝑘,𝑗+ 𝛽𝑝𝑟𝑖𝑐𝑒𝑃𝑟𝑖𝑐𝑒𝑗+ ⁡𝛽𝑠𝑢𝑝𝑠𝑆𝑢𝑝𝑠𝑗+ 𝛽𝑖𝑛𝑡𝑓𝐼𝑛𝑡𝑓𝑗+ 𝛽𝑛𝑜𝑡𝑒𝑁𝑜𝑡𝑒𝑗⁡+ 𝛽𝑟𝑒𝑎𝑓𝑅𝑒𝑎𝑓𝑗

+ 𝛽_{𝑎𝑐𝑒𝑠}𝐴𝑐𝑒𝑠_𝑗

In each choice set, when none of the book offerings is preferred, respondents can always confirm the “no-choice” alternative. In such case, the overall systematic utility V is developed as:

𝑉_𝑖𝑗 = 𝛽_{𝑛𝑜𝑛𝑒}𝑁𝑜𝑛𝑒

In order to test the moderating effect  of the three proposed conjoint settings (hypothetical, incentive-aligned and “cheap-talk”) on respondents’ preference choices, we include interaction effect of each conjoint setting on each attribute for each alternative. The explanation of every variable used in the models below can be found in Table 3.2.

𝑉_𝑖𝑗 = 𝛽_𝑓𝑓_{𝑒𝐵𝑜𝑜𝑘,𝑗} + 𝛽_{𝑝𝑟𝑖𝑐𝑒}𝑃𝑟𝑖𝑐𝑒_𝑗+⁡𝛽_{𝑠𝑢𝑝𝑠}𝑆𝑢𝑝𝑠_𝑗+ 𝛽_{𝑖𝑛𝑡𝑓}𝐼𝑛𝑡𝑓_𝑗+ 𝛽_{𝑛𝑜𝑡𝑒}𝑁𝑜𝑡𝑒_𝑗⁡+ 𝛽_{𝑟𝑒𝑎𝑓}𝑅𝑒𝑎𝑓_𝑗 + 𝛽_{𝑎𝑐𝑒𝑠}𝐴𝑐𝑒𝑠_𝑗+ 𝜂_{ℎ𝑦𝑝𝑜,𝑒𝐵𝑜𝑜𝑘}𝐻𝑦𝑝𝑜_𝑖 ∗ 𝑒𝐵𝑜𝑜𝑘_𝑖+⁡𝜂_{ℎ𝑦𝑝𝑜,𝑝𝑟𝑖𝑐𝑒}𝐻𝑦𝑝𝑜_𝑖 ∗ 𝑃𝑟𝑖𝑐𝑒_𝑖𝑓 +⁡𝜂ℎ𝑦𝑝𝑜,𝑠𝑢𝑝𝑠𝐻𝑦𝑝𝑜𝑖∗ 𝑆𝑢𝑝𝑠𝑖𝑓⁡+⁡𝜂ℎ𝑦𝑝𝑜,𝑖𝑛𝑡𝑓𝐻𝑦𝑝𝑜𝑖∗ 𝐼𝑛𝑡𝑓𝑖𝑓 + 𝜂ℎ𝑦𝑝𝑜,𝑛𝑜𝑡𝑒𝐻𝑦𝑝𝑜𝑖 ∗ 𝑁𝑜𝑡𝑒_𝑖𝑓+⁡𝜂_{ℎ𝑦𝑝𝑜,𝑟𝑒𝑎𝑓}𝐻𝑦𝑝𝑜_𝑖∗ 𝑅𝑒𝑎𝑓_𝑖𝑓⁡+ 𝜂_{ℎ𝑦𝑝𝑜,𝑎𝑐𝑒𝑠}𝐻𝑦𝑝𝑜_𝑖∗ 𝐴𝑐𝑒𝑠_𝑖𝑓 + 𝜂𝑝𝑟𝑜𝑏,𝑒𝐵𝑜𝑜𝑘𝑃𝑟𝑜𝑏𝑖 ∗ 𝑒𝐵𝑜𝑜𝑘𝑖 +⁡𝜂𝑝𝑟𝑜𝑏,𝑝𝑟𝑖𝑐𝑒𝑃𝑟𝑜𝑏𝑖∗ 𝑃𝑟𝑖𝑐𝑒𝑖𝑓 +⁡𝜂_{𝑝𝑟𝑜𝑏,𝑠𝑢𝑝𝑠}𝑃𝑟𝑜𝑏_𝑖 ∗ 𝑆𝑢𝑝𝑠_𝑖𝑓⁡+⁡𝜂_{𝑝𝑟𝑜𝑏,𝑖𝑛𝑡𝑓}𝑃𝑟𝑜𝑏_𝑖∗ 𝐼𝑛𝑡𝑓_𝑖𝑓+ 𝜂_{𝑝𝑟𝑜𝑏,𝑛𝑜𝑡𝑒}𝑃𝑟𝑜𝑏_𝑖 ∗ 𝑁𝑜𝑡𝑒_𝑖𝑓+⁡𝜂_{𝑝𝑟𝑜𝑏,𝑟𝑒𝑎𝑓}𝑃𝑟𝑜𝑏_𝑖 ∗ 𝑅𝑒𝑎𝑓_𝑖𝑓⁡+ 𝜂_{𝑝𝑟𝑜𝑏,𝑎𝑐𝑒𝑠}𝑃𝑟𝑜𝑏_𝑖 ∗ 𝐴𝑐𝑒𝑠_𝑖𝑓 + 𝜂_{𝑐ℎ𝑝𝑡,𝑒𝐵𝑜𝑜𝑘}𝐶ℎ𝑝𝑡_𝑖 ∗ 𝑒𝐵𝑜𝑜𝑘_𝑖+⁡𝜂_{𝑐ℎ𝑝𝑡,𝑝𝑟𝑖𝑐𝑒}𝐶ℎ𝑝𝑡_𝑖∗ 𝑃𝑟𝑖𝑐𝑒_𝑖𝑓+⁡𝜂_{𝑐ℎ𝑝𝑡,𝑠𝑢𝑝𝑠}𝐶ℎ𝑝𝑡_𝑖 ∗ 𝑆𝑢𝑝𝑠𝑖𝑓⁡+⁡𝜂𝑐ℎ𝑝𝑡,𝑖𝑛𝑡𝑓𝐶ℎ𝑝𝑡𝑖∗ 𝑖𝑛𝑡𝑓𝑖𝑓+ 𝜂𝑐ℎ𝑝𝑡,𝑛𝑜𝑡𝑒𝐶ℎ𝑝𝑡𝑖 ∗ 𝑁𝑜𝑡𝑒𝑖𝑓 +⁡𝜂_{𝑐ℎ𝑝𝑡,𝑟𝑒𝑎𝑓}𝐶ℎ𝑝𝑡_𝑖 ∗ 𝑅𝑒𝑎𝑓_𝑖𝑓+ 𝜂_{𝑐ℎ𝑝𝑡,𝑎𝑐𝑒𝑠}𝐶ℎ𝑝𝑡_𝑖 ∗ 𝐴𝑐𝑒𝑠_𝑖𝑓 𝑉_𝑖𝑗 = 𝛽_{𝑛𝑜𝑛𝑒}𝑁𝑜𝑛𝑒 + 𝜂_{ℎ𝑦𝑝𝑜,𝑛𝑜𝑛𝑒}𝐻𝑦𝑝𝑜_𝑖 ∗ 𝑁𝑜𝑛𝑒_𝑖𝑓+ 𝜂_{𝑝𝑟𝑜𝑏,𝑛𝑜𝑛𝑒}𝑃𝑟𝑜𝑏_𝑖∗ 𝑁𝑜𝑛𝑒_𝑖𝑓 + 𝜂_{𝑐ℎ𝑝𝑡,𝑛𝑜𝑛𝑒}𝐶ℎ𝑝𝑡_𝑖∗ 𝑁𝑜𝑛𝑒_𝑖𝑓

Table 3.2. Variable description Print (Format) Printed book eBook (Format) eBook

Price Book price

Sups Online supplements

Comb The offer to combine the printed book with a standard e-version

Cond Book condition

Time Shipping time

Intf Interactive features

Note Notes and highlights function

Reaf Read-aloud function

Aces Accessibility

None The dual-response format / the “no-choice” option

Hypo Hypothetical CBC mechanism

Prob RandOrder aligning CBC mechanism

Chpt “Cheap-talk” CBC mechanism

4. Results

4.1. Data Description

treatment for outliners and missing values, we used data from a total of 600 valid respondents (18,000 observations) without significantly changing the sample profile. The total sample is randomly assigned to one of the experiment conditions, and each condition has an equal number of respondents. Besides choice data, this preference survey dataset also recorded the experiment condition each participant takes and created data column for each condition for the purpose of analyzing.

As our conjoint analysis uses MNL models to estimate students’ book preference, the results are presented on an aggregate level with a maximum likelihood procedure. For the interpretation purpose, we apply effect coding technique to all attributes in Table 3.1 except for the “no-choice” option and price. With such an approach, we can treat the selected alternatives as a deviation from a hypothetical mean of zero. In other words, we use effect coding to allow the positive partworth utilities represent greater preferences for that level in comparison with all levels’ mean partworth utility of the attribute. Thus, instead of comparing with the absolute level, these positive values are compared to the average value of the levels that are involved in the experimental design (Eggers et al., 2016). With regard to the no-choice option, we apply dummy coding with “choosing none of the book offerings” as one and “choosing one offering” as zero. Besides, we assume a linearized price attribute as a premise. This precondition allows us to conduct the study in a simpler manner while carrying valid conclusions and easier interpretations.

We use dummy coding technique to make the “cheap-talk” design as a nominal independent variable. As for the hypothetical and realization probability settings, they are firstly dummy coded as one nominal variable, where zero indicates the hypothetical setting and one means the realization probability setting. Later, we take out the realization probability as a numeric variable for linearization check. In the second step, for constructing the partworth moderation model and gaining insights from different realization probabilities, we separate this numeric variable into four dummy variables, where each variable represents one level of the realization probability that we use in this study (10%, 20%, 50%, and 60%).

4.2. Conjoint Models

The Base Model

each level of the attribute except for price. Generally, BM1 shows face validity (see Appendix II). In particular, the result presents a negative relationship between book preference and price. Moreover, based on the estimates, the main effect of most attributes on utility is significant, except for the interactive function featuring video (p = .714) and homework (p = .197) and the basic package of notes and highlights function (p = .054).

In order to examine modeling alternative possibility, we compare BM1 to a null, vector and ideal-point alternative respectively by applying the likelihood ratio test (Table 4.1). From the test outcome, we find BM1 fits significantly better than the null model (p < .001), as well as the vector alternative (hereafter, BM2), where shipping time is set to linear (p < .001). We further measure BM1 with its ideal-point counterpart for shipping time (hereafter, BM3). We find the two models have the exactly same test outcome (𝑝 = 1). The reason is that BM3 needs the shipping time attribute to include not only a parameter for the linear effect but also one for the squared term. Therefore, it is identical that BM3 has the same model fit as BM1 in which the same shipping time attribute with two parameters is presented. As a decision, we keep BM1 as our base model since it is more analysis-friendly when adding interaction effects.

Table 4.1. Modeling Alternative (Base Model) Selected Model Null model BM1 (Partworth) BM2 (Vector) BM3 (Ideal-Point) #DF 2 17 16 17 LL -6538.5 -5283.8 -5291.8 -5283.8 P-value < 0.001 < 0.001 1.000 AIC 13080.92 10601.68 10615.69 10601.68 R2 _{(%-points) 19.19 19.07 19.19} Adjusted R2 (%-points) 18.93 18.82 18.93

The Moderation Model

a better performance of Akaike information criterion and adjusted coefficient of determination ⁡(𝐴𝐼𝐶_𝐼𝑀2= 10479.09;⁡∆𝑅2

𝐼𝑀2= 19.87%-𝑝𝑜𝑖𝑛𝑡𝑠; 51⁡𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠) while

consisting of only half amount of parameters of IM1 ⁡(𝐴𝐼𝐶_𝐼𝑀1= 10517.02;⁡∆𝑅2

𝐼𝑀1=

19.58%-𝑝𝑜𝑖𝑛𝑡𝑠; 102⁡𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠), we conclude to choose IM2 as our full (moderation) model. Since IM2 has already the same model fit as IM1 and is most parsimonious among the three model forms, we do not necessarily need to compare to the ideal-point form.

The Full Model Simplification

In a further step, we try to find a simpler and valid version of IM2. We systematically combine parameters for moderation (Table 4.2). Based on the comparison, we discover MS3 (hereafter, IM3) is the model with the lowest AIC and highest adjusted R2 _{⁡(𝐴𝐼𝐶}

𝐼𝑀3=

10449.88;⁡∆𝑅2

𝐼𝑀2 = 20.09%-𝑝𝑜𝑖𝑛𝑡𝑠). Thus, we finally determine IM3, which consists of

significant interaction effects from IM2, as the optimal model for this study to work with.

Table 4.2. Model Simplification (Moderation Model)

#DF LL AIC R2 _{(%-points) Adjusted}

R2

(%-points)

Difference towards interaction effects included in IM2

Full

Model IM2 51 -5188.5 10479.09 20.65 19.87

MS1 37 -5189.8 10453.66 20.63 20.06 Exclude highly insignificant effects (p > .5)

MS2 33 -5196.3 10458.55 20.53 20.02 Include effects that are significant in one conjoint condition at least

Selected Model (IM3)

MS3 26 -5198.9 10449.88 20.49 20.09 Include only significant effects

MS4 25 -5205.6 10461.30 20.39 20.00 Include only main attributes for interaction

MS5 24 -5208.7 10465.40 20.34 19.97 Include effects that are literature-proven key attributes for bias reduction

MS6 22 -5207.9 10459.88 20.35 20.01 Include significant effects that the attributes are literature-proven influential on bias reduction

MS7 19 -5268.1 10574.19 19.43 19.14 Include only significant effects in all conjoint conditions

4.3. Moderation Analysis

Internal Validity and Scale Effect

(%-points) of each conjoint setting is generated and then visualized in Figure 4.1. Accordingly,

we see the hypothetical condition has the lowest model fit (𝑅2

𝐵𝑀1ℎ𝑦𝑝𝑜 = 16.5%-𝑝𝑜𝑖𝑛𝑡𝑠), and

the fit gradually rises from “cheap-talk” setting to realization probability conditions. Since a better model fit indicates an improving validation, and therefore, shows a greater performance in hypothetical bias reduction (Murphy et al., 2005), we preliminarily determine that “cheap-talk” design and realization probabilities affect hypothetical bias reduction positively in our case.

Figure 4.1. Model fit of each conjoint condition

As the model fit gets better when the realization probability grows, the scale of the estimates should also go up. To verify this assumption, we apply a scale measurement proposed by Eggers et al. (2017). This measurement calculates the observed scale of each sample by using the same logit model and summing up the range of utilities across attributes. The absolute magnitude of the observed total utility should be larger when the scale is bigger. We employ the scale measurement to all six subsamples by using the same model, BM1. The results (Figure 4.2) of the scale measurement confirm the above assumption and is consistent with the model fit measurement indicating a higher internal validity and lower hypothetical bias as the realization probability increases.

18.52 16.5 19.75 21.03 24.49 24.69 0 5 10 15 20 25 30

"Cheap talk" Hypothetical Realization Probability = 10% Realization Probability = 20% Realization Probability = 50% Realization Probability = 60% (% -p oi nt s)

Figure 4.2. Absolute magnitude of each conjoint condition

When the coefficients of a parameter (e.g., price) increase or decrease in different partworths, we cannot directly indicate, for example, the growing negative estimates of price value can report an increasing price-sensitivity in a rising realization probability setting. Larger price coefficients might due to bigger scales, but the relevant importance of price may be the same among all conditions (Eggers, et al., 2017). By extending the scale measurement, we further determine the relevant attribute importance (hereafter, RAI) of price and online supplements when realization probability equals 0 (hypothetical condition), 10%, 20%, 50%, and 60%. According to Figure 4.3, the RAI of both price and online supplements vary in different conditions. Hence, the scale effect does not play a negative role when interpreting results in our study.

Figure 4.3. RAI of price and online supplements in each realization probability

Interpretation of Results and Discussion of Hypotheses

Table 4.3 depicts the significant estimates of IM3. In comparison of the hypothetical condition, we find realization probability (𝜂_{𝑝𝑟𝑜𝑏,𝑝𝑟𝑖𝑐𝑒} = −0.001, 𝑝 < .001) linearly moderates the effect

10.27 _9.63 11.18 13.1 13.91 13.97 0 2 4 6 8 10 12 14 16

"Cheap talk" Hypothetical Realization Probability = 10% Realization Probability = 20% Realization Probability = 50% Realization Probability = 60% Absolute Magnitude 12.65 13.86 21.63 28.4 30.86 0 10 20 30 40 0 10% 20% 50% 60% (% -p oi nt s)

Magnitudes of Realization Probability RAI of Price 0.17 1.58 2.78 3.88 3.01 0 1 2 3 4 0 10% 20% 50% 60% (% -p oi nt s)

of price on utility significantly, such that price increases respondents’ cognitive efforts on decision making in realization probability setting. In addition, the RAI of price goes up when realization probability is larger (Figure 4.3). Furthermore, its RAI holds a comparatively large share among attributes in all partworths (Appendix III). These RAI outcomes can indicate that respondents increasingly value price when their chances to be rewarded grow bigger. In a broader scope, we first treat the linear realization probability as incentive alignment and claim that respondents become more price-sensitive in incentive-aligned mechanism than in hypothetical setting. This conclusion complies with previous literature (e.g., Dodds et al., 1991; Miller et al. 2011; Cao and Zhang, 2017) and confirms our H1.

With respect to H2b, we interpret the estimates of IM1 (see Appendix II), which has described the moderating effect of four magnitudes of realization probability (10%, 20%, 50%, and 60%) in detail. From H2a, we determine price and dual-response option as two key influencers of how much efforts consumers put into their preference choices. Thus, we check the significance of the two attributes on probability levels. Based on the calculation (see Appendix IV), we notice insignificance between realization probability equals 50% and 60% (p = .376). Therefore, we argue that the realization probability can be linearized in terms of modeling, but respondents can actually become similarly price-sensitive to product profiles when the realization probability is located between 50% and 60%. Based on the graph of model fit (Figure 4.1), scale (Figure 4.2) and the RAI of price (Figure 4.3), we can further argue that the moderating effect on price to utility may not reduce hypothetical bias in much different when realization probability is 50% and 60%. For the dual-response option, its performance is significantly different on each level of realization probability. Thus, the assumption of clustering among the magnitudes of realization probability is not valid for “no-choice”. Eventually, we conclude to partially accept H2b.

is previously defined as a dominant identifier of the attractiveness of product profiles, its interaction effect on utility in “cheap-talk” setting is not included in IM3 due to the insignificance in IM2 (𝑝 = .105). According to Figure 4.1 and 4.2, “cheap-talk” mechanism has a better R2 _{and observed scale than hypothetical mechanism. We indicate that “cheap-talk”}

design performs better in diminishing hypothetical bias. To sum up, we partially accept H3 by claiming, when “cheap-talk” design is applied in a CBC experiment, respondents are more price-cautions and able to think realistically towards their product choices, thus producing less hypothetical bias.

Table 4.3. Maximum likelihood preference estimates of the chosen conjoint model

Estimate results of IM3

(0.001) Observations 6,000 Log Likelihood -5,198.939 Note: *_p<0.1; **_p<0.05; ***_p<0.01

5. Discussion

5.1. Findings

Our research presents that the “cheap-talk” script of highlighting the issue of hypothetical bias in conjoint experiments is effective to guide consumers to react on price-related attributes truthfully. Fundamentally, “cheap-talk” design can be a compromise in CBC experiments. It performs better than the hypothetical format but less than the realization probability (incentive-aligned) format. In particular, “cheap-talk” mechanism can be an alternative when the realization probability in an incentive-aligned condition is set as 10%. However, it is important to emphasize that, according to our results, respondents from “cheap-talk” group consider price as the most relative important attribute while participants of 10% realization probability group measure other attributes being more important (see Appendix III). Therefore, replacing 10% realization probability to “cheap-talk” can alter the performance validity to change the effect on hypothetical bias reduction since the most important relative attribute changes in conditions. This may undermine the predictive accuracy. In general, the findings of “cheap-talk” design answer our secondary research question and partially support the claim of “cheap-talk” being an alternative of incentive alignment from Cummings and Taylor (1999).

5.2. Implementations

CBC experiments have the conceptual advantages to measure preference, and well-designed CBC mechanisms can add great value to these experiments. Derived from our empirical findings, we first propose incentive alignment as the prior choice when conducting conjoint experiments. In addition, we suggest researchers and managers use higher realization probabilities in conjoint design when the full incentive-aligned mechanism (100% realization probability) cannot be applied. Also, it is suggested to include price and dual-response format in the attribute list as both factors are essential to determine hypothetical bias.

10%. However, a tradeoff must be made as choosing the “cheap-talk” setting can reduce some effects on bias reduction.

5.3. Limitations

Bibliography

Baier, D., & Gaul, W. (2001). Market Simulation Using a Probabilistic Ideal Vector Model for Conjoint Data. In A. Gustafsson, A. Herrmann, & F. Huber, Conjoint

Measurement (pp. 97-120). Berlin, Heidelberg: Springer.

Becker, G. M., Marschak, J., & DeGroot, M. H. (1964). Measuring utility by a Single Response Sequential Method. Systems Research and Behavioural Science, 9(3), 226-232.

Cao, X., & Zhang, J. (2017). Prelaunch Demand Estimation. MIT Sloan School of

Management, 1-43.

Carroll, D., Arabie, P., & Chaturvedi, A. (2002). Multidimensional Scaling and Clustering

and their Applications. Philadelphia: Wharton School Press.

Carson, R. T., & Groves, T. (2007). Incentive and informational properties of preference questions. Environmental and Resource Economics, 37(1), 181-210.

Carson, R. T., Groves, T., & List, J. A. (2014). Consequentiality: A Theoretical and Experimental Exploration of a Single Binary Choice. Journal of the Association of

Environmental and Resource Economists, 1(1/2), 171-207.

Chen, L., Petkova, R., & Zhang, L. (2006). THE EXPECTED VALUE PREMIUM .

NATIONAL BUREAU OF ECONOMIC RESEARCH, 14(1), 1-43.

Christianson , M., & Aucoin, M. (2005). Electronic or print books: Which are used? Library

Collections, Acquisitions, & Technical Services, 29(1), 71-81.

Clerides, S. K. (2002). Book value: intertemporal pricing and quality discrimination in the US market for books. International Journal of Industrial Organization , 20(10), 1385-1408.

Cummings, R. G., & Tay, L. O. (1998). Does Realism Matter in Contingent Valuation Surveys ? Land Economics, 74(2), 203-215.

Cummings, R. G., & Taylor, L. O. (1999). Unbiased values estimates for environmental goods: a cheap talk design for the contingent valuation method. The American

Economic Review, 89(3), 646-665.

Diamond, P. A., & Hausman, J. A. (1994). Contingent Valuation : Is Some Number better than No Number? The Journal of Economic Perspectives, 8(4), 45-64.

Ding, M. (2007). An incentive-aligned mechanism for conjoint analysis. Journal of

Marketing Research, 44(2), 214-223.

Ding, M., Grewal, R., & Liechty, J. (2005). Incentive-Aligned Conjoint Analysis. Journal of

Marketing Research , 42, 67-82.

Dodds, W. B., Monroe, K. B., & Grewal, D. (1991). Effects of Price, Brand, and Store Information on Buyers' Product Evaluations. Journal of Marketing Research, 28(3), 307-319.

Dong, S., Ding, M., & Huber, J. (2010). A simple mechanism to incentive-align conjoint experiments. International Journal of Research in Marketing, 27(1), 25-32. Edwards, W. (1954). The Theory of Decision Making. Psychological Bulletin, 51(4),

380-417.

Eggers, F., Eggers, F., & Kraus, S. (2016). Entrepreneurial branding: measuring consumer preferences through choice-based conjoint analysis. International Entrepreneur

Management Journal, 12, 427-444.

Eggers, F., Hauser, J., & Selove, M. (2017). Scale Matters: How Craft in Conjoint Analysis Affects Price and Positioning Strategies . MIT, 1-34.

Frykblom, P. (2000). Willingness to pay and the choice of question format: Experimental results. Applied Economics Letters, 7, 665-667.

Green, P. E., & Rao, V. R. (1971). Conjoint Measurement for Quantifying Judgmental Data.

Journal of Marketing Research, 8(3), 355-363.

Green, P. E., & Srinivasan, V. (1978). Conjoint Analysis in Consumer Research: Issues and Outlook. Journal of Consumer Research, 5(2), 103-123.

Gregory, C. L. (2008). "But I Want a Real Book": An Investigation of Undergraduates' Usage and Attitudes toward Electronic Books. Reference & User Services Quarterly, 47(3), 266-273.

Harrison, G. W., & Rutström, E. E. (2002). Experimental Evidence of Hypothetical Bias in Value Elicitation Methods. Handbook of Results in Experiment Economics, 1, 757-767.

Hauser, J. R., & Rao, V. R. (2004). Conjoint Analysis, Related Modeling, and Applications.

Marketing Research and Modeling: Progress and Prospects, 14, 141–168.

Islam, T., Louviere, J. J., & Burke, P. F. (2007). Modelling the effects of including/excluding attributes in choice experiments on systematic and random components. International

Journal of Research in Marketing, 24, 289–300.

Jedidi, K., & Zhang, J. Z. (2002). Augmenting Conjoint Analysis to Estimate Consumer Reservation Price. Management Science, 48(10), 1350-1368.

Ji, S., Michaels, S., & Waterman, D. (2014). Print vs. electronic readings in college courses: Cost-efficiency and perceived learning. The Internet and Higher Education, 21, 17-24.

List, J. A. (2001). Do explicit warnings eliminate the hypothetical bias in elicitation

procedures? Evidence from Field Auctions for Sportscards. The American Economic

Review, 91(5), 1498-1507.

List, J. A., & Gallet, C. A. (2001). What experimental protocol influence disparities between actual and hypothetical stated values? Environmental and Resource Economics, 20(3), 241-254.

Lusk, J. L., & Schroeder, T. C. (2004). Are Choice Experiments Incentive Compatible? A Test With Quality Differentiated Beef Steaks. American Journal of Agricultural

Economics, 86(2), 467-482.

Manski, C. F. (1977). Structure of random utility models. Theory and Decision, 8(3), 229– 254.

McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka, Frontiers in Econometrics (pp. 105–142). New York: Frontiers in Econometrics.

Miller, K. M., Hofstetter, R., Krohmer, H., & Zhang, J. Z. (2011). How Should Consumers' Willingness to Pay Be Measured? An Empirical Comparison of State-of-the-Art Approaches. Journal of Marketing Research, 48(1), 172-184.

Miller, L. N. (2014). Preference for Print or Electronic Book Depends on User’s Purpose for Consulting. Evidence Based Library and Information Practice, 9(3), 95-97.

Murphy, J. J., Allen, G. P., Stevens, T. H., & Weatherhead, D. (2005). Hypothetical Bias in Stated Preference Valuation. Environmental and Resource Economics, 30, 313-325. Neill, H. R., Cummings, R. G., Ganderton, P. T., & McGuckin, T. (1994). Hypothetical

Surveys and Read Economic Commitments. Land Economics, 70(2), 145-154. Strouse, G. A., & Ganea, P. A. (2017). A print book preference: Caregivers report higher

Tversky, A., & Kahneman, D. (1974). Judgment under Uncertainty : Heuristics and Biases Published. Science, New Series, 185(4157), 1124-1131.

Voelckner, F. (2006). An Empirical Comparison of Methods for Measuring Consumers ' Willingness to Pay. Marketing Letters, 17(2), 137-149.

Wertenbroch, K., & Skiera, B. (2002). Measuring Consumers’ Willingness to Pay at the Point of Purchase. Journal of Marketing Research, 39(2), 228-241.

Wlömert, N., & Eggers, F. (2016). Predicting new service adoption with conjoint analysis: external validity of BDM-based incentive-aligned and dual-response choice designs.

Marketing Letters, 27(1), 195-210.

Appendices

Appendix I. Conjoint design • The “cheap-talk” script

What’s the matter?

In a recent study, several different groups of people made their preference-related choices on a new product just like the one you are about to choose. Payment was hypothetical for these groups, as it will be for you. No one had to purchase their chosen products. The results of these studies were that on average, across the groups, 38% of them stated “yes” that they are willing to buy the chosen product. With another set of groups with similar people choosing the same type of products as you will choose here, but with a real consequence that a certain group of participants (after a lottery session) were really required to buy their selected product. The results on average across the group were that 25% said “yes”. That’s quite a difference, isn’t it?

We call this a “hypothetical bias.” Hypothetical bias is the difference that we continually see in the way people respond to consumers’ hypothetical preferences of a product compared to real preferences.

How can we get people to think about their choice making in a hypothetical scenario as they think in a real situation, where if a bunch of people from the ones who select “yes”, they’ll really have to pay money? How do we get them to think about what it means to really dig into their pocket and pay money if in fact they really aren’t going to have to do it?

Why is it happening?

Let me tell you why I think that we continually see this hypothetical bias, why people behave differently in a hypothetical setting than they do when the purchase is real. I think that when we hear about a survey that asks us to choose a preferred product among a few, our basic reaction in a hypothetical situation is to think: I will just make my choice quickly, and don’t worry, I really would say “yes, I would buy

it” to my chosen product.

But when the purchase is real, and we would actually have to buy the selected product, we think in a different way. We basically still would prefer the product we chose, but when we are faced with the possibility of having to spend money, we think about our options: if I am

obligated to buy this product, I need to make sure this is the one which I want the most (after a careful consideration of all features), and I need to take into account the limited amount of money I have to spend on this. This is just my opinion, of course, but it’s what I

think may be going on in an actual scenario. So, if I were in your shoes, I would ask myself:

- if this were a real purchase, and I had to pay for the book:

o What product attributes (features) do I concern the most, in terms of real use?

o Do I really want to spend my money to buy this book, when taking all other FEASIBLE options into account?

Please take the bias into your consideration

In any case, I ask you to choose just exactly as you would choose if you were really going to face the consequences of your choice: which is to buy the book if you are chosen to be awarded a coupon. Please keep this in mind in our choice experiment.

Appendix II. Estimates of base model and extended model in partworth form

Maximum likelihood preference estimates of conjoint models

Dependent variable:

Partworth Model

Base Model Extended Model

(0.149) I(prob60 * notes.highlights_extravisual.otherreaders) 0.159 (0.150) I(prob60 * notes.highlights_extravisual.teacher) 0.082 (0.147) I(prob60 * read.aloud_included) 0.117 (0.087) I(prob60 * accessibility_computer) -0.144 (0.125) I(prob60 * accessibility_portabledevices) 0.190 (0.124) Observations 6,000 6,000 Log Likelihood -5,283.840 -5,156.509 Note: *_p<0.1; **_p<0.05; ***_p<0.01

Appendix III. Relative attribute importance (RAI) of six conjoint elicitation formats

“Cheap-Talk” Hypothetical

Min. Max. Range Relative Importance (%-points)

Min. Max. Range Relative Importance (%-points) Book format -0.219 0.219 0.438 4.26% -0.024 0.024 0.048 0.51% Online Supplements -0.052 0.052 0.104 1.01% -0.008 0.008 0.016 0.17% Combo -0.384 0.384 0.768 7.47% -0.260 0.260 0.520 5.40% Book condition -0.573 0.573 1.146 11.16% -1.143 1.143 2.286 23.74% Shipping time -0.522 0.711 1.233 12.00% -0.464 0.688 1.152 11.96% Interactive features -0.499 0.742 1.241 12.07% -0.787 0.763 1.550 16.09% Notes and highlights function -1.116 0.942 2.058 20.03% -0.959 1.018 1.977 20.53% Read-aloud function -0.262 0.262 0.524 5.10% -0.181 0.181 0.362 3.76% Accessibility -0.250 0.378 0.628 6.11% -0.194 0.353 0.547 5.68% Price -1.708 -0.427 2.135 20.79% -0.975 -0.244 1.219 12.65% 10.273 9.630

Realization Probability (=10%) Realization Probability (=20%) Min. Max. Range Relative

Importance (%-points)

Min. Max. Range Relative Importance (%-points) Book format -0.408 0.408 0.816 7.30% -0.319 0.319 0.638 4.87% Online Supplements -0.089 0.089 0.178 1.58% -0.182 0.182 0.364 2.78% Combo -0.110 0.110 0.220 1.97% -0.319 0.319 0.638 4.88% Book condition -0.906 0.906 1.812 16.19% -1.015 1.015 2.030 15.49% Shipping time -0.552 1.035 1.587 14.19% -0.835 1.136 1.971 15.04% Interactive features -0.744 0.810 1.554 13.89% -0.765 0.686 1.451 11.07% Notes and highlights function -1.189 1.020 2.209 19.75% -1.245 1.014 2.259 17.24% Read-aloud function -0.279 0.279 0.558 4.99% -0.225 0.225 0.450 3.44% Accessibility -0.253 0.448 0.701 6.27% -0.199 0.268 0.467 3.56% Price -1.240 -0.310 1.550 13.86% -2.266 -0.567 2.833 21.63% 11.184 13.100

Realization Probability (=50%) Realization Probability (=60%) Min. Max. Range Relative

Importance (%-points)

Combo -0.184 0.184 0.368 2.65% -0.226 0.226 0.452 3.24% Book condition -1.083 1.083 2.166 15.56% -0,907 0.907 1.814 12.99% Shipping time -0.564 0.790 1.354 9.73% -0.666 1.234 1.900 13.60% Interactive features -0.828 0.836 1.664 11.96% -0.501 0.651 1.152 8.25% Notes and highlights function -0.949 0.900 1.839 13.22% -0.923 1.100 2.023 14.49% Read-aloud function -0.216 0.216 0.432 3.11% -0.298 0.298 0.596 4.27% Accessibility -0.335 0,397 0.732 5.26% -0.304 0.308 0.612 4.38% Price -3.161 -0.790 3.951 28.40% -3.449 -0.862 4.311 30.86% 13.914 13.968

Appendix IV. P-value comparison of price and dual-response format in realization probabilities

Benchmark Realization Probability

10%

Benchmark Realization Probability

REDUCING HYPOTHETICAL BIAS

IN CHOICE-BASED CONJOINT ANALYSIS:

Investigating Realization Probability in Incentive Alignment

Presented by

Ji Xiao

RESEARCH QUESTIONS

How does realization probability in incentive-aligned

CBC experiments reduce the hypothetical bias?

How does the “cheap-talk” mechanism contributes to

hypothetical bias decrement in CBC experiments?

“

”

CONJOINT DESIGN

Attribute Level 1 Level 2 Level 3 Level 4

M ain At tr ib u te

s Book format Printed book eBook

Price €10 €20 €30 €40

Online supplements No Yes (Access to online resources, tests, and quizzes)

At tr ib u te s of p rin te d b ook

Combo No Yes (With additional

standard eBook)

Condition New Used, shows sign of wear

Shipping time 1 day 3 days 5 days

At tr ib u te s of e B ook

Interactive features No Yes, videos in the eBook Yes, quizzes in the eBook Yes, homework/assignment questions in the eBook Notes and

highlights

No Yes, allow adding highlights and taking notes

Yes, allow adding highlights and taking notes, seeing highlights and notes from other readers

Yes, allow adding highlights and taking notes, seeing highlights and notes from the teacher

Read-aloud function No Yes

Accessibility On a computer On a smartphone, tablet, or e-reader

On all devices

“No-choice” option No, none of the

alternatives is preferred

Yes, one product alternative is preferred

• Fraction factorial and

alternative specific

• 3 alternatives per

choice set

• 10 choice sets per

respondent

• Minimum overlap

• Balanced and

PROCEDURE & MODELING

Hypothetical

“Cheap-Talk”

RP=10%

RP=20%

RP=50%

RP=60%

Introduction Survey

• Study program • Current year of study

• Recently wanted academic book

Conjoint Task

• Task scenario • Choice-based conjoint Information of the reward “Cheap-Talk” script Purchasing task Realization Probability (RP)

𝑈

_𝑖𝑗

= 𝑉

_𝑖𝑗

+ 𝜀

_𝑖𝑗

Random Utility Model:

VALIDATION

Model Selection

The Base Model

Null

Part-worth Vector

Ideal-point

The Moderation Model

Part-worth Vector

The Simplified Moderation Model “Base model + Significant interaction

effects of the moderation model”

Internal Validity

18.52 16.5 19.75 21.03 24.49 24.69 0 5 10 15 20 25 30

"Cheap talk" Hypothetical Realization Probability = 10% Realization Probability = 20% Realization Probability = 50% Realization Probability = 60% (% -p o in ts )

Coefficient of Determination (%-points)

Scale Effect

10.27 _9.63 11.18 13.1 13.91 13.97 0 2 4 6 8 10 12 14 16

"Cheap talk" Hypothetical Realization Probability = 10% Realization Probability = 20% Realization Probability = 50% Realization Probability = 60% Absolute Magnitude 12.65 13.86 21.63 28.4 30.86 0 10 20 30 40 0 10% 20% 50% 60% (% -po int s)

Magnitudes of Realization Probability RAI of Price 0.17 1.58 2.78 3.88 3.01 0 1 2 3 4 0 10% 20% 50% 60% (% -po int s)

Magnitudes of Realization Probability RAI of Online Supplements

Model fit of each conjoint condition Absolute range of each conjoint condition