1
The influence of persuasion strategy and design factors on the
effectiveness of persuasive messages
2
The influence of persuasion strategy and design factors on the
effectiveness of persuasive messages
Rutger van Zanten University of Groningen Faculty of Economics and Business
MSc Marketing Intelligence January 22, 2016 Paulus Potterstraat 53 8932 KK Leeuwarden Tel: +31(0) 6 57563093 E-mail: rutger.van.zanten@hotmail.com Student number: S1815180 Supervisors: University of Groningen
3
Abstract
Online shops often use various persuasion claims about product popularity or product scarcity to convert visitors into buyers. Besides, they spend billions of dollars every year again on banner advertisements and developments of their online shops. Despite those heavy investments, average online conversion rates are still around 2%. Marketers are now expected to come up with creative ideas to enhance conversion rates.
Conversify is a company which tries to enhance those conversion rates by being creative. They show relevant persuasive messages in a pop-up form. To investigate which persuasive strategy and which design factors increase the conversion rates most, a fractional factorial field experiment is executed. Eight different persuasive pop-ups were developed varying on persuasion strategy (scarcity vs social proof), position (bottom left vs top left), exposure duration (4 seconds vs 8 seconds) and animation type (fading vs sliding). Those pop-ups were exposed for a period of two weeks on four different online shops.
The results of the experiments show that statements about scarcity of a product are more effective compared to statements about a products popularity. However, how the message is presented is also found to be of significant influence. Many researchers have established the various effects of design factors of banner advertisements on attention or purchase attention, however this is one of the first studies to link design factors to actual sales. The results suggest that a pop-up on the bottom of the page is more effective compared to a up at the top of the page. Furthermore, exposing the pop-up for 8 seconds is more effective compared to an exposure duration of 4 seconds. Lastly, a fading animation leads to a higher probability of turning a visitor into a buyer compared to a sliding animation.
Those results have to be interpreted with care since the field experiment suffered from several major limitations.
4
Acknowledgements
5
List of content
1. Introduction ... 7
2. Literature review and hypotheses ... 9
2.1 Theories of information processing ... 9
2.2 Persuasion strategies ... 11 2.3.1 Social proof ... 12 2.3.2 Scarcity ... 13 2.4 Design factors: ... 14 2.4.1 Position ... 15 2.4.2 Duration of exposure ... 16 2.4.3 Animation type ... 17 2.5 Conversion... 18 2.6 Conceptual model ... 18 3. Research design ... 19
3.1 Factorial field experiment... 19
3.2 Construction of pop-ups ... 20 3.3 Implementation of pop-ups... 22 3.4 Sampling technique ... 25 4. Methodology ... 27 4.1 Data description ... 27 4.2 Sample ... 29 4.3 Plan of analysis ... 30 5. Results ... 33
5.2 Overall model fit ... 34
6 5.4.1 Probit model ... 37 5.4.2 Validation set ... 38 6. Discussion ... 39 6.1 Contribution to literature ... 40 6.2 Managerial implications ... 41
6.3 Limitations and further research ... 41
References ... 44
7
1. Introduction
As online retailing has grown rapidly worldwide, firms intensify their online activities to reach and attract more consumers. To attract more consumers and get more sales, companies invest heavily in their online presence (Bleier and Eisenbeis, 2015; Hausman and Siekpe, 2009). Consequently online advertising has become a substantial part of the economy with spendings of 17,7 billion dollars only on display advertisements in 2013, which is expected to increase with 53% by 2017 (Hoban and Bucklin, 2015). With this growing amount of
investments in online presence, companies’ online efforts have become an increasing topic of interest to academic research (Hausman and Siekpe, 2009; Hoban and Bucklin, 2015).
Developments in information technology, which made individual online click-stream data available, have enabled practitioners and academics to investigate the effectiveness of online shops and advertising campaigns (Lizhen, Duan and Whinston, 2014). Results showed that despite the amount of investments, there is still much room for improvement (Hausman and Siekpe, 2009). For example, online shoppers abandon their shopping cart in about 25% of the cases (Kukar-Kinney and Close, 2010), e-retailers have an average conversion rate of about 2-3% and advertisement banners click trough rates are as low as 0.08% (Bleier and Eisenbeis, 2015). The amount of banners online shoppers are confronted with, has even led to so called banner blindness which is the tendency for users to avoid attending banner advertisements, or anything that may create the impression to be a banner advertisement (Resnick and Albert, 2014).
Numerous researchers state that in order to increase online results, and in particular conversion rates, it is important to create an online shopping experience for the customer (Hausman and Siekpe, 2009; Mahnke, Benlian and Hess, 2015) and that marketers need to be creative (Smith, Chen and Yang, 2008). The ongoing developments in information technology enabled companies to improve existing methods and start new initiatives to reach this goal (Smith et al., 2008).
8 actions and display relevant persuasive pop-up messages. This can be messages such as “23 Other people bought this product in the last 2 days.”, “This product is almost out of stock, we have 1 left!” and “Our favorite product!”. Those persuasive messages are already widely used by marketers, however, their effects on actual online sales is not established (Jeong and Kwon, 2012). Besides, online shops that already use this type of messages usually show them as a steady component of the product detail page, while Conversify presents those messages in the form of a pop-up.
Another important aspect that can enhance experience is design. For instance, Hausman and Siepke (2009) state that almost half of the visitors of websites pay more attention to the design elements of the site, than to its content. Aside from the potential to enhance
experience, design is also known because of its ability to attract attention (Pieters and Wedel, 2004). For advertisements, elements such as color, brand, pictures, text size, exposure timing and location are known to have considerable effects on the amount of attention the
advertisement attracts (Chan, Jiang and Tan, 2010; Moore, Stammerjohan and Coulter, 2005; Pieters and Wedel, 2004; Pieters, Wedel and Zhang, 2007; Zhang, Wedel and Pieters, 2009). Research about how specific design factors influence shopping behavior or lead to higher conversion rates, is relatively scarce (Hausman and Siekpe, 2009).
This research aims to bring knowledge about the differential effects of different types persuasion strategies and design factors on actual sales. Considering type of persuasion, this research makes a distinction between two types of persuasion tactics classified by Cialdini (1984): social proof and scarcity. With regard to design characteristics, this research focuses on position on the screen, exposure duration, and type of animation.
The research question is:
“Do persuasion strategy and design factors influence the effectiveness of online persuasion messages?”
This question is answered by conducting a fractional factorial field experiment.
9 experiment will be introduced. Chapter four will describe the methodology including some descriptives of the data gathered by the experiment and a plan of analysis. In chapter five the analysis will be carried out and the results will be presented. Chapter 6 discusses the results, their implications and limitations and presents some suggestions for future research.
2. Literature review and hypotheses
The aim of this article is to find out to which extent persuasion strategies and design factors of a persuasive message can influence the conversion rates of a web shop. In this section,
theories of information processing are discussed and how design can play a role in affecting the processing of information. After that, different types of persuasion techniques are discussed. Third, the design factors position on the screen, duration of exposure and type of animation are elaborated on. Although the list of possible design variables is exhaustive and many more variables would be relevant to investigate, those factors are selected on request of Conversify, their interestingness and on the basis of feasibility. Finally, using conversion in terms of actual sales as dependent variable is explained.
2.1 Theories of information processing
Websites are full of decision making stimuli, such as text, images, audio and video’s. All these factors try to influence the thoughts and behaviors of the visitors (Tam and Ho, 2005). When people are presented with all this stimuli, they go through different stages of
information processing. Those different stages are described by the hierarchy of effects (HOE) models. While a lot of different HOE models are proposed by different authors, they all can be generalized as predicting a sequence of three stages: attention, affect and behavioral intention (Smith, et al. 2008).
People are limited in their cognitive capacities and as a consequence, they are not able to process all the information and stimuli presented to them. Because of that, some stimuli will not reach all stages of information processing, or will even not be noticed at all. As Hoyer, MacInnis and Pieters (2013) state, for a message to have a persuasive effect, at least some basic level of information processing is needed. A first step in this process is therefore gaining attention. However, increased attention does not certainly leads to positive attitudes or
10 stimuli in such a way that they are most likely to process all stages of information processing and have most influence on the behavioral intentions of the receiver (Campbell, 1995).
Capacity theories of attention and information processing models assume that attention allocated to a persuasive message is dependent on two types of factors: top-down factors, which consist of consumers’ motivation, involvement and ability and bottom-up factors, which are the properties of the persuasive message (Pieters and Wedel, 2004).
A significant top-down factor is involvement of the consumer. The elaboration likelihood model of Petty, Cacioppo and Schumann (1983) makes a distinction between the central route of information processing, which is taken by the high involved consumers and the peripheral route of information processing, which is taken by the low involved consumers. In the central route the perceiver of a message actively processes the presented information and carefully elaborates on it (Petty et al., 1983). When the consumer is less motivated and follows the peripheral route, he or she does not actively processes and evaluates the arguments presented. The consumer is then guided by peripheral cues (source credibility, style and format of the message, mood, and so on), which are influenced by the design factors of the message (Petty et al., 1983).
To reach the best message effectiveness, personalized messages of which the bottom-up factors harmonize with the top-down factors of each individual are suggested. However, many marketers do not obtain enough personal information, such as demographics or involvement to present every single visitor or segments of visitors with tailored messages (Lin and Chen, 2009). Because involvement of the consumer is found to be a significant moderator of the perceived persuasiveness of a message, Rosenkrans (2010) states that at least the
informational as well as the design properties of a persuasive message need to be considered and optimized to attain the best general effects.
11 Theories on processing fluency state that stimuli that can be easily processed are generally evaluated in a positive way, lead to positive attitudes and are considered more trustworthy compared to non-fluent stimuli (Storme et al., 2015). Processing fluency consists of
perceptual and conceptual fluency (Lee and Labroo, 2004). Perceptual fluency is the ease at which the physical features of a stimulus can be identified whereas conceptual fluency refers to the ease of understanding the meaning of a stimulus (Shapiro 1999; Lee and Labroo 2004). Repeated exposure to a certain stimuli is known to enhance both types of processing fluency, while changes in design features can enhance the perceptual fluency by making the stimuli easier to differentiate from its environment (Lee and Aker, 2004; Rompay and Pruyn, 2011).
Research however also revealed that increased information processing does not always lead to increased effectiveness of persuasive messages. People are confronted with persuasive tactics at a regular basis and consequently developed persuasion knowledge. This means that people are able to recognize the persuasive intend of a message. With increased processing of the message, the chances of noticing the persuasive character of the message increases as well. When people are aware of the persuasive attempt of the message, they might get irritated or counter argue the message (Friestad and Wright, 1994).
2.2 Persuasion strategies
Once decision making stimuli attracted attention, those stimuli will try to influence and persuade the customer. The number of techniques by which persuasion can be used to change attitudes and behaviors of consumers are extensive (Kaptein and Eckles, 2012). Theorists consequently have proposed different taxonomies of influence strategies, of which the six principles of persuasion developed by Cialdini (1984) are most commonly used and have shown to be valid at least in the offline world (Shu and Cheng, 2012). Their effects on actual online sales is however not established (Jeong & Kwon, 2012). The six principles proposed by Cialdini (1984) are: reciprocity, commitment, authority, liking, scarcity and social proof.
12 The principle of commitment represents the fact that once people publicly commit to a certain goal, brand, agreement and so on, they are likely to behave consistent with their commitment (Cialdini, 2001; Shu and Cheng, 2012).
Authority is the extent to which the sender of a message has a certain reputation or expertise. People are generally inclined to follow recommendations of an expert, inferring that
information from an expert is likely to be true, thereby decreasing the risk of making a wrong decision. In e-retailing trustmarks are commonly used to signal authority and “expert reviews” are presented (Cialdini, 2001; Shu and Cheng, 2012).
“Liking” means that people are more likely to be persuaded by someone they like. Liking is a really broad concept and can be strengthened in many ways. It can be as superficial as
designing a website which enhances the shopping experience, which in turn leads the websites’ visitors to like the e-retailer (Cialdini, 2001; Shu and Cheng, 2012).
Although it would be interesting to investigate the effects of the above four principles, this study focuses on the scarcity and social proof principles. These two principles are already widely used by online marketers and, although their effects are already established in the offline world, their effects on conversion rates in online retailing is barely studied (Hausmann and Siepke, 2009; Jeong and Kwon, 2012).
2.3.1 Social proof
Social proof means that people tend to rely on the experiences of others to make difficult decisions. The reasoning behind this principle is that when multiple others have bought a specific product or service, it must be good (Cialdini, 2001). Consumers thus use product popularity to make assumptions about the product quality. E-retailers make use of signaling to bring this principle in practice (Jeong and Kwon, 2012).
13 Consumers therefore tend to rely on a variety of signals to infer product quality. This can be price, brands, warranties, but also social proof messages such as average consumer review scores or statements like “this is a hot item” (Jeong and Kwon, 2012). Moreover, many online shops offer their visitors the possibility to make use of a sorting cue to sort products on review ratings or sales rank (Tam and Ho, 2005).
Next to this quality inference people make, the social proof principle might be explained by the bandwagon effect. The underlying psychology of the bandwagon effect is the desire to belong to a certain group, to behave in accordance with what is accepted in your social environment. As a result, people are more likely to buy the same products as the people in their social environment (Jeong and Kwon, 2012).
2.3.2 Scarcity
The scarcity principle is build on the fact that people evaluate products or services as more valuable and desirable once its availability is threatened (Cialdini, 2001; Lynn, 1991). This scarcity effect is often explained by the commodity theory as well as the signaling theory.
The commodity theory states that anything that can be owned, is of some utility to the possessor and can be transferred from person to person, is valued more desirable when it is difficult to obtain or is unavailable (Brock, 1968; Lynn, 1991). This in turn can be explained by people’s desire to be unique (Eisend, 2008). Besides this desire to be unique, people don’t like it to be restricted in their choices and therefore might buy the product before it comes unavailable (Gierl and Huetll, 2010).
With respect to signaling theory it can be said that people might assume limited product availability to be a consequence of product popularity and product quality (Jeong & Kwon, 2012). E-retailers consequently use statements about limited product availability to signal quality. They do this by showing messages on websites as “almost out of stock”, “limited edition” or, “only 3 items left”. While Jeong & Kwon (2012) did not found any effects of scarcity messages on purchase intention, there has been extensive research about the effects of indicating a product or service as scarce on product desirability, perceived value, brand
14 Both social proof and scarcity strategies are extensively used in online shops. However, on the internet there are for many product categories, many alternatives at hand. This makes it unlikely that scarcity of one single product will be perceived as an important restriction of freedom (Gierl and Huetll, 2010). Besides, consumers tend to rely strongly on opinions of others in online shopping and seem to have more difficulties to believe statements about scarcity compared to statements about product popularity (Jeong and Kwon, 2012). It is therefore expected that a social proof message will have more persuasive effect compared to a scarcity message.
H1: A social influence message has a stronger positive effect on conversion compared to a scarcity message.
2.4 Design factors:
Few studies have empirically tested the effects of design factors on actual online sales (Zhang et al., 2009). Studies have mainly focused on the effects of design factors on attention, attitude towards the advertisement, brand awareness or at most purchase intention (Chan, et al., 2010; Moore et al., 2005; Pieters and Wedel, 2004; Pieters et al., 2007). Besides, prior research on the effects of design factors on the effectiveness of online persuasive messages has focused almost exclusively on advertisements and banner advertisements in particular (Hong, Thong and Tam, 2007). Although the persuasive messages in this research are different from banner advertisements in the sense that they have no intention of building brand awareness, brand preference or generating click-through rates, existing research about design effects on (banner) advertisements can be helpful. The ultimate goal of both, advertisements and the persuasive messages in this research, is to convert the viewer into a buyer (Rosenkrans, 2010).
With the current advancements in information technology, the possibilities to develop different designs are extensive. Size, brand, picture, color, contrast/context, advertisement complexity, advertisement creativity and personalization are just a few of many design features which have been researched and all have their own effects on viewers responses to advertisements (Chan, et al., 2010; Moore et al., 2005; Pieters and Wedel, 2004; Pieters et al., 2007; Zhang et al., 2009). Although the effects of those design characteristics on the
15 exposure and type of animation. Those design characteristics are chosen because they closely relate to so called banner blindness and intrusiveness, two concepts that are known to
decrease the effectiveness of online advertising. Since consumers show an increasing tendency to avoid online advertising, those factors are deemed important (Cho and Cheon, 2002; Edwards, Lee and Li, 2002).
2.4.1 Position
Due to the regular basis at which people are confronted with persuasive attempts, consumers apparently did not only develop persuasion knowledge. It is found that web shop visitors often do not notice the presence of a banner advertisement, this is called banner blindness (Resnick and Albert, 2014). Although there are multiple reasons that can explain the phenomenon of banner blindness, such as lack of contrast or other attention attracting factors, location of the banner is considered the most significant factor leading to banner blindness (Resnick and Albert, 2014). People are becoming more and more experienced with online shopping and they tend to look at locations where they expect to find the most relevant information and hence, they avoid locations where they expect advertisements (Resnick and Albert, 2014).
Where Rosbergen et al. (1997) in their study on print advertisements found that elements on the right side of the page catched most attention, some eye-tracking studies on banner advertisments suggest that web visitors follow an “F-shaped” attention pattern when looking at a web page (Urban, Liberali, MacDonald, Bordley and Hauser, 2014). They found that most attention is paid at the top of the page, while spending least attention at the right side of the web page. Murphy (1999) found that advertisements at the top of the screen received considerably more attention compared to advertisements at the bottom of the screen. Since banner ads are commonly placed at the far right side or bottom of a website, people are less likely to actively process or even pay attention to stimuli in this location (Resnick and Albert, 2014).
Besides the effects of location on attention, some studies investigated the effects of location on attitude towards the advertisement. Those studies found that pictorial advertisements where perceived more favorably when positioned on the left side, while textual
16 Taking into account that the pop-ups in this research consist of both a textual part and a visual part, together with the existence of the phenomenon banner blindness, and the F-shaped pattern, it is hypothesized that the message will be more effective if it is showed at the left side of the page compared to the right side of the page.
H2a: The persuasive message will be more effective in terms of conversion if it is shown at
the left side of the page compared to when it is shown at the right side of the page.
Furthermore, it is hypothesized that presenting the message at the top of the page is more effective compared to showing the message at the bottom of the page.
H2b: The persuasive message will be more effective in terms of conversion if it is shown at
the top of the page compared to when it is shown at the bottom of the page.
2.4.2 Duration of exposure
By showing messages in pop-up mode instead of presenting the message as a part of the product detail page, a form of animation is used. Animation is known to be an important tool in catching viewers attention and can enhance recall (Rosenkrans, 2010). Animation and pop-up messages are also found to be perceived as distractive and intrusive (Cho and Cheon, 2004). While this distraction may enhance the amount of attention paid to the message and increased information processing, it may also lead to negative attitudes. In fact Cho and Cheon (2004) found distraction to be the most significant factor for consumers to avoid online advertisements. In extension of those findings, Edwards, Li and Lee (2002) found that the longer the animation lasts, the more distractive and intrusive it is perceived.
As the messages in this research are relevant for the product the consumer is looking at, they might not be considered as distracting as can be the case with banner advertisements. As multiple researchers state, the attitudes towards banner advertisements can be enhanced by context matching (Urban et al., 2014; Moore et al., 2005). Furthermore, Wang, Shih and Peracchio (2013) found that longer exposure duration can lead to increased processing
fluency, just as with repeated exposure. This will lead to more favorable attitudes towards the message. They find that for advertisements which are moderately difficult to process, the exposure duration has an inverted U-shape relation with the attitude towards this
17 which in turn leads to positive attitudes. Later on, when the duration is increased further, the effect of enhanced processing fluency diminishes and viewers get bored and might get irritated. This is in line with the findings of Boerman, van Reijmersdal and Neijens (2012) who study the effects of television sponsorship disclosure duration on the activation of persuasion knowledge. They found that people may not have the cognitive capacities
available when they are suddenly confronted with a new message. Their findings indicate that the longer the disclosure is shown, the better people are able to process the information. Besides, they state that the effect on persuasion knowledge increases with duration which leads people to counter argue the persuasive intend of the message.
This leads to the following hypotheses:
H3a: The persuasive message will be less effective in terms of conversion with a long
exposure duration compared to a moderate exposure duration
H3b: The persuasive message will be more effective in terms of conversion with a moderate
exposure duration compared to a short exposure duration
2.4.3 Animation type
Conversify chooses between two animation types for the persuasive message to enter the screen. The first option is to slide in from the side, while the second option is to fade in, in which case the message slowly crystallizes on the screen. To the authors knowledge, there is no literature available that could help to directly explain differences in effectiveness between those two animation methods. While there is no literature about this, it is assumed that a sliding message attracts more attention compared to a fading message and that consumers are not differently affected by a sliding or fading message. The increased attention of the sliding message is not expected to lead to increased negative feelings due to distraction, because the message is thought to be relevant for the viewer. The increased attention of the sliding message will in turn be expected to have a better impact throughout the entire HOE model ultimately leading to be more effective compared to a fading message.
18
2.5 Conversion
Studies about (banner) advertisements and other persuasive messages have tried to establish their effectiveness by measuring various outcome variables such as attention, attitude towards the advertisement, brand awareness or at most purchase intention (Chan, et al., 2010; Moore, et al., 2005; Pieters and Wedel, 2004; Pieters et al., 2007). However, the ultimate goal of each persuasive message or advertisement is to convert a shopper into a buyer (Rosenkrans, 2010). In the current literature, this linkage is still missing. While enhanced attention, brand
preference, or enhanced visitors due to banner click-throughs may ultimately lead to increased sales, this relationship is not empirically established.
The persuasive messages in this research do not try to attract more customers or change brand preferences, they try to convert the online shop visitors in buyers thereby increasing the conversion rate of the website. The conversion rate is defined as the percentage of visitors which complete a certain goal (Rosenkrans, 2010). Conversion rate can thus be used for, for example, click-throughs of banner advertisements: the amount of visitors who clicked on the banner advertisement divided by the amount of visitors who are confronted with the
advertisement (Rosenkrans, 2010). The conversion rate in this research is based on sales, measured by the amount of visitors on the website that purchase a product divided by the total amount of visitors on the website. A conversion event is a visitor of a website that purchases a product, also known as a sale. The effectiveness of a persuasive message in this research is assessed by the probability that a person will buy the product after being exposed to the pop-up
2.6 Conceptual model
Based on the theoretical framework, a conceptual model with the factors that are hypothesized to influence the effectiveness of the persuasive pop-ups is created. This model is shown in figure 1.
H1
H2-H4 Figure 1: Conceptual model
19
3. Research design
In this chapter, the research design is discussed. First there will be explained why data collection is done by means of a factorial field experiment. Second, construction of the pop-ups is discussed. Finally, the implementation of the experiment is elaborated on.
3.1 Factorial field experiment
For this research, a factorial field experiment is conducted. Factorial designs are very efficient for studying the effect of two or more independent variables (factors). The effect of an
independent variable can be defined as the change in the dependent variable produced by a change in the level of the independent variable. This is the main effect. Additionally, it may be found that the difference in the dependent variable between levels of independent variables is not the same across all levels of the other independent variables. This is called an
interaction effect. By using a factorial design the main and interaction effects of multiple independent variables can be estimated (Malhotra, 2010; Holland and Cravens, 1973).
The decision to collect the data by means of an experiment instead of using secondary data is guided by the fact that secondary data about the hypothesized relationships was not readily available. Besides, the pop-ups already in practice by Conversify differ on so many factors other than the factors at stake, valid measurement of the hypothesized relationships was considered not feasible. By conducting an experiment, the data is collected with the purpose of answering the hypotheses of this research and hence the effects of the independent variables can be measured more accurately. The decision between a laboratory experiment and a field experiment is made, guided bythe opportunity offered by Conversify to
investigate the effects of the independent variables on conversion rates. Establishing this relationship is not achievable by means of a laboratory experiment. The attempt to establish the relationship of the independent variables with conversion rates, instead of purchase intention, attention or other outcome measures, is one of the main contributions this research tries to make to literature.
20 naturalistic environment instead of an artificial setting. Field experiments are therefore likely to have a higher external validity compared to laboratory experiments (Malhotra, 2010). In this particular field experiment, the visitors are unaware that they are being observed and that they are part of an experiment. This eliminates common potential problems with field
research such as interviewer bias or non-response (Malhotra, 2010).
This experiment presents website visitors with different styles of persuasive pop-ups to determine the influence of persuasion strategy and design factors on the effectiveness of the pop-ups.
3.2 Construction of pop-ups
To test whether, and to which extent, persuasion strategy and the chosen design factors do influence the effectiveness of the persuasive message as hypothesized, the pop-ups that will be shown need to vary over those factors.
An overview of the independent variables and their variations is given in table 1 below.
Attribute Persuasion strategy
Position Exposure
duration
Animation
Level 1 Social proof Top left Short (4 sec) Slide
Level 2 Scarcity Top right Medium (8 sec) Fade
Level 3 Bottom left Long (20 sec)
Level 4 Bottom right
Table 1: Independent variables and their levels
For persuasion strategy, the social proof message is shown as “Dit product staat in de top 25 van meest verkochte producten in 2015.” (“This product is among the top 25 best selling products in 2015.”), while the scarcity message will state “We hebben van dit product nog X op voorraad.” (“Of this product, there are X items in stock.”). The X stands for a number varying from 1 till 9 depending on the actual amount of products in stock. To test whether the position of the message influences the effectiveness of the message, the message can be shown at the top left, top right, bottom left or bottom right position on the screen. For
21 types of animation which can be used by Conversify to enter the screen are slide and fade. In total there are 2 x 4 x 3 x 2 = 48 possible pop-up combinations.
In consultation with Conversify, it is decided that the scarcity pop-up will be able to show varying amounts of products available. This is decision is made because of technological convenience and to enlarge the possibility that clients of Conversify are willing to cooperate in the experiment. Although the number of products available might influence the
effectiveness of the persuasive pop-up, this research will only measure the average effect of the scarcity pop-ups. Besides, it must be mentioned that the pop-ups will be shown with a small picture. This picture is corresponding with the persuasion strategy of the pop-up see figures 2 and 3.
Figures 2 and 3: Scarcity and social proof pop-up
22 other design issues such as shape, color, fonttype, fontsize, and so on, the pop-ups are kept equal.
A pop-up will be shown four seconds after a visitor enters the targeting algorithm of a specific pop-up. For a scarcity pop-up the targeting algorithm is entered when the visitor navigates to a product page at which a scarce product is displayed. For a social proof pop-up, the targeting algorithm will be activated when the visitor navigates to a product page at which a popular product is displayed. The delay of four seconds is incorporated to enhance the attention attracting character of the pop-up, otherwise the pop-up will appear together with the whole new page and people may lack the cognitive ability to process the pop-up.
3.3 Implementation of pop-ups
To attain optimal results, a full factorial field experiment at which all possible combinations of pop-ups are showed at random variation, while the environmental situation is the same for everyone would be best. This is however not feasible considering the fact that it is a field experiment.
It turned out that four websites were willing to cooperate within this experiment, each offering the possibility to test one scarcity message and one popularity message. The design factors of those pop-ups were allowed to vary between those two messages. The total amount of
possible pop-up variations able to test is thereby limited to eight, which is less than expected. To be able to at least measure the main effects of the independent variables some adjustments had to be made. This is accomplished in two ways: by reducing the amount of levels of the independent variables and by using a fractional factorial approach instead of a full factorial approach.
23 As a consequence, the proposed relationships in hypothesis 2a and hypothesis 3a will not be
subject of study in this research.
The variables and their levels which eventually are used in this research can be seen in table 2 below. This leaves 2 x 2 x 2 x 2= 16 possible combinations of pop-ups.
Attribute Strategy (A) Position (B) Exposure duration (C)
Animation type (D)
Level 1 (1) Social proof Top left Medium (8 sec) Slide
Level 2 (-1) Scarcity Bottom left Short (4 sec) Fade
Table 2: Variable levels included in this research
When the full factorial design is carried out, all main effects and interaction effects can be estimated. By using a fractional factorial design and thus showing a fraction of the full factorial design, some effects cannot be estimated separately but will be confounded with other effects. In order to estimate at least the 4 main effects, a resolution IV design is needed. The resolution of a design indicates to which extent variables are confounded. With a
resolution IV design main effects can be measured, without being confounded with each other or with two-way interactions (Bell, Ledolter and Swersey, 2009). To obtain a resolution IV design with four independent variables of two levels each, a minimum of eight pop-ups is required. This is the amount of pop-ups available for this research (Holland and Cravens, 1973).
24 The pop-ups which are used in this research are shown in table 3.
Variable Strategy (A) Position (B) Exposure duration (C)
Animation type (D = ABC) Pop-up 1 Social proof (1) Top left (1) Medium (8 sec) (1) Slide (1)
Pop-up 2 Social proof (1) Top left (1) Short (4 sec) (-1) Fade (-1)
Pop-up 3 Social proof (1) Bottom left (-1) Medium (8 sec) (1) Fade (-1)
Pop-up 4 Social proof (1) Bottom left (-1) Short (4 sec) (-1) Slide (1)
Pop-up 5 Scarcity (-1) Top left (1) Medium (8 sec) (1) Fade (-1)
Pop-up 6 Scarcity (-1) Top left (1) Short (4 sec) (-1) Slide (1)
Pop-up 7 Scarcity (-1) Bottom left (-1) Medium (8 sec) (1) Slide (1)
Pop-up 8 Scarcity (-1) Bottom left (-1) Short (4 sec) (1) Fade (-1)
Table 3: Fractional factorial design of pop-ups
By creating the pop-ups this way, it is ensured that the design is balanced and orthogonal. Balanced means that each level is displayed an equal number of times, in this design each level is displayed two times. Orthogonal means that each level combination is displayed an equal number of times, with each level combination appearing two times in this design, preventing correlation between attributes (Holland and Cravens, 1973). Although the factorial design itself is balanced and orthogonal, we have no influence on the amount of visitors who enter the targeting algorithm of a specific pop-up. As a result, the amount of times each specific pop-up is exposed can differ per pop-up. This is considered a limitation of the research design.
25 All the main-effects of the independent variables are thus confounded with three-way
interactions of the other variables. Due to the following two principles, there is assumed that the main-effects still can be measured, because three-way interactions are said to be negligible in most cases:
- Hierarchical ordering principle: lower order interactions tend to be larger than a higher order interactions (Bell, et al., 2009; Yu, 2013).
- Effect heredity principle: An interaction is significant only if one or both of its main factors are significant (Yu, 2013).
The social proof pop-ups are randomly assigned to the four different websites, the scarcity pop-ups are assigned to the websites based on the assigned social proof pop-ups. It was considered undesirable by Conversify to have two types of pop-ups active at one online shop, differing on more than one design factor. The assignment of the pop-ups to the different websites can be seen in table 4.
Website Pop-ups:
Website 1: Pop-up 1 & pop-up 5
Website 2: Pop-up 2 & pop-up 6
Website 3: Pop-up 3 & pop-up 7
Website 4: Pop-up 4 & pop-up 8
Table 4: Distribution of pop-ups over websites
3.4 Sampling technique
With regard to the sampling technique, some remarks have to be made.
26 The websites participating in this research differ in the type of products they sell (i.e. watches, super foods, toys and sporting weapons) and lay-outs they use, and accordingly are likely to attract different visitors and possibly have different conversion rates. The sample might therefore not be representative for the average online shopper. In attempt to control for those selection bias, every website will have a control group. This means that when a visitor enters the targeting algorithm of one of the two pop-ups active on the specific website, he or she will be randomly assigned to either the treatment group or the control group. People in the
treatment group will be exposed to the relevant pop-up, visitors assigned to the control group do not see a pop-up. Although the pop-ups are presumed to have a positive effect on
conversion, and not showing the pop-up might incur opportunity costs, it is standard
procedure at Conversify to assign about 50% of the visitors to the control group. By adding this control group, already existing differences in conversion rates between the online shops can be controlled for.
Besides the differences between the websites, there are also some important similarities between the websites. The pop-ups are exclusively shown on product detail pages and the product detail pages of all four websites have a white background. This ensures that the pop-ups do not differ in their attention attracting character due to contrast differences(Moore et al., 2005). Additionally, advertising clutter is seen as an important moderator of attention towards persuasive messages, with advertising clutter negatively related towards attention paid to a single advertisement (Pieters et al., 2007). On the product detail pages at which a pop-up is shown, no other advertisements will be present. This goes for all four websites.
27 observations in this period of time. The experiment was planned from november 16th 2015 until november 30th 2015
4.
Methodology
In this section there will be elaborated on data description and data transformation, after that the final sample will be discussed and data insights will be given. Third, a plan of analysis is presented.
4.1 Data description
The data collected and provided by Conversify is at the individual visitor level. There is no information about returning visits available, so every returning visitor counts as a new visitor. Visitors are only included in the dataset if they entered the targeting algorithm of a specific pop-up. Due to the incorporated delay of displaying the pop-ups, visitors that already
navigated to another product page before being exposed to the pop-up are excluded. For each visitor included in the dataset, there is information available about the pop-up number for which the visitor entered the targeting algorithm, if the visitor is assigned to the treatment group or the control group and if the product for which the visitor entered the targeting algorithm is bought or not. Besides this information, which is the minimum information necessary to test the hypotheses, some additional click-stream data is available. Of those click-stream data, “time on site” and “pages viewed” are used as control variables in this research. They are included to control for individual differences. It is assumed that the longer they are on the website and the more pages they viewed, the more likely they are to purchase a product anyway. Both variables however do not indicate the total time on the site or total pages visited during the current session, rather they indicate the time on the site and the pages viewed before they entered the targeting algorithm of the pop-up.
28
Variable Description Levels
Conversion/sale
(Dependent, dichotomous)
Whether the visitor bought the product.
0 = No 1 = Yes
Persuasion strategy:
(Independent, dichotomous)
Whether the visitor saw a scarcity message or social proof message 0 = Scarcity 1 = Social Proof Position: (Independent, dichotomous)
Whether the visitor saw a pop-up on the bottom left location or top left location
0 = Bottom left 1 = Top left
Exposure duration:
(Independent, dichotomous)
Whether the visitor saw a pop-up with a short duration or medium duration. 0 = Short duration 1 = Medium duration Animation type: (Independent, dichotomous)
Whether the visitor saw a pop-up with a fade animation or a slide animation. 0 = Fade 1 = Slide Website conversion: (Control, continuous) Conversion rate of the control group of the website at which the visitor was.
Website 1 = 3,7 Website 2 = 7,5 Website 3 = 3,2 Website 4 = 1,7 Time on site (x100): (Control, continuous) Time in 100 seconds the visitor was already on the website before the pop-up was exposed.
> 0 Pages viewed: (Control, continuous) Number of pages viewed on the specific website before the pop-up was exposed.
> 0
Table 5: Variables used in this research
29 Instead of including the observations of the visitors in the control group, the conversion rate of the control groups of each website is calculated. A new variable “website conversion” is constructed. For this variable, each observation is assigned the conversion rate of the control group of the corresponding website. This new variable enables the possibility to control for existing conversion differences between the websites. By excluding the data of the control groups and creating this new control variable, individual information of the visitors in the control group will be lost. Since the aim of the control group is to correct for website differences, this can be justified.
The variable time on site was initially measured in seconds. As can be seen from table 5, the variable is transformed to a scale of 1/100 to ensure that the estimated parameter of the variable will be interpretable.
4.2 Sample
As a result of technical issues, pop-ups 2 and 6 started at november 18th instead of november 16th, while pop-up 5 started at the third of december. The other pop-ups started at november 16th as intended. To enlarge chances of drawing a sufficient sample it was decided to extend the testing period until the tenth of december, which was the absolute latest due to time constraints. The other 5 pop-ups did run the entire period from november 16th until december 10th.
To reduce possible confounding external variables it would have been best to only use the data of the pop-ups over the period that all pop-ups were active. However, it turned out that for some pop-ups the number of observations is far less than expected and consequently it is decided to use all available information. For pop-ups 1, 2, 4 and 5 the number of observations is below 500 observations per pop-up. This is far below the 2000 observations strived for and will harm the statistical power of the analysis. Since there is no time to gather additional data, the research will be continued with the data available.
30 With regard to the control variables “time on site” and “pages viewed”, no missing values were found. The distribution of this variables is highly skewed, however when assessing the control variables more closely, no values were found which are deemed measurement mistakes and it is decided to keep all observations. The final sample consists of 52.073 observations. An overview of the distribution of the observations over the pop-ups can be seen in table 6.
Table 6: Number of observations and general data
As can be seen from table 6, there are relatively large differences in conversion rates per up, with highest conversion rate (10.5) for up 2 and lowest conversion rate (0.9) for pop-up 4. However, the conversion rates between the control gropop-ups of the websites differ largely as well, ranging from 1.7 at website 4 to 7.5 at website 2. The performance of the pop-ups in terms of conversion rates, cannot be seen separately from the differences in the already present conversion rates of the websites. Furthermore the pop-ups do not consequently outperform the control groups of their websites, with only half of the pop-ups achieving higher conversion rates compared to their own control group.
4.3 Plan of analysis
31 is linked to a cumulative distribution function which ensures that the value of the probabilities stays between 0 and 1 and takes the following form:
Where,
Pi = the probability of each individual
F = the cumulative distribution function Ui = the utility of the individual
Xi = A vector of independent variables for individual i
β = A corresponding parameter vector
The model proposed in this study contains the independent variables used in the experiment as well as the interaction effects between those independent variables. Because the model in this study is an aggregate model, it tries to accommodate for differences between websites and differences in type of visitors by including some control variables. The effects of all the levels of the included variables add up to the total probability that a online shop visitor converts into a shopper.
Where,
Pi = the probability that a visitor buys the product for which the persuasive message is shown.
α = Constant
PS = Persuasion strategy PO = Position
ED = Exposure duration AT = Animation type
INT1 = Persuasion strategy x position = exposure duration x animation type INT2 = Persuasion strategy x exposure duration = position x animation type INT3 = Persuasion strategy x animation type = position x exposure duration WC = Website conversion
32 As can be seen in the proposed model, three interaction parameters are included. Those three interactions measure the confounded effects of all six, two-way interactions.
After estimation of the model, it will be checked if the estimated model suffers from
multicollinearity issues. Multicollinearity is present when some predictor variables are highly intercorrelated. Multicollinearity can lead to inaccurate estimation of the coefficients, while standard errors are likely to be inflated (Malhotra, 2010). A common way to test for
multicollinearity is to check the Variance Inflation Factor (VIF) scores. When the VIF-scores are below 5, multicollinearity is considered not an issue.
When possible multicollinearity issues are solved, the model fit will be checked. After
estimation of the complete model, possible insignificant variables will be excluded one by one and the model will be estimated again. To compare the estimated models and assess the model fit, it is important to look at the Cox and Schnell R2 and Nagelkerke R2, which indicate model performance. Besides looking at those R-squares, information criteria such as Akaike
Information Criterion (AIC) and Bayesian Information Criterion (BIC) need to be considered. Those information criteria penalize for complex models, favoring simple models when
addition variables do not increase model fit enough. All those measures are based on the difference between the log likelihood (LL) of the estimated model (LL(β)) and the log likelihood of the null model (LL(0)) which is the model with only the constant included. The higher the difference in log likelihood between the estimated model and the null model, the better the model predicts and the higher the R-squares. With regard to the information criteria, the lower the scores the better. In this research the BIC will be used since it imposes a larger penalty per additional variable and is recommended when sample size is relatively large.
33
5.
Results
In this chapter, the results of the analysis will be presented. First presence of multicollinearity is tested and resolved. After this, the model fit of the estimated models will be discussed. When the model with the best fit is selected, the parameters of this model will be interpreted. Lastly, some robustness checks will be performed.
5.1 Multicollinearity
When the fractional factorial design is balanced and orthogonal, there will be no correlations between the independent variables and multicollinearity issues should not be present.
However, since the amount of observations per pop-up differs a lot, correlations between independent variables could arise and lead to problems with multicollinearity. Considering the control variables there could arise some multicollinearity problems as well. The control variable “website conversion” only varies over the websites and possibly interferes with the fractional factorial design, while “time on site” and “webpages viewed” could contain similar like information.
It turned out that website conversion is perfectly collinear with the variables position, exposure duration and the interaction between those 2 variables. Because the fractional factorial design is fully occupied, it is decided to solve this problem by sacrificing the interaction effect between duration and position, which is confounded with the interaction between persuasion and animation. By excluding this interaction, it is possible to include the “website conversion” control variable which is deemed to be important due to the large differences between conversion rates of the control groups of the websites.
When evaluating the VIF-scores after exclusion of the interaction between duration and position, VIF-scores for almost all variables are above 5, with some values over 20. Only the control variables time on site and webpages viewed have VIF-scores below 5. This indicates severe multicollinearity problems due to the large differences in observations. When a
34 option involves the disposal of valuable information about individual observations, there is chosen to weight the observations. It must be mentioned that by weighting the observations, the total information of the underrepresented pop-ups becomes equally important to the total information of the overrepresented pop-ups. Because the underrepresented pop-ups actually do not possess enough observations to be a representative sample, this will harm the validity. Since gathering additional data is not possible, this is considered the only option.
If the amount of observations was perfectly balanced, each pop-ups would have (100 / 8 =) 12,5% of the total observations. To determine the weight to be assigned to the individual observations of each pop-up, this percentage must be divided by the actual percentage of observations each pop-up accounts for. The weights assigned to each pop-up can be seen in table 7. Pop-up Total nr of observations Actual percentage of observations (a) Cumulative percentage of observations Weight (12.5/a) 1 158 0.3 0.3 41.20 2 57 0.1 0.4 114.20 3 2462 4.7 5.1 2.64 4 465 0.9 6.0 14.00 5 350 0.7 6.7 18.60 6 10347 19.9 26.6 0.63 7 28279 54.3 80.9 0.23 8 9955 19.1 100 0.65
Table 7: Weighting of observations
After assigning weight to the observations, there is no correlation between the independent variables (see appendix C). The VIF-scores of the included variables after weighting are all below 3 and consequently, multicollinearity is not considered an issue.
5.2 Overall model fit
The complete logistic regression model is overall significant (p = 0.000) and performs with a log likelihood value of -8476.274 better compared to the null model which has a log
35
Table 8: Comparison of the estimated models *p<0.01
When comparing the models based on the model fit measurements, there can be seen that the second model, the model without the insignificant interaction effect, has a log likelihood (-8476,354) comparable with the complete model (-8476.274). The log likelihood of the third model, the model without interaction effects, is somewhat lower (-8484,305) compared to the first and second model. When looking at the Cox and Schnell R2, all models perform equally well 0.035. However, by taking a look at the Nagelkerke R2, there can be seen again that the third model performs a little less well than the first two models with an R2 of 0.115 for the third model and a R2 of 0.116 for the first two models. The BIC accounts for degrees of freedom and tends to prefer simpler models.
5.3 Parameter interpretation
In this section, the parameters of the chosen model will be interpreted. First the main effects are discussed. Second the interpretation of the interaction parameters is given, where after the control effects are elaborated on.
5.3.1 Persuasion strategy
36 can be said that showing a social proof message instead of a scarcity messages decreases the odds of converting a visitor into a buyer.
5.3.2 Position
Due to the limited possibilities in testing different types of pop-ups, the variable position consists of two levels: top left and bottom left. For that reason, H2a which proposed a positive
effect of a pop-up on the left side of the page compared to the right side of the page could not be tested. With regard to H2b which proposed a positive effect of a persuasive message at the
top of the page compared to positioning it at the bottom of the page, there can be said that there is found a significant effect (p = 0.000). However, since the parameter has a negative sign (β = -0.436) and bottom-left is used as reference category, H2b must be rejected.
Showing a pop-up at the top of the page instead of at the bottom of the page thus decreases the odds of being an effective pop-up (Exp(β) = 0.647).
5.3.3 Exposure duration
For the variable exposure duration, only the short and medium duration pop-ups are implemented and hence H3a, which states that a long exposure duration is less effective
compared to a moderate exposure duration, is not tested. The variable exposure duration tested in the model consists of a medium duration level with short exposure duration as reference level. The parameter is significant (p = 0.000) and matches the proposed direction of hypothesis H5b (β = 0.658). H3b is accepted. Exposing the message for 8 seconds instead of
4 seconds increases the odds of converting the visitor into a buyer (Exp(β) = 1.931).
5.3.4 Animation type
With respect to the type of animation used to expose the persuasive message, the parameter shows a negative significant relationship (β = -0.739; p = 0.000). Since fade is used as reference level, this is in contradiction with H4, which assumed a positive effect of a sliding animation in contrast to a fading animation. Accordingly, H4 is rejected. Presenting the message with a sliding animation instead of a fading animation will decrease the odds of being successful (Exp(β) = 0.478).
5.3.5 Interaction effects
37 persuasion strategy by position and exposure duration by animation type. Since the two interactions are confounded, the separate interaction effects cannot be estimated. As all four underlying main effects are significant, it is hard to indicate which interaction effect causes the parameter to be significant. Both interactions might be. Further research should
investigate this.
5.3.6 Control variables
All three control variables are found to be of significant influence and have a positive relationship with sales as was expected (website conversion: β = 0.452, p = 0.000; time on site: β = 0.013, p = 0.000; pages viewed: β = 0.028; p = 0.000). This means that if the visitor visits a website with a higher conversion rate, the odds that the visitor will buy the product will raise with (Exp(β) = 1.571). For time on site counts that with every 100 seconds the visitor is longer on the site before he or she is confronted with the pop-up, the odds of buying the product will increase (Exp(β) = 1.013). The last control variable, pages viewed, indicates that with every additional page viewed before the visitor is confronted with the pop-up, the odds of converting into a shopper increases (Exp(β) = 1.028).
5.4 Model validation
To assess the validity of the results some validity checks will be performed. First the data is analyzed with a probit model to check if the results of the probit model will be similar to the results of the logit model. Second, the decision to not use a validation set is explained.
5.4.1 Probit model
The probit model is very similar to the logit model and hence is expected to produce similar results. To check if this is the case a probit regression will be performed. The probit model takes the following form, where is the cumulative density of a standard normal distribution.
38
Probit model Logit model
Variable β P-value β P-value
Persuasion -0.288 0.000 -0.676 0.000 Position -0.162 0.004 -0.436 0.000 Duration 0.271 0.000 0.658 0.000 Animation -0.314 0.000 -0.739 0.000 Interaction 1 -0.090 0.024 -0.201 0.000 Conversion 0.195 0.000 0.452 0.000 Time on site 0.005 0.006 0.013 0.000 Pages viewed 0.016 0.000 0.025 0.000
Table 9: Comparison probit and logit model
As can be seen, in both models all the included variables are significant at least at the p<0.05 level. Furthermore, the signs of the parameters indicate the same direction of each relationship for both models.
5.4.2 Validation set
To assess validity it is common practice to divide the total sample in an estimation sample and a validation sample. This can be done by randomly taking out a percentage of the total
sample, or by using the data of say the first two weeks for the estimation sample and the data of the last week for the validation sample. Results could have been biased by some
uncontrolled extraneous factors. By testing whether the model estimated with the estimation dataset also holds for the validation dataset, the validity of the estimated model can be checked. Dividing the sample in two subsets will lead to fewer observations available for the estimation of the model, which in turn will harm the power of the estimation. Since the amount of observations for several pop-ups is already far less than the amount strived for, there is decided to not divide the sample in two subsets.
39 a products scarcity when they know many parents are shopping at the same time. It could be as well that people might buy the product anyway, since it is Sinterklaas, regardless of the pop-up is exposed or not. To get a grasp of the effects of Sinterklaas, daily conversion rates of pop-up 6 and the control group of website 3 are plotted in figure 4. Only pop-up 6 is plotted since the amount of observations for pop-up 2 is too small to form a meaningful graph.
Figure 4: Daily conversion rates pop-up 6 and control group website 3
At the end of the testing period conversion rates of both pop-up 6 and the control group decline, so Sinterklaas is probably the reason for the high overall conversion rates of the website and pop-up 2 and 6. The graph shows a small increase in performance of pop-up 6 compared to the control group between november 28th and december 2nd, however additional data is needed to see if this was due to Sinterklaas. Further testing of validity should be done in a follow up study.
6.
Discussion
In this chapter, the results of the previous section will be discussed. Second the limitations and future research ideas are discussed. Finally there is elaborated on the theoretical and managerial implications of this study.
This research was focused on answering the question whether persuasion strategy and design factors influence the effectiveness of persuasive messages. To answer this question, a
40 various persuasive pop-ups to web shop visitors. As results show, differences in persuasion strategy and design factors can significantly influence conversion rates.
Before discussing the results, it has to be mentioned that the findings are probably strongly biased. In the limitations section there will be elaborated on this extensively.
6.1 Contribution to literature
This research contributes to the literature by linking the effects of two persuasion strategies and several design factors to actual sales.
With regard to the persuasion strategy, the results indicate a significant better effect for a scarcity message compared to a social proof message. This is in contrast with the findings of Jeong and Kwon (2012), who found a greater persuasive impact of social proof messages. According to them, this could be because people find scarcity messages less credible compared to social proof messages. In this experiment no information is gathered that can explain why people are influenced by certain factors. It could be that exposing the message in the form of a pop-up with a specific amount of products left is perceived as more credible. Besides, the effects of the persuasion strategy are the combined effects of the text and the picture. In this research, it is assumed that the different effects between the strategies is mainly due to the textual difference. This assumption might be wrong. Further research should investigate this issue.
In previous research design factors are already linked to enhanced attention and purchase intention, suggesting influence on actual sales. The findings of this research show that
41 The only finding in this research which shows a significant effect in accordance with the hypothesis, is duration. For duration it is found that a medium duration has a positive effect on the effectiveness of the persuasive message compared to a short duration which is in line with the findings of Wang et al. (2013). They found an inverted U-shaped relation between exposure duration and attitude towards the advertisment. Since in this research only short and medium duration of exposure are included, we cannot say anything about the form of this relationship.
6.2 Managerial implications
From a managerial perspective, the results indicate that customers are sensitive to the content of persuasive messages as well as to how those messages are presented. To persuade visitors, creation of a shopping experience is important and managers should be creative to find best practices. Slight changes, such as exposing a message a few seconds longer, might already significantly influence the effectiveness of the message.
However, even though managers should be creative, they should also carefully test the effects of their attempts to create a shopping experience. Although it is not further investigated in this study, the basic comparison of conversion rates of the treatment groups with the conversion rates of the corresponding control groups, indicate that persuasive messages do not guarantee higher conversion rates. Consequently, managers should take into consideration possible negative effects of their attempts, which might be larger than the positive effects.
In literature more and more attention is paid to individual differences and consequently different preferences. This research shows that an aggregate model can already help to significantly improve results. When showing a persuasive message as in this research, a scarcity message, positioned at the bottom of the page, fading into the screen for 8 seconds, is expected to result in the highest probability of converting a visitor into a shopper.
6.3 Limitations and further research
42 The first limitation is that the online shops participating in this research are included on the basis of being a client of Conversify and allowing the research to be carried out on their websites. The websites differ on many factors and although this research controls for existing conversion rate differences, it is likely that other extraneous factors play a role. This harms the internal validity of this study. Besides,the visitors of those websites cannot be seen as a representative sample of the average online shopper and therefore the generalizability of this study is low. If different types of pop-ups could be randomly shown at one single website, the internal validity of the research will be higher. In case this proposed procedure can be carried out at different online shops, there can be tested if results hold over different types of
websites, which will increase external validity. When only the opportunity exists to test one or two types of pop-ups at a single website, it becomes increasingly important to match those websites on the basis of several key background variables such as type of products, lay-out, conversion rates and so on.
Besides the differences in websites, there are also large differences in observations per pop-up. Not only between websites, also between social proof and scarcity pop-ups differences are large, suggesting that chances of entering a targeting algorithm for a scarcity pop-up are larger compared to a social proof pop-up. As a consequence, at least four pop-ups have insufficient observations to draw any valid conclusions. By weighting the observations, problems with multicollinearity are solved. At the same time, observations of the pop-ups that are
underrepresented become more important, decreasing the validity of the research even more. Weighting the observations would have not been such a problem if the underrepresented pop-ups would have at least two thousand observations. With average online conversion rates of 2 percent and a sample of less than thousand observations per pop-up, the conversion rates of those pop-ups cannot be seen as representative. To solve this problem more observations for the underrepresented pop-ups are necessary. For future research it is suggested to accurately assess the amount of expected visitors entering the targeting algorithm and to adjust the experimental period accordingly.
43 interaction factor had to be sacrificed to be able to control for website differences.
Furthermore, considering the starting problems of several pop-ups, some spare pop-ups would have been worthwhile. For future research, it is therefore suggested to secure some stretch in the design when planning a field experiment.
