The effects of email personalization on the click-through probability
A weakened effect due to feelings of intrusiveness: How message intimacy and trust seals canmitigate feelings of intrusiveness.
By
Pim Fokkens
2
The effects of email personalization on the click-through probability
A weakened effect due to feelings of intrusiveness: How message intimacy and trust seals canmitigate feelings of intrusiveness
By
Pim Fokkens
June 20, 2016
Faculty of Economics and Business, University of Groningen Marketing department
Master Thesis for MSc Marketing: Intelligence & Management
Arthur van Schendellaan 24 9405 AV Assen, Netherlands
0611441164
p.j.fokkens@student.rug.nl S2023180
Company: Catawiki
First Supervisor: Peter Verhoef Second supervisor: Frank Beke
3
Management summary
This paper discusses effectiveness of email personalization, which is measured with the
click-through probability (CTP). Next to the positive effects, this paper especially focuses on
feelings of intrusiveness due to personalization, which subsequently can decrease the
effectiveness. Moreover, the author suggests two variables which are expected to mitigate
feelings of intrusiveness due to personalization: including message intimacy cues and trust
seals in email messages. Three types of personalization will be discussed in this paper:
medium personalization based on personal identifiable information, medium personalization
based on product preferences, and a high personalization which is a combination of the
previous two variants. To test the assumptions, a customer reactivation email of Catawiki has
been manipulated in terms of personalization (4), message intimacy (2) and trust seals (2). In
sum, sixteen different versions of the email have been created. To test the hypotheses, two
studies have been conducted. In study 1, a survey is distributed, where respondents randomly
got to see one of the versions of the email and had to indicate how intrusive they perceive the
email. A total of 202 respondents participated in this research. It was found that high
personalization increases feelings of intrusiveness and that the inclusion of message intimacy
mitigates this effect. In study 2, the different versions of the email have been sent to real users
of Catawiki and the effects of the manipulations on the click-through probability have been
analyzed. It is found that both medium personalization based on personal identifiable
information and high personalization, increases the click-through probability of an email.
4
Preface
After finalizing all the courses of the master marketing, I decided to write my thesis in combination with an internship. I did my internship with Catawiki, which is an online auction house for special objects. At the moment of writing, Catawiki is according to Deloitte (2016) the fastest growing tech-company worldwide. It was exciting to have the experience of being an intern at such a fast growing company. Due to the fact that it was an interesting, educational and very busy internship, it was sometimes a bit difficult to focus on my thesis. However, I am really grateful that I could perform this research at Catawiki. I got the opportunity to set up my own experiment with 16 different versions of a customer reactivation email campaign which eventually has been sent to approximately 60.000 users in six different countries. It is very unique for a company to give an “intern” the opportunity to do such an extensive experiment where real customers are involved. In particular, I want to thank Jeroen Aldenkamp, employee of Catawiki and former MSc marketing student at the RUG, who supported me with setting up this email campaign at Catawiki.
I also want to thank Peter Verhoef for providing me with feedback during the process of writing my thesis. Finally, I want to mention that I really enjoyed the Master marketing at the RUG. Especially the marketing intelligence course, which I found to be an inspiring and challenging course. I do not regret my choice of studying this master and I would definitely recommend this study to others.
5
Table of Contents
MANAGEMENT SUMMARY ... 3 PREFACE ... 4 TABLE OF CONTENTS ... 5 1. INTRODUCTION ... 6 2. THEORETICAL FRAMEWORK... 9 2.1EMAIL PERSONALIZATION ... 92.2CLICK-THROUGH PROBABILITY ... 10
2.3PERSONALIZATION BASED ON PERSONAL IDENTIFIABLE INFORMATION (PI) ... 12
2.4PERSONALIZATION BASED ON PRODUCT PREFERENCES (PR) ... 12
2.5FEELINGS OF INTRUSIVENESS ... 14 2.6MESSAGE INTIMACY ... 16 2.7TRUST SEALS ... 17 2.8CONCEPTUAL MODEL ... 18 3. RESEARCH DESIGN ... 19 3.1MANIPULATIONS ... 20 3.2STUDY 1 ... 22 3.2STUDY 2 ... 23 4. RESULTS ... 24 4.1STUDY 1:SURVEY ... 24 4.1.1 Data exploration ... 24 4.1.2 Specification ... 26 4.1.3 Estimation ... 27
4.1.4 Validation and results ... 27
STUDY 2 ... 29
4.1.1 Data exploration ... 29
4.2.2 Specification ... 32
4.2.3 Estimation ... 33
4.2.4 Validation and results ... 34
5. CONCLUSION AND DISCUSSION ... 35
6
1. Introduction
The relationship between customers and companies is rapidly changing from mass communication to a much more personal approach. New media and technological developments have led to more cost effective interactive communications between companies and customers, and thus cost effective management of personalized relationships with individual customers (Vlasic and Kesic, 2007). In practice these developments give companies the opportunity to use customers‟ information when making any content (Winer, 2009). One of the most frequently used approaches of using customers‟ information is personalization, which is about delivering messages and services which are tailored to the characteristics and needs of an individual customer (Vesanen, 2007). Before further elaboration on the concept of personalization it is useful to first make the distinction between personalization and customization, since many articles are using these definitions interchangeable. The difference lies in the entity that initiates the action. According to Montgomery and Smith (2009), personalization is automated by the marketer on behalf of the customer, whereas customization is requested by a customer on his own behalf. For example, giving customers the possibility to design their own All-stars shoes through the Converse website is customization, whereas advertising with recommended product offerings based on previous website visits of a person, can be defined as personalization (assuming that the customer has not requested for it).
In the context of new media, personalization has to do with providing personalized services based on a profile of a customer which has been formed during previous transactions with a firm (Montgomery and Smith, 2009). Applying this to the field of marketing, you could think of personalized marketing efforts such as personalized advertisements and personalized email messages. In this context, personalization can better be defined as the degree to which interpersonal communication and interaction based on a consumers‟ personal and preference information exists (Song et al, 2016). This article will further elaborate on personalization of marketing efforts.
7
interesting? First of all, customers benefit since they will receive products and services that have a closer fit to their preferences and are more relevant (Vesanen, 2007). Firms can take advantage of that, since targeted marketing enables companies to charge higher prices and make more profits (Chen et al. 2001). Personalized advertisements are also more relevant, have greater fit and reduce search time, which increases the likelihood of customers to respond (Chellappa and Sin, 2005). Furthermore, personalization can also increase purchase intentions (Lambrecht and Tucker, 2013). Moreover, according to Howard and Kerin (2004) consumers are more likely to respond positively to marketers‟ promotional effort when they are exposed to personalized advertisements.
Although many authors have shown the positive effects of personalization, it does not mean that personalization by definition always adds value for companies and consumers. There are some potential negative effects which decrease the effectiveness of personalization. Personalized advertisements can lead to psychological reactance such as feelings of intrusiveness and/or privacy concerns (Van Doorn and Hoekstra, 2013). Moreover, White et al (2008) states that consumers respond less favorable to emails when they are highly personalized and when there is no justification present. Van Doorn and Hoekstra (2013) describe the presentation of personalized advertisements as a double-edged sword, since it will probably lead to higher purchase intentions, but these advantages will be lower due to increased feelings of intrusiveness which subsequently has a negative effect on purchase intentions. Thus, when considering personalization, one should always take into account that, next to the many positive effects, it also has some potential downsides.
8
Since the amount of personal information available for companies is likely to keep increasing and technological advancements will make personalization even more accessible, it is interesting to get more insights in how to further optimize it. When it becomes more evident what the positive and negative effects of personalization are, and especially how the negative effects can be reduced, companies can further improve their marketing campaigns. Especially the point of getting more insights into which factors can mitigate the negative effects of personalization has not been discussed frequently in the literature. Hence, this paper can be a highly valuable contribution to the existing literature.
Two variables are considered to reduce feelings of intrusiveness due to personalization: The presence of message intimacy and/or trust seals in email messages. Further elaboration on these concepts will follow in the next chapter. The aim of this paper is summarized with the following research question:
“What is the effectiveness of email personalization, and how to improve it by minimizing the negative effects due to increased feelings of intrusiveness?
This research questions will be answered with the following sub questions: 1) What is the effect of email personalization on the effectiveness of an email? 2) Does email personalization lead to increased feelings of intrusiveness? 3) Do feelings of intrusiveness decrease the effectiveness of an email?
4) If personalization causes higher feelings of intrusiveness; which factors can mitigate this effect?
9
2. Theoretical framework
As mentioned in the previous chapter, this paper will focus on personalization of email messages in the field of marketing. Not only the positive effects will be points of interest, but also how the effectiveness can decrease due to feelings of intrusiveness caused by personalization and how these negative effects can be mitigated. This chapter provides an overview of the literature about the concepts concerned in this research. Based on these insights, numerous hypotheses have been formulated. In §2.1 the general concept of email personalization will be discussed. In §2.2 there will be an explanation of the so called „click-through probability‟, which is a common metric of measuring an email‟s success. The subsequent two sections will elaborate on two types of personalization: §2.3 will cover the concept of personalization based on personal identifiable information, where §2.4 covers personalization based on product preferences. §2.5 will go deeper into the negative effect of personalization, namely increased feelings of intrusiveness. In §2.6 the concept of message intimacy, which is expected to decrease feelings of intrusiveness due to personalization, will be discussed. Moreover, §2.7 will explain why the presence of trust seals in email messages, is also expected to decrease feelings of intrusiveness due to personalization. Finally, in §2.8 the conceptual model will presented.
2.1 Email personalization
10
Before going deeper into consequences of email personalization, it is useful to first get deeper into the definition of email personalization. Postma and Brokke (2002) define personalization as a specialized stream of communication that sends individual recipients different messages which are tailored to their individual characteristics and preferences. More recently, Song et al. (2016) defined personalization as the degree to which interpersonal communication and interaction based on a consumers‟ personal and preference information exists. Applying these definitions to the specific case of email marketing, it would mean that every recipient gets a unique email tailored to the individual needs, based on personal characteristics and preferences. All these definitions share two important aspects of personalization. Firstly, email personalization requires the use of individual personal and/or preference information. Secondly, email personalization is about shaping a message such that the content is tailored to the needs and preferences of the recipient.
In line with research conducted by of Sunil et al, 2016, this paper will consider two different types of personalization. These types differ in what sort of information is required, which subsequently determines the degree to which the communication is unique for every recipient (White et al, 2008). For example, the use of transactional data or the name of a customer has a higher distinctiveness than using a customers‟ browsing data, which can be relevant to a larger group of customers (Van Doorn and Hoekstra, 2013). The first type of personalization being discussed has to do with personalizing by the use of personal identifiable information (Sunil et al, 2012). This could be performed by using information such as the recipients‟ name or other information which is unique for the individual. The second type of personalization has to do with personalization based on product preferences, which practically means that the recipient gets product recommendations which are based on revealed preferences during previous transactions with the firm (White et al, 2008). These two types of personalization will be discussed more extensively in sections 2.3 and 2.4. However, before going deeper into the two types of personalization, there will first be an explanation of the click-through probability, which is the metric being used to measure the effectiveness of an email.
2.2 Click-through probability
11
into the percentage of people opening an email, after having received it. In fact it measures the effectiveness of the subject line and initial interest in the email (Goldie, 2006). Moreover, it could also be an indication of well performed targeting (Maccario, 2006). However, although the open-rate can give really valuable insights, it is quite limited as it does not measure the effectiveness of the actual content in the email.
Hence, in this paper the click-through probability (CTP) will be used to assess the effectiveness of an email The CTP is the probability that a recipient clicks on an item when displayed (Cron et al., 2014). In the case of email messages, it would mean that the CTP is the probability of a recipient clicking on a link in an email after having opened it. The CTP is based on the click-through rate (CTR), which is the percentage of recipients that clicks on a link in an email after having opened it. Both definitions have the same essence, but the CTP focuses on the individual level, where the CTR focuses more on a general level. In the remainder of this article the term CTP will be used. Some people calculate the CTP by taking the percentage of clicks relative to the number of emails that has been sent. However, in that case the metric is basically a combination between the open-rate and click-rate of an email. Therefore, in this research the CTP will be interpreted as the probability that a recipient clicks through the email after having opened it.
12
2.3 Personalization based on personal identifiable information (pi)
One way to personalize email messages is by the use personal identifiable information, which is information that is unique for every recipient, such as the recipient‟s name or transactional information (Sunil et al, 2012). It is an explicit way of showing a personalized email message, which means that the recipient should immediately be aware of the fact that personal information has been used (Sunil et al, 2012). Now, the question arises, why this type of personalization adds value for the recipient. According to Tam and Ho (2006) the inclusion of self-referent concepts in a message such as using the customer‟s name, will lead to better recall and an increased likelihood of accepting the offer. Moreover, Levy and Weitz (1992) also found that by using a customer‟s name in the offline marketing context, the probability of accepting an offer will increase. Betrand et al. (2005) found that personalization by using personal identifiable information does not have direct economic benefits for the recipient as it does not lead to services or product offerings that closer fits to the recipient‟s preferences. However, the perceived benefits are associated with the psychological cues in the email (Bertrand et al. 2005). Therefore, it is expected that the increased perceived benefits due to the psychological cues in the email will eventually increase the click-through probability. Based on these insights the following hypothesis has been formulated:Hypothesis 1a: Medium personalization (personal identifiable information) of email messages will have a positive effect on the click-through probability of an email, relative to the situation where no personalization takes place.
2.4 Personalization based on product preferences (pr)
13
message: “We think you are interested in the following products.” In this study we will only consider the implicit use of product based personalization.
Product based personalization has, in contrast to personalization with personal identifiable information, the purpose of increasing the economic value of the offer. First of all, this economic value comes from a reduction in search costs and uncertainty, which could prevent customers from taking any action (Chen and Wu, 2004) Moreover, Lambrecht and Tucker (2013) found that product personalization may also increase purchase intentions. In addition, Tam and Ho (2006) found that users are more willing to accept an offer if it contains relevant content. Moreover, Ansari and Mela (2003) found that customizing links within an email message can potentially improve click through rates by 62%. Therefore, the increased economic benefit due to personalization is expected to positively influence the click-through probability of an email. Based on these insights the following hypothesis has been formulated:
Hypothesis 1b: Medium personalization (product based) of email messages will have a positive effect on the click-through probability of an email relative to the situation where no personalization takes place.
It is also expected that the combination of personalization based on personal information and product based personalization will have a positive effect on an email‟s CTP. This type of personalization will be labeled as: “High personalization‟. The following hypothesis has been formulated for this variant:
Hypothesis 1c: High personalization of email messages will have a positive effect on the click-through probability of an email relative to the situation where no personalization takes place.
In sum this article will distinguish between three types of personalization:
Medium personalization (pi): By the use of personal identifiable information.
Medium personalization (pr): By the use of product preferences.
14
2.5 Feelings of intrusiveness
Although personalization has proven to be effective, there is increasing interest and focus in the literature on the negative effects of personalization. Since personalization is company initiated (Montgomery and Smith, 2009), it is not surprising that customers will not perceive it by definition as something positive. In some cases, personalized advertising demands an unwanted level of personal information, which eventually causes the advertisements to be perceived as ‟too personal‟ (White et al. 2008; Goldfarb, 2011). According to Van Doorn and Hoekstra (2013) the use of information that has great distinctiveness, due to the use of personal and/or transactional information, will increase feelings of intrusiveness and eventually will decrease purchase intentions. Other papers are using slightly different definitions to describe the negative effects of personalization. In the paper of White et al. (2008) the negative effect is defined as „reactance‟, which has to do with the psychological resistance to a perceived inappropriate personalization. Others papers describes the negative effects as „privacy risk‟ (Song et al., 2016; Xu et al., 2011; Chellappa and Sin, 2005). Although there are many different definitions to describe the negative effects of personalization, they share one common aspect; there is some sort of psychological reactance, due to the fact that personal information has been used without explicit permission by the customer.
This paper will focus on feelings of intrusiveness due to personalization. In the psychology literature the definition of intrusiveness is as follow: “Creating an imbalance between closeness and autonomy” (Lavy et al. 2009). In the context of email marketing, closeness has to do with the degree to which both parties (sender and recipient) are related and interdependent, whereas autonomy is about the extent to which the recipient‟s identity can be sustained. Morimoto and Chang (2006) look at it from a consumer privacy perspective and describe intrusiveness as an intrusion of a person‟s solitude and privacy.
15
email, whereas banners just show up on the internet and are therefore by definition more intrusive. However the effect of personalization should be applicable for both situations, since the recipient only notices the personalization after opening the email. In other words, there was no initial intention to be exposed to a personalized message. Moreover, Iacobucci (2006) also found that personalized recommendations can lead to a decreased customer satisfaction, annoyance and irritation. Based on all these insights it is expected that personalization of email messages will cause feelings of intrusiveness. Therefore, the following hypotheses have been formulated:
Hypothesis 2a: Medium personalization (pi) of email messages causes higher feelings of intrusiveness, relative to the situation where no personalization takes place.
Hypothesis 2b: Medium personalization (pr) of email messages causes higher feelings of intrusiveness, relative to the situation where no personalization takes place.
Hypothesis 2c: High personalization (pi + pr) of email messages causes higher feelings of intrusiveness, relative to the situation where no personalization takes place.
After elaborating on why personalization is expected to cause feelings of intrusiveness, the question arises what the effect of feelings of intrusiveness will be on the CTP of an email. According to Stafford and Faber (2015), advertisements which are perceived as disturbing can lead to negative consequences. For instance, it can result in irritation and avoidance of the advertisement. In the case of an email messages, avoidance would mean not clicking through the email. Moreover, Morimoto and Macias (2009) also found that an email which is perceived as intrusive will lead to stronger reactive and eventually to a negative perceptions and avoidance. Finally, Van Doorn and Hoekstra (2013) found that intrusiveness decreases the advertisement‟s effectiveness in terms of purchase intentions.
Based on all these findings it is expected that feelings of intrusiveness will negatively influence the CTP of an email. Therefore, the following hypothesis has been formulated:
16
2.6 Message intimacy
Next to the positive and negative effects of email personalization, this paper also focuses on how to decrease feelings of intrusiveness caused by personalization. Two factors are considered to weaken the effect; „message intimacy‟ and „trust seals‟. This section will elaborate on the inclusion of message intimacy in emails, where the next section will discuss why the inclusion of trust seals is expected to mitigate feelings of intrusiveness.
There is a growing distance between companies and its customers due to the fact that communications go mainly through online media such as email. This distance leads to the problem that customers are not anymore aware of how firms are using and collecting personal information (Premazzi et al (2010). Moreover, it is reasonable to think that customers feel more apprehensive when their personal information is used by an unknown person relative to someone familiar (Posner, 2008). Furthermore, according to Vlasic and Kesic (2007) people are more willing to disclose personal information to companies if they use personal communications, in a sense that they communicate directly with an employee of the company. Moreover, consumers‟ willingness to disclose personal information for the sake of personalization increases with trust, and personal interest (Dinev and Hart, 2006). These insights suggest that customers are less concerned about the use and collection of personal information if they know more about the opposite party. In other words, they are less concerned about their personal information if they interact with a real person instead of a unknown entity.
17
emotional cues or by sending the email in the name of a specific employee, opposed to using the company name. Based on these insights, the following hypotheses have been formulated:
Hypothesis 4a: The presence of message intimacy cues in an email message will decrease feelings of intrusiveness due to medium personalization (pi).
Hypothesis 4b: The presence of message intimacy cues in an email message will decrease feelings of intrusiveness due to medium personalization (pr).
Hypothesis 4c: The presence of message intimacy cues in an email message will decrease feelings of intrusiveness due to high personalization.
2.7 Trust seals
Another variable that is expected to weaken feelings of intrusiveness due to personalization is the presence of trust seals. Trust seals have the purpose to ensure customers that a particular e-retailer is trustworthy (Cook and Wenhong, 2003). Examples of trust seals are seals granted by external parties such as Trustpilot and Truste certified privacy. These seals are signals of trust, which can be placed by an e-retailer on multiple communication channels such as the website or in email messages, in order to be perceived as more trustworthy. These trust seals can be very important, since trust is an important factor in e-commerce as people perceive higher degrees of uncertainty and risk on the internet (Huh & Shin, 2014). Regarding the use of personal information for personalization; prior research states that consumers‟ willingness to disclose personal information increases with personal interest and trust (Dinev and Hart 2006). Moreover, according to Culnan and Bies (2003) individuals are more likely to disclose personal information if they perceive that the benefits are at least equal, or higher than the perceived risks of disclosure.
18
email‟s sender, it is likely that recipients have less resistance to the personalization and will perceive it as less intrusive. Based on these insights the following hypotheses have been formulated:
Hypothesis 5a: Including seals of trust in an email message will decrease feelings of intrusiveness due to medium personalization (pi).
Hypothesis 5b: Including seals of trust in an email message will decrease feelings of intrusiveness due to medium personalization (pr).
Hypothesis 5c: Including seals of trust in an email message will decrease feelings of intrusiveness due to high personalization (pi + pr).
2.8 Conceptual model
All the hypotheses have been visualized in this conceptual model. The model shows all the expected relationships with regard to the direction and the valence of the effects.
19
3. Research design
The hypotheses will be tested by conducting an experiment. This experiment will be done at the company Catawiki, which is an online auction house for special objects. Multiple versions (16) of a customer reactivation email will be created and will be manipulated in the degree of personalization, the presence of message intimacy and the presence of trust seals. The purpose of the email is to reactivate customers who have not been active on Catawiki for at least 45 day. The content of the email is to encourage the recipients to become active again on Catawiki. They will be offered a voucher-code with the amount of €20 which can be redeemed after winning an auction at Catawiki.
Before explaining how the email is manipulated, the research design will be explained first. With the use of the manipulated emails, two studies have been conducted:
In study 1, a survey has been distributed where participants randomly got to see one of the sixteen versions of the email and had to evaluate it on the degree of intrusiveness. The goal of this study is to test the effects of the manipulations on perceived feelings of intrusiveness.
In study 2, the sixteen versions of the email have been sent to real inactive users of Catawiki. The different versions were randomly distribution over the recipients. The goal of this study is to measure the effectiveness of the emails in terms of CTP.
20
intrusiveness affects the CTP on an individual level. The most ideal situation would be to send the email to real customers of Catawiki and subsequently ask them how they perceive the intrusiveness of the mail. However, this would be too complex from a practical perspective and it would also decreases the size of the dataset, since it is not feasible to ask thousands of users about their feelings of intrusiveness.
Since the actual data of intrusiveness cannot be linked to data of the real life dataset, it is not possible to prove the effect of intrusiveness on the CTP of an email. This is also one of the limitations of this study. However, when the direction of the effect corresponds between both studies, it is likely that intrusiveness has an effect on the CTP.
Before further elaboration on the two studies individually, it will first be discussed how the emails have been manipulated. These manipulations will be used in both studies.
3.1 Manipulations
The email has been manipulated in the degree of personalization, the presence of message intimacy and the presence of trust seals. Hence, it will be a 4 (personalization) x 2 (Message intimacy) x 2 (Trust seals) between subjects design. This means that every participant (in both study 1 and 2) will be assigned to only one of the sixteen conditions. Moreover, for the purpose of study 2, the emails will also be written in six different languages; English, Dutch, German, Spanish, Italian, and French, as the email will be sent to users in these countries. Thus, in total there will be 96 different versions of the email.
An explanation of how the email has been manipulated will follow now. For a more detailed explanation of how these manipulations are conducted, see Appendix 1a. For actual examples of the manipulated emails, see Appendix 1b-e.
Personalization
21
medium personalization (pr) condition includes four recommended auction lots. It shows four objects which are being auctioned in the corresponding week. These auction lots will be selected from the category were the users has placed the most bids on in the past. Fourthly, the high personalization condition represents all personalized elements of the previous two variants.
Message intimacy
Two levels of Message intimacy have been manipulated; a condition where message intimacy is present and a condition where message intimacy is not present. Firstly, the condition where message intimacy is present contains some psychological cues, which to a certain extent represents social presence. In practice the social presence has been simulated by pretending that the email is written by an auctioneer of Catawiki. It includes the text: “Our auctioneers have asked me to offer the following”, whereas the no message intimacy variant contains the following text: of “We would like to offer you the following”. Moreover, the closing of the email in the message intimacy variant has been written in the name of a specific Catawiki auctioneer. The name and photo of the auctioneer will be shown. In contrast, the version without message intimacy closes the email with: “Kind regards, Catawiki”. Moreover, the message intimacy variant contains a picture of smiling auctioneers above the email, whereas the version without message intimacy shows a picture of some general auctionable objects.
In addition, the email which includes message intimacy contains an emotional cue. It has the text: “We‟d love to see you again!”, whereas the version without message contains the following text instead: “To help you get going”.
Trust seals
22
3.2 Study 1
The aim of study 1 is to find out whether the manipulated emails score different in terms of perceived intrusiveness. Friends and relatives of the author have been invited to participate in a survey. These people are not by definition users of Catawiki. A visual representation of the relations which will be measured in this study can be found in figure 2.
Figure 2: Model study 1
Participants were asked to image themselves to be a user of Catawiki and have been inactive for a while. Subsequently they will see one of the sixteen versions of the email (Appendix 1). Participants will randomly be assigned to one of the sixteen conditions. Based on the email participants see, they have to indicate to what extent they perceive feelings of intrusiveness due to the email. The complete survey is presented in Appendix 2.
Perceived feelings of intrusiveness have been measured by using a 10 items construct, which is also used by Van Doorn and Hoekstra (2013), who combined the constructs of two other papers (Edwards et al., 2002; Mooradian, 1996). The items will be measured on a 7 points Likert scale.
23
Van Doorn and Hoekstra (2013) found that people with high privacy concerns perceive an ad as more intrusive.
At the end of the survey, two questions will be asked to the recipient with the purpose of ensuring that the participant has read the survey instructions carefully (Appendix 2).
3.2 Study 2
In study 2, the different versions of the email have been sent to real users of Catawiki. The different versions of the email have been distributed randomly over the recipients. Figure 3 gives a visual overview of the relations being measured in study 2.
Figure 3: Model study 1
The sixteen versions of the email have been sent to approximately 60,000 inactive users of Catawiki, which means that they have not visited the website of Catawiki for at least 45 days. The email has been sent to users in 6 different countries; The Netherlands, Germany, France, Italy, Spain and The United Kingdom. Therefore, the emails are translated into the language of the corresponding countries. Thus, it can be assumed that recipients of the email get the email in their native language.
24
4. Results
4.1 Study 1: Survey
4.1.1 Data explorationFrom the 202 participants of the survey, 181 eventually also completed the survey. Moreover, 23 respondents did not answer question 7 correct, which is a check whether the respondent has read the survey instruction carefully (Appendix 2). A one-way ANOVA indicates that respondents who answered question 7 incorrect, significantly differs with regard to the perceived feelings of intrusiveness (F=4.238, p=0.041), relative to the group who answered question 7 correct. Therefore, these respondents have been excluded from the dataset. The intrusiveness has been calculated by averaging the 10 items of intrusiveness. Moreover, as the variable does not have any theoretical relevance, excluding these respondents is justified.
Since the participants were assigned to one of the sixteen versions of the email, the responses of each individual respondent are linked to a single version of the email. However, for the purpose of analyzing these results, the versions had to be recoded in order to get the data for the independent variables (personalization, message intimacy, and trust seals). Since there were four variants of personalization, three dummy variables have been computed for personalization, where the baseline will represent the no-personalization condition. Moreover, a dummy variable is created for both message intimacy and trust seals. Table 1 represents all the variables with regard to the manipulations.
Variable Definition Descriptives
MP_PI The presence of medium personalization (pi) 0= not present 1= present
N(0)= 111 N(1) = 47 No missing values
MP_PR The presence of medium personalization (pr) 0= not present 1= present
N(0)= 125 N(1) = 33 No missing values
HP The presence of high personalization 0= not present 1= present
N(0)= 119 N(1) = 39 No missing values
Messageintimacy The presence of message intimacy 0= not present 1= present
N(0)= 87 N(1) = 71 No missing values
Trustseals The presence trust seals 0= not present 1= present
N(0)= 78 N(1) = 80 No missing values
25
Since intrusiveness has been measured with a 10 scale construct, it is needed to test the reliability of this scale. With a Chronach‟s alpha of α= 0.947, the scale turned out to be highly reliable. However, the Chronbach‟s alpha if item 8 is deleted will be slightly higher α= 0.954 (Appendix 3.1). The squared multiple correlation of item 8 (.332) was also relatively low compared to the other items. Therefore, this item will be excluded for further analysis and a factor analysis will be performed with the remaining nine items. The results of the factor analysis were as follows. The Kaiser-Meyer-Olkin test (KMO) of 0.903 is sufficient (> .5) and the Bartlett's Test of Sphericity is significant (p= .000). This means that the variables in this construct significantly correlates with each other and can be reduced to one factor. This single factor explains 73.46% of the variance of this construct and has been named: Intrusiveness
Next to the variables from the conceptual model, there are also six control variables included which are considered to have an effect on the outcomes. These are presented in the table below.
Variable Definition Descriptives
1. Age The age of the respondent Average: 30 years old Range: 16-60 years
No missing values or outliers
2. Gender Gender of the respondent
0= Female 1= Male
Female: 37% Male: 63%
No missing values or outliers
3. Native language 0= no 1= yes No: 11% Yes: 89%
No missing values or outliers
4. Privacy concerns Construct of privacy concerns Mean centered values
No missing values or outliers
Table 2: Control variables
5. General privacy concerns
26
Sphericity was significant (p= 0.000). Therefore, it can be concluded that variables in this construct significantly correlates with each other. One factor has been created, which explains 74.36% of the variance within this construct. The new factor has been named: Privacyconcerns.
6. Native language
The survey has been distributed in both English and Dutch. Since respondents were also asked to fill in their nationality, it is assumed that if these values do not correspond, the manipulated email is not written in the respondent‟s native language. Since it could be the case that intrusiveness will be perceived different when reading the email in a foreign language, a dummy variable has been created which indicates whether the email is written in someone‟s native language or not. This variable has been labeled as: Nativelanguage¸ where a value of 0 indicates that the email is not in the respondent‟s native language, and 1 indicates the opposite.
4.1.2 Specification
Based on the variables from the conceptual model and the inclusion of the control variables, a model has been specified. In addition to all the main effects, the following interaction effects are also included:
Interaction effects MP_PI Interaction effects MP_PR Interaction effects HP MP_PI * Message intimacy MP_PR * Message intimacy HP * Message intimacy
MP_PI * Trustseals MP_PR * Trustseals HP * Trustseals
MP_PI * Privacyconcerns MP_PR * Privacyconcerns HP * Privacyconcerns
Table 3: Interaction effects
27
4.1.3 Estimation
To test the specified model, a multiple regression analysis has been performed. As the table below indicates, there are five mains effects from the conceptual model, four control variables, and nine interaction effects included in the regression. The results of the regression are presented in the table below. In appendix 3.2 the actual SPSS output can be found.
Parameter β Std Error VIF
Constant -.364 .410 Main effects: MP_PI .291 .336 4.984 MP_PR -.254 .329 3.787 HP .575* .342 4.616 Messageintimacy -.107 .283 4.197 Trustseals -.166 .288 4.386 Control variables: Age -.008 .007 1.280 Gender -.351** .160 1.254 Privacyconcerns .089 .153 4.945 Nativelanguage .725** .221 1.040 Interaction effects: MP_PI * Messageintimacy .096 .385 3.090 MP_PR * Messageintimacy .544 .430 3.162 HP * Messageintimacy -.850** .399 3.242 MP_PI * Trustseals .210 .379 4.047 MP_PR * Trustseals .514 .422 3.037 HP * Trustseals .477 .406 4.018 MP_PI * Privacyconcerns .353* .215 2.322 MP_PR * Privacyconcerns .356* .204 2.517 HP * Privacyconcerns .390* .208 2.214
** significant at p <.01 * significant at < .05 Dependent variable: Intrusiveness
Table 4: results multiple regression analysis
4.1.4 Validation and results
28
penalizes for including parameters in the model is .257, which is much lower, since the model contains 19 parameters. However, since it is not a predictive model, it is not necessary to make the model smaller by excluding variables from the model. Whether an R-squared of .342 can be interpreted as sufficient is quite subjective. Nevertheless, it is at least an indication that the model really does have some explanatory power.
It is also required to test the model for multicollinearity, which occurs when independent variables are highly correlated. This could lead to unreliable estimates of the parameters, since the variance will be estimated too large in such a case. Multicollinearity can be detected by looking at the variance inflation factor (VIF). A VIF score higher than 5 indicates that multicollinearity is an issue. However, as can be seen in table 4, none of the variables has a VIF score higher than 5. Therefore, it can be concluded that multicollinearity is not an issue.
Since the model has been evaluated with respect to the explanatory power and multicollinearity seems to be no issue, the parameters should be given an interpretation. First of all, it turned out that high personalization has a significant positive effect (β3= .575 p
=0.095) on perceived feelings of intrusiveness. This means that an email with high personalization increases feelings of intrusiveness relative to the situation of no personalization. This is in line with hypothesis 2c. With regard to medium personalization (pi) and medium personalization (pr), no significant effects on intrusiveness have been found (β1=
.291 p= 0.387 and β2= -.254 p= 0.442). Moreover, message intimacy and trust seals both have
no direct effect on intrusiveness, which is in line with the expectations. However, it turned out that there is a significant negative interaction effect between high personalization and message intimacy (β8= -.850 p= 0.035). This means that when an email is highly personalized,
the inclusion of message intimacy decreases the perceived intrusiveness due to high personalization. This finding confirms hypothesis 4c. It is even the case that this negative effect is stronger than the direct positive effect of high personalization on intrusiveness. This means that the combination of high personalization and message intimacy, leads to an even lower degree of intrusiveness compared to a situation where the email contains no personalization. All the other moderation effects regarding message intimacy and trust seals were insignificant.
Furthermore, gender has a negative significant effect on intrusiveness (β16= -.351 p= 0.029).
29
perceive the email in general as less intrusive compared to woman. Moreover, it seems that when the email has been written in the participants‟ native language, the email is perceived as more intrusive (β18= .725 p= 0.001). Finally, the interaction effects between privacy concerns
and all the three types of personalization, is positively significant. This means that higher privacy concerns, leads to higher feelings of intrusiveness, under the condition that the emails contains one if the three types of personalization instead of the no-personalization condition.
Study 2
4.1.1 Data exploration
In total, the email has been sent to 59,745 users of Catawiki. Eventually 16,060 users have opened the email and 1,917 users clicked through the email to go the Catawiki website. Before further exploration of the data, the values for the email manipulations have been recoded to dummy variables. This has been done in the same manner as a study 1 (See table 1).
30
Figure 4: Click-through rate by type of personalization
It is also useful to get more insights into the possible effects of message intimacy and trust seals. First of all, for both variables a one-way ANOVA test has been conducted with regard to the click-through rate. Both tests were insignificant, which is in line with the expectations, since no direct effects were expected.
However, it is expected that message intimacy will positively moderate the effect of personalization on the CTR, since study 1 has shown that the combination of message intimacy and high personalization weakens perceived feelings of intrusiveness. Figure 5 gives an indication of how message intimacy influences the effect of personalization on the CTR. It seems that the inclusion of message intimacy negatively affects the relation between medium personalization (pi) on the CTR and positively affects the relation between high personalization and the CTR.
31
Figure 6 represents a similar graph for the presence of trust seals. It seems that there is hardly any effect of trust seals on the relation between personalization and the CTR. However, these insights do show that the presence of trust seals in the high personalization condition could have a negative influence.
Figure 6: Click-through rate by type of personalization and presence of trust seals.
Next to these insights, it is also interesting to look at the differences between countries. A one-way ANOVA shows that there are significant differences between countries with respect to the click-through rates (F=2.120, p=.008). A post-hoc Tukeys test revealed that there are significant differences between The Netherlands and the United Kingdom. Figure 6 shows the differences between click-through rates per country.
32
Figure 7: Click-through rate by country.
These insights have led to the decision to also include country as a control variable in the model. In sum, five dummy variables have been created. There is no dummy variable created for The Netherlands, as it will serve as the base-line category.
4.2.2 Specification
The dependent variable of the model is the click-through probability of an individual recipient who opened the email. Since it is a „probability‟, a binary logistic regression has been used for estimating the parameters. A probit model has also been considered, as this is also an appropriate method to regress on a binary dependent variable. However, the parameters of a logit model are easier to interpret and commonly used in marketing papers.
First, the variables of the conceptual model are included in the same manner as it has been done in study 1. Moreover, the five dummy variables representing the different countries are included.
France United Kingdom Spain Italy Germany
FR UK SP IT GER
Table 5: Dummy variables for countries
The following model has been specified for predicting the click-through probability of an individual recipient, after having opened the email.
33 Click-through probability = ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) ) . 4.2.3 Estimation
The model has been estimated with the use of a binary logistic regression. The results are presented in table 6 (Appendix 3.3 contains the SPSS output)
Parameter β Exp (β) Constant: -2.355** .095 Main effects: MP_PI .465** 1.591 MP_PR .080 1.083 HP .628** 1.873 Messageintimacy -.081 .922 Trustseals .067 1.070 Interaction effects: MP_PI * Messageintimacy -.044 .957 MP_PR * Messageintimacy .087 1.090 HP * Messageintimacy .148 1.160 MP_PI * Trustseals -.081 .922 MP_PR * Trustseals -.043 .958 HP * Trustseals -.171 .843 Control variables: France .088 1.092 UK .254** 1.290 Spain .348* 1.185 Italy .170* 1.004 Germany .004 .095
** significant at p <.01, * significant at < .05 Dependent variable: Click-through probability
34
4.2.4 Validation and results
In line with study 1, the independent variables have been checked for multicollinearity. Since SPSS does not provide the opportunity to get VIF values in a logistic regression, a multiple linear regression has been conducted to get these values. It turned out that none of the variables has a FIV value higher than 5. Hence, multicollinearity is not an issue (Appendix 3.4)
Based on the results of the binary logistic regression analysis it can be stated that both medium personalization based on personal identifiable information (Exp (β) = 1.591 p= 0.000) and high personalization (Exp (β)= 1.873 p= 0.000) positively influences the click through probability of an email, relative to condition without personalization.
No significant direct or moderating effects of message intimacy and trust seals have been found. It was also expected that these effects would not exist for both medium personalization variants. Hence, since study 1 showed that message intimacy mitigates feelings of intrusiveness due to high personalization, it was expected that this would also lead to a positive moderation effect in this model. Hence, it seems that feelings of intrusiveness do not have an effect on actual behavior of the recipients. However, this conclusion cannot be drawn based on statistical evidence.
35
5. Conclusion and Discussion
The goal of this study was to get more insights into the effects personalization. Not only the direct positive effects were point of interest, but also how personalization increases feelings of intrusiveness. Moreover, this research has focused on getting more insights in how to reduce feelings of intrusiveness due to personalization. The research question will be answered with the use of the sub questions which have been defined in chapter 1.
1. What is the effect of email personalization on the effectiveness of an email?
As study 2 shows, medium personalization (pi) of email messages has a significant positive effect on the CTP of an email, relative to a situation without personalization. This means that using personal identifiable information such as a customer‟s name or transactional information increased the likelihood of clicking through the email.
Moreover, study 2 also shows that high personalization of email messages has a significant positive effect on the CTP of an email, relative to a situation without personalization. This effect is even stronger than the effect of medium personalization based on personal identifiable information on the CTP. In practice this means that using personal identifiable information in combination with presenting product recommendations based on product preferences will significantly increase click-through probabilities.
It has also been found that medium personalization (pr) of email messages has no significant effect on the CTP, relative to a situation without personalization. Perhaps this can be explained by the fact that recipients experience it a „strange‟ when a company presents product recommendations based on their revealed interest, but does not even know their name or other personal information.
2. Does email personalization lead to increased feelings of intrusiveness?
36
personalization of email messages is to a certain degree perceived as acceptable, but becomes intrusive when it contains too much personal information.
3. Do feelings of intrusiveness decrease the effectiveness of an email?
As mentioned before, this effect has not been tested in this study. The following situation could have been an indication that feelings of intrusiveness decrease the effectiveness of an email: In study 1 was found that high personalization increases intrusiveness, but when message intimacy is also included, feelings of intrusiveness will mitigate. Thus, when intrusiveness does have a negative effect on CTP, it is likely that message intimacy positively moderates the effect of high personalization on CTP. However this effect turned out not to be significant. It must be mentioned that when this effect would exist, this still would not have been statistical evidence that intrusiveness has an effect on CTP.
4. If personalization causes higher feelings of intrusiveness; which factors can mitigate this effect?
Study 1 shows that feelings of intrusiveness caused by high personalization can be mitigated by including message intimacy. This negative interaction effect is even stronger than the positive direct effect of high personalization on feelings of intrusiveness. Message intimacy has no significant moderation effect regarding the other two types of personalization. The presence of trust seals does not show any significant effects. A possible explanation could be that people do not look so extensively to an email so do not even notice the seals. Moreover, it could be that people do not recognize the seals as signs of trustworthiness.
This chapter will be finalized by presenting an overview of the hypotheses and the outcomes.
Hypotheses Conclusions Study Notes
H1a Accepted Study 2 There is a significant positive effect of medium personalization (pi) on CTP, relative to a situation without personalization.
H1b Not accepted Study 2 Effect of medium personalization (pr) on CTP is insignificant. H1c Accepted Study 2 There is a significant positive effect of high personalization on
CTP, relative to a situation without personalization.
H2a Not accepted Study 1 Medium personalization (pi) of email messages has no significant effect on feelings of intrusiveness.
37
H2c Accepted Study 1 High personalization of email messages significantly increases feelings of intrusiveness.
H3 Not tested The effect of intrusiveness has not been tested.
H4a Not accepted Study 1 No significant effect of message intimacy on feelings of intrusiveness due to medium personalization (pi).
H4b Not accepted Study 1 No significant effect of message intimacy on feelings of intrusiveness due to medium personalization (pr).
H4c Accepted Study 1 Message intimacy significantly weakens feelings of intrusiveness due to high personalization.
H5a Not accepted Study 1 No significant effect of trust seals presence on feelings of intrusiveness due to medium personalization (pi).
H5b Not accepted Study 1 No significant effect of trust seals presence on feelings of intrusiveness due to medium personalization (pr).
H5c Not accepted Study 1 No significant effect of trust seals presence on feelings of intrusiveness due to high personalization.
Table 7: Summary of outcomes per hypothesis
6. Limitations
This research contains a couple of limitations. The most important limitation of this study is that it could not prove whether feelings of intrusiveness negatively influence the click-through rate of an email. A best case scenario would be that for each individual observation the degree of perceived intrusiveness of a specific customer and whether or not this customers clicks on the link in the email, is measured. Another way would be to ask participants for both intrusiveness and click-through intentions in a survey. However as mentioned before, consumers‟ feelings and intentions are often not corresponding with their actual behavior when it comes to privacy concerns.
38
with a company. Moreover, the effects could be different if another type of email will be send, such as a transaction email.
Moreover, some of the manipulations could also be improved in future research. For example, the variant of medium personalization (pr) contains four recommended auction lots based on a user‟s revealed preferences. The versions no-personalization and medium personalization (pi) do not contain auction lots in the email. To really assess for the effect of personalization it would have been better to also include four auction lots in these versions of the email. This could be done by including random auction lots or a fixed set of recommendations. However, this could not have been done in this research, since Catawiki‟s interests also had to be taken into account. Presenting random lots was not likely to contribute to the success of the email.
39
References
Ansari, A., & Mela, C. F. (2003). E-Customization. Journal of Marketing Research, 40(2), 131-145.
Berendt, B., Günther, O., & Spiekermann, S. (2005). PRIVACY IN E-COMMERCE: Stated Preferences vs. Actual Behavior. Communications of the ACM, 48(4), 101-106.
Bertrand, M., Karlan, D.S., Mullainathan, S., Shafir, E., Zinman, J. (2005). What's psychology worth? A field experiment in the consumer credit market. Yale University Economic Growth Center Discussion Paper, No. 918.
Besbes, O., Gur, Y., & Zeevi, A. (2016). Optimization in Online Content Recommendation Services: Beyond Click-Through Rates. Manufacturing & Service Operations Management, 18(1), 15-33
Chellappa, R,K. & Sin, R.G. (2005). Personalization versus privacy: An empirical examination of the online consumer's dilemma. Information technology and management, 6(2), 181-202
Chen, P. Y., Narasimhan, C. Z. & Zhang, J. (2001). Individual marketing with imperfect targetability. Marketing Science, 20(1) 23–41.
Chen, P.Y. & Wu, S.Y. (2004). The impact of online recommendations and consumer feedback on sales. International conference on information systems, 58
Cook, D. P., & Wenhong, L. (2003). The Role of Third-Party Seals in Building Trust Online. E-Service Journal, 2(3), 71-84.
40
Culnan, M.J. & Bies, R.J. (2003). Consumer Privacy: Balancing Economic and Justice Considerations. Journal of Social Issues, 59(2), 323-342.
Deloite (2006). Dutch Catawiki fastest grower Fast500 rankings worldwide. Retrieved from: http://www2.deloitte.com/nl/nl/pages/over-deloitte/articles/dutch-catawiki-fastest-grower-fast500-rankings-worldwide.html
Dinev, T., & Hart, P. (2005). Internet Privacy Concerns and Social Awareness as Determinants of Intention to Transact. International Journal Of Electronic Commerce, 10(2), 7-29
Dinev, T., & Hart, P. (2006). An Extended Privacy Calculus Model for E-Commerce Transactions. Information Systems Research, 17(1), 61-80
Direct Marketing Association (2015). National client email report 2015. Retrieved from: http://dma.org.uk/uploads/ckeditor/National-client-email-2015.pdf
Doorn, J., & Hoekstra, J. (2013). Customization of online advertising: The role of intrusiveness. Marketing Letters, 24(4), 339-351.
Edwards, S. M., Li, H., & Lee, J. H. (2002). Forced exposure and psychological reactance: antecedents and consequences of the perceived intrusiveness of pop-up ads. Journal of Advertising, 31(3), 83–95.
Gefen, D. (2000). E-commerce: the role of familiarity and trust. Omega, 28(6), 725
Goldfarb, A., & Tucker, C. E. (2011). Privacy regulation and online advertising. Management Science, 57(1), 57–71.
41
Hoffman, D. L., & Novak, T. P. (1996). Marketing in hypermedia computer-mediated environments: Conceptual foundations. Journal of Marketing, 60(3), 50.
Howard, D. J., & Kerin, R. A. (2004). The Effects of Personalized Product Recommendations on Advertisement Response Rates: The "Try This. It Works!" Technique. Journal of Consumer Psychology, 14(3), 271-279.
Huh J. & Shin, W. (2014). Trust in prescription drug brand websites: website trust cues, attitude toward the website, and behavioral intentions. Journal of Health Communications, 19(2), 170-191
Iacobucci, D. (2006). Three thoughts on services. Marketing Science, 25(6), 581-583.
Lambrecht, A., & Tucker, C. (2013). When Does Retargeting Work? Information Specificity in Online Advertising. Journal of Marketing Research, 50(5), 561-576.
Lavy, S., Mikulincer, M., Shaver, P. R., & Gillath, O. (2009). Intrusiveness in romantic relationships: a cross-cultural perspective on imbalances between proximity and autonomy. Journal of Social and Personal Relationships, 26(6–7), 989–1008.
Maccario, E. (2006). Getting consumers to open your email by careful targeting. Asia's Media & Marketing Newspaper, 15.
Moe, W. W., & Fader, P. S. (2004). Dynamic Conversion Behavior at E-Commerce Sites. Management Science, 50(3).
Montgomery, A. L., & Smith, M. D. (2009). Prospects for personalization on the Internet. Journal of Interactive Marketing, 23(2), 130–137.
42
Morimoto, M., & Chang, S. (2006). Consumers' Attitudes toward Unsolicited Commercial E-mail and Postal Direct Mail Marketing Methods: Intrusiveness, Perceived Loss of Control, and Irritation. Journal of Interactive Advertising, 7(1), 8-20.
Morimoto, M., & Macias, W. (2009). A Conceptual Framework for Unsolicited Commercial E-mail: Perceived Intrusiveness and Privacy Concerns. Journal of Internet Commerce, 8(3/4), 137-160.
Posner, R. A. (2008). Privacy, Surveillance, and Law. University of Chicago Law Review, 75(1), 245-260.
Postma, O. J., & Brokke, M. (2002). Personalization in practice: The proven effects of personalisation. Journal of Database Management, 9(2), 137-14
Premazzi, K., Castaldo, S., Grosso, M., Raman, P., Brudvig, S., & Hofacker, C. F. (2010). Customer Information Sharing with E-Vendors: The Roles of Incentives and Trust. International Journal of Electronic Commerce, 14(3), 63-91
Song, J., Kim, H., Kim, S., Lee, S., & Lee, J. (2016). Effects of personalized e-mail messages on privacy risk: Moderating roles of control and intimacy. Marketing Letters, 27(1), 89-101
Stafford, M. R. and Faber, J.R. (2015) Advertising, promotion, and new media. London: Routledge
Sunil, W., Rahul, T., Tridas, M., & Peter, B. (2012). What's in a "Name"? Impact of Use of Customer Information in E-Mail Advertisements. Information Systems Research, 23(3), 679-697.
43
Vesanen, J. (2007). What is personalization? A conceptual framework. European Journal of Marketing, 41(5/6), 409-418
Vlasic, G., & Kesic, T. (2007). Analysis of Consumers' Attitudes toward Interactivity and Relationship Personalization as Contemporary Developments in Interactive Marketing Communication. Journal of Marketing Communications, 13(2), 109-129
White, T. B., Zahay, D. L., Thorbjørnsen, H., & Shavitt, S. (2008). Getting too personal: reactance to highly personalized email solicitations. Marketing Letters, 19(1), 39–50.
Winer, R. S. (2009). New Communications Approaches in Marketing: Issues and Research Directions. Journal of Interactive Marketing, 23(2), 108-117
