Can a chatbot with friendly cues improve customer
satisfaction?
An experiment on the influence of chatbot gender and friendly cues on the
attitude towards the chatbot, the attitude towards the company and the customer
satisfaction.
Jeroen Caspar Seelt
10559671
Master’s Thesis
Graduate School of Communication
Master’s Program Communication Science
Supervisor: dr. T. B. Araujo
1
Abstract
Chatbots increasingly replace human customer service agents as the first line of customer service, but the way they provide their service is different from a human customer service agent. This makes it increasingly relevant to find out what type of chatbot leads to a better result, a chatbot with friendly cues or a chatbot without friendly cues? A male chatbot, or a female chatbot? Previous studies have focused on the effects of friendliness and gender of human customer service agents, but very few focused on chatbots. By using a 2 (male vs. female) by 2 (friendly cues vs. no friendly cues) between-subjects experiment (N = 135) with a real world chatbot, this study explores the extent to which chatbot gender and friendly cues influence the attitude towards the chatbot, the attitude towards the company and the customer satisfaction. Findings show friendly cues positively affect the attitude towards the chatbot and the customer satisfaction. However, gender was found to have no effect on the attitude
towards the chatbot, the attitude towards the company or the customer satisfaction.
2
Introduction
The use of chatbots is increasing rapidly (Drift, Audience, Salesforce, & Myclever, 2018), with Facebook reporting a 560% growth year-over-year in 2018 (‘’Chatbot Report 2018: Global Trends and Analysis’’, 2018). According to Dale (2016), chatbots are predicted to generate 623 billion dollars in sales in 2020. A chatbot is an artificially intelligent software application that simulates human-like conversations by allowing users to type questions and generate a relevant answer based on the input (Crutzen, Peters, Portugal, Fisser, & Grolleman, 2011). They can be used for a variety of purposes, such as selling products and helping
customers (Liu, Yan, Phau, Perez, & Teah, 2016). Several companies such as Domino’s Pizza, AliExpress and Alibaba have already replaced their human customer service agents with chatbots as the first line of customer service (Følstad, Nordheim, & Bjørkli, 2018). Since these companies have replaced their human customer service agents with chatbots and still want to satisfy their customers, the question is what type of chatbot characteristics will lead to the best outcomes (Liu et al., 2016). A chatbot with friendly cues or a chatbot without friendly cues? A male chatbot or a female chatbot?
One outcome that is of course very important for companies, is the customer satisfaction. Customer satisfaction refers to the ‘’collective outcome of perception, evaluation and psychological reactions to the consumption experience with a product/service’’ (George & Kumar, 2014, p. 75). Previous studies have shown the customer satisfaction is affected by characteristics such as friendly cues and the gender of the customer service agent.
Studies have also shown that men receive higher service ratings than women, with everything else being constant and with raters from both sexes (Pinar, Wilder, Shaltoni & Stück, 2017; Snipes, Thomson, & Oswald, 2006). In the study of Pinar et al., students were asked to rate the instructional service quality of their instructor. The results showed, with everything else being constant, that male instructors received higher service quality ratings than female
3 instructors. This indicates a bias in service ratings towards males, which has been found in several other studies with differing contexts as well (Hubert, Neale & Northcraft, 1987; Nieva & Gutek, 1980). This begs the question whether chatbot gender has the same influence when rating chatbots as it has when rating human customer service agents.
Besides the gender of the customer service agent, friendly cues also affect the customer satisfaction. Verhagen, van Nes, Feldberg and van Dolen (2014) defined the friendliness of their chatbot according to the definition of Price, Arnould and Deibler (1995): being polite, responsive, giving extra attention and creating mutual understanding. Liu et al. (2016) stated friendliness refers to the warmth and personal approachability of someone and includes having a nice attitude and making customers feel welcome. Based on these definitions, friendly cues is defined as elements of the chatbot which could be perceived to be friendly. When interacting with a human customer service agent, the friendliness of the agent is very important, since it can have a positive effect on the emotions of the customer and thus, positively influence the customer’s satisfaction (Hennig-Thurau, Groth, Paul & Gremler, 2006; Liu et al., 2016; Pugh, 2001). The same might be the case with a chatbot. Besides the customer satisfaction, friendly cues and the gender of the agent also affect the attitude towards the chatbot, which refers to one’s perception of the chatbot, and the attitude towards the company, which refers to one’s perception of the company (Sung & Lee, 2015).
The attitude towards the chatbot is also affected by perceived dialogue. Perceived dialogue refers to the extent to which the interaction with the chatbot is perceived to be a dialogue (Go & Sundar, 2019) and great levels of perceived dialogue positively affect the attitude towards the chatbot (Go & Sundar, 2019).
Customer satisfaction on the other hand, has been shown to be influenced by perceived warmth and perceived competence. Perceived warmth, which refers to the intent of the chatbot (Fiske, Cuddy, Glick & Xu, 2002) has been shown to be a significant predictor of
4 service satisfaction (Smith, Martinez & Sabat, 2016). Perceived competence, which refers to the chatbot’s capability (Fiske et al., 2002) has been shown to improve customer’s satisfaction (Ganguli & Roy, 2013).
This study will extend the existing literature in multiple areas. First, this study extends previous research by looking at how chatbot characteristics will (in)directly influence the attitude towards the company, instead of the emotional connection with the company (Araujo, 2018). Second, the relationship between friendly cues and the customer satisfaction has been studied numerous times before (Chu, Lee, & Chao, 2012; Gong & Yi, 2018; Kim & Lee, 2011; Liu et al., 2016; Park, Robertson, & Wu, 2004; Söderlund & Rosengren, 2010), but those studies focused on human customer service agents, while this study will focus on chatbots. Since the interaction with a chatbot is not the exact same as the interaction with a human customer service agent, it is important to know whether a friendly chatbot has the same effect as a friendly agent. Third, this study will also focus on the effect of friendly cues on the attitude towards the chatbot and the attitude towards the company. Research on chatbots with friendly cues exist (Verhagen et al., 2014), but it did not focus on the effect friendly cues have on the attitude towards the chatbot and the attitude towards the company. This is very relevant since instead of humans, chatbots now represent the company when interacting with customers. Fourth, previous research which studied the use of human-cues focused on the use of human avatars (Ciechanowski et al., 2019; Corti & Gillespie, 2016; Go & Sundar, 2019; Schuetzler et al., 2018), the use of a human name (Araujo, 2018) or animated avatars (Ciechanowski et al., 2019; Schuetzler et al., 2018), while this study will look at two other characteristics of a chatbot: friendly cues and chatbot gender. Fifth, this study will look at the mediating role of perceived competence and perceived warmth on the effect of chatbot gender. Many studies focused on the effect of the gender of the service provider on the service rating (Luoh & Tsaur, 2007; Mokhlis, 2012; Snipes et al., 2006), compared the ratings of
5 male and female voices (Nass, Moon, & Green, 1997; Nelson, Signorella, & Botti, 2016) or studied the competence and warmth of both males and females (Ditonto, 2017; Ebert, Steffens, & Kroth, 2014; Eisenchlas, 2013; Fiske et al., 2002; Lin, Wang, Lin, Lin, & Johnson, 2011), but those studies focused on humans, whilst this study will focus on finding out if the same results are found for gendered chatbots. This is valuable information for a company since customers might prefer one gender over the other, which could lead to a higher level of customer satisfaction for example. Sixth, the study of Go and Sundar (2019) found that perceived dialogue has a positive effect on the attitude towards the chatbot and in this study the possible mediation effect of perceived dialogue on the relationship between friendly cues and attitude towards the chatbot will be studied.
In summary, this study addresses the following question: To what extent do friendly cues and chatbot gender influence the attitudes towards the chatbot, the attitude towards the company and the customer satisfaction?
Theoretical framework
Recent studies (Chung, Ko, Joung & Kim., 2018; Wu, Tsai, Hsiung, & Chen, 2015) found the interactions with customer service chatbots to be similar to interactions with human customer service agents in terms of influencing purchasing decisions, time saving, advice, parasocial benefits and perceptions of service quality and satisfaction. Similarly, chatbots and human customer service agents are equally effective when it comes to emotional, relational and psychological benefits (Ho, Hancock, & Miner, 2018).
These similarities can be partially explained by the Computers Are Social Actors (CASA) paradigm. This states that humans mindlessly apply the same social rules and stereotypes when interacting with a computer (which is similar to a chatbot) as they do when interacting with a human (Nass & Moon, 2000). For example, Kim and Sundar (2012) found participants
6 exposed to a human-like chatbot reported a lower degree of perceived human-likeness, but they mindlessly applied social rules and attributed personal characteristics to a chatbot. A Study by Ho et al. (2018) also found the same disclosure processes and outcomes take place when participants interact with a chatbot, as they do when interacting with a human. Since humans mindlessly apply the same stereotypes and social rules to chatbots as they do to humans, it is highly likely a chatbot is judged in the same way as well.
One characteristic which improves the judgment of a human customer service agent is friendliness. The friendliness of the customer service agent affects the emotions of the customer in a positive way and thus, positively influences the customer’s satisfaction and customer commitment (Liu et al., 2016). This process is called emotional contagion.
Friendliness is defined as being polite, responsive, giving extra attention and creating mutual understanding (Price et al., 1995). It also refers to the warmth and personal approachability of someone and includes having a nice attitude and making customers feel welcome (Liu et al., 2016).
A study by Verhagen et al. (2014) found that a chatbot with friendly cues evokes feelings of personal, sociable and sensitive human contact, which in turn improves the customer
satisfaction. Liu et al. (2016) found a similar result in their study. Honest, friendly and authentic relationships between customers and the customer service agents are crucial for the customer to have a good experience. Furthermore, these customer service agents have to be courteous, helpful and trustworthy (Chung et al., 2018). Friendliness will generate a positive customer emotion through an emotional contagion process, which will then have a positive effect on customer evaluation (Liu et al., 2016), which leads to the following hypothesis:
H1A: A chatbot with friendly cues will lead to a more positive attitude towards the chatbot than a chatbot without friendly cues.
7 As discussed before, part of the definition of friendliness, is being responsive (Price et al., 1995). When a conversation is responsive, individuals will perceive greater levels of dialogue, which means the conversation is perceived to be similar to a human-to-human dialogue (Go & Sundar, 2019). Greater levels of perceived dialogue create the perception of a face-to-face conversation and are positively related to the attitude towards the chatbot (Go & Sundar, 2019), which leads to the following hypothesis:
H1B: Perceived dialogue partially mediates the relationship between friendly cues and the attitude towards the chatbot.
Parasuraman, Zeithaml and Berry (1988) found that a positive emotional display during service interactions has a positive effect on service quality ratings. Positive emotional display refers to employees showing positive emotions (Lam, Huang, & Janssen, 2010) and is similar to being friendly. Being friendly entails being polite, giving extra attention and showing interest in the customer (Price, 1995), which means the employee shows positive emotions.
The service quality, which is positively affected by the display of positive emotions, has been proven to affect the attitude towards a company in a positive way (Kant & Jaiswal, 2017; Wu, 2014; Yang, Hsieh, Li, & Yang, 2012). A study by Engel, Tran, Pavlek, Blankenberg and Meyer (2013) found that employee friendliness also has a positive effect on brand
perceptions. This makes it highly likely the same will be the case for company attitude. Based on the preceding, it is to be expected that being friendly will positively affect the attitude towards the company, which leads to the following hypothesis:
H2: A chatbot with friendly cues will lead to a more positive attitude towards the company than a chatbot without friendly cues.
Söderlund and Rosengren (2010) found that a positive emotional display in service
8 (2016), who stated customer satisfaction is mostly dependent on the behavior of the service employee. Being friendly is similar to showing a positive emotion, because being friendly entails being polite, giving extra attention and showing interest in the customer (Price, 1995), which means the employee shows positive emotions. Friendly cues will also generate a positive customer emotion through an emotional contagion process, which will then have a positive effect on customer evaluation and thus, customer satisfaction (Liu et al., 2016). Besides that, multiple studies have found that service perception is positively related to customer satisfaction (Chu et al., 2012; Gong & Yi, 2018; Park et al., 2004). If a chatbot is perceived as friendly, it is highly likely it will lead to a positive perception of the service provided, which will then lead to a satisfied customer. A chatbot with friendly cues is expected to lead to a higher level of customer satisfaction, resulting in the following hypothesis:
H3: A chatbot with friendly cues will lead to a higher level of customer satisfaction than a chatbot without friendly cues.
According to the attitude theory (Eagly & Mladinic, 1989), society perceives men to be agentic, which means they are perceived to be competent, assertive, independent, masterful and achievement oriented. Woman are perceived by society to be communal, which means they are perceived to be friendly, warm, unselfish, sociable, interdependent, emotionally expressive and relationship oriented. This is due to the stereotypical belief in society that women should be warm and caring, while men should be strong and agentic (Prentice & Carranza, 2002). A study by Ebert et al. (2014) also found that women are associated with warmth, which stems from women traditionally being perceived as the homemakers, which is associated with being warm, but not competent. Men are stereotypically perceived as
competitive and independent, which leads to men being perceived as competent (Asbrock, 2010; Fiske et al., 2002).
9 Part of the dimensions of social judgement, which is how humans are judged socially, are competence and warmth (Lin et al., 2011). This corresponds with the Stereotype Content Model (SCM) of Fiske et al. (2002), which states stereotypes can be captured by two
dimensions: warmth and competence. They argue a high score on one of those dimensions is often paired with a low score on the other dimension.
Multiple experiments have found these stereotypes to occur in customer service as well. Male voices for example, are rated higher in terms of competence than female voices (Nelson, Signorella, & Botti, 2016) and Snipes et al. (2006) found, with all else being equal, male service providers receive a higher service rating than female service providers.
The stereotypical belief in society that women are perceived to be warm has led to some virtual assistants being given a female name, such as Siri and Alexa, because of their stereotypical feminine tasks, such as service work (Hester, 2017; Weber, 2005).
Humans mindlessly apply the same stereotypes and social rules when interacting with a computer (Nass & Moon, 2000), as they would when interacting with a real human. This is referred to as the Computers Are Social Actors (CASA) paradigm. Based on the CASA paradigm it is to be expected that chatbots will be judged based on the same gender stereotypes and social expectations that exist in society, which leads to the following hypothesis:
H4: A male chatbot will lead to a more positive attitude towards the chatbot than a female chatbot.
Service quality, which refers to the efficiency and effectiveness of the service provided (Chu et al., 2012), has been shown to positively influence the image of a company (Kant & Jaiswal, 2017; Park et al., 2004; Wu, 2014; Yang et al., 2012). The image of a company is a
10 & Kotler, 1991), which means it is partially constructed out of the attitude towards the
company. Since service quality has a positive effect on company image, which is thus
partially constructed out of the attitude towards the company, and men receive higher service quality ratings than women (Snipes et al., 2006), it is to be expected that a male chatbot will lead to a more positive attitude towards the company than a female chatbot. This leads to the following hypothesis:
H5: A male chatbot will lead to a more positive attitude towards the company than a female chatbot.
Multiple studies have found service quality is positively related to customer satisfaction (Chu et al., 2012; Gong & Yi, 2018; Mokhlis, 2012; Park et al., 2004) and men have been found to receive higher service ratings than women (Snipes et al., 2006). More importantly, a study by Hekman et al. (2010) found that women received significantly lower customer satisfaction ratings than men, with performance being held constant. This result was due to customer bias. Based on the CASA paradigm, it is to be expected the same results will be found for gendered chatbots, which leads to the following hypothesis:
H6: A male chatbot will lead to a higher level of customer satisfaction than a female chatbot.
Two different views emerge from the literature on what type of characteristic of a customer service agent improves customer satisfaction. One view states competence will improve customer satisfaction and the other view states warmth will improve customer satisfaction. Therefore, both views will be tested in this study.
According to customers, competence is the second most important aspect customer service employees should have when handling customer complaints (Delcourt, Gremler, De Zanet, & van Riel, 2017; Gruber, 2011). If customer service employees are competent, customer satisfaction is highly likely to increase, because the customer service employee has handled
11 the complaint in a satisfactory way. Multiple studies have also found the perception of service quality can be improved by more competent employees (Ganguli & Roy, 2013; Wu et al., 2015). This means competence is an important characteristic of a customer service employee. Since the stereotypical belief in society is that men are perceived to be more competent than women (Eisenchlas, 2013; Fiske et al., 2002), this leads to the expectation that the
relationship between chatbot gender and customer satisfaction is partially mediated by perceived competence. Therefore, the following hypothesis has been created:
H7: Perceived competence partially mediates the relationship between chatbot gender and customer satisfaction.
A study by Smith et al. (2016) found that perceived warmth is a significant predictor of service satisfaction, which is relatively similar to customer satisfaction. Since women are stereotypically perceived by society to be warm, whereas men are not (Ebert et al., 2014), it is to be expected perceived warmth partially mediates the relationship between chatbot gender and customer satisfaction, which leads to the following hypothesis:
H8: Perceived warmth partially mediates the relationship between chatbot gender and customer satisfaction.
Several studies (Moshavi, 2004; Pinar et al., 2017) have found that customers are more
satisfied with a customer service employee from the opposite sex than with a customer service employee from the same sex. According to Moshavi (2004) this was due to good customer service being associated with flirtation, which leads to a more positive perception of the service provided. This leads to the expectation that the effect of chatbot gender on attitude towards the chatbot will be stronger when participants judge a chatbot from the opposite sex, as compared to participants judging a chatbot from the same sex. Based on the preceding, the following hypothesis has been created:
12 H9: The effect of chatbot gender on attitude towards the chatbot will be stronger for a
participant that interacts with a chatbot from the opposite sex.
Methodology
This study used a 2 (male vs. female) x 2 (friendly cues vs. no friendly cues) between-subjects design.
Sample
A total of 143 participants took part in the experiment. Of those participants, 21 had not received a code from the chatbot. If the time they started with the questionnaire matched the time a conversation with a chatbot was started, they were given that particular conversation code. As a result of that, 13 participants were included, because they were matched with a conversation and 8 participants were excluded, because their time did not match any conversation. In total, 135 participants were used for the analysis. The total sample of this study (N = 135) consists of people between the ages of 19 to 61 years old. The average participant was 25.9 years old (SD = 7.43). Participants were recruited via messages on Facebook, LinkedIn and personal contacts. Of the participants, 66.7% was female, 31.9% was male and 1.5% would rather not disclose their gender. A bachelor’s degree was completed by 54.8% of the participants, 37% completed a master’s degree, 7.9% completed high school and .7% completed a doctorate degree. The distribution of the participants per condition can be found in Table 1.
Participants were randomly assigned to any of the four conditions and the conditions were manually added to the dataset, based on the conversation code provided.
13 Table 1.
Distribution of participants over conditions
Chatbot gender
Male Female
Friendly cues Friendly cues No friendly cues 30 31 34 40 Procedure
The first screen of the survey contained information related to the study. The second screen contained the informed consent, with which participants had to agree to proceed. The participants were first asked some questions regarding their demographics and their current attitude towards the company. After that, the participants were instructed to speak to the chatbot, as if they were on the company’s website. They were instructed to request a new product, because they received a damaged product. They were instructed to not spend more than 3 minutes speaking to the chatbot and had to proceed after that time, regardless of the completion of the task. The participants then answered questions regarding their attitude towards the chatbot, their attitude towards the company and the customer satisfaction. The order of the statements was randomized for each question. An example of a conversation can be seen in Figure 1. For the complete list of questions used in the questionnaire, see Appendix I.
14 Figure 1. Example of conversation with female chatbot with friendly cues
Stimuli
The chatbot used in this study was manipulated in two ways: both the friendly cues and the chatbot gender were manipulated. The chatbot with friendly cues greeted users in a nice way (e.g. ‘Hi, my name is John! How can I help you?’) and gave elaborate answers (e.g. ‘I am so sorry to hear that! That should not happen. What kind of product is it?’). The chatbot without friendly cues greeted users in a simple way (e.g. ‘Hi.’) and gave short answers (e.g. ‘What kind of product?’).
One version of the chatbot had a male name: John and one version had a female name: Emma. These names appeared at the top of the screen and at the bottom of every message the chatbot sent.
Measurement
15 Gender. To measure if the manipulation for chatbot gender was successful,
participants were asked ‘’If you had to assign a gender to the chatbot you just talked to, what gender would it be?’’. This item was measured using a 7-point scale ranging from female to male.
Friendly cues. To measure if the manipulation check for friendly cues was successful, a scale by Tsai and Huang (2002) with three items was used (M = 4.79, SD = 1.68, α = .95). The participants were asked to what extend they agreed with the following statements: ‘’The chatbot provided the service in a friendly manner’’, ‘’The chatbot appeared enthusiastic about helping me’’ and ‘’The chatbot treated me nicely’’. These items were measured with a 7-point Likert scale. A factor analysis showed only one component (2.73) and all questions loaded in the same factor.
Outcome Variables
Perceived warmth. Perceived warmth was measured with a 3-item scale which
assessed the chatbot’s warmth, generosity and kindness (Aaker, Vohs, & Mogilner, 2010)(M = 4.63, SD = 1.59, α = .92). These items were measured with 7-point Likert scale, ranging from strongly disagree to strongly agree. A factor analysis showed only one component (2.57) and all questions loaded in the same factor.
Perceived competence. Perceived competence was measured with a 3-item scale,
which assessed the chatbot’s competence, effectiveness and efficiency (Aaker et al., 2010)(M = 5.34, SD = 1.45, α = .93). These items were measured with 7-point Likert scale, ranging from strongly disagree to strongly agree. A factor analysis showed only one component (2.63) and all questions loaded in the same factor.
Perceived dialogue. Perceived dialogue was measured with a 5-item scale (Sundar,
16 to what extent they agreed with statements such as ‘I felt like I was engaged in an active dialogue with the chatbot’ and ‘the chatbot responded quickly to my inputs and requests’. The items were measured with a 7-point Likert scale, ranging from strongly disagree to strongly agree. A factor analysis showed only one component (3.00) and all questions loaded in the same factor.
Attitude towards the chatbot. Attitude towards the chatbot was measured with a
4-item scale (Sung & Lee, 2015)(M = 5.04, SD = 1.24, α = .83). The participants had to evaluate the chatbot on four opposites, such as unfavorable – favorable and bad – good. The items were measured with a 7-point scale. A factor analysis showed only one component (3.73) and all questions loaded in the same factor.
Attitude towards the company before the manipulation. This study used the
company IKEA to measure the attitude towards the company, because IKEA is a well-known and popular company. Pre-existing attitude towards the chatbot was measured with a 4-item scale (Sung & Lee, 2015)(M = 5.92, SD = 0.97, α = .92). The participants had to evaluate the company on four opposites, such as unfavorable – favorable and bad – good. The items were measured with a 7-point scale. A factor analysis showed only one component (3.26) and all questions loaded in the same factor.
Attitude towards the company after the manipulation. To measure the attitude
towards the chatbot after the manipulation a 4-item scale was used (Sung & Lee, 2015)(M = 5.72, SD = 1.15, α = .96). The participants had to evaluate the company on four opposites, such as unfavorable – favorable and bad – good. The items were measured with a 7-point scale. A factor analysis showed only one component (3.61) and all questions loaded in the same factor.
17
Customer satisfaction. Customer satisfaction was measured with a 6-item scale
(Oliver & Swan, 1989)(M = 5.31, SD = 1.53, α = .97). The participants had to evaluate their experience on six opposites, such as poor choice – wise choice and unhappy – happy. The items were measured with a 7-point scale. A factor analysis showed only one component (5.31) and all questions loaded in the same factor.
More detailed information on the construction of the scales can be found in Appendix II.
Pre-test
A pre-test was performed to validate the manipulations beforehand. A total of N = 22 participants took part in the pre-test. To measure if the manipulation check for friendly cues was successful, a scale from Tsai and Huang (2002) was used (α = .98). An independent samples t-test confirmed that participants who interacted with a chatbot with friendly cues perceived it as friendlier (M = 5.95, SD = 1.10) than participants who interacted with a chatbot without friendly cues (M = 3.04, SD = 1.50). This difference is significant t (20) = -5.24, p < .001, CI = [-4.07, -1.75], d = 2.22.
An independent samples t-test confirmed that participants who interacted with a male chatbot mostly assigned the male gender to the chatbot (M = 2.60, SD = 1.58) whereas participants who interacted with the female chatbot mostly assigned the female gender to the chatbot (M = 5.83, SD = 1.57). This difference is significant t (20) = 4.77, p < .001, CI = [1.82, 4.65], d = 2.04.
Results Randomization check
18 A randomization check was performed to validate the randomization of age, participant gender, highest degree completed and nationality over the four conditions. All randomizations were successful. For detailed results, see Appendix III.
Manipulation Check
Friendly cues. An independent samples t-test confirmed that participants who
interacted with a chatbot with friendly cues perceived the chatbot friendlier (M = 5.45, SD = 1.70) than participants who interacted with a chatbot without friendly cues (M = 4.18, SD = 1.43). This difference is significant t (133) = -4.72, p < .001, CI = [-1.80, -0.74], d = .81.
Manipulation check chatbot gender. An independent samples t-test confirmed that
participants who interacted with a male chatbot mostly assigned the male gender to the chatbot (M = 2.93, SD = 1.60) whereas participants who interacted with the female chatbot mostly assigned the female gender to the chatbot (M = 5.27, SD = 1.78). This difference is significant t (133) = 7.95, p < .001, CI = [1.75, 2.92], d = 1.38.
Hypotheses Testing
Friendly cues, chatbot gender and attitude towards the chatbot. Hypothesis 1A
stated that a chatbot with friendly cues will lead to a more positive attitude towards the chatbot than a chatbot without friendly cues. Hypothesis 4 stated that a male chatbot will lead to a more positive attitude towards the chatbot than a female chatbot. To test both hypotheses, a two-way Analysis of Variance was performed to determine whether friendly cues or chatbot gender had an effect on the attitude towards the chatbot and whether there is an interaction effect. Attitude towards the chatbot was used as the dependent variable and Chatbot gender and Friendly cues as the fixed factors.
Results showed that friendly cues had a small, significant effect on attitude towards the chatbot. The attitude towards the chatbot of the participants who spoke to a chatbot with
19 friendly cues was more positive (M = 5.54, SD = 0.21) than the attitude towards the chatbot of participants who spoke to a chatbot without friendly cues (M = 4.65, SD = 0.20). Chatbot gender had no significant effect on attitude towards the chatbot and no significant interaction effect was found between friendly cues and chatbot gender. Detailed information on the results can be found in Table 2.
Table 2.
Results of Analysis of Variance for chatbot gender and friendly cues on attitude towards the chatbot.
DV: Attitude towards the chatbot
IV: Friendly cues F (1, 131) = 9.35, p = .003, η2 = 0.07 With friendly cues (n = 64) M = 5.54, SD = 0.21
Without friendly cues (n = 71) M = 4.65, SD = 0.20
IV: Chatbot gender F (1, 131) = 0.12, p = .735
Male (n = 61) M = 5.14, SD = 0.21
Female (n = 74) M = 5.04, SD = 0.20
IV: Interaction Friendly cues * Chatbot gender F (1,131) = 0.84, p = .362 Friendly cues/male (n = 30) M = 5.72, SD = 0.31 Friendly cues/female (n = 34) M = 5.35, SD = 0.29 No friendly cues/male (n = 31) M = 4.57, SD = 0.30 No friendly cues/female (n = 40) M = 4.73, SD = 0.27
This means Hypothesis 1A, stating that a chatbot with friendly cues will lead to a more positive attitude towards the chatbot than a chatbot without friendly cues, is supported. This also means Hypothesis 4, stating that a male chatbot will lead to a more positive attitude towards the chatbot than a female chatbot, is not supported.
20
Friendly cues, chatbot gender and attitude towards the company. Hypothesis 2
stated that a chatbot with friendly cues will lead to a more positive attitude towards the company than a chatbot without friendly cues. Hypothesis 5 stated that a male chatbot will lead to a more positive attitude towards the company than a female chatbot. To test both hypotheses, a repeated-measures Analysis of Variance was performed, to determine whether friendly cues or chatbot gender had an effect on the attitude towards the company. Attitude towards the company before the manipulation and Attitude towards the company after the manipulation were used as the within-subjects variables and Friendly cues and Chatbot gender were used as the between-subjects factors.
Results showed friendly cues had no significant effect on the attitude towards the company. Chatbot gender also had no significant effect on the attitude towards the company and no significant interaction effect was found between friendly cues and chatbot gender. More detailed information on the results can be found in Table 3.
21 Table 3.
Results of Analysis of Variance for chatbot gender and friendly cues on attitude towards the company.
DV: Attitude towards the company
IV: Friendly cues F (1, 131) = 1.08, p = .301 With friendly cues (n = 64) M = 5.92, SD = 0.12 Without friendly cues (n = 71) M = 5.74, SD = 0.11
IV: Chatbot gender F (1, 131) = 0.21, p = .651
Male (n = 61) M = 5.87, SD = 0.12
Female (n = 74) M = 5.79, SD = 0.11
IV: Interaction Friendly cues * Chatbot gender F (1,131) = 0.32, p = .570 Friendly cues/male (n = 30) M = 6.00, SD = 0.17 Friendly cues/female (n = 34) M = 5.83, SD = 0.16 No friendly cues/male (n = 31) M = 5.73, SD = 0.17 No friendly cues/female (n = 40) M = 5.75, SD = 0.15
This means Hypothesis 2, stating that a chatbot with friendly cues will lead to a more positive attitude towards the company than a chatbot without friendly cues, is not supported. This also means Hypothesis 5, stating that a male chatbot will lead to a more positive attitude towards the company than a female chatbot, is not supported.
Friendly cues, chatbot gender and customer satisfaction. Hypothesis 3 stated that a
chatbot with friendly cues will lead to a higher level of customer satisfaction than a chatbot without friendly cues. Hypothesis 6 stated that a male chatbot will lead to a higher level of customer satisfaction than a female chatbot. To test both hypotheses, a two-way Analysis of
22 Variance was performed, to determine whether friendly cues or chatbot gender had an effect on customer satisfaction. Customer satisfaction was used as the dependent variable and Friendly cues and Chatbot gender as the fixed factors.
Results showed friendly cues had a small, significant effect on customer satisfaction. The level of customer satisfaction of participants who spoke to a chatbot with friendly cues (M = 5.68, SD = 1.49), was higher than the level of customer satisfaction of participants who spoke to a chatbot without friendly cues (M = 4.98, SD = 1.51). Chatbot gender had no significant effect on customer satisfaction and no significant interaction effect was found between friendly cues and chatbot gender. More detailed information on the results can be found in Table 4.
Table 4.
Results of Analysis of Variance for chatbot gender and friendly cues on customer satisfaction.
DV: Customer satisfaction
IV: Friendly cues F (1, 131) = 7.25, p = .007, η2 = 0.05 With friendly cues (n = 64) M = 5.68, SD = 1.49
Without friendly cues (n = 71) M = 4.98, SD = 1.51
IV: Chatbot gender F (1, 131) = 0.19, p = .667
Male (n = 61) M = 5.39, SD = 0.19
Female (n = 74) M = 5.28, SD = 0.18
IV: Interaction Friendly cues * Chatbot gender F (1,131) = 0.03, p = .875 Friendly cues/male (n = 30) M = 5.77, SD = 0.28 Friendly cues/female (n = 34) M = 5.61, SD = 0.26 No friendly cues/male (n = 31) M = 5.02, SD = 0.27 No friendly cues/female (n = 40) M = 4.95, SD = 0.24
23 This means Hypothesis 3, stating that a chatbot with friendly cues will lead to a higher level of customer satisfaction than a chatbot without friendly cues, is supported. This also means Hypothesis 6, stating that a male chatbot will lead to a higher level of customer satisfaction than a female chatbot, is not supported.
Friendly cues, attitude towards the chatbot and perceived dialogue. Hypothesis 1B
stated that the relationship between friendly cues and attitude towards the chatbot is mediated by perceived dialogue. To test this hypothesis, Model 4 of Hayes PROCESS was used
(Hayes, 2013). Attitude towards the chatbot was used as the Y variable, Friendly cues was used as the X variable and Perceived dialogue was used as the mediator.
Results showed friendly cues had a significant, positive effect on perceived dialogue, F (1, 133) = 0.69, p = .001, CI = [0.27, 1.09]. Perceived dialogue had a significant, positive effect on attitude towards the chatbot, F (2, 133) = 0.99, p < .001, CI = [0.82, 1.16]. Friendly cues had no significant direct effect on attitude towards the chatbot, F (2, 133) = 0.19, p = .363. The indirect effect of perceived dialogue on attitude towards the chatbot was 0.67.
This means Hypothesis 1B, stating that the relationship between friendly cues and attitude towards the chatbot is mediated by perceived dialogue, is supported.
Chatbot gender, customer satisfaction and perceived competence and warmth.
Hypothesis 7 stated that the relationship between chatbot gender and customer satisfaction is mediated by perceived competence Hypothesis 8 stated that the relationship between chatbot gender and customer satisfaction is mediated by perceived warmth. To test both hypotheses, Model 4 of Hayes’ PROCESS was used (Hayes, 2013). Customer satisfaction was used as the Y variable, Chatbot gender was used as the X variable and Perceived competence and
24 Results showed chatbot gender had no significant effect on perceived competence, F (1, 133) = 0.22, p = .394. Perceived competence had a significant, positive effect on customer
satisfaction, F (3, 133) = 0.63, p < .001, CI = [0.52, 0.74]. Chatbot gender had no significant effect on perceived warmth F (1, 133) = -0.06, p = .835. Perceived warmth had a significant, positive effect on customer satisfaction, F (3, 133) = 0.38, p < .001, CI = [0.28, 0.48]. The total direct and indirect effects can be found in Table 5.
Table 5.
Results of PROCESS analysis on mediation of competence and warmth.
IV and DV Mediation variables
Indirect effect Total effect Direct effect
IV: chatbot gender .11 (.22) .02 (.14) DV: customer satisfaction Perceived competence .14 (.15) [-.17, .43] Perceived warmth -.02 (.10) [-.24, .18]
Note. * = p < .05, ** = p < .01, *** = p < .001. Unstandardized regression coefficients are given, with standard errors in parenthesis. Confidence intervals (95%) are provided for indirect effects.
This means Hypothesis 7, stating that the relationship between chatbot gender and customer satisfaction is mediated by perceived competence, is not supported. This also means
Hypothesis 8, stating that the relationship between chatbot gender and customer satisfaction is mediated by perceived warmth, is not supported.
25
Chatbot gender, attitude towards the chatbot and perceived dialogue. Hypothesis
9 stated that the effect of chatbot gender on attitude towards the chatbot will be stronger for a participant that interacts with a chatbot from the opposite sex. To test this hypothesis, a two-way Analysis of Variance was performed, to determine whether the effect of chatbot gender on attitude towards the chatbot is moderated by the gender of the participant. Attitude towards the chatbot was used as the dependent variable and Chatbot gender and Participant gender were used as the fixed factors.
Results showed no significant interaction effect between chatbot gender and participant gender, F = 0.36, p = .551.
This means Hypothesis 9, stating that the effect of chatbot gender on attitude towards the chatbot will be stronger for a participant that interacts with a chatbot from the opposite sex, is not supported.
Table 6 summarizes the results of the hypotheses testing.
Table 6
Summary of the results.
Hypothesis Result
Hypothesis 1A: A chatbot with friendly cues will lead to a more positive attitude towards the chatbot than a chatbot without friendly cues.
Supported
Hypothesis 1B: The relationship between friendly cues and attitude towards the chatbot is mediated by perceived dialogue.
Supported
26 positive attitude towards the company than a chatbot without
friendly cues.
Hypothesis 3: A chatbot with friendly cues will lead to a higher level of customer satisfaction than a chatbot without friendly cues.
Supported
Hypothesis 4: A male chatbot will lead to a more positive attitude towards the chatbot than a female chatbot.
Not supported
Hypothesis 5: A male chatbot will lead to a more positive attitude towards the company than a female chatbot.
Not supported
Hypothesis 6: A male chatbot will lead to a higher level of customer satisfaction than a female chatbot.
Not supported
Hypothesis 7: The relationship between chatbot gender and customer satisfaction is mediated by perceived competence
Not Supported
Hypothesis 8: The relationship between chatbot gender and customer satisfaction is mediated by perceived warmth.
Not Supported
Hypothesis 9: The effect of chatbot gender on attitude towards the chatbot will be stronger for a participant that interacts with a chatbot from the opposite sex.
Not Supported
Discussion
The main research question of this study was as follows: To what extent do friendly cues and chatbot gender influence the attitudes towards the chatbot, the attitude towards the company and the customer satisfaction?
27 The results show friendly cues had a significant, positive effect on the attitude towards the chatbot. Participants who interacted with a chatbot with friendly cues rated the chatbot more positive than participants who interacted with a chatbot without friendly cues. Verhagen et al. (2014) found that the friendliness of a virtual customer service agent (VCSA) evokes feelings of personal, sociable and sensitive human contact, which then positively affect service
encounter satisfaction. This study extends those findings in two ways. First, this study extends their findings to a direct effect on chatbot attitudes, instead of a mediated effect on customer service satisfaction. Second, this study used a chatbot which is less similar to a human customer service agent than the VCSA Verhagen et al. (2014) used.
Participants who interacted with a chatbot with friendly cues were also more satisfied with the service provided, which corresponds to results found by Verhagen et al. (2014) and Liu et al. (2016) who both found that friendly cues evoke positive emotions which in turn have a positive effect on customer satisfaction. Liu et al. (2016) found that the friendliness of a human customer service agent results in an emotional contagion process which improves customer satisfaction. This study extends the findings of Verhagen et al. (2014) to a direct effect on customer satisfaction, instead of an indirect effect. It also extends the research of Liu et al. (2016) by finding the same results for chatbots.
The relationship between friendly cues and attitude towards the chatbot is mediated by perceived dialogue. Participants who interacted with a chatbot with friendly cues perceived their interaction with the chatbot to be more similar to a human-to-human conversation than participants who interacted with a chatbot without friendly cues, which led to a more positive attitude towards the chatbot. Go and Sundar (2019) found that perceived dialogue mediates the relationship between message interactivity and attitude towards the website, while this study found perceived dialogue mediates the relationship between friendly cues and attitude towards the website. Message interactivity refers to the extent to which one actor in a
28 conversation acknowledges what the other says. The findings of this study might be due to the fact that when a chatbot contained friendly cues, it gave more elaborate answers, which might have led to the participants perceiving the chatbot as acknowledging their input, which led to greater levels of perceived dialogue.
The results also showed that chatbot gender had no significant effect on the attitude towards the chatbot, the attitude towards the company or the customer satisfaction. These findings are not in line with the results of Snipes et al. (2006), who found that male customer service agents are rated more positively than female customer service agents. The findings of the present study might be due to the relatively new and ongoing gender equality advancements, in which gender stereotypes are challenged. Future studies could investigate if gender
stereotypes mediate the relationship between chatbot gender and attitude towards the chatbot, attitude towards the company and customer satisfaction. If gender stereotypes are
disappearing from society, the relationship would not be mediated by gender stereotypes.
Furthermore, no moderation effect was found for participant gender on the relationship between friendly cues and attitude towards the chatbot, which contradicts the opposite gender bias, found by Moshavi (2004) and Pinar et al. (2017). This could be due to the fact that one cannot flirt with a chatbot, which was why humans preferred a customer service agent from the opposite sex.
No mediation effects were found for perceived competence and perceived warmth on the relationship between chatbot gender and customer satisfaction. These results is not in line with the results of Delcourt et al. (2017) and Gruber (2011) who found that customers perceive competence to be the second most important aspect a customer service agent can have when handling complaints. These results are also not in line with results of Lin et al. (2011), who found that people prefer low-competence and warmth over
high-29 competence and low-warmth, but this study found neither of those profiles were preferred over the other.
The findings that perceived competence and perceived warmth do not mediate the relationship between chatbot gender and customer satisfaction, might also be due to the ongoing gender equality advancements, in which stereotypes are challenged. If participants’ judging is not based on stereotypes, warmth and competence will not mediate the relationship between chatbot gender and customer satisfaction, because neither of the genders will be perceived as more competent or warm than the other.
Implications
This study has two valuable practical implications. Implementing friendly cues in a chatbot can be very beneficial for a company, since it will lead to a more positive attitude towards the chatbot and a higher level of customer satisfaction.
Companies could also use the finding that the effect of friendly cues on attitude towards the chatbot is mediated by perceived dialogue. It is therefore appealing for companies to create a chatbot which makes their customers perceive the interaction with the chatbot as a real conversation, since it will lead to a more positive attitude towards the chatbot.
Limitations
This study has several limitations. First, this study featured a repeated measures design for company attitudes. However, participants already had a certain attitude towards that company, which means their attitude was harder to change. Future studies could use a non-existing company and a single measure design to investigate if the same results are found. Second, the sample of this study contained mostly university students, who are relatively well educated compared to the general population and represent a small and specific sample. It is therefore hard to generalize the results to the general population. Future studies should investigate if the
30 same result are applicable to other parts of the general population. Third, the product used in this experiment was a closet, which is a low involvement product. This means participants might have been indifferent to the problem proposed for the interaction with the chatbot. For future studies it would be worthwhile to find out if the results would change with a high involvement product, such as a new smartphone. It would also be interesting to change the scenario of the experiment to, for example, asking technical questions about a product, instead of asking for a new product. This could possibly change the mediating effect of warmth and competence, because men might be perceived as more competent with regards to technical issues.
Limitations aside, this study proved that friendly cues have a positive effect on the attitude towards the chatbot and the customer satisfaction and that the relationship between friendly cues and the attitude towards the chatbot is mediated by perceived dialogue. Both of which are important practical implications.
Reference list
Aaker, J., Vohs, K., & Mogilner, C. (2010). Non-Profits are Seen as Warm and For-Profits as Competent: Firm Stereotypes Matter. Journal of Consumer Research, 37(August), 224–237. https://doi.org/10.2139/ssrn.1540134
Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
Asbrock, F. (2010). Stereotypes of social groups in Germany in terms of warmth and competence. Social Psychology, 41(2), 76-81.
Barich, H., & Kotler, P. (1991). A framework for marketing image management. Sloan Management Review, 32(2), 94-104.
31 Chatbot Report 2018: Global Trends and Analysis. (2018, March 17). Retrieved from
https://chatbotsmagazine.com/chatbot-report-2018-global-trends-and-analysis-4d8bbe4d924b
Chu, P. Y., Lee, G.Y., & Chao, Y. (2012). Service Quality, Customer Satisfaction, Customer trust, and Loyalty in an E-Banking Context. Social Behavior and Personality, 40(8), 1271– 1284. https://doi.org/10.5539/ijbm.v13n8p108
Chung, M., Ko, E., Joung, H., & Kim, S. J. (2018). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, (September), 1–9. https://doi.org/10.1016/j.jbusres.2018.10.004
Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539–548. https://doi.org/10.1016/j.future.2018.01.055
Corti, K., & Gillespie, A. (2016). Co-constructing intersubjectivity with artificial
conversational agents: People are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior, 58, 431–442.
https://doi.org/10.1016/j.chb.2015.12.039
Crutzen, R., Peters, G. J. Y., Portugal, S. D., Fisser, E. M., & Grolleman, J. J. (2011). An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: An exploratory study. Journal of Adolescent Health, 48(5), 514–519.
https://doi.org/10.1016/j.jadohealth.2010.09.002
Dale, R. (2016). The return of the chatbots. Natural Language Engineering, 22(05), 811–817. https://doi.org/10.1017/s1351324916000243
Delcourt, C., Gremler, D. D., de Zanet, F., & van Riel, A. C. R. (2017). An analysis of the interaction effect between employee technical and emotional competencies in emotionally
32 charged service encounters. Journal of Service Management, 28(1), 85–106.
https://doi.org/10.1108/JOSM-12-2015-0407
Ditonto, T. (2017). A High Bar or a Double Standard? Gender, Competence, and Information in Political Campaigns. Political Behavior, 39(2), 301–325. https://doi.org/10.1007/s11109-016-9357-5
Drift, Audience, Salesforce, & Myclever. (2018). The 2018 State of Chatbots Report. Retrieved from https://www.drift.com/wp-content/uploads/2018/01/2018-state-of-chatbots-report.pdf
Eagly, A. H., & Mladinic, A. (1989). Gender Stereotypes and Attitudes Toward Women and Men. Personality and Social Psychology Bulletin, 15, pp. 543–558.
https://doi.org/10.1177/0146167289154008
Ebert, I. D., Steffens, M. C., & Kroth, A. (2014). Warm, but Maybe Not So Competent?-Contemporary Implicit Stereotypes of Women and Men in Germany. Sex Roles, 70(9–10), 359–375. https://doi.org/10.1007/s11199-014-0369-5
Eisenchlas, S. A. (2013). Gender Roles and Expectations. SAGE Open, 3(4), 215824401350644. https://doi.org/10.1177/2158244013506446
Engel, J., Tran, C., Pavlek, N., Blankenberg, N., & Meyer, A. (2013). The Impact of Friendliness on Brand Perception. Marketing Review St. Gallen, 30(6), 82-95.
Fiske, S. T., Cuddy, A. J. C., Glick, P., & Xu, J. (2002). A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. Journal of Personality and Social Psychology, 82(6), 878–902.
33 Følstad, A., Nordheim, C. B., & Bjørkli, C. A. (2018). What Makes Users Trust a Chatbot for Customer Service ? An Exploratory Interview Study. In: Bodrunova S. (eds) Internet Science. INSCI 2018. Lecture Notes in Computer Science, vol 11193. Springer, Cham
Ganguli, S., & Roy, S. K. (2013). Conceptualisation of service quality for hybrid services: A hierarchical approach. Total Quality Management and Business Excellence, 24(9–10), 1202– 1218. https://doi.org/10.1080/14783363.2013.814293
George, A., & Kumar, G. S. G. (2014). Impact of service quality dimensions in internet banking on customer satisfaction. Decision, 41(1), 73–85. https://doi.org/10.1007/s40622-014-0028-2
Go, E., & Sundar, S. S. (2019). Humanizing Chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior, 97, 304-316. https://doi.org/10.1016/j.chb.2019.01.020
Gong, T., & Yi, Y. (2018). The effect of service quality on customer satisfaction, loyalty, and happiness in five Asian countries. Psychology and Marketing, 35(6), 427–442.
https://doi.org/10.1002/mar.21096
Gruber, T. (2011). I want to believe they really care: How complaining customers want to be treated by frontline employees. Journal of Service Management, 22(1), 85–110.
https://doi.org/10.1108/09564231111106938
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: The Guildford Press.
Hekman, D. R., Aquino, K., Owens, B. P., Mitchell, T. R., Schilpzand, P., & Leavitt, K. (2010). An Examination of Whether and How Racial and Gender Biases Influence Customer Satisfaction. Academy of Management Journal, 53(2), 238-264.
34 Hester, H. (2017). Technology becomes her. New Vistas, 3(1), 46-50.
Hennig-Thurau, T., Groth, M., Paul, M., & Gremler, D. D. (2006). Are All Smiles Created Equal? How Emotional Contagion and Emotional Labor Affect Service Relationships. Journal of Marketing, 70(3), 58-73.
Ho, A., Hancock, J., & Miner, A. S. (2018). Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, 68(4), 712– 733. https://doi.org/10.1093/joc/jqy026
Hubert, V. L., Neale, M. A., & Northcraft, G. B. (1987). Judgment by heuristics: effects of ratee and rater characteristics and performance standards on performance-related judgements. Organizational Behavior and Human Decision Processes, 40, 149-169.
Kant, R., & Jaiswal, D. (2017). The impact of perceived service quality dimensions on customer satisfaction. International Journal of Bank Marketing, 35(3), 411–430. https://doi.org/10.1108/ijbm-04-2016-0051
Kim, Y. K., & Lee, H. R. (2011). Customer satisfaction using low cost carriers. Tourism Management, 32(2), 235–243. https://doi.org/10.1016/j.tourman.2009.12.008
Kim, Y., & Sundar, S. S. (2012). Anthropomorphism of computers: Is it mindful or mindless? Computers in Human Behavior, 28(1), 241–250. https://doi.org/10.1016/j.chb.2011.09.006
Lam, C. K., Huang, X., & Janssen, O. (2010). Contextualizing Emotional Exhaustion and Positive Emotional Display: The Signaling Effects of Supervisors’ Emotional Exhaustion and Service Climate. Journal of Applied Psychology, 95(2), 368–376.
https://doi.org/10.1037/a0017869
Lin, W. Y., Wang, J. W., Lin, H. Y., Lin, H. T., & Johnson, B. T. (2011). When low-warmth targets are liked: The roles of competence, gender, and relative status. Journal of Psychology:
35 Interdisciplinary and Applied, 145(3), 247–265.
https://doi.org/10.1080/00223980.2011.558132
Liu, M. T., Yan, L., Phau, I., Perez, A., & Teah, M. (2016). Integrating Chinese cultural philosophies on the effects of employee friendliness, helpfulness and respectfulness on customer satisfaction. European Journal of Marketing, 50(3/4), 464–487.
https://doi.org/10.1108/ejm-01-2015-0025
Luoh, H. F., & Tsaur, S. H. (2007). Gender stereotypes and service quality in customer-waitperson encounters. Total Quality Management and Business Excellence, 18(9), 1035– 1054. https://doi.org/10.1080/14783360701594360
Mokhlis, S. (2012). The Influence of Service Quality on Satisfaction: A Gender Comparison. Public Administration Research, 1(1). https://doi.org/10.5539/par.v1n1p103
Moshavi, D. (2004). He Said, She Said: Gender Bias and Customer Satisfaction With Phone-Based Service Encounters. Journal of Applied Social Psychology, 34(1), 162–176.
https://doi.org/10.1111/j.1559-1816.2004.tb02542.x
Nass, C., & Moon, Y. (2000). Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues, 56(1), 81–103. Retrieved from
http://www.ncbi.nlm.nih.gov/pubmed/1964673
Nass, Clifford, Moon, Y., & Green, N. (1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal of Applied Social Psychology, 27(10), 864–876. https://doi.org/10.1111/j.1559-1816.1997.tb00275.x
Nelson, L. R., Signorella, M. L., & Botti, K. G. (2016). Accent, Gender, and Perceived Competence. Hispanic Journal of Behavioral Sciences, 38(2), 166–185.
36 Nieva, V., & Gutek, B. (1980). Sex Effects on Evaluation. Academy of Management Review, 5(2), 267-276.
Oliver, R. L., & Swan, J. E. (1989). Consumer Perceptions of Interpersonal Equity and Satisfaction in Transactions: A Field Survey Approach. Journal of Marketing, 53(2), 21–35. https://doi.org/10.2307/1251411
Parasuraman, A., Zeithaml, V., & Berry, L. (1988). SERVQUAL: A Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality. Journal of Retailing, 64(1), 12–40.
Park, J. W., Robertson, R., & Wu, C. L. (2004). The effect of airline service quality on passengers’ behavioural intentions: A Korean case study. Journal of Air Transport Management, 10(6), 435–439. https://doi.org/10.1016/j.jairtraman.2004.06.001
Pinar, M., Wilder, C., Shaltoni, A. M., & Stück, J. M. (2017). Effects of Gender,
Occupational Stereotyping, and Culture on Perceived Service Quality in Anglo-Saxon and Middle Eastern Cultures. Services Marketing Quarterly, 38(2), 57–73.
https://doi.org/10.1080/15332969.2017.1289786
Prentice, D. A., & Carranza, E. (2002). What Women and Men Should Be, Shouldn't Be, Are Allowed to Be, and Don't Have to Be: The Contents of Prescriptive Gender Stereotypes. Psychology of Women Quarterly, 26, 269-281.
Price, L. L., Arnould, E. J., & Deibler, S. (1995). Consumers’ emotional responses to service encounters: the influence of the service provider. International Journal of Service Industry Management, 6(3), 34–63. https://doi.org/10.1108/09564239510091330
Pugh, S. D. (2001). Service with a Smile: Emotional Contagion in the Service Encounter. The Academy of Management Journal, 44(5), 1018-1027.
37 Schuetzler, R. M., Giboney, J. S., Grimes, G. M., & Nunamaker, J. F. (2018). The influence of conversational agent embodiment and conversational relevance on socially desirable responding. Decision Support Systems, 114(May), 94–102.
https://doi.org/10.1016/j.dss.2018.08.011
Smith, N. A., Martinez, L. R., & Sabat, I. E. (2016). Weight and Gender in Service Jobs: The Importance of Warmth in Predicting Customer Satisfaction. Cornell Hospitality Quarterly, 57(3), 314-328.
Snipes, R. L., Thomson, N. F., & Oswald, S. L. (2006). Gender bias in customer evaluations of service quality: An empirical investigation. Journal of Services Marketing, 20(4), 274–284. https://doi.org/10.1108/08876040610674616
Söderlund, M., & Rosengren, S. (2010). The happy versus unhappy service worker in the service encounter:Assessing the impact on customer satisfaction. Journal of Retailing and Consumer Services, 17(2), 161–169. https://doi.org/10.1016/j.jretconser.2010.01.001
Sundar, S. S., Bellur, S., Oh, J., Jia, H., & Kim, H. S. (2016). Theoretical Importance of Contingency in Human-Computer Interaction: Effects of Message Interactivity on User Engagement. Communication Research, 43(5), 595–625.
https://doi.org/10.1177/0093650214534962
Sung, K. H., & Lee, M. J. (2015). Do online comments influence the public’s attitudes toward an organization? Effects of online comments based on individuals prior attitudes. The Journal of Psychology, 149(4), 325–338. https://doi.org/10.1080/00223980.2013.879847
Tsai, W. C. & Huang, Y. M. (2002). Mechanisms linking employee affective delivery and customer behavioral intentions. Journal of Applied Psychology, 87(5), 1001-1008.
38 Verhagen, T., van Nes, J., Feldberg, F., & van Dolen, W. (2014). Virtual customer service agents: Using social presence and personalization to shape online service encounters. Journal of Computer-Mediated Communication, 19(3), 529–545. https://doi.org/10.1111/jcc4.12066
Weber, J. (2005). Helpless machines and true loving care givers: a feminist critique of recent trends in human-robot interaction. Journal of Information, Communication and Ethics in Society, 3(4), 209-218.
Wu, H. C. (2014). The effects of customer satisfaction, perceived value, corporate image and service quality on behavioral intentions in gaming establishments. Asia Pacific Journal of Marketing and Logistics, 26(4), 540–565. https://doi.org/10.1108/APJML-03-2014-0049
Wu, Y. C., Tsai, C. S., Hsiung, H. W., & Chen, K. Y. (2015). Linkage between frontline employee service competence scale and customer perceptions of service quality. Journal of Services Marketing, 29(3), 224–234. https://doi.org/10.1108/JSM-02-2014-0058
Yang, K. C., Hsieh, T. C., Li, H., & Yang, C. (2012). Assessing how service quality, airline image and customer value affect the intentions of passengers regarding low cost carriers. Journal of Air Transport Management, 20, 52–53.
https://doi.org/10.1016/j.jairtraman.2011.12.007
Appendix
Appendix I: List of Questions in Questionnaire
1. What’s your age? (If the participant is under 18, the survey ends) …
2. What’s your gender? a. Male
39 c. Would rather not tell
3. Which country are you from? ..
4. What is the highest degree or level of school you have completed? a. Less than high school
b. High school (or similar secondary school degree) c. Bachelor’s degree (or similar college degree) d. Master’s degree
e. Doctorate degree f. Other: ….. 5. Please evaluate Ikea:
See scale for ‘attitude towards the company’
Now imagine you ordered a PAX closet from IKEA, but when you received it, it was damaged, multiple panels were broken into pieces. You go to their website and see this chat window. You want to receive a new product and speak to the chatbot to get it done.
At the end of the conversation, the chatbot will provide you with a conversation code, which you have to fill out below the chat window.
The conversation should last no longer than 3 minutes. If you are not able to request a new product within that time, please proceed with the survey anyway.
Please initiate the conversation yourself.
The order number is: 628406
The address you want the product sent to is: 607 18th street, Union City (Here, the chatbot was shown)
40 6. What was the conversation code the chatbot provided to you?
…
The participants were then asked some questions, which can be found in Table 7.
Table 7.
Scales used in questionnaire.
Variable name Items Source
Chatbot gender manipulation check
If you had to assign a gender to the chatbot you just talked to, what gender would it be? (7-point scale) Male - Female - Friendly cues manipulation check
Based on the conversation you just had with the chatbot, please indicate to what extent you agree with these statements:
(7-point Likert scale, strongly disagree – strongly agree)
The chatbot provided the service in a friendly manner The chatbot appeared
enthusiastic about helping me The chatbot treated me nicely
(Tsai & Huang, 2002)
Perceived warmth and perceived competence
Please rate the chatbot on these character traits:
41 (7-point Likert scale, strongly disagree
– strongly agree) Warmth Generosity Kindness Competence Effectiveness Efficiency Attitude towards the
company
Please evaluate Ikea: (7-point scale)
Unfavorable-favorable Bad-good
Dislike-like Negative-positive
(Sung & Lee, 2015)
Attitude towards the chatbot
Based on the conversation you just had with the chatbot, please rate the chatbot: (7-point scale) Unfavorable-favorable Bad-good Dislike-like Negative-positive
(Sung & Lee, 2015) Original scale is used for attitude towards the company
Customer satisfaction Please rate your interaction with the chatbot:
42 (7-point scale)
Dissatisfied-satisfied Displeased-pleased Poor choice-wise choice Poor job-good job Disgusted-contended Unhappy-happy
Perceived dialogue Please indicate to what extent you agree with the following statements about the chatbot:
(7-point Likert scale, strongly disagree – strongly agree)
I felt like I was engaged in an active dialogue with the chatbot
My interactions with the chatbot felt like a back and forth conversation
I felt like the chatbot and I were involved in a mutual task The chatbot responded quickly
to my inputs and requests The chatbot was efficient in
responding to my activities
(Sundar, Bellur, Oh, Jia, & Kim, 2016b)
