4. Data Analysis
4.4 Hypotheses testing
To answer the stated hypothesis, multiple test has been used. The first test is the repeated measures ANOVA. This measure determines if there are statistically significant differences among the means of multiple independent and unrelated groups. The repeated measures ANOVA analyses, also named the within-subject ANOVA, is an equivalent of the one-way ANOVA and also an extension of another SPSS test, namely; paired-sample t-test (Ståhle &
Wold, 1989). When using this test, multiple assumptions need to be taken into account. Firstly, there should not be any outliers in the dataset. Secondly, homogeneity among the variances is needed. This can be tested with the Levene’s homogeneity test. Thirdly, the independent variable need to contain at least two categorical and independent groups. Fourthly, the dependent variable should be a normally distributed, when it comes to the category of the independent variable. Fifthly, while measuring the dependent variable there needs to be taken into account that the dependent variables needs to be measured as a ratio level or at interval level. Lastly, there should not be a relationship between the groups (Ståhle & Wold, 1989).
After the assumptions have been met, the ANOVA can be used in case the independent variable is nominal. For this research, the repeated measures ANOVA has been used to answer the hypotheses, since the research includes a within-subject design with 4 different conditions. The same participants participated in all conditions of the experiment and have been measured on the same dependent variable (customer trust) several times (Field, 2009).
The other test that has been used is the repeated measure ANCOVA. This test is used when it is expected that another variable (covariate), apart from the independent variables, influences the dependent variable. As the correlation matrix showed that certain variables were correlating with the control variable, namely gender, the repeated measures ANCOVA was performed.
This to control confounding variables (Field, 2009). The difference between the repeated measures ANOVA and the repeated measure ANCOVA, is that the ANCOVA includes a
covariate that influences the dependent variable, while the ANOVA does not need to take the covariate into account.
Next to the ANOVA and ANCOVA is the PROCESS measure used. This extension is created by The Guilford Press and Hayes (Prado, Korelo, Jose & Mantovani, 2014). The PROCESS extension is a tool that analyses the indirect relationship and both two- or three-way moderated interactions of the conceptual model with a simple slope and regions (see figure 4). This type of analysis has been used to identify the effect of the moderators and can be found in the next chapters. Since the conceptual model contains moderators, this SPSS add-on has been used.
Figure 4: PROCESS extension
4.4.1 Hypothesis 1: direct effect of service type on customer trust
Hypothesis 1 stated “Human service will yield a more positive effect on customer trust, compared to AI service”. As the correlation matrix showed that gender was correlating with average trust, a repeated measure ANCOVA was executed. Gender was executed as covariate, to reduce any bias. The repeated measures ANCOVA showed that Mauchly’s sphericity was violated and therefore the Greenhouse-Geisser correction was used. The tests of within-subjects effects showed that the service types did significantly differ from each other in terms of customer trust (F(1, 333)=7,02, ρ=0,008, np2=0,021) and showed a small effect size. In other words, the customer trust was influenced by the type of service provided. The post-hoc test showed that there was indeed a difference between the service types and customer trust
M
Y X
(ρ=0,000). This is supported by the descriptive results, which showed that human service (M=3,79) resulted in a higher customer trust compared to AI service (M=3,13). It is interesting to see that both human and AI service resulted in positive trust levels, however, that human service was significantly higher. In sum, human service will have a more positive effect on customer trust compared to AI service. This indicates that hypothesis 1 can be supported.
Table 10: Repeated measures ANCOVA findings Hypothesis 1
M df F value p value np2
Service Type 3,46 1 ; 333 7,02 0,008 0,021
N=335
4.4.2 Hypothesis 2a & 2b: moderation effect of TA
Looking at the covariate gender, there could be indicated that there was no interaction effect (F(1, 333)=2,269, ρ=0,133, np2=0,007). Therefore, no further analysis was needed.
Hypothesis 2a stated that “Low TA is expected to have a more positive effect on customer trust, when service is provided by humans, compared to service provided by AI”. As the correlation matrix showed that gender was not correlating with the trust of low TA, a repeated measure ANOVA was executed. To test this hypothesis, the moderator TA was incorporated as a between-subject factor. The test showed that Mauchly’s test of Sphericity was violated and therefore the Greenhouse-geisser was used. This test showed that the TA-groups did significantly differ from each other (F(1,172)=5,260, ρ=0,023, np2=0,030). This means that customer trust of low TA customers is influenced by service type. This was confirmed by the post-hoc test, which indeed showed that there was a difference between the service type and customer trust of low TA customers (ρ=0,000). The descriptive results showed
AI service (M=3,039). This indicates that hypothesis 2a could be supported and that the customer trust of low TA customers is higher for human service compared to AI service.
To answer hypothesis 2b, stating “High technology acceptance is expected to have a more positive effect on customer trust, when service is provided by AI, compared to service provided by humans”, the repeated measures ANOVA has been used. The reason for this is that gender did not show a correlation with high TA. Again, the moderator TA was used as a between-subject factor. Again, the Greenhouse Geisser was used, as Mauchly’s Test of Sphericity was violated. The tests of within-subjects showed that there was no significant interaction effect between the high TA group and service type (F(1,159)=1,989, ρ=0,160, np2=0,012). This indicates that both human- as AI service do have the same effect on customer trust of high TA participants. Looking at the post-hoc test, the customer trust of high TA participants seems to be statistically different between human- and AI service (ρ=0,000). This is confirmed by the descriptive results, which show that the human service results in a higher customer trust (M=3,924) compared to AI service (M=3,225). Although both customer trust of human and AI is positive, this indicates that hypothesis 2b cannot be supported as human service results in a higher trust level for high TA customers. However, it is outstanding that the customer trust of AI service is slightly higher for high TA customers compared to low TA customers. This indicates that high TA customers rate the robotic service a little higher.
Table 11: Repeated measures ANOVA findings Hypotheses 2a & 2b Technology
Acceptance
M df F value p value np2
Low 3,35 1 ; 172 5,260 0,023 0,030
High 3,57 1 ; 159 1,989 0,160 0,012
N=335
Looking at the moderation model with customer trust as dependent variable, service type as independent variable and TA as moderator, the PROCESS SPSS (Model 1, 5000 bootstraps, 95% CI) shows a non-significant interaction effect (b=0,0683, SE=0,1316, -0.1901;0,3267), t(0,5190), ρ=0,6039). This indicates that TA had no moderating effect on service type on customer trust and therefore does not influence this direct effect.
Table 12: PROCESS analysis moderation effect TA
Summary R R-sq MSE F df1 df2 p
0.3936 0.1550 0.7239 30.4848 4.0000 665.0000 0.0000
Model Source Coeff SE t p LLCI ULCI
constant 2.5904 0.3408 7.6014 0.0000 1.9213 3.2596
Service 0.5622 0.2056 2.7341 0.0064 0.1584 0.9659
TA_group 0.1049 0.2081 0.5039 0.6145 -0.3038 0.5135
Int_1 0.0683 0.1316 0.5190 0.6039 -0.1901 0.3267
Gender -0.1952 0.0664 -2.9404 0.0034 -0.3256 -0.0649
4.4.3 Hypothesis 3a & 3b: moderation effect of task characteristics
Looking at hypothesis 3a, this hypothesis states that “Cognitive tasks will have a more positive effect on customer trust when service is provided by AI, compared to service provided by
humans”. As gender was correlating with the cognitive task characteristic, this variable was executed as a covariate. The repeated measures ANCOVA showed that again Mauchly’s sphericity was violated, indicating that the Greenhouse-Geisser was used. The within subjects effects test showed that the customer trust of cognitive tasks was not statistically affected by the different service types (F(1, 333)=3,455, ρ=0,064, np2=0,010). This indicates that, although the test was close to being statistically significant, both human- as AI service do have the same effect on customer trust of cognitive tasks. However, again looking at the post-hoc test; the customer trust for a cognitive task seems to be statistically different between human- and AI service (ρ=0,000). The descriptive results do actually support this, as they show that the human service results in a higher mean (M=3,87), compared to AI service (M=3,28). Again, both service types result in positive trust levels, however, human service seems to result in higher trust levels. This actually means that the hypothesis cannot be supported, as this indicates that human service results in higher trust levels for cognitive tasks compared to AI service.
Furthermore, the control variable gender showed that there was no interaction effect with the customer trust of cognitive tasks (F(1,333)=2,125, ρ=0,146, np2=0,006). Therefore, no further analysis was conducted.
Hypothesis 3b states that “Social intelligence tasks will have a more positive effect on customer trust when service is provided by humans, compared to service provided by AI”.
Again, the repeated measures ANCOVA was performed, with gender as a covariate. Mauchly’s test of Sphericity was again violated and the Greenhouse Geisser correction showed that the service types did significantly differ from each other in terms of customer trust of social intelligence tasks (F(1, 333)=7,538, ρ=0,006, np2=0,022). This indicates that both human- as AI service do have a different effect on the customer trust of social intelligence tasks. This was supported by both the post-hoc test (ρ=0,000) and the descriptive results, as they show that the mean of human service (M=3,717) is significantly higher compared to the mean of AI service
(M=2,979). This means that the hypothesis can be supported, as it indicates that human service results in more customer trust for social intelligence tasks. Besides, it is outstanding that AI service seems to be slightly negative for this type of service and that it is lower compared to cognitive tasks. This indicates that customers do trust AI service related to cognitive tasks more compared to social intelligence tasks. Although this was not a hypothesis of the study, it is an interesting finding.
Looking at the control variable gender, again no interaction effect could be identified (F(1,333)1,394, ρ=2,39, np2=0,004).
Table 13: Repeated measures ANCOVA findings Hypothesis 3a & 3b
Tasks M df F value p value np2
Cognitive 3,58 1 ; 333 3,455 0,064 0,010
Social intelligence
3,35 1 ; 333 7,538 0,006 0,022
N=335
Looking at the moderation model with customer trust as dependent variable, service type as independent variable and cognitive task characteristics as moderator, the PROCESS SPSS (Model 1, 5000 bootstraps, 95% CI) shows a significant interaction effect (b=-01701, SE=0,0409, -0.2505;-0,0898), t(-4.1571), ρ=0,00). This indicates that the cognitive task characteristic influences the level of customer trust, which was identified by investigating hypothesis 3a; which both indicated that the trust levels of cognitive task characteristics did significantly differ.
Table 14: PROCESS analysis moderation effect cognitive characteristics
Summary R R-sq MSE F df1 df2 p
0.8592 0.7383 0.2242 468.9497 4.0000 665.0000 0.0000
Model Source Coeff SE t p LLCI ULCI
constant -0.3950 0.2275 -1.7363 0.0830 -0.8417 0.0517
Service 0.8198 0.1532 5.3509 0.0000 0.5190 1.1206
Cog task 1.0222 0.0612 16.7108 0.000 0.9021 1.1423
Int_1 -0.1701 0.0409 -4.1571 0.000 -0.2505 -0.0898
Gender -0.0623 0.0371 -1.6797 0.0935 -0.1350 0.0105
Looking at the outcome of the PROCESS SPSS data (Model 1, 5000 bootstraps, 95% CI) with the social intelligence task characteristic as moderator, a significant interaction effect has been identified (b=-0,1230 SE=0,0284, -0.1788;-0,0672), t(-4.3278), ρ=0,00). This indicates that the social intelligence task characteristic influences the level of customer trust, which was also identified by investigating hypothesis 3b.
Table 15: PROCESS analysis moderation effect social intelligence task
Summary R R-sq MSE F df1 df2 p
0.9043 0.8178 0.1561 746.2496 4.000 665.000 0.0000
Model Source Coeff SE t p LLCI ULCI
constant 0.3931 0.1525 2.5779 0.0102 0.0937 0.6925
Service 0.5682 0.1022 5.5615 0.0000 0.3676 0.7688
Cog task 0.8710 0.0426 20.4312 0.0000 0.7873 0.9547
Int_1 -0.1230 0.0284 -4.3278 0.0000 -0.1788 -0.0672
Gender -0.0424 0.0309 -1.3711 0.1708 -0.1032 0.0183
4.4.4 Hypothesis 4a & 4b: moderation effect of age
Hypothesis 4a states “Digital natives (people born after 1980), will have a more positive effect on customer trust when service is provided by AI, compared to service provided by humans”.
As gender did not show a correlation with age groups, gender was not used as a control variable. Therefore, the repeated measures ANOVA was used. Again, the moderator age group was used as a between-subject factor. Mauchly’s test of Sphericity was violated and the Greenhouse-Geisser showed that the service type had a significant difference for the customer trust of digital natives (F(1,211)90,464, ρ=0,000, np2=0,300). Looking at the post-hoc test (ρ=0,000) and the descriptive results, this can be confirmed. The descriptive results show that
AI service (M=3,26). Although not expected, this again indicates that human service is preferred by customers born after 1980’s who are expected to be familiar with technology. This means that hypothesis 4a is not supported.
Looking at hypothesis 4b, stating “Digital immigrants (people born before 1980), will have a more positive effect on customer trust when service is provided by humans, compared to service provided by AI”, the repeated measures ANOVA showed that Mauchly’s test of Sphericity was again violated. The Greenhouse-Geisser showed that the service types resulted in significantly different customer trust for digital immigrants (F(1, 122)=65,329, ρ=0,000, np2=0,349). Again, this was supported by the post-hoc test (ρ=0,000). The descriptive results did also confirm this, as the means show that digital immigrants have a significantly lower trust in AI service (M=2,90), compared to human service (M=3,66). This again indicates that human service is more trusted over AI service, which was also expected as digital immigrants (born before 1980) are expected to be less familiar with technology. Therefore, hypothesis 4b is supported.
Table 16: Repeated measures ANOVA findings Hypothesis 4a & 4b
Tasks M df F value p value np2
Digital Immigrants 3,57 1;211 90,464 0,000 0,300
Digital Natives 3,28 1; 122 65, 329 0,000 0,349
N=335
The moderation model with customer trust as dependent variable, service type as independent variable and age groups (digital natives vs digital immigrants) as moderator, the PROCESS SPSS (Model 1, 5000 bootstraps, 95% CI) shows a non-significant interaction effect (b=-01492, SE=0,1356, -0.1171;0,4155), t(1,1003), ρ=0,2716). This indicates that the different age
groups had no moderating effect on the direct effect of service type on customer trust and therefore does not influence this direct effect.
Table 17: PROCESS analysis moderation effect age groups
Summary R R-sq MSE F df1 df2 p
0.4053 0.1643 0.7159 32.6796 4.0000 665.0000 0.0000
Model Source Coeff SE t p LLCI ULCI
constant 3.4342 0.3240 10.5986 0.000 2.7979 4.0704
Service 0.4593 0.1966 2.3358 0.0198 0.0732 0.8453
Age_Gr -0.4993 0.2145 -2.3281 0.0202 -0.9204 -0.0782
Int_1 0.1492 0.1356 1.1003 0.2716 -0.1171 0.4155
Gender -0.1993 0.0659 -3.0235 0.0026 -0.3287 -0.0699