FaceReader Online as a tool for measuring
website design and task complexity
Lisanne Talen (10726616)
Supervisor: Tess den Uyl
24/04/2020
Abstract
Due to the role of the Internet in modern life and the enormous variety of websites, it is important that users have a good experience on the website. Earlier research found that a good experience influences important user behaviour such as further use of the website. Complexity seems to play a role in the usability of websites and a blockage or delay in reaching the goal leads to negative feelings such as frustration. Emotional behaviour of users is an increasing research topic in human-computer interaction, and facial expressions reflect emotional experience. In this research FaceReader Online was used to measure the effect of complexity of website design and the complexity of tasks on the facial expression anger. It was found that the complexity the tasks increased the facial expression of anger. These results suggest that the facial expression anger could be used to measure the usability of a website. Moreover, FaceReader Online proves to be a useful tool in usability research, although there are some important aspects to be aware of.Keywords
FaceReader Online, Usability, Complexity, Websites, Facial Expressions, AngerIntroduction
Nowadays the use of computers is highly incorporated in people’s daily life. The daily use of computers leads to an enormous variety of websites on the Internet. In the labyrinth of all those different kinds of websites it is important that users have a good experience, because this influences important user behaviour such as further use of the site. Research show that a good user experience is associated with greater engagement in the website and increased positive attitudes towards the website (King, Lazard & White, 2019; Nielsen, 2000). Also the sales of a site are affected by the experience of the user (Venkatesh & Agarwal, 2006). To establish a good user experience, it is important that users do not get disturbed or distracted when they explore the website, and be able to reach their goal easily (Lazar et al., 2003). In specific, visual complexity is one aspect that can influence usability of webpages (Comber & Maltby, 1997). Earlier research found a negative relationship between website complexity and satisfaction and pleasure (Nadkarni & Gupta, 2007; Pandir & Knight, 2006). Besides that, it was found that users perform better on search tasks on low visual complex websites compared to high visual complex websites (Tuch et al., 2009). Whenever there is a blockage, delay or interference in obtaining the goal on the website this will lead to frustration by the user (Lazar et al., 2003). Visual complex pages are difficult to use (Harper, Michailidou & Stevens, 2009) and this leads to greater levels of frustration (Hazlett, 2003). The visual complexity of a website depends on the type of elements (amount of text, images, menus) represented and the diversity, density and positioning of them (Harper et al. 2009). To measure the complexity of websites a tool is needed that is capable of measuring the fast initial impression and usability. One way to measure the complexity of websites and tasks is to simply ask the user. Self-report helps in understanding the experience of the user on the website, but there are a few disadvantages. Users may be limited in sharing their true feelings about the website because of social desirability. Inaddition, the initial impression is formed within 50 milliseconds (Lindgaard et al., 2006), but to report thoughts cognitive processing is required. Therefore dissimilarity between the reported thoughts and the true first impression may arise (Poels & Dewitte, 2006). An important and increasing research topic in human-computer interaction is the user’s emotional behaviour (Branco et al., 2005; Hibbeln et al., 2017). Emotions can appear far more quickly than cognitive responses and facial expressions begin to show already a few milliseconds after exposure of the stimulus (Ekman, 1992; Epstein, 1994). This suggests that facial expressions could be an interesting tool for measuring initial impressions and usability. Research shows that facial expressions are a representation of the occurrence of spontaneous emotions (Ekman, Friesen & Ancoli, 1980). Several studies already used facial expressions to measure emotional behaviour. Stone & Wei (2011) for example used the Facial Action Code System (FACS, created by Ekman & Friesen (1978) to describe the movements of the face) to measure change in facial expressions by visual observation. Moreover, facial electromyography (EMG), which measures changes in the electrical activity of muscles in the face, is used to measure facial expressions (Hazlett & Benedek, 2007; Branco et al., 2005). These methods both are labour-intensive and cost a lot of time. Also, facial EMG is only able to tell how positive or negative someone’s emotional state is, not to classify discrete emotions. FaceReader Online is automated facial coding (AFC) software that is able to categorize facial movements into emotions (Lewinski et al., 2014). The software gathers data via a webcam and classifies this data into discrete categories of the five universal basic emotions (Ekman & Cordaro, 2011). The accuracy of FaceReader Online in recognizing the basic emotions is as good as the accuracy of humans. This method is low in labour intensity and accurate in recognizing basic emotions and is therefore a useful tool of measuring facial expressions. In addition, with FaceReader Online it is possible to make recordings with people’s own webcams so there is no lab needed to perform the experiment. In this study FaceReader Online was used to investigate the effect of website design and task complexity on facial expressions. Earlier research with FaceReader shows a relationship between emotional valence and the subjective impression of complexity of the webpage (Goldberg, 2012). Positive facial expressions as joy and surprise can be classified as having positive emotional valence. Negative facial expressions as anger, sadness and fear can be classified as having negative emotional valence. In this study emotional valence was more negative during more complex subjective ratings. Also, both visual complex websites as a delay or blockage in obtaining the goal leads to more frustration by the user (Hazlett, 2003; Harper, Michailidou & Stevens, 2009; Lazar et al., 2003). Hazlett (2003) measured frustration with facial EMG using the corrugator muscle, which lowers the eyebrow. In the FACS brow lowering is described as Action Unit 4 (AU4). Grafsgaard et al., (2013) showed an association between AU4 and increased frustration. AU4 also contributes to negative facial expressions like anger (Ekman & Friesen, 1975; Ekman & Friesen 1978). Based on these findings, it is hypothesized that both website design complexity as task complexity will increase negative facial expressions. In addition, this research will examine whether a difficult task on a website leads to the same facial expressions as a difficult math task. Waterink & van Boxtel (1994) found that increased activity of the corrugator muscle is associated with greater mental effort. Therefore it is also hypothesized that performing a difficult math task leads to more negative facial expressions. This research will measure the effect of website design complexity and task complexity on facial expressions using three different tasks. To measure the effect of
website design complexity on facial expressions, participants were shown start pages of websites with different levels of complexity. To measure the effect of task complexity on facial expressions, two different tasks were set up. First, participants performed search tasks with a various level of difficulty on a website and second, participants solved arithmetic tasks with various levels of difficulty. During these tasks, facial expressions were measured using FaceReader Online. It is expected that the complexity of website design and the complexity of tasks leads to frustration by the user, which leads to increased intensity of the facial expression anger. In addition, it is expected that performing difficult arithmetic tasks lead to greater mental effort, which also leads to increased intensity of the facial expression anger.
Methods
Participants In this experiment, 75 people, participating via Mechanical Turk, Microworkers and the University of Utrecht, finished the experiment with sufficient recording quality. In total 204 participants started the experiment, of which 110 quitted the procedure early and 19 did not reached sufficient recording quality (Fig. 1). The quality of the recording was based on the number of correctly recorded frames and ranges on a scale from 0 to 10. When participants had recordings with a quality of 2 or below, all recordings of that participant were excluded from the experiment. The mean quality of the included recordings was 9.27 (SD = 1.15; range = 3-10). 31 of the included participants were female (M = 30.7 years, SD = 8.9; range = 21-54) and 44 were male (M = 31.5 years, SD = 9.9; range = 18-58). 25% of the participants had a Dutch nationality and 63% a non-Dutch European nationality. Furthermore 84% of the participants had an intermediate or better English language level and 64% had a college degree. Lastly 93% of the participants felt at least somewhat comfortable about using the Internet. People needed a computer with functioning webcam and a Chrome browser to participate. Participants gave online informed consent. Design & procedure This experiment used a factorial repeated measures design with the complexity of website design and the complexity of tasks as a within-subject variable. The experiment existed of three tasks and corresponding questions. Participants performed the experiment online at their own computer and did not have to come to a lab. Before the experiment started the participants were informed about the content of the research and got instructions for the camera position. The research was performed with FaceReader Online embedded in a Qualtrics survey. After the participants filled out a questionnaire about demographic information, the experiment started. The camera was only used when the participants performed the tasks to measure the facial expressions. The entire procedure lasted approximately 15 minutes.Figure 1 Flow diagram of participants Note. In this diagram it is outlined in which part of the experiment participants quitted the procedure and were excluded.
Website design complexity In the first task participants watched homepages of websites with a different level of complexity for ten seconds: a simple page (low amount of features) and a complex page (high amount of features). The used web pages are online travel blogs and participants watched them with the following message: “You’re gathering inspiration for your upcoming trip, you stumble upon the following websites. Where would you tend to click on?”. After the tasks the participants filled out questions about the websites using a 7- point Likert scale to determine the subjective complexity (Appendix A). The stimuli used for this task were videos of ten seconds containing screenshots of the start page of existing websites on the Internet that were themed similar. The experiment contained four stimuli: two conditions with each a simple start page and a complex start page. The original pages that were used existed of a low amount of features and thus were considered as the ‘simple’ pages (Fig. 2). For simple page 2 the font size and colour of the headers were corrected to increase readability. To construct the ‘complex’ pages, extra features were added to the original screenshots (Fig. 3). In the first condition, simple page 1 and complex page 2 were shown, whereas in the second condition, simple page 2 and complex page 1 were shown. Within the conditions, the order of showing the simple or complex page first was randomized. Search tasks on a website In the second task participants searched for specific articles on a website. The participants performed an easy and a difficult task. In the easy task the participants were given the title of an article that literally exits on the website and got instructed to search for this article. The article could be found in one click. In the difficult task the participants were given the title of an article that did not literally exists on the website and got instructed to search for this article. The article was almost impossible to find (see Appendix B for extensive description of the tasks). For both tasks the participants had one minute to search on the website and twenty seconds to indicate whether they found the article or not. After the tasks the participants rated the difficulty of the tasks on a 7-point Likert scale to determine the subjective complexity. This task used two existing travel blogs with a different level of complexity: a simple website (low amount of features) and a complex website (high amount of features) (Fig. 4). The easy task was always followed by the difficult task. The task contained two conditions: in one condition the easy task was performed on the simple website; in the other condition the easy task was performed on the complex website. The order of the conditions was randomized between participants. Arithmetic tasks In the third task the participants had to solve four different arithmetic problems in thirty seconds, which varied from easy to hard, to check whether they showed the same facial expressions as during the difficult task. The participants were offered four multiple-choice options for every problem and were told to leave the question blank if they did not know the answer. After this task the participants also had to rate the difficulty of the tasks on a 7-point Likert scale to determine the subjective difficulty. In this task the participants have to solve the following arithmetic tasks in this order: 4x6, 7x14, 12x18 and 16x17.
Figure 2 Representation of the simple pages Note. The left image represents simple page 1. The right image represents simple page 2. Figure 3 Representation of the complex pages Note. The left image represents complex page 1. The right image represents complex page 2. The two screenshots contain the same extra features, to keep them as similar as possible. Figure 4 Representation of the simple and complex websites Note. The left image shows a screenshot of the start page of the simple designed website. The right image shows a screenshot of the start page of the complex designed website.
Statistical analysis To test the effect of website design complexity and task complexity on facial expressions, a factorial repeated measures ANOVA was used with complexity as between-subjects variable and facial expressions as within-subjects variable. For task one and task two complexity contained two categories: simple and complex/easy and difficult. For task three complexity contained four categories: the arithmetic tasks from easy to hard. For all three tasks facial expressions contained six categories: happy, sad, angry, surprised, scared and disgusted. These six categories were represented by continuous scores, where 0 was the minimum score (minimal intensity of the facial expression) and 1 was the maximum score (maximal intensity of the facial expression). The used scores were the averages of the scores measured during the time the task lasted1. The average facial expression scores measured over the whole task contained a lot of scores that were close to 0 (0.001<). A score of 0 signified minimal intensity of the facial expression, and thus low facial expression intensity was measured in this data sample. An explanation for these low facial expression intensity scores is that facial expressions usually have a rapid onset and a short duration, and so do not last for the whole task. Therefore the average of the five seconds of the maximal angry score were conducted from the data. Lewinski, Fransen & Tan (2014) used a similar method to analyse facial expression data from FaceReader. The scores of the facial expression angry seemed relevant due to the brow lowering Action Unit in it. To test the effect of website design complexity and task complexity on the expression angry a Wilcoxon Signed-Ranks test was performed. To test the effect of arithmetic tasks on the facial expression angry, a Friedman’s ANOVA was used. The subjective complexity and the subjective difficulty of the search task were tested with a Wilcoxon Signed-Ranks test. The completion time of the search task also was tested with a Wilcoxon Signed-Ranks test. For the subjective difficulty and completion time of the arithmetic tasks a Friedman’s ANOVA was used.
Results
Website design complexity Facial expressions A Wilcoxon Signed-Ranks test showed no significant difference in angry scores between the simple website design (Mdn = 0.020) and the complex website design (Mdn = 0.027), Z = -1.13, p = 0.26, r = 0.13 (Fig. 5). Subjective complexity A Wilcoxon Signed-Ranks test showed that the complex website design (Mdn = 2) was rated significantly more complex than the simple website design (Mdn = 4), Z = -5.89, p < 0.001, r = 0.68 1For the design complexity task and the arithmetic task there was no significant interaction found between the level of complexity/difficulty and facial expressions. For the search task a small significant interaction found between the level of difficulty and facial expressions. See appendix C for results.Figure 5 Effect of website design complexity on facial expression angry Note. Boxplot of angry scores of the simple and complex website design. Error bars represent the interquartile range. There is no significant difference in angry scores between the simple website design and the complex website design. Difficulty of search tasks on websites Facial expressions A Wilcoxon Signed-Ranks test showed significantly higher angry scores during the difficult search task compared to the easy search task, Z-value = -2.68, p = 0.007, r = 0.31 (Fig. 6; Table 1). Subjective difficulty and completion time A Wilcoxon Signed-Ranks test showed that the hard search task was rated significantly more difficult than the easy search task, Z-value = -3.07, p = 0.002, r = 0.35. Furthermore participants needed significantly more time to complete the hard search task compared to the easy search task, Z-value = -2.09 p = 0.037, r = 0.24 (Table 1).
Table 1 Medians of outcome measures and percentage of completed search tasks Easy task N = 75 Hard task N = 75 Mdn Mdn Angry score 0.062 0.085 Mdn Mdn Subjective difficulty 4 5 Mdn Mdn Completion time (sec) 63.6 63.4 % % Article found 73% 20% Note. The percentage of article found represents how many participants found the article during the search task. The completion time includes the response time of the participants. Figure 6 Effect of task difficulty on facial expression angry Note. Boxplot of angry scores of the easy and hard task. Error bars represent the interquartile range. Participants showed significantly more angry expression during the hard task compared to the easy task.
Difficulty of arithmetic tasks Facial expressions A Friedman’s ANOVA showed a significant difference between the angry scores of the arithmetic tasks, X2F(3) = 31, p < 0.001. Post-hoc testing showed a significant difference between task one and three (Z = -2.41, r = 0.28) and between task one and four (Z = -4.24, r = 0.49) (Fig. 7; Table 3). The angry scores of participants during task three and four were significantly higher than the scores of task one (Table 2). Table 2 Medians of outcome measures and percentage of correct answered arithmetic tasks 4x6 N = 75 7x14 N = 75 12x18 N = 75 16x17 N = 75 Mdn Mdn Mdn Mdn Angry score 0.013 0.016 0.022 0.041 Mdn Mdn Mdn Mdn Subjective difficulty 1 2 5 6 Mdn Mdn Mdn Mdn Completion time (sec) 4.80 10.0 14.4 19.8 % % % % Correct answer 100% 81% 73% 44% Note. The percentage of correct answers represents how many of the participants knew the correct answer of the task. Table 3 P-values post-hoc Wilcoxon Signed-Ranks test angry scores arithmetic tasks Arithmetic task 4x6 7x14 12x18 7x14 0.051 - - 12x18 0.016 1.00 - 16x17 0.001< 0.069 0.065 Note. The alpha levels of the p-values were Bonferroni-adjusted.
Figure 7 Effect of arithmetic tasks on facial expression angry Note. Boxplot of the angry scores of the arithmetic tasks. Error bars represent the interquartile range. The facial expression angry of participants was significantly higher during the third and fourth task compared to the first task. Subjective difficulty and completion time A Friedman’s ANOVA showed a significant difference between the subjective difficulty of the arithmetic tasks, X2F(3) = 193, p < 0.001. Post-hoc testing showed a significant difference between all the tasks (Table 4). The subjective difficulty of task four was rated highest, followed by task three, two and one in this order (Table 2). Furthermore, a Friedman’s ANOVA showed a significant difference between the time the participants needed to complete the tasks, X2F(3) = 135, p < 0.001. Post-hoc testing showed a significant difference between all the tasks in completion time (Table 5). Participants needed the most time to complete task four, followed by task three, two and one in that order. Table 4 P-values post-hoc Wilcoxon Signed-Ranks test subjective difficulty arithmetic tasks Arithmetic task 4x6 7x14 12x18 7x14 0.001< - - 12x18 0.001< 0.001< - 16x17 0.001< 0.001< 0.001< Note. The alpha levels of the p-values were Bonferroni-adjusted.
Table 5 P-values post-hoc Wilcoxon Signed-Ranks test completion time arithmetic tasks Arithmetic task 4x6 7x14 12x18 7x14 0.001< - - 12x18 0.001< 0.001< - 16x17 0.001< 0.001< 0.001< Note. The alpha levels of the p-values were Bonferroni-adjusted.
Discussion
The goal of this research was to examine the effect of website design complexity and task complexity on facial expressions using FaceReader Online (FRO). It was expected that both the complexity of website design and tasks would increase negative facial expressions. Both the search tasks on the website as the arithmetic tasks show higher anger scores for more difficult tasks. This results show that task complexity increases the negative facial expression anger. Moreover, the complexity of the search tasks and arithmetic tasks show an effect on the subjective difficulty. Participants rated the more complex tasks as more difficult. Also, the completion time of both the search tasks and the arithmetic tasks increased with the more complex tasks. The results of the subjective task difficulty and completion time are in line with the increased facial expression anger and show that the manipulation of the experiment succeeded. In this research, difficulty of searching tasks on a website increased the activity of the facial expression anger. Because it is not likely that searching on a website makes participants really angry, there are other factors that contribute to the increase of the facial expression anger. The facial expression anger contributes Action Unit 4, brow lowering (Ekman & Friesen, 1975; Ekman & Friesen 1978). Action unit 4 is associated with two affective states: frustration (Grafsgaard et al., 2013; Hazlett, 2003) and greater mental effort (Waterink & van Boxtel, 1994). One explanation of the increased anger activity during a difficult task on a website compared to an easy task, is that participants got frustrated. Earlier research shows that people got frustrated when they are not able to reach their goal on a website (Lazar et al., 2003). During the difficult search task in this experiment, participants also were not able to reach their goal. A second explanation of the increased anger activity during a difficult task is greater mental effort. The arithmetic tasks in this experiment showed the same increase in anger activity as the search tasks on the Internet, and for both tasks the subjective difficulty was rated similar. So it is possible that participants were more concentrated because of the greater mental effort during difficult tasks, and thus showed increased expression of anger. Besides task complexity, the effect of website design complexity on the facial expression anger was examined. The results show that website design complexity has no effect on the anger scores. Website design complexity on the other hand does have an effect an effect on the subjective complexity ratings, with the complex website design rated more complex than the simple website design. Earlier research of Goldberg (2012) found a relation between emotional valence and subjective complexity ratings, where emotional valence was more negative with more complex ratings. This researchdid not show similar results, because there was no effect of complexity on facial expressions found. Research of Tuch et al., (2009) also tried to measure the complexity of website design. They used Berlyne’s (1974) aesthetic theory, which states that a viewer’s pleasure is related to the potential arousal of a stimulus, to explain the effect of website complexity on affective valence. In this theory the relationship between pleasure and the arousal potential of a stimulus is expressed as an inverted U-shaped curve. The theory thus suggests that moderate complex stimuli will be considered pleasant, whereas low and high complex stimuli will be considered unpleasant. Tuch et al., (2009) did not find this inverted U-shaped curve between the stimulus complexity and pleasure and suggested that the range of the visual complexity of the start pages they used was not broad enough. The results of this research showed no effect of website design complexity on the facial expression anger, but the complex designed websites were rated more complex than the simple ones. The complex pages were designed by adding more features to the original (simple) ones. As suggested by Tuch et al., (2009), it could be possible that the complexity of the designed pages were still not broad enough, and therefore no change in the facial expression anger was measured. For the first analyses the average of the facial expression scores of whole task were used. This data contained a lot of scores close to 0 (0,001<), which means that a low intensity of the facial expressions was measured. Therefore it was hard to find an interaction between the complexity level and facial expressions. Emotions have a rapid onset and a short duration (Ekman, 1992), and therefore do not last the whole task. It seemed more logical to conduct the average of the five seconds of the maximal angry score from the scores of the whole task. With these maximal angry scores it was tried to better capture the negative experience of the user. Lewinski, Fransen & Tan (2014) did something similar, when measuring facial expressions. It was decided to analyse only the facial expression anger, because it seemed to be a good representation of negative facial expressions. Action Unit 4 (brow lowering) contributed to the facial expression angry, but was also associated with frustration (Grafsgaard et al., 2013) and mental effort (Waterink & van Boxtel, 1994). The analysis with the maximal angry scores does show an effect between task complexity and the facial expression anger. The results of both analysis show that it is important to be aware of the meaning of the FRO data of facial expressions, to perform the analysis with the data. A remarkable observation of this research is the amount of participants that did not finish the procedure. 110 of the 204 participants who started the experiment quitted the procedure early, which is almost 54%. Most of these participants quitted when FRO recordings started. It is possible that participants did not own the required webcam for FRO recording, or that participants did not feel comfortable by being recorded. Other online research that involves webcam recordings, noted privacy concerns as a factor of dropouts of the experiment (Semmelmann, Hönekopp & Weigelt, 2017). Participants also quitted the procedure during the tasks. The design of the experiment could be a reason for these dropouts, because the study consisted of three different tasks. For every task the participants had to accept the use of the webcam again. In this research the reason of quitting the procedure is not specifically investigated, but is an interesting topic to outline in further research with FRO. Another notable result of this research is the 20% of the participants who claimed that they found the article on the website during the difficult search task (task two), despite the fact that the article did not exist on that site. It is possible that participants actually believed that they had found the article. Other possible explanations are that participants did not completely understand the task, or that they
did not pay attention while completing the task. The results of the difficult search task also show that some participants finished faster than the 60 seconds the task lasted. This result suggests that these participants were not committed to finish the task properly. For further research it is important to understand why participants show this conflicting result. The current study has some limitations that could have influenced the results. First, the recordings with low quality that were not excluded, between 3 and 6, could have caused some noise in the recording data. These recordings were not excluded from the data, because mostly it regards one recording with bad quality of the participant, whereas all the other recordings were of good quality (when participants had recordings with a quality 2 or below, all the recordings of that participant were excluded, also the recordings with good quality). Although the mean quality of al the included recordings was 9.27, it is good to be aware that recordings of lower quality were included. Moreover, no premeasures of the faces of participants were conducted to control for neutral facial expressions of the participants. Some people could have a more angry facial expression by nature, which may have affected the results. Furthermore, it is possible that the performance of a task during an experiment leads to different facial expressions than real time exploring of a website. Also, the subjective ratings of the websites could have been influenced by personal degree of interest in travelling. Further research is needed to overcome the limitations of this study. Also, in this study the reason behind the increase in the negative facial expression anger was unclear. Mental effort and frustration could be distinguished more clearly in further research. For example, other outcome measures like self-report and psychophysiological signals could be used. Lastly, this research only focussed on the difficulty of the task while searching on a website to measure the effect on the facial expression anger. In further research it is interesting to investigate whether the level of design complexity of a website also influences the facial expressions of participants while performing searching tasks. A good experience on a website could be key for further use and even have an effect on the sales of the site (King, Lazard & White, 2019; Nielsen, 2000; Venkatesh & Agarwal, 2006). Therefore it is important that the website is easy to use, without any blockages or delays. The results of this research suggest that difficulty of arithmetic tasks is experienced the same as a difficult task on a website, and could be measured with the facial expression anger. This suggests that it is possible to measure the usability of a website with the facial expression anger. Moreover, FaceReader Online proves to be a useful tool in usability research because it is able to measure discrete emotions of participants while performing tasks without the need of a lab, although it is important to be aware of the meaning of the data and the amount of dropouts in online research.
Appendix A.
Questionnaire website design complexity task - Was the text easy to read? (1) Very difficult to read (2) Difficult to read (3) A little difficult to read (4) Neither difficult or easy (5) A little easy to read (6) Easy to read (7) Very easy to read - What is your general impression of the page? (1) Dislike a great deal (2) Dislike a moderate amount (3) Dislike a little (4) Neither like nor dislike (5) Like a little (6) Like a moderate amount (7) Like a great deal - How complex did the site look? (1) Extremely simple (2) Simple (3) Slightly simple (4) Neither simple or complex (5) Slightly complex (6) Complex (7) Extremely complex - What did you think of this complexity? (1) Extremely unclear (2) Unclear (3) Slightly unclear (4) Neither clear or unclear (5) Slightly clear (6) Clear (7) Extremely clear - How likely is it that you would visit the website again? (1) Extremely unlikely (2) Unlikely (3) Slightly unlikely (4) Neither likely nor unlikely (5) Slightly likely (6)Likely (7) Extremely likelyAppendix B.
Task 2 – Website searching experiment – Experiment 1 FaceReader Online Task 1. Website: https://agreekadventure.com/. Design: complex. Easy task (1 click) (max 1 minute): Instruction: “Your friend told you about an amazing article that gave advice about cute hiking gear to wear in the snow. Find it on the website. (Select finished in the right corner, once you are sure you have found it)” Response: I found the article vs. I did not find the article (max. 20 sec). Task 2. Website: https://anywhereweroam.com/. Design: simple. Difficult task (impossible to find) (max 1 minute): Instruction (FaceReader Online): “Your friend told you about an amazing article about a beautiful walk in Belluno, Italy. Find it on the website. (Select finished in the right corner, once you are sure you have found it)”.Response: I found the article vs. I did not find the article (max. 20 sec). Task 2 – Website searching experiment – Experiment 2 Task 1. Website: https://anywhereweroam.com/. Design: simple. Easy task (1 clicks) (max 1 minute): Instruction (FaceReader Online): “Your friend told you about an amazing article about souks and fondouks in Marrakech, Marocco. Find it on the website. (Select finished in the right corner, once you are sure you have found it).” Response: I found the article vs. I did not find the article (max. 20 sec). Task 2. Website: https://agreekadventure.com/. Design: complex. Difficult task (impossible to find): Instruction (FaceReader Online): “Your friend told you about an amazing article about exploring the underwater world in the Ionian Sea, Italy. Find it on the website. (Select finished in the right corner, once you are sure you have found it)” Response: I found the article vs. I did not find the article (max. 20 sec).
Appendix C.
Table 6 P-values of factorial repeated measures ANOVA of Level:Emotion interaction Level:Emotion interaction P-value Website design complexity 0.54 Search task 0.047 Arithmetic task 0.14 Note. This table shows the p-values of the factorial repeated measures ANOVA of the Level:Emtion interaction of all three tasks. The p-values are Greenhouse-Geisser corrected.References Branco, P., Firth, P., Encarnação, L. M., & Bonato, P. (2005, April). Faces of emotion in human- computer interaction. In CHI'05 Extended Abstracts on Human factors in computing systems (pp. 1236-1239). Berlyne, D. E. (1974). Studies in the new experimental aesthetics: Steps toward an objective psychology of aesthetic appreciation. Hemisphere. Comber, T., & Maltby, J. R. (1997). Layout complexity: does it measure usability?. In Human-Computer Interaction INTERACT’97 (pp. 623-626). Springer, Boston, MA. Ekman, P. (1992). An argument for basic emotions. Cognition & emotion, 6(3-4), 169-200. Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion review, 3(4), 364-370. Ekman, P. (86). & Friesen, WV (1975). Unmasking the face: A guide to recognizing emotions from facial clues. Englewood Cliffs, NJ: Prentice Hall. Elfenbein, HA, 86, 203-235. Ekman, P., & Friesen, W. V. (1978). Facial action coding system: A technique for the measurement of facial movement, consulting psychologists press. Palo Alto. Ekman, P., Friesen, W. V., & Hager, J. (1978). Investigator’s guide to the facial action coding system. Ekman, P., Friesen, W. V., & Ancoli, S. (1980). Facial signs of emotional experience. Journal of personality and social psychology, 39(6), 1125. Epstein, S. (1994). Integration of the cognitive and the psychodynamic unconscious. American psychologist, 49(8), 709. Grafsgaard, J., Wiggins, J. B., Boyer, K. E., Wiebe, E. N., & Lester, J. (2013, July). Automatically recognizing facial expression: Predicting engagement and frustration. In Educational Data Mining 2013. Harper, S., Michailidou, E., & Stevens, R. (2009). Toward a definition of visual complexity as an implicit measure of cognitive load. ACM Transactions on Applied Perception (TAP), 6(2), 1-18. Harper, S., Michailidou, E., and Stevens, R. (2009), Toward a Definition of Visual Complexity as an Implicit Measure of Cognitive Load, ACM Trans. on Applied Perception, 6(2), Article 10. Hazlett, R. (2003). Measurement of user frustration: a biologic approach. In CHI'03 extended abstracts on Human factors in computing systems (pp. 734-735). Hazlett, R. L., & Benedek, J. (2007). Measuring emotional valence to understand the user's experience of software. International Journal of Human-Computer Studies, 65(4), 306-314. Hibbeln, M. T., Jenkins, J. L., Schneider, C., Valacich, J., & Weinmann, M. (2017). How is your user feeling? Inferring emotion through human-computer interaction devices. Mis Quarterly, 41(1), 1-21. King, A. J., Lazard, A. J., & White, S. R. (2019). The influence of visual complexity on initial user impressions: testing the persuasive model of web design. Behaviour & Information Technology, 1-14. Lazar, J., Jones, A., Bessiere, K., Ceaparu, I., & Shneiderman, B. (2003). User frustration with technology in the workplace. Lewinski, P., den Uyl, T. M., & Butler, C. (2014). Automated facial coding: Validation of basic Lewinski, P., Fransen, M. L., & Tan, E. S. (2014). Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli. Journal of Neuroscience, Psychology, and Economics, 7(1), 1. emotions and FACS AUs in FaceReader. Journal of Neuroscience, Psychology, and Economics, 7(4), 227. Lindgaard, G., Fernandes, G., Dudek, C., and Brown, J. (2006), Attention web designers: You have 50 milliseconds to make a good first impression!, Behaviour & Information Technology, 25(2): 115-126. Nadkarni, S., & Gupta, R. (2007). A task-based model of perceived website complexity. Mis Quarterly, 501-524. Nielsen, J. (2000). Designing Web Usability: The Practice of Simplicity. Indianapolis: New Riders Publishing. Pandir, M., & Knight, J. (2006). Homepage aesthetics: The search for preference factors and the challenges of subjectivity. Interacting with Computers, 18(6), 1351-1370. Poels, K., & Dewitte, S. (2006). How to capture the heart? Reviewing 20 years of emotion measurement in advertising. Journal of Advertising Research, 46(1), 18-37. Semmelmann, K., Hönekopp, A., & Weigelt, S. (2017). Looking Tasks Online: Utilizing Webcams to Collect Video Data from Home. Frontiers in psychology, 8, 1582.
Stone, R. T., & Wei, C. S. (2011, September). Exploring the linkage between facial expression and mental workload for arithmetic tasks. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 55, No. 1, pp. 616-619). Sage CA: Los Angeles, CA: SAGE Publications. Tuch, A. N., Bargas-Avila, J. A., Opwis, K., & Wilhelm, F. H. (2009). Visual complexity of websites: Effects on users’ experience, physiology, performance, and memory. International journal of human-computer studies, 67(9), 703-715. Venkatesh, V., & Agarwal, R. (2006). Turning visitors into customers: A usability-centric perspective on purchase behavior in electronic channels. Management Science, 52(3), 367-382. Waterink, W., & Van Boxtel, A. (1994). Facial and jaw-elevator EMG activity in relation to changes in performance level during a sustained information processing task. Biological psychology, 37(3), 183-198.
