Enhancing Believability in Responses of Virtual Agents
Through a Computational Model of Leary’s Rose.
SUBMITTED IN PARTIAL FULFULLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE
ALISSA MUFFELS 10163506
MASTER INFORMATION STUDIES HUMAN-CENTERED MULTIMEDIA
FACULTY OF SCIENCE UNIVERSITY OF AMSTERDAM
OCTOBER 3rd, 2017
1st Supervisor 2nd Supervisor
Dr. Tibor Bosse Dhr. Daniel Buzzo
Enhancing Believability in Responses of Virtual Agents
Through a Computational Model of Leary’s Rose.
Alissa Muffels University of Amsterdam Graduate School of Informatics
Science Park 904, Amsterdam Alissa.muffels@student.uva.nl
ABSTRACT
This paper presents the construction and evaluation of a computational model for social skills training. The model is created to serve as part of an embodied conversational agent. The goal is for learners to practice social skills on virtual citizens complementary to role-playing practice with human instructors. The aim is to enhance believability of natural language -and nonverbal behavior of responses of a virtual agent. For this, the interpersonal relations theory of Leary’s Rose is used to shape user-agent interaction. Two models were evaluated. One model, using the theory of Leary’s Rose to shape interaction was compared to a second model generating random responses. The model using the theory of Leary’s Rose performed better on believability and performance. Responses generated by the computational Leary model were considered more logical and like real-life situations than random responses. Many features are combined in building an embodied conversational agent and the results indicate that addition of the cognitive theory Leary’s Rose can enhance believability of responses of a virtual agent. However, to contribute to social skills training, further development is needed. Additional features such as animation, natural language understanding -and generation should be included.
Categories and Subject descriptors
H.1 [natural language]: naturally developed language by use H.2.2 [interpersonal stance]: shape of a communicative interaction.
General Terms
Social Skills, Computational model, Virtual Agent
Keywords
Embodied Conversational Agents, Affective Computing, Leary’s Rose, Serious Game, Virtual Character, user-agent interaction.
1. INTRODUCTION
Several incidents are known of aggressive behavior towards police officers. For example, on February 26th, 2017 two suspects showed significant resistance while they were arrested. Afterwards, one of the police officers got bit in the face1. On May
16th the same year, police officers shot a suspect in his leg to safely arrest him. This was done after he showed aggressive behavior towards these officers by throwing with cans2. Not only
1
Source:https://www.politie.nl/nieuws/2017/februari/26/02- hengelo-agenten-worden-tijdens-aanhouding-mishandeld-en-een-gebeten-in-gelaat.html
police officers suffer from this kind of violence. During New Year’s Eve, several violent incidents were reported towards police officers, ambulance personnel and firemen3.
Incidents as such can be caused by social skills deficits because it influences how people succeed in social situations [32]. Tanaka et al. (2016) describe social skills as being crucial to everyday life and social development. According to Bosse & Gerritsen (2017) one of the solutions to prevent aggressive behavior is to invest in communication skills practice [5].
Practice of social skills is done through social skills training (SST), which is a kind of behavioral therapy to train people in social interactions [32]. Currently, training of these social skills such as aggression de-escalation and professional conversations are trained with human instructors and role-playing games [5]. However, this kind of training is costly,
time-consuming and difficult to repeat [5][6]. Another way of teaching communication skills is through serious gaming. When computer-based games are used to educate they are called serious games [29]. Computer based, virtual games for social skills training are less costly, applicable everywhere, many scenarios are possible, they are easy to manipulate and there are no physical risks involved.
One way of teaching social skills through serious games is simulation-based training. Learners can practice social skills on virtual citizens [5]. These virtual characters, or embodied conversational agents (ECA’s) are animated characters, of which the responses are computer-generated. They have proven to enable users in training social abilities. For instance, ECA’s can be used to answer questions in a certain way so that people can practice with different social approaches and gain more insight in their conversational strategies. The responses of such virtual agents can include both verbal and nonverbal behavior [7]. This kind of training is not only useful for aggression de-escalation. It can be applied to a range of behavioral and cognitive approaches to teach social skills. For instance, communication skills, problem solving, self-management and decision making [10].
However, when developing virtual characters for social skills practice, their actions need to be realistic [7]. The skills taught by the agent must be applied in real-life settings. Therefore, the interaction with users should be natural, which is achieved by incorporating several elements in the virtual agent. One is multimodal understanding –and perception of the user, such as emotion recognition. Another part is designing the agent in a way it can produce natural language and nonverbal behavior [7]. Building ECA’s is a complex process in which these features are 2 Source:https://www.politie.nl/nieuws/2017/mei/16/07- agressieve-en-verwarde-man-na-schot-in-been-aangehouden.html 3 Source:https://www.rijksoverheid.nl/actueel/nieuws/2017/01/17/ geweld-tegen-hulpverleners-tijdens-jaarwisseling-onacceptabel
combined. The focus of current work is in generating realistic -and natural response behavior. The distinction between the different features of an ECA is described in figure 1.
Figure 1. Framework for user-agent interaction in Embodied Conversational Agents4
In the framework, there is a separation between understanding the user and generating responses. In multimodal – and natural language understanding the aim is for the agent to comprehend the meaning of the user, whereas in natural language -and nonverbal behavior generation the agent should produce a fitting response.
Currently, human-agent dialog is often interpreted as stereotypical due to the fixed responses. Input behavior is followed by a pre-defined response. As a result, communication is inflexible and often perceived as predictable [7]. To prevent this, options for dialog need to be extensive and wide-ranging. However, substantial alternatives for dialog are difficult to create and reuse. A possible solution is incorporating generic cognitive models in user-agent communication with ECA’s [7]. In the current paper, Leary’s Rose is used as a cognitive model. The theory of Leary’s Rose, described in the next section is a model for inter-personal communication often used in social-skills training [31]. The research question is:
‘’Given human behavioral input, is it possible to let a computational agent enhance believability by generating automatic responses that fit the behavior theory of Leary’s Rose?’’
Answering this question is done in two phases. First is Construction of a computational model based on Leary’s Rose theory. Second is evaluation of the model to examine its contribution to believability of responses. In the
framework of user-agent interaction, thecognitive model is situated as part of the agent handling both input and output (figure 5). Input in this case is defined as measured behavior of the user, whereas output is the generated response of the agent. The purpose of this paper is to create a computational model that could be used for social skills training and to evaluate whether this enhances the believability of an ECA that uses the model.
4Source:https://confluence.ict.usc.edu/display/VHTK/Architecture
2. RELATED WORK
2.1 Embodied Conversational Agents
Computer environments are adapting embodied conversational agents in conventional web systems, but also in advanced virtual worlds where they interact with humans [23]. ECA’s are highly beneficial, Lester, Converse, Stone, Kahler and Barlow (1997, cited in [23]) found that they can enhance problem solving skills of middle school children. Additionally, in another study was found that most users readily accepted ECA’s as conversational partners [19]. The latest technologies using conversational interfaces engage students to interact with them as if they were talking to humans [24]. The naturalness of this communication can be maximized with the use of both linguistic features, including syntax and semantics as well as paralinguistic features, including pragmatic and social ones [24]. In a learning theory posed by Vygotsky, the importance of social aspects is highlighted. According to Vygotsky, knowledge is socially constructed and involves social skills such as negotiating, sharing and communicating. Within the field of learning technology, these social skills are emphasized together with the significance of technically supported learning. Therefore, various kinds of virtual characters can be used to express facial expressions, movements, voice and language [17]. These studies show that the use of Embodied conversational agents can contribute to the learning process and be perceived as human conversation partners. Therefore, the use of ECA’s for social skills training is examined.The application of ECA’s for social skills training is widely researched and with encouraging outcomes. For example, Bruijnes, Linssen, op den Akker, Theune, Wapperom, Broekema and Heylen (2015) studied the possibility of training social skills with simulation based training in police officers [19]. Moreover, this is done for employee-supervisor communication [35]. In another study was found that learners who practice with virtual humans seriously, found it just as engaging as practicing with real humans [18]. In the same study, after training with a virtual human trainees gained the ability to use practiced interpersonal skills better than before. Furthermore, Craig et al. (2016) looked at improving social emotional functioning of children by a virtual gameplay of social skills practice. Children that received treatment by a game named Zoo U, an interactive game that teaches social emotional learning strategies, showed increased social skills and emotion regulation compared to the control group, who did not receive any treatment [10]. A virtual approach to social skills practice thus, has potential.
Many training systems using virtual agents have already been developed. The mission rehearsal exercise (MRE) is a virtual training system that is used by the military to allow soldiers to practice interactions where high levels of stress apply [27]. Communicate! is used for practicing communication between a health care professional and a patient [20]. Also, Lala, Jeuring, van Dortmont and van Geest (2017) looked at scenarios in learning environments to use in one-to-one communication skills training. They stress the importance of communication protocols where, for example, a doctor can tell a patient of a terrible sickness. However, passing on arguments to communication protocols remains a challenge [21]. One way to create such a communication protocol is using affective computing. The aim of affective computing is to build a computational model that can deal with psychological theories [8]. Where humans can interpret the under-specification in these theories, computers cannot [8]. Thus, psychological theories should be operationalized in a way the computer can interpret them. In the current work, the
psychological theory of Leary’s Rose is used to create such an affective computer model.
2.1 Leary’s Rose
A major consideration in development of virtual agents is modeling of interpersonal stance. With interpersonal stance, typical patterns in people’s reactions while interacting can be modelled [8]. One popular theory which is often used in
communication-modelling and describes these personal stances is Leary’s Rose [8]. The theory provides a better insight in
interpersonal communication by distinguishing behavioral components [22]. The behavioral patterns described in Leary’s Rose are very logical and therefore suit the objective of communication practice very well. According to Leary (1957), a person’s communicative stance towards a conversational partner can be represented as a point in a 2-dimensional space. The general assumption of the model is that behavior that humans exhibit during interaction can be characterized as either dominant (above) or submissive (below) and together or opposed.
o Above: based on displaying influential behavior o Below: based on not or barely displaying influential
behavior
o Together: based on acceptance and cooperative behavior o Opposed: based on disputed behavior and against
acceptance
Within the 2-dimensional space, the horizontal axis describes the relationship with the other person (i.e., together or opposed) whereas the vertical axis describes an attitude (i.e., above or below). This distinction yields four quadrants, which are further defined with the addition of 8 labels, resulting in 8 octants to represent people’s communicative behavior [22], the rose is shown in figure 2.
Behaviors in the rose are labeled by their location in the Rose. For instance, aggressive behavior is on the left side of the rose and above the horizontal axes, therefore it is labeled as OA (i.e., opposed-above). Furthermore, competing behavior is positioned above aggression and is touching the vertical axes. This behavior is more dominant and labeled accordingly as AO. These specifications are made for each quadrant to better understand the subtle behavioral differences between points in the rose. Leary also explains the dynamics between interactions in the Rose [36]:
o Above behavior produces Below behavior. People tend to respond in a submissive matter when facing a dominant character.
o Below behavior produces Above behavior. Leadership behavior is triggered when interacting with people displaying below behavior. The automatic response when handling submissive people is to control them. o Opposed behavior produces opposed behavior, which
means people intend to react more critical to someone that is also behaving in a demanding and critical manner.
o Together behavior produces together behavior. People naturally work together in harmony.
In figure 3 a simplified outline of the interactive behavioral tendencies is shown. Leary’s Rose is widely used for social skills practices because it can be used to understand people’s behavior and change its own behavior. For example, two people that both have a submissive and positive character support each other’s behavior. Two dominant, aggressive individuals continuously compete and lead the conversation [22]. Examples of these behavioral tendencies are outlined in figure 4.
Figure 2. Leary’s Rose including the eight distinct behaviors in a 2-dimensional space5.
Figure 3. Behavioral tendencies in interpersonal communication according to Leary’s Rose theorem.
A virtual agent needs to be able to apply Leary’s Rose theory in communication with a learner. This is where some difficulties arise. The system should be able to recognize the stance of its communication partner. Furthermore, it needs to reason about the input and respond in a suitable manner [8]. In the current paper, the focus is modelling the operationalization of input – output behavior described in Leary’s Rose.
Figure 4. Examples of behavioral tendencies according to the eight octaves of Leary’s Rose.5 Behavior Example
Leading ‘’I will propose some options and we will choose from them’’
Helping ‘’I think you are right’’
Cooperative ‘’Tell me what I can do for you’’
Depend ‘’If you think this is the way, we will go with that’’ Withdrawn ‘’Yeah, ok, whatever, my opinion does not matter that
much’’
Defiant ‘’Yes, maybe, but if you are going to act like that, then’’
Aggression ‘’I absolutely do not agree with that’’
Compete ‘’I know what’s best, this is the way it should be done.’’
5
Note that there are many other examples possible, these examples are based on Van Dijk & Cremers (2013).
3. MODEL DESIGN
Based on the literature review the research is divided in two phases, first the construction of a computational model that generates automatic responses that fit the theory of Leary’s Rose. Second is evaluation of its contribution to believability of responses. The model takes human behavior as its input and generates output based on the theoretical notions as discussed in the theory of Leary’s Rose. In the framework of human-user interaction, the system is situated within the agent as one communication strategy. A revised model is shown in figure 5.
Figure 5. Placement of the Leary model within the human-agent interaction framework.
3.1 Requirements
Based on the literature research, requirements were formulated for the model. Requirements were obtained from Leary’s theory and resemble the most important aspects of the dynamics described in interpersonal communication. Initially, Leary’s Rose is illustrated as an ordering of the stances on a circle, situated on two axes [8]. This regular ordering of stances is known as the interpersonal circumplex [28]. The relation of stances within the interpersonal circumplex is used to define the in –and output of the model. As noted, the stances that are close together are more related than stances that are further apart [8]. Leary (1957) proposed that two people communicating influence each other and calls that ‘interpersonal reflex’. So, one of the requirements of the model is that these reflexes should be simulated.
Second, to make the model able to process interpersonal stances within the circumplex, a computational definition should be determined. As the Rose of Leary consists of two axes, these will be used to define the input and response. In the current model, numerical values are assigned to every position in the rose. This value is used as a representation of the
interpersonal stance position of a user, but also to calculate the interpersonal reflex that is the result of this behavior. The second requirement therefore is that behavioral tendencies of the Rose of Leary need to be represented with numerical values.
3.2 Dynamics
The requirements are resembled by the behavior of the model. For the representation of numerical values, Leary’s Rose is divided in two axes representing affect and dominance. Figure 6 is a schematic representation of how the axes of Leary’s Rose are represented including the representation of numerical values. Here, the values are in a range from 100 to 500.
Figure 6. Leary’s Rose interpersonal communication represented with numerical axes.
The user can give input that determines its stance. The stance of the user and the agent are both displayed. The stance is described as a point with two coordinates. The location of the point falls within one of the octants (figure 2) and can be interpreted as a behavioral tendency. The stance of the agent is randomly initiated at the start of the interaction. The system gets the stance of the user, ‘knows’ its own stance and updates it according to the input of the user. The update of stance is displayed in a new point and represented by two new coordinates. The influence of the user’s stance on the response by the model is determined by Leary’s theory. The input coordinates represent above – below behavior or together – opposed behavior. The point in figure 6, is input in the model. This input is given by a mouse click on that location of the screen. The input value determines the output value of the model. In the current example, the model should return below and opposed behavior as can be seen in figure 7.
Figure 7. Schematic representation of output behavior.
To make sure a user-agent communication strategy is used instead of trial-and-error, a target position is added. This is done to prevent users from interacting with the agent without grasping the interpersonal reflexes of the model. Like the starting position, the target coordinates are randomly initiated. The goal of interaction with the model is to ‘behave’ (i.e., click on octaves in the rose) to move the agent to the target position.
4. MODEL CREATION
4.1 Prototype
In the creation of the prototype, the reflexes of the interpersonal circumplex should be resembled. Leary’s Rose should be represented with numerical values on the axes and the
model should have the ability to translate input into output according to the theory. The computational implementation of these requirements in the model is done using Python software. This language is used for its general-purpose and popularity. The Pygame extension is used to create the interactive model. The implementation of the interpersonal circumplex is a set of functions with parameter d, which is a measure of ‘stubbornness’ of the agent. Input coordinates of the user (x,y) are multiplied by a value of d ε [0,1]. The higher the value of d, the more stubborn the response of the agent. This is because d determines the amount of movement of the agent. The lower the value of d, the more the agent position moves towards the user-input. Different formulas for x and y are created because behavioral tendencies vary for affect (x) and dominance (y). Each formula is applied on the input coordinates for x or y to determine the output, the output is the new position or response of the agent. The x-axis represents affect (opposed – together) and automatic responses are defined by the following formula:
Cnew = d * Cold + (1-d) * Hnew
In which C is the agent position in the rose (x,y) and H is the user position in the rose (x,y). According to this formula, when the input is together-behavior, the output is together-behavior. Likewise, opposed behavior causes opposed behavior. An example movement over the x-axis is shown in figure 8. When Huser inputs together behavior, Cagent moves towards the position
of Huser
Figure 8. Dynamic in the system with starting point C, user-input H and the direction of movement.
For the y-axis, the response of the model is opposite of the input behavior. On the dominance axes, when dominant behavior is the input, submissive behavior is the output and vice versa. The following formula is used:
Cnew = Cold ± d * Hnew
When the input is on the dominant part of the rose, the formula Cnew = Cold + d * Hnew is used to calculate the output.
When the input is on the submissive part, Cnew = Cold - d * Hnew
Calculates the output. When Cagent is in an above position and
Huser displays below behavior, Cagent moves away from Huser. An
example is shown in figure 9.
Figure 9. Dynamic in the system with starting point C, user-input H and the direction of movement.
4.2 Pilot Study
To test if users could project behavioral tendencies on coordinates on points of the model, a pilot study was done. For testing the model, the value of d = 0.2 is used. Five people participated in the study. The average age was 22 (M=22.00; SD=4.85). First some background was given on the Theory of Leary’s Rose, then participants were asked to complete 10 trials. Each trial the goal was moving the starting point (in red) to the target position (in green) in the rose by giving input. This input was given by mouse-clicks within the rose. Afterwards participants were asked to complete a questionnaire (see APPENDIX A) about several aspects of the model. During the study, participants could ask questions and give comments to gather as much information as possible. Questions included: ‘’Can you imagine the dot on the screen represents behavior in the rose?’’ and ‘’ What kind of behavior would you expect when you exhibit above-opposed behavior?’’.
4.3 Current Implementation
Based on results of the pilot test, changes were implemented. Four of five participants were not able to draw correct examples for Above or Below behavior. For example, above behavior was characterized as ‘oppressive’. According to Leary’s theory, ‘oppressive’ belongs to ‘Opposed’ behavior. Additional inquiry revealed the terms ‘’above’’ and ‘’below’’ were considered unclear. Therefore Above – Below was replaced by Dominant – Submissive. Several participants expressed it was difficult to imagine behavioral tendencies in the model within a quadrant. With only four behavioral aspects represented by the model, it felt like too many behaviors cannot be expressed. So, the labels from the eight octants described in Leary’s theory were added to cover a broader range. Finally, the question: ‘’Did you feel like the way the green dot moved in the screen represented responses in real life?’’ was answered mostly with ‘yes’. This was an encouraging outcome for the perceived believability of behavioral responses in the model. The results of these changes and the final outline of the model can be seen in figure 10. The green dot represents the target position. The red dot represents the status of the agent. The user input is shown with a white dot that appears after clicking on a position in the Rose.
Figure 10. Final model after changes are implemented.
5. METHODOLOGY
5.1
Study Design
The model is evaluated by a 2x1 counterbalanced within-subjects design study (Experimental vs. Control). Participants are randomly assigned to condition A or B. The design is chosen to minimize subject-to-subject variation. To avoid practice, -fatigue -and motivation effects the design is counterbalanced. The order in which the models are presented is different for both conditions. Another reason for choosing this design is the increase of power at a lower number of participants. In table 1, the overview of the research design is shown.
Table 1. Research design.
Within Subjects Design
Participants Group: Control Model Leary Model No. of Participants: Group A 1 2 10 Group B 2 1 10
A convenience sample is used to test and evaluate the model’s responses in terms of believability and task performance. Participants are not aware of the condition they are in. The observer is aware of the condition. Believability is operationalized by self-assessment. Participants self-report their actions, attitudes and experience on realism of responses given by the model. Task performance is measured by speed and accuracy. Usability of the system is not measured, because the system itself is not
implemented in communication training. It is rather used as an underlying system to represent the cognitive model of Leary’s Rose to further shape the virtual agents used for training practices.
5.1 Materials
5.1.1 Participants
There were 20 participants included in the research. Of the participants were 8 men and 12 women. The average age of participants was 31 years old (M=30.65, SD=13,33). All 20 participants were analyzed for the control and the experimental system. The participants were mainly coming from the environment of the researcher. There was no reward after the
study was finished, but participants could voluntarily give their e-mail address if they wanted to be informed about the results.
5.1.2 Random Model
A random model is created as comparison for studying the Leary model. It serves as a control for measuring the influence of working with a model that claims to resemble human
interaction. This model looks identical to the Leary model (figure 10), but is not calculating agent responses based on Leary’s theory. It calculates a random number to move the agent position.
5.1.3 Believability Questionnaire
A believability questionnaire of 5 questions (see APPENDIX C) is used to measure the opinion of participants towards responses of the model. Believability of an agent is when its actions exclude disbelief and the character seems life-like [3]. Therefore, the questionnaire includes items about actions, which are the responses of the agent and its perceived character based on these responses. The most important aspect is consistency of expression. It means the agent should send out a unified message [3]. This is operationalized by including questions about the expected behavior of the agent: ‘’Do you consider this a realistic response?’’ and ‘’I felt like I was able to influence the responses the model gave me’’. Finally, interaction with the model was examined by the question: ‘’I think the way the model responded is the same as in real-life communication.’’. Answers were given on a Likert skill varying from 1, totally disagree to 10, totally agree. Answers provide insight in the degree of believability. The believability score on the questionnaire is the sum of all questions in a range from 5 to 50. However, because the virtual agent does not show any emotions or facial expressions, a full comprehension of believability is not obtained.
5.1.4 Task Performance
To further investigate understanding -and processing of information of the Leary (experimental) model, task performance is used. Task performance is measured by speed and accuracy. This is done to explore the added value of the theory of Leary’s Rose on communication with the model. Speed, or reaction time reveals ways in which people deal with the organization of a temporal process [30]. By comparing reaction time for the model with theory to reaction time for the model without theory, the influence of use of Leary’s Rose on temporal information processing is measured. The current interest is the process of deciding where to click to move the agent to the target position. The reaction time is the duration of that mental process [30]. To reduce unwanted variability in this dependent variable, practice, motivation -and fatigue effects are eliminated by counterbalancing between condition A and B. The overall time of the completion of one set is used to investigate participant’s response to the model according to the theory of Leary’s Rose.
To get a full understanding of difference in task performance between the two models, accuracy is measured as well. Accuracy is operationalized by the number of wins in each set. A win is when the goal of a trial is achieved within the number of mouse-clicks allowed for this trial. So, when the agent reaches the target position.
Finally, learning effects are measured. In the current study, learning is defined as a decrease in time required to finish one trial [25]. A learning effect of the model is established when the decrease in time between trials 1 and 5 is significantly higher for the Leary model than the Random model.
5.2 Procedure
First, the participants were randomly assigned to condition A or B. The research setting was with one participant and one observer. In figure 11, the setting of the research is shown. An instruction form (see APPENDIX B) was handed out, containing instructions on the theory, examples and the procedure of the experiment. When the participant finished reading the instruction form, the first set was started. In both conditions the same instructions were given. The first set contains 5 trials. A trial is finished after 20 mouse-clicks or when the target position is reached by the agent. When a trial is finished, a new trial starts automatically. The participant is notified, because the message ‘get ready for your next trial’ appears on the screen. Within 5 seconds, the new trial begins. After the first set, the screen displays ’end of set’ and the program automatically shuts down. The observer gives the first believability questionnaire for the participant to complete.
After completing the first questionnaire, the second set is started in the same manner. Participants in group A first complete five trials with the random model after which trials with the Leary model are presented. Group B performs the procedure vice versa (see table 1). Again 5 trials are completed before the second set is complete. Then, the second believability
questionnaire is completed. Because participants are deliberately not being informed about the underlying algorithms of the models, a debriefing follows. The participant is given full disclosure about the difference between the two models.
Figure 11. The experimental environment in which the research was conducted.
5.3 Data Analysis
For testing the model, the value of d = 0.2 is used. Data was analyzed using SPSS. First, new variables were created for believability score, mean time to complete one trial, total time, number of wins and time enhancement. For both the Random model (N=20) and the Leary model (N=20), separate variables were created so the models could be compared. Believability-score is defined the sum of Believability-scores on each question on the questionnaire. For accuracy, the wins per set are used for the calculation. Time measurements were: total time to complete one set, time between trial 1 and trial 5 and mean time per trial.
To set a clear goal for the data analysis and the expected results, some predictions about the outcomes were defined. First, expected is when the responses of the experimental model are considered logical or rule-based by the user, then information is processed faster. Second, when participants can learn to anticipate on responses of this model, a learning effect will take place. This means that the time-difference between the first and fifth trial for
the Leary model will be bigger than for the Random model. Furthermore, expected is a higher number of wins with the Leary model. The last prediction is that the agent’s believability is rated higher with the experimental model. These predictions for the data analysis resulted in four hypotheses:
1. ‘’After the set with the Leary Model, participants will give a higher believability score on the questionnaire compared to the random model set.’’
2. ‘’The mean total time to complete one set with the Leary Model is less than with the Random model’’ 3. ‘’The time-difference between the 1st and 5th trial for the
Leary model, is bigger than this difference for the Random model.’’
4. ‘’A higher number of wins is achieved for The Leary model than the Random model.’’
6. RESULTS
Participants were randomly assigned to either group A or group B. Each participant completed the questionnaire twice. For the questionnaire, the Cronbach alpha was calculated. The believability questionnaire was reliable with an alpha of α = 0.787. The alpha did also not increase when removing one of the questions. Therefore, none of the questions is removed.
The next step was seeing if the dependent variables, reaction time and believability score were different for the experimental and control model. To perform a paired samples t-test, the data should meet the requirements for outliers and normality of the data distribution. First, the assumption of outliers was tested and for believability score and no outliers were detected. However, for reaction time outliers were present. The boxplots for believability and reaction are shown in figure 12 and 13.
Figure 12. Boxplot to check for outliers in the data distribution of believability score.
Figure 13. Boxplot to check for outliers in the data distribution of reaction time.
Then, the assumption of normality of the distribution of the dependent variables was tested. Because of the small sample in this research the Shapiro-Wilk test is used. For reaction time the distribution was different from the normal distribution in the random model D(20)=0.875, p<0.05. For the Leary model, the sample did not differ from the normal distribution, D(20)=0.930, p>0.05. For believability score the sample was normally
distributed D(20)=0.934, p>0.05 and D(20)=0.941, p>0.05. Thus, for believability score, the assumptions for a paired samples t-test were met. However, for the time-distribution a non-parametric test is used to compare the two models.
The mean of reaction time (to complete one set) with the random model was compared to mean reaction time with the Leary model. For the Random model this was 446 seconds (M=445.65, SD=175.765). For the Leary Model it was 288 seconds (M=287.80, SD=123.47). Also mean scores on the Believability questionnaire were calculated. The mean
believability score on the questionnaire completed after trials with the Leary model was 28 (M=27.85, SD=7.036) and for the Random model this was 16 (M=16.35, SD=6.714).
To see whether these differences in believability score for the Leary vs. the Random model are significant, a paired samples t-test was done for difference in mean believability score between the two models. Participants rated believability of the Leary model, higher than believability of the random model t (1, 19); p<0.05. Analysis of variance in time to complete all trials was done by a Wilcoxon’s test. Participants are faster in completing trials with the Leary model in comparison to the Random model (Z=-3.902, p<0.05).
Next, mean time difference between trials was calculated by subtracting the time to complete trial 1 from the time to complete trial 5. In figure 14 the time difference between the trials is shown in seconds. For both the Leary model and the random model, a reduction in time to complete one trial is observed (see figure 14). To see if the difference between models is significant, the Wilcoxon test for related samples is used. Trials completed with the Leary model did not have more time reduction between the 1st and 5th trial than the Random model (Z=-1.605,
p>0.05).
The number of wins per trial and per set is measured and the results are shown in figure 15. The number of wins is higher for the Leary model than for the Random model.
Figure 14. Decrease in time to complete trials for the Leary model compared to the Random model.
Figure 15. Number of wins per trial for the Leary model (red), compared to the Random model (blue)
Then, some additional analyses were done to further explore the outcomes of the research. First, accuracy was examined. In the Leary model, participants could finish the goal before the end of the trial 20 times. With the Random model, only three of the participants could reach the goal. To see whether the higher believability score for the Leary model can be explained by the number of wins, an additional test was performed. When the number of wins correlates high to the score on believability, some of the variance in believability can be explained by the number of wins, instead of the actual believability of the model. The Pearson correlation between the number of wins and the score on
believability was not significant. So, no relation was found between the number of wins and the score on the believability questionnaire (r=0.264, p>0.05).
Then the questionnaire of believability was examined further. Scores of every question were compared to see which aspect of believability was rated highest. The mean scores on the believability questionnaire for both models are shown in table 2. The question ‘’I think the way the model responded was the same as in real-life communication’’ is considered the main indicator for believability of the model. Additional analysis shows that in both conditions the Leary model got a significantly higher rating on this question than the Random model.
Table 2. Mean believability scores on the separate questions for the Leary and the Random model.
Q: Question: Leary Random
Q1 ‘’I felt like I could influence the responses the model gave me.’’
5.85 2.95
Q2 ‘’I felt like the responses of the model were logical.’’
4.70 2.45
Q3 ‘’I was able to apply a strategy when trying to move the red dot towards the green dot.’’
6.35 3.95
Q4 ‘’I felt like I was improving each trial.’’
5.60 3.40
Q5 ‘’I think the way the model responded was the same as in real-life communication.’’
5.50 3.60
7. DISCUSSION
The results of the study indicate that three of four hypotheses can be accepted. The first is that the Leary model got a higher believability score than the Random model. Second, the set
of trials with the Leary model was completed faster than the set of trials with the Random model. The third hypothesis is not confirmed, that the decrease in time to finish one trial is more for the Leary -than for the Random model. Finally, the fourth hypothesis is correct, in the Leary model, the number of wins was higher.
7.1 Believability
Evidence from the comparison in believability between the Leary -and the Random model suggests a tendency of users towards believability Responses of the Leary model. Users rated the believability of the Leary model higher than believability of the Random model. The number of wins was not associated with the higher score in believability of the Leary model. Therefore, it seems that believability score can be attributed to the type of model. The Leary model is considered more natural or real than the Random model. The additional analysis of scores on the believability questionnaire confirmed that users consider
interaction with the model that gave responses based on the theory of Leary’s Rose more like real-life communication.
However, this could be explained by the model having a structured way of responding itself, apart from applying the theory of Leary’s Rose. A possible interpretation is that the presence of a predictable outcome caused the effect. Instead of attributing the higher believability-rating to the theory of Leary’s Rose, it could have been caused by a combination of two psychological effects. First, the Leary model is based on cue-outcome learning, in which a stimulus (a cue) is used to predict the outcome of an event [15]. Whereas the Random model is based on a lack of predictability. When a stimulus is failing to predict the outcome repeatedly people learn that the cue or outcome is unpredictable [16]. The cue-outcome learning process in the Leary model could have caused a higher feeling of self-efficacy, which is the confidence or expectation of achievement of a learning task [13]. Secondly, according to the attribution theory, a person with high self-efficacy attributes success to internal factors. Opposite of a person with low self-efficacy, who attributes success to external factors [34]. Because questions on the believability questionnaire are asked from first-person perspective, the attribution theory could explain the higher score on the believability questionnaire for the Leary Model. One way to deal with this issue is to compare three models instead of two. In future research, a third model that generates automatic responses opposite of Leary’s Rose theory could be tested as well. This model can reveal whether the results can be assigned to agent responding according to the theory of Leary’s Rose or to the presence of a cue-outcome learning mechanism.
7.2 Task Performance
The results in reaction time show a faster completion of the set with the Leary model than with the Random model. Also, no correlation between accuracy and time was found, so a loss in accuracy does not account for the time difference between the two models. Also, the number of wins was higher for the Leary model than for the Random model. Thus, overall task performance on the Leary model exceeds performance on the random model.
Accordingly using the theory of Leary’s Rose in this model has a better performance and accuracy than lack of a theory. Responses based on the theory of Leary’s Rose therefore seem to be processed faster and result in more accurate in decisions. However, besides a faster completion of the set, expected was more time reduction between trials for the Leary model. This was not confirmed by the results. This means no learning effect was found.
One possible explanation for the absence of this effect is the number of trials and the time of measurement. It was found that reaction time to a stimulus decreases with 3 weeks of practicing [1]. One measurement of 5 trials for each model was probably not enough to find the expected effect. To see if the theory of Leary’s Rose can enhance the performance of the model -and communication with the model, a longitudinal research design should be used. Repeated practice helps with the acquisition of the motor -and cognitive skills [25] and can be discovered with multiple measurements. Although repeated measurement was outside of the scope for this research, further development of the use of the theory of Leary’s Rose in a computational model should investigate of long-term effects. Pre, -and post-training scores could be used to assess the potential of the system in training social skills for users. In a follow-up study, changes in reaction time over a period of several weeks and actual learning outcomes can be measured. The last could be done by using the method of Tanaka et al. (2017). To let an experienced social skills-trainer evaluate and rate the change in social skills of the users [33].
7.3 Study Limitations
One of the limitations of this study was the use of nonparametric tests for the investigation of time-effects. The Wilcoxon matched pairs signed-rank test was used instead of the paired samples t-test to investigate the difference in mean time to complete one set. The reason this test was used, is for the less demands on data distribution. Because the data was not drawn from a normally distributed population, the truth intervals of a paired samples t-test would not have been trustworthy. Because normality is not required in a Wilcoxon signed ranks test this was used. However, one disadvantage is that this test has less power. This is because of the use of ranks instead of actual values. To increase power and draw a conclusion with more confidence, a larger sample size is needed [11].
8. CONCLUSION
In this research, a computational model using the theory of Leary’s Rose is constructed and evaluated. The aim was to see if the model can be used to enhance believability of responses of an agent in social skills practice. The first part focused on creating a computational model that generates automatic responses based on Leary’s Rose. The second part is the evaluation of the model by users to see whether responses can enhance believability. It was found that a computational model can be created based on the theory. Additionally, evidence for enhancement of believability by using Leary’s Rose theory was found. Responses in communication with the model containing Leary’s Rose theory are considered more logical and like real-life communication than a model without a cognitive theory. Furthermore, the feeling that responses could be influenced was higher, meaning a feeling of interaction is achieved.
Additionally, the findings suggest that incorporating the cognitive model of Leary’s Rose in an Embodied Conversational Agent has potential. Further evaluation should be done in discovering learning effects, behavioral changes and cost-efficiency. For this the evaluation model of Kirkpatrick (1994, cited in [4]) could be used. Here, evaluation takes place in 4 phases, starting at level one and as time –and budget allows, move to phase four. Information acquired from each level is used in the successive level. By evaluating training outcomes, the current model can be improved and applied in future development of Embodied Conversational Agents. Then training of social skills could become less costly, time-consuming and easier to repeat
[4][5][6]. When social skills’ training is more accessible, this can result in a reduction of incidents of aggressive behavior.
9. FUTURE WORK
As seen in the discussion, the current implementation of the computational model using the theory of Leary’s Rose should be further developed before it can be used in communication training to replace or add to a human instructor. Real behavioral patterns should be matched to Input and output behavior of the model to make interaction more realistic. Even though the current model shows potential, before it can turn into an embodied conversational agent is still a way to go. First, the model should be able to function across situations. Then, to achieve learning effects a learning theory should be adopted. Finally, an embodied conversational agent can be created.
9.1 Contextual factors
Designing a computation model for interaction requires more than modeling of interpersonal stance [8]. Depending on the context of a situation, interaction is shaped. In some
circumstances it might be true that together behavior does not trigger together behavior, but opposed behavior. Despite the general applicability of Leary’s theory, alternative responses should be included in the model as well. Analysis of use of the computational model based on the interpersonal stance theory of Leary’s Rose, can show inconsistencies between model-responses and human-responses. By studying inconsistencies, contextual aspects can be included in responses of the model. Therefore, tasks given to participants should be varied. Context could be provided before use of the model. For example, scenarios of situations can be used, without posing a solution. People are then asked to complete the scenarios. Afterwards the results are compared to predictions made by the model. This way
inconsistencies and the relation between theory and outcome can be studied [8]. In the current study, variations can be done by adjustment of the target position to match a written scenario. A user can be given the task to communicate in a certain way with the model, keeping the context in mind. This does not only allow for discovering inconsistencies, it also proves if the model can be applied across various situations.
9.2 Learning Effects
One step to add value to this model, is to consider learning strategies. To get users to adopt a new social skill, just practicing with the computational model containing the theory of Leary’s Rose is not enough. The model should be built in a way the communication strategies are understood and applied in real life. One way is shaping the ECA considering the process of observational learning. A trainee learns from observing the consequences of own choices [4]. Observational learning takes an important role in social learning, which is crucial to social skills training. People learn how to behave and respond by observing interaction in their environment. The learning process is described in steps, where attention is when the learner observes something in the environment, then remembers what it was, reproduces it and then experiences the consequence of what happened. This changes the possibility it will happen again [2].
In development of the model observational learning can be modeled by shaping the consequence from the environment. The model already shows the consequence as positive feedback, or reinforcement. This happens when the user ‘wins’ in a set, feedback ‘you won’ is shown. The chance that the user tries to repeat behavior before that sentence is probably bigger than when the feedback ‘you lose’ was given. In future development of the
embodied conversational agent, this positive reinforcement can be implemented in many ways. For example, it seems that positive feedback such as smiling of a virtual agent can contribute positively to task performance. One study found that smiling improves human-machine interaction and can enhance motivation [26]. Furthermore, according to the social agency theory, cues like voice and facial expression can motivate the feeling of interacting with a human instead of a computer [24].
9.3 Embodied Conversational Agent
Only the human-agent dialogue is enhanced using the cognitive theory of Leary’s Rose. To apply the current model to an ECA, many more features should be added. Think of natural language understanding –and generation, animation of the virtual agent, nonverbal behavior generation and multimodal
understanding –and perception. Input -and output of the model should be linked to these features. The features represent behavior. For example, output could be given by facial expressions, which is nonverbal behavior. Or adding spoken or written sentences to the output with natural language generation. On the input side, natural language understanding can improve believability of interaction by taking spoken or written sentences as input instead of just mouse-clicks.
An agent that can interact naturally and is realistic enough to replace a human instructor in role-playing for social skills training does not yet exist. There are some models that try to resemble human instructors [32]. In the study by Tanaka et al. (2016) an agent is created that can recognize speech and language information, and give feedback accordingly. However, no cognitive theory of interpersonal communication is integrated. Just adding a cognitive model is not enough, but combining separate models could result in a virtual agent that is competent enough to replace human instructors.
10. ACKNOWLEDGEMENTS
I would like to thank Tibor Bosse for his helpful feedback as supervisor to this project. I also want to thank participants of the study for their time and cooperation.
Additionally many thanks for two anonymous reviewers who gave me valuable feedback on this study.
11. REFERENCES
[1] Ando S, Kida N, Oda S., (2002). Practice effects on reaction time for peripheral and central visual fields. Percept Mot Skills 95, 747-51.
[2] Bandura, A. (1977). Social learning theory. Englewood Cliffs, N.J.: Prentice Hall.
[3] Bogdanovych A., Trescak T., Simoff S. (2015) Formalizing Believability and Building Believable Virtual Agents. In: Chalup S.K., Blair A.D., Randall M. (eds) Artificial Life and Computational Intelligence. ACALCI 2015. Lecture Notes in Computer Science, vol 8955. Springer, Cham.
[4] Bosse, T., (2016). Opportunities and challenges in computer-based communication training [PowerPoint Slides]. Retrieved from: https://communicate.sites.uu.nl/16-november-2016/.
[5] Bosse, T., & Gerritsen, C. (2017). Towards serious gaming for communication training – A pilot study with police academy students, proceedings of the 8th international conference on intelligent technologies for interactive entertainment, springer. DOI: 10.1007/978-3-319-49616-0_2.
[6] Bosse, T., Gerritsen, C., & Man, de, J. (2016). An intelligent system for agression de-escalation training. Frontiers in artificial intelligence and applications, 285, 1805-1811. [7] Bosse, T. and Provoost, S., (2015) Integrating Conversation
Trees and Cognitive Models within an ECA for Aggression De-escalation Training. In: Proceedings of the 18th International Conference on Principles and Practice of Multi-Agent Systems, PRIMA'15. Springer Verlag, Lecture Notes in Artificial Intelligence.
[8] Bruijnes, M. (2013). Affective conversational models: Interpersonal stance in a police interview context. Affective computing and intelligent interaction (ACII), 2013 Humaine association conference, pp. 624-629.
[9] Bruijnes, M., Linssen, J.M., op den Akker, H.J.A., Theune, M., Wapperom, S., Broekema, C., and Heylen, D.K.J. (2015). Social Behavior in Police Interviews: Relating Data to Theories. In: Conflict and Multimodal Communication, Springer Verlag, pp. 317-347.
[10] Craig, A.B., Brown, E.R., Upright, J., DeRosier, M.E., (2015). Enhancing Children’s Social Emotional Functioning Through Virtual Game-Based Delivery of Social Skills Training. Upright, J. et al. J Child Fam Stud (2016) 25: 959.
https://doi.org/10.1007/s10826-015-0274-8
[11] Dallal, G.E., (2000). Nonparametric Statistics. Retrieved from: http://www.jerrydallal.com/lhsp/npar.htm at 18-09-2017.
[12] Dijk, van, B., & Cremers, M.J. (2013). Actie is reactie: naar effectieve interactie. Zaltbommer: Thema.
[13] Fishman, E.J., Husman, J., (2017). Extending Attribution theory: Considering Student’s Perceived Control of the Attribution Process. Journal of Educational Psychology, 109(4), 559-537
[14] Ghuntla, T.P., Mehta, H.B., Gokhale, P.A., & Shah, C.J., (2014). Influence of practice on visual reaction time. Journal of Mahatma Ghandi institute of medical sciences, 19(2). [15] Griffiths, O., & Thorwart, A. (2017). Effects of Outcome
Predictability on Human Learning. Frontiers in Psychology, 8, 511. http://doi.org/10.3389/fpsyg.2017.00511
[16] Griffiths, O, Mitchell, C.J., Bethmont, A., Lovibond, P.F., (2015). Outcome Predictability Biases Learning. Journal of Experimental Psychology: Animal Learning and Cognition. 41(1):1-17.
[17] Gulz, A. (2004). Benefits of Virtual Characters in Computer Based Learning Environments: Claims and Evidence. International Journal of Artificial Intelligence in Education, 2004, 14, 313-334.
[18] Hays, M., Campbell, J., Trimmer, M., Poore, J., Webb, A., Stark, C., and King, T. (2012). Can Role Play with Virtual Humans Teach Interpersonal Skills? In Interservice/Industry Training, Simulation and Education Conference (I/ITSEC).
[19] Hubal, R.C., Fishbein, D.H., Sheppard, M.S., Paschall, M.J., Eldreth, D.L., & Hyde, C.T. (2008). How do varied populations interact with embodied conversational agents? Findings from inner-city adolescents and prisoners. Computers in Human Behavior, 24(3), 1104-1138. [20] Jeuring, J., Grosfeld, F., Heeren, B., Hulsbergen, M.,
Ijntema, R., Jonker, V., Mastenbroek, N., Smagt, M. van
der, Wijmans, F., Wolters, M., and Zeijts, H. (2015). Communicate! A Serious Game for Communication Skills. In: G. Conole et al. (Eds.): ECTEL 2015, LNCS 9307, pp. 513517.
[21] Lala, R., Jeuring, J., van Dortmont, J. (2017). Scenarios in virtual learning environments for one-to-one communication skills training. Int J Educ Technol High Educ 14: 17. https://doi.org/10.1186/s41239-017-0054-1
[22] Leary, T., (1957). Interpersonal diagnosis of personality. New York. The Ronald press company.
[23] Louwerse, M.M., Graesser, A.C., McNamara, D.S., & Lu, S. (2009). Embodied conversational agents as conversational partners. Applied cognitive psychology, 23, 1244-1255. [24] Louwerse, M.M., Graesser, A.C., Lu, S., & Mitchell, H.H.
(2005). Social cues in animated conversational agents. Applied cognitive psychology, 19, 693-704).
[25] Magallón, S., Narbona, J., & Crespo-Eguílaz, N., (2016). Acquisition of Motor and Cognitive skills through repetition in typically developing children. PLoS One, 11(7):
e0158684.
[26] Ochs, M., Niewiadomski, R., Brunet, P., & Pelachaud, C., (2012). Smiling virtual agent in social context. Cognitive process, 13(2), 519-532.
[27] R. Hill, J. Gratch, S. Marsella, J. Rickel, W. Swartout, and D. Traum, (2003). Virtual humans in the mission rehearsal exercise system. K¨unstliche Intelligenz, 4(03):5–10, [28] Rouckhout, D., & Schacht, R., (2000). Ontwikkeling van een
Nederlandstalig interpersoonlijk circumplex. Diagnostiekwijzer, 4:96-118.
[29] Sawyer, B., (2007). Serious games: Broadening games impact beyond entertainment. Computer graphics forum, 26 (3), pp. xviii. Blackwell Publishing Ltd.
[30] Sternberg, S., (2010). Time Experimentation. Psychology 600-301. Proseminar in Psychological Methods.
[31] Susilo, A.P., Eertwegh, V., Dalen, van, J., & Scherpbier, A. (2013). Leary’s Rose to improve negotiation skills among health professionals: Experiences from a Southeast Asian Culture. Education for Health, 26(1).
[32] Tanaka, H., Sakriani, G., Neubig, G., Toda, T., Negoro, H., Iwasaka, H., & Nakamura, S., (2016). Teaching
communication skills through human-agent interaction. ACM Transactions on interactive intelligent systems, 6(2), 18. [33] Tanaka, H., Negoro, H., Iwasaka, H., & Nakamura, S.
(2017). Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders. PLoS ONE, 12(8), e0182151.
http://doi.org/10.1371/journal.pone.0182151
[34] Thompson S. C. (2002). The role of personal control in adaptive functioning. In C. R. Snyder, S. J. Lopez (), Handbook of positive psychology (pp. 202–213). New York, NY: Oxford University Press.
[35] Vaassen, F. and Wauters, J. (2012). deLearyous: Training interpersonal communication skills using unconstrained text input. In: Proceedings of ECGBL. pp. 505-513.
[36] Vliet, V., (2014). Rose of Leary. Retrieved from
http://www.toolshero.com/communication-skills/rose-of-leary/ at 14-02-2017.
APPENDIX A
Questionnaire pilot study
1. What is the meaning of the red and the green dot on the screen?2. What is meant by Above, Below, Opposed and Together in the screen? Please state the meaning for all of the above. 3. What kind of strategy did you apply while clicking on the screen?
4. How did you feel like you were improving with your answers? 5. Did you finish the challenge?
6. What kind of behavior do you think is represented with 'Above-Opposed' behavior? 7. What kind of behavior do you think is represented with 'Above-Together' behavior? 8. What kind of behavior do you think is represented with 'Below-Opposed' behavior? 9. What kind of behavior do you think is represented with 'Below-Together' behavior? 10. Did you feel like the way the green dot moved in the screen represent responses in real life?
APPENDIX B
Instructions:
Enhancing believability of virtual agents with the theory of Leary’s Rose
The current study investigates a model about the theory of Leary’s Rose. This theory of interpersonal communication provides a better understanding of interaction between people. It is developed by Timothy Leary. He discovered that people react in a predictable manner. Therefore, behavior can be influenced by others. According to the Rose of Leary, a person’s ‘communicative stance’ towards the conversation partner can be represented as a point within a two-dimensional space. Within this space, the horizontal axes describe the relationship with the other person (i.e., together or opposed), the vertical axes describe the attitude towards the other (i.e., above or below). The Rose of Leary with corresponding behavior looks like this:
• Dominant (Above) behavior is acting, initiating and leadership like behavior.
• Submissive (Below) behavior is described as compliant, obedient or passive behavior. Here, the person does not get involved and displays quiet behavior.
• Opposed behavior is described as aggressive and challenging behavior. It is about people that are aggressive and do not agree with people without questioning. Extensive motivations should be given to convince such a person. • Together behavior is about cooperative behavior and people being open-minded and responsive to other people’s
opinion.
These 4 types of behavior are again divided into 8 particles in the rose, which is also shown in the image. The meaning of these types of behaviors is important to understand. In the table below some examples of the 8 distinct behaviors is given.
Behavior Example
Leading ‘’I will propose some options and we will choose from them’’ Helping ‘’I think you are right’’
Cooperative ‘’Tell me what I can do for you’’
Depend ‘’If you think this is the way, the we will go with that’’ Withdrawn ‘’Yeah, ok, whatever, my opinion does not matter that much’’
Defiant ‘’Yes, maybe, but if you are going to act like that, then.’’ Aggression ‘’I absolutely do not agree with that’’
The current study is about you being able to influence another person’s communicative stance. You will be presented with a picture of Leary’s Rose, via which you can ‘communicate’ with the computer. In this picture you will be able to give input by clicking with your mouse on a spot within the rose. For instance, in case you want to approach the computer in a way that is ‘dominant’ and ‘together’, you should click on the upper-right hand part of the rose. The computer responds by moving the red dot on the screen, which represents its own communicative stance. The goal of each trial is to give the model input so that the red dot eventually matches the green dot.
You will be given 10 trials. This will be divided into two sets. After each set you get 5 questions. Try to finish them as fast as possible, but consider the behavior and possible responses of the computer when clicking in the Rose.
APPENDIX C
Questionnaire nr.
I felt like I could influence the responses the model gave me.
Totally disagree Totally Agree
I felt like the responses of the model were logical.
Totally disagree Totally Agree
I was able to apply a strategy when trying to move the red dot towards the green dot.
Totally disagree Totally Agree
I felt like I was improving each trial.
Totally disagree Totally Agree
I think the way the model responded is the same as in real-life communication.
