This site fits YOU mom! The design and implementation of a model underlying an adaptive website to support mothers during decision-making about whether or not to have their daughter vaccinated against HPV

(1)

This site fits YOU mom!

The design and implementation of a model underlying an adaptive website to

support mothers during decision-making about whether or not to have their

daughter vaccinated against HPV

Eva A. van Weel 10244743

Bachelor thesis Credits: 18 EC

Bachelor Opleiding Kunstmatige Intelligentie University of Amsterdam Faculty of Science Science Park 904 1098 XH Amsterdam Supervisors Dr. F.M. Nack Informatics Institute Faculty of Science University of Amsterdam Science Park 904 1098 XH Amsterdam Dr. A. Heuvelink Perceptual and Cognitive Systems TNO Kampweg 5 3769 DE Soesterberg

(2)

Acknowledgements

I wish to thank Annerieke Heuvelink for her guidance, support and inspiration during the process of con-ducting research and writing my thesis at TNO. Furthermore, I would like to express my gratitude to Frank Nack for providing me with clear explanations, instructive drawings and advice during my project. More-over, I would like to express my appreciation to both for calming down my worries, having confidence in me and having a good laugh from time to time. Also, I would like to thank Wilma Otten and Hilde van Keulen for taking the time to participate in my expert evaluation and Mark Neerincx for his help with the interaction patterns described in this thesis. Mirjam Pot I would like to thank for providing me with insights into knowledge representations by showing me the domain from a different perspective. Finally, I want to offer my special thanks to my parents and friends who have been supportive and loving throughout my entire graduation project.

(3)

Abstract

In order to support mothers during decision-making about whether or not to have their daughter vaccinated against HPV, an interactive and adaptive website with two virtual assistants has been devel-opment. To achieve this adaptivity, the adaptive hypermedia approach was followed and four submodels were defined by means of the sCE method. Overall, the implemented adaptations of the web pages and the interactions of the virtual assistants were performed properly according to the expert evaluation. However, some adaptations were not convenient or noticed at all. This thesis thus provides insights into the models necessary to construct an adaptive website and how those models can be implemented.

(4)

1 Introduction

Worldwide, cervical cancer is the third most common cancer among women (Ferlay et al., 2010). In the Netherlands, statistics reveal that in 2012 there were 735 cases diagnosed and 215 deaths from cervical cancer (Integraal kankercentrum Nederland, 2014). Virtually all of these cervical cancers are caused by an infection with the human papillomavirus (HPV) of which types 16 and 18 cause almost 71% of all cases (Munoz et al., 2004). The available vaccination against these types of the virus is believed to be capable of decreasing the number of women who contract cervical cancer by half (Gezondheidsraad, 2008). Therefore, the Dutch government started a national campaign to vaccinate 12 year old girls against these types of HPV in 2009. While the anticipated participation rate for this campaign was 70%, only 52% of the girls who were invited to participate, actually completed all three steps in the vaccination procedure (Ensing et al., 2011). This participation rate was especially low compared to the participation rate of over 90% for the other vaccinations offered by the National Immunization Program (van Lier et al., 2009).

Due to this unexpectedly low participation rate, the Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek (TNO)researched the willingness of mothers and daughters to be vaccinated against HPV. This research was commissioned by the Rijksinstituut voor Volksgezondheid en Milieu and ZorgOnderzoek Ned-erland - Medische Wetenschappen. According to this research, which was conducted by Van Keulen et al. (2010a), mothers fulfilled the most influential role in the decision-making with regard to the HPV vaccina-tion. Further research from Van Keulen et al. (2010b) indicated that mothers wanted more interactive and personalised information about the HPV vaccination. Moreover, 50% of the mothers had neither acquired nor processed detailed information about the vaccination and three months after the decision, 25% of the mothers still felt ambivalent about their choice.

As a result of these studies, TNO is developing a website on which interactive and personalised information about the HPV vaccination will be presented to mothers in order to help them make an informed decision about whether or not their daughters should be vaccinated. Besides receiving interactive and personalised information on the web pages, the mothers will be guided through the website and provided with feedback by two virtual assistants: a peer mother and a doctor. The peer will provide feedback to the mother about her progress on the website, guide her by suggesting what information she should read next and explain why that information would be relevant to her. Furthermore, the peer will show empathy and understanding. The doctor will provide the mother with medical information and personalised feedback based on her answers to knowledge related questions.

In this thesis, we will present our research on the models underlying this interactive and personalised web-site. We will elaborate on those types of information that can be (automatically) acquired from the mother during her interaction with the website, so that the website can be adapted to her information needs. For the website to be able to adapt its content, it needs to decide at what time during interaction certain content needs to be shown and how that content should be presented. We will describe on what information these decisions will be based and how these decisions affect the content of the web pages and the actions of the virtual assistants. Therefore, a model will be constructed in which these decisions and the information to make these decisions are stored. After introducing the relevant submodels, a simplified implementation of the proposed model will be presented. Furthermore, the evaluation of the implementation by two experts from TNO will be described and the results presented. We will finish with conclusions and suggestions for future work.

(6)

2 Related work

The use of web-based tailored, i.e. personalised, interactive programs to support health related decision-making has been researched before in various domains, for instance breast cancer (Banegas et al., 2013) and lifestyle (Schulz et al., 2013; Ezendam et al., 2012). In these studies positive effects of the tailored programs compared to control conditions were found, respectively: reduced decisional conflict towards the decision to take drugs to reduce the risk to contract breast cancer, decreased alcohol consumption and positive short-term effects on diet. Furthermore, research has been conducted to evaluate the effectiveness of tailored web-based interactive programs with virtual assistants. The results from Hudlicka (2013) suggest that virtual coach-based training of mindfulness is potentially more effective than a self-administered program and Friederichs et al. (2014) showed that the use of an avatar in a web-based physical activity intervention increased self-reported physical activity compared to the control condition without the avatar.

The web-based tailored interactive programs deployed in the studies mentioned contain various function-alities, such as acquiring user specific information by means of questions or other forms of user input and providing feedback or information to the user based on the answers given. These functionalities were either provided to the user by means of text on the website, a virtual assistant or a combination of both. To provide these functionalities a model of the user has to be formed based on which the feedback and information on the website can be adapted to the user as "the challenge in an information-rich world is not only to make information available to people at any time, at any place, and in any form, but specifically to say the “right” thing at the “right” time in the “right” way." (Fischer, 2001, p. 65). A possible way to implement this adaptivity and to dispose of the “one-size-fits-all” approach is called Adaptive Hypermedia (AH). “Adaptive hypermedia systems build a model of the goals, preferences and knowledge of each individual user, and use this model throughout the interaction with the user, in order to adapt to the needs of that user” (Brusilovsky, 2001, p. 87). AH systems contain at least three types of models: a user model with user specific information, a domain model with information about the concepts in the domain and the relations between these concepts and an adaptation model which provides information about possible adaptations given the user and the do-main (Knutov et al., 2009). Within an AH system different models and techniques can be used to construct these three models. Two possible approaches to model the user his knowledge are called the overlay model and the stereotype model. An overlay model estimates the user his knowledge of each concept in the do-main model and the stereotype model classifies the user his knowledge by determining to which stereotype the user belongs (Brusilovsky, 1996). Furthermore, different approaches can be followed when modelling the adaptations within an adaptive system (Brusilovsky, 1996), i.e. the presentation of information and the navigation support can be adaptive. Information can for example be emphasised, shown or not be shown or presented differently by means of images or graphs. Adaptive navigation support can consist of for instance hiding, disabling or removing links, providing link sorting or direct guidance.

In order for the virtual assistant to interact with the user, it has to possess several characteristics. It should have the ability to recognise and respond to verbal and nonverbal input, to generate verbal and nonverbal output, deal with conversational functions such as turn taking and provide signals that indicate the state of the conversation. A virtual assistant that has these properties is called an embodied conversational agent (ECA) (Cassell et al., 2000). An ECA engages a user in a dialogue by means of using speech, gestures and other verbal and nonverbal cues which give the experience of human face-to-face interaction and the possibility to build a relationship (Bickmore and Cassell, 2001). Using an ECA has multiple advantages over using only text as it improves recall of the presented information (Beun et al., 2003) and transfer of learning (Atkinson, 2002). Furthermore, the visual presence of the ECA is critical, as a voice alone is less

(7)

effective in conveying a message (Rosenberg-Kima et al., 2007).

Even though web-based tailored interactive programs with ECAs are not new, promoting vaccine uptake by means of such a program is. Therefore, this work will elaborate on the models necessary to provide a web-based tailored interactive program with an ECA to support health related decision-making with a focus on the decision about the HPV vaccination. The corresponding research question is the following: What is the required model underlying an interactive web-based adaptive feedback program with a virtual assistant to support health related decision-making?

During the development of this interactive and adaptive website, the situated cognitive engineering (sCE) method will be used, which describes the process of iterative development cycles (Neerincx and Lindenberg, 2008). The specifications of the website will be refined in each of these cycles, where the initial specifica-tions will be described in this thesis. The specificaspecifica-tions of the website consist of requirements (what), claims (why) and use cases (where, when, who), see Figure 1. These specifications are initially based on analysis of the domain and can change during the process of development. The model underlying the adaptivity of the website, will be described in terms of these specifications and after evaluation be refined by adjusting the specifications.

Figure 1: The relations between the defined requirement, its claims and use cases

3 Models

In order to provide the requested interactivity and personalisation on the website, the introduced adaptive hypermedia (AH) approach is followed. This approach allows to dispose of the idea of “one-size-fits-all” and offers the ability to adapt the content of the website to the user. To achieve this, the website requires information about the domain, i.e. what information can be adapted, the user and the actual adaptations that can be made. Therefore, three submodels are defined which represent this information: a domain model (DM), a user model (UM) and an adaptation model (AM) respectively. Besides these three submodels, as described in the AH approach, a fourth model is defined which represents the interactions of the virtual assistants with the user, i.e. the embodied conversational agent model (ECAM).

Figure 2 shows the relations between the models proposed. The UM and the DM provide input to the AM, based on which the AM can reason about which adaptation to perform given the user and the domain. The ECAM receives its input from the UM and DM, so the ECA can appropriately interact with the user. Also, ECAM receives input from the AM, so the ECA can provide the user with meta-communication about the performed adaptations.

(8)

Figure 2: Relations proposed models

With the sCE approach in mind, the first four requirements (Req) of the model are defined based on these models:

DM-Req 1: the DM shall define the concepts in the domain and the relations between these concepts. UM-Req 1: the UM shall store information about the user that is a relevant source for adaptation. AM-Req 1: the AM shall adapt the content of the web pages to the user based on the input from the DM

and the UM.

ECAM-Req 1: the ECAM shall adapt the interactions of the ECAs based on the input from the DM and the UM or the input from the AM.

For each of these requirements claims (reasons why they should hold) are defined, which are presented with either a+ or a - to indicate a possible positive or negative effect when incorporating the requirement in the system.

Claims DM-Req 1:

+ the DM enables the order and form of the information on the website to be adapted to the user. Claims UM-Req 1:

+ The UM provides the information about the user based on which the information on the website can be adapted to the user’s level of knowledge, needs and interests.

Claims AM-Req 1:

+ The user is likely to understand the provided information better. + The user is likely to enjoy the visit to the website, as the information on the website is adapted to the user’s level of knowledge, needs and interests.

Claims ECAM-Req 1:

+ The user will belief the statements of the ECA. + The user will enjoy the interactions of the ECA.

+ The user will find the interaction with the ECA relevant.

In the following subsections the individual models will be described by means of providing sub requirements and corresponding claims, use cases and rules explaining how the sub requirements can be fulfilled.

(9)

3.1 Domain model

As described in DM-Req 1, the DM contains information about the concepts in the domain and the re-lations between these concepts. It provides a structure of the information in the domain and shows what information can be adapted. The information in the DM is usually be structured hierarchically, where some of the concepts in the hierarchy have child- or sub-concepts. This hierarchical structuring enables a logical navigation of the user through the website. Besides hierarchical relations, it is also possible for relations to exist between concepts in different branches of the hierarchy. These relations can be used to offer alternative routes for navigation to the user (direct links between subparts of the website) or to adapt the content of a subpart of the website given the user’s visit to a related subpart. Two sub requirements and corresponding claims can thus be defined:

DM-Req 1.1: the DM shall define the hierarchical relations between the concepts in the domain. DM-Req 1.2: the DM shall define the relations between concepts in different branches of the hierarchy. Claims DM-Req 1.1:

+ Based on the DM logical navigational routes between the concepts in the domain can be defined. - The DM limits the number of ways in which the content of the website can be approached. Claims DM-Req 1.2:

+ The DM allows logical skip/quick routes between different parts of the website (subconcepts in the domain) to be defined.

+ The DM allows the website content (subconcepts in the domain) to be adapted based on whether related content is yet visited.

In order to fulfil these requirements, a hierarchy needs to be constructed in which these different types of re-lations between the concepts are defined. For the website to support mothers during decision-making about whether or not to have their daughter vaccinated against HPV, a DM is constructed in which these relations are defined. On the highest level in the DM, various knowledge categories are presented, such as basic in-formation, side effects and benefits early age. Each of these knowledge categories contain basic information about the subject and in most cases additional or more detailed information. Besides these hierarchical rela-tions, there also exist relations between the information in different branches of the hierarchy. As described above, these relations indicate either that the same information is present in two or more branches or that information is somehow linked to information in another branch, e.g. that information is also relevant to read given the topic currently being read.

3.2 User model

As described in UM-Req 1, the UM stores those types of information about the user that provide a relevant source for adaption. In order to obtain this information, two methods can be utilised to acquire user input, that is 1) query the user or 2) monitor the behaviour of the user during interaction. The obtained information be either dynamic, i.e. the values change during interaction with the system, or static, i.e. the values do not change during the course of interaction. Furthermore, the information can be domain independent or domain dependent. The former involves information about the user that can be acquired from the user independent of the domain in which the model is applied, for instance age, gender, religion or level of education. The

(10)

latter on the other hand, involves information that is directly related to the domain of the application, such as the user’s level of knowledge about the subject or the information that has already been visited.

For the website about the HPV vaccination, the following requirements are established:

UM-Req 1.1: the UM shall keep track of the user’s degree of knowledge of the concepts in the domain UM-Req 1.2: the UM shall acquire information about the user’s background

In the following subsections each requirement will be elaborated on by presenting the corresponding claims, use cases and rules on how to fulfil the requirements.

3.2.1 User’s degree of knowledge

The claims why to assess the user’s degree of knowledge about the concepts in the domain are presented below:

Claims UM-Req 1.1:

+ The presented information can be adapted to the user’s degree of knowledge.

- The user could experience irritations or impatience by providing the system with information based on which the user’s degree of knowledge can be determined.

For this requirement two more sub requirements are established in order to specify the requirement even further. For each requirement the corresponding claims are presented as well.

UM-Req 1.1.1: the UM shall store the user’s answers to knowledge related questions UM-Req 1.1.2: the UM shall store which concepts in the domain are visited by the user Claims UM-Req 1.1.1:

+ The user’s degree of knowledge about the concepts in the domain can be determined.

- The user could experience irritations or impatience by answering the knowledge related questions as the website is most likely visited to obtain information instead of to provide information.

Claims UM-Req 1.1.2:

+ The website knows which information has been presented to the user and can from that point on be considered as knowledge that the user possesses.

+ The system can keep track of the progress made by the user.

A use case will now be described which illustrates the above mentioned requirements. Furthermore, rules are provided indicating how the requirements are fulfilled.

UM-Use case: acquire the user’s degree of knowledge about facts and fables of the HPV vaccination 1. The mother visits the facts and fables knowledge category on the website

2. Statements are shown to the mother about various facts and fables and she is asked to indicated which statements she thinks are facts and which are fables

3. The mother indicates which statements she thinks are facts and which are fables by clicking either “Fact” or “Fable” for each statement

(11)

4. The website stores the answers

5. The website stores that this specific part of the knowledge category has been visited by setting the concept’s visited boolean to true

6. The website determines the degree of knowledge of the user about the current concept, i.e. facts and fables, in the domain

The final step in this use case describes the website being able to determine the user’s degree of knowledge about facts and fables. As the degree of knowledge does not immediately follow from the mother’s answers, a mapping has to be made from the answers to the corresponding degree of knowledge. Each answer is given a score which represents the user’s knowledge about that specific part of the domain, e.g. 0 for an incorrect answer and 1 for a correct answer. For each concept in the domain, the degree of knowledge is subsequently calculated by means of these scores. By calculating the user’s weighted average knowledge for each concept in the domain, different weights can be assigned in the case that not all knowledge is equally important within the concept in the domain. The following calculation is used, where wi represents

the weight of each question i, si corresponds to the obtained score, n equals the total number of questions

answered so far within that specific concept and t represents the total possible score within that specific concept, i.e. the score that corresponds to the user having answered all the questions correctly.

Kw = n X i=1 wi× si t

In order to provide adaptations based on the degree of knowledge of the user, this score can be mapped to a boolean, i.e. the user has either knowledge or not, discrete values, i.e. low knowledge, average knowledge and high knowledge, or continuous values. Given the adaptations and the interactions of the ECAs which fit this application, as will be described in Sections 3.3 and 3.4, the weighted average knowledge will be mapped to a discrete set of values. The following algorithm describes this mapping:

degreeOfKnowledgeConcept= ""; if Kw< 0.3 then degreeOfKnowledgeConcept ← "low"; end else if Kw< 0.7 then degreeOfKnowledgeConcept ← "average"; end else degreeOfKnowledgeConcept ← "high"; end

Algorithm 1: Mapping of weighted average knowledge to a discrete value to be used by the AM or ECAM The values 0.3 and 0.7 in Algorithm 2 are chosen by estimation. During the development cycles of the website, these values can be adjusted to fit the degree of knowlege of the user better.

3.2.2 User’s background

Background information about the user can be a relevant source to base adaptations on. For UM-Req 1.2, the following claims are established:

(12)

Claims UM-Req 1.2:

+ The information can be adapted to the user’s background information.

- The user could experience irritations or impatience by providing the system with information about its background.

For this specific website, static domain independent variable, such as gender and age, are not relevant as the website is targeted to any mother that has to decide whether or not to have her daughter vaccinated against HPV. The following background information, however, is relevant and therefore needs to be written down in terms of requirements for the website. Why this information is relevant will be described by the claims for each of these sub requirements.

UM-Req 1.2.1: the UM shall store how much information the user has already gathered about the topic before interaction with the system

UM-Req 1.2.2: the UM shall store the user’s level of education The claims corresponding with these requirements are the following: Claims UM-Req 1.2.1:

+ The provided information can be adapted to the user’s initial degree of knowledge.

- The user could experience irritations or impatience by providing the system with information about the amount of information already gathered.

Claims UM-Req 1.2.2:

+ The presented information can be adapted to the user’s level of education.

- The user could experience irritations or impatience by providing the system with information about its level of education.

In order to fulfil these requirements, two questions are posed to the user upon entrance of the website. Therefore, the following rules are used:

amountOfKnowledgeGathered== null; levelOfEducation== null;

if the user enters the web page about background info then if amountOfKnowledgeGathered== null then

pose question about amount of info gathered;

amountOfKnowledgeGathered ← "low|average|high"; end

if levelOfEducation== null then

pose question about amount level of education; levelOfEducation ← "low|average|high"; end

end

(13)

3.2.3 A user template

By fulfilling the requirements introduced above, a profile of the user is constructed which contains the user specific values that are gathered either by posing questions or by monitoring the behaviour of the user on the website, e.g. what links are clicked on. Such a profile is called a user template, which for each user contains the same attributes with corresponding possible values. In the end, different users have obtained different values for these attributes based on their own interaction with the website. A part of the user template for the HPV website is shown below:

Level of education: "low|average|high"

Amount of info gathered: "low|average|high"

Score of knowledge question 1 concept "Facts & Fables": "0|1"

Degree of knowledge concept "Facts & Fables": "low|average|high"

Facts & Fables visited: "true|false"

A user template enables the website to create a profile that is descriptive for each individual user. Reasoning about possible adaptations based on the values of the attributes in the user template occurs within the AM. This reasoning can be complex, as the adaptations can be made based on various (combinations) of values within the template, as will be described in the next section.

3.3 Adaptation model

The AM coordinates the possible adaptations the system can perform, as defined in AM-Req 1. These adaptations should ensure that the system fits the user’s information needs and level of knowledge. The AM links the DM and UM by specifying how the domain needs to be adapted to the user at what time during the interaction. Within adaptive hypermedia systems a distinction is made between two types of possible adaptations, that is adaptive presentation and adaptive navigation support. Adaptive presentation includes those adaptations that change the presentation of the information offered by the system, whereas adaptive navigation support adapts the navigation support to the user, e.g. the colour of the links. These types of adaptations form the two main requirements for the AM of the website.

AM-Req 1.1: the AM shall provide the user with adaptive presentation AM-Req 1.2: the AM shall provide the user with adaptive navigation support The corresponding claims are the following:

Claims AM-Req 1.1:

+ The user will enjoy the presented material.

+ The user will better understand the presented material. Claims AM-Req 1.2:

+ The user will appreciate the presented navigation support.

+ The user will obtain more information, as the navigation support helps to navigate to the most relevant information.

+ The user does not have to search and think what information would be relevant to read next. - The user might experience irritations, as the user would rather decide where to navigate without

(14)

3.3.1 Adaptive presentation

The types of adaptive presentation, that are relevant for this website, are: multimedia adaptation, natural language adaptation and canned text adaptation. Multimedia adaptations consist of adapting the presenting multimedia to the user, such as images, graphs or videos. Adaptation possibilities involve the showing or hiding of certain types of multimedia to specific users, such as showing graphs with various numbers and statistics to users with a higher level of education and more simplified images to users with a lower level of education. Natural language adaptations involve the adjustment of the formulation of the texts presented by the website to the user’s level of education or current degree of knowledge. Lastly, canned text adaptations include the adaption of certain fragments of the text that are presented to the user. Fragments can be inserted or removed, altered or dimmed or highlighted based on the user interacting with the system. The following three sub requirements are established:

AM-Req 1.1.1: the AM shall provide multimedia adaptations AM-Req 1.1.2: the AM shall provide natural language adaptations AM-Req 1.1.3: the AM shall provide canned text adaptations The corresponding claims are the following:

Claims AM-Req 1.1.1:

+ The presented multimedia is understandable for the user. + The presented multimedia is relevant for the user to view. Claims AM-Req 1.1.2:

+ The presented texts are understandable for the user. Claims AM-Req 1.1.3:

+ The presented information is relevant for the user to view.

+ The user does not experience irritations caused by information that is presented more than once within various knowledge categories on the website.

- The user could experience irritations or inconvenience caused by the alterations made to the text. Two use cases will now be described in which all three of the above defined requirements are encountered for mothers with different levels of education.

AM-Use case 1: mother with high level of education visits the knowledge category Probabilities 1. The mother visits the Probabilities knowledge category on the website

2. A graph is shown to the mother indicating the probabilities of contracting HPV

3. The corresponding text on the web page contains information about the statistics described in the graph and references to articles from which the statistics are obtained

4. Some fragments of text are grey of colour, which indicates that the mother has already seen this information

(15)

5. A text fragment is inserted on top of the page explaining in which knowledge category the grey out information has been read before

AM-Use case 2: mother with low level of education visits the knowledge category Probabilities 1. The mother visits the Probabilities knowledge category on the website

2. A descriptive image is shown to the mother indicating the probabilities of contracting HPV

3. The corresponding text on the web page contains information about the image, where the information is provided in a descriptive way, i.e. without the use of statistics

4. Some fragments of text are grey of colour, which indicates that the mother has already seen this information

5. A text fragment is inserted on top of the page explaining in which knowledge category the grey out information has been read before

The following rules show how the website determines which presentation to use: if levelOfEducation== "low" then

show descriptive image probabilities HPV; use descriptive text probabilites HPV; end

else if levelOfEducation== "high" then show graph probabilities HPV; use numerical text probabilies HPV; end

if Facts& Fables visited == "true" then

change colour of fragment about Facts & Fables to grey; Insert fragment on top of page “info read in Facts & Fables”; end

Algorithm 3: Adaptive presentation of the information in knowledge category “Probabilities”

3.3.2 Adaptive navigation support

For the adaptive navigation support, adaptions include direct guidance and adaptive link annotation. Direct guidance includes the presence of for example a next button on a web page. This button guides the user directly to the next (relevant) page. Adaptive link hiding involves the hiding or showing of links to the user based on the interaction with the system. The adaptive annotation of links comprises the changing of the appearance of the links, e.g. changing the colour when the user visited the link or enlarging the font size when the user hoovers over the link. The following sub requirements are established:

AM-Req 1.2.1: the AM shall provide direct guidance AM-Req 1.2.2: the AM shall provide adaptive link hiding AM-Req 1.2.3: the AM shall provide adaptive link annotation

(16)

+ The user is directed through the information in a logical and relevant order and does not have to decide on that order herself.

- The user could experience irritations as she wants to be able to click on every link that is available on the website.

+ The user does not encounter information that is not relevant at that point during the interaction, as only relevant links are presented.

- The user could experience irritations as she wants to be able to click on every link that is available on the website.

+ The user can easily see which information has been visited before, as the appearance of the links is changed.

Two use cases will be described in which the above mentioned requirements are met for mothers with a low and a high level of knowledge about HPV:

AM-Use case 3: mother with a low level of knowledge about HPV visits the knowledge category Basic information 1. The mother visits the Basic information knowledge category on the website

2. An easy readable text is shown about the basic information

3. A link is shown via which the mother can read more detailed information 4. The next button returns the mother to the main menu

5. Back in the main menu, the colour of the link has changed, indicating that the category is visited

AM-Use case 4: mother with a high level of knowledge about HPV visits the knowledge category Basic information 1. The mother visits the Basic information knowledge category on the website

2. A detailed text is shown about the basic information

3. The link to the detailed information is hidden, as the detailed variant of the text has been shown to the mother

4. The next button returns the mother to the main menu

5. Back in the main menu, the colour of the link has changed, indicating that the category is visited The following rules show how the website determines which navigation support to use (in this example for

(17)

the basic information based on the level of knowledge of the mother about HPV): if degreeOfKnowledge(basicInfo)== "low" then

present easy variant basic info about HPV; show link to more detailed info about HPV; end

else if degreeOfKnowledge(basicInfo)== "high" then show detailed variant basic info about HPV; hide link to more detailed info about HPV; end

show next button to main menu; basic info visited== "true"; if nextButton== "clicked" then

navigate main menu;

change colour link basic info in main menu; end

Algorithm 4: Adaptive navigation support in knowledge category “Basic information”and main menu Besides adapting the content of the web pages, the interactions of the available virtual assistant can also be adapted. The following section will elaborate on the aspects of the ECAM that allow these adapta-tions.

3.4 Embodied conversational agent model

The ECAM describes under which conditions the ECA performs which actions, i.e. how to adapt its be-haviour to the user (see ECAM-Req 1). The ECA can perform any of the following actions: inform about relevance of the presented material, provide situated feedback about for instance the user’s answers to knowl-edge related questions and explain about the website’s functioning. For each of these actions requirements are established:

ECAM-Req 1.1: the ECA shall inform the user about the relevance of the presented material ECAM-Req 1.2: the ECA shall provide situated feedback

ECAM-Req 1.3: the ECA shall explain the user about the website’s functioning The corresponding claims are the following:

Claims ECAM-Req 1.1:

+ The user understands why certain information is shown.

- The user could experience irritations because the ECA explains about the relevance, while the user would like to read the information.

+ The user is will listen to the feedback of the ECA, as it is targeted to that user.

+ By providing situated feedback, the user is more likely to understand to the feedback of the ECA, as it is targeted to that user.

(18)

- The user could experience irritations because she wants to read the information on the web pages instead of listening to the spoken feedback.

+ The user understands how the website functions without having to figure that out herself - The user could experience irritations because she wants to figure out how the website works

herself.

3.4.1 Relevance presented material

The ECA can inform the user about the relevance of various types of presented material, i.e. the relevance of posed questions and provided adaptations. Therefore, three sub requirements are defined:

ECAM-Req 1.1.1: the ECA shall inform the user about the relevance of answering the knowledge re-lated questions

ECAM-Req 1.1.2: the ECA shall inform the user about the relevance of answering the background related questions

ECAM-Req 1.1.3: the ECA shall inform the user about the relevance of the adaptations made Below the corresponding claims are presented:

Claims ECAM-Req 1.1.1:

+ The user understands why the questions are posed.

- The user could experience irritations because she does not want to listen to the explanation given. Claims ECAM-Req 1.1.2:

+ The user understands why the questions are posed.

- The user could experience irritations because she does not want to listen to the explanation given. Claims ECAM-Req 1.1.3:

+ The user understands the adaptations that are presented. + The user will appreciate the adaptations that are presented.

- The user could experience irritations because she does not want to listen to the explanation.

The relevance of the presented material can be indicated to the user in various ways, e.g. by bodily move-ments such as pointing or by speech. At the end of this section a large use case will be described in which these requirements are met (in combination with the other ECA requirements).

3.4.2 Situated feedback

The provision of situated feedback by the ECA occurs only in specific situations, for instance after incor-rectly answering a knowledge related question or after reading all available information the system has to offer. The ECA can then inform the user about the progress made on the website. Furthermore, the ECA can suggest to the user what information to read next based on the answers provided by the user to questions posed on the website. To achieve this, the following situated feedback sub requirements are defined:

(19)

ECAM-Req 1.2.1: the ECA shall provide the user with feedback about the user’s answers to the knowl-edge related questions

ECAM-Req 1.2.2: the ECA shall provide the user with feedback about the user’s answers to the back-ground related questions

ECAM-Req 1.2.3: the ECA shall provide the user with feedback about the progress made on the website Below the corresponding claims are presented:

+ The user understands the presented material.

+ The user is more likely to read the presented material, as it is adapted to her.

- The user could experience irritations because she does not want to listen to the feedback. Claims ECAM-Req 1.2.2:

+ The user is understands the presented material.

+ The user reads the presented material, as it is adapted to her.

- The user could experience irritations because she does not want to listen to the feedback. Claims ECAM-Req 1.2.3:

+ The user knows how much of the information is already visited. + The user knows how much of the information is left to visit.

- The user could experience irritations because she does not want to listen to the feedback.

3.4.3 Website functioning

The last interaction that is defined in this model consists of the ECA’s ability to inform the user about the functioning of the website. The following sub requirements are defined:

ECAM-Req 1.3.1: the ECA shall explain how the appearance of the knowledge category links changes after a category has been visited

ECAM-Req 1.3.2: the ECA shall inform the user how the speech of the ECA can be repeated

The corresponding claims will now be presented, after which a larger use case will be described in which all of the above ECA requirements are met:

+ The user understand why the appearance is changed.

- The user could experience irritations because she does not want to listen to the explanation. Claims ECAM-Req 1.3.2:

+ The user knows how to repeat the speech of the ECA in the case that it is not heard or understood. ECAM-Use case 1: from first entrance of website to first knowledge category

(20)

2. The web page shows the user two questions. The first about the user’s level of education and the second about the amount of information already gathered by the user about the subject.

3. The ECA introduces herself and explains that the mother can repeat the virtual assistants speech by double clicking the assistant (ECAM-Req 1.3.2). The ECA asks the user subsequently to answer the questions and informs the user that the answers to the questions enable the ECA to better guide the user on the website (ECAM-Req 1.1.2).

4. The mother answers that her level of education is low and that the amount of information gathered is low as well.

5. The mother enters the main menu and the ECA suggests to read the basic information given that she has not gathered much information before (ECAM-Req 1.2.2).

6. The mother visits the basic information and returns to the main menu

7. The ECA explains that the links that the mother has visited will from this point on look differently to indicate what information she has already read. This information is accompanied by a pointing movement of the hand towards the visited link (ECAM-Req 1.3.1).

8. The mother clicks on the knowledge category “Benefits young age”

9. The web page poses the mother the knowledge related question in order to ascertain the user’s level of knowledge about why the daughters are vaccinated at this age. The ECA explains that the information will be adapted to the mother based on the answers that she provides to the knowledge questions on the website (ECAM-Req 1.1.1). Furthermore, the website shows that some information has already been read before and where on the website. In this specific case a fragment about the benefits of the young age when receiving the vaccination, which was also presented in the basic information. The ECA explains why the fragment of information is coloured differently (ECAM-Req 1.1.3).

10. The mother thinks that the young age has to do with a lower chance of already having contracted the virus. As this is correct, the ECA confirms the mother’s answers (ECAM-Req 1.2.1).

11. When the mother returns to the main menu, the ECA informs the mother about her progress by indicating that she has already seen two categories (ECAM-Req 1.2.3).

(21)

4 Implementation of the models

All the aspects of the four different models, i.e. the DM, UM, AD and ECAM, described in Section 3 could be used to create the ultimate adaptive system. In this system the information, the navigation and the interaction of the ECA(s) are all adapted to the user’s needs and current location within the domain. However, during the implementation of such a system, different elements can influence the feasibility of the system. Examples of such elements are memory capacity, system speed and the possibilities within the cho-sen programming environment. The following subsections will elaborate on the programming environment used to create the website to support mothers during decision making about the HPV vaccination and the implementation of the proposed models.

4.1 TailorBuilder

To create the adaptive website, a programming environment called TailorBuilder (TB)1 is used. TB is an environment which provides an interface to easily create questionnaires by constructing questions and advice items. These advice items contain information that is to be presented to the user only under certain conditions. Based on the answer of the user to the posed question, an advice item is selected and presented to the user. By pressing a ”next” or ”previous” button the user can subsequently navigate through the questions and the information.

4.2 WebSpeaking

The ECAs used within the adaptive system are created by a company called WebSpeaking (WS)2 that pro-vides the movements and speech of the ECAs. In Figure 3 the ECAs are shown which are used for the implementation: a doctor (left) and a peer mother (right).

Figure 3: The ECAs used within the system: a doctor (left) and a peer mother (right) 1For more information about TB visit: https://www.tailorbuilder.com

(22)

4.3 A simplified implementation of the models

In order to implement (aspects) of the models proposed in Section 3, a more sophisticated structure than question/advice items would be necessary. This structure would for example allow to store information about the user, other than its answers to questions within the questionnaire, such as the clicking behaviour of the user. Based on such information about the user, concepts in the domain can subsequently be adapted, after which the interaction of the ECA can be adapted as well. Unfortunately, it became clear during my research that the TB environment (chosen by TNO at the beginning of the project) is not very suitable for programming an adaptive website. Because of the limitations offered and the necessity to work around the predefined TB structures, a simplified version of the proposed models was programmed. Besides limitations provided by the programming environment, other requirements could not be implemented yet, because the content was not available. No natural language adaptations are performed, as the texts were not available in different variants of reading difficulty. Also, there were no multimedia adaptations implemented, as there were no variants of the images which could be used for mothers with different levels of education or knowledge.

The following sub requirements of the AM and the ECA are implemented, as described in Sections 3.3 and 3.4:

• AM-Req 1.1.3: the AM shall provide canned text adaptations

Three types of canned text adaptations are implemented within the website, namely the insertion, alteration and highlighting of text fragments. For each of these adaptations, implementation examples will be provided below.

1. Fragment insertion - occurs when a user visits a web page on which information is presented which he or she already viewed on another web page. At the top of the page, a small text appears explaining that the page contains information that the user has already seen and is therefore pre-sented in a different colour. This way, the user can decide whether or not to read the information again or to skip it. For each reoccurring fragment of text on the website, variables are stored that indicate whether the fragment has already been presented to the user. During loading of the web page, the system decides to insert the specific fragment or not based on the values of the variables connected to that web page.

2. Fragment alteration - occurs based on the answers the user provides to the knowledge related questions. The information presented on the web page then consists of feedback that is adapted to the answer of the user. Given the predefined relations between the possible answers and the corresponding text fragments, the systems can decide which fragment to show when a certain answer is provided. The content of the web page is thus altered based on these predefined relations and the answer(s) given.

3. Fragment highlighting - occurs based on the answers of the user as well. Fragments are high-lighted in two different ways in the implementation, that is either by changing the colour or by underlining the text fragment. An example is provided in Figure 4, where the colour of the text fragments changes to green when the question is answered correctly by the user and changed to red when the answer is incorrect.

(23)

Figure 4: Changing the color of a text fragment based on the answer provided by the user

• AM-Req 1.2.1: the AM shall provide direct guidance, i.e. the mother is guided from one page to the next by means of “next” and “previous” buttons.

• AM-Req 1.2.3: the AM shall provide adaptive link annotation

Within the website adaptive navigation support is included as well, namely adaptive link annotation. Figure 5 shows the difference in the appearance of the links when unvisited (left) and visited (right).

Figure 5: Adaptive link annotation: appearance visited link (left) and unvisited link (right)

All but one of the ECAM requirements defined in Section 3.4 (ECAM-Req 1.1.1-1.1.3, 1.2.1-1.2.3 and 1.3.1) are implemented by calling predefined segments, which contain the speech and movements of the ECA. These segments are development by WS and are approached by means of Javascript calls. Given values from the UM, such as the amount of information gathered and the location of the user on the website, different segments are called. In ECAM-use case 1, an ECA fragment corresponding to the mother having gathered few information is for example called, where in the case of a large of amount of information gathered a different segment would be called.

ECAM-Req 1.3.2: the ECA shall explain how the appearance of the knowledge category links changes after a category has been visitedhas not been implemented in the website as such a speech fragment was not yet recorded for the ECA.

As the TB environment does not allow to export code, a few examples of code are include in Ap-pendix A. The adaptive website itself can be found here: https://www.tailorbuilder.com/ cgi-bin/runtime_login.pl?grid=49&prid=11&taal=NL. The following section will elaborate on the method of evaluation of the implementation.

(24)

5 Evaluation

In order to evaluate the implementation, two experts at TNO were approached. Dr. W. Otten, who has extensive experience in the field of informed decision making and the development of decision-aids, and Dr. H.M. van Keulen, who conducted the prior research about the HPV vaccination uptake and has experience in the field of changing health behaviours among adults.

The experts were asked to visit the website and evaluate the adaptions both on the web pages and the interactions of the ECAs, i.e. are the adaptions logical and correct from an expert point of view (no user experience evaluation). For this evaluation there were no time restrictions set. To be able to compare both evaluations, the experts received guidelines that had to be followed when interacting with the website, see Appendix B. This way, it was ensured that both experts received the same kind of adaptations. Besides receiving the same kind of adaptations, the guidelines were structured such that the experts received various kinds of adaptions during the visit on the website.

By following these guidelines, the experts encountered the following specific adaptations:

(a) The suggestion of the ECA to view the basic information after entering the main menu for the first time (ECAM-Req 1.2.2: the ECA shall provide the user with feedback about the user’s to the background related questions →based on the information gathered from the user by means of a question that little information was gathered beforehand (UM-Req 1.1.2)).

(b) Feedback about the effectiveness of possible other methods to protect your daughter against HPV and cervical cancer (ECAM-Req 1.2.1: the ECA shall provide the user with feedback about the user’s answers to the knowledge related questions →based on the answers to the questions on how effective the user thinks other methods of protection than the vaccination are to prevent against cervical cancer (UM-Req 1.1.1)).

(c) A highlighted sentence in the feedback about the effectiveness of having safe sex (AM-Req 1.1.3: the AM shall provide canned text adaptations → based on the answer that safe sex is be-lieved to be more effectiveness to prevent cervical cancer than the vaccination (UM-Req 1.1.1)). Furthermore, the experts received other adaptations which were not based on the information provided by following the guidelines, but which were based on the actions they choose themselves. These adaptations correspond to the requirements described in Section 4.3, such as adaptive link annotation and feedback on the user’s progress.

Before interacting with the website, the experts received a questionnaire, see Appendix C, that con-tained one question about each possible adaptation that could be encountered during the interaction with the website. This questionnaire had to be filled in during or after the visit on the website. More-over, the experts were asked to write down the order in which they visited the topics on the website, such that this information could be used when analysing the provided answers. Lastly, the experts were asked to write down an explanation for each answers if they felt that the adaptation was not performed correctly or optimally. In the next section the results of this evaluation will be presented.

(25)

6 Results

Each question of the questionnaire consisted of a 5-point Likert scale, where a score of 1 indicates a negative score, 3 neutral and 5 positive. In Table 1 the results are presented for both experts, showing the Likert scale scores provided for each question, the mean and the difference. Green indicates a positive mean, i.e. a score above 3, white a neutral mean, i.e. a score of 3, and red a negative score, i.e. a mean below 3. Blue indicates a difference of 2 between the scores of the experts. The results of question 3 can not be used, as expert 1 did not give a score, because the question was not understood.

Actor Question

Score expert 1

Score

expert 2 Mean Difference

ECA 1: Feedback KQ 5 3 4 2

2: Suggestions KQ 4 3 3.5 1

3: Suggestions BQ - 4 -

-4: Suggestions info visited 3 4 3.5 1

5: Inform about progress 2 3 2.5 1

Web pages 6: Feedback KQ 4 5 4.5 1

7: Repeated info 3 5 4 2

8: Back-references 4 2 3 2

9: Inform about progress 5 4 4.5 1

Table 1: Results questionnaire expert evaluation implementation

For each question, the experts also provided notes to elaborate on the scores they provided on the Likert scales. Below, these notes will be presented (translated from Dutch), where the number of the enumeration corresponds to the number of the question of which the notes are presented:

1. Expert 1 indicats that there is a nice connection between the feedback provided by the ECA and the answers given to the knowledge related questions. However, the ECA’s speech was too fast and there was no possibility to repeat what had been said. The ECA’s text is too long, therefore the tendency arose to read the text on the web page, which subsequently led to missing what the ECA had to say. The movements of the ECA on the other hand were pleasant and supported well what was said. Expert 2 noticed that the speech of the doctor ECA was rather controlling. Furthermore, the doctor indicated that she would check the experts’s answers to the knowledge question within the “Andere manieren om te beschermen” section, which was experienced as taking an exam. According to expert 2, the ECA should help the mother, not act controlling. 2. The notes indicate that expert 1 only encountered such suggestions twice, which was experienced

as nice. Expert 2, on the other hand, indicates that such suggestions were not encountered at all. 3. Expert 1 indicated that the question was not understood. Expert 2 noted that the provided sug-gestions of the ECA correspond quite well with the answers given to the background related questions, i.e. about the information gathered so far, level of education and the intention whether or not to get the daughter vaccinated. A suggestion is done to view the practical information or the general information in a pleasant manner. When entering the main menu, a suggestion is done based on the information that few information was gathered about the topic beforehand. Expert 2 wonders what happens with the information about the level of education.

(26)

4. Expert 1 indicated that only 2 suggestions were encountered and that therefore the answer is not really applicable. Expert 2 reported that it was not clear why the suggestion to view “Protection and how does it work” after reading the basic information was made. Furthermore, the sugges-tion is made to consider the advantages and disadvantages of the vaccinasugges-tion after viewing half of the available components on the website. Expert 2 indicates that the wording for this sugges-tion is a little strange. It was pleasant that the ECA encouraged at that moment. According to expert 2, the ECA can appear more often within the main menu.

5. Expert 1 reported that no information about the progress was received. Expert 2 indicated that the ECA explained how the progress is indicated the first time the main menu is visited. That was good. After that the ECA was observed once more, as soon as half of the components of the main menu have been visited. Expert 2 points out that the ECA is providing few information about the progress made, that can be done more.

6. Expert 1 reported that the layout of the information could be improved and the amount of text reduced. Scrolling should be avoided as much as possible. Expert 2 indicates that this is done well within the following components: “Facts & Fables”, “Other methods of prevention” and “Probabilities”, however too patronising (especially with the “Probabilities” component). 7. Expert 1 indicated that the grey text is not convenient. It does not matter if text is repeated. That

can be nice sometimes, because it can be placed within different contexts. Expert 1 recommends to leave out the grey colour change, but to keep the text. Expert 2 experienced difficulties to assess this adaptation. “Was that the reason I saw grey text within the “Probabilities” compo-nent?”. Expert 2 indicates no text had been presented more than once and therefore assumed that the adaptation was successful.

8. Expert 1 reported that a back-reference had been encountered once, which was fine, but person-ally this did not have to be done by means of grey text. Expert 2 indicated that this was not encounter at all and posed the question whether this involved the grey text presented within the “Kans” component.

9. Expert 1 indicated that this was nicely done, only different colours would be preferred (same colours as logo). Expert 2 reported that if this question considers the main menu that it is all clear then. However, if the question considers the progress bar, then it is less clear. That bar is confusing as it seems that it indicates the progress per component, while the layout of the bar remains the same. A suggestion is made to either use different colours or explain how the progress bars work. Furthermore, expert 2 reported that it is unclear which component is being viewed, as there is no heading indicating which component is currently viewed. Maybe the text about the website being a decision aid (banner on top of web page) could change per component indicating which component is currently viewed. The progress bar might also be more clear then.

In the next section the notable results will be discussed, i.e. the questions were the difference is 2 apart or the mean is 3 or below 3.

(27)

7 Discussion

Overall the adaptations appear to be performed correctly, as most of the means are 3.5 or higher and the experts for most questions do not differ more than 1 point in score. Below the results from question 1, 5, 7 and 8 will be discussed, as these results had a mean of 3 or lower, a difference of 2 or a combination of both.

Question 1: a difference of 2 between the expert’s scores

The explanations provided by the experts about the first type of adaptation, i.e. the connection between the feedback and the answers given to the knowledge related questions, indicate that expert 2 found the texts of the ECA too controlling and directive. No remarks were made about a wrong connection and therefore it can be concluded that the score is based on remark about the content of the speech, not the adaptation itself.

Question 4: a mean of 2.5

From the explanations provided by the experts it can be concluded that the progress adaptation of the ECA was not or barely observed by both experts. The progress is thus provided to the user in a too subtle way and should be made more explicit or applied more often.

Question 7: a difference of 2 between the expert’s scores

From the comments of the experts, it appears that expert 1 noticed that the repeated text’s font changed colour, but that no added value was experienced. Expert 2 appears not to have noticed the adaptations, as she indicates that no information has been observed twice. As the guidelines were followed by the experts, this adaptation was present and thus not noticed by expert 2. Therefore it can be concluded that the use of grey text to indicate that information has been seen before, is not a convenient way to indicate this. Instead of notifying the user about this, the text could either be removed or not changed of colour. As expert 1 indicates “it does not matter if text is repeated”.

Question 8: a difference of 2 between the expert’s scores and a mean of 3

The explanations provided by the experts show that the same observation holds as for the results of question 7. The canned text adaptation of inserting an extra fragment to indicate where the grey text has been read before, i.e. a back-reference, is not experienced as convenient. Expert 1 noticed the adaptation, but did not see the added value and expert 2 did not notice the adaptation. This adaptation should either be made more explicit or not performed at all.

(28)

8 Conclusion and future work

For the design and implementation of a model underlying an adaptive website to support mothers during decision-making about whether or not the have their daughter vaccinated against HPV, the following submodels were defined: DM, UM, AM and ECAM. Each of these models were defined in terms of requirements and claims as to why these requirements should hold.

From the expert evaluation it can be concluded that the implemented requirements of the AM and the ECAM overall performed properly. It can indirectly also be concluded that the requirements of the DM and UM have been implemented appropriately, as the AM and the ECAM are based on this information. Some results, however, indicate a need for further research. The progress should be made more explicit by the ECA in a next implementation and the use of grey text and back-references should be investigated further. Is the use of this canned text adaption necessary or should the same information be presented twice in different contexts? Besides improvements of the current implementation, extra adaptations, such as natural language adaptations or multimedia adaptations, could be added to a next implementation. To enable this, different versions the existing texts and images need to be created. Most importantly, a user evaluation will have to be conducted once the implementation has been refined in order to evaluate the user experience of the website. During this evaluation the provided claims for the user experience, e.g. the user understands the presented information, can be verified. In the case that some claims do not hold for certain requirements, then these requirements need to be implemented differently in a next development cycle. After these refinements, the new implementation should be evaluated again to see if the stated claims are fulfilled. These cycles should be repeated until all the claims hold for the proposed requirements.

(29)

References

Robert K Atkinson. Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2):416, 2002.

Matthew P Banegas, Jennifer B McClure, William E Barlow, Peter A Ubel, Dylan M Smith, Brian J Zikmund-Fisher, Sarah M Greene, and Angela Fagerlin. Results from a randomized trial of a web-based, tailored decision aid for women at high risk for breast cancer. Patient education and counseling, 91(3):364–371, 2013.

Robbert-Jan Beun, Eveliene De Vos, and Cilia Witteman. Embodied conversational agents: effects on memory performance and anthropomorphisation. In Intelligent virtual agents, pages 315–319. Springer, 2003.

Timothy Bickmore and Justine Cassell. Relational agents: a model and implementation of building user trust. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 396–403. ACM, 2001.

Peter Brusilovsky. Methods and techniques of adaptive hypermedia. User modeling and user-adapted interaction, 6(2-3):87–129, 1996.

Peter Brusilovsky. Adaptive hypermedia. User modeling and user-adapted interaction, 11(1-2):87– 110, 2001.

Justine Cassell, Tim Bickmore, Lee Campbell, and Hannes Vilhjálmsson. Designing embodied con-versational agents. Embodied concon-versational agents, page 29, 2000.

E Ensing, M Steen, R Laferte, I Zonnenberg, and P De Hoogh. Opkomst hpv-vaccinaties per 31 december 2010. deel ii van de rapportage: de cijfers, 2011.

Nicole PM Ezendam, Johannes Brug, and Anke Oenema. Evaluation of the web-based computer-tailored fataintphat intervention to promote energy balance among adolescents: results from a school cluster randomized trial. Archives of pediatrics & adolescent medicine, 166(3):248–255, 2012.

Jacques Ferlay, Hai-Rim Shin, Freddie Bray, David Forman, Colin Mathers, and Donald Maxwell Parkin. Estimates of worldwide burden of cancer in 2008: Globocan 2008. International journal of cancer, 127(12):2893–2917, 2010.

Gerhard Fischer. User modeling in human–computer interaction. User modeling and user-adapted interaction, 11(1-2):65–86, 2001.

Stijn Friederichs, Catherine Bolman, Anke Oenema, Janneke Guyaux, and Lilian Lechner. Moti-vational interviewing in a web-based physical activity intervention with an avatar: Randomized controlled trial. Journal of medical Internet research, 16(2), 2014.

Gezondheidsraad. Vaccinatie tegen baarmoederhalskanker. Den Haag: Gezondheidsraad, 2008. Eva Hudlicka. Virtual training and coaching of health behavior: Example from mindfulness

(30)

Integraal kankercentrum Nederland. Nederlandse kankerregistratie, June 2014. URL http://www. cijfersoverkanker.nl/.

Evgeny Knutov, Paul De Bra, and Mykola Pechenizkiy. Ah 12 years later: a comprehensive survey of adaptive hypermedia methods and techniques. New Review of Hypermedia and Multimedia, 15(1): 5–38, 2009.

Nubia Munoz, F Xavier Bosch, Xavier Castellsagué, Mireia Díaz, Silvia de Sanjose, Doudja Ham-mouda, Keerti V Shah, and Chris JLM Meijer. Against which human papillomavirus types shall we vaccinate and screen? the international perspective. International Journal of Cancer, 111(2): 278–285, 2004.

Mark A Neerincx and Jasper Lindenberg. Situated cognitive engineering for complex task environ-ments. Naturalistic decision making and macrocognition, pages 373–390, 2008.

Rinat B Rosenberg-Kima, Amy L Baylor, E Ashby Plant, and Celeste E Doerr. The importance of interface agent visual presence: Voice alone is less effective in impacting young womenâ ˘A ´Zs attitudes toward engineering. In Persuasive Technology, pages 214–222. Springer, 2007.

Daniela N Schulz, Math JJM Candel, Stef PJ Kremers, Dominique A Reinwand, Astrid Jander, and Hein de Vries. Effects of a web-based tailored intervention to reduce alcohol consumption in adults: Randomized controlled trial. Journal of medical Internet research, 15(9), 2013.

HM Van Keulen, M Fekkes, W Otten, S Van der Pal, P Kocken, R Ruiter, and TGWM Paulussen. Onderzoek naar de hpv-vaccinatiebereidheid bij moeders en dochters naar aanleiding van de in-haalcampagne in nederland. A study into willingness of mothers and daughters to be vaccinated against HPV within the framework of the catch-up campaign in the Netherlands. Report KvL/GB, 2010a.

HM Van Keulen, M Fekkes, and TGWM Paulussen. Factsheet tevredenheidsonderzoek hpv-vaccinatie campagne 2010. Leiden: TNO Kwaliteit van Leven, 2010b.

EA van Lier, PJ Oomen, MWM Oostenbrug, SLN Zwakhals, IH Drijfhout, PAAM de Hoogh, and HE de Melker. Vaccinatiegraad rijksvaccinatieprogramma nederland: verslagjaar 2009.[immunisa-tion coverage na2009.[immunisa-tional immunisa2009.[immunisa-tion programme in the netherlands; year of report 2009]. Bilthoven, the Netherlands: RIVM, 210021010, 2009.

(31)

Appendices

A

Code examples

The javascript call to ECA fragment

Show advice items given a condition

In this example, show the appropriate feedback, call the corresponding ECA fragment (above men-tioned Javascript is situated in [AV_RE_VEILIG]) and show the appropriate graph given that the user indicated that the practising safe sex is more effective to prevent cervical cancer than the vaccination. IF (VR_RE_VEILIG GE VR_RE_INENTING) { [AV_RE_VEILIG]; [ADI_RE_VEILIG]; [GR_RE_VEILIG]; }

Change the colour of a fragment of text by means of HTML and Javascript

When the text fragment below "leeftijdHPV" has been visited before, the colour of the text is changed. <span id="leeftijdHPV">Het is het beste om de HPV-inenting op jonge leeftijd te doen, als een meisje nog niet seksueel actief is. Vandaar de leeftijd van 12 jaar. Het heeft geen zin om een HPV-inenting te halen als je eenmaal besmet bent met HPV. Door de inenting op 12-jarige leeftijd te doen, is een meisje al beschermd als zij wÃl’l seksueel actief wordt.</span>

if(localStorage && localStorage.getItem(’leeftijdHPV’) == ’waar’){ document.getElementById("leeftijdHPV").style.color="#A4A4A4"; }

(32)

B

Expert evaluation guidelines

1. Visit the website as if you are a mother who is about to make the decision whether or not to let her daughter get vaccinated against HPV

2. When asked how much information you have already gathered about HPV and the vaccination, indicate that you have gathered little information so far. After providing this answer, the vir-tual assistant will offer the possibility to view general or practical information, select practical information

3. Within the Other methods of protection component indicate that you think that every method to protect your daughter against HPV and cervical cancer is more efficient than the HPV vaccina-tion

4. Follow the suggestions made by the virtual assistant when interacting with the website

5. Aside from the guidelines mentioned above, you are free to navigate through the website in a manner that fits you best

This site fits YOU mom! The design and implementation of a model underlying an adaptive website to support mothers during decision-making about whether or not to have their daughter vaccinated against HPV