USING A FACEBOOK
GAME TO TRAIN
ALCOHOL INHIBITION
A study in motivation and user experience
Student: Astrid (Ans) de Nijs
Student ID: 10648143
Supervisor: W.J. (Wouter) Boendermaker
Second reader: S.C.J. (Sander) Bakkes
Information studies: Human Centered Multimedia
University of Amsterdam
Using a Facebook game to train alcohol inhibition
A study in motivation and user experience
Astrid (Ans) de Nijs
Student UvA ID: 10648143
Information Studies: HCM
University of Amsterdam
ans.denijs@student.uva.nl
ABSTRACT
Alcohol addiction is a serious problem among the youth population in Europe. Therefore there is a need to train controlled processes in the context of alcohol. The current research is a pilot study based on the work by Georgiadis and Boendermaker [9] in which a serious game was designed to make a response inhibition task more entertaining and motivating. In this study, we researched a trainingwhich builds on the serious game by adding social elements. Participants were subjected to a control training, original training, serious game training, or social game training. We investigated whether the training increased inhibition of the automatic tendency to respond to alcoholic drinks, player enjoyment and motivation; and decreased alcohol consumption. Player enjoyment was higher for the social game condition compared to the other conditions. No difference in response inhibition and drinking behavior was found between the pretest and posttest over the conditions. Also, no difference was found for motivation between the conditions. Possible limitations to this study and implications for future work are discussed. In conclusion, our study showed that adding social elements to a serious game can yield a higher player enjoyment, but this does not necessarily lead to higher motivation for participants and better training results for response inhibition in context of alcohol.
Keywords
Serious game, social game, addiction, alcohol, consumption, response inhibition, motivation, enjoyment
1. INTRODUCTION
Alcohol addiction is one of the three major causes of physical and mental health issues and premature deaths on a global scale [5]. Alcohol is one of the top five risk factors causing diseases and disability [19] such as cancer, cardiovascular diseases and liver diseases. Next to health problems, drinking large doses of alcohol also increases the risk of injuries, homicides, traffic accidents and suicides [4]. The European Union is the region with the highest and heaviest alcohol consumption worldwide. In 2009 the adult per capita alcohol consumption in the EU was 12.45 liter of pure alcohol, which is more than double the world average [5]. In the Netherlands, 8.4 percent of the population is considered excessive alcohol drinkers, which implies drinking at least 14 (for women) or 21 (for men) glasses of alcohol per week [18].
Especially young people show higher frequencies of drinking to intoxication. Drinking large doses of alcohol in particular can lead to brain damage and neurocognitive deficits with implications for learning (memory) [10] and intellectual development [30] for young people. It has also been shown that regular alcohol use in teenage years increased the risk of alcohol dependence during the young adult period [6]. Therefore, alcohol addiction is one among many other forms of addiction which demands serious treatment.
Earlier research has shown that it is possible to train controlled processes [25] and in particular response inhibition for alcohol related responses. Training response inhibition in the context of alcohol can cause a negative implicit association with alcohol and decrease alcohol consumption [12]. The Go/No-Go task is an example of a task to train response inhibition and can be used in the context of alcohol [24]. We discuss this material in more detail in section 2.1.
The response inhibition tasks can be tedious to perform, and therefore it can be difficult to motivate individuals to stay engaged in these tasks. One way to increase motivation is to convert the training tasks, such as the ‘Go/No-Go task’, into serious games to make them less tedious and more enjoyable. Serious games are (digital) games used for purposes other than mere entertainment [15; 21]. Iuppa and Borst state that serious games are designed to transfer and improve knowledge and skills. Furthermore, persuasive techniques and content can be integrated into serious games to change social or personal behavior [14].
Research has already been done to investigate whether transforming psychological training tasks into serious games makes them more motivating and enjoyable. For example, Boendermaker and Georgiadis [9] designed a Cheese ninja game as a serious game variant of the Go/No-Go task. Another way to improve the motivation of patients could be adding social elements to psychological tasks. Williamson et al. [28] support this view by stating that games are most powerful when they are personally meaningful, experimental, social, and epistemological at the same time. Adams [1] adds that socializing is a way to make a game entertaining. Furthermore, Yee has shown that a few of the many motivating factors to play online games are socializing, teamwork and relationship [29].
Based on these findings, we propose a design for a social game to train response inhibition in the context of alcohol. Our research includes investigating two aspects: whether the user was motivated when playing the game and whether the user enjoyed playing the game, i.e. the user experience. By means of a social game we intend to place the game within the environment of a social network, Facebook in particular.
In section 2 we will cover literature relevant to our study, and section 3 will cover our research questions. Subsequently, we will describe the methods used in our study in section 4, and report results in section 5. We will discuss these results and give recommendations for future work in sections 6 and 7, and we finish with a conclusion in section 8.
2. RELATED WORK
In this section we will cover the psychological fundaments for the Go/No-Go task, as well as motivation, user experience, and other studies on serious games. We will also indicate the contribution of our research to the field.
2.1 Response inhibition
The underlying theory for this research consists of the dual-process model as proposed by Wiers et al. [26]. This model explains that alcohol addiction results from an imbalance between two systems, an appetitive approach-oriented system and a regulatory executive system. The first system is sensitive to developing automatic tendencies towards alcohol and acts largely automatic. The second system is able to regulate these automatic tendencies and acts largely under conscious control. Earlier research has shown that people with alcohol dependence lack the ability to inhibit their automatic responses to alcohol [13]. According to the dual-process model, automatic responses from the appetitive system can be regulated by the controlled processes of the regulatory system.
One of these controlled processes is response inhibition. This process can be described as the ability to stop or control dominant responses [11; 13]. Response inhibition can be measured and trained by use of the Go/No-Go task. In the Go/No-Go task the participant is presented with pictures which are paired either with a Go or No-Go cue. The participant has to press a key as quickly as possible or refrain from pressing a key, dependent on the presented cue. Pairing a specific stimulus consistently with a No-Go cue can help a participant to acquire better self-control over automatic tendencies towards that stimulus. Consistently presenting No-Go cues in combination with specific stimuli – such as pictures of alcoholic beverages – results in devaluation of those stimuli and an increase in inhibition of automatic tendencies towards alcohol increases. Research has shown that this effect can be long-lasting [12].
In the Go/No-Go task designed by van Deursen et al. [24], the letters ‘P’ and ‘F’ are used to indicate either a Go or No-Go cue. These cues are shown in one of the four corners of a picture presented on the screen. Participants in the training condition are only exposed to no-go trials with alcohol pictures and go trials with non-alcoholic pictures. Houben, Nederkoorn, Wiers, and Jansen showed the effectiveness of the Go/No-Go task on excessive alcohol use [12].
2.2 Motivation and user experience
In order to train response inhibition it is important that participants are motivated to perform the training task. Vallerand et al. [23] state that motivation is associated with outcomes such as curiosity, persistence, learning, and performance. Liu and Chu [17] used digital learning programs to improve motivation. The aim of their research was to motivate students to learn English in a context-aware ubiquitous game learning environment. Liu and Chu [17] stated that active motivation will encourage students to strive for better performance, achievement and ability. For their study they applied the attention, relevance, confidence and satisfaction (ARCS) motivation model. From this model, a motivation survey has been developed. Results from the motivation survey showed that the students who used the digital learning programs had a higher measured learning motivation for attention, relevance, confidence and satisfaction.
In our study we want to research the motivation of participants performing a response inhibition training in the context of alcohol. For measuring motivation we will make use of the ARCS model applied in the study of Liu and Chu [17].
Because the original Go/No-Go task was tedious, we also want to investigate the value of a serious game in terms of user experience. Many definitions of user experience can be found in the literature. One clear definition of experience was given by
Alben [2]: “All the aspects of how people use an interactive product: the way it feels in their hands, how well they understand how it works, how they feel about it while they are using it, how well it serves their purposes, and how well it fits into the entire context in which they are using it.” User experience is often associated with terms such as interaction, senses, (ease of) use, perception, feel, meaning and emotion [3]. The study of Law et al. [16] has shown that user experience experts agree on user experience features as being dynamic, context-dependent and subjective.
A way to evaluate the user experience for games is the player enjoyment evaluation model of Sweetser and Wyeth [22]. Their model is based on heuristics from the games usability and user experience literature. The player evaluation model is structured by the theory on flow [7] and consists of the following eight elements: concentration, challenge, player skills, control, clear goals, feedback, immersion, and social interaction. The player enjoyment evaluation model is of value to our study to measure the user experience of the response inhibition training.
2.3 Serious games
A way to motivate participants and make a training task more enjoyable is to transform the task into a serious game. Earlier work on serious games has already been done in the field of psychology. One example is the study of Dovis et al. [8] on improving the motivation and task persistence of children with Attention Deficit/Hyperactivity Disorder (ADHD) in working memory training tasks by using computer games. Results from this study showed that strong motivational incentives can improve the working memory performance of ADHD children. However, these incentives did not normalize the performance to children without ADHD diagnosis. Nevertheless, the incentives of money (10 euros) and gaming normalized the persistence of performance of ADHD children in the inhibition task as compared to a less money incentive (1 euro). Since it is unpractical to give a child money for every optimally performed task Dovis et al. concluded that a cost-effective way to improve performance of children with ADHD is to translate the task into a game [8].
Another example of a psychological task that has been translated into a game is the Cheese Ninja game designed by Georgiadis and Boendermaker [9]. They designed a serious game based on the Go/No-Go task to improve the entertainment factor and motivation of the users. The participants rated the game as more motivating than the original task. However, since their research was limited to a measurement study it was not possible to investigate whether the game was as effective in increasing inhibition of automatic alcohol tendencies as the original task. Our study will continue on the study of Georgiadis and Boendermaker [9] by adding social elements to their game. These social elements were added in order to improve the motivation of the user and to give the user a better game experience in comparison with the original game and tasks. In this study we will also address a limitation of Georgiadis and Boendermaker’s study [9]. Whereas their research was limited to a measurement study, we will include an inhibition response training and investigate the response inhibition performance and alcohol consumption.
3. EXPECTATIONS
In our study we will compare player enjoyment and motivation between the original task, the serious game and the social game, as well as their effectiveness in increasing inhibition in the context of alcohol. Based on the literature we found, we formulated three research questions with their hypotheses and expectancies.
The first research question of our study is:
“Will adding social elements to a serious game for prevention of alcohol abuse enhance the user experience?”
The hypothesis derived from this research question is that adding social elements to a gaming task will improve the user experience compared to a game without social elements. We expect that participants in the social game condition will report a higher player enjoyment score than participants in the other conditions. Furthermore, we expect that participants in game conditions will report a higher player enjoyment score than participants in the original task condition.
The second research question is:
“Will adding social elements to a serious game for prevention of alcohol abuse improve the motivation of the user for performing the task?”
Our hypothesis is that adding social elements to a gaming task will improve the motivation of the user compared to a situation without social elements. We expect that the reported motivation score by the participant will be higher in the social game condition in comparison with the other conditions. In addition, we expect that participants in the game conditions will report a higher motivation score than participants in the original task condition. The third research question is as following:
“Will adding social elements to a serious game for prevention of alcohol abuse lead to an improved effect on response inhibition towards alcohol tendencies?”
We hypothesize that by adding social elements to a gaming task the user will drink less alcohol compared to a situation without social elements. Furthermore, we hypothesize that the performance on the task will improve more after playing the social game than in other conditions. Therefore, we expect that the alcohol consumption a week after the training is lower compared to the alcohol consumption in the week before the training when playing social game. Furthermore, we expect participants to have an increased response inhibition towards alcohol.
4. METHODOLOGY
We set up an experiment to answer the research questions. First we will describe the participants of our study. Then, we will cover the materials and measures used in our study and discuss the procedure and design of the experiment.
4.1 Participants
We recruited participants for our study based on age ranging between 18 and 30. The participants had to be native Dutch speakers and drink alcohol occasionally or on a regular basis. Participants were recruited to execute the experiment in either a controlled lab environment or remotely from their home in order to maximize the sample size.
In total 69 individuals participated in our study. 35 participants performed the experiment at home and 34 in a controlled lab environment. 53.6% of the participants were male (37) and 46.4% were female (32). There was an even distribution of the participants over the conditions for gender (χ2 (3)= 5.516, p = 0.145) and remote or lab version (χ2 (3) = 0.159, p = 1.000). The participants were aged between 18 and 29, the mean age was 22 (SD=3.0). The participants in the controlled lab setting were psychology students from the University of Amsterdam, whereas the remote participants consisted of both students as well as non-students. There was a difference in age between participants in the
controlled lab and participants at home, t(55.96)=-5.33, p=0.001. The group participants in the controlled lab had a slightly younger age (M=20.44, SD=1.862) than the participants at home (M=23.71, SD=3.102).
4.2 Materials and measurements
We measured player enjoyment, motivation, response inhibition, and alcohol consumption to answer the research questions.
4.2.1 Questionnaires
All the questionnaires, instructions and tasks were in Dutch. Both the user experience and motivation questions were measured on a 5 level Likert scale with the options “strongly agree”, “disagree”, “neutral”, “agree”, and “strongly agree”. A smiley was paired with each of the options to help visualize them, with the intent of helping participants indicating the option closest to their opinion. The player enjoyment of the tasks was measured with a questionnaire based on the eight elements of the GameFlow model [22]. Each element consisted of a set of criteria to evaluate games on player enjoyment. For each criterion, a score between 0 and 5 was given, where 0 was interpreted as not applicable, 1 as lowest score and 5 as the highest score. We translated the criteria of the elements into 20 statements which reflected the opinion of the user and were relevant for our tasks. We compiled them into one questionnaire. Because three of the four conditions in our study did not include social elements, we excluded the element social interaction criteria from the player enjoyment questionnaire for all conditions. Instead of the social interaction criteria, we added a monitoring question where we asked the participants from the control group, original training group and serious game training group whether they thought adding social elements (such as social interaction with other players and competition features) to the task would be a good idea. We asked the participants from the social game training whether they thought the social elements in the tasks were a good addition to the task. The questionnaire items are included in appendix A.
Our measurement of motivation was based on the study by Liu and Chu [17]. The questions were based on the four sub-scales attention, relevance, confidence, and satisfaction. We adjusted the questions from [17] to our tasks. There were 13 motivation questions in total, which are included in appendix B.
To measure the alcohol consumption of the participants, we used a modified version of the Timeline Follow-Back questionnaire (TLFB) [27] based on the original questionnaire by Sobell. This was a shorter version than the original Timeline Follow-Back survey in which the alcohol use of the participant per day over the past week was measured. We also asked whether the participants drank more than 4 (female participants) or more than 5 glasses (male participants) of alcohol during one occasion during the week before and the week after the training to determine the number of binge drinking occasions. We also included the question whether the week in particular differed from regular weeks and therefore led to more alcohol consumption than usual. Participants could indicate whether there were special occasions during the week before or after the last training session. Answers to this question could provide an explanation when no difference in alcohol consumption is found.
Participants were tested on having alcohol-related problems with the Alcohol Use Disorder Identification Test (AUDIT) [20]. The test included 10 multiple-choice questions regarding alcohol consumption and alcohol related problems. The overall AUDIT score ranged between 0 and 40. An AUDIT score of 8 or higher indicated an increased risk of alcohol-related problems, and an AUDIT score of below 8 indicated non-hazardous alcohol consumption.
4.2.2 Go/No-Go task
The Go/No-Go task had three versions: the measurement task, the training task, and the non-training task.
The Go/No-Go task we used for our study was similar to the one used by van Deursen, Salemink, Smit, Kramer and Wiers [24]. The training and measurement tasks had the same interface and images shown to the participant. The measurement task was shorter than the training tasks and the data from the measurement task was used to compare the response inhibition performance, whereas the training task data was only used for training the participant.
A trial consisted of either an alcoholic beverage image, non-alcoholic beverage image, or image of another object, and a cue. The cue indicated whether the participant had to press a key (the spacebar) or not. In each trial the participant had 1500 milliseconds to respond. The measurement task consisted of 80 trials and some practice trials. Each training session consisted of 200 trials.
The difference between the training Go/No-Go task and non-training Go/No-Go task was the pairing of the images and cues. In the training task, images of alcohol were paired with No-Go cues and alcohol images are paired with Go cues. In the non-training task there was no relation between the content of the images and the cues. The interface of the Go/No-Go task is displayed in Figure 2.
To measure inhibition, we compared the alcohol bias reaction time of the Go/No-Go pretest to the Go/No-Go posttest between the conditions. The alcohol bias reaction time was calculated by subtracting the average reaction time in a go-trial of a participant with alcohol image to the average reaction time in a go-trial without an alcohol image. A low alcohol bias score indicated that there was a bias towards alcohol. A higher alcohol bias indicated a smaller bias for alcohol. Furthermore, we compared the sum and average number of mistakes over the participants between the conditions.
4.2.3 Serious game
For the serious game condition the Cheese Ninja game was used. The main character in the game was a ninja mouse that walked through a hallway. When walking he passed by posters of objects. The mouse has the ability to drop objects which were presented in front of the posters being held by a mechanical claw. The goal of Cheese Ninja was to collect as many food objects as possible while ignoring harmful objects such as traps. The alcohol and non-alcoholic pictures from the Go/No-Go task trials were within the poster frames in the Cheese ninja game. The objects in front of the posters were cues, which indicated a Go or No-Go situation. At the start of each level, the Go cue and No-Go cue were presented to the participant. The cues were objects which were either pleasant or harmful for the mouse. In the Go situation stimuli of non-alcoholic beverages were paired with pleasant objects and in the No-Go situation stimuli of alcoholic beverages were paired with harmful objects.
In the social game condition, the serious game was registered as a Facebook game on the social network site Facebook1. The players had to sign into Facebook in order to play the game. We registered Facebook test accounts for participants to create a controlled environment within Facebook and secure the participants’ privacy. The Facebook accounts were personalized with the name of the participants, which they provided to us. The test accounts all had a neutral but unique profile picture. The participants were able to view the accounts of other participants, for the accounts were linked by Facebook friendship. Facebook’s ‘Like’ and ‘Share’ buttons were implemented underneath the game. Additionally, after each level the player could choose to post his level score and achievement to his Facebook timeline. The Facebook timeline is the personal page for each Facebook account. Content on the timeline could be viewed by other Facebook users. The interface of the social game within the Facebook environment is shown in Figure 1.
4.3 Procedure
In our study the participants were divided over four conditions: the placebo training, the original Go/No-Go training, serious game training, and social game training. Only the training tasks differed per condition. The placebo condition was the control group, in which the participants performed the original Go/No-Go task. In
1http://www.facebook.com
this version of the Go/No-Go task, the alcohol images were paired with both Go-cues as No-Go cues. In the original Go/No-Go condition participants were trained with the original Go/No-Go task. In the serious game condition participants were trained with the Cheese ninja game. In the fourth condition participants were trained with the social version of the serious game.
All participants filled in the AUDIT questionnaire, TLFB questionnaire, motivation and player enjoyment questionnaire, and the TLFB follow-up questionnaire. Furthermore, participants executed the same inhibition pretest and posttest. The setup of the experiment and flow of the tasks during the experiment are shown in Figure 3.
The experiment consisted of three sessions. In all of the three sessions, the participant was subjected to a training task. All sessions were executed behind a computer. Sessions were at least a day apart and at most a week.
For the first session each participant filled in some general questions regarding their age, gender, gaming experience and experience with other alcohol studies. Then, the participant filled in the AUDIT questionnaire. Subsequently, the participant performed the inhibition pretest which is the Go/No-Go measurement task. The TLFB questionnaire followed after the inhibition pretest. The first training task was the last part of the first session. The second session consisted only of a training task. The third session started with a training task. Then, the participant filled in a questionnaire on motivation and player enjoyment. The last part of the session was the inhibition posttest. A week after the third session, the participant received the TLFB follow-up questionnaire by email.
5. RESULTS
This section covers analysis of the results obtained from the experiment. The α value used in all of our tests was 0.05.
5.1 Player enjoyment
The internal consistency of the sub-scales of our questionnaire for player enjoyment was measured. The Cronbach’s α of most of the sub-scales was below 0.7 and therefore not reliable enough to use as independent scales for our study. However, the overall reliability of the questionnaire was high, Cronbach’s α = 0.77. The overall player enjoyment score of the game was measured by the average over the sum of all the items in the questionnaire. We compared the serious and social game to the original task. The player enjoyment scores from the placebo and original training were combined. The reason for this was the small sample size of the original training group. Moreover, the interface of the Go/No-go task of the placebo group and original training group were identical.
The player enjoyment score mean of the original task was 3.21 (SD=0.36). For the serious game, the mean enjoyment score was 3.22 (SD=0.50), and for the social game it was 3.50 (SD=0.41). The average player enjoyment scores per condition are shown in Figure 4. There was a marginal significant difference between conditions in player enjoyment, F(2,66) = 1.01, p = 0.056, η𝑝2 = 0.08.
Comparisons between the conditions revealed that the social game was more enjoyable than the original task, p = 0.021. There was a borderline significant effect for better enjoyment on the social game over the serious game, p = 0.057. However, the results should be interpreted carefully, for the normality assumption for the serious game was violated (W(17) = 0.81, p = 0.003). For the serious game condition the skewness was -1.71 (SE=0.55) and
Figure 4 - Experiment setup
Figure 3 - Average motivation scores
2 2,5 3 3,5
Original task Serious game Social game
Mot
iv
at
ion sc
or
e
Figure 5 - Average player enjoyment scores
2 2,5 3 3,5 4
Original task Serious game Social game
A ve ra g e pl ay er en jo ym en t sc or e
kurtosis 2.97 (SE=1.06). We therefore performed the Kruskal– Wallis test which revealed a significant difference of player enjoyment between groups, H(2) = 6.61, p = 0.037, with a mean rank player enjoyment score of 30.25 for the original task, 34.97 for the serious game and 45.72 for the social game. A Mann-Whitney test revealed a significant difference between the original task and the social game condition, U = 163.5, p = 0.013. In the original task, serious game, and social game condition participants gave an average score of 3.33 (SD=1.02), 2.80 (SD=1.01), and 2.82 (SD=0.88), respectively, to the question whether they thought social elements would be a good addition to the task. Participants in the social game condition had a 3.19 average score (SD=0.54) on the question whether they think the social elements were a good addition to the task. The scores of 2 indicate disagreement, whereas scores of 3 indicate neither agreement nor disagreement with the statement. Of the 17 participants in the social game condition, 2 participants posted achievements on their Facebook timeline. Unfortunately, we could not measure the number of participants who pressed the ‘like’ button. None of the participants shared the game on their Facebook timeline.
We compared the player enjoyment scores of both game version conditions to the non-game conditions. There was no difference between the player enjoyment experience of the participants who performed the game version of the task and players who performed the non-game version of the task, t(67) = -1.48, p = 0.143.
5.2 Motivation
The motivation sub-scales had an insufficient Cronbach’s alpha (below 0.7). For the overall motivation survey a Cronbach’s α of 0.87 was found. Because the overall motivation survey was reliable, the calculation of the motivation score was the grand mean of the questions. The average motivation scores were 2.92 (SD=0.53) for the original task, 2.81 (SD=0.62) for the serious game, and 3.16 (SD=.48) for the social game. The mean and standard error per condition are visualized in Figure 5.
We compared the serious game and social game conditions to the original task for the same reasons mentioned in the player enjoyment analysis. No effect on motivation was found between the conditions, F(2, 66) = 1.09, p = 0.166,η𝑝2= 0.05.
Also, no significant difference in motivation was found when comparing the serious game condition and social game condition to the original task separately. However, there was a marginal significant difference between the social and serious game in motivation, p = 0.066.
We compared the game conditions to the non-game conditions. There was no significant difference between the motivation of participants playing the game versus the participants who performed the original task, t(67) = -0.44, p = 0.663.
5.3 Alcohol bias
In all conditions, the average alcohol bias increased between pretest and posttest. The reaction bias means at the pretest and posttest per condition are shown in Table 1. Note however, that in the social game condition the results for both pretest (W(16) = 0.82, p = 0.006) and posttest (W(16) = 0.85, p = 0.014) were not normally distributed. We looked at the non-parametric Friedman test, but could not apply such test as we needed at least three measurement occasions. Therefore we applied a repeated measures design ANOVA. There was a significant difference in alcohol bias between pretest and posttest, F(1,65) = 4.72, p =
0.033, η𝑝2
= 0.07. No significant effect was found in alcohol bias over time between conditions, F(3, 65) = 0.60, p = 0.617. Because of the violation of normality, the results should be interpreted with caution.
Regarding to the non-normality of the data, the pretest in the social condition had a skewness of -1.54 (SE=0.56) and kurtosis of 1.99 (SE=1.09). The posttest in the social condition had a skewness of -1.08 (SE=0.56) and a kurtosis of 0.27 (SE=1.09).
Table 1 - Alcohol bias means at pretest and posttest
The placebo condition had no normal distribution for the posttest, W(21) = 0.71, p = 0.000, with skewness of 2.52 (SE=0.50) and kurtosis of 7.83 (SE=0.97).
We also compared the alcohol reaction time bias from the three training conditions combined to the placebo condition. The results showed again a significant difference in time, F(1, 67) = 5.81, p = 0.019, η𝑝2 = 0.08. However, there was no significant difference in alcohol bias over time between the non-training and training conditions, F(1, 67) = 0.72, p = 0.399. Please again note that both the training and placebo conditions had a non-normal distribution at the posttest, W(21) = 0.71, p = 0.000, W(48) = 0.95, p = 0.046. The placebo condition had a skewness of 2.518 (SE=0.501) and kurtosis of 7.83 (SE=0.97). The training conditions had a skewness of -0.84
Table 2 – Average number of errors per condition
Condition Mean Std. Deviation No. participants Pretest Placebo 1.39 1.53 21 Original task 1.41 1.46 15 Serious game 2.11 1.45 17 Social game 1.11 1.05 16 Posttest Placebo 1.52 1.50 21 Original task 1.80 1.57 15 Serious game 1.65 1.66 17 Social game 1.13 1.15 16 Condition Mean Std. Deviation No. participants
Alcohol bias at pretest
Placebo -1.15 35.05 21
Original task 7.04 38.17 15
Serious game 11.16 31.85 17
Social game 2.44 28.42 16
Alcohol bias at posttest
Placebo 18.72 54.70 21
Original task 26.85 28.73 15
Serious game 14.32 27.18 17
Table 3 – Alcohol consumption results per condition
Condition Mean Std. deviation
No. participants
Alcohol consumption pretest
Placebo 10.91 10.76 20
Original task 16.82 18.67 15
Serious game 13.71 8.04 16
Social game 18.88 20.19 16
Alcohol consumption posttest
Placebo 11.88 13.21 20
Original task 13.20 17.51 15
Serious game 17.76 23.84 16
Social game 18.50 13.41 16
(SE=0.34) and kurtosis of 0.92 (SE=0.67). Therefore, results should be interpreted with caution for we could not apply a non-parametric test.
We looked at the average number of errors made during the pretest and posttest. These are shown in Table 2. We also looked at the total errors made by participants. The total number of errors for all participants was 115 for the pretest, and 105 for the posttest. On average, the number of errors per participant were more or less the same in the posttest (M=1.52, SD=1.47) and the pretest (M=1.49, SD=1.41). Finally, there was no correlation between alcohol bias and the AUDIT score of the participants, r=.04, p = 0.718.
5.4 Alcohol consumption
From two participants we had no data of the TLFB pretest or posttest. Therefore, data of two participants were excluded from this analysis.
The average amount of alcohol consumed in the week before the pretest was 14.8 glasses2 (SD = 14.98). The mean AUDIT score of the participants was 9.34 (SD=4.64) and there was no significant difference in AUDIT score between the conditions, F(3,63) = 1.81, p = 0.155. 66.7% of the participants had an AUDIT score of 8 or higher, which indicates hazardous drinking [20]. 44.9% scored 10 or higher which is considered to be a cut-off score for alcohol problems [13; 26].
We investigated whether there was a significant decrease of alcohol consumption by comparing the overall amount of consumed glasses of alcohol in a week for the pretest and posttest. There was a significant difference in alcohol consumption between the male and female participants at the alcohol consumption pretest, t(48.35) = 2.346, p = 0.023. Male participants had an average of 18.5 glasses of alcohol, and female participants had an average of 10.5 glasses. The average alcohol consumption per condition for the pretest and posttest is shown in Table 3. There was no significant difference between the alcohol consumption of the pretest and posttest, F(1, 63) = 0.03, p = 0.870. Also, no significant difference was measured of the alcohol consumption over time between the conditions, F(3,63) = 0.999, p = 0.399.
2 One glass is the alcoholic equivalent of 25 cl of beer containing 5% alcohol
It should be noted that the assumption of normal distribution has been violated over the placebo, original task and serious game condition for the pretest. For the posttest, normality for the placebo, original task and serious game condition was violated. For this data we also could not perform a Friedman test due to reasons discussed in section 5.3. For the pretest, the placebo condition (W(20) = 0.86, p = 0.009) had a skewness of 1.39 (SE=0.51) and kurtosis of 1.90 (SE=0.99). The original task (W(15) = 0.72, p = 0.001) had a skewness of 2.54 (SE=0.58) and kurtosis of 7.56 (SE=1.12). The social game condition (W(16) = 0.79, p = 0.002) had a skewness of 2.00 (SE=0.56) and kurtosis of 5.00 (SE=1.09). In the posttest, the placebo condition (W(20) = 0.83, p = 0.002) had a skewness of 1.38 (SE=0.51) and a kurtosis of 1.49 (SE=0.99). The original task (W(15) = 0.70, p = 0.001) had a skewness of 2.51 (SE=0.58) and a kurtosis of 7.08 (SE=1.12). The serious game condition (W(16) = 0.76, p = 0.001) had a skewness of 1.82 (SE=0.56) and kurtosis of 3.07 (SE=1.09). We measured how many participants had a week with special events which could influence the alcohol drinking behavior. For 28.4% of the participants this was the case for the pretest. 47.8% of the participants had special events in the week after the last training. We also investigated whether having a week with special events was related to a decrease in alcohol consumption over the conditions during the pretest and posttest. No difference in alcohol consumption was found when combining condition and special week, F(8,52) = 1.76, p = 0.107.
Finally, we exploratively looked whether participation in other alcohol research studies influenced the alcohol consumption and alcohol bias during the pretest. In total, 30.4% of the participants stated they had experience with other alcohol studies. No significant difference was found in participation in other alcohol researches and the pretest alcohol consumption, t(65) = 0.07, p = 0.944. Neither was there a significant difference measured between participation in other alcohol studies and alcohol bias during the pretest, t(67) = -0.26, p = 0.798. We looked at the influence of participation in other alcohol studies on the training for alcohol bias and alcohol consumption. No effect in difference for alcohol bias was found in a repeated measures analysis with condition and participation in other alcohol studies as independent variables, F(3,61) = 1.14, p = 0.340. Also, no effect in difference for alcohol consumption was found in a repeated measure analysis with condition and participation in other alcohol studies as independent variables, F(3,59) = 1.36, p = 0.263.
6. DISCUSSION
In this section we will elaborate on the results we presented above. Furthermore, we will cover the limitations of our study.
6.1 Player enjoyment
Our study showed a borderline significant difference in player enjoyment between conditions. More specifically, there were indications that participants may have experienced the social game as more enjoyable than the original task and the serious game which is partly in line with our expectations. We believe the trend we found may become significant given more power. The results we found could indicate that placing a serious game of a psychological task on Facebook makes psychological tasks more enjoyable. An explanation could be that adding a social network integration to a serious game makes it more enjoyable. Another possibility could be that participants have a positive association with the social network Facebook in particular and therefore enjoyed the game more than participants in other conditions. The personalized Facebook test account we created for the participants
in the social game condition could have influenced the player enjoyment as well. Participants might have liked playing the game on an individual Facebook account with their personal name. It is hard to indicate what specific social element in the game or part of Facebook would make the game more enjoyable. Therefore, we suggest that more research should be done on this matter for future work.
It is surprising that only 2 of 17 social game participants actually posted achievements to their timeline and none of the participants shared the game on Facebook while the player enjoyment was higher for the social game condition than the other conditions. An explanation could be that people do not feel the need to share their achievements and activities to others, who they do not know in their personal lives.
However, a limitation of our study is that it is hard to measure exactly what makes the social game more enjoyable compared to the serious game. For example, we could not measure whether the presence of the Facebook elements makes the game more enjoyable, or being able to share game achievements to friends. Results from our study show that the serious game was not more enjoyable compared to the original task. This is in contrast to our expectation; we expected a significantly higher average player enjoyment score for the serious game over the original task. When we look at the comments which participants left behind in the survey, it emerges that some participants view the game as frustrating, tiresome and tedious. Others are confused about whether they would pass a level and would have liked to have a progress bar. Because of the comments on the game, the game should be altered to make it more enjoyable. This would likely make the social game also more enjoyable, because the game is exactly the same but integrated into Facebook.
6.2 Motivation
In our study, the motivation was not significantly different over the conditions. This is contrary to our expectation that the social game would be more motivating than the serious game and original task. The average scores for motivation in the conditions were around 3, which is neither a low or high score for motivation. This implies that the motivation of the participants for each condition was roughly the same; participants were motivated but not more motivated for the serious game or social game compared to the original task.
From our results can be interpreted that the game version is not more motivating than the original task. Adding social elements to a game will also not make the game more motivating over the original task. Making a psychological task more visually appealing might increase the player enjoyment but not the motivation. One possible explanation could be that participants recruited online were less aware of a training towards decreasing alcohol consumption than the psychology students in the lab who were aware of a training study in context of alcohol. Therefore, the online recruited participants might not have viewed the training as useful.
Another cause for the lack of difference between motivation scores could be that participants were not motivated to change their alcohol drinking behavior. One third of the participants had an AUDIT score of 7 or lower, which indicates they do not have a drinking problem. These participants have a very low risk of alcohol problems, which would explain why the training would not have any value to them. Training response inhibition towards alcohol tendencies would only be necessary when people have a hazardous alcohol drinking behavior.
The results might also be explained because participants who were motivated in performing the training might not have noticed an immediate training effect in their everyday life, for the participants were retrained in an indirect and automatic behavior. In this case, telling participants they were trained in gaining better control over their alcohol consumption might have given them high expectations in training.
6.3 Inhibition performance
The alcohol bias increased significantly over time which implies that the bias for alcohol decreased. We could interpret that the response inhibition towards alcohol tendencies increased. However, the alcohol bias was not significantly different between conditions, suggesting the training was not more effective than the placebo task. This is in stark contrast with our expectations: we expected the training to perform better than the placebo task, and we expected the game version to perform better than the non-game version. We also expected participants to move towards the neutral bias of 0, as suggested by van Deursen et al. [24]. However, we observed a bias of above zero, which indicates a bias towards non-alcoholic drinks.
A cause for these results could be that participants did not have alcohol related problems. This would mean that participants did not have an alcohol bias and then training would not have been useful. Another explanation could be that participants already performed the tasks in another alcohol study and therefore are already trained in response inhibition in the context of alcohol. However, test results showed no relation between participation to other alcohol studies and alcohol bias. Nevertheless, more research into this is needed as our sample for this test was quite small. Finally, an explanation could be that the training of three sessions and 600 trials was too short to measure a difference in response inhibition. More sessions and more trials might be necessary to train the participants better in response inhibition towards alcohol tendencies.
6.4 Alcohol consumption
There was no significant difference found between the alcohol consumption in the week before the training and week after the training. This is in contrast with our hypothesis: we expected participants performing the social game training to have a decreased alcohol consumption in comparison with other conditions.
An explanation for the result could be that almost one third of the participants stated they had a week with special events in the week before the training. This means the participants drank more alcohol in that particular week than they would drink in a regular week. Also, almost half of the participants stated they had a week with special events after the training which influenced their alcohol consumption behavior. These outcomes could explain why no difference was found in alcohol consumption. However, our test results showed that having a week with special events was not related to alcohol consumption, although our sample size was small for a special week condition and the task conditions. Another reason for these results could be that part of the participants did not have alcohol related problems. Therefore, the training could have improved their response inhibition but not decreased their alcohol consumption. Furthermore, participation in earlier alcohol studies might have influenced the alcohol consumption behavior. However, no significant difference was found between pretest and posttest in alcohol consumption for participants who participated in other alcohol research and participants who did not.
Finally, it is possible that 600 trials is not sufficient to cause a significant change in alcohol bias, and therefore, alcohol consumption.
7. FUTURE WORK
We believe there could have been more results in line with our expectations if we had a bigger sample size and a more homogeneous group. In the current study, half of the participants were psychology students (lab setting) whereas the other half of the participants (home setting) were a very heterogeneous group of students from other studies. The division of participants in controlled lab setting and at home setting also could have influenced our results, for participants might behave differently in a controlled lab than in an environment of their own choice. The differences in results between the lab group and remote group would have been an interesting study on its own if we had a bigger sample size for each group. However, for future execution of this study we would advise to have the same controlled environment for every participant to minimize other influences such as distractions during the experiment.
Also, the non-normal distributions we found for the groups might have been normally distributed if we had a larger sample size. The results for non-normally distributed groups in our study should be interpreted with caution. Therefore, more time should be invested into the study to recruit participants and execute the experiments. In section 6.3 and 6.4 we described the limited amount of 600 trials and three sessions as a possible explanation for the unexpected results for alcohol bias and alcohol consumption. Therefore, we suggest that participants be subjected to more trials in future projects. We recommend more research into the amount of time and trials necessary to make a significant difference in alcohol consumption.
Additionally, for future replication of the study we would advise to select only participants having alcoholic problems (an AUDIT of at least 8 and above) or excessive drinkers (14 or more glasses per week) who are willing to change their alcohol drinking. This way, the training would be more valuable to participants.
With regard to our measurements we recommend evaluating the player enjoyment and motivation surveys in a pilot study and possibly adjust the surveys before the training study. The player enjoyment survey could be evaluated by asking participants how they would rate the experience of the game and if that corresponds to the questions asked in the questionnaire. Also, a short interview with the participants after performing the tasks would be a good addition to further evaluate the experience of the task. For motivation, we gathered data using only a survey. With a short interview we could ask in more depth about participants' motivation for the task. For future work, metrics for motivation in behavior could also be taken into account, such as posture or facial expressions while playing the game.
As for the game, we advise adjusting and evaluating it on gaming experience with a small participant group or expert group as a pilot. It seems that the requirements to pass a level are not clear and a progress bar is needed because the levels are too long for concentration. The levels should also be made more challenging with more diverse gameplay. However, the game should not become too hard, otherwise the player enjoyment could be negatively influenced. Examples of changes are that the running pace of the character in the game becomes faster, the posters pass by faster, changing the direction the mouse runs in, or that the character in the game performs other actions than only dropping objects.
As a further matter, feedback in the game and scoring should be adjusted. Now, it seems the player has a high score and enough hearts to pass the level, but still does not have a sufficient score, and therefore fails the level. For further development, the composition of the scores has to be more in balance with the bonus points a player can receive during the game: now it is impossible to pass a level without enough bonus points.
Furthermore, players should be able to make errors and repeat the errors until they answered the trial correctly without having to redo the level. Otherwise participants in the gaming conditions will be presented more trials in comparison to the participants in the original task and placebo condition.
In addition, the used set of alcohol and non-alcohol images should be enlarged, as the limited set of images makes the task repetitive. Also, images included in the measurement task should be disjoint from the images used in the training task. Otherwise, there is no way to verify whether the training was effective outside of the image sample used in the training. Finally, effort should be put into improving the way the moving trial images are displayed in-game, for some participants had trouble viewing them.
8. CONCLUSION
We expected a higher user experience score for the social game than for the serious game. We measured this with the player enjoyment scale, with which we found no difference between the conditions. However, there were indications that the social game may be more enjoyable than the original task and serious game. This means we can partly confirm our hypothesis: adding social elements to the serious game for alcohol inhibition might increase player enjoyment.
We expected an increased participant motivation for the social game compared to the other conditions. We did not find any significant difference in motivation between the conditions, therefore the hypothesis that a serious game with social elements is more motivating has been rejected. Adding social elements to a serious game for alcohol control will not make the task more motivating.
The last hypothesis concerned the effectiveness of the training by a serious game with added social elements. We measured this on alcohol consumption, alcohol reaction time bias and number of mistakes on the inhibition pretest and posttest. We did not find significant differences in glasses of alcohol consumed, alcohol bias and number of mistakes. Therefore, our hypothesis that the social game training will be more effective on decrease of alcohol consumption, alcohol bias and number of mistakes has been rejected. We can conclude that the effectiveness of an alcohol inhibition training will not improve when using a serious game with social elements.
In general, our study shows that adding social elements to a serious game can yield a higher player enjoyment. This can have an added value for serious applications such as psychological training tasks to make them more enjoyable. Online social elements such as Facebook integration can improve the enjoyment of performing tedious tasks. These tasks can train the alcohol reaction time bias of players. However, more research is needed on this aspect, as well as alcohol consumption behavior. In addition, more research is needed into how to improve participant motivation.
9. ACKNOWLEDGEMENTS
Many thanks go to Tim de Jong for the installment, his instructions, and support with the Lotus system. The Lotus system
was used as an online experiment automation system for participants. Without his work, data gathering would have been problematic for 69 participants both offline and online. I also would like to thank Johanna Quist for supporting and coaching me throughout the study and especially with the analysis. Finally, I would like to thank all the participants who voluntarily took part in the experiment.
10. REFERENCES
[1] Adams, E., 2010. Fundamentals of game design. New Riders.
[2] Alben, L., 1996. Quality of experience: defining the criteria for effective interaction design. interactions 3, 3, 11-15. [3] Allaboutux.org, 2012. User experience definitions. [4] Anderson, P. and Baumberg, B., 2006. Alcohol in Europe: a public health persepctive. A report for the European Commission. Alcohol in Europe: a public health persepctive. A report for the European Commission.
[5] Anderson, P., Møller, L., and Galea, G., 2012. Alcohol in the European Union: consumption, harm and policy approaches. Stylus Pub Llc.
[6] Bonomo, Y.A., Bowes, G., Coffey, C., Carlin, J.B., and Patton, G.C., 2004. Teenage drinking and the onset of alcohol dependence: a cohort study over seven years. Addiction 99, 12, 1520-1528.
[7] Csikszentmihalyi, M. and Csikzentmihaly, M., 1991. Flow: The psychology of optimal experience. HarperPerennial New York.
[8] Dovis, S., Van der Oord, S., Wiers, R.W., and Prins, P.J., 2012. Can motivation normalize working memory and task persistence in children with attention-deficit/hyperactivity disorder? The effects of money and computer-gaming. Journal of abnormal child psychology 40, 5, 669-681.
[9] Georgiadis, A. and Boendermaker, W., 2013. Cheese Ninja: A serious game for alcohol addiction prevention in adolescents.
[10] Heffernan, T., O’Neill, T., Ling, J., Holroyd, S., Bartholomew, J., and Betney, G., 2006. Does excessive alcohol use in teenagers affect their everyday prospective memory? Clinical Effectiveness in Nursing 9, Supplement 3, 0, e302-e307. [11] Houben, K., Havermans, R.C., Nederkoorn, C., and Jansen, A., 2012. Beer à No‐Go: Learning to stop responding to alcohol cues reduces alcohol intake via reduced affective associations rather than increased response inhibition. Addiction 107, 7, 1280-1287.
[12] Houben, K., Nederkoorn, C., Wiers, R.W., and Jansen, A., 2011. Resisting temptation: decreasing alcohol-related affect and drinking behavior by training response inhibition. Drug and alcohol dependence 116, 1, 132-136.
[13] Houben, K. and Wiers, R.W., 2009. Response inhibition moderates the relationship between implicit associations and drinking behavior. Alcoholism: Clinical and Experimental Research 33, 4, 626-633.
[14] Iuppa, N., Borst, T., and Simpson, C., 2012. End-to-End Game Development: Creating Independent Serious Games and Simulations from Start to Finish. Taylor & Francis.
[15] Keitt, T., 2008. It’s Time to Take Games Seriously. Forrester Research, Cambridge.
[16] Law, E.L.-C., Roto, V., Hassenzahl, M., Vermeeren, A.P., and Kort, J., 2009. Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ACM, 719-728.
[17] Liu, T.-Y. and Chu, Y.-L., 2010. Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation. Computers & Education 55, 2, 630-643.
[18] Mulder, M., 2013. Zware drinkers 2012. In Volksgezondheid Toekomst Verkenning, Nationale Atlas Volksgezondheid RIVM, Bilthoven.
[19] Nutt, D.J. and Rehm, J., 2014. Doing it by numbers: A simple approach to reducing the harms of alcohol. Journal of Psychopharmacology 28, 1, 3-7.
[20] Saunders, J.B., Aasland, O.G., Babor, T.F., de la Fuente, J.R., and Grant, M., 1993. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption‐ II. Addiction 88, 6, 791-804.
[21] Susi, T., Johannesson, M., and Backlund, P., 2007. Serious games: An overview.
[22] Sweetser, P. and Wyeth, P., 2005. GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE) 3, 3, 3-3.
[23] Vallerand, R.J., Pelletier, L.G., Blais, M.R., Briere, N.M., Senecal, C., and Vallieres, E.F., 1992. The Academic Motivation Scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and psychological measurement 52, 4, 1003-1017.
[24] van Deursen, D.S., Salemink, E., Smit, F., Kramer, J., and Wiers, R.W., 2013. Web-based cognitive bias modification for problem drinkers: protocol of a randomised controlled trial with a 2x2x2 factorial design. BMC public health 13, 1, 674. [25] Verbruggen, F. and Logan, G.D., 2008. Automatic and controlled response inhibition: associative learning in the go/no-go and stop-signal paradigms. Journal of Experimental Psychology: General 137, 4, 649.
[26] Wiers, R.W., Bartholow, B.D., van den Wildenberg, E., Thush, C., Engels, R.C., Sher, K.J., Grenard, J., Ames, S.L., and Stacy, A.W., 2007. Automatic and controlled processes and the development of addictive behaviors in adolescents: a review and a model. Pharmacology Biochemistry and Behavior 86, 2, 263-283. [27] Wiers, R.W., Hoogeveen, K.J., Sergeant, J.A., and Gunning, W.B., 1997. High‐and low‐dose alcohol‐related expectancies and the differential associations with drinking in male and female adolescents and young adults. Addiction 92, 7, 871-888.
[28] Williamson, D., Squire, K., Halverson, R., and Gee, J., 2005. Video games and the future of learning. Phi Delta Kappan 87, 2, 104-111.
[29] Yee, N., 2006. Motivations for play in online games. CyberPsychology & Behavior 9, 6, 772-775.
[30] Zeigler, D.W., Wang, C.C., Yoast, R.A., Dickinson, B.D., McCaffree, M.A., Robinowitz, C.B., and Sterling, M.L., 2005. The neurocognitive effects of alcohol on adolescents and college students. Preventive medicine 40, 1, 23-32.
APPENDIX A – Player enjoyment
questionnaire
Geef aan in hoeverre je het met de volgende stellingen eens bent. (helemaal oneens, oneens, neutraal, eens, helemaal eens)
De opdrachten waren voldoende uitdagend voor mij
De objecten waarop ik moest letten waren moeilijk uit elkaar te halen
Ik had het idee dat ik steeds beter in de opdrachten werd
Ik vond de beloningen voor mijn prestaties bij de opdrachten goed
Ik vond de beloningen voor mijn prestaties bij de opdrachten te hoog
Ik vond de beloningen voor mijn prestaties bij de opdrachten te laag
Ik vond dat er geen overbodige elementen in de opdrachten zaten
De opdrachten vereisten mijn volledige concentratie
Ik hoefde me niet erg te concentreren om de opdrachten uit te voeren
Ik werd makkelijk afgeleid van de opdrachten
Ik vond de andere visuele elementen op het scherm afleidend
Ik had het idee dat ik volledige controle had tijdens de taak
Ik had het gevoel dat mijn handelingen invloed hadden op de rest van de opdrachten
De besturing van de opdrachten was gemakkelijk
Het was voor mij duidelijk wat van mij verwacht werd tijdens de taak
Voor mij was het duidelijk hoe ik presteerde tijdens de opdracht
Ik vond alle informatie die op het scherm getoond werd tijdens de opdrachten duidelijk
Ik ging helemaal op in de taak
Ik vergat de tijd tijdens het uitvoeren van de opdrachten
Ik was me tijdens het uitvoeren van de opdrachten niet bewust van mijn omgeving
Ik zou sociale elementen (bijv. sociale interactie met andere spelers en competitie) bij de opdrachten een goede toevoeging vinden
Ik vind de sociale elementen bij de opdrachten een goede toevoeging
APPENDIX B – Motivation questionnaire
Geef aan in hoeverre je het met de volgende stellingen eens bent. (helemaal oneens, oneens, neutraal, eens, helemaal eens) Ik werd gemotiveerd om mijn best te doen
De afbeeldingen trokken mijn aandacht
De manier waarop de opdrachten werden gepresenteerd hielp mij om mijn aandacht erbij te houden
Ik kon me goed concentreren op de opdrachten
Ik vond de training nuttig
De training was de moeite waard
Ik denk dat ik iets aan de training heb in het dagelijks leven
Ik vond de opdrachten makkelijk om te doen
Ik heb het gevoel dat ik de opdrachten goed heb uitgevoerd
Ik heb het gevoel dat ik steeds beter werd tijdens het uitvoeren van de opdrachten
Ik heb de opdrachten met plezier uitgevoerd
Ik had langer willen doorgaan met de opdrachten
