1
Gamification for
education
A study on the effects of gamification, on student motivation and engagement
Burcu Akcay 10054219
University of Amsterdam
Bachelor thesis: Entertainment Communication Supervisor: Monique Timmers
13-02-2017 Word count:
2 CONTENTS Abstract 1. Introduction 2. Conceptual framework 2.1 Research question 2.2 Conceptual model 2.3 Dependent variables 2.3.1 Student motivation 2.3.2 Engagement 2.4 Independent variables 2.4.1 Leaderboard 2.4.2 Badges 2.4.3 Feedback
2.5 Research questions based on conceptual model 3. Method 3.1 Participants 3.2 Procedure 3.3 Material 3.4 Instruments/ measurements 4. Results
5. Conclusion & Disussion Literature References
Appendix A : Questionnaire Appendix B: Vignettes
3 ABSTRACT
The use of gamification has been linked to the increase of engagement and (intrinsic) motivation by several scholars. Gamification is the use of game elements in non-game contexts and the trend of creating game like experiences seems to be everywhere nowadays. Gamification gained a lot of popularity in different fields such as marketing and business. It now also continues to grow in popularity in the field of education as a method to promote student engagement and the intrinsic motivation of students. This study focusses on how game elements affect student motivation and engagement in a statistics class. Two conditions of a vignette that described a statistics class were read by two groups of participants and a questionnaire on intrinsic motivation and student engagement was given. The results show that there are no significant effects found for the impact of these game elements on the intrinsic motivation of students and the student engagement.
4 1. INTRODUCTION TO THE EMPIRICAL STUDY
During the past few years, the concept of ‘gamification’ has emerged in the education literature. Gamification is the use of game elements in a non-game context (Deterding,
Siacart, Nacke, O’Hara & Dixon, 2011) often to increase motivation and/or engagement (Kuo & Chang, 2016; de Rocha Seixas, Gomes & Melo Filho, 2016; de Macros, Garcia-Lopez & Carcia-Cabot, 2016; Simoes, Redondo & Vilas, 2013). An investigation of the literature has yielded that despite the popularity of gamification, scientific research has not found a consensus on the effectiveness of gamification motivation and engagement. The existing literature about the effectiveness of gamification in education show contradicting results. While there are studies that do report positive findings regarding to student motivation (Barata, Gamma, Jorge & Goncalves, 2013; Domínguez, Saenz-Navarrete,
De-Marcos, Fernández-Sanz, Pagés & Martínez-HerráIz, 2013), there are also studies that do not exhibit these same results and are more critical about the use of gamification in education (Dominiquez et al., 2013; Hanus & Fox, 2015). Some scholars state that the use of game elements might even backfire and have a negative impact on student motivation in the long run (Hanus & Fox, 2015).
It is important that more research is done in order to gain more insight in the effects of gamification in education on student motivation and engagement. This is important for
several reasons. First of all, if the effectiveness can be empirically proven, gamification could be a great cost effective alternative to creating costly games for educational purposes.
Secondly, if proven effective, gamification can be a great tool to upgrade the quality of the existing educational system. However, if a lack of empirical evidence shows that
gamification’s effect on student motivation and engagement is overrated, it is pointless to develop gamification in education. If it has a negative impact on student motivation and engagement, developing gamification in educational settings could even be unethical.
Within this context, this thesis aims to determine the effectiveness of several gamification elements on student motivation and engagement.
5 2.CONCEPTUAL FRAMEWORK
2.1 Research question
Playing a videogame is often experienced as fun, engaging and often elicits motivation to keep on playing. While games are often considered fun and motivating, studying is often considered boring (McGonigal, 2011). But what if game elements could be implemented in an educational setting and make it more fun, motivating and engaging? That is the aim of the use of gamification in education. However is gamification also an effective tool in educational settings? To gain more knowledge about this context, I propose the following research question:
RQ: What is the effect of gamification elements on intrinsic student motivation and student
engagement?
2.2 Conceptual model
To answer this research question, the following conceptual model has been developed:
Fig 1: Conceptual model gamification in education
In the above figure, gamification elements are shown to consist of ‘leaderboard’, ‘badges’ and ‘feedback’. These elements are the independent variables. Shown below are the elements student motivation and student engagement. These are the dependent variables. The aim of
GAME ELEMENTS
leaderboard Badges Feedback
Intrinsic
motivation Engagement
6 this research is to measure the extent of the effect of the gamification elements on intrinsic motivation and student engagement. Additionally it is gauged whether student engagement influences the intrinsic motivation of students.
2.3 Dependent variables
In the following section the dependent variables of the conceptual model will be further explained.
2.3.1 Intrinsic motivation of student’s
Many scholars explain the link between gamification and motivation in education with the self-determination theory (De-Macros, Dominguez, Saenz-de-Navarete & Pages, 2014; Fotaris, Mastoras, Leinfellner, & Rosunally, 2016; Dominiquez et al., 2013, Wilson,
Calongne & Henderson, 2016; Kuo & Chang, 2016). Drawing from the field of psychology, this theory by Ryan and Deci (2000) has two primary constructs, which are the intrinsic and extrinsic motivation. Intrinsic motivation stands for doing something out of interest or enjoyment and extrinsic motivation is about doing something only for the outcome/reward. And these two types motivations are fluid, it presents a sort of continuum form one to the other (de-Macros et al., 2014)
Ideally, teachers want students to learn out of their own interest or enjoyment, that the students are intrinsically motivated to learn. This is because when students are intrinsically motivated they are more engaged and retain information better (Hanus & Fox, 2015; Ryan & Deci, 2000).Often game elements are perceived as an extrinsic motivator, but these game elements can be seen as a tool to promote intrinsic motivation (de-Macros et al., 2014). This can again be be explained by the Self determination theory. This holistic theory,
discusses three psychological human needs: autonomy, competence and relatedness. It states that if these needs are satisfied it results in development and intrinsic motivation (Ryan & Deci, 2000). And most game elements are able to satisfy these psychological human needs. For example Badges satisfy the need for accomplishment (competence) , (positive) feedback would both lead to a satisfaction of competence and achievement. In short, if an extrinsic motivator is found to be pleasurable and meaningful , it can be adopted by people as though it were intrinsic. Which means that although game elements such as badges,
points and leaderboards are often perceived as external motivators in the first place ,they can lead to intrinsic motivation (Wang, Khoo, Liu & Divaharan, 2008).
7 Yet there are also critical voices about the use of badges and rewards in education for the increase of motivation (Deci et al., 2001; Hanus & Fox, 2015). Interestingly those scholars who are critical about the badges system use the same theory to explain this and state that when a student is already intrinsically motivated and genuinely interested in a class, the use of badges might decrease the intrinsic motivation. In this case a shift will occur from intrinsic motivation (doing something because you enjoy it) to extrinsic motivation (doing it for the badges), the level of control affects the level of autonomy of the student, which in turn can decrease the level of intrinsic motivation (Deci, Koestner & Ryan, 1999). This is best illustrated in an experiment that is carried out by Deci (1971). In this particular experiment there are two groups of students who solve a puzzle. One group of students was paid for each solved puzzle, the other was not. When the experiment ended, the paid group stopped solving puzzles and the group who was not paid kept on solving the puzzle, even though the
experiment was over. The group that kept on solving the puzzles was perceived as interested. Deci (1971) suggested that the group who was paid to solve the puzzle were perhaps also interested in solving the puzzles but the extrinsic reward, in this case the money, had reduced that interest. The study of Hanus and Fox (2015) about gamification in education is a good example of the badges system impacting the intrinsic motivation of student’s negatively
Previous research to the effectiveness of gamification in education on the motivation of students has shown that at the use of gamification in education increases the motivation of students (Barata et al., 2013; Dominiquez et al., 2013 and Donovan, Gain & Marais, 2013). In the study of Barata et al. (2014), a course was gamified by implementing game elements such as leaderboards, points and badges into the course. This study states that students in the gamified course were more motivated. This increase in motivation was noticeable by the increase in the number of downloads of lecture slides and the increase in lecture attendance. The study of Dominiquez et al. (2013) also concludes that students who were in the gamified group were more motivated than the control group.
It is important to note that in the above studies, researchers have not distinguished between intrinsic and extrinsic motivation. However, SDT’s assessment that intrinsic and extrinsic motivation are codependent and thus are to be expected concurrently makes that the results from the above studies warrant expectation that intrinsic motivation will be affect by game elements such as leaderboards, badges and feedback.This enables gamification to motivate intrinsically.
8 In this study, I choose to propose research questions based on the part of the literature that shows evidence for the effectiveness of game elements on student motivation. Although the study of Hanus and Fox (2015) state that the negative effect of the game element badges on intrinsic motivation only occurs on the long run. This study is not a longitudinal study. Based on the literature it is thus more likely that badges will promote the intrinsic motivation.
2.3.2 Engagement
Next to intrinsic motivation, engagement is also an important concept when discussing
gamification in education. Intrinsic motivation and engagement are seen as intertwined. When a student is engaged, it is expected that the student is also intrinsically motivated, thus
if student engagement increases, the intrinsic motivation of students is also expected to increase (Deci & Ryan, 2000)
Engagement is thought of as an important factor of student success and it is believed that gamification can potentially improve student engagement. The game elements are
considered an important part of the development of knowledge and engagement of students in education (Ellis, Heppell, Kirriemuit, Krotoski & MacFarlane, 2006).
Literature provides conflicting results about the effect of gamification on student motivation. Yet the literature displays consistent results when it comes to the effects of gamification in education on student engagement. The majority of studies concerning gamification in education seem to provide evidence to support the claim that the use of gamification in education is beneficial for the student engagement (da Rocha Seixas, Gomes & Melo Filho, 2016; Goehle, 2013; Ibanez et al, 2014; Rose, Meara, Gerhardt & Williams, 2016).
2.4 Independent variables
Game elements are considered to be basic components and features of games; such as points, levels, badges (Wilson et al., 2015). Below there is a description of three commonly used game elements and an explanation to how they affect engagement and/or motivation.
The reason for the choice of these three game elements – leaderboard, badges and feedback – is the following. The most common game elements that are used in the literature to create a gamified course are: points, badges and leaderboards (PBL) (Antin & Churchill, 2011; Narasimhan, Chiricescu, & Vasudevan, 2011; Werbach & Hunter, 2012). Two of these three
9 game elements are selected to use in the vignette. The reason for excluding the element
‘points’ is the similarity between points and badges, as they are both used as reward. Instead of points, the game element feedback is used as a third game element in the vignette.
Although feedback is not in the top three of most used game elements in gamified courses, it still is a commonly used game element in gamification. And the purpose of feedback is not to reward, which clearly distinguishes it from the other two elements.
2.4.1 Leaderboard.
In the context of education, leaderboards will rank the results of students and visually display the ranked results. Leaderboards promote competition, which is believed to encourage the intrinsic motivation and engagement of students (Dominiquez et al., 2013; Hanus & Fox, 2015; Kapp, 2012). One study that used leaderboards and competition to teach course concepts, indicated an increase in engagement (Bellotti et al., 2013) The leaderboard is considered as an effective game element that could be implemented in educational
environments and is commonly included as variable in related research (Barata et al., 2013; Berklingen & Thomas, 2013;Dominiguez et al., 2013; Fotaris et al., 2016; Hanus & Fox, 2015; Ibanez et al., 2014; Kuo & Chang, 2016; de-Macros et al., 2014; O’Donnovan et al., 2013)
2.4.2 Badges.
There are two options regarding to the effects of badges on motivation and engagement. Proponents state that badges initially trigger the extrinsic motivation, but can be used as a tool to promote intrinsic motivation (Dertringen, 2011). Since a badge is often given as a reward, it is a visible, and sometimes tangible, acknowledgement of an accomplishment. Therefore it satisfies the desire for accomplishment (Richter, Raban & Rafaeli, 2015). On the other hand, based on the Cognitive Evaluation Theory by Deci and Ryan (1985), which is explained earlier in the literature review previous to this empirical research, which indicates that the use of a badge system might backfire if students are already interested in the class/course. The reasoning behind this is that giving rewards for something a student would want to do voluntarily, might feel as an attempt to control the free will, thus affecting the autonomy of the individual, decreasing the intrinsic motivation instead of increasing it (Deci & Ryan, 1985). Hanus and Fox (2015) have confirmed these results in an experimental study
10 2.4.3 Feedback
Direct feedback, like in games can be more effective than the feedback that is given in class, which is often restricted. Feedback in class is often given to one student at a time, which in most cases makes it impossible to give feedback to every student individually. Feedback through grading can be given to one student at a time, but this does not happen directly and it is also very time consuming for the teacher (Kapp, 2012) . Direct feedback like in games is often given in real time and allow the player of a game to try again. When this game element, feedback is implemented in educational settings it will allow students the freedom to fail without fear and this will increase the student engagement (Lee & Hammer, 2011).
Furthermore positive feedback encourages personal growth, and gives students the feeling that they are more qualified. This can be motivating (Lee & Hammer, 2011; Nah, et al., 2014; Zichermann & Cunningham, 2011; Kapp, 2012; Simões, Díaz, & Fernández, 2013)
2.5 RESEARCH QUESTIONS
The following research questions are based on the literature review above.
Q1: Does the implementation of game elements in a traditional classroom, increase the intrinsic student motivation?
Q1a: Does the use of a leaderboard increase the intrinsic student motivation?
Q1b: Does the use of badges/tangible rewards increase the intrinsic student motivation? Q1c: Does the use of feedback increases intrinsic student motivation?
Q2: Does the implementation of game elements in a traditional classroom, increase the student engagement?
Q2a: Does the use of a leaderboard increase the student engagement
Q2b: Does the use of badges / tangible rewards increase the student engagement? Q2c: Does the use of feedback increases student engagement?
11 3. METHOD
In order to answers the research questions in section 2.5, I have conducted an online experiment with a ‘between subjects’ design. The online experiment consisted of two conditions - a condition with a vignette, which contained game elements and a control
condition with a vignette without game elements. Vignettes are short scenarios that describe a hypothetical situation, participants are asked to place themselves in the situation. Participants were randomly assigned to one of the two conditions and were asked to complete a
questionnaire afterwards that measured the intrinsic motivation and student engagement. The total questionnaire consisted of 61 items measuring demographics, intrinsic motivation and student engagement.
3.1 Participants
All participants selected for the online experiment are university students. The participants (N = 60) were selected by means of convenience sampling. The questionnaire and vignette used for the online experiment were written in Dutch. The hypothetical situation featuring in the vignettes was a statistics class. The link to the online experiment was posted on the following two Facebook groups aimed at Dutch speaking university students: ‘Athena Studies’ and ‘Meer en Vaart 358’ (the Facebook group of a student apartment). The link was also send through Whatsapp to relatives and friends who are students All participants were asked to share the link of the online-experiment with friends they know who are students. More than half of the participants (n=39, 65.0 %) participated through the link on Facebook. The other remaining (n=21, 35.0 %) participated through the link they received on Whatsapp. Average age of the respondents was 23.5 (SD=2.5). 26.7% of participants were male (N = 16) and 73.4% were female (N = 44). The majority of the participants were wo students (n=52, 86.7%) and the remaining participants were hbo students (n=8, 13.3%)
3.1 Procedure
The online-experiment was available online on the platform Qualtrics between December 5, 2015 and December 11, 2016. This enabled the participants to participate independently and in a private setting. Participants were first asked to agree with the informed consent. The participants were randomly assigned to one of the two conditions of the experiment (the
12 condition with the vignette that contained game elements and the control condition with without). First the participants were first asked to answer questions that addressed personal information such as gender, age and the level of education. Next, each respondent was requested to read the vignette and try to imagine themselves in the situation that was
described in the vignette they were about to read. After being presented with the vignette, the participants were redirected to the last part of the experiment, a questionnaire with questions concerning the intrinsic motivation and engagement of the participants.
3.1 Material
The two versions of the vignette that were used in this study are displayed in appendix B. Both of the vignettes describe a situation in a statistics class. That first starts with the teacher handing back grades of an earlier assignment, then an in-class assignment and at last the students get a take home assignment. In version B.1, the situation with the game elements leaderboard, badges and feedback is presented. The reasoning for the choice of these three game elements is explained above in the conceptual model (2.4). The leaderboard is presented at the beginning of the vignette when the teacher is handing back the grades of the earlier assignment, in the vignette with game elements the teacher reveals the grades by using a leaderboard. The second game element are badges, in the vignette with game elements the badges can be earned with the in class assignment. And the last game element, feedback is implemented in the vignette with the take home assignment.
The vignette that was presented to the control group B.2 describes the same class situation, but without the game elements that are presented in version B.1. Instead, the grades in this condition were not presented on a leaderboard. There were no badges to earn by making the in-class assignment and there was no direct feedback on the take home assignment from the lecturer. These choices were made to emulate a traditional classroom setting.
3.4 Instruments/ Measurements
Intrinsic motivation
Intrinsic motivation was measured by adapting items from the intrinsic motivation inventory (Ryan, Koestner, & Deci, 1991). This inventory was build out of items that measure interest and enjoyment, effort and importance, pressure and tension and value and usefulness. Included items were formulated like: “I thought this was a boring activity”, “I put a lot of effort into this”, “I felt pressured while doing these”, “I believe doing this activity could be beneficial to me”. These items were asked to assess how the respondents experienced the
13 class they have had read about in the vignette in general. Each question was adjusted to the vignette to gain knowledge about the class in general (overall intrinsic motivation) and to gain knowledge about the effect of each game element (leaderboard, badges and feedback) on intrinsic motivation (See appendix A). For example: “I felt pressured during class”, “I felt pressured when the grades were announced”, “I felt pressured while making the in-class assignment”, “I felt pressured while making the take home assignment”. All items were measured on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree).
Engagement
To measure student engagement, 17 items were constructed. Student engagement was mainly measured with items based on the national survey of student engagement (NSSE) (Survey Instrument, (n.d.). Items that were relevant for this study were selected from the NSSE survey and adjusted to the vignette to gain knowledge about the class in general (overall
engagement) and to gain knowledge about the effect of each game element
(leaderboard, badges and feedback) on the student engagement (See appendix A). For example “I summarize what I learned in class”, “I will ask question in class.” Finally, two other items were added to the survey. These two items were items that give a general idea of engagement. For example ‘’I was enthusiastic about this class’. All items were measured on a 7-point Likert scale from 1 (definitely not) to 7 (definitely yes) In the original NSSE a 4-point likert scale is used, yet for this thesis a 7-4-point Likert scale is chosen to match the scale of the questions that measure intrinsic motivation.
4.RESULTS
intrinsic motivation scales
First, all scales for intrinisc motivation were tested fot its reliability. The twelve items that together created the scale for overall intrinsic motivation was tested and had a Cronbach’s Alpha score of α=.67. It is therefore not a reliable scale. Based on the corrected item total correlation and the Cronbach's alpha if item deleted, three items could be removed to increase the reliability of the scale. The following three items “Ik voelde me niet nerveus tijdens de les’’(r=.04), “ik voelde me onder druk gezet tijdens de les”(r=-.08) and “Ik was relaxed tijdens de les” (r=.03) were deleted. The Cronbach's if items deleted does meet the standard of a reliable scale α=.83
14 The items that together created the scale for the effect of leaderboard on intrinsic motivation was tested for its reliability. The scale consisted of five items and had a
Cronbach’s Alpha score of .60, this is not a reliable scale. This scale could be improved if the item “ Ik vond de bekendmaking van de cijfers belangrijk (r=.13) is deleted. This resulted in a new Cronbach’s Alpha of α=.69, which meets the standard of a moderately reliable scale.
The scale for the effect of badges on intrinsic motivation was tested for its reliability. The scale consisted of eleven items and it had a Cronbach’s Alpha of α=.73 and therefore meets the standard of a reliable scale
The items that together created the scale for the effect of feedback on intrinsic motivation was tested for its reliability. The scale consisted of ten items and it has a
Cronbach’s Alpha of .68 this is a moderately reliable scale. This scale could be improved by removing the item ” Ik voelde me niet nerveus tijdens het maken van de opdracht” (r=.07) is deleted this resulted in a new Cronbach’s Alpha of α=.71, which meets the standard of a reliable scale.
Student engagement scales
Furthermore all scales for student engagement were tested for its reliability.
The ten items that together created the scale for overall engagement of students was tested for its reliability and had a Cronbach’s Alpha score of α=.86 it is therefore a reliable scale. The two items that togehter created the sale for the effect of leaderboard on student engagement was tested for its reliability and had and Cronbach’s Alpha score of α=.76 . The scale for the effect of badges on student engagement was tested for its reliability. The scale consisted of five items and it had a Cronbach’s Alpha of α=.65. This scale could not be further improved and therefore this scale is moderately reliable. The last scale that was tested for its reliability was the scale for the effect of feedback on student engagment. This scale consisted out of five items and showed an Cronbach’s Alpha score of α=.52. This scale could be further improved by deleting the following items “ De kans is er dat ik naar de volgende les kom zonder de thuisopdracht gemaakt te hebben”( r=.12), “Ik ga samenwerken met mijn medestudenten tijdens het maken van de thuisopdracht”(r=.31), “Ik vroeg mijn
medestudenten om hulp toen ik de opdracht niet begreep” (r=.17). If these items are deleted it becomes a moderately reliable scale α=.69.
T-test & descriptives
15
Descriptive statistics (Means, Standard deviations, Minimum, Maximum) of intrinsic motivation and engagement scales by condition, with independent t-tests for differences in means between the conditions
There is a significant difference between the control (M = 4.62, SD = 0.69) and game
elements groups (M = 5.06, SD = 0.92) in overall motivation, t(58) = 2.12, p = .038; Cohen’s
d = 0.54. The conditions have a medium effect on general motivation. The motivation in the
control group is generally lower. No significant differences are found between the means of the condition for the other intrinsic motivation and engagement scales. In this sample we do see higher means for all intrinsic motivation and engagement scores. The results of the t-tests are not controlled for confounding variables. Looking at the variances between the conditions, there is a significant difference between game elements (SD = 1.08) and controls (SD = 0.75) for badges engagement, F = 4.17, p = .046. With game elements, the variation is greater than for the control group. The variances on the other intrinsic motivation and engagement-scales were not significantly unequal.
Randomization check
In order to see if the conditions are randomized, the age and gender characteristics of both groups are compared with a t-test and a chi-square test. The mean age in control group (M = 23.64, SD = 2.29) and game elements group (M = 23.33, SD = 2.68) is not significantly different, t(58) = -0.47, p = .64. The percentage of females in the control group (n = 28, 84.8%) and in the game elements group (n = 16, 59.3%) did not differ significantly, χ2 (1) = 3.75, p = .053; ϕ = .29. However, there is medium effect, therefore gender will be used as a control variable
Condition t-test for difference in
means
game elements (n = 27) controls (n = 33)
t-valu e
df p-value
(2-tailed)
M SD Min Max M SD Min Ma x Overall_motivation 5.06 .92 1.56 6.11 4.62 .69 2.89 5.67 2.12 58 .038 Leaderboard motivation 3.56 1.34 1.00 6.33 3.30 1.46 1.00 7.00 0.69 58 .49 Badges motivation 4.65 .83 2.82 6.27 4.60 .65 3.09 5.73 0.26 58 .80 Feedback motivation 4.74 .80 3.11 6.22 4.60 .70 2.67 5.89 0.75 58 .46 Overall engagement 4.60 1.06 2.10 6.30 4.57 .88 1.60 5.80 0.10 57 .92 Leaderboard engagement 4.80 1.37 1.50 6.50 4.50 1.10 2.00 6.00 0.92 57 .36 Badges engagement 4.78 1.08 1.60 6.20 4.58 .75 2.40 6.40 0.80 45 .1 .43 Feedback engagment 4.24 1.43 1.00 6.50 4.08 1.28 1.50 6.50 0.47 58 .64
16 Correlations
Table 1.2
Pearson correlations between the intrinsic motivation and student engagement-scales and age for control (above diagonal) and game element group (below diagonal)
intrinsic motivation engagement
Age gene
ral
leaderboard badges feedbac k genera l leaderboar d badges feedba ck Age 1.00 .08 -.42* -.08 -.24 -.21 -.09 -.10 .05 Overall motivation -.39* 1.00 .09 .71** .55** .61** .49** .52** .38* Leaderboard motivation -.37 -.22 1.00 .45** .45** .12 -.04 -.03 -.36* Badges motivation -.29 .72** .22 1.00 .50** .55** .43* .53** .12 Feedback motivation -.36 .57** -.24 .18 1.00 .63** .41* .35 .38* Overall engagement -.35 .71** .08 .63** .48* 1.00 .86** .78** .55** Leaderboard engagement -.28 .61** -.05 .58** .21 .82** 1.00 .73** .52** Badges engagement -.29 .56** .05 .60** .33 .88** .85** 1.00 .30 Feedback engagement -.35 .47* -.26 .11 .68** .68** .56** .56** 1.00 **. p < .01 (2-tailed) *. p < .05 (2-tailed)
In table 1.2 the correlations between the dependent variables and age are presented, separately for the control and the game element groups. Below the diagonal, the correlations for the game element group are presented, while above the correlations of the control group are shown. In this table, the relationship between the different scales can be inspected per group. For intrinsic motivation, more significant correlations are found for the control group than for the game elements group. Significant correlations can be found for the relation between badges and leaderboard (r = .45, p < .01), feedback and leaderboard (r = .45, p < .01) and badges and feedback (r = .50, p < .01) for the control group. For the game element group, these correlations are not significant (all r < .25) or even negative. For engagement, similar positive correlations are found in both conditions. Age has a negative relationship with general intrinsic motivation in the game elements group, r = -.39, p < .05. The older the respondent, the less he or she is generally intrinsically motivated. In the control group no relationship is found between age and intrinsic overall motivation, r = .08, ns. In this group, a significant negative relationship of age is found with intrinsic leaderboard motivation, r = -.42, p < .05. The older the respondent, the lower a score on the leaderboard motivation scale is expected.
17 Regression analyses
Table 2.1
Linear multiple regression analysis to investigate the effect of condition on the 4 scales of intrinsic motivation (overall, leaderboard, badges and feedback) as dependent variables
intrinsic motivation
overall leaderboard badges feedback
b* p-value b* p-value b* p-value b* p-value (Constant) 5.91 <.001 9.45 <.001 6.43 .00 6.23 <.001 Gender 0.04 0.86 -0.42 .29 -0.23 .30 0.25 .25 Age -0.06 0.18 -0.23 <.01 -0.06 .13 -0.09 .02 Condition 0.44 0.05 0.08 .83 -0.03 .89 0.18 .35 R2 0.103 0.178 0.058 0.119 F 2.14 0.11 4.05 0.01 1.15 0.34 2.53 0.07
In order to answer the research questions Q1 (a, b, c), a regression analysis was performed. Four separate regressions give answers to respectively Q1, Q1a, Q1b and Q1c, by looking at the effect of condition. The reference group is the control group, so a positive effect of condition means a positive effect of the game elements condition on the intrinsic motivation. Gender and age were added as control variables, because age has influence on some of the intrinsic motivation and engagement scales (see table 1.2). Gender was added because the percentage of women in the control group was much higher. The reference category is female. As can be seen from tabel 2.1, no effects were significant, so all questions can be answered with “no”, which means that game implementation of game elements does not influence the intrinsic motivations of students. The effect of condition predicting overall intrinsic
motivation is not significant, in contrast to the t-test. The explained variance in the regression predicting leaderboard is highest (17.8% explained) and significant, F(3,56) = 4.05, p = .011. This is due to the significant effect of age, b* = -0.23, p < .01.
18 Table 2.2
Linear multiple regression analysis to investigate the effect of condition on the 4 scales of engagement (overall, leaderboard, badges and feedback) as dependent variables
engagement
overall leaderboard badges feedback
b* p-value b* p-value b* p-value b* p-value (Constant) 7.56 <.001 7.61 <.001 6.83 0.00 6.35 <.001 Gender -0.20 0.49 -0.42 0.27 -0.22 0.44 -0.11 0.80 Age -0.11 0.03 -0.10 0.13 -0.08 0.11 -0.09 0.23 Condition -0.06 0.82 0.16 0.63 0.12 0.63 0.11 0.76 R2 0.089 0.079 0.065 0.03 F 1.78 .16 1.45 .24 1.28 .29 .58 .63
In order to give answers to the research questions Q2 (a, b, c), a regression analysis was performed. Four separate regressions give answers to respectively Q1, Q1a, Q1b and Q1c, by looking at the effect of condition. Results are presented in table 2.2 No effects were
significant, which means that game implementation of game elements does not influence the engagement of students.
Not any of the regression model we significant in prediction one of the engagement scales (all
ps > .15).
Table 2.3
Linear multiple regression analysis to investigate the effect of overall engagement on overall intrinsic motivation as dependent variable
Overall intrinsic motivation
b* p-value (Constant) 1.643 .136 Gender .168 .374 Age .005 .881 Condition .445 .010 Overall engagement .562 <.001 R2 0.485 F 12.71 <.001
To address Q3, a regression analysis is performed with overall intrinsic motivation as
dependent variable and overall engagement as predictor as independent variable. The analysis is also controlled for gender, age and condition. The model is significant, F (4,54) = 12.71, p < .001, R2 = .49. In total, 48.5% of the variance of overall intrinsic motivation is explained. As can be seen from the correlation table (table 1.2), overall motivations has a significant positive relation with overall engagement, r = .61, p < .01. This effect is large (r > .50). The effect in the regression model is also positive and significant, b* = .56, p < .001. This means
19 that the Q3 “Does the intrinsic motivation increase if the student engagement increases?” is confirmed. It is remarkable that in the regression, the effect of condition is significant, b* = .45, p = .010. The effect is positive, so the game element condition has a positive effect on the overall intrinsic motivation, when is controlled for the difference in the overall engagement
5. CONCLUSION & DISCUSSION
The aim of this study was to contribute to the literature on the effectiveness of gamification on intrinsic motivation and student engagement in educational settings The results of the online experiment will be discussed in the light of the research questions that were set up in this study. At first after an independent t-test, it looked like that the participants who were assigned to the condition with game elements felt more intrinsically motivated than the control group. The significant difference that was found was for the overall intrinsic motivation, looking at the class as a whole. But further investigation of the results with an regression showed that there was no significant result for the effect of game elements on the overall intrinsic motivation of students. Therefore the first RQ (Q1) : Does the
implementation of game elements in a traditional classroom, increase the intrinsic student motivation?” could be answered with a, no the implementation of game elements in a
traditional classroom do not increase the intrinsic motivation of students. These results is not in line with the discussed literature.
For the remaining Q1, a, b,c and Q2, Q2,a,b,c. were no significant results found either after the regression analysis. But there was an interesting significant effect found that enables to answer the last RQ : Q3: Does the intrinsic motivation increase if the student engagement
increases? With an yes. The remarkable result of the significant effect of condition when
controlled for engagement, shows that engagement plays a role in the effect of game elements on intrinsic motivation. It is possible there is a mediating effect of engagement and this could be investigated in further research.
The results of the correlation analysis show that the relationship between the intrinsic motivation and the different game elements is stronger for the control group than for the condition group. A possible explanation for this can be that without any interference of game elements, it can be that participants who are already intrinsically motivated to study or who are already motivated by themselves for a class, show more intrinsic motivation for each
20 aspect of the class. Because the impact of each game element was measured by how
intrinsically motivated each student felt to participate in a certain aspect of the class (receiving grades, carrying out the in-class assignment and carrying the the take-home assignment). Participants who were not motivated from the start will in the control group, without the interference of game elements, be likely continue to be less intrinsically motivated throughout the class. On the other hand, by implementing game elements in a classroom, it can be that some students who were not intrinsically motivated from the start show different levels of intrinsic motivation for the different aspects of the class (receiving grades, carrying out the in-class assignment and carrying the the take-home assignment) thanks to the game elements. And some game elements have more effect than others this can explain why in the condition group the relationship between intrinsic motivation and game elements not as strong as the control group. Future studies should more specifically investigate different conditions under which gamification is effective for individual participants
The majority of the results are not in line with the discussed literature. And it is important to note that this study is limited. First it is important to note that the findings of this study are limited to the game elements leaderboard, badges and feedback. For this reason, these results are not indicative for all gamification systems. Another important limitation of this study is the fact that a vignette was used instead of a real-life situation. Perhaps it was not easy for all participants to imagine the situation in the vignette and the outcomes of the study could be different if the described situation of the vignette was performed in a real-life situation. Furthermore not all measurements were reliable. For this study existing scales for intrinsic motivation and student engagement were used and adjusted to the vignette. However analysis showed that not all scales were reliable. The scale for the effect of leaderboard on intrinsic motivation and the scales for the effect of badges and feedback on student engagement could be improved in future research by using other items .
21 The results of the correlation analysis show that the relationship between the intrinsic
motivation and the different game elements is stronger for the control group than for the condition group. A possible explanation for this can be that without any interference of game elements, it can be that participants who are already intrinsically motivated to study or who are already motivated by themselves for a class, show more intrinsic motivation for each aspect of the class. Because the impact of each game element was measured by how intrinsically motivated each student felt to participate in a certain aspect of the class (receiving grades, carrying out the in-class assignment and carrying the the take-home assignment). Participants who were not motivated from the start will in the control group, without the interference of game elements, be likely continue to be less intrinsically motivated throughout the class. On the other hand, by implementing game elements in a classroom, it can be that some students who were not intrinsically motivated from the start show different levels of intrinsic motivation for the different aspects of the class (receiving grades, carrying out the in-class assignment and carrying the the take-home assignment) thanks to the game elements. And some game elements have more effect than others this can explain why in the condition group the relationship between intrinisc motivation and game elements not as strong as the control group. Future studies should more specifically investigate different conditions under which gamification is effective for individual participants
The majority of the results are not in line with the discussed literature. And it is important to note that this study is limited. First it is important to note that the findings of this study are limited to the game elements leaderboard, badges and feedback. For this reason, these results are not indicative for all gamification systems. Another important limitation of this study is the fact that a vignette was used instead of a real-life situation. Perhaps it was not easy for all participants to imagine the situation in the vignette and the outcomes of the study could be different if the described situation of the vignette was performed in a real-life situation. Furthermore not all measurements were reliable. For this study existing scales for intrinsic motivation and student engagement were used and adjusted to the vignette. However analysis showed that not all scales were reliable. The scale for the effect of leaderboard on intrinsic motivation and the scales for the effect of badges and feedback on student engagement could be improved in future research by using other items .
22 LITERATURE
Barata, G., Gama, S., Jorge, J., & Gonçalves, D. (2013, September). Engaging engineering students with gamification. In Games and Virtual Worlds for Serious Applications (VS-GAMES), 2013 5th
International Conference on (pp. 1-8). IEEE.
Barata, G., Gama, S., Jorge, J., & Gonçalves, D. (2013, October). Improving participation and learning with gamification. In Proceedings of the First International Conference on gameful design, research,
and applications (pp. 10-17). ACM.
Bellotti, F., Berta, R., De Gloria, A., Lavagnino, E., Antonaci, A., Dagnino, F. M., & Ott, M. (2013, July). A gamified short course for promoting entrepreneurship among ICT engineering students. In
advanced learning technologies (ICALT), 2013 IEEE 13th international conference on (pp. 31-32).
IEEE.
da Rocha Seixas, L., Gomes, A. S., & de Melo Filho, I. J. (2016). Effectiveness of gamification in the engagement of students. Computers in Human Behavior,58, 48-63.
De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning.Computers & Education, 75, 82-91.
de-Marcos, L., Garcia-Lopez, E., & Garcia-Cabot, A. (2016). On the effectiveness of game-like and social approaches in learning: Comparing educational gaming, gamification & social networking.
Computers & Education,95, 99-113
Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011, May). Gamification. using game-design elements in non-gaming contexts. In CHI'11 Extended Abstracts on Human Factors in
Computing Systems (pp. 2425-2428). ACM.
DomíNguez, A., Saenz-De-Navarrete, J., De-Marcos, L., FernáNdez-Sanz, L., PagéS, C., & MartíNez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes.
23
Ellis, H., Heppell, S., Kirriemuir, J., Krotoski, A., & McFarlane, A. (2006). Unlimited learning: Computer and video games in the learning landscape.London: Entertainment and Leisure Software
Publishers Association, 66.
Fotaris, P., Mastoras, T., Leinfellner, R., & Rosunally, Y. (2016). Climbing up the leaderboard: An empirical study of applying gamification techniques to a computer programming class. Electronic
Journal of E-Learning, 14(2).
Goehle, G. (2013). Gamification and web-based homework. Primus, 23(3), 234-246.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance.
Computers & Education, 80, 152-161.
Ibanez, M. B., Di-Serio, A., & Delgado-Kloos, C. (2014). Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on Learning Technologies, 7(3), 291-301.
Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and strategies
for training and education. John Wiley & Sons.
Kuo, M. S., & Chuang, T. Y. (2016). How gamification motivates visits and engagement for online academic dissemination–An empirical study. Computers in Human Behavior, 55, 16-27.
Lee, J. J., & Hammer, J. (2011). Gamification in education: What, how, why bother?. Academic
exchange quarterly, 15(2), 146.
McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the
world. Penguin.
Narasimhan, N., Chiricescu, S., & Vasudevan, V. (2011). The gamification of television: Is there life beyond badges. In CHI 2011 Workshop Gamification, Vancouver, Canada.
O'Donovan, S., Gain, J., & Marais, P. (2013, October). A case study in the gamification of a university-level games development course. In Proceedings of the South African Institute for
24
Richter, G., Raban, D. R., & Rafaeli, S. (2015). Studying gamification: the effect of rewards and incentives on motivation. In Gamification in education and business (pp. 21-46). Springer International Publishing.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary educational psychology, 25(1), 54-67.
Simões, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345-353.
Survey intrument(n.d)Retrieved from http://nsse.indiana.edu/html/survey_instruments.cfm
Wilson, D., Calongne, C., & Henderson, S. B. (2016). Gamification Challenges and a Case Study in Online Learning. Internet Learning, 4(2), 8.
Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in
25 Appendix B.1With game elements
Het is maandag ochtend 9:00 uur. Je loopt de klas binnen om je les statistiek te gaan volgen. Je neemt plaats. Terwijl je je spullen uit je tas pakt, begint de docent met de les. Ze kondigt aan dat ze allereerst de cijfers van de vorige opdracht bekend gaat maken. Al snel verdwijnt het geluid van gelach en gefluister in de klas en richten alle ogen zich op het scorebord. De docent loopt naar het digitale bord toe. Ze klikt het grote lichtgevende scorebord aan. Jouw cijfer en die van jou medeleerlingen verschijnt op het scorebord. Je ogen zoeken snel naar jouw naam. Jouw naam staat achter een goudkleurige ster met het nummer vijf erop. Dit betekent dat er vier klasgenootjes hoger dan jij hebben gescoord. Achter je naam staat jouw cijfer.
Nadat iedereen zijn cijfer gezien heeft, zet de docent het scorebord weer uit. De les gaat verder. Je krijgt nu een opdracht die je in de les moet maken. De opdracht gaat over de lesstof van deze week en staat online. Je opent je laptop, logt in met je studentgegevens en klikt op de opdracht.
Een wit scherm verschijnt nu in beeld, met een rechtsboven een klokje. Zodra je op start klikt begint het klokje te lopen en verschijnt de opdracht. Wie als eerste de opdracht het beste maakt binnen 30 minuten, krijgt een digi-badge. Deze badge wordt op je collegekaart gezet. Dit doet de docent met behulp van een kaart scanner. Deze badges kan je sparen en bij de coffee corner inruilen voor diverse consumpties zoals thee/koffie, 3 badges = 1kop koffie/ thee. Deze week wil jij wel een badge winnen want je hebt er al twee op je kaart staan en een koffie zou er nu wel ingaan!
Als afsluiter krijg je een opdracht mee die je thuis online moet maken. Het handige daarvan is dat je direct tijdens het maken ervan feedback kunt krijgen. Met deze opdrachten kun je de stof extra oefenen voor het tentamen. De opdracht bestaat uit tien verschillende onderdelen, waarvan de eerste vier verplicht zijn en de overige zes optioneel.
Appendix B.2 Without game elements
Het is maandag ochtend 9:00 uur. Je loopt de klas binnen om je les statistiek te gaan volgen. Je neemt plaats. Terwijl je je spullen uit je tas pakt, begint de docent met de les. Ze kondigt aan dat ze allereerst de cijfers van de vorige opdracht bekend gaat maken. Al snel verdwijnt het geluid van gelach en gefluister in de klas en richten alle ogen zich op de docent. De docent loopt naar de stapel met cijfers toe. Jouw cijfer en die van jou
medeleerlingen worden uitgedeeld. Je krijgt een a4tje terug met je naam erop .Achter je naam staat jouw cijfer.
Nadat iedereen zijn cijfer gezien heeft, haalt de docent de uitgedeelde cijfers weer op De les gaat verder. Je krijgt nu een opdracht die je in de les moet maken. De opdracht gaat over de lesstof van deze week. De docent deelt de opdracht uit. Zodra je de opdracht hebt kan je beginnen. Je krijgt ongeveer 30 minuten de tijd om aan deze opdracht te werken.
Als afsluiter krijg je een opdracht mee die je thuis moet maken. Met deze opdrachten kun je de stof extra oefenen voor het tentamen. De opdracht bestaat uit tien verschillende onderdelen, waarvan de eerste vier verplicht zijn en de overige zes optioneel.
26 Appendix A questionnaire General question: Geslacht? x man x vrouw leeftijd in jaren?
Hoe heb je de survey gevonden Xvia Facebook
Xvia Whatsapp
Volg je op dit moment een studie Xja
xNee
(If nee is selected) Wat is je hoogst afgeronde opleiding? xBasisonderwijs xMiddelbaar onderwijs xHBO xWO-bachelor xWO-master (If ja is selected) Op welk niveau? xMBO xHBO xWO
Is het vak statistiek onderdeel van je studie xJa
xNee
27 *Randomize
vignet 1 vignet 2
Ik vond het leuk om deze les te volgen
Ik beleefde veel plezier aan het maken van de thuisopdracht Ik vond het leuk om de opdracht in de les te maken
Ik vond deze les saai
Het was voor mij van belang om de opdrachten goed te maken Ik vond het maken van de opdracht in de les saai
Ik vond het maken van de thuisopdracht saai Ik zou deze les als interessant beschrijven
Ik zou de opdracht in de les als interessant beschrijven Ik zou de thuisopdracht als interessant beschrijven
Ik heb veel moeite gedaan voor de opdracht die we in de les moesten maken Ik heb veel moeite gedaan voor de thuisopdracht
Het was belangrijk voor mij om de opdrachten goed te maken Het was belangrijk voor mij om de opdracht in de les goed te maken Het was belangrijk voor mij om de thuisopdracht goed te maken Ik heb niet veel energie gestoken in deze les
Ik heb niet veel energie gestoken in het maken van de opdracht tijdens de les Ik heb niet veel energie gestoken in het maken van de thuisopdracht
k voelde me niet nerveus tijdens de les
Ik voelde me niet nerveus tijdens de bekendmaking van de cijfers Ik voelde me niet nerveus tijdens het maken van de opdracht in de les Ik voelde me niet nerveus tijdens het maken van de thuisopdracht Ik was relaxed tijdens de les
Ik was relaxed tijdens de bekendmaking van de cijfers Ik was relaxed tijdens het maken van de opdracht in de les
28 Ik was relaxed tijdens het maken van de thuisopdracht
Ik voelde me onder druk gezet tijdens de les
Ik voelde me onder druk gezet tijdens de bekend making van de cijfers Ik voelde me onder druk gezet tijdens het maken van de opdracht in de les Ik voelde me onder druk gezet tijdens het maken van de thuisopdracht Ik geloof dat deze les waardevol kan zijn voor mij
Ik geloof dat het maken van de opdracht tijdens de les waardevol kan zijn voor mij Ik geloof dat het maken van de thuisopdracht waardevol kan zijn voor mij
Ik ben in staat om deze les opnieuw te volgen omdat het waardevol kan zijn voor mij Ik vond de les belangrijk
Ik vond de bekendmaking van de cijfers belangrijk Ik vond de opdracht in de les belangrijk
Ik vond de thuisopdracht belangrijk
( De boven genoemde items zijn gemeten met een 7 punt schaal ,1, helemaal mee oneens - 7 helemaal mee eens
(De items hieronder worden gemeten met een 7 punts schaal 1 zeker niet – 7 zeker wel) Ik vraag dingen over de behandelde stof tijdens de les
De kans is er dat ik naar de volgende les kom zonder de thuis opdracht gemaakt te hebben Ik vroeg mijn medestudenten om hulp toen ik de opdracht niet begreep
Ik geef graag uitleg over de lesstof aan mijn medestudenten
Ik ga samenwerken met mijn medestudenten tijdens het maken van de thuis opdracht Ik heb samengewerkt met mijn medestudenten om de opdracht in de les te maken
Na deze les zal ik vaak terugkijken naar de aantekeningen die ik van deze les gemaakt heb Ik maak een samenvatting van het geen wat ik in deze les geleerd heb
Ik maak ook de optionele onderdelen van de thuisopdracht Deze les daagde mij uit om mijn best te doen
Ik zou dit vak opnieuw kiezen
Ik besteed weinig tijd aan het maken van de opdracht in de les Ik besteed veel tijd aan het maken van de thuisopdracht
De kwaliteit van mijn interacties met mijn medestudenten waren goed De kwaliteit van mijn interacties met de docent waren goed
Ik voelde me betrokken bij wat er in de les gebeurde Ik was enthousiast over de les
