How Does the Complexity of Game Mechanics Affect the
Understanding of the Gameplay in Serious Games
SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER
OF SCIENCE
NAVDEEP SINGH
10004603
M
ASTER
I
NFORMATION
S
TUDIES
G
AME
S
TUDIES
F
ACULTY OF
S
CIENCE
U
NIVERSITY OF
A
MSTERDAM
August 28, 2017
1st Supervisor 2nd Supervisor
Dr. Frank Nack Mr. Daniel Buzzo
How Does the Complexity of Game Mechanics Affect the
Understanding of the Gameplay in Serious Games
Navdeep Singh
University of Amsterdam 10004603 Supervisor: Dr. F. Nackricky.singh72@gmail.com
ABSTRACT
In this paper it is hypothesized that the understanding of gameplay is affected by the complexity of game mechanics in serious games. Complex game mechanics are considered here as complex systems consisting of smaller mechanics that influence each other. In this article it is investigated if the complexity of the game mechanics in Minecraft1 influence
the understanding of the gameplay within the context of a serious game that addresses the teaching of digital root in math. Two versions of a Minecraft custom map will be created and tested. This custom map will be known as Num-bers Prison, and there will be a complex and a simple ver-sion.
The study shows that the players of the complex version of Numbers Prison enjoyed the game more than the players of the simple version. The gameplay was experienced more difficult by the players of the complex version. Finally, the players of the complex versions forgot more often the ac-tivities they performed in the game than the players of the simple version.
It can be concluded that complexity of the game mechanics affect the understanding of the gameplay in serious games in a negative sense, however more research must be conducted to investigate where this effect comes from.
1.
INTRODUCTION
With the growth of video games, researchers started to see the usefulness of video games for the purpose of learning and instruction. Games in this context are referred to as serious games [1]. Serious games are grounded in active learning and encourages learning activities by building on engagement and challenges to achieve the intended learning objectives [2].
1
https://minecraft.net/en-us/
However, the rise of serious games was met with failure as 80 percent of the serious games predicted not to work by 2014 due to poor design [3]. Research also showed that serious games are effective in terms of learning and retention, but not more motivational than conventional instruction meth-ods [1].
Serious games try to tackle a problem by either designing a game that intrinsically merges the gameplay with the prob-lem domain (mainly tasks) or design gameplay that incor-porates the addressed problem in some loose metaphorical manner. In other words serious games try to merge two problem spaces. The first addresses the comprehension of the domain problem, e.g. the educational purpose, such as particular math strategies. Here the ”player” tries to gain knowledge and/or learn skills through playing. The second problem space addresses the gameplay (i.e. the mechanics). The ”player” needs to understand the game mechanics to make progress in the serious game. In video games, for ex-ample, the core mechanics are mostly hidden, but players will learn to understand them while playing. Expert players will deduce what the core mechanics must be by watching the behavior of the game many times; they will learn how to use a game’s core mechanics to their advantage [4].
In serious games the ”player” faces these two problems spaces simultaneously. This can be problematic because the player might focus mainly on one problem area and therefore lose access to a large part of the game experience. This might be one reason why serious games are not as successful as as-sumed [3]. Another problem can be the complexity of tasks to be performed that degrades the appeal of serious games. The player must learn the game mechanics, understand the learning material of the serious game, and understand what the relation is between the learning material and the game mechanics in order to figure out the gameplay.
Most studies regarding serious games were to be found fo-cusing whether players understand the learning material [5], engagement and motivation [1]. However, no studies (as of this date) were found that investigate whether players were able to understand the gameplay of the serious game, which provides an essential contribution to its appeal. The appeal comes from play which is a fundamental part of the moral self-being, of the healthy and mature human life [6].
of the gameplay is affected by the complexity of the game mechanics in serious games. Gameplay is a set of activi-ties that can be performed by the player and other entiactivi-ties that belong to the virtual world during the experience, as a response to the actions of the player [7].
The thesis is structured in the following way to explore the topic mentioned in the previous paragraph. First, the paper starts with discussing related work. This is where it is ex-plored and established what (serious) games are. Next the paper dives into different components of games, and focuses on a specific component, i.e. game mechanics. This is fol-lowed by looking at complexity, and a definition for complex game mechanics is given based on complex systems. Based on what serious games are and how (complex) game mechan-ics work, two problem spaces are identified and explained. This is where the research question will be formulated based on previous research. Finally, the methodology will be dis-cussed, followed by the results from the experiment, discus-sion, and conclusion.
2.
RELATED WORK
2.1
(Serious) Games
Throughout time many researchers and philosophers have given their definition for what a game is. A game is consid-ered to be a free and fun activity standing quite consciously outside ordinary life as being not serious, but at the same time absorbing the player intensely and utterly [8].
One of the earliest philosopher to discuss games was Ludwig Wittgenstein [1953]. He argued that elements such as play, rules, and competition fail to define what games are. The term game is applied to a range of disparate human activities [9].
According to Roger Caillois [1957] a game is an activity with the following characteristics: fun, separate (as it is circum-scribed in time and place), uncertain (where the outcome of the activity is unforeseen), non-productive (participation does not accomplish anything useful), rules, and fictitious (there is an awareness of a different reality) [10].
An interesting definition comes from Salen and Zimmerman [2003]. They define a game as a system in which players en-gage in an artificial conflict, defined by rules, that results in a quantifiable outcome [11]. Noticeable here is the mention conflict, which indicates the competitive nature of games.
Crawford [2003], in his definition, talks about conflict in more detail. If no goals are associated with a plaything, it is a toy. If it has goals, a plaything is a challenge. If a challenge has no ”active agent against whom you compete with,” it is a puzzle; if there is one, it is a conflict. If the player can only outperform the opponent, but not attack them, the conflict is a competition. However, if attacks are allowed, then the conflict qualifies as a game [12].
A game is a problem-solving activity, approached with a playful attitude [13]. According to Schell [2008] games:
• are entered willfully • have goals
• have a conflict • have rules
• can be won or lost
• are interactive • have challenges
• can create their own internal value
• engage players
• are closed and formal systems.
Another definition concerning video games state that video games are designed for players to actively engage with their systems and for these systems, in turn, to react to players’ agentive behaviors [14].
So, games can be described as having goals, rules, challenges, interaction and stimulation. These definitions essentially show that games can be seen as a problem-solving activity in form of some sort of conflict or puzzle (goal / challenge) that is structured by rules.
Serious games are games that have other purposes next to entertainment. Serious games have the properties of the def-initions as described before. These type of games are being used for training [15], education [16] and health [17]. Seri-ous games are games that engage the user, and contribute to the achievement of a purpose other than pure entertain-ment. This purpose may be formulated by the user self or by the designer of the game. Therefore, commercial off-the-shelf (COTS) games, used for non-entertainment purposes, can be considered as serious games [18].
Minecraft2is such an example. Minecraft is an open-world, sandbox video game [19]. Key purposes of this game is the survival of the player and the exploration of the procedu-rally generated world. The player can harvest numerous of materials in the world, which can be used to craft other ma-terials and items. Next to the survival mode, the game has a creative mode. In this mode, the survival element of the game is disabled. Also, the player has infinite resources to all the materials and items at its disposal. This allows the player to create and manipulate the world as he/she sees fit. This has led to many worlds created by different players. This paved way to the application of Minecraft in the world of serious games.
Minecraft also introduced an educational edition, where stu-dents can be taught various subjects using the mechan-ics of Minecraft [20]. This was original a mod created by TeacherGaming 3 called Minecraftedu [21]. This version
of Minecraft includes features that benefit teachers using Minecraft in the classroom. Teachers can switch their view-point for that of an avatar to spectate mode, which allows teacher fast mobility and a bird’s eye view to easily survey the students [20]. The teacher can freeze all students in the Minecraft world to draw their full attention [20]. Exploring
2https://minecraft.net/en-us/ 3
a pre-built world, the teacher can disable the ability of stu-dents to mine and place blocks [20]. This way the teacher can limit the actions of the students, to keep them focused on the task at hand. Another feature in the educational version is the ability to give assignments [20]. This is an in-game item which contains instructions. The students can fill it in as well, which can be graded by the teacher. The teacher is also able to provide students with certain items which is needed to complete assignments, using simple com-mands [20]. This way the students do not need to craft the items themselves (if it is not a part of the game), and the teacher does not need to travel to individual students in the game.
One example of a non-COTS serious games in education a serious game called ”The Math Garden” [22]. In this game pupils try to maintain their garden. This is achieved by play-ing small games in the context of mathematics. Correlations show a high relation between the use of Math Garden and the dutch CITO4 examinations scores. Other research also
support the effectiveness of serious games in math. Research shows that even relatively short mathematical game inter-ventions can be effective. Well designed games can engage learners and facilitate a positive learning attitude. Games can provide non-intrusive ways to assess learners mathemat-ical knowledge and skills. And it is possible to achieve trans-fer [23].
Research shows mixed results when it comes to the effective-ness of such serious games. A meta analysis shows that a few studies found that games improved learning, while other studies shows no positive effects on knowledge and skill ac-quisition when compared to traditional methods of learning [5].
Another study shows the possibility of increasing primary school students multiplicative reasoning ability through an intervention with multiplicative mini-games. The mini-games were found to be most effective when played at home and afterwards debriefed at school. When utilized in this way, mini-games were found to promote students multiplicative operation skills (procedural knowledge) as well as their in-sight in multiplicative number relations (conceptual knowl-edge) [24].
2.2
Game Mechanics
According to the Mechanics Dynamics and Aesthetics (MDA) model [25] a video game consists of these three main com-ponents. Mechanics describes the particular components of the game, at the level of data representation and algorithms. This can be seen as the rule-set of the game. Dynamics de-scribes the run-time behaviour of the mechanics acting on player inputs and each others outputs over time. Aesthetics describes the desirable emotional responses evoked in the player, when he/she interacts with the game system [25]. Some of the components tie in with the previous mentioned definitions. The mechanics of the game are considered as a rule-set which help structure the play. The dynamics is caused by the interaction of the player with the systems and these systems, in turn, react to the player. In this paper, the focus will be on the mechanics component of the game.
4
http://www.cito.nl/
This will be discussed in more detail later on.
Game mechanics are the rules, processes, and data at the heart of a game. They define how play progresses, what happens when, and what conditions determine victory or defeat[4].
Core mechanics is a term often used to indicate mechanics that are the most influential, affecting many aspects of a game and interacting with mechanics of lesser importance, such as those that control only a single game element [4]. Primary mechanics can be directly applied to solving chal-lenges that result in the desired end state. These mechanics are available from the start, which are explained in the early stages of the game, and are constant throughout the game experience. In Grand Theft Auto IV5, primary mechanics are shooting, melee fighting, and driving. These mechan-ics are always available to the player, mapped to the most obvious and tradition-conforming controller inputs and re-main consistent throughout the game experience. Secondary mechanics ease the player’s interaction with the game to-wards reaching the end state. Secondary mechanics are ei-ther available occasionally or require their combination with a primary mechanic in order to be functional. The cover me-chanic in Grand Theft Auto IV is an example: it cannot be used exclusively to solve the main challenges of the game, but once mastered, it can prove of help to achieve the end state of the game [26].
Mechanics of serious games are similar to those found in en-tertainment video games. Since the main objective of each is different, there are some differences to be aware of. Whereas entertainment video games main purpose is to entertain and the mechanics used in them allows for various modes of play. Serious games for education have a different path. Educa-tional games focus on learning outcomes that are dependent upon an appropriate pedagogy and the underlying game me-chanics and how the content is integrated into the game so the learning is intrinsic to play [27]. Serious Game Mechanic (SGM) can be defined as the design decision that transitions the learning goal into a mechanical element of game-play [28]. SGMs act as the game elements/aspects linking ped-agogical practices (represented through learning mechanics) to game mechanics related to the player’s actions.
2.3
Complexity
The science of complexity mainly focuses on vast, complex systems [4]. There is no precise definition of a ”complex sys-tem” [29]. Most researchers do agree that it is a system com-posed of many interacting parts. The collective behaviour of those parts together is more than the sum of their indi-vidual behaviors. The collective behaviours are sometimes also called ”emergent” behaviours [29].
In terms of complex game mechanics, not much research was found. However, complex systems can be observed in games as well. Complex game mechanics can be considered as smaller game mechanics that influence each other. This can be observed in tower defence games which consist of a number of relatively simple parts [30]. Enemies follow a fixed path toward the player’s fortress. Each enemy has a
5
particular speed, and a number of hit points. The player places towers to defend his fortress. Each tower fires pro-jectiles at enemies within a certain range and at a certain rate. Some of the defending towers of the player deal damage while others produce other effects (e.g. such as slowing en-emies down). Sometimes towers will boost the performance of neighboring towers [29]. In this example smaller simpler mechanics influence each other. The relation between these mechanics then create a more complex system, which can be considered as complex game mechanics. Therefore, simple mechanics are regarded to be mechanics that are not inter-acting with other mechanics. Whereas complex mechanics are mechanics that do interact with other mechanics, creat-ing a complex mechanics system.
3.
METHOD
Previous studies show that players will deduce what the core mechanics must be by watching the behavior of the game many times [4]. This could be seen as a problem space by itself, which is the understanding of the game mechanics. Complex mechanics can be seen as complex systems, that would make the problem space of game mechanics bigger. However, no studies were found exploring the effect of com-plex mechanics in serious games.
Serious games have an educational goal [31]. This can be seen as a problem space where a game ( or its elements) is being added to, e.g. learning math. Research shows mixed results regarding the effectiveness of serious games [1].
So, a serious game can be considered having two problem spaces, one being the being the game part and the other be-ing learnbe-ing material part. As the player must figure its way through both problem spaces, this could result in problems in either the understanding the gameplay of a serious games or its learning material, or both.
In summary, research already shows how (serious) games work, and how to define game mechanics. However, not much research is found on the complexity of game mechan-ics, and the effect of those type of mechanics in serious games. Therefore, the research question in this paper is: how does the complexity of game mechanics affect the un-derstanding of the gameplay in serious games.
To answer the research question a quantitative experiment will be conducted. In this experiment mechanics will be ma-nipulated in terms of complexity. Complex game mechanics will be considered as a number of smaller mechanics creating a complex mechanics system. This might disturb the un-derstanding of the gameplay in serious games, as the player will also be confronted with the mathematical problem space (see figure 1). Simple game mechanics will be considered as number of smaller mechanics that do not affect each other, while the player faces the mathematical problem space. A serious game called ”Numbers Prison” for learning the ”Dig-ital Root” will be created using Minecraft6.
3.1
Numbers Prison
Numbers Prison is a serious game that teaches the players about the so-called ”Digital Root”. It is a mathematical
6
https://minecraft.net/en-us/
Figure 1: Problem spaces in complex serious games
iterative process of adding numbers until left with a single digit. The digits of any number are added together, and then the digits of the resulting number are added together. This cycle continues until the resulting number is a single digit. That single digit is the digital root of the original number. It is assumed that this approach is relative easy, but overall unknown to people.
The serious game is constructed using Minecraft version 1.12 with the help of a mod. A mod (stand short for modifica-tion) is an alteration made to the game. The mod used to help construct Numbers Prison is called WorldEdit7. The narrative of the game is that the player is locked inside an underground prison. The underground prison consists of five rooms. The player starts with a book and several pieces of paper in the inventory. The book was written by someone who already had escaped the prison. This book contains instructions on how to play the game, how to escape, and hints for each room (see figure 2). Each room has a puzzle that the player needs to solve, a locked chest, and an anvil.
Figure 2: Screenshot of instructions
These puzzles are based on the digital root method. Each room contains a set of hidden numbers (see figure 3). The player searches for these numbers then applies the digital root method. The hidden numbers are added, and if the resulting number is a double-digit number then process of
7
adding continues until left with a single digit number. This number is the pass code needed to open the chest. The player is instructed to change the name of the paper to the right pass code using the anvil. The chest will contain an item that helps the player to proceed to the next room. This is the layout of the simple version of the game.
Figure 3: Screenshot of crafting
The complex version is similar to the simple version of the game, but with added complex game mechanics. Through-out the underground prison there are other prisoners trapped. The player needs to find these prisoners so that the player can trade with them. To trade the player needs to craft the required item. To craft the player needs to find the right materials. After finding the right materials the player needs to find a crafting table to craft the item. The item that the player receives from the trade helps the player to move onto the next room. So, the complexity is added with the crafting (see figure 4) and trading systems (see figure 5).
Figure 4: Screenshot of crafting
4.
EXPERIMENT
The goal of the experiment is to see whether the complexity of game mechanics affects the understanding of gameplay in serious games. And if it does, how does this effect look like. The experiment will consist of two experimental conditions. In the simple condition the participants are going to play the simple version of the game. In the complex condition the participants are playing the complex version. Participants of both conditions will fill in the same questionnaire.
Figure 5: Screenshot of trading
4.1
Setup
Numbers Prison will be shared online with existing Minecraft communities. For each version of the game a post has been made onto either PlanetMinecraft8 or on Minecraft
Curse-Forge website9 with the description of the study, a
descrip-tion of the game, and a link to the online quesdescrip-tionnaire. Both version will not be posted onto the same website. This will be done to make sure that the same players do not play both versions, as this is a between-subjects study. This way carry-over effects are avoided. The simple version will be uploaded first. After sufficient amount of participants have played the simple version, this version will be taken offline, and the complex version will be uploaded.
A power analysis was conducted to determine the sample size. With the confidence level of 95 percent, and the confi-dence interval of 10. The power analysis shows a sample size of 97 participants. Around 100 participants are expected to participate. That will be 50 participants per condition.
Members of the community can download the Numbers Prison save file and play it using their version of Minecraft. This is the PC version of the game, and it is required to play with keyboard and mouse. The players will play the game alone (single player mode). The game is set in adventure mode, and the difficulty is set on peaceful. This means that the player cannot manipulate any of the structures (e.g. cannot destroy or place blocks), and the players cannot die in the game. The expected duration of the simple version is 10 minutes, whereas the expected duration of the complex ver-sion is 20 minutes. It is not required to finish the game to fill in the questionnaire. These participants can be compared to the ones that did complete the game in the analysis. Par-ticipants who did not finish the game can be considered as either not understanding the gameplay or not understand-ing the Digital Root concept. After the player finishes the game, an in-game message will prompt asking them to fill in the questionnaire with a link to the online questionnaire. The questionnaire is does not take more than 5 minutes.
4.2
Assessment
The questionnaire used for this study derives from the Core Elements of the Gaming Experience Questionnaire (CEGEQ)
8https://www.planetminecraft.com/ 9
[32] to measure the understanding of the gameplay. The questionnaire consists of different scales, which are: Enjoy-ment, Frustration, Video-game, Puppetry, Game-Play, En-vironment, Control, Ownership and Facilitators. The aim of the study is to see if players understand the game mechanics in serious games, only several scales were needed.
The following scales were included for the assessment: En-joyment, Frustration, Puppetry (control), and Gameplay. Enjoyment and frustration can tell how the player perceived the gameplay. Low enjoyment and high frustration can in-dicate to a low understanding of the gameplay. Puppetry (control) is split into two separate scales. One scale assesses if the players understood the controls of the game. This is to see whether the game controls influenced the players in either version. The second Puppetry (control) scale judges to what degree the players understood the gameplay. This is to see if players indicate that they knew what they were suppose to do. Gameplay scale focus more directly on their opinion of the gameplay of Numbers Prison. This is to see to what degree the players found the gameplay difficult and challenging. This led to 16 questions from the CEGEQ. The questions use a 7-point likert scale.
The questionnaire includes an open question asking the par-ticipants to state all of the activities the player performed in the game. This is done to see if the participants actually un-derstood the gameplay by reproducing the knowledge they gained about the gameplay in Numbers Prison. The an-swers will be compared with an answer model. The answer model contains all the activities that can be performed in the game that are crucial for the progression of the game. The model distinguishes two kind of answers. Answers that focus on mathematical related mechanics, and answers that focus of the non-mathematical mechanics. Whenever the participants failed to state an activity that was crucial for the progression of the game that was marked as forgotten.
The questionnaire also includes several general questions about the player and their experience with Minecraft. It is asked how much Minecraft they play per week, and what their expertise level is. The expertise level the participants can choose from are: novice, beginner, intermediate, ad-vanced, and expert. This questionnaire has been programmed online into Qualtrics10.
5.
RESULTS
The data gained from the online questionnaire is analyzed using IBM’s SPSS 11. The results section is divided into several sections. First, the participants will be discussed. Second, the different scales are compared between the two conditions. Third, the analysis will focus more on the game-play and activities the participant performed in the game.
A total of 42 participants filled in the online questionnaire. The average age off the participants was 18.40 with a stan-dard deviation of 5.01. The participants indicated to play an average of 14.73 hours per week Minecraft with a standard deviation of 12.11. The simple condition covers 22 partic-ipants, and the complex condition consisted of 20
partici-10 11
https://www.ibm.com/analytics/us/en/technology/spss/
pants. This is less than the participants needed according to the power analysis. In the discussion section this will be discussed in more detail.
First, it is interesting to see whether the amount of Minecraft played per week and the level of expertise differentiated be-tween the participants of both conditions. Therefore, the dependent variables are: playtime per week in hours and expertise level, and the independent variable is the experi-mental conditions. This might influence the understanding of the gameplay as more experienced Minecraft players will have less trouble with the mechanics problem space than less experienced Minecraft players. To analyze this an in-dependent t-test was utilized. The analysis shows no statis-tical difference on average playtime per week between both conditions. Also, no statistical difference was found on the expertise level between both groups. In other words, partic-ipants of both groups played an equal amount of Minecraft per week, and indicated the same expertise level (see table 1).
Table 1: Score playtime hours per week and expertise level (higher score equals higher
expertise)
Condition Mean Standard deviation Playtime hours per week
Simple 14.32 11.06 Complex 15.21 13.51 Expertise level
Simple 3.73 0.83 Complex 3.84 0.83
The next step is to see whether the participants of each condition differentiated on the scales of the CEGEQ. As mentioned before, there are in total of five scales (enjoy-ment, frustration, puppetry (control) I and II, and game-play). These are also the dependent (continuous) variables. The (categorical) independent variable is the experimental conditions. Based on the types of variables independent t-tests were utilized to analyze the data.
Table 2: Playtime per week and expertise level
CEGEQ Scales t-value p-value Enjoyment 2.35 0.02 Frustration 0.95 0.35 Puppetry Control I -1.42 0.16 Puppetry Control II -1.46 0.16 Gameplay 2.15 0.04
The analysis shows a significant difference on the Enjoyment scale (see table 2). The Levene’s test for equality was not significant (F = 1.63, p = .21), therefore equal variances were assumed. The participants of the complex condition scored higher on the enjoyment scale than the participants of the simple condition (see table 3).
There was a significant difference in score found on the Gameplay scale (see table 2). There is a significant result on
the Levene’s test for equality, F = 5.23, p = .03. Therefore, equal variances is not assumed. To compensate for this a slight adjustment is made to the degree of freedom. The participants of the complex condition scored higher on the Gameplay scale than the participants of the simple condition (see table 3).
Table 3: Mean scores and standard deviation (SD) on the CEGEQ scales (lower number means higher
score)
CEGEQ Scales Condition Mean SD Enjoyment Simple 2.92 1.09
Complex 2.18 0.94 Frustration Simple 5.55 1.01 Complex 5.18 1.44 Puppetry Control I Simple 1.70 0.47 Complex 2.00 0.88 Puppetry Control II Simple 1.87 0.68 Complex 2.38 1.43 Gameplay Simple 3.23 1.04 Complex 2.63 0.73
There were no significant results found on the puppetry (control) scales (see table 2). For the first scale it means that the participants of both conditions had no trouble with the controls of the game. The second scale indicates that the participants of both condition knew what they were suppose to do to progress and win the game. In other words, par-ticipants stated that they understood the gameplay. To see whether this is really the case, that the participants under-stand the gameplay, the open question was included.
For the open question a number of participants were ex-cluded based on either not answering it or giving an answer that was not related to the question. Only 10 participants for the simple condition and 13 participants for the complex condition were used. The participant was marked as forgot if they failed to state an activity crucial for the progression of the game. If all activities were stated, then the partic-ipant was marked as complete. The dependent variable is categorical. The independent variable is the experimental condition, which is categorical. To do this analysis a Pear-son Chi-Square was performed. The analysis shows that the participants of the complex conditions forgot the mechanics significantly more than the participants of the simple condi-tion, chi-square = 4.20, p = 0.04.
6.
DISCUSSION
The results do provide a number of insights that help answer the research question. First, the CEGEQ scales show that the players of the complex version enjoyed the game more than the players of the simple version. Also, the results indi-cate that the players of the complex version found the game more challenging and difficult. This can be linked to the enjoyment scale. Enjoyment increases with difficulty up to a point, where the level becomes to difficult to enjoy [33]. As the participants were acquired from online Minecraft communities, it can be assumed that the both version of Numbers Prison were not to difficult to be enjoyable. Also,
69 percent of the participants indicated that their expertise level of the game is advanced or higher.
The second Puppetry (control) scale focuses on the partic-ipants perception of their understanding of the gameplay. The analysis does show that participants of both condition indicated to understand the gameplay well, and there was no significant difference between those conditions. However, combining this result with the previous result regarding the Gameplay scale some light can be shed on the research ques-tion. The analysis shows that the participants of the com-plex version have the perception that their understanding of the gameplay is high. However, further analysis show that the participants found the gameplay more challenging and difficult. This might indicate that their actual understand-ing of the gameplay is lackunderstand-ing compared to the participants of the simple version. To further research this the open question was included.
The analysis shows that the participants of the complex condition forgot to state activities significantly more than the participants from the simple condition. Even though the participants stated to have a good understanding of the game. They also stated that the gameplay was more chal-lenging and difficult, and forgot to mention gameplay ac-tivities more than the participants from the simple version. However, the analysis of the open question shows a correla-tion. Therefore, nothing can be said about causality.
Some shortcomings need to be mentioned as this could have had a significant impact on the study. First, the number of participant was too low. As the power analysis suggested, at least 100 participants were needed. This was not reached due to time constraints and unforeseen circumstances. The study was done online using online Minecraft communities. It was expected that the participation would be much higher. Both versions combined were downloaded several hundred times, however the people who filled in the questionnaire was much lower. This was beyond expectations. An explanation for this is that people were hesitant to click the link that led to the online questionnaire. Whenever the link was clicked in the post or in-game, a pop-up warning message showed to be cautious of external links. This could have caused people to withdraw from the experiment.
An alternative explanation for the low participation is that people were not able to finish the game. Because of the online nature of the study there was no direct feedback to the experimentor. The game was play tested before it was put online. However, without the direct feedback it cannot be said for sure whether the users who downloaded Numbers Prison were also able to finish the game. This might explain the high download rate, but the low participation rate.
In the light of the amount of participants, the number of participants that filled in the open question was much lower. The Minecraft community consist of a significantly young player base. A poll on a Minecraft forum showed that 46 percent of the users were between 15 and 21 years old, and 20 percent was younger that 1512. In the current study the average age is 18 years old where the youngest participant
12
is 10. As the questionnaire was filled online and possibly alone, the participants might have not taken it seriously. As some of the answers were either blank or not related to the question.
Another point to address is that the target audience is well experienced with Minecraft. This was to ensure that the par-ticipants were familiar with the controls of the game. There-fore, the controls of the game did not needed to be explained and the instructions could be focused on the gameplay of Numbers Prison. However, the complex version consisted of the crafting and trading mechanics, which Minecraft is pop-ular for. For the experienced player this could be considered as not complex, as the player knows what to do and what to expect.
Final point to discuss is that serious games have an educa-tional goal [31]. With a serious game it is aimed to teach or train the player with learning material and through play. The current study, however, does not measure if the players understood the learning material. To pass Numbers Prison as a serious game, it is necessary to see whether the play-ers actually undplay-erstood the goal of the game, and if they learned the basic principles of Digital Root.
Future studies should refrain from doing an online study. Too much control is lost when doing such study. Having di-rect feedback whether the game is playable or not is crucial. Even after play testing, the game can still be too difficult or too easy for the target audience. Having a game that is too easy or too difficult can lead to different results. Also, a much higher amount of participants is needed to ensure statistical sound results. It is also interesting to test play-ers that have no experience with Minecraft and compare them to the experienced players. Experienced players could have a better understanding of the gameplay as they are playing in a familiar in-game environment. Finally, future studies should measure if the goal of the serious game is reached. Measurement and testing of the acquired knowl-edge and skills from the learning material in serious games needs to be incorporated.
7.
CONCLUSION
This paper started out with the aim to shed more light onto why Serious Games are failing. The study tries to look from the aspect of the game instead of the aspect of the problem that the Serious Game tries to tackle. The reason for this is that it is believed that games can be considered as a problem space. The player need to figure out and understand what the rules are of the game (mechanics), and how the system reacts to the input of the player (dynamics) in order to make progress. In Serious Games another problem space is added (e.g. learning math). Therefore, the research question that came forth was how complex mechanics in serious games af-fects the understanding of the gameplay. In this paper it is assumed that the player faces these two problem spaces at the same time making the Serious Game more difficult to play. Enjoyment of the game does increase with diffi-culty, however stops at the point when the game becomes to difficult [33].
The results of the study do show that complex mechanics affect the understanding of the gameplay to some degree.
Players who played Numbers Prison with complex mechan-ics indicated that the gameplay was more difficult and chal-lenging. Also, these players failed to state all the activi-ties they performed in the game. At first, it can be said that complex mechanics interferes with the understanding of gameplay in Serious Games. However, the study cannot give an explanation to why this is the case as the study does not reach that far.
All in all, the current study provides some indication that the complexity of the game mechanics affects the under-standing of the gameplay. How the complexity causes this effect is still yet to be answered. More research is needed to get to that answer. Hopefully, future studies will be conducted to research this topic and help improve Serious Games.
8.
REFERENCES
[1] Pieter Wouters, Christof Van Nimwegen, Herre Van Oostendorp, and Erik D Van Der Spek. A meta-analysis of the cognitive and motivational effects of serious games., 2013.
[2] Margarida Romero, Mireia Usart, and Michela Ott. Can serious games contribute to developing and sustaining 21st century skills? Games and Culture, 10(2):148–177, 2015.
[3] Brian Burke. Gamification 2020: What is the future of gamification. Gartner, Inc., Nov, 5, 2012.
[4] Ernest Adams and Joris Dormans. Game mechanics: advanced game design. New Riders, 2012.
[5] Coralie Girard, Jean Ecalle, and Annie Magnan. Serious games as new educational tools: how effective are they? a meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3):207–219, 2013. [6] Miguel Sicart. Play matters. MIT Press, 2014. [7] Carlo Fabricatore. Gameplay and game mechanics
design: A key to quality in videogames (2007), 2007. [8] Johan Huizinga. Homo Ludens; a study of
play-elements in culture. Boston: Beacon Press, 1955. [9] Ludwig Wittgenstein. Ludwig wittgenstein (1997
[1953]): Philosophische untersuchungen/philosophical investigations, selected remarks (bemerkungen) 1-6, 10, 11, 23-26, 29, 30, 43. oxford: Blackwell. 2014. [10] Roger Caillois. Les jeux et les hommes. Le masque et
le vertige. Editions Gallimard, 2015.
[11] Katie Salen and Eric Zimmerman. Rules of Play: Game Design Fundamentals. The MIT Press, 2003. [12] Chris Crawford. Chris Crawford on Game Design.
New Riders Publishing, Thousand Oaks, CA, USA, 2003.
[13] Jesse Schell. The art of game design: A book of lenses, 2008.
[14] Isabela Granic, Adam Lobel, and Rutger CME Engels. The benefits of playing video games. American Psychologist, 69(1):66, 2014.
[15] Warren Viant, Jon Purdy, and Jason Wood. Serious games for fire and rescue training. In Computer Science and Electronic Engineering (CEEC), 2016 8th, pages 136–139. IEEE, 2016.
[16] Daniel Rodr´ıguez-Cerezo, Antonio Sarasa-Cabezuelo, Mercedes G´omez-Albarr´an, and Jos´e-Luis Sierra.
Serious games in tertiary education: A case study concerning the comprehension of basic concepts in computer language implementation courses. Computers in Human Behavior, 31:558–570, 2014. [17] Mathijs FG Lucassen, Sally N Merry, Simon Hatcher,
and Christopher MA Frampton. Rainbow sparx: A novel approach to addressing depression in sexual minority youth. Cognitive and Behavioral Practice, 22(2):203–216, 2015.
[18] Per Backlund and Maurice Hendrix. Educational games-are they worth the effort? a literature survey of the effectiveness of serious games. In Games and virtual worlds for serious applications (VS-GAMES), 2013 5th international conference on, pages 1–8. IEEE, 2013.
[19] Mojang. Minecraft, 2017.
[20] Mojang. Minecraft educational edition, 2017.
[21] Aaron J Hultstrand. Minecraft in the math classroom. 2015.
[22] H Maas, S Klinkenberg, M Straatemeier, et al. Rekentuin. nl: Combinatie van oefenen en toetsen. Examens, (4):10–14, 2010.
[23] Kristian Juha Mikael Kiili, Keith Devlin, and Jari Multisilta. Editorial: is game-based math learning finally coming of age. Intl. J. Serious Games, 2(4), 2015.
[24] Marjoke Bakker. Using mini-games for learning multiplication and division: A longitudinal effect study. PhD thesis, Utrecht University, 2014. [25] Robin Hunicke, Marc LeBlanc, and Robert Zubek.
Mda: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI, volume 4, 2004.
[26] Miguel Sicart. Defining game mechanics. Game Studies, 8(2):1–14, 2008.
[27] Ant´onio Coelho, Enrique Kato, Jo˜ao Xavier, and Ricardo Gon¸calves. Serious game for introductory programming. In International Conference on Serious Games Development and Applications, pages 61–71. Springer, 2011.
[28] Sylvester Arnab, Theodore Lim, Maira B Carvalho, Francesco Bellotti, Sara Freitas, Sandy Louchart, Neil Suttie, Riccardo Berta, and Alessandro De Gloria. Mapping learning and game mechanics for serious games analysis. British Journal of Educational Technology, 46(2):391–411, 2015.
[29] Mark EJ Newman. Complex systems: A survey. arXiv preprint arXiv:1112.1440, 2011.
[30] Aura Hern´andez-Sabat´e, Meritxell Joanpere, N´uria Gorgori´o, and Llu´ıs Albarrac´ın. Mathematics learning opportunities when playing a tower defense game. International Journal of Serious Games, 2(4):57–71, 2015.
[31] Nergiz Ercil Cagiltay, Erol Ozcelik, and Nese Sahin Ozcelik. The effect of competition on learning in games. Computers & Education, 87:35–41, 2015. [32] Eduardo H Calvillo-G´amez, Paul Cairns, and Anna L
Cox. Assessing the core elements of the gaming experience. In Game User Experience Evaluation, pages 37–62. Springer, 2015.
[33] Willian MP Reis, Levi HS Lelis, et al. Human
computation for procedural content generation in platform games. In Computational intelligence and games (cig), 2015 ieee conference on, pages 99–106. IEEE, 2015.
