Iris Rigter (10331336) 31-01-2017
Master Corporate Communication Word count: 7492
Graduate School of Communication Master's Thesis
University of Amsterdam Supervisor: Joost Verhoeven
Work hard, play hard:
Can business games increase risk awareness?
Gamification, serious games and business games are a growing business and used for things such as training and education. While much research into these developments has been done, many of these studies have used quantitative approaches. Because of this, not much is known about the actual processes and game-elements that make such games effective. To offer a deeper insight into these aspects, this study has looked at the question of how the use of gamification can raise awareness and promote certain behaviors in a qualitative way. With the use of a case study through semi-structured, individual interviews this research has attempted to uncover the underlying processes and game-elements that make a business game effective. Elements that were found to influence the effectiveness of a business game were the emotional experience a game evokes, the relevance of the game to its players, the discourse of players during and about the game and the context in which the game was played.
Introduction
Organizations nowadays face all kinds of risks. A few of the top risks that can endanger organizational performance are information security, data privacy, poorly planned strategic changes and a lack of risk awareness (Audit & Risk, 2015). By taking measures like raising awareness about existing risks or promoting compliance with both organizational and national regulations, organizations can prevent risks from harming them (Bulgurcu, Cavusoglu & Benbasat, 2010). Shell, for example, has a 'Goal Zero' framework (Shell, n.d.) that includes safety standards and training aimed at preventing any harm happening to their personnel, their processes and their clients.
Raising awareness and increasing knowledge of risks that can harm a company can be done in many ways, like through a training program (Puhakainen & Siponen, 2010). An
or the use of game-like elements, has already been linked to improved knowledge acquisition and increased behavioral skills (Connolly et al., 2012). Serious game Friend Inspector, for example, creates an environment in which users can playfully discover their privacy setting on Facebook (Cetto et al., 2014). The game has shown to successfully increase privacy awareness.
The gamification market is expected to reach over 2.8 billion dollars in direct spending in the US alone by the end of 2016 (M2 Research, 2011). Surprisingly, over 80% of gamified applications fail to meet their business objectives (Webb & Cantú, 2013). This may be because not much is know about the underlying processes that make gamification effective. As a
consequence, the design of failing games is insufficiently tailored towards the aims of the game and its target audience. Roughly 84% of studies into the effects of gamification use quantitative methods like experiments or surveys (Connolly et al., 2012). Most studies thus look at the generalized relation between gamification and outcome variables like job performance or knowledge acquisition. While quantitative research offers the opportunity to study a large number of respondents, it lacks the ability to ask in-depth questions regarding the topic
(Hopkins, 2008). This means that there is little research on the aspects of the game that underlie the positive effects gamification has on raising awareness and knowledge. For example, do people who play with a colleague rather than alone notice more positive effects? This study therefore aspires to look at the underlying processes and game elements that make gamification effective by asking the question:
How does the use of gamification raise awareness and promote certain behaviors?
This study will attempt to add to existing knowledge on the effects that gamification has on awareness and behavior. By offering an in-depth look into game-elements or personal
experiences that influence the effects of a game on awareness, a possibility opens up for further research. In addition to the scientific field, organizations may benefit from this research. Many organizations jump on the gamification bandwagon without being fully aware of its effects, as can be seen by the high failure rate of business games (Webb & Cantú, 2013). By generating knowledge on the processes and elements in the game that create the effectiveness of
gamification, this study may increase understanding into processes that make gamification effective. A better understanding can help organizations to improve the development of business games, lowering the failure rate and making gamification more effective.
Theoretical framework A definition of gamification
In spite of the first video game being designed by William Higinbotham in 1958 (Kalning, 2008), gaming only became popular in the 1970s with the introduction of games like Space Invaders and Breakout (Donovan, 2010). While games were first only seen as entertainment, their educational value soon became apparent (Randel et al., 1992).
With an increase in interest for educational games, came the rise of gamification. A term first used by Nick Pelling in 2002 (Fitz-Walter, 2013). Despite its introduction in the early 2000s, it was not picked up by both research and businesses until 2010 (Jakubowski, 2014). A resurgence in interest in 2011 led to increasing research and with that several attempts to define gamification (Fitz-Walter, 2013). In spite of great diversity in definitions, there are some
similarities in most of them. All refer to the use of elements of game design, and a majority also mentions the use of these elements in a ‘non-gaming’ context. Therefore this study will use an adjusted version of the definition as given by Domínguez et al. (2012): 'Gamification is the use of game design elements, such as leaderboards and levels, and game mechanics in non-game
contexts, like education or employee training'. This definition fits both the similarities mentioned
before, and is broad enough not to exclude any type of gamification used within organizations.
Business games
There are different kinds of business games available for various aims (Kankanhalli et al., 2012). All games can be divided into solo play, cooperative play or competitive play (Waddel & Peng, 2014). Solo mode only lets players interact with the gaming environment and with
game-controlled characters (Waddel & Peng, 2014). A game element that is often used in solo mode is a personal avatar, which offers participants the option to determine who or what they want to be in-game (Blohm & Leimeister, 2012) and increases personal motivation (Deterding et al., 2011). Social play, however, offers participants an environment in which they can either collaborate or create a competition (Jansz & Martens, 2005). In collaborative play people work towards a collective goal (Brewer & Gardner, 1996) or support each other in reaching certain goals (Bosworth, 2012). A so-called viral loop usually indicates the use of a collaborative game. In such a loop people need to invite other players and work together with them to progress in the game (Kankanhalli et al., 2012). Competitive play on the other hand, happens when one
employee attains his or her goals at the expense of someone else (Johnson et al., 1986). This can be done by adding a leaderboard to games (Kankanhalli et al., 2012). In all types of games, scoring systems such as points or badges can be used (Meder, Plumtbaum and Hopfgartner, 2013). This can offer participants a sense of achievement (Blohm & Leimeister, 2013) and with that a positive reinforcement to continue using the application (Dale, 2014). Collaboration during a game has been shown to lead to prosocial behaviors (De Simone & Riddle, 2001), whereas competition has been linked to aggression and conflict (Waddell & Peng, 2014).
Using these types of games and elements, companies can create a business game that fits their aims (Callan & Landers, 2012). For example, Deloitte uses gamification to motivate employees to learn on the job about among other things management styles and interpersonal communication (Harvard Business Review, 2013). With their Deloitte Leadership Academy platform, participants are asked to complete certain trainings to improve their management and work-related skills and in return can collect badges and see their scores on the leadership board. This type of gamification uses the game-elements of a scoreboard and the rewards in the form of badges.
Effectiveness of business games
Throughout the years, research has frequently shown that gamification can lead to positive results (Connolly et al., 2015). A concrete example of this can be seen in the KEAS wellness program, which promoted better nutrition and more exercise among employees using a game (Bosworth, 2012). The use of a gamification platform not only triggers motivation, but also increases employee engagement (Bosworth, 2012). Other studies have found that gamification can have a positive impact on work-related motivation (Ruhi, 2015), on enjoyment (Herzig, Ameling & Schill, 2015) and on group cohesion (Hornaday, 2014). A meta-analysis done by Connolly et al. (2012) also shows an improvement of knowledge acquisition through the use of gamification.
In spite of these positive results, there is no consensus yet on whether gamification actually has positive effects. Dominguez et al. (2013) in their study showed that gamified
activities can actually hinder the understanding of theoretical concepts when applied to learning, making gamification unsuitable for certain kinds of training. In addition, enjoyment can decrease as employees use the game for a prolonged period of time. This can point to a novelty value of
gamification (Koivisto & Hamari, 2014) or the increase of boredom after using gamification for a while (Ruhi, 2015). Mollick & Rothbard (2014) also warn for what they call the paradox of mandatory fun; an externally-imposed structure of fun does not automatically make the game enjoyable. When gamification is implemented without consent from the employee, it may thus decrease performance. Overall it can be said that research on the effectiveness of gamification paints a mixed picture so far (Seaborn & Fels, 2015).
Risk awareness
Before it can be established how business games create awareness, it is important to define this concept. Risks can harm or endanger an organization and its stakeholders. A way to minimize the negative effects risk has is by creating employee awareness (Bulgurcu et al., 2010). Risk awareness encourages workers to 'stop and think before starting work, so that they may identify hazards and take action to control the associated risks' (Borys, 2009). It can be defined as an employee of the organization feeling, experiencing or noticing risk that can potentially harm the organization (Bin Ishaq Alseiari, 2015).
An example of employee awareness that is becoming increasingly important for organizations is that of cyber crime awareness. Organizations rely heavily on information
systems and any damage to these systems can have severe consequences, like monetary losses or corporate liability (Bulgurcu et al., 2010). Currently employees are the weakest link in these information systems (Mitnick & Simon, 2002), as they are prone to making mistakes or
disregarding regulations (Bulgurcu et al., 2010). Raising employee awareness through education and training can reduce these problems, making it a valuable consideration for organizations (Furnell et al., 2002)
Method
Research design
To dive deeper into the underlying processes of business games, the reasons why they are effective and the elements that make them effective, this study has used a qualitative approach. Qualitative research offers the opportunity to ask people in-depth questions about their
experiences with business games, and thus offers a better look into the processes at work when someone uses a business game (Flick, 2009).
Participants (N = 20) were asked to play a mobile, turn-based business game that aims to develop security awareness on cyber risks. The game is a platform game, with each level
showing a map of an office floor with several rooms and tools in these rooms. By using tools and talking to characters in the game, participants had to make their way to the exit. The game used a collaborative approach: players had to choose one of the aforementioned roles, and based on the role they chose they had certain skills. Their co-player then had a different set of skills, and by working together they could progress through the game as a team. The game was designed by a game-design company. The researcher was allowed to use this game for study, but the game itself cannot be named and information had to be used anonymously. A further description of this game is provided in Appendix A. The experiences and opinions of participants were discussed during a semi-structured interview with the use of an interview guide.
Participants
There were several prerequisites for participants to join this study. Participants needed to work at an organization that faces security risks. In addition, participants needed to be available between the period of October 31st and November 25th to participate in the game and the interview afterwards.
Potential participants were contacted by e-mail via snowballing. An initial e-mail was sent out to acquaintances of the researcher, who in turn provided information of possible
participants to be invited. An invitation was sent out with a short explanation of this study and its topic. It was also made clear that participation was anonymous, and that participants could withdraw from the study at any time without further consequences. After agreeing to join the study, 26 people received an email with instructions on how they could play the game. After signing up, 6 people indicated that they did not have sufficient time to participate and decided to leave the study. The remaining participants (N = 20) who joined this study came from two organizations. Half of the participants (N = 10) worked at a consultancy firm, the other half of the participants (N = 10) was employed by a digital agency. Follow-up interviews were
scheduled with all participants in the two weeks after playing the game. Interviews were held at the organizations where the participants worked. Before the interview, participants were asked to sign an informed consent form. This form can be found in Appendix B, and included information on the voluntary conditions of the study and the recording of the interviews for further analysis. No incentive was given to participants of this study.
During the interviews some participants indicated that they played a large number of levels of the game, but did not completely finish it. As this study looks at the underlying processes and game-elements that make business games effective, these participants were still included in this study. This because not finishing the game with valid reasons could still provide this study with relevant information on game-processes and elements that may make it difficult to continue the game.
As this study has used qualitative, semi-structured interviews to collect data, an interview guide was set up as a research tool. This interview guide consists of 18 questions, and is composed of 4 themes. The themes discussed were personal information, experiences with the game, risk
awareness before and after the game and behavior after the game. Easier questions were asked first to put the interviewee at ease. The entire interview guide can be found in Appendix C. Each interview was recorded and then transcribed. With transcribing all interviews are listened to and written down word for word, for further analysis. Transcripts were then coded by using the grounded theory approach (Flick, 2009). This approach divides coding into three phases: open coding, axial coding and selective coding. Through open coding certain codes are added to sentences or word groups, and axial coding then aims to look for connections between the different codes that have been found in the transcripts. These connections create certain categories within the text, for example positive sides of playing with a colleague and negative sides of playing with a colleague. Selective coding analyses these categories and looks for examples and exceptions per category, so that a fully inclusive analysis of the transcripts can be provided. A founded theory approach has been chosen as this approach is known as a fitting tool to uncover explanations of processes and experiences from transcripts (Flick, 2009). The codes found in the transcripts were categorized in four themes; emotional experience, relevance, discourse and context of play. These themes will be discussed in the results section of this research.
Reliability
To ensure a reliable study, several measures were taken. First of all, individual and anonymous interviews were chosen to minimize the possibility of socially desirability. Making use of
be traced back to them (Opdenakker, 2006). Secondly, notes taken during the interview can help with the later analysis of the transcripts. When an interviewee seemed nervous or uncomfortable, and this is noted during the interview, it offers the possibility to better understand the context in which the answers were given. Lastly, the use of an interview guide ensures that all participants are asked the same questions. This way the answers to those questions can be properly
compared, making analysis more reliable.
Results
In this results section several topics will be discussed to explain the way business game
influenced risk awareness. These topics address the underlying processes and game-elements of effective serious games: emotional experience, relevance, discourse and context of play. First, however, some information will be given on how respondents rated the game they had to play as this experience may influence how they further answered the questions asked during the
interview.
Appreciation of the game
Respondents’ answers paint a very mixed picture on how employees rate the game they played. Participants were asked to rate the game between 1 and 10, with 1 being the lowest and 10 being the highest. The lowest grade given was a 3. According to the respondent:
“Because I can say that the game is designed so you can play at your own pace, but because you are dependent on another person that is extremely annoying. […] I think the game is too difficult for it to reach its goal, it is such a slow way to learn about cyber
On the other side of the spectrum, however, one of the respondents rated the game with a 9:
"Well, I'm never allowed to say 10, so that means I would go with a 9. I have really
enjoyed playing this game, and I am eagerly waiting for new levels to be released. [...] It
has really made me more aware as well." (Respondent 6)
Generally, participants rated the game with grades between 6 and 8, indicating that most were quite positive about the game but that some improvements could be made. Frequently mentioned positive aspects of the game were the design of the game and the fact that the game used humor. Negative aspects mentioned were that the notifications of the game didn’t work properly and that the game was quite slow to play.
Emotional experience
Both the positive and negative aspects of the game evoked an emotional reaction from respondents in this study. One of the emotions that was most frequently experienced was
frustration. There were a number of game elements that led to these feelings. First of all, several respondents mentioned that they did not find the game user-friendly. Especially the beginning of the game, right after the tutorial, was said to be complicated to understand:
At first it took some time getting used to. Like ‘okay, how exactly does this work?’. You can look at the instruction video of course, but everyone understands that straight away.
The point is more, like, how does the hacking and social engineering work?
Some respondents even indicated that they found the gameplay difficult and therefore struggled to progress in the game:
“I think that in the beginning, well, what I mostly noticed was that we had the feeling that we didn’t understand the game. We have, I think, in the first week. Well, we are now two weeks in and we have finally finished the game” (Respondent 4)
A second, frequently mentioned point of frustration was the turn-based aspect of this game. Even though the game was designed for people to play occasionally during small breaks, only little progress could be made per turn. As respondents had to wait for their co-player to finish his or her turn before they could play again, it could take a few hours up to a few days before they could continue the game. For players whose co-player only played at infrequent intervals, this posed a problem:
"I tried playing it in the evening while I was sitting on the couch, uhm, but yeah, then you
can only do one move or a little part of a level and then you have to wait again. And
sometimes that one [other player] can't play and then you want to continue but you can't.
I found that a bit of a killer" (Respondent 3)
Two respondents got so frustrated with this feature that indicated they would prefer to play all
alone, rather than having to wait for someone else. According to some respondents, their
"[...] it was so unclear to us what was happening and we kept going round and round and
it was really sort of [deep sigh]. Because of that you also learn very little. "
(Respondent 14)
As opposed to the frustration felt by a large number of respondents, another group indicated that they really enjoyed the game while playing it. Several respondents even called the experience fun:
"I found the setting really fun. That was also because there was some humor included,
you know? It wasn't that I felt like I had to play the game again. It was more like, yeah,
I'm going to enjoy playing that game again!" (Respondent 18)
The respondents who said they had enjoyed the game also formed the majority of people who finished all levels. Enjoyment is thus an important factor in keeping people motivated to play. Since finishing all levels means they were confronted with more examples of risk awareness, including humor and fun elements can be seen as an important way to increase the effectiveness of a serious game.
Relevance
In this study respondents were asked to mention the cyber risks they learnt from the game. The majority of respondents mentioned risks that they could connect to their everyday experience. Several other respondents indicated cyber risks that they found in the game and that they could relate to their own experiences at work.
"Sometimes I sit, not always, but often I sit next to the door. And in this building people
walk in every now and then, and they could be clients or freelancers, but I don't know.
And sometimes the office manager isn't here, and out of politeness I usually ask people
who they are visiting and offer them coffee. But someone could, and that is what you
realize when playing the game, easily say they are from any company. So that is
something I will look out for in the future." (Respondent 10)
The importance of relevance to the players became apparent from this. Especially since one of the respondents expressed frustration about not being able to identify with the game:
"So you have this level that is sort of logistics where you go in with a bus and whatever.
And then I'm like, this is so far from reality. What good is it to me that this happens? I
could not identify with this" (Respondent 11)
When a game is relevant for an employee, this can also contribute to the engagement an employee experiences.
Discourse
In the game respondents played, they were asked to team up with a colleague and take turns to advance in the game. Because of the co-operative element in the game, discourse became an important aspect of the experience. Contact between co-players was maintained in several ways: by playing during working hours whilst sitting next to each other, by using personal online-messaging service or by using work-related communication channels.
The game is set up in such a way that people can take turns in playing and can play the game whenever they want. This offers players the opportunity to check the game at times they find convenient, for example when they are waiting for the bus or sitting at home. However,
several respondents indicated that they made specific appointments to meet at the office and sit
down to play the game:
"At a certain moment we decided like, what if we communicate better on all the things we've seen in the game? That would make it a lot easier. At some point we even sat down, last week on Friday, to play together. That is when you really make progress in the game, and without that communication I think it might be too difficult" (Respondent 1)
Being able to communicate with your co-player outside of the gaming context turned out to be an
important issue. Other players mentioned the use of personal communication channels such as
Whatsapp, or work-related communication channels like Slack and Skype:
"Eventually we had to use Slack to solve who would play against each other. And then we began, and everyone would still use Slack to communicate as a channel next to it [the game] or from my co-player via Whatsapp. In other words, the fact that we all used Whatsapp and Slack to solve things means that the communication feature in the game doesn't work properly." (Respondent 12)
This need for a constant communication between players got very mixed reactions, ranging from very positive to quite negative. People who were positive about this social feature mentioned that
they felt they learnt more from playing the game together with a colleague and discussing the game with that person:
"I really like collaboration and working together to solve things. That you can discuss it
afterwards. Because that is what we did, we really talked about how we were going to
complete the next level." (Respondent 6)
In addition, playing with a colleague increased the commitment to the game. When you have someone waiting for you to make a move you are more likely to pick up the game as to not to
disappoint the other, as one respondent mentioned. The downside mentioned was the fact that
you had to wait for your co-player to finish his or her turn. However, most of the respondents
who mentioned this downside of playing together also saw the benefit in having a partner in this
game. One respondent even called it a 'double edged sword':
"I think it helps when you play together, because you also talk about it outside of the game. But it is, uhm, it is a double edged sword. You can't really keep playing and because of that you can't really get into the game" (Respondent 3)
Other respondents offered solutions for this problem. By creating a game that first gave the
opportunity to play alone and get acquainted with the process, and adding the social feature later
on, the game could tap into the best of both worlds and keep players motivated throughout.
"Maybe they could add a message option to the app. Because you can leave information or a message for the next person, but maybe it would be better if you could do that constantly instead of only after each turn. That way you can also discuss during [the game]." (Respondent 20)
Most respondents see an added value to a social feature within a game, but it is absolutely
essential that the social feature works properly and offers sufficient features to communicate
within the game.
Context of play
The turn-based and mobile aspects of this game gave players the freedom to choose when and
where they preferred to play the game. Here a divide can be seen between people who chose not
to play during work hours and people who did play during work hours. Some respondents
mentioned that playing the game did not feel like part of their tasks, and thus mostly played
outside of office hours:
'In the beginning I didn't play it much because I didn't want to play during work hours, but when I was at home watching the news I would think about playing the game at the same time. You know?" (Respondent 11)
Other players made the decision to sit together and play the game during scheduled meetings.
This group also had an overall more positive opinion of the game. People who played during
work hours likely did not see the game as something they had to do on top of their work, but as a
become more acquainted with certain risks, but do not feel like they are given time to do so
during work, they might experience the game as an infringement of their personal time.
Conclusion
Through a case study of a mobile-based business game, this study has looked at the relationship
between these games and learning: the underlying processes and game-elements that can explain
the impact of a business game on awareness. Twenty respondents were interviewed about their
experiences with playing a business game aimed at creating cyber- and information risk
awareness in order to find these processes. Respondents played the business game specifically
for this study. Previous literature on this topic has already set several prerequisites for learning,
which also apply to gamification. Fogg's behavioral model (2009) for example, states that
learning requires motivation to learn, the ability to learn and a trigger to learn. Aspects that can
either support or hinder learning through gameplay such as motivation to play or a lack thereof
were also found in this study. Firstly, the emotional experience evoked can be an important
contribution to the amount of awareness that a serious game establishes. Previous studies have
already shown that games are motivating when it comes to learning because of their impact on
emotional areas of players (Dominguez et al., 2013). This study, however, shows that the kind of
emotion felt also has an impact. Once a game creates a feeling of frustration, this can negatively
influence game-motivation and with that the effectiveness of a serious game. On the other hand,
positive experiences such as fun can increase motivation to play.
Secondly, the relevance of a game to its players has an influence on game effectiveness.
Once employees feel like the situations they encounter in the game are relevant for their own
life, this can increase their motivation to play the game. An explanation for this can be found in
When employees see the relevance in a business game they will most likely also find it more
interesting, as they can apply the knowledge from the game to their everyday life. This higher
motivation leads to more gameplay and through that to raised awareness, which indicates a more
effective business game.
A third game-element that can have a positive effect on creating awareness is a social
element and the promotion of discourse about the game. It helps people to connect and discuss
the game with others, making the topic of the game more top of mind. This discourse, however,
can be seen as a double edged-sword. Once the social element of the game is lacking the inability
to communicate with a co-player can create frustration, which can lessen the effect a business
game has on awareness. A final element that is important for effective use of business games is
the context of play. Since a business game is often imposed on employees by
upper-management, there is a risk of the game feeling like yet another task. If this task cannot be done
during office hours, it might create feelings of resentment towards the game. To prevent this
negative spiral from happening, it is important for employees to get sufficient time to play
business games during work-time.
Discussion
Motivation to play
Before a business game can create awareness, it is important for employees to actually (continue
to) play the game. An important factor here is motivation, which is often connected to the theory
of self-determination (Ryan & Deci, 2000). This is a theory of human motivation ‘concerning
people’s inherent growth tendencies and their innate psychological needs’ (Francisco et al., 2012) that divides motivation into intrinsic and extrinsic. Intrinsic motivation can occur when a task is interesting or enjoyable (Perryer, Scott-Ladd & Leighton, 2012) and is a natural drive in
people to seek out challenges and aim to further develop themselves (Ryan & Deci, 2000). Extrinsic motivation occurs when a task leads to external desirable outcomes, for example an award or a bonus (Dale, 2014). Three variables are said to influence the level of intrinsic motivation a person experiences: autonomy, competence and relatedness (Ryan & Deci, 2000).
Autonomy 'involves acting with a sense of volition and having the experience of choice' (Gagné
& Deci, 2005). The feeling of competence indicates a 'state of having requisite or adequate
ability or qualities' to complete a task (Ryan & Deci, 2010). Finally, relatedness refers to 'the
desire to interact with and be connected to other people' (Ryan & Deci, 2000). This study has
shown several game-elements and processes that are relevant in creating motivation through
these variables. First, the emotional experience has an influence on whether employees want to
play a business game. Respondents who enjoyed playing the game were motivated to keep
playing, and all of them finished the entire game with a positive attitude. On the other hand,
respondents who indicated that they felt frustrated while playing the game mentioned that they
often quit playing the game after experiencing several levels. A lack of user-friendliness and the
perceived difficulty of the game were important reasons for this. These feelings of frustration in
turn led people to feel like they did not understand the game, which negatively influenced their
feelings of competence. As the self-determination theory mentions, competence is an important
predictor of motivation (Ryan & Deci, 2000). It can thus be stated that emotional experiences
influence motivation, with enjoyment leading to more motivation whereas frustration leads to a
lack of motivation.
A second game-element that influenced players' motivation was the relevance of its
content. Respondents connected in-game experiences to their own reality, indicating that they
value in the serious game they are playing and the benefits they can reap from the game, their
motivation to play will go up as it gives them another reason to finish the game. Using a game
that is relevant for players has already been linked to more effective use of serious games
(Blohm & Leimeister, 2013). It is even the center argument of the theory of player-centered
design (Kumar, 2013). This design process puts the player at the center of the design, by first
understanding the player and his/her motivation to play (Kumar, 2013). By keeping the player in
mind when developing the game, you create a serious game that is relevant for the player and
will thus increase motivation.
Creating discourse within or about the game can also increase motivation to play when
implemented correctly. On the one hand playing together with a colleague showed to be an
important motivator as players did not want to let their colleague down, and because they
enjoyed playing the game with someone else. This can also be traced to relatedness in the
self-determination theory, which states that the desire to connect to other people is an important
motivation. The game offers players an opportunity to connect with their colleagues, as they are
obliged to play the game with someone else. According to the self-determination theory, that in
turn should lead to higher motivation. On the other hand, however, the waiting period between
players’ moves was said to be demotivating as it takes the speed out of the game and creates feelings of frustration when players have to wait for a long time.
A final factor that influenced players' motivation was the context in which they played
the game. Players who played during work hours had an overall more positive attitude towards
the game, and these feelings of enjoyment increased their motivation to play. People who felt
like they had to play the game outside of work hours, on the other hand, seemed less enthusiastic
theory of self-determination. An important aspect in this autonomy is feeling like you have a
choice. When employees are told that they have to play a game to learn, this leaves them with
little room to make their own choice. If they play during work hours, however, they get the
opportunity to choose whether they want to play the game or engage in other work activities.
Especially for people why enjoyed the game, being able to choose something fun to do during
work hours can increase motivation to play the game. People who play outside of work hours, on
the other hand, may experience even more limitation in their choices. They are not only
obligated to play the game, but also feel like their choice of how to fill in their spare time is
being taken away as they have to play the game. This feeling of less autonomy in turn leads to a
lower motivation, and can be prevented by offering employees time to play the game during
work hours. Playing the game should thus be seen as in-role behavior by employees, and
facilitated as such. The role theory divides work between in-role and extra-role behavior (Zhu,
2013). In-role behavior refers to the kind of behavior that is ‘described and defined as part of employees’ work and reflected in the official salary system’ (Morrison, 1994). Extra-role behavior, on the other hand, are behaviors that are not part of an employee’s formal job requirements (Zhu, 2013). Forcing employees to take on extra-role behavior, for example
playing a game in their own time, can lead to feelings of resentment (Zhu, 2013). This can have a
negative effect on motivation.
These findings show that a first prerequisite in creating risk awareness through a serious
game is thus to get players motivated. The aforementioned game-elements can either promote or
hinder this motivation.
Respondents in this study indicated that they thought playing the game together and being able to
discuss the contents of the game made them learn more. The collaborative learning theory
supports these observations. Collaborative learning is a method in which people are grouped
together to achieve a common goal (Gokhale, 1995). Through this shared learning, participants
get the opportunity to engage in discussion, which has been linked to higher levels of thought
and better information retainment (Johnson & Johnson, 1986). In this light a business game that
encourages collaborative gameplay can be expected to also stimulate collaborative learning.
Adding a social feature that promotes collaboration may thus be one of the features that makes a
business game effective in creating risk awareness. These assumptions are supported by earlier
studies that have already shown that players who worked in groups learned more than those who
worked separately (Wouters et al., 2013) and that cooperation is a key factor in both motivation
and engagement (Romero et al., 2012).
Respondents also mentioned the relevance of the game as something that made them
remember the content. Several even said that remembering certain lessons from the game made
them change their behavior. A similarity between players’ own reality and the game-reality can thus be seen as an important factor in creating risk awareness. Generating new knowledge based
on what you already now, is an important foundation of the constructivist learning theory
(Dewey, 1986). This theory states that people learn by experiencing things, reflecting on those
experiences and connecting this new information to prior knowledge (Fosnot & Perry, 1996).
Respondents in this study indicated that they already had some knowledge about the situations
created in-game from their work experience. The game then enabled them to experience these
they could connect to their existing knowledge. By adding to this knowledge, people learnt more
on the topic of (cyber) risks and thus increased their risk awareness.
Implications of this study
From a scientific standpoint this research offers a different perspective on the effectiveness of
business games than most studies on this topic. The use of a qualitative study offers a more
in-depth overview of aspects that are important to the effectiveness of a game. The concept of
emotional experience and its link to intrinsic motivation has already been established in several
other studies (Herzig, Ameling & Schill, 2005) as well as the fact that a business game has to be
relevant to employees to be effective (Blohm & Leimeister, 2013) and the idea that collaborative
learning increases effectiveness (Johnson & Johnson, 1986). This study thus supports earlier
findings in these regards. However, the proposition that business games are more effective when
they are seen as in-role work behavior rather than an extra task is added knowledge to the field of
business games. Future studies can use a quantitative approach to see if this concept is also found
in larger groups of respondents, and with that expand on the generalized knowledge there is of
processes that influence business gaming effectiveness. In addition to these findings, this study
also uncovered some game features that can enable effective business gaming. First, a clear
manual or help desk while playing the game seem to be important factors in preventing
frustration to keep players motivated. Second, a communication feature that helps players discuss
the gameplay in-game can increase motivation and enjoyment.
From a practical standpoint, this study can be of great relevance for both organizations
that create business games and organizations that are considering the design and implementation
of such a game. Frustration was found to have a negative effect on game effectiveness. A
clear instruction before the game and a possibility to ask for help during the game. This can be done by adding a manual or offering help desk services with support employees. In addition, the way employees will play the game has to be taken into account during the design. Questions like how and when people will play and what features are necessary to avoid frustration need to be asked before the game is implemented. Another finding is that topics and games that employees see as relevant to their own life get more attention, which positively influences risk awareness. It is thus important for a company to ensure that a business game is relevant for its employees. Especially in larger organizations with bigger and more diverse teams, this can pose a challenge. As different kinds of employees have different preferences and even more so different fields of interest, creating a game that can be personalized based on these characteristics might be crucial to an effective implementation. Games can for example be tailored per employee or group of employees, or can offer a large number of modules so that employees can choose depending on their interests. In addition, this study found that it is advisable for organizations to frame business games implemented as an in-job task rather than an extra task and offer time during work hours to play the game. This way people do not feel like they have to take on more work outside of office hours, which can prevent frustration and resistance from occurring.
For studios that design business games, these elements are of value in the creation of a
good game. Corporations that want to use a business game for their employees, know that they
have to look at a well-designed game to prevent frustration, to implement games that employees
will see as relevant to their own experiences and to promote the game as an in-job task. This can
increase the effective implementation of business games, which will mature the domain and
make it a more attractive way of learning for organizations.
The use of an actual business game played by employees of different companies specifically for
this study combined with the fact that face-to-face interviews were used in which personal
experiences of participants could be discussed, has led to a high external validity. In addition,
this study offers insights into the processes and game-elements that can influence the
effectiveness of business games. However, it is also important to note some limitations of this
research. Due to a short time-span and miscommunication between players, several respondents
indicated that they had not completely finished the game. Respondents in this study were given
two weeks in which they had to finish the game, and for some respondents this time-span turned
out to be too short. It can be said that these unfinished games create an incomplete picture of the
elements and processes that make a business game effective. However, players who did not
finish the game still had relevant experiences and opinions. Their answers are therefore still
included in the results. In addition, when organizations implement a business game among their
employees they may also encounter a certain drop-out rate. Participants in this study not
finishing the game in time can thus also be a representation of the real-life context in which these
games are played.
A second limitation of this study lies in the interview questions that were used. While the
questions did bring to light several processes that influence effective gamification, it is possible
that not all processes that influence the effectiveness of business games have been uncovered. A
possible explanation for this limitation can be found in the research method used: asking people
to reflect on their personal learning processes through interviews can be extremely difficult for
respondents, as people are often not aware of exactly how they learn. Future studies on this topic
study on learning processes in business games, such as an observation in combination with
interviews.
In spite of these limitations, this study has provided an insight into the underlying
processes that contribute to effective serious gaming and offers an opportunity for future
research to expand on this.
Literature
Audit & Risk (2015) Top ten risks facing organizations in 2015. Retrieved from
http://auditandrisk.org.uk/features/top-ten-risks-facing-organisations-in-2015
Bin Ishaq Alseiari, K. (2015). The Management of Risk Awareness in Relation to Information Technology (MERIT) (Doctoral dissertation, University of Gloucestershire).
Blohm, I., & Leimeister, J. M. (2013). Gamification: Design of IT-based enhancing services for motivational support and behavioral change. Business & Information Systems Engineering (BISE), 5(4), 275-278.
Borys, D. (2009). Exploring risk-awareness as a cultural approach to safety: Exposing the gap between work as imagined and work as actually performed. Safety Science Monitor, 13(2), 1-11.
Bosworth, A. (2012). KEAS: developing a successful game-based employee wellness program. GAMES FOR HEALTH: Research, Development, and Clinical Applications, 1(3), 189-191.
Bulgurcu, B., Cavusoglu, H., & Benbasat, I. (2010). Information security policy compliance: an empirical study of rationality-based beliefs and information security awareness. MIS quarterly, 34(3), 523-548.
Brewer, M. B., & Gardner, W. (1996). Who is this "we"? Levels of collective identity and self representations. Journal of personality and social psychology, 71(1), 83.
Callan, R. C., Bauer, K. N., & Landers, R. N. (2015). How to avoid the dark side of
gamification: Ten business scenarios and their unintended consequences. In Gamification in education and business (553-568). Springer International Publishing.
Cetto, A., Netter, M., Pernul, G., Richthammer, C., Riesner, M., Roth, C., & Sänger, J. (2014). Friend inspector: a serious game to enhance privacy awareness in social networks. Proceedings of the 2nd International Workshop on Intelligent Digital Games for Empowerment and Inclusion.
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661-686.
Dale, S. (2014). Gamification Making work fun, or making fun of work? Business Information Review, 31(2), 82-90.
De Simone, J. J., & Riddle, K. (2011). Does cooperation decrease state hostility? An exploration of cooperative play in a violent, war-themed video game. Paper presented at the Association for Education in Journalism and Mass Communication Conference, St. Louis, MO.
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (9-15).
Dewey, J. (1986). Experience and education. The Educational Forum 50(3), 241 - 252. Taylor & Francis Group.
Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380-392.
Donovan, T. (2010). Replay: The History of Video Games. East Sussex, UK: Yellow Ant.
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance improvement quarterly, 6(4), 50-72.
Fitz-Walter, Z. (2013). A brief history of gamification. Retrieved from
http://zefcan.com/2013/01/a-brief-history-of-gamification/
Flick, U. (2009). An introduction to qualitative research. Sage.
Fogg, B. J. (2009). A behavior model for persuasive design. In Proceedings of the 4th international Conference on Persuasive Technology (p. 40). ACM.
Fosnot, C. T., & Perry, R. S. (1996). Constructivism: A psychological theory of learning. Constructivism: Theory, perspectives, and practice, 2, 8-33.
Furnell, S. M., Gennatou, M., & Dowland, P. S. (2002). A prototype tool for information security awareness and training. Logistics Information Management, 15(5/6), 352-357.
Gagné, M., & Deci, E. L. (2005). Self‐ determination theory and work motivation. Journal of Organizational behavior, 26(4), 331-362.
Gartner (2011). Hype cycle special report evaluates the maturity of 1,900 technologies. Retrieved from http://www.gartner.com/newsroom/id/1763814.
Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7(1).
Harvard Business Review (2013). How Deloitte made learning a game. Retrieved from
https://hbr.org/2013/01/how-deloitte-made-learning-a-g,
Herzig, P., Ameling, M., & Schill, A. (2015). Workplace Psychology and Gamification: Theory and Application. In Gamification in education and business, 451-471. Springer International Publishing.
Hopkins, W. G. (2008). Quantitative research design. Retrieved from http://www.citeulike.org/group/6675/article/3424132.
Hornaday, R. W. (2014). Sex composition, cohesion, consensus, potency and performance of simulation teams. Developments in Business Simulation and Experiential Learning, 28.
Jakubowski, M. (2014). Gamification in Business and Education–Project of Gamified Course for University Students. Developments in Business Simulation and Experiential Learning, 41.
Jansz, J., & Martens, L. (2005). Gaming at a LAN event: the social context of playing video games. New media & society, 7(3), 333-355.
Johnson, R. T., & Johnson, D. W. (1986). Action research: Cooperative learning in the science classroom. Science and Children, 24, 31-32.
Johnson, R. T., Johnson, D. W., & Stanne, M. B. (1986). Comparison of computer-assisted cooperative, competitive, and individualistic learning. American Educational Research Journal, 23(3), 382-392.
Kalning, K. (2008). The anatomy of the first video game. Retrieved from
http://www.nbcnews.com/id/27328345/ns/technology_and_science-games/t/anatomy-first-video-game/
Kankanhalli, A., Taher, M., Cavusoglu, H., & Kim, S. H. (2012). Gamification: A new paradigm for online user engagement. 33rd International Conference on Information Systems.
Hamari, J., Koivisto, J., & Sarsa, H. (2014) Does gamification work?--a literature review of empirical studies on gamification. In System Sciences (HICSS), 2014 47th Hawaii International Conference on (pp. 3025-3034)
Kumar, J. (2013). Gamification at work: Designing engaging business software. In International Conference of Design, User Experience, and Usability, 528-537. Springer Berlin Heidelberg.
Lainema, T., & Makkonen, P. (2003). Applying constructivist approach to educational business games: Case REALGAME. Simulation & gaming, 34(1), 131-149.
M2 Research (2011). Gamification Market to Reach $2.8 Billion in US by 2016, According to New Findings. Retrieved from http://m2research.com/Gamification.htm.
Meder, M., Plumbaum, T., & Hopfgartner, F. (2013). Perceived and actual role of gamification principles. In Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing (pp. 488-493). IEEE Computer Society.
Mitnick, K. D., & Simon, W. L. (2011). The art of deception: Controlling the human element of security. John Wiley & Sons.
Mollick, E. R., & Rothbard, N. (2014). Mandatory fun: consent, gamification and the impact of games at work. The Wharton School research paper series.
Morrison, E. W. (1994). Role definitions and organizational citizenship behavior: The
importance of the employee's perspective. Academy of management journal, 37(6), 1543-1567.
Opdenakker, R. (2006). Advantages and disadvantages of four interview techniques in
qualitative research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 7(4).
Perryer, C., Scott-Ladd, B., & Leighton, C. (2012). Gamification: Implications for workplace intrinsic motivation in the 21st century. AFBE J, 371–381.
Puhakainen, P., & Siponen, M. (2010). Improving employees' compliance through information systems security training: an action research study. Mis Quarterly, 757-778.
Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of games for educational purposes: A review of recent research. Simulation & Gaming, 23(3), 261-76.
Romero, M., Usart, M., Ott, M., Earp, J., & de Freitas, S. (2012). Learning through playing for or against each other? Promoting collaborative learning in digital game based learning. Learning, 5.
Ruhi, U. (2015). Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification. Technology Innovation Management Review.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist, 55(1), 68.
Ryan, R. M., & Deci, E. L. (2010). Self-determination theory. Rochester: University of Rochester.
Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of Human-Computer Studies, 74, 14-31.
Shell (n.d.), Our Goal Zero Ambition. Retrieved from
http://www.shell.com/sustainability/safety/our-approach.html
Waddell, J. C., & Peng, W. (2014). Does it matter with whom you slay? The effects of
competition, cooperation and relationship type among video game players. Computers in Human Behavior, 38, 331-338.
Webb, E. N., & Cantú, A. (2013). Building internal enthusiasm for gamification in your
organization. In International Conference on Human-Computer Interaction, 316-322. Springer Berlin Heidelberg.
Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249 - 265.
Zhu, Y. (2013). Individual behavior: In-role and extra-role. International Journal of Business Administration, 4(1), 23.
Appendix A: Game description
The game used in this study was what is called a mobile, turn-based game. Players were asked to download the game to their mobile device and create an account using their work-related e-mail address. After an account was created, players had to choose whether they wanted to team up with a random colleague or invite a specific colleague. Teams of two were connected to each other, and players had to choose between playing a social engineer or a hacker. Social engineers were very capable in social interactions and getting information or tools from people but had little technical skills. Hackers were very good with technical aspects like breaking into computer systems, but had little to no social skills.
The game itself is a platform game, each level showing a map of an office floor with several rooms and tools in them. Both players can take turns, and during their turn walk around, talk to people or look at objects placed in the rooms. Characters in the game can give players information, this is something the social engineer could do, and objects can contain useable tools or offer an opportunity to access the company’s computer network. The hacker has to do these things. In between each turn, players can communicate via a small text box in the game and
exchange tools they have found. The goal of each level was to reach the exit in for each floor by hacking systems and talking to people, so both players could access the next floor.
Appendix B: Consent form Informed consent: English
I hereby state that I have been clearly instructed on the nature and method of this study, as earlier explained to me in the invitation I have received for this study.
I voluntarily participate in this study, and with that have the right to withdraw my consent at any time during the study without having to provide a reason for doing so. I am aware of the fact that the results and information of this study will be published anonymously, and that my personal information will not be shown to third parties without my clear consent.
I understand that any audio from this study will only be used for further analysis. I also am aware of my rights to request to read this study or parts of it at all times, and where necessary make changes to the quotes that have been included in this study from my interview.
If I want more information, now or in the future, I can turn to Iris Rigter via +31633609596. For any complaints with regards to this study I can contact the Ethics Commission of ASCoR via the address: ASCoR secretariaat, Commissie Ethiek, Universiteit van Amsterdam,
Name participant:
Date: Signature participant: For the researcher
I have given a verbal and written explanation of this study. I will answer any further questions on this study to the best of my ability. The participant will not experience any negative
consequences should he/she choose to stop participation in this study or withdraw his/her data.
Name researcher:
Date: Signature researcher:
Informed consent: Dutch
Ik verklaar hierbij op duidelijke wijze te zijn ingelicht over de aard en methode van het onderzoek, zoals uiteengezet in de uitnodigingsmail voor dit onderzoek.
Ik stem geheel vrijwillig in met deelname aan dit onderzoek. Ik behoud daarbij het recht deze instemming weer in te trekken zonder dat ik daarvoor een reden hoef op te geven. Ik besef dat ik op elk moment mag stoppen met het onderzoek. Ik weet dat de gegevens en resultaten van het
onderzoek in overleg geanonimiseerd gepubliceerd zullen worden. Mijn persoonsgegevens worden niet door derden ingezien zonder uitdrukkelijke toestemming.
Ik begrijp dat audiomateriaal of bewerking daarvan uitsluitend voor analyse en/of
wetenschappelijke presentaties zal worden gebruikt. Ook behoud ik het recht om het onderzoek ten alle tijden in te zien en waar nodig door mij gedane uitspraken te corrigeren.
Als ik meer informatie wil, nu of in de toekomst, dan kan ik me wenden tot Iris Rigter via 0633609596. Voor eventuele klachten over dit onderzoek kan ik me wenden tot het lid van de Commissie Ethiek namens ASCoR, per adres: ASCoR secretariaat, Commissie Ethiek,
Universiteit van Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, 020-525 3680; ascor-secr-fmg@uva.nl.
Naam deelnemer:
Datum: Handtekening deelnemer:
Voor de onderzoeker
Ik heb een mondelinge en schriftelijke toelichting gegeven op het onderzoek. Ik zal resterende vragen over het onderzoek naar vermogen beantwoorden. De deelnemer zal van een eventuele voortijdige beëindiging van deelname aan dit onderzoek geen nadelige gevolgen ondervinden.
Naam onderzoeker:
Datum: Handtekening onderzoeker:
Appendix C: Interview guide Introduction
First of all, thank you for partaking in this interview. As previously mentioned I will be asking you a few questions on your experiences with Elevator the Game. Feel free to answer them as you see fit, there are no right or wrong answers. If you feel uncomfortable with any of the questions asked, please let me know. If you want to stop the interview at any point, please indicate this. I would also like to record the conversation as we have previously discussed. The recordings will be stored anonymously and will only be listened to by me, so I can use them for further analysis. Do you still agree with me recording this conversation? Then I would like to ask you to carefully read this informed consent document for a minute, this states that you have been
fully informed about the kind of research I am conducting and that you are aware of the fact that your anonymity will be safeguarded. If you understand the text and agree to participate, please sign the document and then we will continue.
Once agreed upon, start interview.
First: Experience of gamification
The aim of this topic is to gain an understanding of how the respondent experienced the game. Did he or she enjoy it? This can be relevant for the influence the game has on awareness and behavior later on, people who do not enjoy the game may feel different about the process than people who did enjoy the game.
I would first like to ask you a quick question on how you played the game. ● Which role did you take on in the game?
○ Possible follow-up question: Why did you choose this role?
I would also like to know how you experienced the game while you were playing it and what your opinion is on games and gamification.
● What was your initial reaction when you heard about playing this game? ○ If short answer is given: And why did you have this reaction/feeling?
● Can you tell me about your experiences while playing the game?
● How did you feel while playing the game?
○ If short answer is given: Could you please explain this feeling a bit more?
● How did you feel about playing this game with a colleague?
○ Possible primers are: comfortable, uncomfortable, happy, frustrated etc. ○ Follow-up question: What aspects of the game made you feel [insert feeling
mentioned]?
● How did you feel about the turn-based aspect of this game?
○ Possible follow-up: What part of it did you find [insert initial reaction]?
● If you had to rate this game between 1 being the lowest and 10 being the highest, what would you rate it and why?
● What aspects of the game make you say that it is [insert answer to previous question]?
Second topic: Risk awareness after playing the game
The aim of this topic is to gain an understanding in the risk awareness (related to information security) of the respondent after he or she played the game. Has he or she gained any new knowledge? Have attitudes towards information security changed?
As you have probably noticed, Elevator the Game was focused on your knowledge of and experience with information security and cyber risk. For the following questions I would like to focus on this:
● What information- and cyber risks have you become more aware of?
○ Possible follow-up questions: Could you please elaborate a bit more on that? What specific facts/points did you take away from the game?
If respondent indicates that he/she has gained awareness on cyber risk through the game: ● What parts of the game did you find particularly helpful in creating this awareness?
● How do you feel those parts helped you with becoming more aware of cyber risk?
If respondent indicates that he/she has not gained awareness on cyber risk through the game: ● What do you think is the reason you didn't learn anything from this game?
● How do you think this game can be improved?
Continue with following question:
● Do you now think differently about cyber risks than before you played the game? ○ Follow-up question: How do you think the game has helped with this [mention
difference if needed]?
Third topic: Behavior after playing the game
The aim of this topic is to gain an understanding in possible behavioral changes of the respondent after he or she played the game. A change in awareness is frequently linked to a
change in behavior, so these questions can provide more insight into the effects of the game on awareness and actions.
As well as a focus on your knowledge about cyber risk, this game also looks at possible behaviors that can make you handle information security and cyber risks better. The following questions are aimed towards that:
● What behaviors or actions to avoid cyber risks were mentioned in the game? ○ Follow-up question: How were these behaviors incorporated in the game?
● Do you expect to change your behavior?
If respondent indicates that he/she will change behavior after playing the game: ● Why do you expect this behavior to change?
○ Follow up question: What parts of the game did you find particularly in helping you change behavior?
If respondent indicates that he/she will change behavior after playing the game: ● Why don’t you expect this behavior to change?
End of interview
Great! These were all the questions I have. Is there anything else you would like to add?
[possible response from interviewee] Thank you very much for participating in this interview. In the next few months I will be analyzing this and other interviews and writing a conclusion on my thesis. If you would like to receive this thesis after it has been finalized, please let me know and I will send it to you. If you have any questions on this interview, my research or for any reason wish to withdraw your interview please contact me whenever you want.