• No results found

Work hard, play hard : can business games increase risk awareness?

N/A
N/A
Protected

Academic year: 2021

Share "Work hard, play hard : can business games increase risk awareness?"

Copied!
42
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

Iris Rigter (10331336) 31-01-2017

Master Corporate Communication Word count: 7492

Graduate School of Communication Master's Thesis

University of Amsterdam Supervisor: Joost Verhoeven

Work hard, play hard:

Can business games increase risk awareness?

(2)

Gamification, serious games and business games are a growing business and used for things such as training and education. While much research into these developments has been done, many of these studies have used quantitative approaches. Because of this, not much is known about the actual processes and game-elements that make such games effective. To offer a deeper insight into these aspects, this study has looked at the question of how the use of gamification can raise awareness and promote certain behaviors in a qualitative way. With the use of a case study through semi-structured, individual interviews this research has attempted to uncover the underlying processes and game-elements that make a business game effective. Elements that were found to influence the effectiveness of a business game were the emotional experience a game evokes, the relevance of the game to its players, the discourse of players during and about the game and the context in which the game was played.

Introduction

Organizations nowadays face all kinds of risks. A few of the top risks that can endanger organizational performance are information security, data privacy, poorly planned strategic changes and a lack of risk awareness (Audit & Risk, 2015). By taking measures like raising awareness about existing risks or promoting compliance with both organizational and national regulations, organizations can prevent risks from harming them (Bulgurcu, Cavusoglu & Benbasat, 2010). Shell, for example, has a 'Goal Zero' framework (Shell, n.d.) that includes safety standards and training aimed at preventing any harm happening to their personnel, their processes and their clients.

Raising awareness and increasing knowledge of risks that can harm a company can be done in many ways, like through a training program (Puhakainen & Siponen, 2010). An

(3)

or the use of game-like elements, has already been linked to improved knowledge acquisition and increased behavioral skills (Connolly et al., 2012). Serious game Friend Inspector, for example, creates an environment in which users can playfully discover their privacy setting on Facebook (Cetto et al., 2014). The game has shown to successfully increase privacy awareness.

The gamification market is expected to reach over 2.8 billion dollars in direct spending in the US alone by the end of 2016 (M2 Research, 2011). Surprisingly, over 80% of gamified applications fail to meet their business objectives (Webb & Cantú, 2013). This may be because not much is know about the underlying processes that make gamification effective. As a

consequence, the design of failing games is insufficiently tailored towards the aims of the game and its target audience. Roughly 84% of studies into the effects of gamification use quantitative methods like experiments or surveys (Connolly et al., 2012). Most studies thus look at the generalized relation between gamification and outcome variables like job performance or knowledge acquisition. While quantitative research offers the opportunity to study a large number of respondents, it lacks the ability to ask in-depth questions regarding the topic

(Hopkins, 2008). This means that there is little research on the aspects of the game that underlie the positive effects gamification has on raising awareness and knowledge. For example, do people who play with a colleague rather than alone notice more positive effects? This study therefore aspires to look at the underlying processes and game elements that make gamification effective by asking the question:

How does the use of gamification raise awareness and promote certain behaviors?

This study will attempt to add to existing knowledge on the effects that gamification has on awareness and behavior. By offering an in-depth look into game-elements or personal

(4)

experiences that influence the effects of a game on awareness, a possibility opens up for further research. In addition to the scientific field, organizations may benefit from this research. Many organizations jump on the gamification bandwagon without being fully aware of its effects, as can be seen by the high failure rate of business games (Webb & Cantú, 2013). By generating knowledge on the processes and elements in the game that create the effectiveness of

gamification, this study may increase understanding into processes that make gamification effective. A better understanding can help organizations to improve the development of business games, lowering the failure rate and making gamification more effective.

Theoretical framework A definition of gamification

In spite of the first video game being designed by William Higinbotham in 1958 (Kalning, 2008), gaming only became popular in the 1970s with the introduction of games like Space Invaders and Breakout (Donovan, 2010). While games were first only seen as entertainment, their educational value soon became apparent (Randel et al., 1992).

With an increase in interest for educational games, came the rise of gamification. A term first used by Nick Pelling in 2002 (Fitz-Walter, 2013). Despite its introduction in the early 2000s, it was not picked up by both research and businesses until 2010 (Jakubowski, 2014). A resurgence in interest in 2011 led to increasing research and with that several attempts to define gamification (Fitz-Walter, 2013). In spite of great diversity in definitions, there are some

similarities in most of them. All refer to the use of elements of game design, and a majority also mentions the use of these elements in a ‘non-gaming’ context. Therefore this study will use an adjusted version of the definition as given by Domínguez et al. (2012): 'Gamification is the use of game design elements, such as leaderboards and levels, and game mechanics in non-game

(5)

contexts, like education or employee training'. This definition fits both the similarities mentioned

before, and is broad enough not to exclude any type of gamification used within organizations.

Business games

There are different kinds of business games available for various aims (Kankanhalli et al., 2012). All games can be divided into solo play, cooperative play or competitive play (Waddel & Peng, 2014). Solo mode only lets players interact with the gaming environment and with

game-controlled characters (Waddel & Peng, 2014). A game element that is often used in solo mode is a personal avatar, which offers participants the option to determine who or what they want to be in-game (Blohm & Leimeister, 2012) and increases personal motivation (Deterding et al., 2011). Social play, however, offers participants an environment in which they can either collaborate or create a competition (Jansz & Martens, 2005). In collaborative play people work towards a collective goal (Brewer & Gardner, 1996) or support each other in reaching certain goals (Bosworth, 2012). A so-called viral loop usually indicates the use of a collaborative game. In such a loop people need to invite other players and work together with them to progress in the game (Kankanhalli et al., 2012). Competitive play on the other hand, happens when one

employee attains his or her goals at the expense of someone else (Johnson et al., 1986). This can be done by adding a leaderboard to games (Kankanhalli et al., 2012). In all types of games, scoring systems such as points or badges can be used (Meder, Plumtbaum and Hopfgartner, 2013). This can offer participants a sense of achievement (Blohm & Leimeister, 2013) and with that a positive reinforcement to continue using the application (Dale, 2014). Collaboration during a game has been shown to lead to prosocial behaviors (De Simone & Riddle, 2001), whereas competition has been linked to aggression and conflict (Waddell & Peng, 2014).

(6)

Using these types of games and elements, companies can create a business game that fits their aims (Callan & Landers, 2012). For example, Deloitte uses gamification to motivate employees to learn on the job about among other things management styles and interpersonal communication (Harvard Business Review, 2013). With their Deloitte Leadership Academy platform, participants are asked to complete certain trainings to improve their management and work-related skills and in return can collect badges and see their scores on the leadership board. This type of gamification uses the game-elements of a scoreboard and the rewards in the form of badges.

Effectiveness of business games

Throughout the years, research has frequently shown that gamification can lead to positive results (Connolly et al., 2015). A concrete example of this can be seen in the KEAS wellness program, which promoted better nutrition and more exercise among employees using a game (Bosworth, 2012). The use of a gamification platform not only triggers motivation, but also increases employee engagement (Bosworth, 2012). Other studies have found that gamification can have a positive impact on work-related motivation (Ruhi, 2015), on enjoyment (Herzig, Ameling & Schill, 2015) and on group cohesion (Hornaday, 2014). A meta-analysis done by Connolly et al. (2012) also shows an improvement of knowledge acquisition through the use of gamification.

In spite of these positive results, there is no consensus yet on whether gamification actually has positive effects. Dominguez et al. (2013) in their study showed that gamified

activities can actually hinder the understanding of theoretical concepts when applied to learning, making gamification unsuitable for certain kinds of training. In addition, enjoyment can decrease as employees use the game for a prolonged period of time. This can point to a novelty value of

(7)

gamification (Koivisto & Hamari, 2014) or the increase of boredom after using gamification for a while (Ruhi, 2015). Mollick & Rothbard (2014) also warn for what they call the paradox of mandatory fun; an externally-imposed structure of fun does not automatically make the game enjoyable. When gamification is implemented without consent from the employee, it may thus decrease performance. Overall it can be said that research on the effectiveness of gamification paints a mixed picture so far (Seaborn & Fels, 2015).

Risk awareness

Before it can be established how business games create awareness, it is important to define this concept. Risks can harm or endanger an organization and its stakeholders. A way to minimize the negative effects risk has is by creating employee awareness (Bulgurcu et al., 2010). Risk awareness encourages workers to 'stop and think before starting work, so that they may identify hazards and take action to control the associated risks' (Borys, 2009). It can be defined as an employee of the organization feeling, experiencing or noticing risk that can potentially harm the organization (Bin Ishaq Alseiari, 2015).

An example of employee awareness that is becoming increasingly important for organizations is that of cyber crime awareness. Organizations rely heavily on information

systems and any damage to these systems can have severe consequences, like monetary losses or corporate liability (Bulgurcu et al., 2010). Currently employees are the weakest link in these information systems (Mitnick & Simon, 2002), as they are prone to making mistakes or

disregarding regulations (Bulgurcu et al., 2010). Raising employee awareness through education and training can reduce these problems, making it a valuable consideration for organizations (Furnell et al., 2002)

(8)

Method

Research design

To dive deeper into the underlying processes of business games, the reasons why they are effective and the elements that make them effective, this study has used a qualitative approach. Qualitative research offers the opportunity to ask people in-depth questions about their

experiences with business games, and thus offers a better look into the processes at work when someone uses a business game (Flick, 2009).

Participants (N = 20) were asked to play a mobile, turn-based business game that aims to develop security awareness on cyber risks. The game is a platform game, with each level

showing a map of an office floor with several rooms and tools in these rooms. By using tools and talking to characters in the game, participants had to make their way to the exit. The game used a collaborative approach: players had to choose one of the aforementioned roles, and based on the role they chose they had certain skills. Their co-player then had a different set of skills, and by working together they could progress through the game as a team. The game was designed by a game-design company. The researcher was allowed to use this game for study, but the game itself cannot be named and information had to be used anonymously. A further description of this game is provided in Appendix A. The experiences and opinions of participants were discussed during a semi-structured interview with the use of an interview guide.

Participants

There were several prerequisites for participants to join this study. Participants needed to work at an organization that faces security risks. In addition, participants needed to be available between the period of October 31st and November 25th to participate in the game and the interview afterwards.

(9)

Potential participants were contacted by e-mail via snowballing. An initial e-mail was sent out to acquaintances of the researcher, who in turn provided information of possible

participants to be invited. An invitation was sent out with a short explanation of this study and its topic. It was also made clear that participation was anonymous, and that participants could withdraw from the study at any time without further consequences. After agreeing to join the study, 26 people received an email with instructions on how they could play the game. After signing up, 6 people indicated that they did not have sufficient time to participate and decided to leave the study. The remaining participants (N = 20) who joined this study came from two organizations. Half of the participants (N = 10) worked at a consultancy firm, the other half of the participants (N = 10) was employed by a digital agency. Follow-up interviews were

scheduled with all participants in the two weeks after playing the game. Interviews were held at the organizations where the participants worked. Before the interview, participants were asked to sign an informed consent form. This form can be found in Appendix B, and included information on the voluntary conditions of the study and the recording of the interviews for further analysis. No incentive was given to participants of this study.

During the interviews some participants indicated that they played a large number of levels of the game, but did not completely finish it. As this study looks at the underlying processes and game-elements that make business games effective, these participants were still included in this study. This because not finishing the game with valid reasons could still provide this study with relevant information on game-processes and elements that may make it difficult to continue the game.

(10)

As this study has used qualitative, semi-structured interviews to collect data, an interview guide was set up as a research tool. This interview guide consists of 18 questions, and is composed of 4 themes. The themes discussed were personal information, experiences with the game, risk

awareness before and after the game and behavior after the game. Easier questions were asked first to put the interviewee at ease. The entire interview guide can be found in Appendix C. Each interview was recorded and then transcribed. With transcribing all interviews are listened to and written down word for word, for further analysis. Transcripts were then coded by using the grounded theory approach (Flick, 2009). This approach divides coding into three phases: open coding, axial coding and selective coding. Through open coding certain codes are added to sentences or word groups, and axial coding then aims to look for connections between the different codes that have been found in the transcripts. These connections create certain categories within the text, for example positive sides of playing with a colleague and negative sides of playing with a colleague. Selective coding analyses these categories and looks for examples and exceptions per category, so that a fully inclusive analysis of the transcripts can be provided. A founded theory approach has been chosen as this approach is known as a fitting tool to uncover explanations of processes and experiences from transcripts (Flick, 2009). The codes found in the transcripts were categorized in four themes; emotional experience, relevance, discourse and context of play. These themes will be discussed in the results section of this research.

Reliability

To ensure a reliable study, several measures were taken. First of all, individual and anonymous interviews were chosen to minimize the possibility of socially desirability. Making use of

(11)

be traced back to them (Opdenakker, 2006). Secondly, notes taken during the interview can help with the later analysis of the transcripts. When an interviewee seemed nervous or uncomfortable, and this is noted during the interview, it offers the possibility to better understand the context in which the answers were given. Lastly, the use of an interview guide ensures that all participants are asked the same questions. This way the answers to those questions can be properly

compared, making analysis more reliable.

Results

In this results section several topics will be discussed to explain the way business game

influenced risk awareness. These topics address the underlying processes and game-elements of effective serious games: emotional experience, relevance, discourse and context of play. First, however, some information will be given on how respondents rated the game they had to play as this experience may influence how they further answered the questions asked during the

interview.

Appreciation of the game

Respondents’ answers paint a very mixed picture on how employees rate the game they played. Participants were asked to rate the game between 1 and 10, with 1 being the lowest and 10 being the highest. The lowest grade given was a 3. According to the respondent:

“Because I can say that the game is designed so you can play at your own pace, but because you are dependent on another person that is extremely annoying. […] I think the game is too difficult for it to reach its goal, it is such a slow way to learn about cyber

(12)

On the other side of the spectrum, however, one of the respondents rated the game with a 9:

"Well, I'm never allowed to say 10, so that means I would go with a 9. I have really

enjoyed playing this game, and I am eagerly waiting for new levels to be released. [...] It

has really made me more aware as well." (Respondent 6)

Generally, participants rated the game with grades between 6 and 8, indicating that most were quite positive about the game but that some improvements could be made. Frequently mentioned positive aspects of the game were the design of the game and the fact that the game used humor. Negative aspects mentioned were that the notifications of the game didn’t work properly and that the game was quite slow to play.

Emotional experience

Both the positive and negative aspects of the game evoked an emotional reaction from respondents in this study. One of the emotions that was most frequently experienced was

frustration. There were a number of game elements that led to these feelings. First of all, several respondents mentioned that they did not find the game user-friendly. Especially the beginning of the game, right after the tutorial, was said to be complicated to understand:

At first it took some time getting used to. Like ‘okay, how exactly does this work?’. You can look at the instruction video of course, but everyone understands that straight away.

The point is more, like, how does the hacking and social engineering work?

(13)

Some respondents even indicated that they found the gameplay difficult and therefore struggled to progress in the game:

“I think that in the beginning, well, what I mostly noticed was that we had the feeling that we didn’t understand the game. We have, I think, in the first week. Well, we are now two weeks in and we have finally finished the game” (Respondent 4)

A second, frequently mentioned point of frustration was the turn-based aspect of this game. Even though the game was designed for people to play occasionally during small breaks, only little progress could be made per turn. As respondents had to wait for their co-player to finish his or her turn before they could play again, it could take a few hours up to a few days before they could continue the game. For players whose co-player only played at infrequent intervals, this posed a problem:

"I tried playing it in the evening while I was sitting on the couch, uhm, but yeah, then you

can only do one move or a little part of a level and then you have to wait again. And

sometimes that one [other player] can't play and then you want to continue but you can't.

I found that a bit of a killer" (Respondent 3)

Two respondents got so frustrated with this feature that indicated they would prefer to play all

alone, rather than having to wait for someone else. According to some respondents, their

(14)

"[...] it was so unclear to us what was happening and we kept going round and round and

it was really sort of [deep sigh]. Because of that you also learn very little. "

(Respondent 14)

As opposed to the frustration felt by a large number of respondents, another group indicated that they really enjoyed the game while playing it. Several respondents even called the experience fun:

"I found the setting really fun. That was also because there was some humor included,

you know? It wasn't that I felt like I had to play the game again. It was more like, yeah,

I'm going to enjoy playing that game again!" (Respondent 18)

The respondents who said they had enjoyed the game also formed the majority of people who finished all levels. Enjoyment is thus an important factor in keeping people motivated to play. Since finishing all levels means they were confronted with more examples of risk awareness, including humor and fun elements can be seen as an important way to increase the effectiveness of a serious game.

Relevance

In this study respondents were asked to mention the cyber risks they learnt from the game. The majority of respondents mentioned risks that they could connect to their everyday experience. Several other respondents indicated cyber risks that they found in the game and that they could relate to their own experiences at work.

(15)

"Sometimes I sit, not always, but often I sit next to the door. And in this building people

walk in every now and then, and they could be clients or freelancers, but I don't know.

And sometimes the office manager isn't here, and out of politeness I usually ask people

who they are visiting and offer them coffee. But someone could, and that is what you

realize when playing the game, easily say they are from any company. So that is

something I will look out for in the future." (Respondent 10)

The importance of relevance to the players became apparent from this. Especially since one of the respondents expressed frustration about not being able to identify with the game:

"So you have this level that is sort of logistics where you go in with a bus and whatever.

And then I'm like, this is so far from reality. What good is it to me that this happens? I

could not identify with this" (Respondent 11)

When a game is relevant for an employee, this can also contribute to the engagement an employee experiences.

Discourse

In the game respondents played, they were asked to team up with a colleague and take turns to advance in the game. Because of the co-operative element in the game, discourse became an important aspect of the experience. Contact between co-players was maintained in several ways: by playing during working hours whilst sitting next to each other, by using personal online-messaging service or by using work-related communication channels.

(16)

The game is set up in such a way that people can take turns in playing and can play the game whenever they want. This offers players the opportunity to check the game at times they find convenient, for example when they are waiting for the bus or sitting at home. However,

several respondents indicated that they made specific appointments to meet at the office and sit

down to play the game:

"At a certain moment we decided like, what if we communicate better on all the things we've seen in the game? That would make it a lot easier. At some point we even sat down, last week on Friday, to play together. That is when you really make progress in the game, and without that communication I think it might be too difficult" (Respondent 1)

Being able to communicate with your co-player outside of the gaming context turned out to be an

important issue. Other players mentioned the use of personal communication channels such as

Whatsapp, or work-related communication channels like Slack and Skype:

"Eventually we had to use Slack to solve who would play against each other. And then we began, and everyone would still use Slack to communicate as a channel next to it [the game] or from my co-player via Whatsapp. In other words, the fact that we all used Whatsapp and Slack to solve things means that the communication feature in the game doesn't work properly." (Respondent 12)

This need for a constant communication between players got very mixed reactions, ranging from very positive to quite negative. People who were positive about this social feature mentioned that

(17)

they felt they learnt more from playing the game together with a colleague and discussing the game with that person:

"I really like collaboration and working together to solve things. That you can discuss it

afterwards. Because that is what we did, we really talked about how we were going to

complete the next level." (Respondent 6)

In addition, playing with a colleague increased the commitment to the game. When you have someone waiting for you to make a move you are more likely to pick up the game as to not to

disappoint the other, as one respondent mentioned. The downside mentioned was the fact that

you had to wait for your co-player to finish his or her turn. However, most of the respondents

who mentioned this downside of playing together also saw the benefit in having a partner in this

game. One respondent even called it a 'double edged sword':

"I think it helps when you play together, because you also talk about it outside of the game. But it is, uhm, it is a double edged sword. You can't really keep playing and because of that you can't really get into the game" (Respondent 3)

Other respondents offered solutions for this problem. By creating a game that first gave the

opportunity to play alone and get acquainted with the process, and adding the social feature later

on, the game could tap into the best of both worlds and keep players motivated throughout.

(18)

"Maybe they could add a message option to the app. Because you can leave information or a message for the next person, but maybe it would be better if you could do that constantly instead of only after each turn. That way you can also discuss during [the game]." (Respondent 20)

Most respondents see an added value to a social feature within a game, but it is absolutely

essential that the social feature works properly and offers sufficient features to communicate

within the game.

Context of play

The turn-based and mobile aspects of this game gave players the freedom to choose when and

where they preferred to play the game. Here a divide can be seen between people who chose not

to play during work hours and people who did play during work hours. Some respondents

mentioned that playing the game did not feel like part of their tasks, and thus mostly played

outside of office hours:

'In the beginning I didn't play it much because I didn't want to play during work hours, but when I was at home watching the news I would think about playing the game at the same time. You know?" (Respondent 11)

Other players made the decision to sit together and play the game during scheduled meetings.

This group also had an overall more positive opinion of the game. People who played during

work hours likely did not see the game as something they had to do on top of their work, but as a

(19)

become more acquainted with certain risks, but do not feel like they are given time to do so

during work, they might experience the game as an infringement of their personal time.

Conclusion

Through a case study of a mobile-based business game, this study has looked at the relationship

between these games and learning: the underlying processes and game-elements that can explain

the impact of a business game on awareness. Twenty respondents were interviewed about their

experiences with playing a business game aimed at creating cyber- and information risk

awareness in order to find these processes. Respondents played the business game specifically

for this study. Previous literature on this topic has already set several prerequisites for learning,

which also apply to gamification. Fogg's behavioral model (2009) for example, states that

learning requires motivation to learn, the ability to learn and a trigger to learn. Aspects that can

either support or hinder learning through gameplay such as motivation to play or a lack thereof

were also found in this study. Firstly, the emotional experience evoked can be an important

contribution to the amount of awareness that a serious game establishes. Previous studies have

already shown that games are motivating when it comes to learning because of their impact on

emotional areas of players (Dominguez et al., 2013). This study, however, shows that the kind of

emotion felt also has an impact. Once a game creates a feeling of frustration, this can negatively

influence game-motivation and with that the effectiveness of a serious game. On the other hand,

positive experiences such as fun can increase motivation to play.

Secondly, the relevance of a game to its players has an influence on game effectiveness.

Once employees feel like the situations they encounter in the game are relevant for their own

life, this can increase their motivation to play the game. An explanation for this can be found in

(20)

When employees see the relevance in a business game they will most likely also find it more

interesting, as they can apply the knowledge from the game to their everyday life. This higher

motivation leads to more gameplay and through that to raised awareness, which indicates a more

effective business game.

A third game-element that can have a positive effect on creating awareness is a social

element and the promotion of discourse about the game. It helps people to connect and discuss

the game with others, making the topic of the game more top of mind. This discourse, however,

can be seen as a double edged-sword. Once the social element of the game is lacking the inability

to communicate with a co-player can create frustration, which can lessen the effect a business

game has on awareness. A final element that is important for effective use of business games is

the context of play. Since a business game is often imposed on employees by

upper-management, there is a risk of the game feeling like yet another task. If this task cannot be done

during office hours, it might create feelings of resentment towards the game. To prevent this

negative spiral from happening, it is important for employees to get sufficient time to play

business games during work-time.

Discussion

Motivation to play

Before a business game can create awareness, it is important for employees to actually (continue

to) play the game. An important factor here is motivation, which is often connected to the theory

of self-determination (Ryan & Deci, 2000). This is a theory of human motivation ‘concerning

people’s inherent growth tendencies and their innate psychological needs’ (Francisco et al., 2012) that divides motivation into intrinsic and extrinsic. Intrinsic motivation can occur when a task is interesting or enjoyable (Perryer, Scott-Ladd & Leighton, 2012) and is a natural drive in

(21)

people to seek out challenges and aim to further develop themselves (Ryan & Deci, 2000). Extrinsic motivation occurs when a task leads to external desirable outcomes, for example an award or a bonus (Dale, 2014). Three variables are said to influence the level of intrinsic motivation a person experiences: autonomy, competence and relatedness (Ryan & Deci, 2000).

Autonomy 'involves acting with a sense of volition and having the experience of choice' (Gagné

& Deci, 2005). The feeling of competence indicates a 'state of having requisite or adequate

ability or qualities' to complete a task (Ryan & Deci, 2010). Finally, relatedness refers to 'the

desire to interact with and be connected to other people' (Ryan & Deci, 2000). This study has

shown several game-elements and processes that are relevant in creating motivation through

these variables. First, the emotional experience has an influence on whether employees want to

play a business game. Respondents who enjoyed playing the game were motivated to keep

playing, and all of them finished the entire game with a positive attitude. On the other hand,

respondents who indicated that they felt frustrated while playing the game mentioned that they

often quit playing the game after experiencing several levels. A lack of user-friendliness and the

perceived difficulty of the game were important reasons for this. These feelings of frustration in

turn led people to feel like they did not understand the game, which negatively influenced their

feelings of competence. As the self-determination theory mentions, competence is an important

predictor of motivation (Ryan & Deci, 2000). It can thus be stated that emotional experiences

influence motivation, with enjoyment leading to more motivation whereas frustration leads to a

lack of motivation.

A second game-element that influenced players' motivation was the relevance of its

content. Respondents connected in-game experiences to their own reality, indicating that they

(22)

value in the serious game they are playing and the benefits they can reap from the game, their

motivation to play will go up as it gives them another reason to finish the game. Using a game

that is relevant for players has already been linked to more effective use of serious games

(Blohm & Leimeister, 2013). It is even the center argument of the theory of player-centered

design (Kumar, 2013). This design process puts the player at the center of the design, by first

understanding the player and his/her motivation to play (Kumar, 2013). By keeping the player in

mind when developing the game, you create a serious game that is relevant for the player and

will thus increase motivation.

Creating discourse within or about the game can also increase motivation to play when

implemented correctly. On the one hand playing together with a colleague showed to be an

important motivator as players did not want to let their colleague down, and because they

enjoyed playing the game with someone else. This can also be traced to relatedness in the

self-determination theory, which states that the desire to connect to other people is an important

motivation. The game offers players an opportunity to connect with their colleagues, as they are

obliged to play the game with someone else. According to the self-determination theory, that in

turn should lead to higher motivation. On the other hand, however, the waiting period between

players’ moves was said to be demotivating as it takes the speed out of the game and creates feelings of frustration when players have to wait for a long time.

A final factor that influenced players' motivation was the context in which they played

the game. Players who played during work hours had an overall more positive attitude towards

the game, and these feelings of enjoyment increased their motivation to play. People who felt

like they had to play the game outside of work hours, on the other hand, seemed less enthusiastic

(23)

theory of self-determination. An important aspect in this autonomy is feeling like you have a

choice. When employees are told that they have to play a game to learn, this leaves them with

little room to make their own choice. If they play during work hours, however, they get the

opportunity to choose whether they want to play the game or engage in other work activities.

Especially for people why enjoyed the game, being able to choose something fun to do during

work hours can increase motivation to play the game. People who play outside of work hours, on

the other hand, may experience even more limitation in their choices. They are not only

obligated to play the game, but also feel like their choice of how to fill in their spare time is

being taken away as they have to play the game. This feeling of less autonomy in turn leads to a

lower motivation, and can be prevented by offering employees time to play the game during

work hours. Playing the game should thus be seen as in-role behavior by employees, and

facilitated as such. The role theory divides work between in-role and extra-role behavior (Zhu,

2013). In-role behavior refers to the kind of behavior that is ‘described and defined as part of employees’ work and reflected in the official salary system’ (Morrison, 1994). Extra-role behavior, on the other hand, are behaviors that are not part of an employee’s formal job requirements (Zhu, 2013). Forcing employees to take on extra-role behavior, for example

playing a game in their own time, can lead to feelings of resentment (Zhu, 2013). This can have a

negative effect on motivation.

These findings show that a first prerequisite in creating risk awareness through a serious

game is thus to get players motivated. The aforementioned game-elements can either promote or

hinder this motivation.

(24)

Respondents in this study indicated that they thought playing the game together and being able to

discuss the contents of the game made them learn more. The collaborative learning theory

supports these observations. Collaborative learning is a method in which people are grouped

together to achieve a common goal (Gokhale, 1995). Through this shared learning, participants

get the opportunity to engage in discussion, which has been linked to higher levels of thought

and better information retainment (Johnson & Johnson, 1986). In this light a business game that

encourages collaborative gameplay can be expected to also stimulate collaborative learning.

Adding a social feature that promotes collaboration may thus be one of the features that makes a

business game effective in creating risk awareness. These assumptions are supported by earlier

studies that have already shown that players who worked in groups learned more than those who

worked separately (Wouters et al., 2013) and that cooperation is a key factor in both motivation

and engagement (Romero et al., 2012).

Respondents also mentioned the relevance of the game as something that made them

remember the content. Several even said that remembering certain lessons from the game made

them change their behavior. A similarity between players’ own reality and the game-reality can thus be seen as an important factor in creating risk awareness. Generating new knowledge based

on what you already now, is an important foundation of the constructivist learning theory

(Dewey, 1986). This theory states that people learn by experiencing things, reflecting on those

experiences and connecting this new information to prior knowledge (Fosnot & Perry, 1996).

Respondents in this study indicated that they already had some knowledge about the situations

created in-game from their work experience. The game then enabled them to experience these

(25)

they could connect to their existing knowledge. By adding to this knowledge, people learnt more

on the topic of (cyber) risks and thus increased their risk awareness.

Implications of this study

From a scientific standpoint this research offers a different perspective on the effectiveness of

business games than most studies on this topic. The use of a qualitative study offers a more

in-depth overview of aspects that are important to the effectiveness of a game. The concept of

emotional experience and its link to intrinsic motivation has already been established in several

other studies (Herzig, Ameling & Schill, 2005) as well as the fact that a business game has to be

relevant to employees to be effective (Blohm & Leimeister, 2013) and the idea that collaborative

learning increases effectiveness (Johnson & Johnson, 1986). This study thus supports earlier

findings in these regards. However, the proposition that business games are more effective when

they are seen as in-role work behavior rather than an extra task is added knowledge to the field of

business games. Future studies can use a quantitative approach to see if this concept is also found

in larger groups of respondents, and with that expand on the generalized knowledge there is of

processes that influence business gaming effectiveness. In addition to these findings, this study

also uncovered some game features that can enable effective business gaming. First, a clear

manual or help desk while playing the game seem to be important factors in preventing

frustration to keep players motivated. Second, a communication feature that helps players discuss

the gameplay in-game can increase motivation and enjoyment.

From a practical standpoint, this study can be of great relevance for both organizations

that create business games and organizations that are considering the design and implementation

of such a game. Frustration was found to have a negative effect on game effectiveness. A

(26)

clear instruction before the game and a possibility to ask for help during the game. This can be done by adding a manual or offering help desk services with support employees. In addition, the way employees will play the game has to be taken into account during the design. Questions like how and when people will play and what features are necessary to avoid frustration need to be asked before the game is implemented. Another finding is that topics and games that employees see as relevant to their own life get more attention, which positively influences risk awareness. It is thus important for a company to ensure that a business game is relevant for its employees. Especially in larger organizations with bigger and more diverse teams, this can pose a challenge. As different kinds of employees have different preferences and even more so different fields of interest, creating a game that can be personalized based on these characteristics might be crucial to an effective implementation. Games can for example be tailored per employee or group of employees, or can offer a large number of modules so that employees can choose depending on their interests. In addition, this study found that it is advisable for organizations to frame business games implemented as an in-job task rather than an extra task and offer time during work hours to play the game. This way people do not feel like they have to take on more work outside of office hours, which can prevent frustration and resistance from occurring.

For studios that design business games, these elements are of value in the creation of a

good game. Corporations that want to use a business game for their employees, know that they

have to look at a well-designed game to prevent frustration, to implement games that employees

will see as relevant to their own experiences and to promote the game as an in-job task. This can

increase the effective implementation of business games, which will mature the domain and

make it a more attractive way of learning for organizations.

(27)

The use of an actual business game played by employees of different companies specifically for

this study combined with the fact that face-to-face interviews were used in which personal

experiences of participants could be discussed, has led to a high external validity. In addition,

this study offers insights into the processes and game-elements that can influence the

effectiveness of business games. However, it is also important to note some limitations of this

research. Due to a short time-span and miscommunication between players, several respondents

indicated that they had not completely finished the game. Respondents in this study were given

two weeks in which they had to finish the game, and for some respondents this time-span turned

out to be too short. It can be said that these unfinished games create an incomplete picture of the

elements and processes that make a business game effective. However, players who did not

finish the game still had relevant experiences and opinions. Their answers are therefore still

included in the results. In addition, when organizations implement a business game among their

employees they may also encounter a certain drop-out rate. Participants in this study not

finishing the game in time can thus also be a representation of the real-life context in which these

games are played.

A second limitation of this study lies in the interview questions that were used. While the

questions did bring to light several processes that influence effective gamification, it is possible

that not all processes that influence the effectiveness of business games have been uncovered. A

possible explanation for this limitation can be found in the research method used: asking people

to reflect on their personal learning processes through interviews can be extremely difficult for

respondents, as people are often not aware of exactly how they learn. Future studies on this topic

(28)

study on learning processes in business games, such as an observation in combination with

interviews.

In spite of these limitations, this study has provided an insight into the underlying

processes that contribute to effective serious gaming and offers an opportunity for future

research to expand on this.

Literature

Audit & Risk (2015) Top ten risks facing organizations in 2015. Retrieved from

http://auditandrisk.org.uk/features/top-ten-risks-facing-organisations-in-2015

Bin Ishaq Alseiari, K. (2015). The Management of Risk Awareness in Relation to Information Technology (MERIT) (Doctoral dissertation, University of Gloucestershire).

Blohm, I., & Leimeister, J. M. (2013). Gamification: Design of IT-based enhancing services for motivational support and behavioral change. Business & Information Systems Engineering (BISE), 5(4), 275-278.

Borys, D. (2009). Exploring risk-awareness as a cultural approach to safety: Exposing the gap between work as imagined and work as actually performed. Safety Science Monitor, 13(2), 1-11.

Bosworth, A. (2012). KEAS: developing a successful game-based employee wellness program. GAMES FOR HEALTH: Research, Development, and Clinical Applications, 1(3), 189-191.

Bulgurcu, B., Cavusoglu, H., & Benbasat, I. (2010). Information security policy compliance: an empirical study of rationality-based beliefs and information security awareness. MIS quarterly, 34(3), 523-548.

Brewer, M. B., & Gardner, W. (1996). Who is this "we"? Levels of collective identity and self representations. Journal of personality and social psychology, 71(1), 83.

(29)

Callan, R. C., Bauer, K. N., & Landers, R. N. (2015). How to avoid the dark side of

gamification: Ten business scenarios and their unintended consequences. In Gamification in education and business (553-568). Springer International Publishing.

Cetto, A., Netter, M., Pernul, G., Richthammer, C., Riesner, M., Roth, C., & Sänger, J. (2014). Friend inspector: a serious game to enhance privacy awareness in social networks. Proceedings of the 2nd International Workshop on Intelligent Digital Games for Empowerment and Inclusion.

Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661-686.

Dale, S. (2014). Gamification Making work fun, or making fun of work? Business Information Review, 31(2), 82-90.

De Simone, J. J., & Riddle, K. (2011). Does cooperation decrease state hostility? An exploration of cooperative play in a violent, war-themed video game. Paper presented at the Association for Education in Journalism and Mass Communication Conference, St. Louis, MO.

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (9-15).

Dewey, J. (1986). Experience and education. The Educational Forum 50(3), 241 - 252. Taylor & Francis Group.

Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380-392.

(30)

Donovan, T. (2010). Replay: The History of Video Games. East Sussex, UK: Yellow Ant.

Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance improvement quarterly, 6(4), 50-72.

Fitz-Walter, Z. (2013). A brief history of gamification. Retrieved from

http://zefcan.com/2013/01/a-brief-history-of-gamification/

Flick, U. (2009). An introduction to qualitative research. Sage.

Fogg, B. J. (2009). A behavior model for persuasive design. In Proceedings of the 4th international Conference on Persuasive Technology (p. 40). ACM.

Fosnot, C. T., & Perry, R. S. (1996). Constructivism: A psychological theory of learning. Constructivism: Theory, perspectives, and practice, 2, 8-33.

Furnell, S. M., Gennatou, M., & Dowland, P. S. (2002). A prototype tool for information security awareness and training. Logistics Information Management, 15(5/6), 352-357.

Gagné, M., & Deci, E. L. (2005). Self‐ determination theory and work motivation. Journal of Organizational behavior, 26(4), 331-362.

Gartner (2011). Hype cycle special report evaluates the maturity of 1,900 technologies. Retrieved from http://www.gartner.com/newsroom/id/1763814.

Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7(1).

Harvard Business Review (2013). How Deloitte made learning a game. Retrieved from

https://hbr.org/2013/01/how-deloitte-made-learning-a-g,

(31)

Herzig, P., Ameling, M., & Schill, A. (2015). Workplace Psychology and Gamification: Theory and Application. In Gamification in education and business, 451-471. Springer International Publishing.

Hopkins, W. G. (2008). Quantitative research design. Retrieved from http://www.citeulike.org/group/6675/article/3424132.

Hornaday, R. W. (2014). Sex composition, cohesion, consensus, potency and performance of simulation teams. Developments in Business Simulation and Experiential Learning, 28.

Jakubowski, M. (2014). Gamification in Business and Education–Project of Gamified Course for University Students. Developments in Business Simulation and Experiential Learning, 41.

Jansz, J., & Martens, L. (2005). Gaming at a LAN event: the social context of playing video games. New media & society, 7(3), 333-355.

Johnson, R. T., & Johnson, D. W. (1986). Action research: Cooperative learning in the science classroom. Science and Children, 24, 31-32.

Johnson, R. T., Johnson, D. W., & Stanne, M. B. (1986). Comparison of computer-assisted cooperative, competitive, and individualistic learning. American Educational Research Journal, 23(3), 382-392.

Kalning, K. (2008). The anatomy of the first video game. Retrieved from

http://www.nbcnews.com/id/27328345/ns/technology_and_science-games/t/anatomy-first-video-game/

Kankanhalli, A., Taher, M., Cavusoglu, H., & Kim, S. H. (2012). Gamification: A new paradigm for online user engagement. 33rd International Conference on Information Systems.

(32)

Hamari, J., Koivisto, J., & Sarsa, H. (2014) Does gamification work?--a literature review of empirical studies on gamification. In System Sciences (HICSS), 2014 47th Hawaii International Conference on (pp. 3025-3034)

Kumar, J. (2013). Gamification at work: Designing engaging business software. In International Conference of Design, User Experience, and Usability, 528-537. Springer Berlin Heidelberg.

Lainema, T., & Makkonen, P. (2003). Applying constructivist approach to educational business games: Case REALGAME. Simulation & gaming, 34(1), 131-149.

M2 Research (2011). Gamification Market to Reach $2.8 Billion in US by 2016, According to New Findings. Retrieved from http://m2research.com/Gamification.htm.

Meder, M., Plumbaum, T., & Hopfgartner, F. (2013). Perceived and actual role of gamification principles. In Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing (pp. 488-493). IEEE Computer Society.

Mitnick, K. D., & Simon, W. L. (2011). The art of deception: Controlling the human element of security. John Wiley & Sons.

Mollick, E. R., & Rothbard, N. (2014). Mandatory fun: consent, gamification and the impact of games at work. The Wharton School research paper series.

Morrison, E. W. (1994). Role definitions and organizational citizenship behavior: The

importance of the employee's perspective. Academy of management journal, 37(6), 1543-1567.

Opdenakker, R. (2006). Advantages and disadvantages of four interview techniques in

qualitative research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 7(4).

Perryer, C., Scott-Ladd, B., & Leighton, C. (2012). Gamification: Implications for workplace intrinsic motivation in the 21st century. AFBE J, 371–381.

(33)

Puhakainen, P., & Siponen, M. (2010). Improving employees' compliance through information systems security training: an action research study. Mis Quarterly, 757-778.

Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of games for educational purposes: A review of recent research. Simulation & Gaming, 23(3), 261-76.

Romero, M., Usart, M., Ott, M., Earp, J., & de Freitas, S. (2012). Learning through playing for or against each other? Promoting collaborative learning in digital game based learning. Learning, 5.

Ruhi, U. (2015). Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification. Technology Innovation Management Review.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist, 55(1), 68.

Ryan, R. M., & Deci, E. L. (2010). Self-determination theory. Rochester: University of Rochester.

Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of Human-Computer Studies, 74, 14-31.

Shell (n.d.), Our Goal Zero Ambition. Retrieved from

http://www.shell.com/sustainability/safety/our-approach.html

Waddell, J. C., & Peng, W. (2014). Does it matter with whom you slay? The effects of

competition, cooperation and relationship type among video game players. Computers in Human Behavior, 38, 331-338.

(34)

Webb, E. N., & Cantú, A. (2013). Building internal enthusiasm for gamification in your

organization. In International Conference on Human-Computer Interaction, 316-322. Springer Berlin Heidelberg.

Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249 - 265.

Zhu, Y. (2013). Individual behavior: In-role and extra-role. International Journal of Business Administration, 4(1), 23.

Appendix A: Game description

The game used in this study was what is called a mobile, turn-based game. Players were asked to download the game to their mobile device and create an account using their work-related e-mail address. After an account was created, players had to choose whether they wanted to team up with a random colleague or invite a specific colleague. Teams of two were connected to each other, and players had to choose between playing a social engineer or a hacker. Social engineers were very capable in social interactions and getting information or tools from people but had little technical skills. Hackers were very good with technical aspects like breaking into computer systems, but had little to no social skills.

The game itself is a platform game, each level showing a map of an office floor with several rooms and tools in them. Both players can take turns, and during their turn walk around, talk to people or look at objects placed in the rooms. Characters in the game can give players information, this is something the social engineer could do, and objects can contain useable tools or offer an opportunity to access the company’s computer network. The hacker has to do these things. In between each turn, players can communicate via a small text box in the game and

(35)

exchange tools they have found. The goal of each level was to reach the exit in for each floor by hacking systems and talking to people, so both players could access the next floor.

Appendix B: Consent form Informed consent: English

I hereby state that I have been clearly instructed on the nature and method of this study, as earlier explained to me in the invitation I have received for this study.

I voluntarily participate in this study, and with that have the right to withdraw my consent at any time during the study without having to provide a reason for doing so. I am aware of the fact that the results and information of this study will be published anonymously, and that my personal information will not be shown to third parties without my clear consent.

I understand that any audio from this study will only be used for further analysis. I also am aware of my rights to request to read this study or parts of it at all times, and where necessary make changes to the quotes that have been included in this study from my interview.

If I want more information, now or in the future, I can turn to Iris Rigter via +31633609596. For any complaints with regards to this study I can contact the Ethics Commission of ASCoR via the address: ASCoR secretariaat, Commissie Ethiek, Universiteit van Amsterdam,

(36)

Name participant:

Date: Signature participant: For the researcher

I have given a verbal and written explanation of this study. I will answer any further questions on this study to the best of my ability. The participant will not experience any negative

consequences should he/she choose to stop participation in this study or withdraw his/her data.

Name researcher:

Date: Signature researcher:

Informed consent: Dutch

Ik verklaar hierbij op duidelijke wijze te zijn ingelicht over de aard en methode van het onderzoek, zoals uiteengezet in de uitnodigingsmail voor dit onderzoek.

Ik stem geheel vrijwillig in met deelname aan dit onderzoek. Ik behoud daarbij het recht deze instemming weer in te trekken zonder dat ik daarvoor een reden hoef op te geven. Ik besef dat ik op elk moment mag stoppen met het onderzoek. Ik weet dat de gegevens en resultaten van het

(37)

onderzoek in overleg geanonimiseerd gepubliceerd zullen worden. Mijn persoonsgegevens worden niet door derden ingezien zonder uitdrukkelijke toestemming.

Ik begrijp dat audiomateriaal of bewerking daarvan uitsluitend voor analyse en/of

wetenschappelijke presentaties zal worden gebruikt. Ook behoud ik het recht om het onderzoek ten alle tijden in te zien en waar nodig door mij gedane uitspraken te corrigeren.

Als ik meer informatie wil, nu of in de toekomst, dan kan ik me wenden tot Iris Rigter via 0633609596. Voor eventuele klachten over dit onderzoek kan ik me wenden tot het lid van de Commissie Ethiek namens ASCoR, per adres: ASCoR secretariaat, Commissie Ethiek,

Universiteit van Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, 020-525 3680; ascor-secr-fmg@uva.nl.

Naam deelnemer:

Datum: Handtekening deelnemer:

Voor de onderzoeker

Ik heb een mondelinge en schriftelijke toelichting gegeven op het onderzoek. Ik zal resterende vragen over het onderzoek naar vermogen beantwoorden. De deelnemer zal van een eventuele voortijdige beëindiging van deelname aan dit onderzoek geen nadelige gevolgen ondervinden.

(38)

Naam onderzoeker:

Datum: Handtekening onderzoeker:

Appendix C: Interview guide Introduction

First of all, thank you for partaking in this interview. As previously mentioned I will be asking you a few questions on your experiences with Elevator the Game. Feel free to answer them as you see fit, there are no right or wrong answers. If you feel uncomfortable with any of the questions asked, please let me know. If you want to stop the interview at any point, please indicate this. I would also like to record the conversation as we have previously discussed. The recordings will be stored anonymously and will only be listened to by me, so I can use them for further analysis. Do you still agree with me recording this conversation? Then I would like to ask you to carefully read this informed consent document for a minute, this states that you have been

(39)

fully informed about the kind of research I am conducting and that you are aware of the fact that your anonymity will be safeguarded. If you understand the text and agree to participate, please sign the document and then we will continue.

Once agreed upon, start interview.

First: Experience of gamification

The aim of this topic is to gain an understanding of how the respondent experienced the game. Did he or she enjoy it? This can be relevant for the influence the game has on awareness and behavior later on, people who do not enjoy the game may feel different about the process than people who did enjoy the game.

I would first like to ask you a quick question on how you played the game. ● Which role did you take on in the game?

○ Possible follow-up question: Why did you choose this role?

I would also like to know how you experienced the game while you were playing it and what your opinion is on games and gamification.

● What was your initial reaction when you heard about playing this game? ○ If short answer is given: And why did you have this reaction/feeling?

● Can you tell me about your experiences while playing the game?

● How did you feel while playing the game?

(40)

○ If short answer is given: Could you please explain this feeling a bit more?

● How did you feel about playing this game with a colleague?

○ Possible primers are: comfortable, uncomfortable, happy, frustrated etc. ○ Follow-up question: What aspects of the game made you feel [insert feeling

mentioned]?

● How did you feel about the turn-based aspect of this game?

○ Possible follow-up: What part of it did you find [insert initial reaction]?

● If you had to rate this game between 1 being the lowest and 10 being the highest, what would you rate it and why?

● What aspects of the game make you say that it is [insert answer to previous question]?

Second topic: Risk awareness after playing the game

The aim of this topic is to gain an understanding in the risk awareness (related to information security) of the respondent after he or she played the game. Has he or she gained any new knowledge? Have attitudes towards information security changed?

As you have probably noticed, Elevator the Game was focused on your knowledge of and experience with information security and cyber risk. For the following questions I would like to focus on this:

● What information- and cyber risks have you become more aware of?

○ Possible follow-up questions: Could you please elaborate a bit more on that? What specific facts/points did you take away from the game?

(41)

If respondent indicates that he/she has gained awareness on cyber risk through the game: ● What parts of the game did you find particularly helpful in creating this awareness?

● How do you feel those parts helped you with becoming more aware of cyber risk?

If respondent indicates that he/she has not gained awareness on cyber risk through the game: ● What do you think is the reason you didn't learn anything from this game?

● How do you think this game can be improved?

Continue with following question:

● Do you now think differently about cyber risks than before you played the game? ○ Follow-up question: How do you think the game has helped with this [mention

difference if needed]?

Third topic: Behavior after playing the game

The aim of this topic is to gain an understanding in possible behavioral changes of the respondent after he or she played the game. A change in awareness is frequently linked to a

change in behavior, so these questions can provide more insight into the effects of the game on awareness and actions.

As well as a focus on your knowledge about cyber risk, this game also looks at possible behaviors that can make you handle information security and cyber risks better. The following questions are aimed towards that:

● What behaviors or actions to avoid cyber risks were mentioned in the game? ○ Follow-up question: How were these behaviors incorporated in the game?

(42)

● Do you expect to change your behavior?

If respondent indicates that he/she will change behavior after playing the game: ● Why do you expect this behavior to change?

○ Follow up question: What parts of the game did you find particularly in helping you change behavior?

If respondent indicates that he/she will change behavior after playing the game: ● Why don’t you expect this behavior to change?

End of interview

Great! These were all the questions I have. Is there anything else you would like to add?

[possible response from interviewee] Thank you very much for participating in this interview. In the next few months I will be analyzing this and other interviews and writing a conclusion on my thesis. If you would like to receive this thesis after it has been finalized, please let me know and I will send it to you. If you have any questions on this interview, my research or for any reason wish to withdraw your interview please contact me whenever you want.

Referenties

GERELATEERDE DOCUMENTEN

In the same way that the young people are aspiring to have a single fixed dwelling filled with the positive social experiences often associated with the normative UK vision,

[r]

Our study on how to apply reinforcement learning to the game Agar.io has led to a new off-policy actor-critic algorithm named Sampled Policy Gradient (SPG).. We compared some state

Voor een nieuw medisch hulpmiddel waarvan nog geen vergelijkbaar product op de markt is, moet de veiligheid en effectiviteit door klinisch onderzoek worden aangetoond voor het

- Voor waardevolle archeologische vindplaatsen die bedreigd worden door de geplande ruimtelijke ontwikkeling en die niet in situ bewaard kunnen blijven: wat is

In particular, this requires firstly equity, sustainability and efficiency in the protection, development and utilisation of water resources, as well as the institutions that

Bespreek met de cliënt (en eventueel met een naaste van de cliënt) welke mensen en contacten in zijn so- ciale netwerk (nog) niet helemaal worden benut.. Vraag hier goed

Most general-purpose methods feature hyperparameters to control this trade-off; for instance via regularization as in support vector machines and regularization networks [16, 18]..