PAUL SCHÜREN
University of Groningen, Faculty of Economics & Business MSc. Business Administration
Specialization: Change Management
Company supervisor: dr. G. Dekker Second company supervisor: M. Taverne, MSc University supervisor: dr. C. Reezigt Second university supervisor: dr. J. Rupert
ABSTRACT
This thesis has the aim of unveiling how gamification can aid in attaining management consultancy critical success factors (CSFs) and what the underlying principles are in doing so. Gamification is the process of using game design elements and game-thinking to engage users and to solve problems in non-game contexts. The focus lies on how gamification works in a management consultancy setting. Based on eight expert interviews, 14 distinct game elements are linked to management consultancy CSFs which were derived from a literature review. These 14 links are all supported by a minimum of three out of the eight experts. The linked CSFs and game elements function as a guideline for applying gamification in management consultancy. Future research on such links could make gamified applications, and with that game design, more usable and would help in reducing the number of gamification failures.
PREFACE
In choosing the subject for my thesis I aimed at studying a new, surprising, meaningful and intriguing management consultancy related topic. That topic became gamification; it was immediately clear that it would fulfill the aims which I set up front. The innovative consultancy firm, and developer of gamified consultancy tools, MadLogic, helped me in learning about gamification, increasing my network and showing me, through real-life change projects with their game 3TGO, that their gamified consultancy tool actually works at MNEs and SMEs. Answering the research question became an important quest while writing my thesis the past months and surprisingly created lots of attention from practitioners and enthusiasts throughout the Netherlands. I am pleased with my results in writing this thesis which can aid many consultancy firms in setting up gamification in a thoughtful and well designed manner.
understanding of my lack of time for her. Without a single exception, all the people that I mentioned above have helped me gain insight on all sorts of things. Insights that have gained me a plethora of experiences and knowledge which I will carry with me the coming years.
“Play is a uniquely adaptive act, not subordinate to some other adaptive act, but with a special function of its own in human experience”
SUMMARY
TABLE OF CONTENTS Abstract 2 Preface 3 Summary 5 Table of Contents 6 1. Introduction 7
1.1 Comparing Management Consultancy and Gamification Practices 8
2. Problem statement 8 3. Definitions 10 3.1 A Game 10 3.2 Serious Gaming 10 3.3 Gamification 11 3.4 MDA Framework 11
4. Relevancy and Purpose 12
5. Adjacent Fields 14
5.1 Category 1: Management Consultancy 15
5.1.1 Critical success factors 15
5.1.2 Categorization of the CSFs 18
5.2 Category 2: Behavioral Change Theories 20
5.2.1 Motivation 20
5.2.2 Engagement 21
5.2.3 Learning through serious gaming 21
5.3 Category 3: Game-related Research 24
5.3.1 Flow 24
5.3.2 Individual motivators: why people play 25
5.3.3 Player types 25
6. Methodology 26
6.1 Literature Findings 27
6.2 Expert Interviews 27
6.3 Data Analysis Strategy 28
6.4 Design Quality 28
7. Results 29
8. Discussion 35
9. References 39
10. Appendices 46
10.1 Eight CSFs Derived From Kotter (1996) 46
10.2 Six CSFs Derived From Pitman (1994) 47
10.3 Expert Interviews 47
10.4 Standardized Interview: The Use of Gamification in Management 48
1. INTRODUCTION
Games are serious business, something that was already known by the medieval monarch Alfonso X (1252 - 1284). “As patron of the ludic arts, he [Alfonso X] commissioned the richly-illustrated Libro de los juegos (“Book of games”), featuring a panoply of games of skill and chance; [and] as legislator, he (…) [made] an effort to regulate gambling (…)” (Carpenter, 1998:333). This thesis is dedicated to the subject of applying game elements in consultancy trajectories. More specifically, it will focus on the link between the application of game elements and its relationship towards the attainment of critical success factors of management consultancy interventions. This thesis is set up as an explorative research. Further in this thesis, an overview of the management problem and a justification of the usage of gamification in consultancy trajectories are provided. Following this, adjacent topics regarding gamification in other sectors are discussed. From this a knowledge
gap on the application of game elements in consultancy trajectories will be unveiled. The methodology section will explain how this gap is covered by this research. Finally, results from a qualitative research are displayed to the reader and a number of propositions will be elaborated on in the discussion chapter. Since the application of game elements in consultancy trajectories is rather unknown to many academia and practitioners this introduction chapter will provide the reader with a clear guiding example of application of gamification (see the text box). In addition, analogies are presented between management consultancy and gamification practices.
A Guiding Example
For the sake of further understanding of the application of game elements in consultancy interventions, it is chosen to describe a case study in short.
Imagine an advisory intervention where a consultant will start his investigation of the company in a classical way, which includes personnel interviews (which can be perceived as dull, distracting and repetitive), scans of the organization based on his observations in the office (which can be subject to stereotypes) and group interventions which can be one-sided, non-dynamic and lacking a match with the audience, i.e. the client. During these sessions, the client(s) can become distracted, unengaged and demotivated. The latter might have severe consequences for the implementation and anchoring of the change.
A different kind of consultancy intervention involves the application of game elements. In such an intervention the consultant can start off by creating a challenge (the first game element) for his clients. Such as challenge can be created by inviting the client(s) into an office space at a certain pre-agreed time without giving any information regarding what is about the happen. Following this, the consultant can introduce some kind of game or simulation (online or offline) to the participants (called players from now on) and apply game elements such as roleplaying situations, multiplayer (team) and single player (individual) efforts, missions (long term) and intermediate goals (short term). In addition, during the game or simulation points can be rewarded to teams or individuals. The points that are distributed over the players can be used as a benchmarking tool for external or internal use.
1.1 Comparing Management Consultancy and Gamification Practices
If we compare the definitions of management consultancy and gamification practices we can see several similarities. Six analogies are presented below. For creating the analogies the definition of management consultancy by Kubr (2002) is compared to game elements from various writers in the field of gamification (McGonigal, 2011; Dignan, 2011; Zichermann & Cunningham, 2011).
Kubr (2002) defined management consultancy as a professional and independent advisory service (Kubr, 2002). This means that both the consultant and client have a choice as to who they work for or hire; which seems very much alike the basic game trait of ‘voluntary participation’ (McGonigal, 2011). The achievement of organizational purposes and objectives mentioned by Kubr 2002) fits the basic trait of a game, namely ‘to have a specific goal’ (McGonigal, 2011). Third, management consultancy focuses on solving management and business problems (Kubr, 2002). This shows resemblance with the idea that games consists of a series of little challenges and struggles and artificial conflict (Dignan, 2011). Fourth, the aspect of “(…) identifying and seizing new opportunities (…)” (Kubr, 2002:11) corresponds with the idea that gaming is all about playing the system, taking risks, confronting our fears and encouraging failure. This makes perfect sense when exploring and exploiting new opportunities (Dignan, 2011). Fifth, gaming and consultancy both enhances learning (Kubr, 2002; Dignan, 2011). Gaming is playing and “play is not a way to make learning fun, play is learning” (Dignan, 2011:28). Finally, management consultancy is also about implementing change (Kubr, 2002). Games can support change implementation through motivating and engaging players (Zichermann & Cunningham, 2011). In addition, “games have the power to shape our attitudes, beliefs and even our thoughts” (Dignan, 2011:48).
2. PROBLEM STATEMENT
engagement and loyalty. The chairman of GSummit is Gabe Zichermann. He wrote the books The Gamification Revolution (2013), Gamification by Design (2011) and Game-Based Marketing (2010). His books have helped to define the industry’s standards and frameworks. Zichermann states that “since the beginning of the gamification industry in 2010, over 350 companies have launched major gamification projects. These include consumer brands like MLB, Adobe, NBC, Walgreens, Ford, Southwest, eBay, Panera and Threadless among others. For (…) companies [like] Oracle, SAP, Jive, Cisco, Pearson and Salesforce, gamification has emerged as a key element in their consumerization of the enterprise strategy. (…) [In addition,] consulting behemoths Deloitte, Accenture, NTTData and Capgemini began practices targeting gamification at Fortune 500 companies”.1
Another proponent of gamification, Brian Burke from the Gartner Group, has released a report that suggests that by 80% “of current gamified applications will fail to meet business objectives primarily because of poor design.”2 Such news as provided by Gartner is often misunderstood as an indictment of the future of gamification application, however the article clearly argues that the 80% failure chance comes from wrong gamification design and related challenges of making fundamental vicissitudes in the enterprise. Exactly the failure of properly applying gamification is the reason that questioning why it works is much more relevant than whether it works, since that is already been proven by practice as the abovementioned examples display. To conclude, we have seen that both smaller enterprise and larger multinational firms are starting to gamify (part of) their processes. Despite the practical applications, there seems to be a lack of academic foundations for using gamification in consultancy trajectories and more specifically why that works. Therefore this thesis has the goal of unveiling the added value of using gamification in management consultancy undertakings and seeks the linkages between game elements from gamification undertakings and CSFs required to be attained in management consultancy interventions. The guiding research question is two-folded and is: How can gamification aid in attaining management consultancy CSFs? And what are the underlying principles?
1
http://www.huffingtonpost.com/gabe-zichermann/gamification_b_2516376.html?utm_hp_ref=tw
3. DEFINITIONS
The terms serious gaming and gamification are often used interchangeably, despite the fact that these are two different things. Serious gaming and the term gamification has become relatively trending the last couple of years. Some even call it the marketing buzzword of our time (Zichermann & Cunningham, 2011). Others refer to gamification as simply making games for advertisement purposes or training usage. Nonetheless, many people in the field of business and consultancy might still be relatively unknown with these terms. Therefore this thesis urgently requires a definition for a game, serious gaming and gamification. Hereafter, the main principles behind gamification, the game mechanics, dynamics and aesthetics are elaborated on.
3.1 A Game
Let us start by defining a game. Crookall (2010) indicates that there seems not to be an agreed-upon definition of a game, “[a]s Wittgenstein said of game, it is almost impossible to define, but we recognize one when we see it.” (Crookall, 2010:904; Uden, 2009). However, four defining traits, as described by McGonigal (2011), are useful to describe here. First of all, all games have a goal; this creates a sense of purpose to the players. Secondly, rules are set as boundaries on how players can achieve the goal. Third, all games have some sort of feedback system which indicates players how far they are from reaching their (short- or long-term) goal. Fourth, voluntary participation creates the freedom to enter or leave a game at will (McGonigal, 2011).
3.2 Serious Gaming
3.3 Gamification
Based on the abovementioned principles we can describe how gamification can be set apart from serious gaming. Gamification is described as the “use of game design elements in non-game contexts” (Deterding, Khaled, Nacke & Dixon, 2011). A more elaborate definition is held by Zichermann and Cunningham (2011) who define gamification as “the process of game-thinking and game mechanics to engage users and to solve problems” (Zichermann & Cunningham, 2011: xiv). Zichermann and Cunningham (2011) add the concept of game mechanics. For our definition of gamification, we do not only use game mechanics, but the entire umbrella term; namely game elements. Zichermann and Cunningham (2011) refer to game elements as a combined package of game mechanics (M), dynamics (D) and aesthetics (A). In addition, the latter definition of gamification adds the important notion of game-thinking. The idea behind game-thinking is the effective use of the MDA framework based on three distinct variables. These are: the non-game context, the goal of the gamification effort, and the type of user involved. These three variables influence the construction and mixture of the MDA framework. By combining both aforementioned definitions and the added principles, a rather complete description of gamification is created, namely: gamification is the process of using game design elements and game-thinking to engage users and to solve problems in non-game contexts (Zichermann & Cunningham, 2011; Deterding et. Al, 2011).
3.4 MDA Framework
Hunicke, LeBlanc and Zubek (2004) presented the MDA framework in a research paper for the first time. Game mechanics describe certain elements which are afforded to the player within a game context and have the aim of triggering a specific action (Hunicke et. al, 2004; Zichermann & Cunningham, 2011). They are particular components of the game and include data representation and algorithms. Examples are: rules of the game, goals, earnable points, levels and scoreboards.
The game aesthetics are the eventual desired emotional responses that are triggered through playing the game. This is also known as the ‘fun’ factor. Fun however, is a rather indirect and vague term. So for describing game aesthetics, a more direct vocabulary would include the following taxonomy (see table 2). The combined mechanics, dynamics and aesthetics are called game elements.
TABLE 1 Game Aesthetics
A game as: Aesthetic
Sense-pleasure Sensation
Make-believe Fantasy
A drama Narrative
An obstacle course Challenge
A social framework Fellowship
Uncharted territory Discovery
Self-discovery Expression
Pastime Submission
TABLE 1 - Derived from: Hunicke et. al, 2004
4. RELEVANCY AND PURPOSE
This investigation is deemed relevant for several reasons. First of all, gamification is getting more important these days. Gartner, an information technology research and advisory company, estimates increasing usage of serious gaming. In 2011 they indicated that by 2015, 50% of all companies managing innovation and research will use gamification to drive their innovative thinking. According to Gartner there are great opportunities for businesses concerning the achievement of higher levels of engagement, the ability to change behaviors and stimulate innovation, both for customers and employees.
three points in their article from 2006 on how serious gaming offers potential for companies in the Netherlands. Amongst others they mentioned that gamification makes the learning process more attractive and therefore more efficient and, since serious games have the potential to be spread out internationally immediately from the creation, companies can easily increase their international network. STT (2009, 2011a, 2011b) wrote three distinct explorative studies on serious gaming in general. They see similarities between managing and serious games in that managing can be seen as a continuous process of making calculations, weighing of interests and decision making processes that influence certain situations. SST (2011b) expects a growth of 6% per annum for the gaming industry in the Netherlands.
5. ADJACENT FIELDS
Literature has already discussed many topics that are directly involved in this research. In this chapter, adjacent research fields will be discussed that are relevant in identifying how the use of gamification contributes to attaining the CSFs of general management consultancy interventions. Chapter 5 is divided into three distinct research categories: category 1: management consulting, category 2: behavioural change theories and category 3: game-related research. The idea is that this chapter links to our research objective, namely the link between CSFs (category 1: management consultancy theory) and game elements (category 3: game-related research). The second category to be discussed in this chapter (behavioral change theory) is meant as a justification for using game-related tools in management consultancy settings.
The latter statement requires some further explanation. The second category is meant as a catalyst for the link between CSFs and game elements. The link between CSFs (category 1) and game elements (category 3) cannot be created merely by the two categories themselves. Another category (category 2) is needed to provide this abovementioned linkage. This can be explained through the fact that many CSFs are aimed at motivation and engaging employees (for example: show of visible commitment by management, continuing involvement in the project, reward for support, seek buy-in from all involved, social orientation); hence the link between category 1 and 2. On the other side of the model, we find a link between category 2 and 3. Here, category 3 deals with game-related research. Within this category, we find research on game elements, flow, reasons why people play and player types. These topics have the aim of motivating and engaging people by using specific game elements such as levels, scoreboards, challenges, sensations and discoveries in order to motivate and engage the player. To conclude, both CSFs (category 1) and game elements and related research (category 3) aim at satisfying feelings of motivation and engagement. The following schema provides a visual representation of the composition of this chapter.
5.1 Category 1: Management Consulting
This sub-chapter will relate to management consultancy theories which are within our research scope. The focus will be laid upon CSFs for management consultancy applications derived from Kotter (1996), Pitman (1994) and Prabhakar (2008).
5.1.1 Critical success factors. Critical success factors (CFSs) were first mentioned in an article by Daniel (1961). He discussed the problem of a knowledge gap on critical information that is needed for “setting objectives, for shaping alternative strategies, for making decisions, and for measuring results against planned goals” (Daniel, 1961:111). In order to overcome and to fill this gap, Daniel (1961) came up with success factors. He found that most industries could be downsized to three to six success factors that determined success. These factors were key jobs which “must be done exceedingly well for a company to be successful” (Daniel, 1961:116). CSFs are also useful to apply for processes and projects, although one might need to alter the general nature of CSFs into factors that contain more specific know-how to better support decision-making (Zwikael & Globerson, 2006).
5.1.1.1 Kotter’s CSFs. A prime example of critical success factors of management consultancy interventions is created by Kotter (1996). His famous Eight-Stage Process consists of a sequence of steps for which “successful change of any magnitude goes through all eight stages (…) skipping even a single step or getting too far ahead without a solid base almost always creates problems (Kotter, 1996:23).
TABLE 2
Kotter’s CSFs
Creating a sense of urgency Forming a guiding coalition Envisioning the change Communicate the vision Remove any obstacles Create short-term wins Build on the change
For all the eight steps mentioned above, Kotter (1996) strictly says: “Whenever you leave one of the steps in the eight-stage change process without finishing the work, you usually pay a big price later on” (Kotter, 1996:83). The CSFs by Kotter (1996) are more extensively discussed in the appendix (see 11.1).
5.1.1.2 Pitman’s CSFs. Pitman (1994) also summed up six CSFs to organizational change, which are closely related to Kotters ideas (1996). They are summarized in table 4. According to Pitman (1994), an absence of one of the aforementioned CSFs would result in less success for an organizational change effort.
TABLE 3
Pitman’s CSFs
Create a strong business-related need for change
Show of visible support and commitment by management Install a high degree of communication
Prepare properly
Let the users and clients participate Reward for support of necessary changes
TABLE 3 – Pitman’s CSFs
5.1.1.3 Prabhakar (2008) CSFs. Prabhakar (2008) wrote a literature review on project success. He saw a great divergence of opinions in this field. His literature study “attempts to put forth the points of views of different researchers in this field” (Prabhakar, 2008:3). From this literature review CSFs mentioned by Martin (1976) Lock (1984) Cleland and King (1983), Sayles and Chandler (1971), Baker, Murphy and Fisher (1983), Pinto and Slevin (1989), Jiang, Klien & Discenza (2002) and Morris and Hough (1987) are derived. They can be found in the next sup-chapter 5.1.1.4 where all CSFs are combined in one table.
TABLE 4 - CSFs
CSF Authors
Create a sense of urgency
Create a strong business-related need for change
Kotter (1996)
Pinto & Slevin (1989) Pitman (1994)
Form a guiding coalition
Show of visible support and commitment by management
General management support Top management support
Kotter (1996) Pitman (1994) Martin (1976)
Cleland & King (1983)
Envision the change Set clear goals
Seek simplicity, not complexity, in goal and path
Kotter (1996)
Baker, Murphy and Fisher (1983)
Jiang, Klien & Discenza (2002)
Communicate the vision
Install a high degree of communication
Facilitate information and communication channels Monitoring and feedback
Client consultation
Maintaining ongoing communications
Kotter (1996) Pitman (1994)
Cleland & King (1983) Sayles and Chandler (1971) Pinto & Slevin (1989) Jiang, Klien & Discenza (2002)
Remove any obstacles Bypass an obstacle
Kotter (1996)
Jiang, Klien & Discenza (2002)
Create short-term wins
Continuing involvement in the project
Kotter (1996)
Sayles and Chandler (1971)
Build on the change Progress meetings
Follow a standardized process
Kotter (1996) Locke (1984)
Jiang, Klien & Discenza (2002)
Prepare properly
Minimum start-up difficulties
Pitman (1994)
Baker, Murphy and Fisher (1983)
Let the users and clients participate Reward for support of necessary changes Create a good task vs. social orientation
Pitman (1994) Pitman (1994)
Baker, Murphy and Fisher (1983)
Community involvement Seek buy-in from all involved
Morris & Hough (1987) Jiang, Klien & Discenza (2002)
TABLE 4 – CSFs
This table provides us with a broad overview of all CSFs. These CSFs will be used in the interviews for unveiling the links to game elements of gamified consultancy.
5.1.2 Categorization of the CSFs. For further understanding and practical application with regards to a gamification design the CSFs mentioned in table 4 are categorized based on a change management model by Beckhard and Harris (1987) as elaborated on by Cawsey, Deszca and Ingols (2011). This change model begins the process with an exploration on why the change is needed, followed by urgency for change and a desired state in relation to an organization’s present situation. The difference between the desired and the current state can be identified as a gap. Knowing this gap, a transition can be managed and change action can be implemented (Cawsey et. al, 2011). The model by Bechkard and Harris (1987) forms the framework for our categorization of game elements in order to clarify and generalize the findings in this thesis.
Stage 1: Organizational analysis. This first stage deals with an analysis of the forces for and against the change. A thorough understanding of the organization and its stakeholders is important in this stage (Cawsey et. al, 2011).
“minimum start-up difficulties” (Baker, Murphy and Fisher, 1983) and “top management support” (Cleland & King, 1983).
Stage 2: Need for change. “Many assume that the need for change is easily recognizable, obvious, and evident from the environment. (…) However, people may not accept the need or believe that they need to change. (…) It is important to understand that the perception of the need for change is exactly that: a perception” (Cawsey et. al, 2011:46,47).
The CSFs that belong to this stage are: “create a sense of urgency” (Kotter, 1996; Pinto & Slevin, 1989) and “create a strong business-related need for change” (Pitman, 1994).
Stage 3: Creating a change vision. Once the most important stakeholders are convinced of the need for change, manager mistakenly think others will understand and be convinced as well. This notion ignores the “the lag that will occur as the message about change moves through the organization” (Cawsey et. al, 2011:48). Therefore a strong change vision is needed to be spread.
The CSFs that belong to this stage are: “envision the change” (Kotter, 1996), “communicate the vision” (Kotter, 1996) “set clear goals” (Baker, Murphy and Fisher, 1983) and “seek simplicity, not complexity, in goal and path” (Jiang, Klien & Discenza, 2002)
Stage 4: Gap analysis. This stage focuses on the differences between the current and the future state. More specifically it focuses on setting the correct diagnosis for change. The idea is that if the diagnosis is wrong, the chances for inappropriate action increase. The gap analysis addresses why change is needed by analyzing both formal and informal dimensions, various stakeholders and the change agents themselves. This analysis creates a complete image of the situation and the gaps that require attention (Cawsey et. al, 2011).
(Pinto & Slevin, 1989), “maintaining ongoing communications” (Jiang, Klien & Discenza, 2002), “community involvement” (Morris & Hough, 1987), “seek buy-in from all involved” (Jiang, Klien & Discenza, 2002).
Stage 5: Manage change transition. The final stage emphasizes on “getting from here to there” and “managing during the transition state”. This transition management is needed because the action plans are mostly linear and straightforward, whilst the reality is often more complex and neither linear nor straightforward (Cawsey et. al, 2011). The CSFs that belong to this stage are: “create short-term wins” (Kotter, 1996), “continuing involvement in the project” (Sayles and Chandler, 1971), “build on the change” (Kotter, 1996), “progress meetings” (Locke, 1984), “follow a standardized process“ (Jiang, Klien & Discenza, 2002), “translate design into reality” (Kotter, 1996), “let the users and clients participate” (Pitman, 1994), “reward for support of necessary changes” (Pitman, 1994) and “create a good task vs. social orientation” (Baker, Murphy and Fisher, 1983).
5.2 Category 2: Behavioral Change Theories
This second sub-chapter within the scope of the adjacent fields of this thesis deals with organizational theories. Many subject areas are already discussed in relation to other research fields. In order to apply gamification in organizational context, it is important to emphasize on related established theories. We will discuss motivational theories, engagement and theories related to learning. It is chosen to highlight these topics to create further understanding in the key factors that form the core of gamification utilization practices.
possibility to fully participate in their own change intervention and thus get relatively high levels of attention from higher management levels (who can also be involved in the game) during and after the gamified consultancy intervention. The term ‘player’ is used on purpose in this sub-chapter. In any non-game consultancy trajectory, one may refer to the opposing party as a consultancy’s client or principal. However, by referring to our clients as players, we stop looking at the immediacy of one single financial transaction on the short-term, and start focusing on their engagement with consultancy products and services on the long-term. This results in a stronger focus towards people, rather than towards monetary values. From this, higher motivation levels amongst the players are likely to occur. Player motivation for playing games is nowadays seen as a factor derived from three interrelated variables, namely flow, individual motivators and player types; these values will be further discussed in the third category in chapter 5.3 on game-related theories.
5.2.2 Engagement. Gamification can also drive engagement in a consultancy process. Gartner3 distinguishes four ways in which gamification can aid in driving engagement, namely through accelerated feedback cycles which keeps the people involved engaged. Secondly, gamified processes often contain clear goals and rules of play, whilst the real world is rather fuzzy; the result is that players feel empowered to achieve goals. Third, game elements help in building a narrative that is more compelling than real-world activities, which stimulates engagement and the will of achieving activity related goals. Fourth, Gartner argues that gamification can drive engagement when tasks are challenging but achievable. Real-world challenges are often large and rather long-term, whilst gamification may provide various short-term and achievable goals which keeps the players connected, involved and in flow.
5.2.3 Learning through serious gaming. Learning is an important externality of a gamified management consultancy intervention, since learning is a change and vice versa. Normally a learning experience can be hard to develop in a short intervention. Games can help here, because in addition to the fact that serious games can motivate, engage, educate, train or inform, enhance decision making, change behaviour and attitudes (Subrahmanyam & Greenfield, 1994; Chapman & Martin, 1995; Ruohomaki, 1995; Chwif and Barretto, 2003; Smeds, 2003; Lainema and Hilmola, 2005; Lewis and Maylor, 2007; Ferguson 2007; Bogost, 2007), serious
games can also aid in facilitating learning possibilities (Hummel, Houcke, Nadolski, Hiele, Kurvers & Löhr, 2011; Yu-Hao, Heeter, Magerko, Medler, 2012). This can be done either because the serious games are deliberately designed for learning or it just happens by coincidence. Regardless of whether the learning effect within a serious game was deliberately constructed or not, learning goes further than simply acquiring knowledge of a specific content. Learning is concerned with much more factors such as applying the learned knowledge “for certain problem situations in the workplace (…) and acquiring competences like information gathering, general business skills, media literacy, problem-solving, communication, collaboration and critical reflection on wicked problems. Such competences are usually not addressed by other learning platforms” (Hummel et. al, 2011: 1030).
mind-set participants, the number of mistakes would predict loss of attention. Based on their research Yu-Hao et. al (2012) concluded the following important notes regarding players behavior and game outcomes: “in the context of serious games, good performance in a game should indicate that learning has occurred. We found that growth mind-set players performed better than fixed mind-set players, their mistakes did not affect their attention to the game, and they read more learning feedback than fixed mind-set players. In addition, growth mind-set players were more likely to actively seek difficult challenges, which are (…) essential to self-directed learning” (Yu-Hao et. al 2012: 193).
average appreciation of virtual collaboration among the focus group included in their research (Hummel et. al, 2011).
5.3 Category 3: Game-related Research
This sub-chapter will elaborate on known research in the field of games. In particular we will focus on aspects that lay in the nature of gamification undertakings. We first focus on the concept of flow. In line with the flow principle, we will focus on why people play. Finally a further elaboration on the type of players that a gamified concept has to deal with is presented.
FIGURE 2 - Derived from Zichermann & Cunningham, 2011
5.3.2 Individual motivators: why people play. Zichermann and Cunningham (2011) maintain that there are four individual motivators for playing games, these are: for mastery, to distress, to socialize and the have fun. In almost all cases the player is looking for winning some kind of competition, exploring the system and engaging into interactions with other players. These attributes do not merely apply to video game players, but also exist organically in the strata of our society (Zichermann & Cunningham, 2011). It is important to denote that one should take notice of these aspects with regard to designing a gamified consultancy intervention. Without doing so, there is the risk of losing grip on the players. While your aim was to motivate, engage and stimulate learning.
end itself” (Zichermann & Cunningham, 2011:22). Killers, finally, are defined by a focus on winning, rank and direct peer-to-peer competition. Unlike achievers, winning is not enough. A killer is only satisfied if they win and consequently someone else loses. For all types it must be noted that people are not exclusively one specific type of player. Most people have at least some percentage of each. In addition, the player type may vary from game to game and during life phases. If the scores were mutually exclusive, this would make socializers 75% of all players, explorers and achievers would each make up 10% of the player population and killers account for 5 (Zichermann & Cunningham, 2011).
FIGURE 3 – Player Types
6. METHODOLOGY
6.1 Literature Findings
To begin with, the CSFs of change management interventions were retrieved from various authors. It is chosen to pick often cited authors, where the citation search is performed with the help of the ‘Web of Knowledge Cited Reference Search’. We have used the following authors to construct our CSF list: Baker, Murphy and Fisher (1983, 27 citations), Cleland and King (1983, 103 citations), Jiang, Klien and Discenza (2002, 16 citations). Kotter (1995, 808 citations), Lock (1984, 81 citations), Martin (1976, 14 citations), Morris & Hough (1987, 142 citations), Pinto and Slevin (1989, 47 citations) and Sayles and Chandler (1971, 73 citations). Pitman (1994) is only cited three times but is used in this research for the resemblance on other CSFs by the aforementioned authors but stated differently. This might have created a better sense of recognition amongst the interviewed experts. The game elements in the literature framework are derived from Hunicke, LeBlanc and Zubek (2004), it is chosen not to explicate extensively on the game elements, since they are countless and not bound by any theoretical frameworks.
6.2 Expert Interviews
Since we answer a ‘how’ question, on which we as researchers have little or no control in our research, Yin (2003) suggests to use a case study research, which is closely related to the expert-interviews that were held in this research. That is the reason why the links between the CSFs and game elements are constructed through expert interviews with management consultants and researchers in the field of gamification and management consultancy. A total of eight interviews are conducted. Yin (2003) proposes that multiple ‘case studies’ (from now on referred to as expert interviews), eight in our case, are preferred over a single study design, since support of multiple sources to provide evidence uses data triangulation. This strengthens its construct validity of the data gathering (Yin, 2003).
interviewees and lasted approximately one hour. Every interview started by asking general questions on the overall process of applying gamification in consultancy. The final questions of the interview narrowed down towards creating the links between CSFs and game elements. All interview questions are open-ended as advised by Yin (2003). In addition, all interviewees agreed to record their interviews, this provides more accurate interpretation (Yin, 2003).
6.3 Data Analysis Strategy
A preferred analytical technique in trying to answer a “how” research question is pattern-matching (Yin, 2003). This technique relates information from the eight interviews to the predicted patterns (namely the game elements and CSFs). Yin (2003) acknowledges that there is “no precise way of setting the criteria for interpreting these types of findings” (Yin, 2003:27). Therefore, this research has set its own criteria in extracting findings from the expert interviews. In order to answer the question “how close does a match have to be in order to be considered a match?” (Yin, 2003:27), we have stated the following consensus criteria before consulting the interviewees: A link between the CSFs and a certain game element is seen as a relatively strong and predictive link when 3 or more interviewees mentioned the same link by using the same words, substitutes or a depiction of the meaning of the word. The links which are made by 3 or more persons are elaborated on in the results section most extensively since they are deemed most predictive. Equal links made by only 2 persons are shortly discussed, and the links provided by only 1 of the 8 persons are merely mentioned without detailed information. This method is chosen since it is believed that 3 out of 8 experts provide for a relative strong predictor concerning whether or not a link between a certain game-element and a CSF is actually present.
6.4 Design Quality
research is valued. The reliability of this research deals with the extent to which another researcher is able to produce the same results when performing similar interviews. In order to increase reliability the steps included in the case study are operationalized and documented. In addition, the interview questions and transcripts of the interviews are documented in the appendix. This makes sure that the reader can track where the results are derived from (Yin, 2003).
7. RESULTS
Based on the literature review that revealed the foundations of gamification on the one hand and numerous CSFs for a change project on the other hand, we have conducted eight interviews with experts in the field of gamification and change management. All interviews resulted in data containing links between CSFs and game elements. The links as presented by the interviewees are presented as follows. Links made by 3 or more persons are elaborated on in the results section most extensively. Since this research is explorative, the links made by 2 persons are shortly discussed, and the links provided by only 1 of the 8 persons are merely mentioned. The results show numerous links between CSFs and game elements, that is, numerous ways in which specific game elements can be used to trigger certain factors that will increase the success of a change intervention. These links are provided below based on the change model stages by Cawsey et. al (2011).
Stage 1: Organizational analysis (Cawsey et. al, 2011)
CSFs: guiding coalition, show commitment by management, (top) management support, remove obstacles, prepare properly, minimum start-up difficulties.
itself. “Telling the participants that you will have extensive discussions about the organization’s functioning, people tend to be anxious. But if you explain that you will be playing a game with them, the main emotion in the room is commitment”. Van Bree works the other way around, he only mentions the date and time of the meeting and does not tell anything upfront. “This has some consequences, some people for example will lose track and get confused (what’s the agenda? Who I am representing? Who is involved?). These people often cannot cope with the uncertainty. This leads to a self-selection of the participants for the gamified intervention, since you would not want such people as obstacles during the workshop. In addition, the challenge of not knowing what to expect makes the level of energy of the participants increase”. Fourth, Michiel Postma (Gridplay) also acknowledges the fact that challenges help in removing obstacles. “I often use challenges, in the form of assignments and missions, which tends to create a positive form of peer pressure and help players to get in a flow [see figure 2] and as a consequence players evade a negative atmosphere [full of obstacles]”.
Teaming: the game element of forming teams during the intervention is mentioned by three interviewees. First of all, Van Bree dictates that teaming can remove obstacles: By making teams “people stop paying attention on individual hierarchical positions and actions. [The new team] cuts the participants loose from their normal function and thinking. As a result complete focus on the problem at hand grows and involvement for the process at hand arises”. Secondly, Postma denotes that a team formation also helps in showing visible support by (top) management in that you can select teams in which at least one board member or manager is involved. Third, Postma also sees possibilities of teaming in letting the users participate. He “tries to break hierarchies through team play” and tries to construct “multi-disciplinary teams in which stakeholders can participate too”.
Two game elements were mentioned each by one interviewee for forming a guiding coalition, namely role play and visuals.
Stage 2: Need for change (Cawsey et. al, 2011)
CSFs: create a sense of urgency, create a strong business-related need for change
with regards to their role in the greater organization” (Stoppelenburg) and that “obtaining various perspectives of the greater whole increased the understanding for each other’s situation and the overall problem. In a standardized mathematical analysis such roles are set hardcoded, whilst games provide for more creativity here” (Durk-Jouke van der Zee, University of Groningen). Another interviewee mentions that a role play “widens the perspective of the players” (Martijn van Ooijen, Pentascope).
Imagination (imaging): “People tend to get stuck in old paradigms, mental models and behavior that belong to the old situation. By focusing on the new situation, and letting them experience this, people tend to change on their own”, says Stoppelenburg. She holds that “experiencing the change helps”. Another interviewee denotes that “the players will notice at the end that they have lived through the game and that reality has become much more explicit (Van Bree). Postma says that “involving people early on through imagining the change makes them owner of their own change. This makes them feel they own part of the solution and increases the need for change”. Fourth, Koster, notes that “if you tell people what to change, they will sometimes not find a need to change. While by gamifying the process and letting managers learn playfully the need for change becomes clear more easily”.
Game elements that are mentioned once for creating a sense of urgency are visuals, epic meaning, and reflection.
Stage 3: Create the vision (Cawsey et. al, 2011)
CSFs: envision the change, communicate the vision, set clear goals, seek simplicity in goal and path
visuals are also mentioned as a game element to provide for simplicity in this third phase of the change process. Koster elucidates that “visualization by using images help in creating simplicity and challenging group dynamics. Using visuals helps to make experiencing the change more simple.
Missions: Stoppelenburg uses missions in order to set clear goals for the upcoming change. “The long and short term missions act as intermediate goals to get from the current situation A to [the new and desired situation] B.” Van Bree agrees with this and uses missions as ways of obtaining collective goals. Van Bree makes sure that without finishing these missions, one cannot win [or finish] the game. Third, Postma uses missions in this phase of the change process to seek simplicity. “Complex things are broken into smaller pieces; this is done through missions and assignments during the game. At the end of such a mission, people often experience some kind of Eureka moment. That simplifies the game”
Role play: a third game element that is mentioned by three interviewees as an important link towards CSFs in this phase is role play. Van Bree clearly links role play to creating a shared vision. He indicates that “without such game elements such as role play, many stakeholders would above all mark their opposing interests. Role play helps in understanding each other’s perspective which makes the change more durable”. Van Ooijen continues on this by making clear that experiencing the different perspectives, through the role play, stimulates communication and makes for very concrete shared visions. Angeline van Gils and Marjolijn Traas - van Oort (Twynstra Gudde) mainly see role play as a mean of setting clear goals for the change ahead. “The role play forces people to think on what needs to be done and forces the players to set clear goals. During the intervention, the goals need to be met, so setting the correct goals is important.”
Game elements that are mentioned only by one interviewee are feedback, epic meaning (communicate the vision), metaphors (seek simplicity), decor (setting clear goals).
Stage 4: Gap analysis (Cawsey et. al, 2011)
Feedback: in the fourth stage feedback is mentioned as an important game element by three of the eight interviewees. In order to install a high degree of communication during this phase, “interaction is provided through feedback based on tracking mechanisms during the game”, as Van der Zee explains. “By registering this feedback, decision traces appear and can in turn be used to trigger interactions [amongst participants]”. “Feedback is an important part of a gamified platform since it let people participate and cooperate during the change. The feedback makes sure that the gamified process becomes a communicative model. More analytical non-gamified techniques cannot arrange for such levels of communications easily”. Secondly, Stoppelenburg explains that feedback can also facilitate communication. “Feedback in the form of learning loops creates a ‘solution-driven’ energy. This energy makes behavior discussable [and thus facilitates communication]. I call these discussions ‘multilogues’ since it links thinking to doing via the communication flows, feedback loops and debriefing exercises.” Third, Postma discusses the fact that feedback provides for ongoing communications. “During the game I formulate assumptions together with the players. After a session we discuss the assumptions and check to what extent they appeared valid. The visualized result of a session becomes a communication portal in this way.
Points: this game element is mentioned twice. Van der Zee and Van Ooijen both tell us that “you can grant points for desired behavior” and with that create community involvement or a high degree of communication.
Multiplayer: both Van der Zee as Van Gils and Traas - van Oort mention the possibility of multiplayer aspects in installing a high degree of communication. A multiplayer session namely forces people to think on what they are going to do and how they are going to do it, which lead to increased communication flows.
Stage 5: Manage change transition (Cawsey et. al, 2011)
CSFs: create short term wins, continuing involvement in the project, build on the change, progress meetings, follow a standardized process, translate design into reality, let the users and clients participate, reward for support of necessary changes, create a good task vs. social orientation
Wins: the game element ‘wins’ is mentioned by five of the interviewees. By some interviewees it is explicitly mentioned in the form of points, others generally refer to it as ‘rewards’. Van der Zee says he uses points to make sure players will attain the final performance elements by giving points in attaining the short term mission on the route towards the final performance elements. He denotes that “there must be wins involved in the game for it to succeed [that is, being effective]”. Van Ooijen adds to this by discussing the idea that by linking points to creative and wanted behavior, people will display this behavior more easily, since “people like to win; they like to get the euphoric feeling of a win”. Koster uses points in order to create continuing involvement in the project, which is an important CSFs in the 5th stage of the change process. Points make sure that every game or mission is different, “if players know what the game will be like and how it ends, you will lose a lot of involvement”. Koster acknowledges that the competitive thinking behind pointing systems depends on the player type that you are dealing with (see figure 3). Van Bree sees wins as a way of rewarding for support of necessary changes. The reward in his gamified interventions is the winning itself. From his perspective, making it possible to win something increases dedication and commitment towards the change. Fifth, Postma uses rewards to build on the change. He denotes that “when I ask people to give a pitch, nobody is willing to. But it appears that a reward is connected to the pitch, the second time I ask all of a sudden everyone wants to pitch. For me, that proves that rewarding helps for commitment and dedication which helps to build on the change.”
Flow: the idea of flow as a game element (see figure 2) is mentioned twice. Both Koster and Van Ooijen illustrate that flow helps to translate the gamified design into reality. This CSF is important in the 5th stage of implementation. Koster says that “people that participated [in the gamified intervention] cannot simply say ‘but this is just a theory’, they feel the change, they live through it. Accepting the reality is what seems to be left, it is inevitable”. Van Ooijen says that “the pure inclusion of the participants in the change process increases the commitment, since people themselves become the owner of the game and thus the change. The consultant is all of a sudden not present anymore (in a positive way) and becomes a mere facilitator.” It is all about their organization, their change. The flow that arises helps in the implementation phase.
8. DISCUSSION
the type of user involved. The findings presented in this thesis are useful to be applied in game-thinking. This thesis focussed on picking the correct game elements for the CSFs you want to attain. This is part of the game-thinking research which explains that you have to take the context, goal and type of user into account when gamifying a consultancy process. Many gamification projects are not constructed this thoughtfully, which is exactly the reason that Gartner suspects 80% of all gamification projects will fail.4 The idea behind this thesis is that it helps in further understanding how to apply gamification and that simply throwing in some points, team play and assignments will not satisfy the change goals. Very recent research by Domínquez et. al (2013) builds on this by concluding their article with the notion that gamification is “not trivial to achieve [the effect of increased motivation], and a big effort is required in the design and implementation of the experience for it to be fully motivating for participants” (Domínquez et. al, 2013:391). Hamari (2013) found similar results in his field experiment. This experiment held a gamification of a utilitarian peer-to-peer trading service by implementing the game elements badges that users can earn from a variety of tasks. Based on this experiment Hamari (2013) did not find any “support for the claims that implementing gamified features would alone lead to significant overall increases in usage frequency, quality or social interaction in a utilitarian trading service (…)” (Hamari, 2013:8). In other words, the mere application of game elements does not automatically lead to the desired outcomes.
This thesis dealt with some limitations. First of all as for every research, collecting the literature and data takes a lot of time. By the time this thesis is finished, many congresses on this topic are held, new gamification undertakings are started. In addition, some new literature is published, but unfortunately not on the application of game elements in consultancy. Secondly, in collecting our linkages we have aimed at selecting a wide range of consultants with expert knowledge in this field. However, the time frame severely limited the search for more heterogeneous experts from different fields which could have enriched the data. In addition, we have limited to number of interviewees to eight experts, after the 6th interview the level of repetition was already relatively high and the satisfactory level of saturation for this explorative research was reached after the 8th interview. However, since the level of saturation is based on the perspective of the researcher this is relatively subjective. This brings us at our third limitation,
namely subjectivity. No research was found dealing with the links between CSFs of a management consultancy trajectory and game elements from gamification undertakings. Therefore an explorative study was required to create this academic basis. This means that the level of subjectivity is comparatively high. As mentioned previously this becomes visible when picking a saturation level on the number of interviews, but also with regard to extracting CSFs – game element links from the interviews when they are not explicitly mentioned by the interviewees.
Future research based on this paper is possible in several ways. We suggest first of all the possibility to undertake more research on links between CSFs and game elements by trying to overcome some of the limitations described above. There must be more links possible to construct, they were probably limited for this research because of the limited interviews and time span. More research on the links would make game design more usable and would help in reducing the number of gamification failures. Secondly, future research can be built on CSFs of other processes than management consultancy or in specific industries, for example healthcare or manufacturing. Third, we propose to base future research on the latest academic findings proposed in the papers by Hamari (2013) and Domínquez et. al (2013) in which it is found that randomly applying game elements will not suffice the attainment of certain CSFs or project goals. Building on this, future research could focus more specifically on game-thinking research which includes research on the change context of the game, the goal and the type of player. This could strengthen the links between CSFs and game elements.
points for example) makes sure that players show a continuing involvement in the project and that they therewith build on the change.
To conclude, whilst writing this thesis it has appeared that the attention by academia and businesses on the practice of gamification and corresponding game thinking, which was largely under addressed whilst writing this thesis, is sprouting. This notion creates possibilities for more field research on gamification application. We hope to have contributed in creating guidelines for setting up a gamified undertaking and future research in this interesting field of study.
“Seriousness seeks to exclude play, whereas play can very well include seriousness”
9. REFERENCES B
Baker, B. N., Murphy, D. C. & Fisher, D. (1988) Factors affecting project success. In: Cleland, D. I. & King, W. R. (Eds.) Project Management Handbook, Second Edition pp. 902 – 909. New York: Van Nostrand Reinhold.
Bartle, R. (1996) Hearts, Clubs, Diamonds, Spades: Players Who suit MUDs. The Journal of Virtual Environments, Vol. 1, No. 1
Beckhard, R. and Harris, R.T. (1987) Organizational Transitions: Managing Complex Change. Reading, MA: Addison-Wesley.
Ben-Zvi, T. (2010) The efficacy of business simulation games in creating Decision Support Systems: an experimental investigation. Decision Support Systems 49 (1) (2010) 61–69.
Bogost, I. (2007). Persuasive games: The expressive power of videogames. Cambridge, MA: The MIT Press.
C
Carpenter, D.E. (1998) ‘Alea jacta est’: at the gaming table with Alfonso the Learned. Journal of Medieval History, Vol. 24, No. 4, pp. 333–345, 1998
Chapman, G.M. & Martin, J.F. (1995) Computerized business games in engineering education, Computers in Education 25 (1/2) (1995) 67–73.
Cawsey, T.F., Deszca, G. and Ingols, C.A. (2011) Organizational Change: An Action-Oriented Toolkit. California: SAGE Publications.
Johansson, B., Johnsson, J., Kinnander, A., Chick, S., Sánchez, P.J., Ferrin, D., Morrice, D.J. (2003) Simulation in Manufacturing Systems.
Cleland, D.I. and King, W.R. (1983). Systems analysis and project management. New York: Mc Graw Hill.
Csikszentmihalyi, M. (1975/2000). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass.
Csikszentmihalyi, M., & Csikszentmihalyi, I. (1988). Optimal experience. Psychological studies of flow in consciousness. Cambridge, MA: Cambridge University Press.
Csikszentmihalyi, M., & Nakamura, J. (2010). Effortless attention in everyday life: a systematic phenomenology. In B. Bruya (Ed.), Effortless attention: A new perspective in the cognitive science of attention and action (pp. 179–190). Cambridge, MA: The MIT Press.
Crookall, D. (2010) Serious Games, Debriefing, and Simulation/ Gaming as a Discipline. Simulation Gaming 2010, 41 (6): 898-920
D
Deterding, S., Khaled, R., Nacke, L.E., & Dixon, D. (2011) Gamification: Toward a Definition. In: CHI 2011 Gamification Workshop Proceedings. Vancouver, BC: Canada
Domínquez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C. & Martínez-Herráiz, J.J. (2013) Gamifying learning experiences: Practical implications and outcomes. Computers & Education 63 (2013) 380–392
F
Ferguson, C. J. (2007) The good, the bad and the ugly: a meta-analytic review of positive and negative effects of violent video games. Psychiatric Q, 78, 309–316.
G
Gee, J.P. (2003) What video games have to teach us about learning and literacy. Computers in Entertainment (CIE) - Theoretical and Practical Computer Applications in Entertainment archive, Volume 1, Issue 1, October 2003, pp. 20-20
H
Hamari, J. (2013). Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applications, 12. doi: 10.1016/j.elerap.2013.01.004
Hummel, H.G.K., Houcke, van, J., Nadolski, R.J., Hiele, van der, T., Kurvers, H., Löhr, A. (2011) Scripted collaboration in serious gaming for complex learning: Effects of multiple perspectives when acquiring water management skills. British Journal of Educational Technology, Vol. 42 No. 6, pp. 1029–1041
Huizinga, J. (2008) Homo Ludens: Proeve Eener Bepaling van het Spel-element Der Cultuur. Amsterdam, the Netherlands: Amsterdam University Press
Hunicke, R., LeBlanc, M., Zubek, R. (2004) MDA: A Formal Approach to Game Design and Game Research. Proceedings of the Challenges in Game AI Workshop, Nineteenth National Conference on Artificial Intelligence. Pp. 1-5
J
K
Kotter, J.P. (1996) Leading Change. Boston, MA: Harvard Business Review Press. 978-0-87584-747-4
Kotter, J.P. (1995) Why Transformation Efforts Fail, Harvard Business Review. March-April 1995:61
L
Lainema, T. & O.P. Hilmola (2005) Learn more, better and faster: computer-based simulation gaming of production and operations. International Journal of Business Performance Management 7 (1) (2005) 34–59.
Levit, S.D. & List, J.A. (2011) Was There Really a Hawthorne Effect at the Hawthorne Plant? An Analysis of the Original Illumination Experiments. American Economic Journal: Applied Economics 3 (January 2011): 224–238
Lewis, M.A. & H.R. Maylor (2007) Game playing and operations management education, International Journal of Production Economics 105 (2007) 134–149.
Lock, D. (1984). Project Management. New York: St. Martins Press
M
Martin, C.C. (1976) Project Management. New York: Amaco
McGonigal, Jane (2011) Reality if Broken – Why Games Make Us Better and How They Can Change the World. London, Great Britain: The Random House Group
P
Pinto, J.K., & Slevin, D.P. (1989). Critical success factors in R&D projects. Research Technology Management. pp. 31-35.
Pitman, B. (1994) Critical success factors to organizational change. Journal of Systems Management. Sep. 94, Vol. 45 Issue 9, pp. 40. 1p. 1 Black and White Photograph.
R
Ronald, D.D. (1961) Management Information Crisis. Harvard Business Review, Sep/Oct61, Vol. 39 Issue 5, p111-121
Ruohomaki, V. (1995) Learning and education with simulation games, in: J.O. Riis (Ed.), Simulation games and learning in production management, London, United Kingdom: Chapman & Hall, pp. 13–25.
S
Sawyer, Ben (2009) Foreword. In Ritterfeld, Ute, Michael Cody & Peter Vorderer (2009), Serious Games: mechanisms and effects, p.XIV. New York: Routledge
Smeds, R. (2003) Simulation for accelerated learning and development in industrial management – guest editorial. Production Planning & Control 14 (2) (2003) 107–110.
Sayles, L.R. and Chandler, M.K. (1971). Managing large systems. New York: Harper and Row
Stichting Toekomstbeeld der Techniek (STT) (2011a) Serious Games, Playful Business. Toekomstbeelden van de spelende organisatie [Future images of the playing organization] ISBN 9789080961302, No. 76. Den Haag, the Netherlands
Stoppelenburg, A., de Caluwé, L., Geurts, J. (2012) Gaming: Organisatieverandering met spelsimulaties. Deventer, the Netherlands: Kluwer
Subrahmanyam, K. & Greenfield, P. M. (1994) Effect of video game practice on spatial skills in girls and boys. Journal of Applied Developmental Psychology, 15, 13–32.
Sung, K.T. & Gibson, D.V. (1998) Critical Success Factors for Business Reengineering and Corporate Performance: The Case of Korean Corporations. Technological Forecasting and Social Change Vol. 58, pp. 297–311
T
Thomas M. Connolly, T.M., Boyle, E.A., MacArthur, E.A., Hainey, T., Boyle, J.M. (2012) A systematic literature review of empirical evidence on computer games and serious games. Computers & Education 59 (2012) 661–686
U
Uden, van, J. (2009) Toekomstverkenning Serious Gaming [Horizonstudy Serious Gaming] Stichting Toekomstbeeld der Techniek, Position Paper. http://www.stt.nl/gaming
Ullén, F., de Manzano, Ö., Almeida, R., Magnusson, P.K.E., Pedersen, N.L., Nakamura, J., Csíkszentmihályi, M., Madison,G. (2011) Proneness for psychological flow in everyday life: Associations with personality and intelligence. Personality and Individual Differences 52 (2012) 167–172
V
Van der Zee, D.J., Holkenborg, B., Robinson, S. (2012) Conceptual modeling for simulation-based serious gaming. Elsevier, In Press, Corrected Proof doi:10.1016/j.dss.2012.03.006
Z
Zichermann, G. & Cunningham, C. (2011) Gamification by Design. Sebastopol, California: O’Reilly Media, Inc
Zwikael, O. & Globerson, S. (2006) From Critical Success Factors to Critical Success Processes. International Journal of Production Research, Vol. 44, No. 17, 1 September 2006, 3433–3449
Y
Yin, R.K. (2003) Case study research: Design and methods. Thousand Oaks, California: Sage Publications
