Educating Goal Setting Theory usingGaming-Simulation

(1)

RIJKSUNIVERSITEIT GRONINGEN

Educating Goal Setting Theory using

Gaming-Simulation

The design and educational validity of a

gaming-simulation

Master of Business Administration: Operations & Supply Chains August 26th, 2014

Jelle Geertsema (s1903349; s1903349@st.rug.nl) Supervisor: W.M.C. van Wezel

(2)

1. Introduction

In the field of Behavioral Operations Management (BOM), Goal setting theory (GST) has the highest impact within a company (Locke, 2004). Of the 73 management theories it is GST that ranks number one on the list. It is therefore no surprise that GST gets the most attention within the field of BOM. GST its main premise is that encouraging people to pursue a goal that is specific and difficult will yield better performance than encouraging them to pursue a specific but easy goal or to simply do their best (Kleingeld, van Mierlo, & Arends, 2011). While this is true in most cases, conflicting research showed that there are moderators that have to be taken into consideration. Being a part of BOM, GST researchers looked for answers in the interactions between the users actions, its context – the operation processes - and the users cognitive state. Without the proper know-how on goal setting any attempts on goal setting will cause a misalignment between the employees’ goals, the system goals and the company’s goals.

Research in operations management used to be limited to the technical aspects. Operations Management investigates the design, management and improvement of processes aimed at the development, production and delivery of products and services (Gino & Pisano, 2008). BOM researchers have advocated to design or improve operation processes with consideration of both technical and human aspects (Bendoly, Donohue, & Schultz, 2006; Boudreau, Hopp, McClain, & Thomas, 2003; Gino & Pisano, 2008). With continuously new research in the field of BOM the scope of operations management has expanded to servicing and supply chain management as well (Gino & Pisano, 2008). The majority of operation processes are designed with the assumption that their users are predictable in their actions, independent from others, not part of the product or service and emotionless. Goal setting for these processes is used to create incentives to induce such behavior (Gino & Pisano, 2008). This ignores the fact that organizations are complex social systems where human behavior is a central driver. Goal setting can then become a disrupting factor. When goal setting uses human behavior as the central driver, the goals are designed to be high and difficult to challenge the user. While remaining realistic to give the user a sense of success, pleasure, satisfaction and happiness (Latham & Locke, 2006).

The need for further research in GST finds its foundation in the continuously changing workplace. The modern workplace is characterized by high task complexity, stress and work overload. Given the context the employee can feel either threatened or challenged by these factors. Goal setting moderates the employees’ perspective towards the workload (Drach-Zahavy & Erez, 2002). Identifying and familiarizing with the effects of GST within the modern workplace is therefore quickly becoming a managerial necessity.

(4)

self-assess on how goal setting theory affects them. The tendency to protect the self-image makes people resistant to any tendencies to see themselves in a negative fashion (Skowronski, Sedikides, Heider, Wood, & Scherer, 2010).

These educational problems are not new. The educative system uses the Bloom’s taxonomy to create a common language to address these problems. The same taxonomy is used in the paper to help understand the problems and possible solutions. The taxonomy describes several cognitive domains reaching from the lowest level being Knowledge – ability to recall learned material – to the highest level Evaluation – ability to judge the value of material for a given purpose (Tyran, 2010).

If educators wish for their goal setting theory classes to have its impact in the students’ lives, they need to ensure that the students get an understanding on how their own personality traits affect the goal-performance relationship. Using Bloom’s taxonomy we can describe the issue as the difference between the second cognitive domain level Comprehension – understand and interpret the material -and the third cognitive domain level Application – ability to use learned material in new situations (Tyran, 2010).

Exploratory research showed that gaming-simulations have the potential to learn application of the theory. Using gaming-simulations for teaching goal-setting theory benefits from the game’s potential to mimic human behavior and visualize the invisible. The potential of gaming-simulation for goal-setting theory can already be seen in other business gaming-simulations. In 'The Beer Game' the objective is to let the players understand the power of the bullwhip-effect (Sparling, 2002). The bullwhip effect occurs in supply chains where small changes in ultimate demand create exponentially large buffers up-stream the supply chain. In 'The Beer Game' the intangible notion of the bullwhip effect becomes tangible when the player piles up inventory and doesn't seem to be capable of stopping it. Mimicking the human role within the process and have visual effects to intangible relations make playing “The Beer Game” an intensive experience. Gaming-simulation could prove to do the same for goal-setting theory.

Whether gaming-simulation is the appropriate educational tool depends on the learning that needs to take place (Lederman, 1987). The learning subject and gaming-simulation go well together if the students benefit from the main strengths of gaming-simulation. These strengths comprise of (Lankford & Watson, 2007):

1. Gaming-simulations allow for testing risky actions which would not be acceptable in the real world;

2. Passing the barrier of language;

3. Processing several scenarios with complex interactions through the computer; 4. Mimic human behavior;

5. Creating an outdoor environment indoors; 6. Visualize the invisible;

(5)

Prior business gaming-simulations already have proven its validity as educational tool. Business gaming-simulations have grown considerably in use and have moved from being a supplemental exercise in business courses to a central mode of business instruction (A. J. Faria, Hutchinson, Wellington, & Gold, 2009). This is justified by a major research project across several universities that showed that using gaming-simulations gave better results than the control group with class lectures (A. J. Faria & Weffington, 2005). This shows that the standard class lectures are not necessarily flawed, but that gaming-simulations got strengths by themselves. The involvement of the student directly taps in the student’s intrinsic desire to learn (Kashibuchi & Sakamoto, 2001).

Through a gaming-simulation students can reflect on their own behavior when affected by goal setting theory. Gaming-simulations mimic the human behavior and can visualize the moderating effects (Lankford & Watson, 2007). The goal of a goal-setting gaming-simulation would be to help participants experience the following: (1) the goal-performance relationship and (2) the moderating effects on the goal-performance relationship.

Goal of this research paper is to develop an educational game on goal setting theory. The strengths of gaming-simulation – mimic human behavior and visualize the invisible – provides the solution to the before-mentioned issues when teaching goal setting theory.

(6)

2. Research design

The design of the research was added to the paper to answer questions on the research constraints; how the problem was finally stated; how the research (sub-questions) came to be and how the information was gathered.

2.1 Introduction to the research

To successfully fulfill the research objective, the limitations of the research has to be taken into account when designing the research.

The objective is to improve the toolset for educators to teach goal setting theory. Improvements in the educative toolset will help broaden the scope of users and improve the usage of goal setting theory. As mentioned in the introduction the toolset for educating GST is improved with the addition of a gaming-simulation.

The research has its constraints. To develop a game and find a population to statistically prove the validity of new hypothesizes would have exceeded the time constraints. Instead there was chosen to combine proven goal setting theory relationships and make these relationships more visible through gaming.

2.2 Problem statement

Erroneous real-life application of goal setting theory will do more harm than good. As described in detail in chapter 3, goal setting has a set of pitfalls that negatively effect on the goal setting-performance relationship. Examples of these pitfalls are: goals that are considered threatening create an adverse effect on risk-taking; overstating performance caused by monetary incentives (Latham & Locke, 2006). The educational goal is to adequately be able to apply goal setting in real life to avoid such erroneous actions.

Using standard classes to teach goal setting theory does not provide certainty that people will not fall into these pitfalls. In standard classes learning is removed from contexts in which the moderators are found (Wideman et al., 2007). Gaming-simulation has the potential to solve these issues by providing the context. The gaming-simulation would have to be designed so that its basic elements mirror goal-setting elements (Snow, Gehlen, & Green, 2002) . Exploratory research found no gaming-simulation on Goal Setting Theory elements.

2.3 Research Question

Given the research problem, the research question is formulated as:

“How can the usage of gaming-simulation as an educational tool accomplish the educational goal of participants to correctly apply goal setting in real life?”

(7)

educational validity of a gaming-simulation, the gaming-simulation should be on the stand as well. Therefore the research question was split into two sub-questions. The two sub-questions are:

Sub-question 1: How are the gaming-simulation elements designed to simulate the effects of goal setting in real life?

Sub-question 2: After participating in the gaming simulations, in which ways did participants show an improved understanding of goal setting theory?

The structure of the research is based on the research question and its sub-questions: sub-question 1 is addressed in chapter 5; sub-question 2 is addressed in chapter 6.

2.4 Methodology

2.4.1 Conceptual Model

By translating the research problem to the research questions a set-up towards the solution is made. As the research problem lies within the interdisciplinary field of goal setting and educational gaming-simulation the scope of the research is expanded to include all necessary concepts from these fields. Figure 1 shows the conceptual model to visualize the relationships between the concepts. Following Stainton’s theory (2010) the conceptual model includes two parts:

- Validate the gaming-simulation by testing it on its game design.

- Validate the gaming-simulation by testing its success on conveying the educational goal to the participants - being able to correctly apply goal setting in their own real life situations.

Figure 1 : Conceptual Model

2.4.2 Research Setup

To conclude, further parameters such as the chosen type of research, the chosen way of information gathering and the inevitable restrictions on the research have been grouped in this section.

Research Type

(8)

implication, it can be used in divers situations, its foundations lie in goal setting and game design theory and its output can be used as validating tool for the used theory. The last step in the research is best characterized as evaluation research. This last step is to measure the performance of the gaming-simulation in achieving its educational goals.

Information gathering

For the literature review studies from the main journals in the different fields were used. For the evaluation research the preferable method is to use controlled experiments. There is a strong presumption that goal setting lends itself easily for controlled experiment. The regular testing method used for studies in social psychology and group dynamics is through a controlled experiment (Bendoly, Croson, Goncalves, & Schultz, 2010). This method is used in this research. The controlled experiment consists of playing the gaming-simulation and debriefing it afterwards. Measurements in the debriefing show if the educational goal is met.

Limitations

(9)

3. Goal setting theory

Leading the front on goal setting theory for the past decades are Locke and Latham. Their paper (2002) summarized the main discoveries in goal setting theory (in which they played a large role themselves) and helped structure the chapter. New studies from the past decade - including conflicting studies – were added where appropriate. The information is limited to the GST elements that were used in the gaming-simulation. The summary of the GST elements can be found in Table 1.

3.1 Short Introduction to GST

The field of goal setting theory includes studies on (Latham & Locke, 2006): The effectiveness of specific, difficult goals; the relationship of goals to affect; the moderators of goal effects; the relation of goals to self-efficacy; the moderators of goal effects; the generality of goal effects across people, tasks, countries, time spans, experimental designs, goal sources, and dependent variables. Including all GST elements into a gaming-simulation would be a too large an effort for the research's objective. Therefore only the most important studies are described. See Figure 1 for the concepts that are included in this chapter and their relations.

Figure 1 : Relationships between goal setting theory concepts (Lock e & Latham , 2002)

(10)

3.2 Goal Setting Theory concepts

As Figure 1 shows, the potential of goal setting to positively affect performance depends on the goal, what you consider good performance and on the moderators. This paragraph is on how to correctly set the goals, how to decide on the performance measurement and how to take the moderators into account.

Goal Core – Specificity & Difficulty

An outcome goal is a measurable objective which is to be obtained at a set time. Outcome goals intend to promote maximizing task utility and increase output. A learning goal is considered more difficult in terms of the number of possibilities to perform a task. Learning goals focus on learning the task first and utilizing the task second (Selits & Latham, 2001). Whether to set either a learning goal or an outcome goal depends on the current skill-set of the person.

When the person has the skill-set to perform the task a specific, difficult outcome goal motivates the person to exert effort and persistence to achieve the outcome (Latham & Locke, 1991). If the person has used strategies which have been proven to be effective in the past, they are substantiated to repeat the strategies. If the person is lacking the skill-set to perform the task a learning goal is more appropriate. Learning goals give high self-efficacy to the person (Selits & Latham, 2001). Finding new strategies increases the commitment to the goal and increases the self-efficacy. Which leads again to finding new strategies. This is also known as the high performance cycle (Locke & Latham, 2002). The cycle also shows in Figure 1.

Allowing the subject to set its own goal increases the task enjoyment the subject experiences and increases his determinacy to perform well. As seen in Figure 3 this also increases the self-efficacy. By self-regulating the goals, people enter the high performance cycle faster. Benefits of self-regulating were found most strongly in a mid-long term (6 months in the study) where the benefits of the high performance cycle are best visible (Locke & Latham, 2002).

Figure 1 : Locke & Latham's Model for subsequent venture growth indicate a positive relationship between tenacity, self-efficacy and venture growth. (Bau m & Locke , 2004)

(11)

2002). In the design of the gaming simulation, one of the players has the misfortune to have a specific difficult outcome goal without having the required skill-set.

Performance Improvements

Improved performance can either target the productivity - which is generally achieved by setting high outcome goals – or task improvements – which is generally achieved by setting learning goals. If a high goal is achieved its benefits do not limit themselves to the company. Employees or students that achieved a high goal feel high appraisal and satisfaction. The high performance cycle can thus have similar effects for high outcome goals. Satisfied employees do not necessarily make productive employees. But productive employees can lead to satisfied employees who in return will be more productive (Locke & Latham, 2002). Both outcome goals as well as learning goals have longer-lasting effects than the time-span that is set for reaching the objective. If fulfilled, both will increase the self-efficacy of the person. All leading researchers (Baum & Locke, 2004; Locke & Latham, 2002; Selits & Latham, 2001) cannot emphasize enough the importance of self-efficacy. Baum (Baum & Locke, 2004) even goes as far as putting self-efficacy as one of the driving forces behind venture growth in its model for venture growth (see Figure 3). Both objective goals as well as learning goals are proved methods to improve performance. Both share an emphasis on the importance of high self-efficacy. In the design of the gaming-simulation, one of the players is given a learning outcome goal. They are expected to show the most enjoyment in the gaming-simulation, and most notably a higher participation in the feedback rounds.

Moderator - Self-efficacy

One way to increase goal commitment is the high performance cycle. Self-regulation usually increases the commitment as well. The most common method to increase goal commitment is by creating – usually monetary – incentives. All these steps have a positive effect on the goal-performance relationship (Locke & Latham, 2002).

Moderators – Feedback

The effects of feedback on the goal-performance relationship are most significant when the goal is long-term. Without feedback people have no idea if their current actions will lead them to the goal, and don’t get motivated. If the goal is split in sub-goals with each its feedback round, people are inclined to even increase the level of future goals if current sub-goals are met (Locke & Latham, 2002). Feedback rounds also affect risk-taking. Locke's research on error management shows that providing us with (a) ample opportunity to make mistakes and (b) explicitly encouraging us to learn from our errors, improves our performance (Locke & Latham, 2002).

There are 2 feedback rounds in the game. The first feedback round after the first phase is mostly intended to clearify any misunderstandings on the game and keep them motivated to continue investing in the game. It is expected that part of the improved performance in the second phase is caused due to the feedback round after the first phase. In the gaming-simulation the players with the learning goals are expected to be more involved during the feedback rounds as they are more committed to 'finding the answer'.

Moderators – Task complexity

(12)

impact. With learning goals the people are given the time and space to develop the strategies. High learning goals also increases performance when compared with low learning goals. Whether learning goals are set high or low, they will outperform outcome goals if the goal is to develop the required skill-set for the task (Locke & Latham, 2002).

To ensure that each player is able to develop strategies for the task the tasks are simplified as much as possible. The strategy then lies in balancing the tasks without being limited by any miscomprehensions on the tasks.

Moderators – Environmental uncertainty

A distinction is made between controllable causes and uncontrollable causes. Stable causes are causes that the person feels that he can control. Examples are the skill level of a person or the time spent on the goal. With the controllable causes the person can see areas to improve on and expect to reach the goal on the next attempt. If the cause for the unexpected results was stable, the goals are mildly adjusted. Unstable causes are causes that the person feels he can’t control. Examples are bad weather or sickness. With the uncontrollable causes the person will account for the possibility that similar causes are to occur again next time and adjusted the goal accordingly. If the cause for the unexpected results was unstable, the goals are strongly adjusted (Donovan & Williams, 2003).

Mechanisms - Direction

An idealized goal pronounced by a leader, group, or organization helps focus the efforts on achieving the goal. But it is not without its risks. Non-goal performance dimensions sometimes get ignored. (Locke & Latham, 2002). It can also lead to irrational, risky actions aimed at attaining the goal regardless of possible costs and consequences (Locke & Latham, 2002). The importance of a well communicated vision was seen earlier in Figure 3 in Baum’s (Baum & Locke, 2004) model on organization growth.

Mechanisms – Strategies

Past successes increase the participant's satisfaction. This satisfaction can lead the person to continue to draw upon his previously successful strategies. High satisfaction typically leads to increasingly high self-confidence and the setting of even higher goals to attain. This in turn frequently leads to dysfunctional persistence of previously used strategies if there is a radical change in the business environment(Locke & Latham, 2002).

To include a radical change in the business environment and witness its effects would extend the game's time-span. To an extend that it would needed to be played over a multitude of sessions. Therefore this was GST-element was left out of the gaming-simulation.

Satisfaction

(13)

4. Gaming-simulations

In the introduction it is suggested that gaming-simulation would be an appropriate medium to educate goal setting theory. This chapter is committed to explore the potential of gaming-simulation to maximize its utility. The chapter starts with a short introduction to gaming-simulation in section 4.1. The educational validity of gaming-simulation is further explored in section 4.2. Detailed descriptions of the required gaming elements are found in section 4.3 to pave the way towards the gaming-simulation on goal setting theory.

4.1 Introduction to gaming-simulation

Reading up on gaming-simulations, it became clear that within the gaming-simulation community there is still debate over what defines a gaming-simulation. Feinstein (Feinstein, Mann, & Corsun, 2002) emphasises the human element calling games 'interactions among players placed in a prescribed setting and constrained by a set of rules and procedures.' With the upcoming of the computer the term simulation often became short for computer simulation and all other simulations became known as games. Lankford & Watson (2007) have a more general description stating that simulations aim “to enrich the gaming-simulation sufficiently so it will effect actions in the real world.” In common use the terms gaming and simulation are interchangeable. For the research the usage of the term simulation was chosen. It covers both descriptions as the gaming-simulation its ultimate goal is to raise interaction and increase the cognitive level to apply their knowledge in real life.

The current dominant type of educational games provides a game shell to content that is found in the standard classes. For example, in the game “Grab” players improve on the score track by answering questions on time management correctly (Kirk ,2001). While it will keep students more active and involved, it can hardly be said users will be able to apply the gained knowledge in real-life. Classes and game shells help people comprehend the material, but application is on a higher cognitive level. Bloom’s taxonomy (1956) shows it best by placing knowledge within its cognitive domains. As mentioned in the introduction, Bloom’s taxonomy is used to show how the knowledge impacts the user. At the lowest level the user is able to recall the learned material. At the highest level the user is able to judge the value of material for a given purpose (Tyran, 2010). At higher levels the learning will be represented through more in-depth explanations of cause-and-effect relationships, as well as an ability to apply concepts more broadly (Stainton 2010). When relating these cognitive domains to the different educational mediums, class courses provide the user with the ability to recall learned material. If the material is successfully put within a gaming shell, the extra engagement towards the learning material will help users to recall the learned material over a longer period of time.

4.2 Educational Validity of a business gaming simulation

(14)

personally meaningful and valued social and material worlds in which game learning takes place may be “virtual” from an outsider’s perspective; however, they have a psychological reality for the player that directly mediates the player’s level of immersion, persistence in the face of challenges, and intrinsic desire to learn (Wideman et al., 2007).

Gaming-simulation also works well for business themes. Operating plans and systems are characterized by its large size, the high costs and the high complexity. Wrong decisions on its design have a strong negative impact on the entire business. Gaming-simulation can make it possible for new, situated understanding to be developed through embodied experiences in complex domains that are otherwise inaccessible (Wideman et al., 2007).

In business courses gaming-simulation has to fit with what is taught and be well-prepared. These are considered the main problems of using gaming simulations (A. J. Faria & Wellington, 2004).

In the following paragraph the design of a gaming-simulation is further explained.

4.3 Designing and validating processes

Since the gaming-simulation is created as part of answering the research question, both the creation of the gaming-simulation (section 4.3.1) as well as the end results (section 4.3.2) has to be accounted for.

4.3.1 Design Process

The design process was structured to mirror the process used for the gaming-simulation “The Marble Company” by Lederman (1987). See Figure 1 for these steps.

Figure 1 : The Simulation Design Process (Lederman 1987)

Defining the problem

The objective of the game needs to address the problem. This includes a clear specification of the learning problems. At this point the constraints like money, energy and resources on what can be spent are taken into consideration. Estimation on whether using gaming-simulation is the most appropriate communication mode is done here to justify further continuation. The justification of the game is repeatedly questioned during the test-play.

Constructing the model of reality

To link the gaming-simulation with real-life, Lederman suggests to follow these five steps: 1. Create roles for players to fulfil

(15)

Company” the goal is to teach participants differences in organizational communication and therefore places the roles on different places in the organizational hierarchy.

2. Create interactions between those roles

With the variation of roles comes the variation of interactions. In the example of “Marble Company” the roles are hierarchal (top, middle and lower management), therefore the interactions are designed to be hierarchal (top to middle, middle to lower management).

3. Rules to control the interactions

This include how the roles may interact with each other as well as the tasks the roles perform and the rewards and punishments they receive for performing the task well. In the example of “Marble Company” the rules are modelled to the rules of interaction in an organizational structure as that’s what the game tries to mimic.

4. Goals that promote the interaction

The participants need some goals or purposes for engaging in activities. In the example of the “Marble Company” activities are fulfilled successfully by communicating the tasks through the organizational structure. With a randomizer the rules of interaction are changed between the rounds. Via this way players find how different communication forms impact the success-rate of communicating tasks through the organization.

5. Outcome criteria for game boundaries.

One final criteria to determine when the gaming-simulation is over. In the “Marble Company” this is simply by a beforehand determined time limit. This can be designed to ensure that all participants have met the learning objectives of the gaming-simulation.

Specification of sequence of events

The sequence of events follows from the game lay-out as constructed by the 5 previous steps. A clear specification of the sequence and modification of the tasks makes the tasks function in-game. The events can be found in full detail in the appendix.

Testing the model

When test-playing the gaming-simulation we check for its validity, reliability and utility. The reliability refers to the replay ability of the gaming-simulation, including the predictability of the outcomes. For the outcome there is the distinction between externally set and internally set (Lederman, 1987). Externally set outcomes have the benefit of being predetermined and therefore predictable. One example is to teach someone to shift gear in the car. The process has one clear right way and only requires repetition to master the skill. Internally set outcomes depend on how the players make their decisions. Internally set outcomes can still be considered predictable if the process of discovery within the game is consistently replicated.

(16)

mixture of externally and internally set goals within a game. In the gaming-simulation the outcome is internally set, and the skills are learned by consistently replicating the process each turn.

When checking for the utility of the gaming-simulation the benefits of the game are weighed against the costs (time, energy, money) of the game. This can also include the opportunity costs of the gaming-simulation.

Creation of materials and manuals

Materials and manuals are needed for the participants and facilitator. Materials help symbolize elements in the real world, manuals help the reliability of the game. The rulebook can be found in Appendix I: Rulebook for the game-simulation.

4.3.2 Validation process

In short the validity of gaming simulation depends on the simulation design and the manner in which the simulation is implemented (Stainton et al., 2010).The design phase of the gaming-simulation is therefore added in chapter 5. Stainton (2010) identified a number of key design aims that will help in benchmarking a gaming-simulation with other gaming-simulations:

1. The representation of real-world business environment

2. Content should be relevant, stimulating, rewarding, expand on the current knowledge level and have several learning objectives.

3. Implementation. Players have to be accountable for decisions, be able to apply the theory and reflect on theory. The game has to provide a dynamic environment to repeat these steps.

(17)

5. Designing the gaming-simulation

As mentioned in section 2.3, Stainton(Stainton et al., 2010) addressed the issue that studies on gaming-simulation as an educational tool should assess both the gaming-simulation design as well as its educational validity. This led to the following two sub-questions:

Sub-question 1: How are the gaming-simulation elements designed to simulate the effects of goal setting in real life?

Sub-question 2: After participating in the gaming simulations, in which ways did participants show an improved understanding of goal setting theory? The first sub-question on the gaming-simulation design is answered in the current chapter. The second sub-question is answered via measuring the experiences of participants after playing the gaming-simulation and is addressed in chapter 6.

Both Lederman's (1987) design process for the game “Marble Company” and Kirk’s (2001) design process for the game “GRAB” were used for structuring the design process. Lederman’s designing stages formed the main structure as it is meant for designing an advanced endogenous game like “Marble Company”. Whereas Kirk’s design is for creating game shells like “GRAB’. However, Kirk’s design includes elements such as facilitator qualifications to ensure proper usage in courses. The paragraphs in 5.1 follow the design stages of Lederman, but elements of Kirk’s design processes were added where appropriate. Section 5.2 describes the game balancing to finalize the gaming simulation. The chapter is summarized in section 5.3. Appendix 1 has the rulebook for the gaming-simulation with the final details on the game. This chapter is meant to give insight in the creation of the game and with it its validity. It is recommended to read the rulebook in the appendix first, however not required.

5.1 Design Stages

5.1.1 Defining the problem

The gaming-simulation needs to fill a gap in gaming-simulations designed for educating goal setting theory. The educational goal is to be able to apply goal setting theory in real life to avoid goal setting pitfalls in real life. Given the educational setting the gaming-simulation falls under serious gaming and has to meet the standards that are set for educational games.

Purpose & Learning Objectives

Possible interested parties are students who are interested in goal setting theory, managers that are looking to redesign the goal setting to increase effectiveness and employees that desire a better understanding on how goal setting affects their work.

The game is intended to be part of a seminar, accompanied by the proper start-up and feedback and guidance provided by the facilitator. The role of the facilitator is further described later on.

(18)

5.1.2 Constructing the model of reality

5 steps were taken to link the gaming-simulation with real-life:

Create roles for players to fulfil

In order to measure the goal-performance relationship players are given goals and tasks which they can perform in variable degrees of success. The tasks need to be clear enough to be easily explained and complex enough for players to make mistakes and have a sub-optimal performance.

In the gaming-simulation the players are given the role of forecasters. They need to make predictions based on previous results.

Create interactions between the roles

All participants take on the same role as to enable comparing their individual performances. As a result their interactions are limited. As they have similar roles they can learn from each other during the feedback rounds. A direct player-to-player connection in the game would strongly influence the other players performance to the point where the outcomes aren't predictable anymore. This would harm the replay ability of the game and discredit the gaming-simulations validity as an educational tool. The interaction is therefore limited to the feedback rounds for a more controlled environment.

Rules to control the interactions

The rules control the interaction between the gaming-simulation and the players. It is up to the facilitator to make the rules clear at the start of the game. And make sure players play by the rules during the game. The rules describe the actions the players can perform and how these actions relate to the games' distribution of rewards and punishments.

 Role tasks

The players assume the role of forecaster and as such it is their task to forecast. In order to keep the game easily approachable the tasks are heavily simplified. This allows the game to be structured using only a deck of cards and a limited set of rules. The set of rules is described in the rulebook, which is to be found in the appendix.

 Rewards and punishments

Once the draw pile is depleted, the player his performance is measured. He receives 1 point for each card laid face open on the table and is penalized 1 point for each card lying in the discard pile. The final amount of points indicates his performance. The higher the score, the higher the performance.

Goals that promote the interaction

(19)

Outcome criteria for game boundaries.

The game is split into two phases. Each phase consists of 3 rounds of going through the gaming-simulation. This allows for pre-set times for the facilitator to conduct group discussions. After the second phase the session comes to an end. At this point the facilitator debriefs the players.

Creation of materials and manuals

The rulebook can be found in Appendix I: Rulebook for the game-simulation. The rulebook includes:  A list of required components

 The full set of order cards and meeting cards in printable sizes

 The manual for the facilitator including how to control the game, how to arrange the group discussions and how to arrange the debriefing

The rulebook is placed in the appendix as its content is meant to guide the gaming-simulation. Reading the rulebook will increase the understanding of the game.

5.1.3 Facilitator qualifications and duties

The facilitator guides the players through the game rounds. It is the task of the facilitator to keep the tempo going during the game and the conversation going during the feedback. To successfully facilitate the game the facilitator requires the following skill-set (Kirk et al., 2001):

 The know-how on goal setting and related BOM-concepts.  Experience on facilitating gaming-simulations

 Understanding of the game structure and the reasoning behind it.  Skilled in explaining the procedures from in-game and from real-life.  Able to adapt to the situation as every game will pan out different.  Qualities of a host to provide a pleasant learning environment. The duties of the facilitator during the seminar are:

 Introduce the players to the field of goal setting with a basic introduction course.  Introduce the players to the game rules

 In case of multiple groups, split the group up to 4 people per group.

 Ensuring that the game-play is at a proper pace and everyone remains involved.  Debriefing the players.

5.1.4 Debriefings Ideas and Suggestions

(20)

situations beyond the gaming simulation experience. To conduct a gaming simulation without including adequate debriefing is ineffective and even unethical (Kriz, 2010).

Kriz (Kriz, 2010) wrote an excellent debriefing guide called the Six Phases of Quality Debriefing. These steps were followed to create a guideline for the debriefing. In the end however the quality of the facilitator has the largest impact to the quality of the debriefing. The six phases can be found be found in the rulebook. These phases were also used as the framework for chapter 6.

5.1.5 Other Details

Number of Players

The game is designed to be played with a group of four players.

Time & Resources Required

The game is intended to fit in a hour and a half during seminar. This includes the introduction and the feedback. A run-through of the gaming-simulation requires a facilitator, the components as described in the rulebook and a proper gaming environment.

Game Rules

All the components – including the game rules – that make game-play possible are enclosed in the rulebook.

5.2 Balancing the Game

Multiple researchers (Kirk et al., 2001; Kriz, 2010; Lankford & Watson, 2007; Lederman, 1987) acknowledge that real-life elements don’t always translate well into game elements. There is a trade-off between the realism of the elements and the playability of the elements. By play-testing, adjusting and further play-testing a correct balance was found between the playability of the game and its relationship to reality.

Physical and/or material qualities

The aim was to make the required amount of materials as low as possible. A deck of cards was chosen as the main source of materials. Its advantages are that a deck of cards are easy attainable, have low costs and - given everyone's familiarity with cards - lowers the amount of required explanation. One of the major drawbacks is that the game does not mimic the business environment well.

Metaphysical qualities

Levity

(21)

translate the gaming-simulation into real-life situations. The importance of the facilitator for the success of the gaming-simulation cannot be overlooked. A large part of the rulebook directly addresses the facilitators role for that reason.

Specifity

The level of detail that is put into the game strongly affects the games’ realism. More realistic versions of the business environment proved to add too much distraction within the gaming-simulation. Although specifity helps with the realism, it can distract the player from the goal-performance relationship. It generally also adds more material requirements, time, rules and assumed know-how.

Spatiality

In the gaming-simulation the player has a helicopter-view on all the processes at work. Its downside is that it takes away some realism of the game. The upside is that it provides a sense of context of their actions and allows group discussions to happen with full disclosure.

Attachment

Depending on the reasoning behind participating in the game players can be emotionally attached or neutral when starting the game. Since lower performances at some points of time are inevitable given their goal cards, frustration and irritation is expected to develop. This is realistic to real-life reactions to feeling inapt. Besides realism it has value as goal setting phenomenon. This is why it is left intact as it is, but the facilitator should be aware of possible irritations. During phase 1 in the debriefing players are allowed to express their emotional state. For the facilitator a moment to act carefully, but also an opportunity to address the cognitive role within goal setting theory.

Ingenuity

Players are offered the opportunity to explore different strategies, but they can also stick to “what they know". Whether they choose to stick to what they know, depends for a large part on their assignment within the game.

5.3 Summary and conclusion on designing the gaming-simulation

The current chapter was added to ensure that the gaming-simulation its design lies in the scientific gaming-simulation papers. This also gives answer to the first of the two sub-questions: How are the gaming-simulation elements designed to simulate the effects of goal setting in real life?

The game is intended to be part of a seminar, accompanied with the proper briefing and debriefing by the facilitator. The role of the facilitator is further described later on. At the end of the seminar players will have gained a better understanding of the key elements of goal setting.

(22)

The most important role is that of the facilitator. Facilitator qualifications contain of proper goal setting theory knowhow, understanding the gaming-simulation to its core principles and able to create links to real-life. During the seminar the quality of the facilitator when briefing, when controlling the game tempo and when debriefing determine for a large part the successful outcome of the seminar.

During test-play the game was balanced between its realism and its playability. The link between goal-setting and players their performance relied much stronger on the game design than did the link between the game and real-life. The latter can be addressed at the debriefing.

6. Assessing the gaming-simulation

To see if goal-setting is applicable in real-life situations participants were tested on their ability to apply goal setting theory in real-life situations. The information is gathered during the debriefing during play-testing the gaming simulation. Since goal setting theory also includes concepts such as the cognitive state of the user, all debriefing steps will be included in this chapter.

This chapter gives answer to the second of the two sub-questions: After participating in the gaming-simulation, in which way did participants show an improved understand of goal setting application in real-life situations?

The data was gathered from a series of play-rounds with different colleagues. They all had academic backgrounds, but not in the field of business nor in the field goal setting theory. Based on the limited dataset it lacks the statistical significance of the results to be used as scientific proof. Section 2.4.2 on the research limitations already described that, given the constraints of the research paper, the desirable outcome of the research is an indication on whether gaming-simulation is a valid educational tool for teaching GST or not.

Phase 1: Discuss the emotional state

(23)

Phase 2: Observations on the activities

After explaining the rules the game proceeded without much problem. Players who had to find the median took longer each round, which kept the others waiting. Perhaps it is preferable to limit these players to two rounds when using the gaming-simulation in larger groups.

Phase 3: The relationship between experiences and reality

None of the players are active in roles that are similar to the roles they played in the gaming-simulation. Here it depends on the quality of the facilitator whether he can generalize the gained knowledge. In the rulebook are case examples for the facilitator to use. After the first phase people were unable to see the gained knowledge outside the context of the game. After all, it is just a simple card game and their choices were limited and not very impactful. Only after they saw how other participants with other goals viewed the gaming-simulation were they able to broaden the discussion.

Phase 4: Generalize the gained knowledge

It prove difficult to shift the conversation from the game to a more general discussion on GST. Players felt they had to justify their scoring as being the result of their limited influence and the high randomness within the game. Even though their scoring and behavior were as to be expected when applying GST. This showed the importance of the first phase where players need to cut the emotional ties with the role they played in the game.

One of the main concepts the players picked up on were the different attitudes the different goals toward the game created. A second main concept is the incentive to improve the performance. No actual incentives were given for better performance. One would only gain higher scores. They had entered the high performance cycle by improving their results by increasing control over their actions by increasing their skill level. This by itself was enough to keep them motivated. Admittedly, this didn't necessarily happen without some suggestions and tips from the facilitator. Nonetheless, players always showed a willingness to go another round to reach for higher scores.

A third main concept was the role of uncertainty in the game. In the first round they focused on the uncertainties and how it affected their performances. In following rounds they considered the uncertainties as part of the system, but focused on themselves and how to improve despite the uncertainties. One of the participants worked for a detachment agency. He noticed that he had similar experiences on the work floor. Irregularities that he once considered to be a nuisance later became part of the system (the company's culture).

Phase 5: Speculate on the game

At the end of the game they felt that the game could have more variety of strategy and more viable solutions. I would agree with them that the re playability of the game is low, as that is not the main concern with an educational goal. It would be interesting to see if more complex simulations would increase the difference in outcome between the goal cards. But this falls outside the scope of this research.

Phase 6: Address real-life issues

(24)

the gaming-simulation, was an heightened understanding of how the goal setting directly affects their cognitive state. Some of the participants felt discouraged by their companies to develop themselves within the company. The general consensus was that if the company was unable to set proper goal setting, the employees should consider setting goals for themselves as well.

Conclusion

An issue yet to overcome in current GST courses is how to convey the impact of personality traits on the goal – performance relationship and vice versa. To overcome the issue educators need to include material in the course that approaches the student on a higher cognitive level. Only then the students elevate from comprehending the GST material to be able to apply them in their own lives. The potential of gaming-simulations to mimic human behaviour and visualize the invisible makes it a welcome addition to GST course material. The goal of such a goal-setting gaming-simulation would be to help participants experience the goal-performance relationship and the moderating effects on the goal-performance relationship. The research question is formulated as: “How can the usage of gaming-simulation as an educational tool accomplish the educational goal of participants to correctly apply goal setting in real life?”

In order to properly measure the educational validity of a gaming-simulation, the gaming-simulation should be on the stand as well. When test-playing the gaming-simulation was checked for its validity, reliability and utility. As is common with serious gaming used in higher education, the gaming-simulation is accompanied by the proper start-up and feedback and guidance provided by the facilitator. The gaming-elements were designed to pass the validity test for usage as an educational tool. The game has to be easily linked with reality, consists of easily recreated materials and a manual, and the game needs to be conducted by a qualified facilitator.

As GST predicted the players with the learning goals led the discussion in the feedback rounds. They also showed a more deeper understanding of the game elements. Whereas the player with the self-setting outcome goal often felt victimized by the randomness within the gaming-simulation, while showing no deeper understanding on the game dynamic.

It prove difficult to shift the conversation from the game to a more general discussion on GST. Players felt they had to justify their scoring as being the result of personal mistakes or well-doing. Even though their scoring and behavior were as to be expected when applying GST. Players saw in the feedback round that the different goals made for very different attitudes towards the game. Between the two sets of round a change in perception of uncertainty was seen as well. In the second set of rounds they considered the uncertainties as part of the system, but focused on their own role in the system and how to improve on that role.

We did agree that for future work environments it could be preferable to alleviate their goal. That if you wanted to make a career in that environment one could best focus on self-improving their role in the system to keep themselves motivated and passionate and not been set back by any uncertainties.

(25)

player had a different goal card. Several people engage simultaneously on an exercise in GST and are motivated to converse with each other about GST.

(26)

Appendix I: Rulebook for the game-simulation

Introduction

The game is part of my final thesis. I refer to the thesis paper for explanation on used concepts and reasoning behind the set-up of the game. This rulebook is used to describe the set-up and the mechanics of the game. It also describes the requirements for the facilitator and the tasks of the facilitator. The components necessary to play the game can be found at the end of the rulebook. After reading the rulebook and obtain or create the game’s components, anyone should be able to facilitate the gaming-simulation.

Table of contents

1. Introduction... 3

2. Research design... 6

2.1 Introduction to the research...6

2.2 Problem statement...6 2.3 Research Question...6 2.4 Methodology...7 2.4.1 Conceptual Model... 7 2.4.2 Research Setup... 7 Research Type...7 Information gathering...8 Limitations...8

3. Goal setting theory... 9

3.1 Short Introduction to GST...9

3.2 Goal Setting Theory concepts...10

Goal Core – Specificity & Difficulty... 10

Performance Improvements... 11

Moderator - Self-efficacy... 11

(27)

Moderators – Task complexity... 11

Moderators – Environmental uncertainty... 12

Mechanisms - Direction... 12

Mechanisms – Strategies... 12

Satisfaction... 12

4. Gaming-simulations... 13

4.1 Introduction to gaming-simulation...13

4.2 Educational Validity of a business gaming simulation...13

4.3 Designing and validating processes...14

4.3.1 Design Process... 14

Defining the problem...14

Constructing the model of reality...14

Specification of sequence of events...15

Testing the model...15

Creation of materials and manuals...16

4.3.2 Validation process... 16

5. Designing the gaming-simulation... 17

5.1 Design Stages...17

5.1.1 Defining the problem... 17

Purpose & Learning Objectives...17

5.1.2 Constructing the model of reality... 18

Create roles for players to fulfil...18

Create interactions between the roles...18

Rules to control the interactions...18

Goals that promote the interaction...18

(28)

Creation of materials and manuals...19

5.1.3 Facilitator qualifications and duties... 19

5.1.4 Debriefings Ideas and Suggestions... 19

5.1.5 Other Details... 20

Number of Players...20

Time & Resources Required...20

Game Rules...20

5.2 Balancing the Game...20

Physical and/or material qualities...20

Metaphysical qualities...20 Levity...20 Specifity...21 Spatiality...21 Attachment...21 Ingenuity...21

5.3 Summary and conclusion on designing the gaming-simulation...21

6. Assessing the gaming-simulation... 22

Phase 1: Discuss the emotional state... 22

Phase 2: Observations on the activities... 23

Phase 3: The relationship between experiences and reality...23

Phase 4: Generalize the gained knowledge...23

Phase 5: Speculate on the game... 23

Phase 6: Address real-life issues... 23

Conclusion... 24

(29)

Introduction... 26

About the Game... 30

Game components... 31

Game setup... 31

The Rounds... 31

Step 1. Draw a card ...31

Step 2. Voice the prediction...31

Step 3. Turn the card...31

Feedback at the end of each round... 33

Six Phases for Quality Debriefing...33

Phase 1: Discuss the emotional state...33

Phase 2: Observations on the activities...33

Phase 3: The relationship between experiences and reality...33

Phase 4: Generalize the gained knowledge...34

Phase 5: Speculate on the game...34

Phase 6: Address real-life issues...35

Components for print... 35

(30)

About the Game

The objective in the gaming-simulation is to increase insight into the goal-performance relationship. All players work similar jobs at the same company. The job is to forecast if the next demand is higher or lower than the current demand. The goals are the only part where the players differ from each other. The game has two sets of three rounds and all rounds are played out the same. After each round the facilitator conducts a small group discussion. Mainly to make sure everyone understands what is expected of them. At the end of the game the facilitator conducts a longer, more in-depth group discussion. These feedback moments allow the players to share their experiences and give surface to any misconceptions.

At the start of the game the four goal cards (they can be found with the components at the end of the rulebook) are randomly divided over the players. Each goal represents a square in the typology with goal determination on one axe and the objective on the other axe (see Table 1).

Self-regulating goal Managerial determined goal

Outcome (1) (2)

Learning (3) (4)

Table 1 : Each player will randomly receive one of these 4 goals at the start of the game.

This leads to the goal cards as shown in Table 1

(1) At the start of each round, make a prediction beforehand on how many points you expect to make.

(2) Get a score of 25 points

(3) Experiment the first two rounds to find the best tactic.

(4) Find the median by tallying the value of each card during the first round.

Table 1 : The 4 goal cards in the gaming-simulation

As said, players may show different reactions towards the game depending on the goal card they receive. For the self-regulating outcome goal (1) the player might feel overwhelmed. They have never played the game before, and are now expected to know their end results. This is one of the pitfalls as mentioned in GST theory. Given people the freedom while they lack the knowhow, they easily start to feel overwhelmed.

For the managerial determined outcome goals (2) the player might feel pressured. The goals are set to high and not knowing the game they show greater risk-taking. This is again one of the pitfalls as mentioned in GST theory. Whilst setting high objectives can work motivational, the user has to feel he can actually reach that goal.

(31)

The managerial determined learning goal (4) is the most effective within the gaming-simulation. This player will most likely enjoy the game the most, and show a leading role in the feedback rounds.

Game components

You can find the components at the end of the rulebook. These are the necessary game components:  1 Standard deck of cards per player

 4 Goal cards

Each player receives at the start of the game a goal card. It shows for each player what its goal during this game is. The goal card remains to be kept hidden for the rest of the players during the first round.

Game setup

 Outside the vision of the participants, remove the followings card from each deck of cards: '  3 times the '2' and 3 times the '3'

 2 times the '4', '5', '6', '7', '8', '9' and '10'  1 Jack, 1 Queen, 1 King, 1 Ace.

This should leave each deck with 30 cards. Shuffle the cards of each deck and place a deck face-down in front of each participant. This now becomes their draw pile

 For each player, draw the top card from the draw pile and place it face-up next to the draw pile.

 Give each player a goal card.

After these steps are finished, the game is ready for play.

The Rounds

A player's round ends when he has depleted his draw pile. Each round, the player performs the following actions:

Step 1. Draw a card

The player draws a card facedown from the draw pile. Step 2. Voice the prediction

The player announces if the drawn card has a higher value or a lower value then the current top open card.

Step 3. Turn the card

(32)

Step 1 to 3 is repeated till the draw pile is depleted. See figure 6 for an example of the gaming-simulation in progress.

Figure 1 : Example of the gaming-simulation in progress

1. The player receives his goal card. In this case the player received the card that guides the player towards learning the system. The player is told to find the median card value of the draw pile.

2. The player looks at the top open card and makes a prediction if the next card drawn will have a higher or lower value than the top open card (two being lowest, ace being highest). His prediction is based on his knowledge of the draw pile. In this case the player expects the next card to have a higher value than four.

3. The player flips the top card of the draw deck. If his estimation was correct he places the card open on the table on top of the current pile of open cards. If his estimation was incorrect he places the card open on the discard pile. In this case the player's estimation is incorrect so the card goes on the discard pile.

4. As suggested by the goal card, the player notes on a sheet that a card with the value of two was drawn on his sheet.

(33)

Feedback at the end of each round

At the end of each round the facilitator announces the winner. Once the game is over, it is the facilitator’s task to discuss the goal setting theory concepts that were woven into the game. Participants might have taken offence by the game and can show emotional reactions during and after participating in the simulation. To ensure a successful conclusion of the gaming-simulation, it is essential to follow the six phases for quality debriefing as shown below.

Six Phases for Quality Debriefing

Phase 1: Discuss the emotional state

Before discussing goal-setting theory the players need to be able to listen. If players build up tension or resentment during the game they will take distance from anything related to the game. This touches the players on a personal level, and it is up to the facilitator to create an open climate. If people do not want discuss their emotional state, pressing the matter usually only increases the tension. Do ensure that players understand that with the debriefing the learning occurs and that the gaming-simulation was used as preparation for the debriefing discussions.

Phase 2: Observations on the activities

The goal is to get perspective if the players understood the intentions of gaming-simulation. If the facilitator or other players saw a player struggle with understanding the game then this is an opportune time to take the player through the game and discuss their decision moment. If they had problem-solving issues, suggest how they could have improved their play. Here the “correct” decisions in the gaming-simulation that were left unused can be introduced as well. This is all to secure that people understand the game and with it can see its validity as teaching methods. The purpose of this phase, as it was with phase 1, is to open up the players for the debriefing.

Phase 3: The relationship between experiences and reality

The facilitator can use the following case examples to show how these different goals work out in reality.

Case example #1: In 1989, Kanfer and Ackerman published an extensive paper on goal setting within the American Air Force. They measured the improvements of a thousand Air Traffic Controllers, of which some were given specific goals while others were told to try their best. Before given the differences in results, keep in mind that they had to function in a highly complex environment: that of air trafficking. Lacking the knowhow the students that were given specifics goals found difficulties reaching these goals. Their behavior became negative, they began to avoid the problem they had difficulties with and showed worst results. The Air Force was trying to help the students by telling them what they value most by making those their goals. It resulted however in a stressful environment where the valued goals where avoided most.

(34)

Case example #2: Harackiewicz did a research on self-setting goals amongst students. They were allowed to choose themselves if they rather be judged on their performance at the end, or on their learning efforts. The result was as following: those that wished to be judged on their performance, showed higher results at the end of the course. Those that wished to be judged on their learning efforts showed higher participation and a broader interest on the subject. It's an interesting example how within one setting the goal setting can make for two very different outcomes. The author advocates that teachers should try to make a mixture of both for optimal results.

In the game the player who is given the managerial determined learning objective is likely to show higher participation and a broader interest on the gaming-simulation. In the second case example, it is to note that the students could follow their personal preference. Individual preferences can increase or decrease the effects goal setting has on them.

If players already have real-life experiences, ask them what they would change after what they’ve now experienced. There is no reason to limit the discussion to the work-floor. Ask the players if they can see connections to other points in their life. The more relations are found, the more the relations are embedded in the players mind. If for instance only one relation can be found the players will most likely consider it a fluke or at best hold very little relevance for them.

The facilitator needs to understand why the game is designed as it is. Deconstruct the game and compare each role, each rule and each resource with real-life. Of course not each element has a strong link with reality. These elements were introduced or altered for the playability of the game. Acknowledging this helps players make the right connections as well, instead of making weak conclusions based on the weak links.

Phase 4: Generalize the gained knowledge

Assessing the gained knowledge does not limit itself to tangible relationships. Personal experiences like feeling agitated at the end of the game also provide knowledge. This phase can be seen as a summary of the previous two phases, but new conversation topics of course doesn’t need to be shut down. If possible, it helps to start summarizing the newfound knowledge on relationships between goal-setting concepts and related business concepts on a whiteboard or something similar. If players don’t find these relations themselves, the facilitator can actively ask on players’ current perceptions. Ifs players’ perception on goal-setting concepts conflicts with experiences during the game, deepen the discussion on these differences. The objective is to incorporate their experiences into their cognitive structure. The intention of gaming-simulations is that players increase the knowledge towards the point where they can apply the knowledge in real-life. These aimed long-termed effects only work if the players perceptions are altered. The facilitator should repeat comparing personal perceptions to game experiences to create a general view on how goal-setting theory should be applied in real-life.

Phase 5: Speculate on the game

(35)

Phase 6: Address real-life issues

The last step encourages applying the knowledge in real-life. This is especially valuable if the game is used as a consultancy-tool for a clear problem at hand.

Components for print

Goal Card #1

Each round, make a prediction beforehand on how many points you expect to make.

Goal Card #2

Get a score of 25 points

Goal Card #3

Experiment the first two rounds to find the best tactic.

Educating Goal Setting Theory usingGaming-Simulation