The influence of autonomy in SCM decision-making games Suffocation or self-determination:

(1)

Suffocation or self-determination:

The influence of autonomy in SCM decision-making games

Master Thesis

February 2017

Word count: 11,453

Abstract: This paper explores the relationship between autonomy, learning motivation and knowledge retention in the context of supply chain management games (SCM). An embedded single case study on a SCM game, semi-structured interviews, evaluation forms, performance scores and observations of the game play will provide the basis of this investigation. Recent literature found mixed results regarding the effectiveness of serious games in supporting learning motivation and knowledge retention. Therefore, possible improvements in the game design, that would support successful learning of SCM decision making, are also approached. This study suggests that the complexities of SCM experienced during the game-play hindered students' persistence in the learning process. Further, it was concluded that autonomy does not independently support learning motivation and knowledge retention, unless students feel competent and related to those involved in the game play.

(2)

(3)

ACKNOWLEDGEMENT

(4)

In recent years, the potential of serious games to motivate students to learn about SCM decision-making was discussed by many authors (Chang et al., 2010; Sweeney et al., 2010; De Leeuw et al., 2015; Lau, 2015). However, Wouters et al. (2013) found in their meta-analysis that, while these games support knowledge retention, they are not more motivating than traditional teaching methods. The authors further suggest that a potential solution for increasing learning motivation is by designing serious games that support autonomy. Furthermore, as implied by Black and Deci (2000), autonomy supportive learning environments do not only increase learning motivation, but they also improve student performance. Hence, understanding the role of autonomy in SCM decision-making games was the trigger for this study. Serious games are a useful tool to create active learning environments where individuals can train problem solving, critical thinking and decision-making (Dondlinger, 2007; Chang et al. 2010; Kebritchi et al., 2010; Lau, 2015). Games can recreate SCM complexities and provide students with the opportunity to experiment with real-life SCM issues (Sterman et al. 1989; Sterman, 1994; Chang et al., 2010; Sweeney et al., 2010; De Leeuw et al., 2015; Lau, 2015). However, due to these complexities, learning about optimal SCM decision-making can be very challenging and students often fail to arrive at sound solution (Sterman, 1994). While many authors support the potential of serious games in retaining learning motivation (Garris et al., 2002; Chang et al., 2010; Kebritchi et al., 2010; Ahrens, 2014) there are no general agreements on this matter (Gunter et al., 2007; Wouters et al., 2013). If SCM games would motivate students to use their knowledge and persist at solving complex problems, then these students would be more capable to deal with real-life SCM decision-making (Lau, 2015).

(7)

2

studies address how the self-determination needs, and implicitly autonomy, support learning motivation. While autonomy supportive teaching methods have been widely applied in educational contexts, no studies address how they can be embedded in serious game designs.

This study seeks to explore how autonomy influences learning motivation and how serious games can be designed in order to support successful learning of SCM decision-making. Hence, this study will evaluate how autonomy influences students’ knowledge retention and motivation to learn about optimal SCM decision-making through serious games. In this regard, an embedded single case study on a SCM game designed by the University of Groningen will be used. Semi-structured interviews, evaluation forms, performance scores, course information and visual observations of the game play will provide the basis of this evaluation. Further, the findings of this study will contribute to the literature by providing an understanding of how self-determination and autonomy can be applied in educational contexts, such as SCM courses and serious games.

(8)

3 2. Theoretical background

In this section the theoretical background and the research framework of this paper will be described. First, the definitions of SCM and SCM decision-making complexity will be presented, together with some underlying concepts of SCM complexity. Second, the struggles of understanding complexity and the limitations in decision-making will be discussed in more detail. Third, the role of serious games as an educational tool for recreating the complexities of SCM will be explained, together with their motivational aspects. Fourth, the methods for increasing learning-motivation will be explored through the self-determination theory. Fifth, the research framework will be presented by exposing the research questions and the conceptual model.

2.1. Supply chain management

A supply chain is defined as a network composed of several interdependent entities involved in activities such as distribution, procurement and manufacturing (Swaminathan et al., 1998; Mentzer et al., 2001; Surana et al., 2005). These activities initiate flows of products, materials, information and services which constantly move downstream and upstream the supply chain. The effort of managing these flows is known as supply chain management (Jones and Riley, 1985; Cooper et al. 1997). Mentzer et al. (2001) further elaborates on this matter by stating that the objective of supply chain management is to boost the performance of each entity of the supply chain with the ultimate goal of improving the supply chain as a whole.

2.1.1. Decision-making in supply chain management: the influence of complexity

(9)

4

2.1.2. Limitations to decision-making: understanding the complexity

In order to understand the impact of uncertainty on the complexity of the decision-making process, it is important to provide a definition of the concept. Peidro et al. (2009) define uncertainty as “the difference between the amount of information required to execute a task and the information that is actually available”. Thus, when individuals do not possess all the needed information for making SCM decisions, or when they are limited in their information processing ability, their efforts can lead to sub-optimal solutions (Wu and Pagell, 2011). That is because, under complexity and uncertainty, decision makers often rely on very simple and intuitive set of rules, rather than optimal decisions. (Manuj and Sahin, 2011; Wu and Pagell, 2011). Likewise, De Martino et al. (2006) arrive at a similar conclusion stating that when the information necessary for decision-making is complex and uncertain, individuals rely on simple heuristics and rules of thumb. Further, Manuj and Sahin (2011) explain in their findings that SCM professionals must understand how different decisions are interrelated to each other. However, due to this complexity and uncertainty, they fail to see the bigger picture, which leads to unwanted outcomes. Furthermore, they state that “human cognitive processes often cut through the large levels of objective complexity to arrive at decision problems of low perceived complexity”. This means that individuals avoid complexity and provide very simplistic solutions to something much more complex.

The complexity of SCM does not only affect professional decision makers, but it also impedes individuals from learning and understanding SCM. Sterman (1994) points out that complexity negatively influences the learning process and the ability of individuals to arrive at sound decisions. The author explains that unsuccessful learning in complex contexts stems from the inability of individuals to understand the outcome of their decisions. For example, individuals cannot discern if the performance changes in a system are a result of their own decisions or someone else’s. This is because in complex contexts, such as SCM, informational delays are a major source of uncertainty (Giannoccaro et al. 2002). Sterman (1994) states that individuals “fail to appreciate time delays between action and response” and that this inability becomes more of an issue as the level of complexity increases. Another important finding of the author was that, when facing complex environments, individuals spend less time in the decision-making process than in less complex ones. This leads to the question of how complexity influences the motivation to arrive at optimal decision.

(10)

5

management. Hence, one can argue that the greater the level of complexity, the greater the engagement, devotion of effort and persistence must be.

2.2. Learning SCM through serious games 2.2.1. The role of SCM games

Boyle et al. (2011) define serious games as “games which are intentionally designed for the purpose of learning, skill acquisition and training”. These games initiate learning processes and enable individuals to assimilate new knowledge more easily (Boyle et al., 2011; Wouters et al., 2013; Ahrens, 2014). This is because serious games have the potential to stimulate the human cognitive mind and to educate individuals into problem solving, critical thinking and decision-making (Dondlinger, 2007; Chang et al., 2010). Furthermore, serious games use action over explanation, which is more effective for teaching about complex contexts and improving the aforementioned skills (Kebritchi et al., 2010, Lau, 2015). Many authors recognize the benefits of teaching SCM through serious games (Sterman et al. 1989; Sterman, 1994; Chang et al., 2010; Sweeney et al., 2010; De Leeuw et al., 2015; Lau, 2015). These authors stress that, besides their ability to recreate real-world supply chains, serious games allow individuals to explore the underlying complexities of SCM and SCM decision-making. Furthermore, they found that individuals consider SCM courses more valuable when these serious games are integrated. Well known examples of SCM serious games are the Beer Distribution Game and the Fresh Connection. The Beer Distribution Game demonstrates basic complexities of SCM, such as time delays, misperceptions of feedback and uncertainty and focuses on single decision-making environment (Sterman et al. 1989; Sterman, 1994; De Leeuw et al., 2015; Lau, 2015). On the other hand, De Leeuw et al. (2015) state that the Fresh Connection is more complex and focuses on a more tactical and multi decision-making environment with cross-functional departments. However, the authors stress that both successfully educate individuals into SCM decision-making.

2.2.2. Learning motivation and knowledge retention

(11)

6

On the other hand, Gunter et al. (2008) imply that serious games are only motivating individuals to play the game and do not increase their knowledge retention, simply because they are more interested in the entertaining aspect, than the academic one. Thus, the authors suggest that serious games are a motivational tool for the sole purpose of playing, rather than learning. Nevertheless, Wouters et al. (2013) contradict all the above in their meta-analysis, suggesting that serious games do improve the learning process and increase knowledge retention, but do not retain learning motivation more than traditional teaching methods do. They further mention that knowledge retention increases both when a game is played alone, but also when additional instructional methods are used. However, the positive impact on knowledge retention is higher in the second scenario.

2.2.3. Instructional methods for integrating serious games in learning environments

The learning success of serious games is not self-sustaining, but depends on the way they are complemented by other instructional methods (Garris et al., 2002, O’Neil et al. 2005; Gunter et al. 2008; Chang et al. 2010; Wouters et al. 2013; Lau, 2015). Chang et al. (2010) suggest that SCM games should create a competitive environment and provide the opportunity of receiving support and encouragement. Other suggestions are to provide a game manual that could familiarize the players with the game setting. Lau (2015) proposes supplementing the game with lectures and practice hours that could prepare the players with prior knowledge. They further suggest that these instructions should clearly communicate how the game relates to the desired learning outcomes. Another suggestion of Lau (2015) is to complement the game with a report where they need to reflect on their performance. The authors state that by reflecting on their performance, the learning process improves and students retain knowledge better. Further both papers of Chang el al. (2010) and Lau (2015) recommend playing the game twice. Students that fall behind their competitors might get demotivated, while those who perform really well might have missed some important insights (Lau, 2015). Thus, running the game twice gives them the opportunity to improve their learning outcomes even more.

2.2.4. Supporting learning motivation

(12)

7

learning processes and motivates individuals to engage in effective SCM decision-making, instead of using simple heuristics and rules of thumb.

2.3. The self-determination theory

2.3.1. The role of intrinsic and extrinsic motivation on self-determined learning

The self-determination theory describes the factors that generate intentional or motivated behaviours (Deci et al., 1991). Motivation relies on the engagement, devotion of effort and persistence at a certain activity and can be both intrinsic and extrinsic (Garris et al., 2002). Intrinsic motivation is what makes individuals engage in activities with a full sense of volition and for its own self (Malone, 1981; Deci et al., 1991). This happens when the activity stimulates the curiosity of individuals and when they perceive it as being fun, interesting and challenging (Malone, 1981; Garris et al., 2002). On the other hand, extrinsic motivation is triggered by external factors that make the action seem important or pressuring, but not interesting (Denis and Jouvelot, 2005).

Deci et al. (1991) discuss the role of self-determination in educational contexts. They state that individuals with intrinsically motivated behaviours are more likely to be self-determined at improving their learning outcomes. However, they mention that this can also happen when individuals are extrinsically motivated by an identified regulation. An identified regulation is an autonomous form of external motivation that does not create pressure, but is rather perceived as being personally important. Hence, even when individuals do not perceive a learning topic as being interesting, they will engage in it because they perceive it as being important for their personal growth. The authors further propose the need for autonomy, competence and relatedness as the most important needs that generate self-determined behaviours.

2.3.2. The role of competence and relatedness

(13)

8

2.3.3. The role of autonomy: creating autonomous learning environments

Ryan and Deci (2006) identify autonomy as the root of intentional and motivated behaviours. Autonomy is in fact what volition and self-determination refer to (Deci et al., 1991; Ryan and Deci, 2000). In educational contexts, autonomy supports the learning process and knowledge retention by creating an environment where control on individuals’ actions is minimized as much as possible and where individuals have enough choices and freedom of action (Deci et al., 1991; Ryan et al., 2006). For example, controlling or pressuring behaviours of teachers can be replaced with the provision of information, feedback and emotional support that will boost autonomy, and consequently self-determined behaviours. Furthermore, it is of great importance to clarify that autonomy does not refer to being independent from external inputs (Ryan and Deci, 2000, 2006), but rather to the means in which individuals volitionally consent to these inputs (i.e. rules, deadlines, feedback). Thus, in order to sustain self-determination, individuals should not feel pressured or controlled by these external inputs.

A way to enable autonomy in learning environments is through serious games. Boyle et al. (2011) mention that, opposed to recreational games, serious games can reduce the autonomy and enjoyment if they are not designed properly. This is a result of serious games being part of a mandatory activity from their educational track. Black and Deci (2000) suggest that autonomous motivated (self-determined) students perform better than those who feel controlled or pressured by external inputs. Furthermore, they mention that their performance increases even more when they perceive their teachers as being less controlling and more autonomy supportive. Another interesting finding is that when the teachers support autonomy, students feel more competent. Thus, when autonomy is successfully supported, perceived competence also increases. Prior research suggests that individuals feel more autonomous when the significant other (i.e. teacher, coach etc.) avoids pressuring them, supports their self-initiation, and provides the opportunity for choices, independent problem solving and involvement in decision-making (Grolnick and Ryan, 1989; Mageau and Vallerand, 2003; Hagger et al., 2007).

2.4. Theoretical framework and research questions

Firstly, the theoretical background of this paper looked into the concept of SCM decision-making and its underlying complexities. In summary, the review hints at the negative impact of complexity on SCM decision-making motivation, suggesting that, the greater the level of complexity, the greater the engagement, devotion of effort and persistence must be. This implies that, when arriving at optimal solutions, the motivational threshold of SCM decision-makers is higher. Hence, the following research sub-question is formulated:

(14)

9

Secondly, the role of serious games in teaching SCM decision-making was reviewed. While many authors praise the potential of serious games to motivate students into learning processes, the meta-analysis of Wouters et al. (2013) suggests that, in fact, they are not more motivating than traditional teaching methods, but increase knowledge retention. The authors further propose the support of autonomy in serious games as means to increase motivation. Therefore, the final part of the theoretical background reviewed the role of self-determination and autonomy, concluding that an autonomy supportive learning environment creates more performant students (Black and Deci, 2000). Therefore, the main research question is defined:

How does autonomy in complex serious games influence students’ knowledge retention and the motivation to learn about optimal SCM decision-making?

The research framework of this study is illustrated in Table 2.4-1:

Independent problem solving Engagement Devotion of effort Optimal SCM decision-making

Enablers of motivation Learning outcomes

Persistence

Table 2.4-1. Main research framework

(15)

10 3. Methodology

To answer the questions of this research, an embedded single case study design is adopted. Case studies are one of the most powerful methods used in operations management (Voss et al., 2002). They allow researchers to investigate complex real life problems where both humans and technology are involved (Runson and Host, 2008). Researchers can integrate qualitative with quantitative data, which allows for a more holistic approach and understanding of the complex real-life problem under study (Baxter and Jack, 2008). Furthermore, Voss et al. (2002) discuss in their paper the different types of research purposes that a case study has. These are exploration, theory testing, theory building and theory extension/refinement, out of which the first is applicable for this research. That is because the aim of this paper is to explore how autonomy influences students’ knowledge retention and motivation to learn about optimal SCM decision-making through serious games.

3.1. Case design 3.1.1. Game play

This study analyses the Universal Exports SCM game developed by the University of Groningen. The game recreates a supply chain with two distribution centres, two plants and two suppliers where they must produce and sell both tablets and smartphones. It is a virtual team-based game which involves the active participation of first year Bachelor students into the management of a SC. Student teams are self-assigned and a maximum of three students can enrol.

Figure 3.1-1. Universal Exports Timeline

The game runs continuously throughout 318 periods, where each period lasts 60 minutes in real life. Figure 3.1-1 illustrates the timeline of the game. Out of all the phases shown above, students are only allowed to make decisions in the second one. In the start-up phase they can use the game dashboard to inform themselves about the SC status and decide on the strategy that they would like to follow. In the third phase, the game will continue running based on the final decisions made in the second phase. The final status of their SC performance will be available at the end of all periods.

The game starts with lost sales as demand is high and the warehouses are empty. Thus, in order to serve their customers and increase their sales, students need to develop a strategy. When they prepare their strategy, they must consider the trade-offs between the different performance dimensions. These are People, Planet and Profit (3Ps). Students must indicate which dimension is the most important by

(16)

11

assigning a percentage to it and then dividing the rest to the other two. At the end of the game, their performance is calculated based on how they score on each of these dimensions.

In order to make decisions and formulate a strategy, students need to use the management dashboard which provides information about the status of the SC. The dashboard can be accessed at any time and contains information about sales, cash position, demand, stock at distribution centres, finished goods at plants, and raw material at plants. More decision parameters are illustrated in Table 3.1-1. Students must use the data for building mathematical and spreadsheet models. Based on the outcomes of their models they further need to make decisions.

Each decision affects the game in different ways. Firstly, students need to be aware that the demand for each product is different at each distribution centre, that customers pay different prices for these products, and that the holding costs are different for each product and location. Secondly, each plant has limited capacity and is different for each product. Thirdly, the price of the raw materials for each product is different depending on the chosen supplier. Another important aspect is that the shipping distance has both a financial and environmental impact. Long shipping distances imply longer lead times, but also large CO2 transmissions, which will affect the Planet dimension. Lastly, poor working conditions, at either the plant or the supplier, will result in a higher humanitarian cost and will influence the People dimension.

Distribution management Capacity management Supply management

Decisions

When to request new products from your plants?

Is additional capacity needed? When to order new raw materials for from suppliers?

How many products to ship from your plants?

When to buy additional capacity? How many units of raw material to order from suppliers?

Where to ship the products from? How much additional capacity to buy?

Which supplier to order from?

(17)

12

Parameters

Unit selling price Humanitarian costs per products Humanitarian cost of one unit of raw material

Unit holding cost Cost per capacity unit Unit cost of raw material

Unit shipping cost per period Production capacity per period per capacity

Unit shipping cost of raw material

Fixed cost per shipment Holding cost per period per unit raw material

Fixed cost per shipment of raw material

Shipping time Holding cost per period per unit finished goods

Shipping time in periods

Environmental cost per shipment of goods

Environmental cost per shipment of raw material

Location

Western Europe Northern Europe Asia

The United States Southern Europe Eastern Europe

Table 3.1-1. Overview of the Universal Exports decision parameters

After playing the game, students are asked to reflect on their strategy by writing a report. Apart from the strategy, they must demonstrate their ability to use Excel for supporting their decisions. In the end, students are assessed on both the game and the report.

3.1.2. Instructional methods

(18)

13

explanations for building the models (i.e. mathematical and spreadsheet models). Furthermore, they must also prepare weekly assignments which are discussed during the tutorials and computer practicals.

Figure 3.1-2. Course timeline

A game manual is also provided where each part of the game is thoroughly elaborated and explained. The manual contains information about how to access the game platform, how to download information, the types of decisions involved and their decision parameters. Further, the task of the report is explained. Before the game starts, an introductory lecture is offered, where the teacher presents the managerial problem of the Universal Exports game. The lecture discusses the steps they must follow, the types of decisions involved and how they are interconnected to each other. Besides this, they are asked to create graphs and models that will help them make optimal decisions. Figure 3.1-3 summarizes the steps they must follow.

Figure 3.1-3. Universal Exports decision-making steps

Immediately after the introductory lecture, students must attend a three hour kick-off session of the game. From this point, the Universal Exports starts running and stops after the 318 periods. The purpose of this session is to allow students to get familiar with the game environment. In the classroom there are ten student assistants and two teachers that can provide them with guidance. Thus, when in doubt, they can ask questions and receive immediate feedback. Furthermore, the decision-making steps from Figure 3.1-3 are projected and elaborated on the electronic whiteboard of the classroom. These steps advise students to

Pre-game 6 weeks

•Lectures: providing the mathematical knowledge (1 per week)

•Tutorials: translating management problems into mathematical models (1 per week) •Computer practicals: transfering the mathematical models into spreadsheet models (1 per

week)

Game 1 week

•Game lecture: introducing the Universal Exports management problem •Kick-off session: assisting students to start the game

•Playing the game: solving the Universal Exports management problem

Post-game 3 days

•Report: reflecting on the Universal Exports strategy and spreadsheet models

(19)

14

develop their strategy and start the game by forecasting demand which allows them to sufficient capacity for future production.

After the kick-off session, students continue playing the game until the 318 periods end. During these days, consultation hours are provided daily where two student assistants are present. These hours take place in computer rooms. They are meant to provide support for students that have questions about the game and the models. Thus, they can access the game during the consultation hours and student assistants can have a look at the difficulties they encountered. Students are encouraged to prepare their questions before these consultation hours.

3.2. Data collection

For this research an embedded case study is used. Embedded case studies evaluate multiple units of analysis (UoAs) within the same case (Runson and Host, 2008). Case studies with embedded units allow for a richer and various data collection (Baxter and Jack, 2008; Alvarez et al., 2010). The UoA of this research is divided into four sub-units: (1) students, (2) student assistants, (3) lecturers and (4) game developers. It is important to mention that one of the lecturers is also a game developer.

Selecting more than one UoA does not only allow the researcher to gather rich data, but also to achieve triangulation. Runson and Host (2008) mention that there are different types of triangulation: observer triangulation, data triangulation, methodological triangulation and theory triangulation, where only the first two are relevant for this research. The first type of triangulation is achieved by selecting multiple UoAs and multiple respondents within each of these UoA. The second type of triangulation is ensured by using an extensive procedure to collect data such as: (1) interviews, (2) evaluation forms, (3) game data such as performance scores and log-in frequencies, (4) course information and (5) visual observations of students playing the game (took place before conducting the interviews). Table 3.2-1 summarizes the different data collected.

Data source Description

Semi-structured interviews (Appendix 1)

Students 15 semi-structured interviews of 40-50 minutes

Student assistants 5 semi-structured interviews of 45-90 minutes

Teachers

(20)

15

Evaluation forms 1

Students Critical comments about the course content

Game data

Performance scores Rankings from the game and report grades

Log-in frequency The number of times students logged into the game

Course information

Ocasys Details about the course content and learning outcomes

Visual observations

Kick-off session Observations of how students handled the game and their main struggles

Table 3.2-1. Overview of data collection

With the exception of two student interviews, the interviews took place face-to-face in several offices of the University of Groningen. All interviews were recorded and transcribed in English. However, it is important to mention that interviews were not always transcribed verbatim, as some interviewees lacked sufficient English skills and small reformulations were needed.

As suggested by Yin (2009), an interview protocol was developed to ensure data reliability. The interview protocol was formulated in line with the theoretical background of this paper (see Appendix 1). After the first interview with each of the UoAs, more insight was gained and the protocol was further refined to avoid leaving out important notions or repetitions. Semi-structured questions together with a set of pre-defined follow-up questions were used to encourage interviewees to provide detailed answers. The protocol was formulated to address several categories that refer to the game play and the additional instructional methods, which provided information about how students were motivated and supported to learn about optimal SCM decision-making. Hence, in line with the theoretical background, the interview protocol included questions about motivation, complexity, instructional methods, autonomy, competence, relatedness, and learning outcomes. Furthermore, the visual observations of the kick-off session allowed for an initial overview of the main struggles that students faced, which provided additional ideas for questions that could be added to the protocol. Additionally, the evaluation forms consisted of student opinions regarding the course content (i.e. lectures, tutorials, computer lessons and game) which provided support for the preparation of the protocols designed for the student assistants and teachers.

(21)

16

It is of particular interest to clarify how the motivation was measured. O’neil et al. (2005) mention that previous studies of serious games determine motivation by asking for student opinions which are highly unreliable. They further stress that motivation should be analysed through its actual constructs. Thus, in order to ensure reliability for this research, students were asked questions about their (1) engagement, (2) devotion of effort and (3) persistence (Garris et al., 2002). Furthermore, to ensure more reliability, the log-in frequencies were used as an additional measure for motivation.

3.3. Data analysis

In order to establish a chain of evidence, the different types of data mentioned in Table 3.2-1 were analysed (Voss et al., 2002). Hsieh and Shannon (2005) propose the directed content analysis method which is designed to validate or elaborate on existing theories, in this case, the role of autonomy on self-determination, and implicitly on the knowledge retention and the motivation to learn about optimal SCM decision-making. For the purpose of this research, optimal SCM decision-making is operationalized as the use of mathematical and spreadsheets models for making decisions.

Following the directed content analysis proposed by Hsieh and Shannon (2005), the initial step was coding the data (i.e. interviews). The authors explain that initial codes should be deducted from the key elements of the theoretical background. Hence, motivation, learning outcomes, instructional methods, autonomy, competence and relatedness were selected as initial coding categories. Further, each coding category was divided in multiple sub-categories. These sub-categories were operationalized in relation to the existing theory and case design (Appendix 2).

For the purpose of this study, facing complexity was defined as ones’ competence to process information (Wu and Pagell, 2011), understand the interconnectedness of decisions (Manuj and Sahin, 2011) and appreciate informational delays (Sterman, 1994; Giannoccaro et al. 2002). Hence, as observed in Table 3.3-1, due to this definition, complexity, which was a key element of the theoretical background, was added as a sub-category of competence.

(22)

17

Motivation (see section 2.3.1)

Effort Persistence Engagement Intrinsic Extrinsic

Learning outcomes (see section 2.2.2)

Performance scores Knowledge retention

Instructional methods (see section 2.2.3)

Preparation Game manual and

kick-off Report

Communicating the goal of the game/learning outcomes

Autonomy (see section 2.2.4 and 2.3.3)

Independent problem solving Involvement in decision-making Self-initiation Provision of

choice Feedback Pressure

Competence (see section 2.3.2)

Mathematical models Spreadsheet models Decision-making

SCM complexity (see section 2.1.2) Vastness of decision parameters Interconnectedness of decisions Informational delays

Relatedness (see section 2.3.2)

Enjoyment Encouragement

Suggestions for improving the game (new codes)

Technical issues Speeding up the

game Gradual complexity Practice mode

Playing the game twice Table 3.3-1. Coding categories and sub-categories

(23)

18

allowed for a clear comparison between the differently motivated groups in relation with each category and sub-category of codes. This method was applied throughout the entire data analysis.

Table 3.3-2. The different levels of learning motivation

One assumption derived from Sterman (1994) and Garris et al. (2002) was that individuals do not “engage in, devote effort to, and persist longer” at effective decision-making when dealing with the complexities of SCM. Hence, an initial focus of this analysis was on the exploration of this assumption. This was possible by investigating the first research question of this study (i.e. how does complexity influence individuals’ motivation to learn about optimal SCM decision-making?). Therefore, statements about motivation and its sub-categories were analysed in relation to those about complexity (i.e. information processing, interrelatedness of decisions and informational delay) and their competence to arrive at optimal SCM decisions (i.e. use of mathematical and spreadsheet models).

The next phase of the analysis was answering the main question of this study (i.e. How does autonomy in complex serious games influence students’ knowledge retention and the motivation to learn about optimal SCM decision-making?). This analysis followed the same pattern as the one of the first research question, only this time, all coding categories and sub-categories were compared. Thus, the first step was to identify the relationships between autonomy, motivation, and learning outcomes. However, it was prudent to investigate if these relationships were biased by competence, relatedness, or instructional methods. This allowed for a thorough understanding of the role of autonomy on self-determination, implicitly on the learning motivation and learning processes. As suggested in the theoretical background, autonomy is the root of motivated behaviours and self-determination (Deci et al., 1991; Ryan and Deci, 2000; Ryan and Deci, 2006). Hence, the main purpose was to identify whether autonomy stands alone as a driver for learning motivation and knowledge retention, or if it is dependent on competence and relatedness. Further, statements about the instructional methods provided an explanation of how students were equipped with the knowledge for being competent at optimal SCM decisions.

Apart from interviews, other types of data were used for this analysis, as illustrated in Table 3.2-1. Log-in frequencies were used as an additional measure of motivation, while performance scores as an additional

High motivation

•Students who were engaged, devoted effort to, and persisted at optimal SCM decision-making during the game-play

Medium motivation

•Students who were engaged, devoted effort to, but did not persist at optimal SCM decision-making during the game-play

Low motivation

(24)

19

(25)

20 4. Findings

This section will address the main findings of this study. The first part will present an overview of how students handled the decision-making process in the game. In this regard, the influence of SCM complexity (i.e. vastness of decision parameters, interconnectedness of decisions and informational delays) on learning motivation (i.e. engagement, devotion of effort and persistence at optimal SCM decision-making) will be presented. The second section will describe the instructional methods and how the game design supported the need for autonomy, competence and relatedness. The goal is to illustrate the influence of autonomy on learning motivation and knowledge retention in relation to competence and relatedness. As mentioned above, in this paper, learning motivation refers to the engagement, devotion of effort and persistence at arriving at optimal SCM decisions through the use of spreadsheet models.

4.1. Learning motivation and performance: influence of competence, pressure and SCM decision-making complexity

Table 4.1-1 displays the overall engagement, devotion of effort and the persistence of students to arrive at optimal SCM decisions and their learning outcomes. A representative statement for most highly motivated students is presented below:

“We wanted to do it good, but it was very difficult to understand what you have to do. That was a bit discouraging. […] When I got stuck I tried to figure out a solution by looking in my book or asking the teachers, because I actually wanted to make things good. I didn’t just come up with any kind of simple,

but wrong solution, just because that would have been easier” (student #2)

Student # Engagement Effort Persistence Motivation Level Game score Report grade

1    Medium 8 Fail

2    High 9 8

3    High 9 8

4    Low Fail Fail

5    Medium 4 9

7    Medium Fail Fail

8    High 8 8

9    Low 6 Fail

10    Medium Fail Fail

11    High 6 9

12    High 9 8

14    High 6 8

15    High 6 7

(26)

21

Overall, students that were highly motivated also scored good grades for both the game and the report, with the exception of student #15. However, an unexpected finding is the one of student #5. As can be seen from the following statement, something might have gone wrong in the grading:

“We scored an 8.8 for the report and I don’t understand how and why we got that grade” (student #5) From the interviews it was clear that, when the game started, students faced many struggles. In the first days, they were not aware of the decisions they could make, the information needed for those decisions (i.e. decision parameters), and the models they have to use. They relied on simple mathematical calculations, and trend line charts, that allowed them to determine demand. However, after consulting the course materials (i.e. course manual, game manual, report outline, tutorials and computer practicals) and the student assistant, it became clear that they must use Excel to build the models. While most students were engaged and devoted effort to build the models, more than half didn’t persist. A representative statement for these students is presented below:

“We were motivated to make the best of it […] We read the book and the manual, but we still didn’t know what to do […] We tried to use the models from the tutorials but we didn’t understand them […] By the

end of the week we were already not that engaged anymore “(student #5)

The lack of competence was very common throughout students that didn’t persist or who didn’t devote any effort. Further, their overall motivation was diminished even more when students felt time pressured:

“[…] the game it was just pressed into one week, and when you don’t understand things you get frustrated and don’t want to put time into it anymore. That’s why after all we didn’t do much with it”

(student #9)

However, this pressure had a larger influence when students faced the complexities of SCM decision-making. Table 4.1-2 illustrates these different complexities:

Student # Vastness of decision parameters Interconnectedness of decisions Informational delays

Ability to use the spreadsheet

models

Use of

intuition Pressure Motivation 1 processing

information - -    Medium

2 processing

information -

misunderstanding

decision outcomes    High

3 processing

information -

misunderstanding

decision outcomes    High

(27)

22

5 processing

information -

misunderstanding

decision outcomes    Medium

6 - - -    Low 7 processing information optimization of decisions delayed gratification    Medium 8 processing information - delayed gratification    High 9 processing information optimization of decisions -    Low

10 - balance between 3Ps delayed

gratification    Medium

11 processing

information -

misunderstanding

decision outcomes    High

12 processing

information -

delayed

gratification    High

13 processing

information balance between 3Ps -    Low

14 processing

information balance between 3Ps

misunderstanding

decision outcomes    High

15 processing

information -

delayed

gratification    High

Table 4.1-2. Influence of SCM complexity and pressure on performance (cells marked with “-“mean that the student did not answer to that particular issue)

One of the most mentioned SCM complexities was the large amount of decision parameters involved in the game. This hindered their ability to understand what information they should use for the spreadsheet models:

“You get a lot of data that you don’t know what to do with” (student #8)

“The information was in the game, but you had to find it and put it together and then make the models. I think that was the most difficult thing (student #3)”

“They have to do so much when they start, so they get overwhelmed by the multitude of information” (student assistant #3)

(28)

23

“If they want to perform well in the Planet dimension, the solution is to reduce the CO2 emissions. This is done by killing your production. When production is killed, the humanitarian performance also increases. This allowed them to decrease both humanitarian costs and carbon dioxide transmissions (teacher #1)”

Further, while interviewing students it seemed that most of them only focused on one or two of the models, and very rarely optimized their decisions by integrating the models together. This was also confirmed by the teachers and students assistants:

“They had to build a model for forecasting and a network model. If they combine those two they already have the solution for the game” (students assistant #2)

“We used some models, such as network models […] I didn’t understand the others well enough” (student #7)

The last source of complexity was the informational delay. As the game ran continuously (i.e. one period in the game was the equivalent of 60 minutes in real life), students had to wait a long time for their decisions to have an impact.

“The transportation of goods takes up to ten days, so ten hours in real life which is really a lot of time to see the effects of your actions (teacher #2)”

“You had to wait very long to see if you did something right” (student #9)

Firstly, some could not understand if their ranking was changing due to their decisions or the ones of the other teams involved. Secondly, some simply didn’t know when and how their decisions are affecting the game. Some representative statements that refer to both situations are presented below:

“We never knew what happened after we made a decision. This was very demotivating” (student #5) “We couldn’t really relate back our decisions to what happened in the game sometimes. So a bit more direct feedback in that respect would have helped […] we changed something and weren’t really sure if it

had an impact, and then we were already about to make a new decision” (student #11)

“We could see when our ranking was changing, but we didn’t know if it was because of our decisions or the other teams’ decisions. We actually never knew when our decisions were impacting the game. It was

very demotivating in the beginning because we didn’t know if we were making the right decisions and every few hours we were changing something hoping to see something changing in the game (student

(29)

24

4.2 Instructional methods and game design: the role of autonomy, competence and relatedness Table 4.2-1 shows how student autonomy was supported before the game (i.e. lectures, tutorials, computer practicals) for each student. This is illustrated in relation to their competence and motivation during the game.

Student # Independent problem solving

Involvement in

decision making Pressure

Ability to use the

spreadsheet models Motivation Level

1     Medium 2     High 3     High 4 - -   Low 5     Medium 6 - -   Low 7     Medium 8     High 9 -    Low 10     Medium 11     High 12     High 13 - -   Low 14     High 15     High

Table 4.2-1. Autonomy support during the lectures, tutorials and computer practicals (cells marked with “-“mean that the student did not answer to that particular issue)

The general feeling was that the course design did not support their self-initiation. It focused too much on building models, and too less on the involvement of students in decision-making scenarios. Many students mentioned that, during the game, they were not competent to build the models on their own, because they learned them mechanically, without understanding how to use them for decision-making.

“Being guided it was doable, but on our own it was very hard […] we didn’t really know what we were doing and what that was for” (student #11)

“The examples that we worked with were very mathematical and too less focused on how to make decisions” (student #12)

However, a small group of students had a different opinion, which made it clear that some student assistants created an autonomy supportive learning environment, while others didn’t:

(30)

25

also the level of motivation in my class was a bit higher, because every week I showed them again where they can apply their knowledge” (student assistant #2)

A very representative statement of highly motivated students that were assisted by one of the autonomy supportive teaching assistants is presented below:

“There was this student assistant which was very motivated and really made things very clear for us. He didn’t provide us with the exact answers, but guided us and helped us to figure it out on our own”

(student #8)

On the other hand, less autonomy supportive student assistants diminished the students’ self-initiation, rather than allowing them to solve problems independently:

“In the tutorials I tell them exactly what they need to do” (student assistant #4)

“You have the steps of the questions and you can follow that, but when you get your own information you don’t really know how to use the Solver then […] we only typed in numbers before without learning

much” (student #1)

Furthermore, other students could not really provide a clear explanation of how these instructional methods prepared them for the game, simply because they were too overwhelmed by the entire course:

“I couldn’t tell because I was completely overwhelmed and had no clue what was happening there” (student #13)

Before the Universal Exports game started, students were also provided with additional support, such as (1) a game manual, (2) a game lecture, (3) a kick-off session and (4) consultation hours. However, none of them were perceived very well.

“We weren’t really told how the game actually works […] It was just hard to make the right assumptions in the beginning, even with reading the manual and so on” (student #1, game manual)

“I think the group was too big. So when you had questions you had to wait like half an hour before somebody came by” (student #15, kick-off session)

“There was one classroom a couple of hours a day with a few assistants per day, and that wasn’t really enough. Basically too many students for too few assistants, that wasn’t really helpful for getting help and improving our decision making. If you got help it was nice, but most of the time they were too busy to help

(31)

26

Once the game started, as opposed to the course design, the autonomy of students was very high. They had to be self-initiated, solve models independently, and use them for making decisions. Hence, the role of these consultation hours was to provide support for students that struggled with the game platform, building the models and the decision-making process. However, the lack of capacity during the consultation hours was mentioned by all students and it was clear that it had a negative impact on their motivation and performance. This can be seen in Table 4.2-2. Furthermore, Table 4.2-2 encompasses the three needs for self-determination and illustrates how they were supported throughout the game. In addition, the level of motivation and the knowledge retention of students is added as means to understand the influence of autonomy, competence and relatedness on learning motivation and knowledge retention.

Student #

Autonomy Competence Relatedness

Motivation Knowledge retention Provision of

choices

Ability to use the

models Team members Student assistants

1 limitation of cash    Medium 

2     High  3     High  4 overwhelming    Low  5 overwhelming    Medium  6 limited    Low  7     Medium  8 supply and transportation limitations    High  9 overwhelming    Low  10 overwhelming    Medium  11     High  12     High  13 overwhelming    Low  14     High  15     High 

Table 4.2-2. The influence of autonomy, competence and relatedness on learning motivation and knowledge retention during the game-play (cells marked with “-“mean that the student did not answer to that particular issue)

(32)

27

A representative statement referring to the knowledge retention of highly overwhelmed students is mentioned below:

“I don’t feel that the game improved in any way my knowledge and modelling skills. I think that this would only apply for those who understood the models prior the game” (student #5)

Table 4.2-2 contradicts this statement. There are clear patterns of students that lacked the competence to build the models, but still improved their knowledge retention after playing the game. However, this only happened when relatedness was high (i.e. with both their team-members and students assistants).

(33)

28 5. Discussion

5.1. General insights about the SCM game

The Universal Exports game was, in general, positively perceived by the students. A very common remark was its potential to engage them in more practical learning experiences, as opposed to traditional teaching methods. This is because they enjoyed doing something on their own and engaging in real-life practices, such as the management of a supply chain. This finding confirms the added value of integrating serious games into SCM courses, as suggested by many previous studies (Sterman et al. 1989; Sterman, 1994; Chang et al., 2010; Sweeney et al., 2010; De Leeuw et al., 2015; Lau, 2015).

The game supported the intrinsic motivation of students, as almost all of them perceived the game as being interesting and had fun while playing it (except #student 5). However, it is important to mention that, generally, students were intrinsically motivated from the entertaining aspects of the game, and not from applying the decision-support models. Hence, in line with the suggestions of Gunter et al. (2008), the game sometimes missed its aim, as some students engaged more in the entertaining aspect of the game, rather than its actual learning purposes. Nevertheless, those were less motivated students that lacked the competence to build the models.

On the other hand, the rankings and the competitive environment were another form of motivation that engaged students in the game-play. Students identified with this form of extrinsic motivation and perceived the competitiveness of the game as being important for their self-development. However, apart from the highly motivated students, this form of identified regulation still didn't increase the persistence of students to arrive at better decisions through decision-support models.

5.2. Influence of SCM decision-making complexity on learning motivation

One assumption derived from Sterman (1994) and Garris et al. (2002) was that individuals do not “engage in, devote effort to, and persist longer” at optimal decision-making when dealing with SCM complexities. The findings of section 4.1 confirm this assumption. When students encountered the different sources of SCM decision-making complexity, they often stopped persisting. However, this happened when students lacked the competence to use the decision-support models and when they felt time pressured. Nevertheless, the learning process of all students was negatively influenced, to some extent, by these complexities (except students #4 and #6, which were simply not interested enough in the game).

(34)

29

from the game dashboard, students were not able to determine what information to use for which model. This happened among all the differently motivated groups of students. However, the highly motivated ones persisted and eventually managed to build the correct models, while others simply didn't persist anymore and relied on intuitive decisions.

Another source of complexity encountered in the game was the interconnectedness of decisions. It is a source of complexity that limits SCM decision-making, as explained by Manuj and Sahin (2011). Students often struggled making trade-offs between the different performance dimensions involved in the game (i.e. People, Planet and Profit). Generally, students focusing on an environmental strategy had many difficulties in finding the right balance that would allow them to continue producing tablets and smartphones, while reducing the carbon dioxide transmissions. The same happened with students focusing on a humanitarian strategy, only this time it involved the minimization of humanitarian costs. Under this particular source of complexity, less motivated students didn't persist at using the models, and relied on intuitive decision (i.e. shutting down production or simply choosing suppliers with the shortest routes). Furthermore, students also failed at optimizing their decisions and often focused on local decisions (i.e. either forecasting, inventory or network models). However, it was particularly difficult to conclude this from the student interviews. Nevertheless, the information provided by the student assistants and teachers provided an overall impression about this particular issue. They explained that students could not realize that one model influences the other. Hence, because they were not able to approach their decision-making in a holistic way, their rankings decreased. Furthermore, some students were only able to build one of the models (student #7 and #9).

Due to the fact that the game ran continuously (i.e. 318 hours), students experienced informational delays, as they had to wait for many hours before their decisions were impacting the game. This source of complexity and its impact were described by Giannaccaro et al. (2002) and Sterman (1994). Due to this informational delay, students misunderstand the outcomes of their decisions, which was very common during the Universal Exports, as can be seen in section 4.1. Furthermore, the informational delay hindered the ability of students to appreciate when their decisions would take place (i.e. delayed gratification). Because of this, they were not aware when they should make new decisions and if the ones they already made were effective.

5.3. The role of autonomy supportive instructional methods

(35)

30

decision-making, self-initiation and provision of choices) will increase their learning motivation and competence. The findings from section 4.2 partially support this claim. Almost all students mentioned that they lacked sufficient preparation, which hindered their ability to build models without guidance. They suggested that more assignments, in which they could independently engage in decision-making processes, would have helped them to develop a better understanding of the applicability of these models. Hence, the relationship between autonomy and competence is highlighted. Furthermore, the additional teaching methods, such as the kick-off session and consultation hours, did not always provide them with much help. Firstly, because both were very crowded and students had to wait a long time to receive help. Secondly, some student assistants were less encouraging than others. Students lacking the competence to build the models, and that did not receive encouraging support from the student assistants, considerably lost their learning motivation and lagged behind. Hence, the relationship between autonomy, competence and relatedness is highlighted. However, contrary to the claim of Black and Deci (2000), there is no reliable evidence whether autonomy supportive instructional methods increases the learning motivation. Yet, there is strong evidence that their lack of autonomy diminished their competence to build the models.

5.4. The role of autonomy on learning motivation and knowledge retention

Table 4.2-2 provides the information for answering the main research question of this study. The findings suggest that autonomy does not act as an enabler of learning motivation and knowledge retention, unless it is also supported by the other two needs of self-determination (i.e. competence and relatedness). For example, the interviews suggested that, during the game, the provision of choices, and the fact that they were free in their actions, increased their interest in performing well in the game. However, when they realized that they were not able to build the models, their learning motivation was diminished. Further, when they were also missing the relatedness to either their team members, and/or student assistant, they stopped persisting. Consequently, their knowledge retention was not supported, as they did not manage to better understand the decision-support models. On the other hand, learning motivation and knowledge retention was supported by autonomy only for students that were able to build the models and had relatedness to either their team members or their student assistants.

(36)

31

(37)

32 6. Conclusion

This study adopted a single embedded case study that explored the influence of autonomy on learning motivation and knowledge retention in SCM serious games. As suggested by many previous authors, the game successfully recreated the complexities of SCM and allowed students to experience real-life decision-making problems (Sterman et al. 1989; Sterman, 1994; Chang et al., 2010; Sweeney et al., 2010; De Leeuw et al., 2015; Lau, 2015).

The first assumption of this paper was that individuals do not engage in, devote effort to, and persist longer at optimal decision-making when dealing with SCM complexities. Hence, the initial question of this paper concerned the influence of complexity on the learning motivation of students. The findings confirm this assumption, as the sources of complexity recreated in the game (i.e. vastness of decision variables, interconnectedness of decisions and informational delays) hindered the learning motivation of students, particularly their persistence.

The second research question treated the influence of autonomy on the learning motivation and knowledge retention of students. Autonomy acted as an enabler of motivation and knowledge retention only when students had the competence to build the models and the relatedness to their team members or student assistants. On the other hand, students that lacked the other two needs of self-determination, felt suffocated by firstly: the provision of choices from the game, and secondly, implicitly: by autonomy. Hence, the answer to this research question is that autonomy does not independently support learning motivation and knowledge retention, unless students feel competent and related to those involved in the game play. Another important finding was that the lack of autonomy support prior to the game prevented the development of students’ competence, which later constrained them to perform well in the game. The findings of this study have several theoretical contributions. Firstly, they provide an overview of the different sources of SCM complexity and their influence on students' ability and motivation to learn, for instance, about SCM decision-making. Secondly, this study investigates serious games through the self-determination framework, where the main focus is on the autonomy support framework of Grolnick and Ryan (1989). This is of great interest as the applicability of these two particular frameworks in serious games was not studied in prior literature.

(38)

33

complexity gradually, which is also in line with the design of the Fresh Connection game. This will avoid overburdening students with large amounts of decision variables on their first contact with the game. Further, it might avoid students' losing their learning motivation on an early stage of the game. Furthermore, allowing the participants to play the game twice, as suggested by Lau (2015) can increase students' knowledge retention. Alternatively, a "practice" mode could be implemented. In this practice mode, students would be able to test different models and decisions and study their effects. This could be implemented during the tutorials and computer practicals, as means to teach the students about the decision-support models. Furthermore, another suggestion which is in line with the findings of this paper, is to support the autonomy of students, both in the game and its instructional methods, by allowing them to be self-initiated and to get involved in decision-making processes.

A few limitations were encountered throughout this study. Firstly, the limited English skills of certain interviewees could have led to certain omissions of relevant information. Secondly, only some participants of the game were interviewed. However, the different UoAs analysed throughout this study minimized these limitations and allowed for an extensive data collection.

The influence of autonomy in SCM decision-making games Suffocation or self-determination:

Suffocation or self-determination: