University of Groningen Improving delirium education: the role of experiential learning in a serious game Buijs-Spanjers, Kiki

Improving delirium education: the role of experiential learning in a serious game

Buijs-Spanjers, Kiki

DOI:

10.33612/diss.126540538

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from

it. Please check the document version below.

Document Version

Publisher's PDF, also known as Version of record

Publication date:

2020

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

Buijs-Spanjers, K. (2020). Improving delirium education: the role of experiential learning in a serious game.

https://doi.org/10.33612/diss.126540538

Copyright

Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.

The sound design and video did help to complete the image described in textbooks - Medical student after 3 years of medical education and playing The Delirium Experience

General Discussion

7

7

Aim

In this thesis, we aimed to investigate whether a serious game on delirium that uses principles of experiential learning is a suited strategy to improve delirium education (chapter 2, 3, 4, and 5). Improvements in delirium education should focus on patient- oriented, interactive teaching methods that address attitudes, understanding of the patient’s experience, and behavior and communication skills in delirium care.1–4 _In

addition, this thesis aimed to identify the aspects of a serious game that contribute to specific improvements in delirium education (chapter 4, 5, and 6) because in delirium education as well as in serious games, there are unresolved questions on which aspects contribute to achieving its goals.2,5–10

Main findings

In addition to the description of the main findings below, Figure1 provides an overview of the main findings of this thesis.

In chapter 2 we described a three-arm randomized controlled trial in which we

investigated the effect of The Delirium Experience in 156 medical students. We compared playing the serious game to watching a video on delirium and a patient experience, or to watching a video on healthy ageing. In this study, we showed that students who played The Delirium Experience gave better advice for delirium care and reported higher learning motivation and engagement compared to both control groups. In addition, students who played the serious game also scored higher on self-reported knowledge compared to students who watched the video on healthy ageing. However, we were unable to identify differences in students’ skills in delirium screening and rating, nor in their attitudes towards delirium. We concluded that The Delirium Experience is suitable as an educational intervention to teach delirium care to medical students and has added value to a lecture on delirium. In chapter 3 we conducted a before-after study with The Delirium Experience as

delirium education for 128 applied university nursing students. We showed that nursing students enhanced their advice for delirium care, and their perceived self-efficacy in providing care for delirious patients increased after playing the serious game. Playing The Delirium Experience also resulted in high learning motivation and engagement. We thereby concluded that The Delirium Experience is a practical, useful, and effective educational tool for nursing students to improve delirium training.

In chapter 4 we described a two-arm randomized controlled trial to gain insight in

the effect of different play strategies on game effectiveness in 157 medical students. Also, we explored the different game aspects of these play strategies and how these aspects contributed to improvements in delirium education. We compared two play strategies, normal and dark play. Players in the ‘dark play’ condition were instructed to take

7

delirium as severe as possible. Players in the ‘normal play’ condition were instructed to try to achieve high scores and provide the patient with the best possible care. The different play strategies did not affect game effectiveness regarding quality of advice for delirium care, attitude, or learning motivation and engagement. We identified that irrespective of normal or dark play, the patient’s and nurse’s perspective are important aspects in creating empathy for the patient. Practicing how to care, feedback, and seeing the importance of interaction and consequences of care were identified as important game aspects to support students’ self-efficacy in providing care to delirious patients.

Chapter 5 described our second study on the different play strategies. In this study,

160 medical students were allowed to choose their preferred play strategy within The Delirium Experience. We investigated differences in these students’ characteristics, reasons to choose the play strategy, and learning experiences. Almost half of the students chose to play the serious game using the dark play strategy. Males were found to be more likely to choose dark play, but we were unable to identify other student characteristics (e.g. age, experience with caring for older or delirious patients, and number of completed clerkships) that influenced students’ choice for play strategy. We found that regardless of the play strategy medical students choose, participants reported the same lessons learned after playing the serious game: students reported having learned how to provide care to a delirious patient and having gained insight in what a delirious patient endures. We therefore concluded that either using a serious game as a safe environment to practice for real-life, or using it to explore options that are not ethical in real life lead to the same lessons learned.

In chapter 6 we explored in depth in an interview study how the narrative and

aspects of the narrative within The Delirium Experience affected medical and nursing students’ attitudes and learning experiences toward delirium. We identified that the patient’s and nurse’s perspective, interactivity to experiment, realistic views and care options, and feedback on care actions were important aspects for students attitudes and learning experiences toward delirium. These aspects were found to create better understanding of the patient’s experience and contributed to better knowledge, behavior, and communication in delirium care. We concluded that a patient-oriented narrative within a serious game provides an experience to students on which they reflect. It also creates a medium for active experimentation to create better understanding of how the learner can influence the patient’s experience of delirium.

Figure 1. Overview of main findings

- Character’s perspectives - Realism - Interactivity - Feedback

Which aspects of a serious game on delirium contribute to improvements in delirium education?

Is a serious game a suited strategy to improve delirium education?

✓

- Quality of care advice - Perceived self -efficacy - Self -reported learning

- Learning motivation and engagement

×

- Delirium screening - Delirium severity rating

?

- Attitude

Normal play or dark play

- Similar game effectiveness

7

indicate that The Delirium Experience may prepare students on how to respond to delirious patients.

Perceived self-efficacy in providing care to delirious patients

Playing The Delirium Experience was related to increased perceived self-efficacy in providing care to delirious patients. Perceived self-efficacy plays an important role in clinical performance.23,24 _{With regard to the clinical performance of delirium care, the}

increased perceived self-efficacy holds implications for the behavior and communication skills that need to be addressed in delirium education.1,2 _{It is known that practicing skills}

increases perceived self-efficacy in general.25 _{We showed that practicing delirium care}

skills in a safe environment such as a serious game, also increases students perceived self-efficacy in providing care to delirious patients. Our results thereby confirm the need to incorporate simulations into delirium education to allow students to practice their skills and increase their perceived self-efficacy.26–28 _{This increased perceived self-efficacy may}

also decrease the discomfort healthcare professionals experience in delirium detection and management.29–31

Learning motivation and engagement

We also showed that using The Delirium Experience as delirium education increases students’ learning motivation and engagement. Students’ motivation and engagement can be enhanced by changes in the educational environment, such as an increase in autonomy.32,33 _{Autonomy is not only associated with increased motivation and engagement}

in learning, but also in gameplay.34 _{The autonomy provided to students when playing}

The Delirium Experience may therefore have contributed to the enhanced learning motivation and engagement.

Enhanced learning motivation and engagement is important for the uptake of serious games,35 _{and to improve learning experiences and outcomes.}36,37 _{Especially in forms of}

learning that depend on the active involvement of students, motivation and engagement is of great importance for learning experiences and outcomes.37,38 _{In addition, improved}

learning motivation and engagement also increases student reflection during their learning process.32 _{Reflection helps students to make sense of the experience and helps}

to solidify the experience into the learner’s memory.39,40

Self-reported knowledge on delirium

In general, students overestimate their knowledge, and self-reported knowledge has therefore been found to have little agreement with objectively assessed knowledge.41,42

In that respect, it is important to note that students showed higher self-reported knowledge after playing The Delirium Experience and may have overestimated their knowledge on delirium.

Reflection on the findings

A serious game to teach delirium care

In this thesis we aimed to investigate if a serious game on delirium that uses principles of experiential learning is a suited strategy to improve delirium education (chapter 2, 3, 4, and 5). We showed that a serious game that makes use of experiential learning principles is suited as delirium education to improve quality of care advice (chapter 2, 3, and 4), learning motivation and engagement (chapter 2, 3, and 4), perceived self-efficacy in providing care to delirious patients (chapter 3 and 4), and self-reported knowledge on delirium (chapter 2) in medical and nursing students. We were however unable to show effect of the serious game on skills in delirium screening and rating (chapter 2), or on attitudes towards delirious patients (chapter 2, 4 and 5).

Outcomes that did improve after playing the game

Quality of care advice

The improved quality of care advice after playing The Delirium Experience indicates that students were able to give more specific delirium care advice and increased their knowledge on how to manage delirium. This yields important implications for improving delirium education, as it is known that inadequate knowledge and education are important barriers in good delirium management.3,11–14 _{It is also known that current delirium education}

often fails to educate healthcare professionals in such a way that they can overcome these barriers.1,5,15

More specifically, the improved care advice of students revealed that students became more aware of the importance of non-pharmacological interventions in the management of delirium. This resonates with recent findings on effective management of delirium because the treatment or management of delirium with pharmacological interventions is not proven to be effective in decreasing the duration or severity of delirium.16–19 _{The use of non-pharmacological interventions are therefore recommended}

nowadays.18,20,21 _{This is also in line with the recommendations for good quality delirium}

care of the Dutch delirium guidelines.22 _{The care advice of students showed that they are}

aware of the implementation of multicomponent non-pharmacological interventions for delirium management as described in the guidelines, such as treatment or prevention of possible precipitating medical conditions, re-orientation, avoidance of sensory deprivation, hydration, and involvement of family. This is an important finding, because it is estimated that multicomponent non-pharmacological interventions may reduce delirium incidence by 30 to 50%.16,18,21

Another important finding in the improved quality of care advice was that the advice of students often focussed on guiding the patients through the experience of delirium. The Delirium Experience showed students how to communicate with and what to say to patients who experience or experienced delirious episodes. Because many healthcare

7

Delirium Experience embedded in other delirium education. Multiple interventions, such as e-learnings, have been proven effective in improving delirium screening and severity rating.57–60 _{This encourages the use of multifaceted interventions to teach delirium care.}

Multifacated interventions use different approaches to teach delirium care. The use of multifaceted interventions in delirium education is recommended to increase the effectiveness of delirium education.4,15,61

Attitude towards delirium

We were unable to identify changes in attitudes towards delirium as measured by the Delirium Attitude Scale. Although attitude scales are a convenient way to measure attitudes, they have an important drawback as well: these scales do not always reveal the underlying dynamics of an attitude like cognitions or emotions.62 _{Healthcare professionals}

seem to be less aware of their negative attitudes towards delirium.2 _{This points towards}

more implicit attitudes, which need a more indirect measurement of attitude because the attitudes that people are unaware of, are hard to measure with (explicit) attitude scales.62

We will elaborate on this later in this chapter, after we explained how aspects of The Delirium Experience may have contributed to attitudinal change towards delirium.

Effective aspects of a serious game to teach delirium care

This thesis also aimed to identify the aspects of a serious game that contribute to specific improvements in delirium education (chapter 4, 5, and 6). We identified the characters’ perspectives, interactivity to experiment with care options, realistic care options, and feedback of importance to improve delirium education.

Characters’ perspectives

The patient

The patient’s perspective in The Delirium Experience was highly valued by students. Most importantly, the first-person perspective of the patient created understanding of the patient’s experience of delirium. This is important because healthcare professionals often fail to understand how frightening experiencing delirious episodes is.1,2 _{The patient’s}

perspective thereby functions as a way of providing students with patient feedback, which can facilitate reflective discussions.63 _{It supported students in making sense of the}

behavior of delirious patients, as was expected.39,64,65 _{This indicates that students not only}

created a new learning experience, but also linked this experience to their existing knowledge on behaviour of delirious patients. Because this is an important principle of experiential learning,39,66 _{we showed that a first-persons perspective not only supports}

learners in making sense of the experience,64,65 _{but that it also facilitates experiential}

learning. In contrast, recent studies show that students underestimate their learning when

they experienced increased cognitive effort.37,43 _{Students seem to perceive the cognitive}

effort as a sign of inferior learning and prefer not to choose for learning strategies that involve increased cognitive effort such as active learning.43 _{This is unfortunate, because}

the increased cognitive effort related to learning is associated with more effective learning and important in self-regulatory learning.43,44 _{Students therefore need to be convinced of}

the benefit of increased cognitive effort in learning.37,43 _{However, simply showing or}

telling students which strategy is best for learning, often does not convince them of the added value of increased cognitive effort.45

In our studies, we showed that students reported higher knowledge after playing The Delirium Experience, an active way of learning, compared to learning strategies that require less cognitive effort (e.g. watching a video). These results indicate that a serious game may support students to engage in learning activities that require more cognitive effort without the feeling of inferior learning. This is an important finding, because the perceived learning of students is essential for the learning strategy they choose, and therefore for the outcomes of learning and self-regulatory learning.43 _{A serious game may}

therefore also contribute to the promotion of self-regulatory learning, and aid in determining specific conditions that can enhance self-regulatory learning.46

Outcomes that did not improve after playing the game

Delirium screening and delirium severity rating

Important barriers for healthcare professionals in screening patients for delirium are lack of delirium awareness, inadequate levels of knowledge, and lack of education on delirium.3,11,12,29 _{However, delirium screening and severity rating are important elements}

of delirium care, and should therefore be taught to students.47 _{Although The Delirium}

Experience indirectly contributes to these barriers in screening for delirium by letting students become more familiar with the behavior of delirious patients, the game does not explicitly train delirium screening and severity rating skills. That, most likely, explains why we were unable to find improvements in students’ skills in delirium screening and severity rating in using the commonly applied clinical scales.48–50

Following from this, we will elaborate on two different strategies to improve delirium screening and severity rating as these are important for good quality delirium care.51,52

First, our results confirm that practicing and experimenting in a safe environment such as a serious game contributes to effective learning.53–55 _{We can therefore assume this also}

works for training students in using scales for delirium screening and severity rating, especially as training in using these scales is important for proper use of the scales.56 _This

indicates that adapting The Delirium Experience so it explicitly trains delirium screening and severity rating, would improve students’ skills in delirium screening and severity rating.

7

thereby decrease the discomfort healthcare professionals often experience with delirium management29–31

Interactivity

The interactive nature of the game allowed students to practice and experiment with different delirium care action in The Delirium Experience in a safe environment.53–55 _This

allowed students to fail to provide good care to the patient in the game, either intentionally (dark play) or unintentionally. In case students played the game dark play and intentionally failed to provide good quality care, they still achieved the same learning outcomes as the students who played the game with the intention to provide good quality care. This indicated that The Delirium Experience can serve students with different learning goal orientations (e.g. performance or master oriented) having the same learning outcomes.74

Differences in learning outcomes when using simulations as educational tool is a known disadvantage,75 _{so it is promising to see that our findings showed that interactivity as}

incorporated in The Delirium Experience can overcome this disadvantage.

For students who unintentionally did not provide good good care to the delirious patient in the game, the game was a safe environment for them to fail without the risk of being judged or assessed, or hurting the patient. This is important because there is a high fear of failure among students, and one of the reasons reported is the pressure to perform well and the experienced shame and embarressment when failing.76,77 _{The learning}

environment of a student can influence the fear of failure a student experiences.78

However, failing can also enhance learning outcomes, which is referred to as productive failure.79,80 _{It is therefore important to provide students with learning environments in}

which they can safely fail and learn.

The interactivity also allowed students to create new knowledge via active experi-mentation.39,66 _{In addition, the interactivity made students reflect on their choices and}

theorize on how to make better choices, all important principles of experiential learning.39,66 _{They had to actively recall knowledge, which is more effective than simply}

restudying information.44 _{The interactivity of a serious game is therefore an important}

aspects to promote experiential learning in delirium education.

Realism

A third aspect of The Delirium Experience that supported students in how to provide good quality care to delirious patients were the realistic care options. In The Delirium Experience, realistic videos were used in the design to show players both the patient’s and the nurse’s perspectives. Our findings showed that the use of a realistic design in delirium education may have the potential to improve reflection and knowledge transfer to practice.

Another important finding related to the patient’s perspective in the serious game was that it can address attitudes towards delirium and generate empathy for the patient. Because delirium education should address the existing negative attitudes of healthcare professionals towards delirium,1,2 _{incorporating the patient’s perspective holds promise to}

improve delirium education. The integrated emotions of the patients in the perspective evoked emotional responses in students that enhanced their learning experiences and addressed their attitudes.67,68 _{These emotional responses in combination with the patient}

feedback also increased reflection on the learning experience that is related to attitudes that are more persistent over time and a better predictor of their behavior when providing care to patients with delirium.39,63,69,70 _{Students were often surprised by how intense and}

frightening the patient’s experience is, which enhanced the learning outcomes.71

Finally, the patient’s perspective also showed students important elements of delirium care, and in particular why this matters to patients. Students therefore became more motivated to provide good quality care to delirious patients. Because of this, students also reflected on their actions within the game and in reality when caring for a delirious patient, an important principle of learning and experiential learning.40,66 _{It thereby also}

enhanced their perceived clinical importance of delirium, which is an important barrier in delirium detection.11,29

To summarize, our findings reveal that the patient’s perspective is important for students learning about delirium in multiple ways. But most importantly, the patient’s perspective can contribute to more patient-oriented delirium education, without causing extra burden to patients.1,2,24,28 _{The narrative of the patient, incorporated in the patient’s}

perspective, also promoted the more humanistic aspects of medicine.72 _{Previous research}

suggested that the involvement of patients and patient’s experience may be important in addressing negative attitudes towards delirium.1,3,6 _{We indeed showed that the patient’s}

perspective can address the negative attitudes towards delirium, without the involvement of real patients.

The nurse

For both nursing and medical students the nurse’s perspective was found to be important too. Current delirium education often fails to address collaboration between the different professions involved in delirium care.5,12 _{Our results indicated that taking another person’s}

perspective may enhance collaboration between healthcare professionals because medical students stated collaboration with nurses would be easier because they gained insight in their activities because of the nurse’s perspective.

The nurse’s perspective was also important for students because it provided them with a role model on how to provide care to delirious patients.73 _{Both medical and nursing}

students reported this was primarily on how to communicate with delirious patients and what to say to them. This is important, as communication skills in delirium care are important to address in delirium education.1,2 _{The care actions that were provided in}

7

experimentation increases learning outcomes and experiences.37,38 _{Both active}

experi-mentation and patient feedback in a narrative also play an important role in knowledge transfer in delirium care and medical performance, which is important to address in delirium education.3,39,63

In addition, the feedback of The Delirium Experience also provoked reflection on the learning materials and choices made regarding delirium care. Again, students reflected on their care choices if the written feedback was insufficient in answering their questions. Meaningful in-game feedback has shown to encourage students to reflect on mis-conceptions and the transfer of knowledge to new contexts.35 _{Although feedback is most}

effective when the messages are specific and clear,84,85 _{our results showed that students}

achieve learning goals in a different way if feedback is less specific. Because students do so by active experimentation and learning, this can also have a positive effect on learning outcomes.43,44

Reflections and future perspectives

Methodology

Design

As explained in the introduction of this thesis, there are many challenges in studying serious games and methodological issues that need to be addressed in future research.7,8,10,81,86–89 _{We aimed to take the suggested improvements into account when}

studying the effectiveness of the serious game The Delirium Experience.

To start with the need for (cluster) randomized controlled trials in studying serious games to be able to draw more reliable conclusions on the effectiveness of serious games,7,8,86,88 _{we conducted two (cluster) randomized controlled trials (chapter 2 and 4).}

In these studies we took the more specific suggestions on how to improve the study design of serious games into account. We therefore used a sufficient number of participants to reach adequate power in our studies. Also, both the study participants and the assessors were blinded to the study conditions to prevent bias. In these studies, we used validated measurement instruments. By conducting two randomized controlled trials we tested the game multiple times, showing consistent and reliable results. We thereby contribute to drawing more reliable conclusions on the effectiveness of serious games.

However, as was to be expected, practical challenges interfered with taking into account all the suggested improvements regarding the study design in all of our studies, so we also used two quasi-experimental designs (chapter 3 and 5). Practical challenges such as students’ schedule, research locations, and logistics make it hard to conduct randomized controlled trials investigating a serious game in medical education. Previous research also indicated that these challenges are major drawbacks of the use of To start with reflection, students reported that the realism in the serious game made

them reflect upon their care options chosen in the game. This was mainly because of the realism of the setting and its characteristics in The Delirium Experience. The realistic time limit players have within the game encouraged them to evaluate the most important care options. In addition, the realistic progression of the delirious episodes also made them reflect upon the care options they had chosen earlier. This reflection on care options is essential to integrate new knowledge into existing knowledge, but it often is challenging to support learners in this process.40,66,82 _{Our findings indicate that in case of learning to}

provide care to patients, a realistic design of options and settings can improve reflection and thereby knowledge integration.

Students also reported that the realism in the serious game enhanced the transfer of their knowledge into practice. This is an important finding as there is a need to identify possible ways to transfer knowledge into practice of delirium care.5 _{With regard to The}

Delirium Experience, students perceived both the care options players could choose from and the visuals on how to perform these care options resembling real life. Students expect that they will better remember and recognize care options and how to perform them because of the game’s resemblance to practice, and therefore expect to be able to transfer their knowledge into practice more easily. However, the literature shows that the transfer of knowledge to practice is complex and difficult even when healthcare professionals have a clear sense of what they are expected to do.83 _{In that respect, it is important to}

note we did not measure actual knowledge transfer or clinical performance.

In contrast to our findings, previous research showed that schematic, textual or cartoonlike visual designs are often equally or more effective compared to more realistic designed serious games.81,82 _{Because of the bigger challenges of designing realistic}

serious games, it was emphasized to focus more on learning content than on realistic designs. Research also showed that the use of realistic spoken information accompanied by visual information is more effective for learning than written information.82 _{Our results}

indicated that for delirium education the use of a realistic design seems to be important; however, it is important to note that we did not investigate the effect of a schematic design. We therefore feel more research is needed on how levels of realism may enhance learning outcomes in specific domains.81

Feedback

A final, but important, aspect of The Delirium Experience that contributed to supporting students on providing good quality care to delirious patients was the incorporated feedback, both written and in the narrative. The feedback contributed to principles of experiential learning in different ways. The feedback provoked active experimentation with the learning material because it encouraged students to try different options and strategies in delirium care.39,66 _{Because the written feedback did not always fulfill the}

7

Measurements

In previous studies on serious game research, it was recommended to use standardized or validated measurement instruments to enhance the quality of evidence and comparability of serious games.7,10,94 _{In our studies, we did use validated measurement instruments to investigate}

The Delirium Experience’s effectiveness. In addition, we used an Learning Motivation and Engagement questionnaire that had already been used in previous serious games research to enhance comparability of our outcomes with other games.95 _{However, we encountered an}

important challenge in using standardized measurements: outcomes relevant to measure The Delirium Experience’s effectiveness (e.g. quality of care advice for delirium) differs from outcomes that are relevant to measure the effectiveness of other serious games that try to achieve other goals. The focus in our studies was to investigate the effectiveness of The Delirium Experience as an educational intervention on delirium. We therefore used measurements with which we could compare the game to other educational interventions on delirium, such as scales for screening and severity rating of delirium, and specific attitudes towards delirious patients.48,49

Hence, these measurements are relevant to compare this serious game with other delirium education interventions, but they do not to enhance comparability with other serious games.

As mentioned before, we were unable to identify changes in attitudes towards delirium by making use of an attitude scale (chapter 2 and 4). However, with in-depth interviews we were able to identify aspects of the game that affected students’ attitudes towards delirium (chapter 6). This points towards more implicit attitudes of which students are less aware.2

Using a scale to measure these attitudes assumes that students both have the ability and motivation to accurately report their attitudes, which seems to be invalid for measuring implicit attitudes towards delirium. Implicit attitudes need a more indirect measurement of those attitudes. Techniques that have proven to be effective in measuring implicit attitudes are for example response-window evaluative priming, the Implicit Association Test, and the response-window Implicit Association Test.96 _{These techniques use reaction time and error}

rates in predefined tasks (e.g. link good and bad words to two attributes or objects) to measure implicit attitudes or beliefs. We recommend including such measures in future studies on delirium education.

Delirium education

We believe our findings are also important for the education on other health conditions besides delirium. For example, the importance of incorporating the patient’s perspective into delirium education also yields implications for using a patient’s perspective on other health conditions. In particular, incorporating the patient’s perspective may be well suited to support healthcare professionals in understanding the experience of patients with dementia and their attitudes towards them.97,98 _{But also in mental healthcare in general,}

healthcare professionals often lack understanding of the patient’s experience and education often fails to address this.99,100 _{We therefore consider it also highly beneficial to}

incorporate the patient’s perspective in education on other healthcare conditions. randomized controlled trials to investigate the dynamic context of serious games.90–93

Apart from these practical challenges that interfered with the intentions to conduct randomized controlled trials, we also encountered some ethical considerations. We already showed the game’s effectiveness with a randomized controlled trial in medical students, and also wanted to investigate the effect in nursing students because of the broad target audience of The Delirium Experience (chapter 3). However, using a randomized controlled trial would have to include a control group of students who would not be allowed to play the game during their regular practical (they would be allowed to play The Delirium Experience, but in their spare time). We therefore decided to use a before-after test instead to allow all students to play The Delirium Experience in their scheduled practical. The quasi-experimental design of the study described in chapter 5, where we were interested in student’s reason to choose for a specific play strategy, obviously did not allow randomization.

Outcomes

Another plea was made to also investigate the effects of serious games on attitudinal outcomes, patient outcomes, and clinician behavior in addition to the most common outcomes on knowledge, skills, and player satisfaction.7,8 _{We did measure the effect of the}

serious game on attitudinal outcomes (chapter 2, 4, 5, and 6), on which we will elaborate later on. However, we did not measure patient outcomes nor clinician behaviors, which are also important to judge the effectiveness of the serious game. This was mainly because our target group, medical and nursing students, did not have direct contact with patients at the moment of delirium education in most of our studies. We were therefore unable to measure patient outcomes and clinician behavior. This limitation also relates to another limitation in our research design: we did not include research settings in which participants were working with delirious patients.10 _{We would therefore recommend for future studies}

to investigate the game’s effectiveness in other populations besides the student population, and thereby investigate its effect on patient outcomes and clinician behavior.

It is emphasized that to develop more efficacious serious games, we need to know what makes a game effective in achieving its goals.7–10,86,89 _{To gain this knowledge, we}

need to distinguish between individual game aspects when investigating serious games, instead of studying it as a whole artifact with a predefined outcome (e.g. knowledge).8,10,86,89

We therefore examined how specific game aspects were related to improvements in delirium education (chapter 4, 5, and 6). The insights gained in these studies can contribute to the development of new serious games in, for example, the field of medical education. However, to gain even deeper insight in how the identified aspects contribute to achieving a serious game’s goals, it is important in future studies to use adaptable serious games in which individual aspects can be removed and added.35,91,92 _{Since we used The Delirium}

Experience in its original form, we were unable to gain insights in individual game aspects without tampering with other game aspects.

7

To enhance students’ understanding of the patient’s experience it is important to incorporate the patient’s perspective and experiences into delirium education. In addition, the perspectives and experiences of the patients need to be realistic for both audio and visuals. To increase knowledge on how to provide good quality care to delirious patients, delirium education should include perspectives of people involved in delirium care. Furthermore, delirium education should be interactive in such a way that students can experiment with different care options in a safe environment. This environment should be realistic and resemble the real-life caring for a patient with delirium. Finally, delirium education should incorporate feedback in multiple ways, encouraging and supporting students to find answers to their questions via active experimentation.

Our studies focused mainly on students in the preclinical phase of their education, and our results are therefore limited to how delirium education was improved by the game in this phase. For now, we would therefore recommend to use The Delirium Experience embedded into the current delirium education in the preclinical phase to teach more general delirium management and the patient’s experience. In the clinical phase, delirium education could have a more specific focus on delirium screening and severity rating, and refreshing knowledge on delirium management and the patient’s experience. Further research is needed on how a serious game on delirium is best embedded into the curriculum and continuous professional development.

Serious games

The Delirium Experience incorporated both explicit feedback (written form) and implicit feedback (the progression of the narrative). Our findings showed that the written feedback in itself was often insufficient for players to answer unresolved questions on why care actions were correct or incorrect and how to improve. That is most likely because the written feedback in the game is outcome-oriented (corrective) feedback, and it is known that process-oriented (explanatory) feedback is more beneficial for learning.101 _{In contrast,}

as explained earlier, some players use the feedback incorporated into the game’s narrative to find answers to their questions by active experimentation in the game. However, there are also players who were not triggered to find answers to their questions in this way, and they valued the game’s feedback as insufficient. Further research is warranted on how we can support these students in making use of other feedback types to fulfill their learning needs, especially as research on how learner characteristics influence the effect of feedback in serious games is sparse.101 _{A strategy that may support students in making}

use of different feedback types is modeling. Modeling is showing how to solve a particular problem, or which information is important to solve this problem.82 _{Serious games that}

make use of modeling support players in the selection of relevant information and improve the learning potential.82

Our findings indicated that in case of delirium education, realistic design of options and settings can improve reflection and thereby knowledge integration. This stands in contrast to earlier reviews that showed that schematic, textual or cartoonlike visual designs are often equally or more effective compared to more realistic designed serious games.81,82

However, it was also suggested that more research is needed on how levels of realism may enhance learning outcomes in specific domains.81 _{We argue that in case of delirium}

education, or other patient-oriented education, a realistic design can support learners to interpret and make sense of the experience, but this needs to be further studied.

Conclusions

A serious game that makes use of experiential learning principles is a well-suited strategy to improve delirium education in enhancing understanding of the patient’s experience

7

26. Coyle, M. A., Chang, H. C., Burns, P. & Traynor, V. Impact of Interactive Education on Health Care Practitioners and Older Adults at Risk of Delirium: A Literature Review. J. Gerontol. Nurs. 44, 41–48 (2018).

27. Robles, M. J., Esperanza, A., Pi-Figueras, M., Riera, M. & Miralles, R. Simulation of a clinical scenario with actresses in the classroom: A useful method of learning clinical delirium management. Eur. Geriatr. Med. 8, 474–479 (2017).

28. Fisher, J. M. & Walker, R. W. A new age approach to an age old problem: using simulation to teach geriatric medicine to medical students. Age Ageing 43, 424–428 (2014).

29. Trogrlić, Z. et al. Attitudes, knowledge and practices concerning delirium: A survey among intensive care unit professionals. Nurs. Crit. Care 1–8 (2016). doi:10.1111/nicc.12239

30. Maximous, R. et al. Pain, agitation and delirium assessment and management in a community medical-surgi-cal ICU: results from a prospective observational study and nurse survey. BMJ Open Qual. 7, e000413 (2018). 31. Rowley-Conwy, G. Barriers to delirium assessment in the intensive care unit: A literature review. Intensive Crit.

Care Nurs. 44, 99–104 (2018).

32. Orsini, C., Binnie, V. I. & Wilson, S. L. Determinants and outcomes of motivation in health professions education: a systematic review based on self-determination theory. J. Educ. Eval. Health Prof. 13, 19 (2016).

33. Ryan, R. M. & Deci, E. L. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78 (2000).

34. Ryan, R. M., Rigby, C. S. & Przybylski, A. The motivational pull of video games: a self-determination theory approach. Motiv. Emot. 30, 344–360 (2006).

35. Lameras, P. et al. Essential features of serious games design in higher education: Linking learning attributes to game mechanics. Br. J. Educ. Technol. 48, 972–994 (2017).

36. Carini, R. M., Kuh, G. D. & Klein, S. P. Student Engagement and Student Learning: Testing the Linkages*. Res. High. Educ. 47, 1–32 (2006).

37. Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K. & Kestin, G. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc. Natl. Acad. Sci. U. S. A. 116, 19251–19257 (2019).

38. Wagenaar, A., Scherpbier, A. J. J. A., Boshuizen, H. P. A. & Van Der Vleuten, C. P. M. The importance of active involvement in learning: A qualitative study on learning results and learning processes in different traineeships. Adv. Heal. Sci. Educ. 8, 201–212 (2003).

39. Kolb, D. A. Experiential learning: Experience as the source of learning and development. (Englewood Cliffs, 1984).

40. Sandars, J. The use of reflection in medical education: AMEE Guide No. 44. Med. Teach. 31, 685–695 (2009). 41. Snibsøer, A. K. et al. Self-reported and objectively assessed knowledge of evidence-based practice

terminology among healthcare students: A cross-sectional study. PLoS One 13, e0200313 (2018).

42. Baessler, F. et al. Delirium: Medical Students’ Knowledge and Effectiveness of Different Teaching Methods. Am. J. Geriatr. Psychiatry 27, 737–744 (2019).

43. Kirk-Johnson, A., Galla, B. M. & Fraundorf, S. H. Perceiving effort as poor learning: The misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cogn. Psychol. 115, (2019).

44. Rowland, C. A. The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychol. Bull. 140, 1432–1463 (2014).

45. Yan, V. X., Bjork, E. L. & Bjork, R. A. On the difficulty of mending metacognitive illusions: A priori theories, fluency effects, and misattributions of the interleaving benefit. J. Exp. Psychol. Gen. 145, 918–933 (2016).

46. Zeidner, M. Self-regulated learning: Current fissures, challenges, and directions for future research. High Abil. Stud. 30, 255–276 (2019).

47. Inouye, S. K. Delirium-A Framework to Improve Acute Care for Older Persons. J. Am. Geriatr. Soc. 66, 446–451 (2018).

48. de Rooij, S. E. et al. Delirium subtype identification and the validation of the Delirium Rating Scale—Revised-98 (Dutch version) in hospitalized elderly patients. Int. J. Geriatr. Psychiatry 21, 876–882 (2006).

49. Schuurmans, M. J., Shortridge-Baggett, L. M. & Duursma, S. A. The Delirium Observation Screening Scale: A Screening Instrument for Delirium. Res. Theory Nurs. Pract. 17, 31–50 (2003).

References

1. Copeland, C., Fisher, J. & Teodorczuk, A. Development of an international undergraduate curriculum for delirium using a modified delphi process. Age Ageing 47, 131–137 (2018).

2. Teodorczuk, A., Mukaetova-Ladinska, E., Corbett, S. & Welfare, M. Reconceptualizing models of delirium education: findings of a Grounded Theory study. Int. Psychogeriatr. 25, 645–55 (2013).

3. Brooke, J. & Manneh, C. Caring for a patient with delirium in an acute hospital: The lived experience of cardiology, elderly care, renal, and respiratory nurses. Int. J. Nurs. Pract. e12643 (2018). doi:10.1111/ijn.12643 4. Yanamadala, M., Wieland, D. & Heflin, M. T. Educational interventions to improve recognition of delirium:

A systematic review. Journal of the American Geriatrics Society 61, (2013).

5. Morandi, A. et al. An interdisciplinary statement of scientific societies for the advancement of delirium care across Europe (EDA, EANS, EUGMS, COTEC, IPTOP/WCPT). BMC Geriatr. 19, 253 (2019).

6. Fisher, J. M. et al. Towards an understanding of why undergraduate teaching about delirium does not guarantee gold-standard practice--results from a UK national survey. Age Ageing 44, 166–170 (2015). 7. Gentry, S. V. et al. Serious gaming and gamification education in health professions: systematic review. J. Med.

Internet Res. 21, (2019).

8. Haoran, G., Bazakidi, E. & Zary, N. Serious Games in Health Professions Education: Review of Trends and Learning Efficacy. Yearb. Med. Inform. 28, 240–248 (2019).

9. Lu, A. S. & Kharrazi, H. A State-of-the-Art Systematic Content Analysis of Games for Health. Games Health J. 7, 1–15 (2018).

10. Gorbanev, I. et al. A systematic review of serious games in medical education: quality of evidence and pedagogical strategy. Med. Educ. Online 23, (2018).

11. Morandi, A. et al. Consensus and variations in opinions on delirium care: a survey of European delirium specialists. Int. Psychogeriatr. 25, 2067–75 (2013).

12. Nydahl, P. et al. Survey among critical care nurses and physicians about delirium management. Nurs. Crit. Care 23, 23–29 (2018).

13. Palacios-Ceña, D. et al. How do doctors and nurses manage delirium in intensive care units? A qualitative study using focus groups. BMJ Open 6, 1–11 (2016).

14. Morandi, A. et al. Delirium superimposed on dementia: A quantitative and qualitative evaluation of informal caregivers and health care staff experience. J. Psychosom. Res. 79, 272–280 (2015).

15. Fowler, B. M. Clinical education to decrease perceived barriers to delirium screening in adult intensive care units. Crit. Care Nurs. Q. 42, 41–43 (2019).

16. Lee, S. et al. Recognition, Prevention, and Treatment of Delirium in Emergency Department: An Evidence-Based Narrative Review. Am. J. Emerg. Med. 158454 (2019). doi:10.1016/j.ajem.2019.158454

17. Barbateskovic, M. et al. Pharmacological interventions for prevention and management of delirium in intensive care patients: a systematic overview of reviews and meta-analyses. BMJ Open 9, e024562 (2019). 18. Siddiqi, N. et al. Interventions for preventing delirium in hospitalised non-ICU patients. Cochrane Database

Syst. Rev. (2016). doi:10.1002/14651858.CD005563.pub3

19. Burry, L. et al. Pharmacological interventions for the treatment of delirium in critically ill adults. Cochrane Database Syst. Rev. (2019). doi:10.1002/14651858.CD011749.pub2

20. Abraha, I. et al. Non-pharmacological interventions to prevent or treat delirium in older patients: Clinical practice recommendations the SENATOR-ONTOP series. J. Nutr. Health Aging 20, 927–936 (2016).

21. Hshieh, T. T. et al. Effectiveness of Multicomponent Nonpharmacological Delirium Interventions. JAMA Intern. Med. 175, 512 (2015).

22. Dautzenberg, P. L. J. et al. Herziene richtlijn ‘Delier volwassenen en ouderen’. Ned. Tijdschr. voor Geneeskd. 158, (2014).

23. Lee, T. W. & Ko, Y. K. Effects of self-efficacy, affectivity and collective efficacy on nursing performance of hospital nurses. J. Adv. Nurs. 66, 839–848 (2010).

24. Leigh, G. T. High-Fidelity Patient Simulation and Nursing Students ’ Self-Efficacy : A Review of the Literature. Int. J. Nurs. Educ. Scholarsh. 5, 1–17 (2008).

7

77. Beiter, R. et al. The prevalence and correlates of depression, anxiety, and stress in a sample of college students. J. Affect. Disord. 173, 90–96 (2015).

78. Giel, L. I. S., Noordzij, G., Noordegraaf-Eelens, L. & Denktaş, S. Fear of failure: a polynomial regression analysis of the joint impact of the perceived learning environment and personal achievement goal orientation. Anxiety, Stress. Coping 33, 123–139 (2020).

79. Kapur, M. Productive Failure. Cogn. Instr. 26, 379–424 (2008).

80. Kapur, M. Learning from productive failure. Learn. Res. Pract. 1, 51–65 (2015).

81. Wouters, P., van Nimwegen, C., van Oostendorp, H. & van der Spek, E. D. A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249–265 (2013).

82. Wouters, P. & van Oostendorp, H. Overview of Instructional Techniques to Facilitate Learning and Motivation of Serious Games. in Instructional Techniques to Facilitate Learning and Motivation of Serious Games 1–16 (Springer International Publishing, 2017). doi:10.1007/978-3-319-39298-1_1

83. Kahlke, R. M., McConnell, M. M., Wisener, K. M. & Eva, K. W. The disconnect between knowing and doing in health professions education and practice. Adv. Heal. Sci. Educ. (2019). doi:10.1007/s10459-019-09886-5 84. van de Ridder, J. M. M., Mcgaghie, W. C., Stokking, K. M. & ten Cate, O. T. J. Variables that affect the process and

outcome of feedback, relevant for medical training: A meta-review. Med. Educ. 49, 658–673 (2015). 85. Paterson, C., Paterson, N., Jackson, W. & Work, F. What are students’ needs and preferences for academic

feedback in higher education? A systematic review. Nurse Educ. Today 85, 104236 (2020).

86. Wang, R., DeMaria, S., Goldberg, A. & Katz, D. A Systematic Review of Serious Games in Training Health Care Professionals. Simul. Healthc. J. Soc. Simul. Healthc. 11, 41–51 (2016).

87. Kharrazi, H., Lu, A. S., Gharghabi, F. & Coleman, W. A scoping review of health game research: Past, present, and future. Games Health J. 1, 153–164 (2012).

88. Girard, C., Ecalle, J. & Magnan, A. Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. J. Comput. Assist. Learn. 29, 207–219 (2013).

89. Boyle, E. A. et al. An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput. Educ. 94, (2016).

90. Mayer, I. et al. The research and evaluation of serious games: Toward a comprehensive methodology. Br. J. Educ. Technol. 45, 502–527 (2014).

91. Warmelink, H., Valente, M., van Tol, R. & Schravenhoff, R. Get it Right! Introducing a framework for integrating Validation in applied game design. Springer Int. Publ. 461–470 (2016). doi:10.1007/978-3-319-40216-1_51 92. Van der Kooij, K., Hoogendoorn, E., Spijkerman, R. & Visch, V. Validation of Serious Games. Int. J. Serious Games

2, (2015).

93. Connolly, T. M., Boyle, E. A., Macarthur, E., Hainey, T. & Boyle, J. M. A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 59, 661–686 (2012).

94. Hamari, J., Koivisto, J. & Pakkanen, T. Do Persuasive Technologies Persuade?-A Review of Empirical Studies. Persuas. Technol. 8462, 118–136 (2014).

95. Dankbaar, M. E. W. et al. Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness. BMC Med. Educ. 17, 30 (2017).

96. Cunningham, W. A., Preacher, K. J. & Banaji, M. R. Implicit attitude measures: consistency, stability, and convergent validity. Psychol. Sci. 12, 163–70 (2001).

97. Banerjee, S. et al. How do we enhance undergraduate healthcare education in dementia? A review of the role of innovative approaches and development of the Time for Dementia Programme. Int. J. Geriatr. Psychiatry 32, 68–75 (2017).

98. Frank, C. & Forbes, R. F. A patient’s experience in dementia care: Using the "lived experience" to improve care. Can. Fam. Physician 63, 22–26 (2017).

99. Knaak, S., Mantler, E. & Szeto, A. Mental illness-related stigma in healthcare: Barriers to access and care and evidence-based solutions. Healthc. Manag. forum 30, 111–116 (2017).

100. Lester, H. & Tritter, J. Q. ‘Listen to my madness’: understanding the experiences of people with serious mental illness. Sociol. Heal. Illn. 27, 649–669 (2005).

101. Johnson, C. I., Bailey, S. K. T. & Van Buskirk, W. L. Designing Effective Feedback Messages in Serious Games and Simulations: A Research Review. in Instructional Techniques to Facilitate Learning and Motivation of Serious Games 119–140 (Springer International Publishing, 2017). doi:10.1007/978-3-319-39298-1_7

50. Scheffer, A. C., van Munster, B. C., Schuurmans, M. J. & de Rooij, S. E. Assessing severity of delirium by the delirium observation screening scale. Int. J. Geriatr. Psychiatry 26, 284–291 (2011).

51. Kalish, V. B., Gillham, J. E. & Unwin, B. K. Delirium in Older persons: Evaluation and Management. Am. Fam. Physician 90, 150–158 (2014).

52. Miller, M. O. & Miller, M. O. Evaluation and management of delirium in hospitalized older patients. Am. Fam. Physician 78, 1265–1270 (2008).

53. Wattanasoontorn, V., Boada, I., García, R. & Sbert, M. Serious games for health. Entertain. Comput. 4, 231–247 (2013).

54. Cook, D. A. et al. Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Medical Teacher 35, (2013).

55. Graafland, M., Schraagen, J. M. & Schijven, M. P. Systematic review of serious games for medical education and surgical skills training. Br. J. Surg. 99, 1322–1330 (2012).

56. De, J. & Wand, A. P. F. Delirium Screening: A Systematic Review of Delirium Screening Tools in Hospitalized Patients. Gerontologist 55, 1079–1099 (2015).

57. van de Steeg, L., IJkema, R., Langelaan, M. & Wagner, C. Can an e-learning course improve nursing care for older people at risk of delirium: a stepped wedge cluster randomised trial. BMC Geriatr. 14, (2014).

58. McCrow, J., Sullivan, K. A. & Beattie, E. R. Delirium knowledge and recognition: a randomized controlled trial of a web-based educational intervention for acute care nurses. Nurse Educ. Today 34, 912–7 (2014).

59. van Velthuijsen, E. L. et al. Can education improve clinical practice concerning delirium in older hospitalised patients? Results of a pre-test post-test study on an educational intervention for nursing staff. BMC Med. Educ. 18, 59 (2018).

60. Detroyer, E. et al. The effect of an interactive delirium e-learning tool on healthcare workers’ delirium recognition, knowledge and strain in caring for delirious patients: a pilot pre-test/post-test study. BMC Med. Educ. 16, 17 (2016).

61. Norman, S. L. & Taha, A. A. Delirium knowledge, self-confidence, and attitude in pediatric intensive care nurses. J. Pediatr. Nurs. 46, 6–11 (2019).

62. Perloff, R. M. The dynamics of persuasion: communication and attitudes in the 21st century. (Routledge, 2010). 63. Baines, R. et al. The impact of patient feedback on the medical performance of qualified doctors: a systematic

review. BMC Med. Educ. 18, 173 (2018).

64. Reeve, C. Narrative-Based Serious Games. in Serious Games on the Move (eds. Petrovic, O. & Brand, A.) 73–89 (Springer Vienna, 2009). doi:10.1007/978-3-211-09418-1_5

65. Koivisto, J.-M., Niemi, H., Multisilta, J. & Eriksson, E. Nursing students’ experiential learning processes using an online 3D simulation game. Educ. Inf. Technol. 22, 383–398 (2017).

66. Yardley, S., Teunissen, P. W. & Dornan, T. Experiential learning: AMEE Guide No. 63. Med. Teach. 34, e102–e115 (2012).

67. Dewey, J. Experience and education. (NY: Collier Books, 1938).

68. Gordon, M. et al. Patient/service user involvement in medical education: A best evidence medical education (BEME) systematic review: BEME Guide No. 58. Med. Teach. 42, 4–16 (2019).

69. Petty, R. E. & Cacioppo, J. T. The Elaboration Likelihood Model of Persuasion. in Advances in experimental social psychology (ed. Berkowitz, L.) 19, 123–205 (Academic Press, 1986).

70. Petty, R. E. & Krosnick, J. A. Attitude strength : antecedents and consequences.

71. van der Spek, E. D., van Oostendorp, H. & Ch. Meyer, J.-J. Introducing surprising events can stimulate deep learning in a serious game. Br. J. Educ. Technol. 44, 156–169 (2013).

72. Easton, G. How medical teachers use narratives in lectures: a qualitative study. BMC Med. Educ. 16, 3 (2016). 73. Fida, R., Laschinger, H. K. S. & Leiter, M. P. The protective role of self-efficacy against workplace incivility and

burnout in nursing: A time-lagged study. Health Care Manage. Rev. 43, 21–29 (2018).

74. Kool, A., Mainhard, T., Brekelmans, M., van Beukelen, P. & Jaarsma, D. Goal orientations of health profession students throughout the undergraduate program: a multilevel study. BMC Med. Educ. 16, 100 (2016). 75. Meyer, M. et al. Simulation as a Learning Experience: Perceptions of New RNs. Clin. Simul. Nurs. 10, 384–394 (2014). 76. Elliot, A. J. & Thrash, T. M. The intergenerational transmission of fear of failure. Pers. Soc. Psychol. Bull. 30, 957–71