A serious game for learning alarm combinations in the pediatric intensive care
Bachelor’s Project Thesis
Steven Warmelink, s.warmelink@student.rug.nl, Supervisor: Dr. Fokie Cnossen
Abstract: Although a serious game should strive to be entertaining and engaging, it also has another purpose: to educate its user. In his master thesis, Koen Brinks reported a shortcoming in the ability of nurses in the pediatric intensive care unit of the UMCG to recognize alarms and their corresponding causes and treatments. In a previous bachelor project Pim van der Meulen built a serious game that would educate its users about these alarms. In the serious game, users control a nurse in an Intensive Care unit who selects the correct causes and treatments given certain alarms of vital functions. We updated this serious game with a new learning model, various graphical features and presentations of new cases. We compared the updated serious game to a training program that was identical but stripped of all game elements. The test group (serious game) and control group (training without game elements) were tested regarding their knowledge gained during training. No significant difference between the two groups was found.
This means the game elements of the program do not influence performance in this particular setting.
1 Introduction
In environments such as a pediatric intensive care unit (PICU), patient monitor alarms are very im- portant. Patient monitor alarms alert nurses of any abnormalities that might occur. However, not every alarm is clinically important. In research by Law- less (1994), it was revealed that there was “one significant alarm sounding for every 17.3 false or induced alarm soundings”. With this many false alarms going off, it is very important for a nurse to be able to discern the various alarms they might encounter and know what might cause them.
1.1 Previous research
In 2015, Brinks researched alarm hazards at the PICU of the University Medical Center Gronin- gen (UMCG). Brinks identified a lack of knowledge among nurses about the causes of alarms that could go off on the PICU. Brinks noted that “[..] there is no manual or list of protocols that is used to in- struct new employees on the skills, rules or knowl- edge that they need to manage alarms” (p43).
Based on both structured and unstructured in- terviews with PICU nurses, Brinks developed a
knowledge base which included the most common and most crucial alarms and their corresponding causes and treatments (see appendix D). All inter- viewees noted that the most important physiologi- cal measurements were blood pressure (BP), heart rate (HR) and blood oxygen saturation (Sat). In case one (or more) of these values would be too high or too low, an alarm went off indicating whether the measured value was too high or too low. If a physio- logical measurements has a normal value, the alarm is absent in the knowledge base entry.
One of the main difficulties of developing the knowledge base was its specificity: by adding more specific cases to the knowledge base, the number of cases represented in the knowledge base increased greatly, making the knowledge base significantly more complex. In the end, lower complexity was favored over higher specificity. Brinks developed a very rudimentary training mode in which users could select alarms while keeping the corresponding cause and treatment hidden. After thinking about the possible diagnoses they could then click a but- ton to reveal the actual cause and treatment (See figure 1.1).
For future work, Brinks suggested expanding the training by keeping scores and by using smart se-
Figure 1.1: Alpha training mode developed by Brinks.
quencing algorithms, which could be used to deter- mine which case to show based on the users per- formance so far, e.g.: performing well on a certain case causes that case to show up less frequently, while performing poorly on a case causes that case to show up more frequently.
Using serious games for training medical pro- fessionals is becoming more and more popular (Graafland et al., 2012, Wang et al., 2016). Seri- ous games have the benefit of improving a user’s motivation to learn (or keep on learning).
In 2016, Van der Meulen developed a serious game called ‘PIC Play’ based on Brinks’ research.
PIC Play used part of the knowledge base devel- oped by Brinks and in collaboration with nurses of the PICU of the UMCG. PIC Play would allow its users to learn all the causes and treatments corre- sponding to particular alarms that could occur on the PICU. In the serious game, users took on the role of a nurse working on the PICU. Patients could have an underlying problem which would cause an alarm to go off, in which case the user could treat the patient by selecting the correct cause and treatment. Treating patients correctly earned points and ultimately cured patients, whereas in- correctly treating patients earned no points and could ultimately cause the patient to die, remov-
ing points.
Van der Meulen’s aim was to determine whether the use of a serious game yielded better results than that of a standard learning method for learning this knowledge base. Van der Meulen did this by comparing two learning methods: one group had trained using a serious game, and another group had trained using a version of the serious game where all game elements had been taken out, leav- ing just the learning part of the program intact. Af- terwards, both groups made an identical test on pa- per, and test scores were compared to assess perfor- mance. Van der Meulen’s results showed that there was no significant difference between training using the serious game and training using the standard learning method. However, having a low statisti- cal power due to low participant numbers, along with various proposed improvements such as an im- proved learning model and better logging during training and of user demographics, offered enough incentive to pursue a follow-up study.
1.2 Current study
In our current study, we wanted to answer the same research question as Van der Meulen: “Does the use of a serious game yield better results than the use of
a standard learning method in helping to find the most likely cause and treatment for certain com- binations of patient monitor alarms in the pedi- atric intensive care?”. We did this by comparing two learning methods, where one group trains us- ing a serious game (PIC Play) and the other group trains using a more traditional learning method (NO Play) which is the serious game stripped of all game elements. Whereas Van der Meulen used 10 entries from Brinks’ knowledge base, our database consists of eight entries. Pilot studies indicated that the knowledge base used by Van der Meulen might be too complex, so in order to reduce complexity, we removed two entries that had treatments which were identical to other treatments already occur- ring in the database. The final database used can be found in appendix E.
1.2.1 PIC Play: The Game
While playing the serious game, the user controls a nurse in charge of the PICU. There are various beds that may or may not hold a patient (see Figure 1.2). Each bed with a patient has an alarm that can go off.
When an alarm goes off, the user can move the nurse to the patient to start diagnosing and treat- ing the patient. Clicking on a patient opens the so-called problem window (see Figure 1.3). In the problem window, alarms are shown on the left, and possible causes are shown on the right. The user has to choose the most probable cause given the alarms.
If they do so correctly, they are allowed to go to the treatment tab and select the most probable treat- ment. However, should they select the wrong cause, they are sent back to the PICU and the patient’s state will deteriorate. This hard exclusion of contin- uing to treat the patient was used because selecting a treatment without knowing the underlying cause could cause users to learn wrong cause-treatment pairs. After treating a patient well three times in total, the patient is saved and allowed to go home (netting the player bonus points), whereas after treating a patient incorrectly three times in total, the patient is lost and the player loses points.
1.2.2 NO Play: The Traditional Learning Method
Training using NO Play is a lot more straightfor- ward: there are no nurses, no points, no beds and no patients. The learning model, presentations and new alarm icons are still used, but users only receive feedback after an answer (correct/incorrect) and a total tally of cases handled correctly or incorrectly (see Figure 1.4).
1.2.3 Learning Model
One of the main issues with the programs devel- oped by Van der Meulen was the learning model, which was a simple learning algorithm based on ELO. It needed a high number of trials for the learning model to adapt to the user, and the rate at which the learning rate changed remained stable over time. For the updated serious game, a learning model based on a spacing algorithm used in Dren- then’s 2015 Master Thesis. Drenthen developed a spacing algorithm which calculates the interval at which a new case should be presented. This spacing algorithm should adapt to the user’s performance faster than the ELO-based learning algorithm since its learning rate is not static. The spacing algorithm uses intervals to determine which flash card to show next. For example, a flash card with an interval of 5 means that the ideal time for that flash card to appear is after 5 other flash cards have appeared.
The interval is based on previous performance of the particular flash card and whether the the flash card was shown on time (i.e. at the time specified by the interval value), too early or too late. We did this using the following formulas, as developed by Drenthen (2015):
I0= startinterval (1.1) In= max(startinterval, In−1+ dn∗ In−1∗ jn)
(1.2) Inis the interval as calculated at time n. Since In is influenced by its previous state, we must be able to look back in time, in particular at In−1. The dn variable is the due-factor. A due-factor represents whether a case is presented on time (d = 1), too early (0 ≤ d) or too late (d > 1). The due factor is calculated by dividing the actual interval after which the case was represented by the ideal interval
Figure 1.2: Two patients with alarms going off
Figure 1.3: PIC Play problem window
Figure 1.4: NO Play problem window
(which is the interval that was calculated when this case was last encountered).
dn =In−1actual
In−1
(1.3)
The jnfactor of formula 1.2 is the so-called judg- ment factor. If a case is answered correctly, it takes a positive value (1.2), and if a case is answered in- correctly it takes a slightly negative value (-0.2).
This means that correctly answered cases get a larger interval (and thus appear later than usual), whereas incorrectly answered cases get a shorter interval (and thus appear earlier than usual).
The initial pool of cases consists of three cases.
Once a user has mastered a case (the In value is above the threshold t = 7), a new case is added to the pool of cases. Now the pool consists four cases.
Once the user has mastered a total of two cases, an additional new case is added to the pool. This process continues until all eight available cases are in the pool.
There is also a refractory period which prevents cases from being shown in quick succession. This was done by keeping track of how many problems ago the case was shown. If this number was too small, this meant that the case was shown very re- cently and can not be shown now. To accomplish this, rather than returning the interval In, an ar- bitrarily large interval was returned instead. Since the algorithm looks for the case with the lowest interval, the smallest non-recent case is returned.
1.2.4 Presentation of new cases
One aspect of the original versions of PIC Play and NO Play was the fact that users were thrown in at the deep end: the first time they encountered a case they immediately had to give a cause and treat- ment, without having had the opportunity to study the case first. This effectively meant that whenever a new case was presented, the user just had to take a guess. This felt very counter-intuitive, since the program is supposed to be training the user, not punishing them for not knowing something they could not possibly have known.
In response, deliberate presentation of new cases was added to the program. When starting the pro- gram the user is shown all cases that are in the pool of available cases and instructed to study the rela- tions between alarms and cause/treatment. When a new case is added to the pool of possible cases it is first presented to the user to be studied.
1.2.5 Logging
Logging has been updated to include (time- stamped) all possible answering options, which op- tion the participant ended up choosing, time spent looking at a new presentation, user mouse actions such as confirming an answer and going to the treatment tab, etc. This way we can trace all user actions during the training phase. Apart from the log file, all answers and times spent at questions and sub questions (correct/incorrect only) are stored in a .csv file for easier statistical analysis.
Figure 1.5: Alarm icons
1.2.6 Miscellaneous
Apart from these major improvements, some minor improvements were made as well. The instructions that used to be on paper have been partly digitized and made more compact. Where the old program used a plain text representation of alarms (i.e. ‘BP Low’ for a low blood pressure alarm), the new pro- gram shows the icons to represent alarm (see figure 1.5).
We improved the feedback about points lost or gained in PIC Play. In the old program, the points were hidden in the top right of the UI, and the only time the user received any explicit feedback about gaining or losing points was after saving or losing a patient. The new PIC Play gives a popup after each patient encounter stating how many points the user gained, along with a short message such as ‘Good job! +10 points’ or ‘The patient is doing less well now’ (see figure 1.6).
1.2.7 Smart Play
In a parallel project, De Jong changed the learning model to be more heavily based on human cogni- tion: it is modeled in such a way that cases are offered when they are just about to dip below a recall threshold. By offering them just before this happens, learning should be more effective. In ad- dition, users can indicate their certainty about cer- tain alarm combinations. If they are very certain,
Figure 1.6: Feedback after patient treatment
they can double the consequence of their decision:
if they answer correctly, that particular case will occur later than if the user would have answered it ‘normally’, and the patient will be an extra step closer to being saved. However, answering incor- rectly with a double consequence also brings the patient one extra step closer to being ‘lost’, and the learning model will show the case more often than if the user would have answered it ‘normally’.
For a more in-depth look into this particular learn- ing model, the readed is referred to De Jong’s First Year Research Project paper (2016).
It should be noted that a large part of both this research and De Jong’s research was done in col- laboration, including but not limited to the new alarm icons, digitizing of the instructions, presen- tations of new cases and increased user feedback in the serious game version.
1.3 Experiment
This all brings us to the following research question:
”Does the use of a serious game yield better results than the use of a standard learning method in help- ing to find the most likely cause and treatment for certain combinations of patient monitor alarms in the pediatric intensive care?”. In the next section, we will describe the methods and procedures used during the experiment.
2 Method
2.1 Participants
32 participants participated in the experiment. 2 trials had to be excluded due to program crashes
during the training phase, invalidating their data.
30 participants remain in our analysis. Of these 30 participants, 15 were quasi-randomly assigned to the PIC Play condition (N=15, age range: 21-66, Agemean= 28.53), and 15 were quasi-randomly as- signed to the NO Play condition (N=15, age range:
19-62, Agemean = 26.86). Only participants with sufficient command of the Dutch language were al- lowed to participate. The participants had varying educational backgrounds and were compensated for their time. Informed consent was obtained before running the experiment.
2.2 Experimental Design
The study used a between-subject design. The test group used PIC Play during the training phase. The control group used NO Play during the training phase. By using a between-subjects design carry- over effects are avoided.
2.3 Materials and Procedure
Both ’PIC Play’ and ’No Play’ were created in Java.
Experiments took place at the university in a desig- nated experiment room. The room contained three cubicles. Each cubicle contained a desk with a Mac- book, chair, keyboard and mouse. Each experiment session had a maximum of three participants.
When a participant entered the room, they were given a short explanation about the experiment and assigned to either the control group or the test group quasi-randomly. The participants were not aware whether they were in the test group or the control group. All participants were given an explanation of the experiment (see appendix A).
Then they were asked to fill out a demographics form (age, gender, experience working in a hospital, and how often they play video games. See appendix B) and the informed consent form. Next, subjects were seated behind a computer to start the training phase. Depending on whether the participant was in the test group or the control group, they would train using ’PIC Play’ or ’NO Play’ respectively.
In either case, participants were given an instruc- tion sheet (see appendix G and appendix H for re- spectively PIC Play and NO Play) and an overview of terminology. The overview contained definitions of words not commonly used outside of a hospital
context such as ‘hypoxic’, ‘cardiac tamponade’ and
‘sedation’ (see appendix F).
When starting the training phase, the users re- ceived in-program instructions about the User In- terface and goal of the training phase. Addition- ally, the initial pool of cases they were supposed to learn was presented to them before the actual game started.
The training phase lasted 40 minutes or 40 cases, whichever came first. Once one of these conditions occurred the training phase was over, a pop-up ap- peared and the participants were asked to alert the experimenter by asking quietly.
After the training phase the participant was given a pen, an answering sheet and a test on pa- per about the cases they were supposed to have learned. They were then given eight minutes to take a written multiple-choice test, in which they were asked to select the most probable cause and treat- ment given certain alarms for all eight cases (see appendix C).
After making the test, participants were asked to hand in their answer sheet and fill in a short questionnaire about their training experience using PIC Play or NO Play, containing statements such as ‘I would like to learn this way more often’, ‘I felt motivated to keep on training’ and ‘I had trouble learning the material’ (See Appendices I and J).
2.4 Data Collection, Measures and Analysis
For every participant, demographics, test results, questionnaire results and training phase actions were collected. During the training phase, all ac- tions made by the users, such as going to the
‘treatment’ part of a problem or selecting an an- swer, were logged using timestamps. This enabled us to analyze user interaction. Actual answers dur- ing the training phase were logged to a semicolon- separated data file for later analysis in R. The ques- tionnaire used a Likert-scale from 1-5 in which the participants were asked to rate their experience us- ing PIC Play and NO Play in order to get to know more about users’ motivations and experience. Test results and questionnaire results were digitized.
Since both the training phase and the test phase used multiple-choice examination, the correct and incorrect answers had to be transformed into a
Table 3.1: PIC Play and NO Play test score means, medians and standard deviations.
Condition Mean Std dev. Median
PIC Play 7.33 3.02 8.33
NO Play 7.45 2.05 6.67
grade. We did this using formula 2.1 G = ((Qc+ Qt) − T ∗ 0.25) ∗ 10
T ∗ 0.75 (2.1) where G is the grade, Qc is the number of cor- rect causes, Qtis the number of correct treatments and T is the total number of questions (16, since we have 8 causes and 8 treatments). Each question had four options, therefore we have to compensate for the 25% of cases in which the answer might be guessed correctly. This number is then subtracted from all correct answers, and the end grade is mul- tiplied by the grade variable (T ∗0.7510 ), resulting in a grade from 0 - 10.
3 Results
3.1 Test Results
The data distribution is visualized in a boxplot form in figure 3.1. Data distribution in table for- mat can be found in table 3.1.
A Shapiro-Wilk normality test showed the PIC Play and NO Play test grades to be normally dis- tributed (w = 0.88, p = 0.00). An unpaired Two- sample t-test was performed using the PIC Play and NO Play test grades resulting from the experi- ment. There was no significant difference in test re- sults between the two groups (t = 0.12, df = 24.65, p = 0.91).
The test results indicate that there might be a ceiling effect, especially in the PIC Play condition.
It is worth noting that, although their means are very close, the medians of PIC Play and NO Play differ substantially.
There was no significant correlation between time spent during the training phase and test score.
No correlation was found between the number of rehearsals of a case during the training phase and performance on that case during the test. Figure 3.3 shows the frequency of total answers correct on the test for both PIC Play and NO Play conditions.
NO Play PIC Play
0246810
Test Score by Condition
Condition
Score
Figure 3.1: Boxplot of PIC Play vs. NO Play test scores.
The overall percentage of (sub-)questions correct during the training phase for both conditions can be found in figure 3.4.
3.2 Questionnaire Results
The questionnaire (figure 3.2) shows some insight into the (self-reported) experience participants had during the experiment. There was no significant dif- ference between NO Play users’ and PIC Play users’
self-reported motivation to keep on training (t = - 0.13, df = 26.93, p = 0.90).
There was a moderate Spearman correlation (0.62) between motivation and test score for PIC Play, whereas there was a weak negative Spear- man correlation (-0.28) between motivation and test score for NO play.
Overall, three strong positive (0.70+ Spearman correlation) correlations were found: I enjoyed the training — I would like to train using a similar method more often; I enjoyed the training — I felt motivated to keep on training; I thought the train- ing was challenging — I felt motivated to keep on training.
20%
13%
67%
73%
13%
13%
13%
7%
53%
57%
33%
36%
20%
36%
53%
57%
27%
7%
40%
57%
27%
21%
33%
21%
31%
7%
62%
71%
8%
21%
20%
21%
60%
57%
20%
21%
50%
36%
43%
50%
7%
14%
13%
13%
73%
80%
13%
7%
21%
33%
57%
67%
21%
0%
I enjoyed the training
I feel like I know more about the material than before the training
I felt motivated to keep on training
I found it difficult to learn the material
I had enough time during the training
I really tried my best to understand the material
I thought the training was challenging
It was clear what I had to do during the training
I would like to learn in a similar way more often NOPLAY
PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
NOPLAY PICPLAY
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
PIC Play vs NO Play Questionnaire Results
Figure 3.2: Questionnaire results
PIC Play Test Answers Correct
# of Correct Answers
Frequency
6 8 10 12 14 16
012345
NO Play Test Answers Correct
# of Correct Answers
Frequency
6 8 10 12 14 16
012345
Figure 3.3: Frequency of correct answers on the test for PIC Play and NO Play
C1 T1 C2 T2 C3 T3 C4 T4 C5 T5 C6 T6 C7 T7 C8 T8
Overall Percentage of Correct Answers During Test
Test Question
Percentage Correct
0 20 40 60 80 100
Figure 3.4: Performance per question during the test; The blue bars represent ‘cause’ parts of the question. Red bars represent ‘treatment’ parts of the question.
4 Discussion
4.1 General discussion
Going back to our hypothesis, “Does the use of a serious game yield better results than the use of a standard learning method in helping to find the most likely cause and treatment for certain combi- nations of patient monitor alarms in the pediatric intensive care? ”, we conclude that the use of a se- rious game does not yield better results in helping to find the most likely cause and treatment for cer- tain combinations of patient monitor alarms in the pediatric care.
Our data suggests that there is a small to no correlation between the number of rehearsals dur- ing the training period and the user’s performance on that case during the test. This could be because there is no effect, or because cases people find dif- ficult (and thus more often answer wrong) appear more often. This means harder cases will have more rehearsals, but not necessarily better performance during the test.
There were some notable differences between our versions of PIC Play and NO Play and Van der Meulen’s versions of PIC Play and NO Play, and the way the experiments were run:
The sample size was almost doubled, giving our tests a higher statistical power; The questionnaire provided more insight into participants’ strategies, experience and motivations; Average scores for our versions of NO Play and PIC Play were higher across the board. It is hard to attribute this to a single factor, since many factors may have had an effect:
- The database used in our experiment was slightly smaller than the database used by Van der Meulen (8 items vs. 10 items in the datase), making it easier to learn the material. In par- ticular, the cases which were removed were cases that had duplicate treatment options, i.e.: the original PIC Play database had mul- tiple cases of which the correct treatment was
“Check the patient”. Removing the duplicates made the database easier to learn, since all causes and treatments now consisted of unique options.
- A more responsive learning algorithm was used, which may have increased participants’
performance by showing appropriate cases de- pending on user performance instead showing them random cases.
- The presentations in the training phase may have helped the participants with learning the material by allowing them to see the material before they have to answer questions about it.
- The increased feedback in the PIC Play condi- tion about points gained and/or lost may have shifted the participants’ attention to the point aspect of the game, which in turn could have motivated them to score more points (and thus perform better).
In conclusion, we can note that performance was higher overall compared to Van der Meulen’s ver- sion. However, it is unclear to which factor(s) we can attribute this improvement.
One thing that was noted in the questionnaire among participants in both conditions was the fact that using pattern-matching rather than trying to learn the underlying cause-effect was quite effec- tive. As mentioned before, this is possible due to the database consisting of only unique causes and treatments. When learning new material, it is im- portant to make connections and try to understand the material. In this (pediatric intensive care) con- text, understanding the material rather than re- membering is very important since it may make the difference between life and death. As such, it may be important to make users unable to depend on simple pattern-matching in order to force them to learn the underlying relations.
Participants in both conditions reported that they enjoyed training and felt motivated to keep on training to roughly the same extent. How- ever, more participants in the PIC Play condition agreed strongly with these statements than those in the NO Play condition. This could indicate that the gamification elements of PIC Play elicited a stronger positive response to the questions.
PIC Play seemed to have a ceiling effect: 33%
of all participants in the PIC Play condition had a perfect score (16 out of 16 questions correct dur- ing the test). Although having a perfect score may seem great looking at it from a learning perspec- tive, it indicates that the database may have been too small or too easy for these participants. This
brings us to another point: participants’ learning abilities.
For future work, it would be very interesting to test the effect of Serious Games when applied to people who have a harder time learning new mate- rial. Often people who have more difficulties learn- ing new material lack motivation and do not enjoy learning. A serious game environment could alle- viate some of these obstacles: the questionnaire of the experiment showed strong correlations between
“I would like to train this way more often” and
“I enjoyed the training”. There was also a strong correlation between “I enjoyed the training” and
“I was motivated to keep on training”. A serious game may increase a users’ enjoyment while train- ing, which might cause them to train for longer pe- riods of time or train more often than they other- wise would have.
Aside from the motivational aspect, the learn- ing algorithm could also help: the organizational aspects of learning are automated since users no longer have to keep track of which questions they are good or bad at, and they do not have to deter- mine when to add new items or revisit old items.
One point of consideration for future research would be the participant sample: our participants were all either students or ex-students, although they differed greatly in discipline and age. Ideally the participant sample would consist of participants from the same discipline (preferably the same study and academic year). This way effects of age and education (and possibly prior knowledge) can be avoided.
4.2 SMART Play
In a parallel project, De Jong compared our version of PIC Play to SMART Play, a serious game similar to PIC Play with a different learning algorithm and the ability for its users to indicate how certain they are regarding their answer. The results of De Jong’s research that are available at the moment of writing indicate no significant difference between SMART Play and PIC Play test scores and no significant difference between SMART Play and NO Play test scores.
4.3 Real world applications
As mentioned in the introduction, serious gam- ing for training medical professionals is becoming more and more popular. However, developing a se- rious game is usually a costly affair. Serious games may cost from $9.000 upwards to $900.000, with development costs typically hovering around the
$40.000 to $60.000 range.∗It may seem hard to jus- tify these costs when the existing learning methods perform well enough. However, especially for those who struggle learning, serious games may be a cost- effective way to learn certain materials. Although costs may be too high for a single classroom, a par- ticular serious game could be used in many different classes covering the same subject matter across the country (or world). Plus, serious games could be used outside of a classroom environment, be it at home or while in transit to or from work or school on (for example) a smartphone. If the serious game is fun enough, people will start learning (while us- ing the game) on their own accord.
Often, serious games can be a substitution for real-life scenarios that can be hard to practice in real life, such as total knee replacement surgery procedures. Traditional simulators are much more expensive to develop than serious games, and more expensive to deploy since each simulator uses cus- tom software-hardware combinations (Ricciardi et al., 2014). In these cases, serious games can offer a cheaper and more easily deployable alternative, since they are not built for very specific hardware.
All in all, serious games have a lot to offer, and while they might not be the answer to all problems, they have a lot of potential as an educational tool.
References
Brinks, K. (2015). Alarm hazards on the pediatric intensive care unit. Master’s thesis, University of Groningen, Groningen.
De Jong, A. (2016). Smart play: [title tbd]. First year research project, University of Groningen, Groningen.
∗Although finding exact numbers in research papers proved difficult, a short analysis of information found on- line provided this broad estimate.
Drenthen, A. (2015). Effective learning with an ap- plication for summary maps and flashcards. Mas- ter’s thesis, University of Groningen, Groningen.
Graafland, M., Schraagen, J. M., and Schijven, M. P. (2012). Systematic review of serious games for medical education and surgical skills training.
British Journal of Surgery, 99(10):1322–1330.
Lawless, S. T. (1994). Crying wolf. Critical Care Medicine, 22(6):981985.
Ricciardi, F. and Paolis, L. T. D. (2014). A compre- hensive review of serious games in health profes- sions. International Journal of Computer Games Technology.
Van der Meulen, P. (2016). Pic play: A serious game for alarms in the paediatric intensive care. Bache- lor’s thesis, University of Groningen, Groningen.
Wang, R., Demaria, S., Goldberg, A., and Katz, D.
(2016). A systematic review of serious games in training health care professionals. Simulation in Healthcare: The Journal of the Society for Sim- ulation in Healthcare, 11(1):4151.
Welkom! Leuk dat u mee wilt doen aan het experiment.
Het experiment gaat over het leren van de oorzaak en bijbehorende behandeling op basis van verschillende alarmen die af kunnen gaan op de kinder-Intensive Care.
Zometeen krijgt u eerst twee formulieren: het 'geïnformeerde toestemming’ formulier en een formulier voor persoonsgegevens. Nadat u beide formulieren heeft ingevuld mag u ze teruggeven aan mij en krijgt u een computer toegewezen. Vervolgens vindt de trainingsfase van het experiment plaats.
Tijdens de trainingsfase is het de bedoeling om te leren welke alarmen welke oorzaak en bijbehoren behandeling hebben. Na 40 minuten is de trainingsfase afgelopen en wordt het programma afgesloten.
Vervolgens komt de testfase. In de testfase krijgt u een een meerkeuzetoets over de stof die u zojuist (hopelijk) heeft geleerd.
Mocht u vragen hebben over het onderzoek, dan kunt u deze nu stellen. Mocht u achteraf nog vragen hebben, dan kunt u contact opnemen met de onderzoeker:
s.warmelink@student.rug.nl
APPENDIX A: Verbal instructions experiment
Proefpersooninformatie Datum: Pp. Nr:
Leeftijd:
Geslacht:
Opleiding:
Studiejaar:
Heb je ooit in een ziekenhuis gewerkt of stage gelopen? Ja/Nee Zo ja, heb je ooit op de Intensive Care gewerkt of stage gelopen?
Speel je wel eens computerspelletjes (games)? Ja/Nee Zo ja, hoeveel uur per week speel je ongeveer? ___uur
APPENDIX B: General demographics questionnaire
1. Sat Low
Oorzaak:
A: Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
B: Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope middelen.
C: De patiënt heeft mogelijk meer zuurstof nodig.
D: De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
Behandeling:
A: Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
B: Check de patiënt.
C: Check de machine en de patiënt.
D: Verlaag de eventuele hersenzwelling met medicijnen, of door in het geval van een drain deze af te laten lopen.
APPENDIX C: Multiple choice test
2. HR High, BP High
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
B: Mogelijk is er sprake van een algemene verbetering.
C: De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te diep.
D: Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope middelen.
Behandeling:
A: Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd.
B: Check de patiënt.
C: Check de machine en de patiënt.
D: Check of de patiënt moet hoesten.
APPENDIX C: Multiple choice test (cont.)
3. HR Low, BP Low
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie of een beademingsprobleem.
B: De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
C: De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te diep.
D: Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
Behandeling:
A: Check de patiënt en check de machine voor hartdrukmetingen. Indien pacemaker, check of deze nog goed werkt.
B: Check of de patiënt moet hoesten.
C: Verlaag de eventuele hersenzwelling met medicijnen. Indien beademingsprobleem, check of de patiënt moet hoesten.
D: Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
APPENDIX C: Multiple choice test (cont.)
4. HR Low, BP High
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
B: De patiënt heeft mogelijk last van inklemming of een vagale reactie of een beademingsprobleem.
C: Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope middelen.
D: De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te diep.
Behandeling:
A: Check de patiënt.
B: Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd.
C: Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
D: Verlaag de eventuele hersenzwelling met medicijnen, of door in het geval van een drain deze af te laten lopen.
APPENDIX C: Multiple choice test (cont.)
5. HR Low, Sat Low
Oorzaak:
A: De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te diep.
B: De patiënt heeft mogelijk last van sputum.
C: Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
D: De patiënt heeft mogelijk meer zuurstof nodig.
Behandeling:
A: Check of de patiënt moet hoesten.
B: Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd.
C: Check de machine en de patiënt.
D: Check de patiënt.
APPENDIX C: Multiple choice test (cont.)
6. HR Low, BP High, Sat Low
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie of een beademingsprobleem.
B: De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te diep.
C: De patiënt heeft mogelijk meer zuurstof nodig.
D: Mogelijk is er sprake van een algemene verbetering.
Behandeling:
A: Verlaag de eventuele hersenzwelling met medicijnen, of door in het geval van een drain deze af te laten lopen.
B: Check de patiënt en check de machine voor hartdrukmetingen. Indien pacemaker, check of deze nog goed werkt.
C: Verlaag de eventuele hersenzwelling met medicijnen. Indien beademingsprobleem, check of de patiënt moet hoesten.
D: Check de patiënt.
APPENDIX C: Multiple choice test (cont.)
7. HR Low, BP High, Sat High
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
B: Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope middelen.
C: De patiënt heeft mogelijk last van sputum.
D: Mogelijk is er sprake van een algemene verbetering.
Behandeling:
A: Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
B: Check of de patiënt moet hoesten.
C: Verlaag de eventuele hersenzwelling met medicijnen, of door in het geval van een drain deze af te laten lopen.
D: Verlaag de eventuele hersenzwelling met medicijnen. Indien beademingsprobleem, check of de patiënt moet hoesten.
APPENDIX C: Multiple choice test (cont.)
8. HR High, Sat High
Oorzaak:
A: De patiënt heeft mogelijk last van inklemming of een vagale reactie of een beademingsprobleem.
B: Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
C: Mogelijk is er sprake van een algemene verbetering.
D: Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope middelen.
Behandeling:
A: Check de machine en de patiënt.
B: Verlaag de eventuele hersenzwelling met medicijnen. Indien beademingsprobleem, check of de patiënt moet hoesten.
C: Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd.
D: Check de patiënt en check de machine voor hartdrukmetingen. Indien pacemaker, check of deze nog goed werkt.
APPENDIX C: Multiple choice test (cont.)
Nr Alarmen Oorzaak Behandeling
1 Sat Low De patiënt heeft mogelijk meer zuurstof nodig. Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
2 Sat High De patiënt heeft mogelijk minder zuurstof nodig. Check de machine en de patiënt.
3Sat Low HR Low
De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze
mogelijk niet toegankelijk of zit deze te diep. Check de machine en de patiënt.
4 Sat Low HR High
De patiënt is mogelijk onrustig of heeft pijn, krijgt niet genoeg medicatie, of heeft beademingsproblemen door
sputum. Check de patiënt.
5Sat Low BP Low
De patiënt heeft mogelijk oversedatie, een
ademhalingsprobleem, of er is sprake van een algemene
verslechtering. Check de patiënt.
6 Sat Low BP High
De patiënt heeft mogelijk ondersedatie, een
ademhalingsprobleem, of er is sprake van een algemene
verslechtering. Check de patiënt.
7 Sat High
HR High Mogelijk is er sprake van een algemene verbetering.
Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd.
8Sat High
BP High Mogelijk is er sprake van een algemene verbetering. Check of het zuurstofpercentage of de beademingsdrukken moeten worden afgebouwd. Check of de bloeddruk reëel is.
9 HR High BP High
Er is mogelijk sprake van ondersedatie, de patiënt heeft
koorts, of de patiënt heeft te veel inotrope middelen. Check de patiënt.
10 HR High
BP Low Er kan sprake zijn van ondervulling (een vochttekort).
Geef medicijnen zodat de bloeddruk weer op een goed niveau komt. Onderzoek de patiënt voor een volledige diagnose, check of deze een hartritmestoornis heeft.
11HR Low
BP High De patiënt heeft mogelijk last van inklemming of een
vagale reactie. Er kan ook een drain zijn. Verlaag de eventuele hersenzwelling met medicijnen, of in het geval van een drain door deze af te laten lopen.
12 HR Low BP Low
Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
Check de patiënt en check de machine voor hartdrukmetingen.
Indien er een pacemaker is, check of deze nog goed werkt.
13 Sat High HR High BP High
Er is mogelijk sprake van ondersedatie, de patiënt heeft
koorts, of de patiënt heeft te veel inotrope middelen. Check de patiënt.
14 Sat High HR High BP Low
Er is mogelijk een probleem met de inotrope medicatie of
een circulatie-probleem. Check de patiënt.
15 Sat High HR Low
BP High De patiënt heeft mogelijk last van sputum. Check of de patiënt moet hoesten.
16 Sat High HR Low
BP Low De inotrope medicatie loopt mogelijk niet goed, of er is
oversedatie. Check de patiënt.
17 Sat Low HR High BP High
De patiënt heeft mogelijk stress, pijn of verdriet. De oorzaak hiervan kan ondersedatie of een
beademingsprobleem zijn. Check de patiënt.
18 Sat Low HR High
BP Low De patiënt is mogelijk in shock. Check de patiënt en de beademingsmachine, check of de inotrope middelen aankomen.
19 Sat Low HR Low
BP High De patiënt heeft mogelijk last van inklemming, een vagale
reactie of een beademingsprobleem. Verlaag de eventuele hersenzwelling met medicijnen. Indien er een beademingsprobleem is, check of de patiënt moet hoesten.
20 Sat Low HR Low BP Low
Er is mogelijk een circulatoir of ventilatoir probleem. Het kan ook een probleem met inotrope middelen zijn. Dit kan veroorzaakt worden door een verstopping van de tube.
Check de patiënt en neem de oorzaak van het probleem weg. Geef meer zuurstof, verhoog de beademing wanneer nodig, verhoog de inotropie, check de tubes. Indien dit niet helpt, start reanimatie.
APPENDIX D: Knowledge Base developed by Brinks
Nr Alarmen Oorzaak Behandeling
1 Sat Low De patiënt heeft mogelijk meer zuurstof nodig. Check de machine en de patiënt. Als het een cardio patiënt betreft, check of de shunt dicht zit.
2
Sat Low HR Low
De patiënt heeft mogelijk meer zuurstof nodig, en kan hypoxisch zijn. Als de patiënt een tube heeft is deze mogelijk niet toegankelijk of zit deze te
diep. Check de machine en de patiënt.
3
Sat High HR High
Mogelijk is er sprake van een algemene verbetering.
Check of het zuurstofpercentage of de
beademingsdrukken moeten worden afgebouwd.
4
HR High BP High
Er is mogelijk sprake van ondersedatie, de patiënt heeft koorts, of de patiënt heeft te veel inotrope
middelen. Check de patiënt.
5
HR Low BP High
De patiënt heeft mogelijk last van inklemming of een vagale reactie. Er kan ook een drain zijn.
Verlaag de eventuele hersenzwelling met medicijnen, of in het geval van een drain door deze af te laten lopen.
6 HR Low BP Low
Mogelijk treedt een harttamponade op, is er sprake van oversedatie of is er een verslechtering van de patiënt.
Check de patiënt en check de machine voor
hartdrukmetingen. Indien er een pacemaker is, check of deze nog goed werkt.
7
Sat High HR Low
BP High De patiënt heeft mogelijk last van sputum. Check of de patiënt moet hoesten.
8
Sat Low HR Low BP High
De patiënt heeft mogelijk last van inklemming, een vagale reactie of een beademingsprobleem.
Verlaag de eventuele hersenzwelling met medicijnen.
Indien er een beademingsprobleem is, check of de patiënt moet hoesten.
APPENDIX E: Knowledge Base as used in experiments
Begrippenlijst
Tijdens het spel krijg je te maken met drie verschillende alarmen:
- HR: Heart Rate (hartslagfrequentie).
- BP: Blood Pressure (bloeddruk).
- Sat: Saturation (de zuurstofsaturatie van het bloed. Kan worden gezien als de effictiviteit van de ademhaling van de patiënt).
In het spel kunnen termen voorkomen waarmee je niet bekend bent. De meest voorkomende zijn:
- Drain: wordt gebruikt om wondvocht af te voeren.
- Hypoxisch: er is sprake van te weinig O2 (zuurstof).
- Inklemming: uitzetting van de hersenen.
- Inotrope middelen: zorgen ervoor dat de bloeddruk stijgt - Sedatie: verlagen van bewustzijn; in slaap brengen.
- Shunt: wordt gebruikt om een dialysemachine aan te sluiten op de bloedbaan - Sputum: slijm vermengd met speeksel dat vanuit de luchtwegen wordt opgehoest.
- Tube: beademingsbuis
- Vagale reactie: overdreven werking van een deel van het zenuwstelsel. Zorgt voor verwijding van bloedvaten, daling van hartslagfrequentie en vernauwing van de luchtwegen.
- Harttamponade: vocht of bloed tussen de hartspier en het hartzakje, waardoor de hartspier wordt belemmerd om te kunnen pompen. Zorgt voor kortademigheid, hoge hartslagfrequentie en lage bloeddruk.
APPENDIX F: Glossary of Terms, used during training and
test
Instructies PIC Play
Het is de bedoeling om spelenderwijs te leren wat de meest voor de hand liggende oorzaak en behandeling is van bepaalde alarmen op de kinder-IC.
Het doel van het spel is om zoveel mogelijk punten te halen. Punten halen doe je door goede diagnoses te stellen en de juiste behandelingen te kiezen. Maar let op: je kunt ook punten verliezen door elk keer een verkeerde diagnose te stellen of de verkeerde behandeling te kiezen!
Het spel is afgelopen zodra je 40 patiënten hebt behandeld of 40 minuten hebt gespeeld.
Werk zo accuraat mogelijk en neem je tijd!
APPENDIX G: PIC Play instructions
Hierboven is het overzicht van het spel te zien. De kinder-IC bestaat uit zes bedden waarop een patiënt kan liggen. Naarmate de tijd verstrijkt zul je nieuwe patiënten zien verschijnen. Patiënten die net binnenkomen op de kinder-IC hebben altijd een "probleem", patiënten die al op de IC liggen kunnen na verloop van tijd een nieuw probleem krijgen.
Wanneer je drie keer een patiënt goed hebt behandeld, blijft deze leven en krijg jij 25 bonuspunten! Maar als je bij dezelfde patiënt drie keer een verkeerde behandeling of diagnose hebt geselecteerd is de patiënt verloren en krijg je 10 minpunten.
Is er iets mis met een patiënt, dan brandt er een rood lampje op de monitor naast het bed van de desbetreffende patint. Nadat je de pati ënt behandeld hebt gaat het lampje weer uit.
Aan weerszijden van de Intensive Care staan speciale quarantaine-kamers. Bij deze kamers is het zicht, net zoals op de Intensive Care, beperkt. Dit houdt in dat er geen zicht is op de kamer, tenzij de verpleegkundige naast het bed staat of aan het bureau zit.
Als de verpleegkundige geen zicht heeft op de kamers kun je ook niet zien of er iets met de patiënt aan de hand is, dus vergeet niet om regelmatig deze kamers te checken!
Tijdens het spel wordt een aantal zaken weergegeven:
- De score, die je verhoogt door goede antwoorden te geven en patiënten te redden.
- Het aantal patiënten dat je hebt gered of bent verloren.
- De tijd.
- Het aantal problemen dat je hebt behandeld.
Mocht je het spel willen pauzeren, dan kun je linksboven op de pauze-knop drukken.
APPENDIX G: PIC Play instructions (cont.)
INSTRUCTIES NOPLAY
Het is de bedoeling te leren wat de meest voor de hand liggende oorzaak en behandeling is van bepaalde alarmen op de kinder-IC.
Het onderdeel is afgelopen zodra je 40 problemen hebt gehad of langer dan 40 minuten bezig bent geweest. Werk zo accuraat mogelijk en neem je tijd!
APPENDIX H: NO Play Training instructions
Evaluatievragen
In hoeverre ben je het eens of oneens met de volgende stellingen?
Sterk
mee oneens
Neutraal
Sterk mee eens
Ik vond de training leuk om te doen.
o o o o o
Ik zou vaker op een soortgelijke manier willen leren.
o o o o o
Ik vond het duidelijk wat ik tijdens de training moest
doen.
o o o o o
Ik heb het idee dat ik de stof beter beheers dan voor de
training.
o o o o o
Ik had voldoende tijd om de stof te leren.
o o o o o
Ik had moeite met het leren van de stof.
o o o o o
Ik heb mijn best gedaan om de stof zo goed mogelijk te
leren.
o o o o o
Ik vond de training uitdagend.
o o o o o
Ik voelde me gemotiveerd om de training te blijven
doen.
o o o o o
Heb je nog andere opmerkingen of suggesties?
Gebruik bij te weinig ruimte de achterkant van het formulier.
APPENDIX I: NO Play questionnaire
Evaluatievragen
In hoeverre ben je het eens of oneens met de volgende stellingen?
Sterk mee
oneens Neutraal
Sterk mee eens
Ik vond het spel leuk om te spelen.
o o o o o
Ik zou vaker op een soortgelijke manier willen leren.
o o o o o
Ik vond het duidelijk wat ik tijdens het spel moest doen.
o o o o o
Ik heb het idee dat ik de stof beter beheers dan voor het
spelen van het spel.
o o o o o
Ik had voldoende tijd om de stof te leren.
o o o o o
Ik had moeite met het leren van de stof.
o o o o o
Ik heb mijn best gedaan om de stof zo goed mogelijk te
leren.
o o o o o
Ik vond het spel uitdagend.
o o o o o
Ik voelde me gemotiveerd om het spel te blijven spelen.
o o o o o
Ik probeerde zoveel mogelijk punten te halen.
o o o o o
Ik probeerde zoveel mogelijk patiënten te redden.
o o o o o
Heb je nog andere opmerkingen of suggesties?
Gebruik bij te weinig ruimte de achterkant van het formulier.
PP nr:
