The attitude of Dutch medical students toward the use of serious games in medical education

(1)

The attitude of Dutch medical students

toward the use of serious games in

medical education

Bachelor Thesis Information Science

2019-2020

Lucas Berkvens 11000996 Supervisor: Jacobijn Sandberg

(2)

Abstract

The aim of this study was to chart the attitude of Dutch medical students towards the use of serious games in medical education. A survey was sent out to Dutch medical students at the UvA. They were asked questions about their general opinion on the use of new technology as an educative tool, their previous experience with serious games, and a list of statements regarding the potential motivation to play serious games was presented. The results showed that most of the respondents did not have prior experience with playing serious games, but their opinion on the use of serious games in medical education was very positive. The general opinion of respondents with prior experience in playing serious games was significantly higher than respondents with no experience at all. There was no difference between male and female respondents. From these results it can be concluded that Dutch medical students have a positive attitude towards the use of serious games in medical education.

(3)

1. Introduction

1.1 General introduction

With the exponential growth of new technologies, the way people carry out everyday tasks is changing a lot. One of the things that is undergoing a lot of change is the education system. For centuries people were only educated from books and lecturing, but nowadays all sorts of new technological developments provide opportunities for new ways of education. According to Raja & Nagasubramani (2018) the main advantage of using new technologies in education is that the use of technology makes students more excited to learn.

One of the most recent technological developments in education is the use of ‘serious games’ in learning.

1.2 Definition

To get a better understanding of this subject, it’s important to determine the definition of the concept ‘serious game’, as used in this paper. This does not seem to be an easy task because a lot of different definitions are described in existing literature. What most of these definitions do agree on is that serious games are (digital) games used for purposes other than mere entertainment (Susi et al., 2007). A lot of different purposes can be fulfilled with the use of serious games, for example training militants, helping in curing addictions and using them to educate people. In this paper the term serious game indicates a game that is especially designed for the use in educational purposes.

1.3 The history of serious games

The first time the term serious game as we know it now, seems to be used in literature in the book “Serious Games”, written by Clark Abt (1970). Abt was a researcher during the cold war, who wanted to use games for training and educational purposes. In his book Abt talks about digital games and non-digital games, and states that: ‘serious games have an explicit and carefully thought-out educational purpose and are not intended to be played primarily for amusement. This does not mean that serious games are not, or should not be, entertaining.’

(5)

1.4 Serious games in medical education

One of the professional fields that is already making a lot of use of serious games is medical education. According to Zhonggen, the wide use of serious games in medical education can be explained by their positive outcomes among participants. Serious games in medical education have the potential to be the next revolutionary innovation on the existing market for education tools. The main reason for this is that these games allow for a very wide accessibility and reduced operating costs, while they enjoy a lot of the same advantages of other simulation forms. They can for example be developed for specific learning objectives and they enhance patient safety as you do not need an actual human being to perform actions on, but a simulated patient in a virtual world (Wang et al., 2016).

(6)

2. Theoretical background

2.1 Recent studies on serious games

According to a recent study on serious games by Zhonggen, Y. (2019), the number of publications on serious games in education has risen a lot over the past decade, as shown in this figure:

Figure 1. Histogram of publications related to serious games over the years 2009-2018. Reprinted from “Medical student attitudes toward video games and related new media technologies in medical education” by Kron, F. W., Gjerde, C. L., Sen, A., & Fetters, M. D., 2010, BMC medical

Zhonggen systematically reviewed 792 publications about serious games in the past decade, and concluded that the majority of these publications found a lot of positive effects on the use of serious games in education. However, also a number of negative effects were found, and it was concluded that the use of serious games in education is a very complicated issue. A lot of future research on the positive and negative effects must be done to get a good picture on the overall effects of the use of serious games in education.

2.2 Serious games versus traditional learning

Two factors of serious games that are assumed to have an influence on learning are ‘affecting motivation’ and ‘changing cognitive processes’. Wouters et al.

(7)

(2012) used meta-analytic techniques to investigate if serious games are better in terms of motivating and learning than conventional methods. Their research showed that serious games did in fact have a positive effect on learning and retention compared to traditional learning techniques. There was evidence that learners who played serious games learned more. They did not find evidence that serious games were more motivating than traditional learning techniques.

According to a study conducted by Chung et al. (2019), the most important factor a serious game needs to have in order to make it attractive for students to use is performance expectancy. Performance expectancy refers to the degree to which an individual perceives that using a system will help him or her to attain a gain in job performance (Venkatesh et al., 2003).

2.3 Studies on serious games in medical education

In a study by Ricciardi & De Paolis (2014), serious games developed for health professions were reviewed in order to understand if they were useful tools for medical training. The serious games they reviewed were grouped by application area. The groups they distinguished were: surgery, odontology, nursing, cardiology, first aid, dietitian and diabetes and psychology. Their results showed that the field with the highest number of developed serious games was first aid. This can be explained by the fact that continuous training here is very important, and can be achieved by serious games with lower costs. Out of all the scientific papers they reviewed where the results of evaluation studies were reported, all of them had a positive outcome. The users who practiced the serious games all had better results than users that used experienced traditional learning processes.

To get better understanding of what these serious games are exactly, two examples of serious games in medical education are described.

Surgery

A serious game designed to simulate a knee replacement surgery was created by Sabri et al. (2010). This game allowed players to get a clear understanding of the procedure of the operation and the surgical decision making processes.

(8)

First aid

Triage is an interactive serious game designed by Kurenov et al. (2009). The game simulates a real-life emergency event involving 40 burn victims in a theme park. The goal is to correctly stabilize, sort, tag and transport the burn victims. If done correctly the player gets the role of a burn care provider with access to computer-simulated hospital devices to provide clinical needs to the victims.

Both of these games provide scenarios which are hard to simulate without the use of technology. For many other medical implications, serious games can be a serious replacement for traditional learning processes because of their realistic simulation and cost-reduction possibilities.

2.4 Studies on the attitude of medical students towards serious games A study by Johnsen et al. (2018), a serious game was used in simulation courses in nursing education. The game was developed to teach the nursing students clinical reasoning and decision-making skills. Overall the students that had followed the simulation courses thought the serious games they played had educational value and were easy to use. There was no evidence found that there was a difference between nursing students with no prior experience in healthcare and students that did have experience. The students that participated in the study thought serious game simulations should be used more in nursing education.

In another study among 217 American healthcare students showed that 98% liked the idea of using technology to enhance healthcare education, and 80% believed that video games can have educational value (Kron et al., 2010). It is stated that duplicating this work in other countries is needed to determine if their findings generalize to medical students with different ethnicities and cultural backgrounds. The aim of this paper is to see what the attitude of Dutch medical students is toward the use of serious games in medical education. This raises the following research-question: “What is the attitude of Dutch medical students toward the use of serious games in medical education?”

(9)

2.5 Hypothesis

There is strong evidence that serious games can have a positive influence on education. Studies have shown that students were able to learn easier with the use of serious games than with traditional educational tools. Researchers also found that students who participated in studies with serious game simulations were positive on the use and effect of them. In the gaming world, there are generally more male players than female players. This might also have an effect on the stance toward the use of serious games in education.

For this research the following three hypotheses were created.

Hypothesis 1

Students will have a positive stance towards the use of new technologies, including serious games, in education. The mean of their scores will be higher than neutral (higher than a score of 3 on the 1-5 Likert-scale).

Hypothesis 2

Students with experience in serious games will have an even significantly more positive stance towards the use of new technologies, including serious games, in education than students with no experience.

Hypothesis 3

Male respondents will have a more positive stance toward the use of serious games than female respondents.

(10)

3. Methodology

In order to investigate the attitude of Dutch medical students toward serious games as an educational tool, a survey was sent out to medical students at the UvA (University of Amsterdam). This survey contained a set of questions regarding the use of serious gaming in medical education, similar to some of the questions from the survey used in the study by Kron et al.. The relevant questions in that survey were then translated into Dutch, and a Likert-scale was applied to get a better insight in the attitude of the students.

3.1 Participants

One week after the survey was sent out, twenty-eight medical students from the UvA filled out the survey (22 male and 6 female). Their ages ranged from 21-29 with an average age of 24.2. They were approached personally in real life or through whatsapp- or facebook-groups especially for medical students at the UvA. The answers were all provided anonymously.

3.2 Design

The study reported here is a survey study. The survey contained statements about the general opinion and motivation to play serious games. The independent variables in this survey are ‘experience with serious games’ and ‘gender’. The rest of the variables are the dependent variables.

3.3 Materials

The survey contained questions and statements regarding the familiarity with computers and (serious) gaming, the degree to which the participants agreed to make more use of new technologies in education and certain questions about the motivation to use serious games as a learning mechanism in the medical education. These questions would later be grouped into new variables to perform tests on.

The survey was divided into four sections, each with a different set of questions to get a better insight in the attitude of the students.

The first section started off with an introduction. After that the first two questions were about the age and gender of the respondents to get a little insight in the demographics.

(11)

1. What is your age? 2. What is your gender?

The next two sections made use of a Likert-scale. The respondents had to choose a number from 1 to 5 to indicate whether they agreed or not with the following statements (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree).

The second section existed of some general questions. 3. I have experience with computers

4. I have experience with gaming

5. I have experience with playing serious games

6. Are serious games being used in your current education?

The third section existed of some more in-depth questions about the motivation for the use of serious games in education.

7. I like the idea of using technology to improve the quality of education 8. I think that education should make better use of new technologies 9. I feel that video games can have educational value

10. If serious games were proven to work, I would play them in my free time

11. Serious games would only be interesting to me if I could really learn how to perform physical operations

12. In my free time, I would rather play a serious game than read a textbook

13. I think serious games could help me to obtain factual knowledge 14. Serious games would be interesting for me if they could help me

acquire skills needed for good interaction with patients

15. Serious games would be interesting for me if they could help me acquire skills needed to perform medical operations

16. Serious games would be interesting for me if they could help me acquire communicative skills that are needed to maintain contact with colleagues

(12)

17. Serious games would be interesting for me if they could help me acquire factual knowledge.

In the fourth, and last section, five statements were shown again, and the respondents were asked to arrange them from most important to least important. These questions were created to get a clear view on the most important aspects of serious games for these students, and to see if the results would be consistent with the previous answers.

1 Enjoyment in the game 2 Acquiring factual knowledge

3 Acquiring skills needed to perform medical operations

4 Acquiring communicative skills needed to maintain contact with colleagues 5 Acquiring skills needed for good interaction with patients

To create the survey, google forms was used. The reason behind this, is that google forms has a very convenient design to create survey questions and sections, and the data can then be converted into a csv file which you can easily analyze later.

The survey was written in Dutch, because it was sent out to Dutch medical students at the UvA only. In this section the questions will be translated and presented in English. The full Dutch version of the survey can be found in the appendix.

3.4 Procedure

After the survey was created, a link to the survey was sent out to the students. The link directed the students to the first section of the survey where an introduction was shown with a little explanation about serious games and instructions for the survey. Then the respondents proceeded to the question sections where answering every question was required to proceed to the next section of the survey. After the survey was completely filled out, the students were asked to send in their answers, and a ‘thank you’ message was shown. Everything was filled out anonymously.

(13)

3.5 Analysis

To analyze the data, the data from the survey was converted and downloaded to a .CSV file. The data was then imported into the statistical data editor IBM SPSS (version 26). The question variables and answers were renamed to English names to make it more insightful for this paper and then labelled with the original translated survey question. SPSS was then used to analyze the data which is presented in the results section below. To test the reliability of the questions a Cronbach's alpha was computed. Then groups of questions were clustered into new variables by using the compute variable function in SPSS so they could be easily used to compare and perform tests on. These clusters were based on the similarity of the nature of the questions, and they are presented below. To test the reliability of these clusters, the Cronbach’s alpha of these individual clusters was also computed. An Independent Samples T-Test was performed to see if there was a significant difference between the answers of people with- and without experience with playing serious games, and in addition to this it was tested if there was a difference in the answers of male and female respondents.

The next clusters were formed by adding them up and dividing them by the number of questions used in the cluster:

General opinion

I like the idea of using technology to improve the quality of education I think that education should make better use of new technologies I feel that video games can have educational value

If serious games were proven to work, I would play them in my free time In my free time, I would rather play a serious game than read a textbook

Operations

Serious games would only be interesting to me if I could really learn how to perform physical operations

Serious games would be interesting for me if they could help me acquire skills needed to perform medical operations

(14)

Interaction

Serious games would be interesting for me if they could help me acquire skills needed for good interaction with patients

Serious games would be interesting for me if they could help me acquire communicative skills that are needed to maintain contact with colleagues

Factual Knowledge

Serious games would be interesting for me if they could help me acquire factual knowledge.

(15)

4. Results

4.1 Reliability analysis

To test the reliability of the questions provided with a Likert-scale, the Cronbach's alpha coefficient was computed. The minimum value for this coefficient should be at least .7 to be reliable. On these questions the coefficient turned out to be .776 which means the scale of the questions is reliable, and the survey can be used for further research.

To test the reliability of the clustered variables, the Cronbach’s alpha of every individual cluster was also computed. The alpha’s are presented in the table below.

Table 1. Cronbach’s alpha of clustered variables

Cronbach’s alpha N of items General opinion .709 5

Operations .599 2

Interaction .877 2

Factual knowledge N/A 1

As shown in the Table 1, the Cronbach’s alpha of the variables ‘General opinion’ and ‘Interaction’ had were bigger than .7 which means they are reliable. The Cronbach’s alpha of ‘Operations’ was .599 which can be explained by the small number of respondents and the fact that the cluster only consists of two variables. Any conclusions taken from tests with these variable should not be taken for granted. ‘Factual knowledge’ does not have a Cronbach’s alpha because it only consists of one variable.

4.2 Item statistics

In the table below we can see the mean scores for each question, where a mean of 5 would imply the respondents fully agree with the statement, and a mean of 1 would mean the respondents do not agree at all. Next to the mean, the standard deviation is shown, and next to that the number of respondents for each question.

(16)

Table 2. Item statistics of variables

Mean Std.

Deviation N

I like the idea of using technology to improve the quality of education 4.43 .690 28 I think that education should make better use of new technologies 4.29 .763 28 I feel that video games can have educational value 4.00 .770 28 If serious games were proven to work, I would play them in my free

time 3.61 .994 28

Serious games would only be interesting to me if I could really learn

how to perform physical operations 3.79 .995 28 In my free time, I would rather play a serious game than read a

textbook 4.04 .962 28

I think serious games could help me to obtain factual knowledge 4.00 .816 28 Serious games would be interesting for me if they could help me

acquire skills needed for good interaction with patients 3.32 1.188 28 Serious games would be interesting for me if they could help me

acquire skills needed to perform medical operations 4.21 .787 28 Serious games would be interesting for me if they could help me

acquire communicative skills that are needed to maintain contact with colleagues

2.86 1.208 28

Serious games would be interesting for me if they could help me

acquire factual knowledge. 4.07 .979 28

I have experience with computers 4.46 .744 28 I have experience with gaming 3.82 1.090 28 I have experience with playing serious games 1.93 1.215 28

Table 3. Item statistics of clustered variables

Mean Std. Deviation N

General Opinion 4.07 .574 28

Operations 4.00 .758 28

Interaction 3.09 1.13 28

Factual Knowledge 4.07 .977 28

Looking at these statistics, it can be said that the respondents all have experience with computers, but most of their experience with serious games is very low.

(17)

4.3 Independent samples T-Tests

With this test we are able to see if there is a significant difference between certain groups using the independent variable. The test was performed on two independent variables to test the hypotheses.

4.3.1 Experience

Table 4. Group statistics of clustered variables I have experience with playing serious games N Mean Std. Deviation Factual Knowledge 1 15 4.2000 .17457 4 6 3.8333 .65405 General Opinion 1 15 3.9467 .16032 4 6 4.5667 .15846 Operations 1 15 4.1000 .16330 4 6 4.5000 .12910 Interaction 1 15 2.8333 .28312 4 6 3.7500 .46098

If we compare people who had no experience with serious games to people who gave a score of 4 (=agree) on having experience with serious games, there were no significant differences on the answers except for the general opinion. Because the p value of 0.035 < 0.05 we can assume the general opinion significantly differs between people who had experience with serious games and people with no experience at all. If we look at the mean differences we can see that the group with experience gave a much higher score to the questions (mean = 4.5667) than people with no experience (mean = 3.9467).

4.3.2 Gender

Conducting the same test to compare between male and female respondents didn’t show any significant difference in any of the variable scores. The means of the answers were very much alike.

(18)

5. Discussion/Conclusion

As seen in the literature, more and more research on serious games is finding evidence of serious games having a positive effect on learning. With this research the goal was to see what the attitude of Dutch medical students at the UvA is toward the use of serious games in their education.

5.1 Hypothesis

1. Students will have a positive stance towards the use of new technologies, including serious games, in education. The mean of their scores will be higher than neutral (higher than a score of 3 on the 1-5 Likert-scale).

The first hypothesis has been met. As seen in Table 3, the mean score of the clusters is higher than 3 for every cluster. ‘General opinion’ (mean = 4.07), ‘Operations’ (mean = 4.00) and ‘Factual knowledge’ (4.07) all scored far higher than neutral. ‘Interaction’ was the only variable which scored about 1 point lower than the rest with a mean score of 3.07. This implicates that students are generally positive about serious games, and they would like to use serious games as a tool for obtaining factual knowledge and acquiring skills needed to perform medical operations.

2. Students with experience in serious games will have an even significantly more positive stance towards the use of new technologies, including serious games, in education than students with no experience.

The second hypothesis has been partly met. There was not a significant difference between the two groups for any of the variables, except for ‘General opinion’. The mean of the scores for this variable was significantly higher for people with experience with serious games. This shows that people with prior experience have a more positive stance in general towards the use of new technologies, including serious games, in education.

3. Male respondents will have a more positive stance toward the use of serious games than female respondents.

The third hypothesis has not been met. There was no significant difference in the answers of male and female respondents. This means gender does not affect the opinion about any of the questions.

(19)

5.2 Conclusion

By looking at the results of this research, it can be said that Dutch medical students at the UvA have a positive stance towards the use of serious games in education. The mean scores implied that the respondents mostly agreed on the statements from the survey. The only variable that had a lower score was ‘Interaction’, which implies that the respondents think interaction is the least important skill that can be acquired by playing serious games.

The study by Johnsen et al. (2019) found that students who participated in a nursing course which used serious games had an increased positive stance towards the use of serious games in education. The research conducted for this paper confirms this, as respondents with experience with serious games had a significantly higher score on their general opinion than respondents with no experience. There was no significant difference in answers found between male and female respondents.

5.3 Discussion

Several remarks about this research have to be made. First of all, the number of respondents was a little bit low (N=28). Also only six of the respondents were female, against twenty-two male respondents. Because of these low numbers, outliers had a big impact on the results and the reliability of the answers is lower.

Secondly, the choice to cluster similar questions into new variables was made after the survey was sent out. This caused two clusters to only consist of two questions and one cluster to be only one question.

For future research it is advised to find a bigger sample group and to categorize the questions more evenly in order to create bigger clusters with more similar questions, to increase the validity of the research.

(20)

References

Abt, Clark C. Serious Games. New York: Viking, 1970, 176 pp., $5.95, L.C. 79-83234. (1970). American Behavioral Scientist, 14(1), 129–129.

Chung, C. H., Shen, C., & Qiu, Y. Z. (2019). Students' Acceptance of Gamification in Higher Education. International Journal of Game-Based Learning (IJGBL), 9(2), 1-19.

Johnsen, H. M., Fossum, M., Vivekananda-Schmidt, P., Fruhling, A., & Slettebø, Å. (2018). Nursing students' perceptions of a video-based serious game's educational value: A pilot study. Nurse education today, 62, 62-68.

Kron, F. W., Gjerde, C. L., Sen, A., & Fetters, M. D. (2010). Medical student attitudes toward video games and related new media technologies in medical education. BMC medical education, 10(1), 50.

Kurenov, S. N., Cance, W. W., Noel, B., & Mozingo, D. W. (2009). Game-based mass casualty burn training. Studies in health technology and informatics, 142, 142-144.

Raja, R., & Nagasubramani, P. C. (2018). Impact of modern technology in education. Journal of Applied and Advanced Research, 3(1), 33-35.

Ricciardi, F., & De Paolis, L. T. (2014). A comprehensive review of serious games in health professions. International Journal of Computer Games Technology, 2014.

Sabri, H., Cowan, B., Kapralos, B., Porte, M., Backstein, D., & Dubrowskie, A. (2010). Serious games for knee replacement surgery procedure education and training. Procedia-social and behavioral sciences, 2(2), 3483-3488.

(21)

Venkatesh, V., Brown, S. A., & Bala, H. (2013). Bridging the

qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS quarterly, 21-54.

Wang, R., DeMaria Jr, S., Goldberg, A., & Katz, D. (2016). A systematic review of serious games in training health care professionals. Simulation in Healthcare, 11(1), 41-51.

Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. (2013). A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games. Journal of Educational Psychology, 105(2), 249–265.

Zhonggen, Y. (2019). A meta-analysis of use of serious games in education over a decade. International Journal of Computer Games Technology, 2019.

Appendix

Dutch survey as sent out to medical students at the UvA

Het gebruik van serious games in de geneeskundestudie.

Mijn naam is Lucas Berkvens, en voor mijn bachelor studie Informatiekunde aan de UvA onderzoek ik hoe Nederlandse geneeskundestudenten tegenover het gebruik van serious games als leermiddel staan.

Een serious game is een game met een ander primair doel dan puur vermaak. Het is een leermiddel dat dankzij het spelelement de motivatie en concentratie van de speler probeert te verhogen terwijl de leerstof wordt gepresenteerd. Hieronder is een klein filmpje te zien van 2 minuten over het gebruik van serious games op de hogeschool in Utrecht om iets meer inzicht te krijgen in het

onderwerp.

https://www.youtube.com/watch?v=dkMP6gKhauA

Het beantwoorden van de vragenlijst duurt ongeveer 3 minuten. Alvast bedankt voor je medewerking.

(22)

1. Wat is je leeftijd? [ ]

2. Wat is je geslacht? a. Man

b. Vrouw

Voor de volgende stellingen moet worden aangegeven in hoeverre je het eens bent met de volgende stellingen (1 = zeer oneens, 2 = oneens, 3 = neutraal, 4 = eens, 5 = zeer eens).

3. Ik heb ervaring met computers 4. Ik heb ervaring met gaming

5. Ik heb ervaring met het spelen van serious games

6. Wordt er in je opleiding momenteel gebruik gemaakt van serious games? 7. Ik vind het een goed idee om technologie te gebruiken voor het

verbeteren van de kwaliteit van de geneeskundestudie

8. Ik denk dat de huidige opleiding beter gebruik moet maken van nieuwe technologie

9. Ik denk dat videogames een bijdrage kunnen leveren aan de opleiding 10.Als serious games goed zouden werken, zou ik ze ook in mijn vrije tijd

spelen

11.Serious games zouden voor mij alleen interessant zijn als ik er echt van kan leren om medische handelingen te verrichten

12.Ik zou in mijn vrije tijd liever een serious game spelen dan een tekstboek lezen

13.Ik denk dat serious games goed zouden kunnen helpen om nieuwe feitelijke kennis op te doen

14.Serious games zouden voor mij interessant zijn als ze me helpen met vaardigheden om een goede interactie met patiënten te ontwikkelen 15.Serious games zouden voor mij interessant zijn als ze me helpen

vaardigheden te ontwikkelen die nodig zijn voor de uitvoering van medische handelingen

16.Serious games zouden voor mij interessant zijn als ze me helpen communicatieve vaardigheden te ontwikkelen die nodig zijn in het onderhouden van contacten met collega’s

17.Serious games zouden voor mij interessant zijn als ze me kunnen helpen algemene kennis op te doen

Belangrijk - minder belangrijk - plezier in het spel

- feitelijke kennis verkrijgen

(23)

- communicatieve vaardigheden te ontwikkelen die nodig zijn in het onderhouden van contacten met collega’s

- Vaardigheden voor een goede interactie met de patiënten te ontwikkelen

https://docs.google.com/forms/d/1zpZQFYx3Xv-lrPelknTwSMbF_rR3-GqiBk2K1c R7iao

The attitude of Dutch medical students toward the use of serious games in medical education