The important characteristics of Serious Games in Healthcare

University of Groningen

Faculty of Economics and Business, Business Administration

MSc Strategic Innovation Management

Master Thesis

The important characteristics of Serious Games

in Healthcare

Master Student:

Evangelos Chalas (s3165132)

Supervisors:

Dr. Wilfred Schoenmakers

Co-assessor:

Dr. Killian McCarthy

2

Abstract

This study aims to identify the characteristics that a Serious Game needs in order to be accepted and implemented in Healthcare operations. In order to do so the P-B-A framework (Healthcare Profile - Benefits of Serious Games - Applications) has been created. The Healthcare profile was used to identify the main operations of a Healthcare institution, the main drivers and directions that the Healthcare members seek for improvements on their operations and finally the actual needs that appear. During the next step of the thesis, the benefits that Serious Games are able to offer in three Healthcare operations were identified. The three Healthcare operations that this thesis focuses on is the Training of the Healthcare staff, the distant care and the Care for chronic diseases. The findings of the research specify not only the benefits that Serious Games can offer but also several obstacles that impede their use, together with the translation of the effect of both the benefits and obstacles on the elements of the value propositions block of the Serious Games. The elements that were used for this research are from an Internet of Things business model and the most important elements of a Serious Game in Healthcare are the following: ‘’getting the job done’’; Convenience/Usability; Accessibility; Performance; Cost/Risk Reduction; Security & Privacy. These elements create a threshold, the absence of which can make the acceptance of Serious Games applications in Healthcare less likely. There are also some more characteristics that are important in projects of Serious Games in Healthcare and can offer value. So, the partners/stakeholders of such a project and their involvement in it should be taken into account and in addition, the careful planning of all the stages of the project from the initial stages and the essential terms of co-creation and design thinking that have to be the ‘’philosophy’’ behind the actions on these projects. Finally, there are managerial implications and further research options presented. This research is concentrated on the northern part of The Netherlands (Friesland – Groningen – Drenthe) so the participating healthcare institutes and serious game developers are also from this territory.

3

Table of Contents

1. Introduction……….……….……...……...page 04 2. Literature Review...……….……….…....………….………..…....page 07 2.1. Healthcare sector…..……….……...………...…………....…………...page 07 2.2. Internet of Things……….…...……….……...………page 09 2.3. Internet of Things in Healthcare………...………...…...………page 10 2.4. Serious Games.……….……...………....…….….……….page 11 2.5. Serious Games in Healthcare……….………...……….……….…....…….page 13 2.6. Business Models………...……….…...….….page 16 3. Theory – Conceptual Model………..…………..……….…...…...…..page 18 4. Methods………..……….…………...……...….…..….page 24 4.1. Study Population………...……….………...…..…...page 24 4.2. Method of Measurement………...………….………...……..page 26 4.3. Method of Analysis………...……….………...…...page 27 5. Results………...………...……….………...………page 30 5.1. Healthcare Profile………..……….…...………...……..page 30 5.2. Ranking of the Elements……….………...…………...……...…..page 33 5.3. Influence of Serious Games on three Healthcare operations……...…...page 35 5.3.1. Training of the Healthcare staff………...………..…...…...…..page 36 5.3.2. Distant Care……….…...…….…..…….……...……..page 37 5.3.3. Care for Chronic Diseases………...…….…….……...……page 38 5.3.4. Translation of the influence on the elements………...…….…..…...…..page 39 5.4. Additional findings from the interviews………...…...….page 43 6. Discussion………...……….……….………….……page 45 7. Conclusion……….….…………....……..page 51 7.1. Managerial Implications………..……….……….………...……..……...page 53 7.2. Limitations & Further Research…………..………...…...…..…...…...…..page 53 7.3. Further Research………...…...……….page 54 8. References………...……...…...….……...….….…..page 56 9. Appendices………...………...…………..……...……..….page 61 List of Abbreviations: HC: Healthcare SGs: Serious Games G/D: Game Developers [P #]: Participant #

4

1. Introduction

Computers, robots and other technological applications are being used for several years and in many different sectors of the society and industry (Costa et al., 2017). Although, questions like ‘’are these applications effective?’’, ‘’how can we build them in order to take their total value?’’, ‘’what are the advantages and disadvantages that come together with their use?’’, ‘’are people willing and able to use them?’’ are always on the table and have yet to be fully answered (Akl et al., 2013, Callies et al., 2017).

A term that is highly attractive to many researchers over the previous years, as its use is being expanding over many different industries, is the Internet of Things (IoT). Many application of IoT have been reported in transportation and manufacturing (Glova et al., 2014), the energy sector (Zhou et al., 2016), in security (Wortmann & Fluchter, 2015), healthcare (Wade, 2015), agriculture (Dijkman et al., 2015) and education (Pfisterer et al., 2016). IoT is linked with the connectivity of devices and the flawless transmission of information among them (Bernsdorf et al., 2016, Ullah et al., 2016). A term often used when referring to IoT, is ‘’Pervasive Computing’’ (Glova et al., 2014). The use of individual devices, or even individual devices within whole systems, is a direction for companies to discover new ways of finding and absorbing value.

As organizations target on gaining value, the construction of effective business models is a crucial aspect of their business operations (Ju et al., 2016). Business Models are frameworks of a company that define the ways that business is made up and subsequently the ways it transforms its operations into value (Teece, 2010). The expansion and increased use of IoT has created the need for new business models that take into account its characteristics and effects (Wortmann & Fluchter, 2015, Glova et al., 2014). Although, several studies have focused on that purpose (Dijkman et al., 2015, Pfisterer et al., 2016, Zhou et al., 2016), the value that can be grasped through the use of IoT is still vague (Ju et al., 2016). The ability of the Internet of Things to connect devices and share information has shed light on technological applications that can be used on that direction.

5

that is essential for IoT devices. Serious Games are targeting objectives further than pure entertainment. The amusement of the user is used as a mean to acquire new skills or enhance his/her abilities through playing (Rapp, 2017, Callies et al., 2017). Serious gaming developers have visualized the potential of their applications and have shown an increasing interest in expanding their use around several sectors (Susi et al., 2017). Serious games are being and can be used in areas like military, government, education and corporate operations. The objectives linked with its use can be several like enhancing abilities (Graafland et al, 2014), performing distant connection and communication (Zielke et al, 2015) or producing scientific data (Baaden et al., 2017).

A sector where these technological applications have a lot of use and potential is the Healthcare sector (Andersen, 2012, Pang, 2013, Soares et al, 2016, Andreatta, 2017). Applications and new methods are being used at the healthcare institutions to enhance their offerings and to optimize the final outcomes for both, themselves and their patients (Bateman et al., 2012, Goris et al., 2014, Fovet et al, 2017). The needs at the Healthcare sector are multiple and are evolving through the years along with society (Ullah et al., 2016). Previous studies have analyzed and discussed several needs of the Healthcare sector as well as the advantages that Internet of Things and Serious Gaming can offer to respond to them (Breslin et al., 2007, Bui & Zorzi, 2011, Anguera et al., 2013, Brasil et al., 2013, Cruz & Uresti, 2017). Many of these studies agree that in this changing environment, of new needs and opportunities, there is a need for new business models (Islam et al., 2015).

6

approach the idea of implementing Serious Games in their operations and also identify where and how value from the Serious Gaming can be gained in Healthcare operations. The focus of this thesis will be on the three categories that Healthcare can use Serious Games, namely the training of the Healthcare staff, Distant care and Care for chronic diseases. Members from both the Healthcare sector and Serious Games developers will contribute by sharing information about their perspective and insights on such projects. The main research question of this thesis is the following:

‘’Which are the important characteristics of Serious Games for Healthcare?’’

7

2. Literature Review

2.1 Healthcare sector:

As discussed, most of the sectors of humanity’s operations are being affected and transformed because of the never-stopping technological advances and achievements. In the same sense, technology and computers are already being used in Healthcare sector. Too many applications have overrun specific human skills and are used to optimize operations (Ju et al., 2016). Technology is being used at Healthcare institutions for more efficient and faster communication among the healthcare staff as well as with the patients (Wade, 2015). Furthermore, Doctors use applications for training sessions and enhancement of their abilities (Graafland et al, 2014, Ricciardi, 2014). Other applications help with the monitoring of the patient’s health conditions and the coordination of their treatment procedures (Torres et al., 2017). The term that has been commonly used for such applications is ‘’eHealth’’. At the European Commission this term refers to ‘’tools and services using information and communication technologies (ICTs) that can improve prevention, diagnosis, treatment, monitoring and management’’ (European Commission). There are already several devices that have delivered successful and promising results. An example towards this is the ‘’Wize Mirror’’ project. This device is a smart mirror which was designed to replace doctor’s diagnosis and it was able for the patients to use it at home (European Commission, 2017). This Mirror has a reflective surface and makes it able for the users to interact with it so as to analyze their health conditions. There is a face recognition system that identifies the user and specific data are collected through multispectral cameras, 3D scanner and a breath analyzer. Another interesting example is the Oculus Health. Oculus designed a whole new system of Healthcare operations with the objective to ‘’keep doctors, patients and caregivers connected between visits with clinical information that is insightful, actionable, and engaging’’ (Oculus Health). By this system, patients are offered an improved quality of healthcare service by access and continuity, care management, comprehensiveness and coordination, patient and caregiver engagement and also planned care and population health (Oculus Health). The options and abilities for such systems are always expanding and more ideas towards their use are being introduced.

8

and on the web that mention the issues that Healthcare operations are confronting and where there are needs for enhancements.

Numerous needs and problems that are being faced during healthcare operations have been expressed through the years. In their article, Ricciardi & Paolis (2014) state that the training and education (de Wit‐ Zuurendonk & Oei, 2011) of the Healthcare staff is very important for the patient’s safety. The traditional methods of training are time and money consuming and also involve threat for the health of the participating patients (Verdaasdonk et al, 2009, Graafland et al, 2012, 2014, Andreatta, 2017). In addition to that, the ‘’ineffective communication among healthcare professionals is one of the leading causes of medical errors and patient harm” (Zielke et al, 2015). Moreover, Healthcare institutions have to find ways to decrease the overall expenses and make their services more affordable (IBM: ‘’Smarter Games for Health’’, Bernsdorf et al., 2016). There is a tremendous amount of data being generated and not efficiently communicated around the interested members of a Healthcare institution. An idea that was expressed towards a solution of this problem is the need of an easily accessible and informed database (Pustisek, 2017). Another direction for improvement should be the level of effective management and decision making at a Healthcare institutions. Andersen (2012) discusses about simulation models that have been proved successful in sharpening the decision making and management skills of both experienced practitioners and trainees.

Furthermore, many articles express the need for more personalized and customized healthcare according to the patient's needs and conditions (Ju et al., 2016). Some healthcare incidents are common and have to be treated with the same ‘’formula’’, but there are others that the approach has to be out of the standardized procedures to cover individual needs (McCallum, 2012). The personalized health is also a contributing factor towards the increased engagement and motivation that patients feel, something that is still a challenge for the Healthcare institutions (IBM: ‘’Smarter Games for Health’’), especially on chronic diseases (Vugts et al, 2017). For example, the ‘’iManageCancer’’ is a project with the objective to help patients with chronic cancer manage their own care. It is basically a platform that promotes ‘’clinician-patient communication, maximizes compliance to therapy, informs about drug interactions and contributes to the management of pain and other side-effects of cancer treatments’’ and encourages the patient (European Commission: Cordis).

9

their isolation level was decreased and also, by stimulation of cognitive abilities and access to internet enabled services, their wellbeing and independence were increased. In the research of Bateman et al. (2017), where mHealth (mobile Health) is discussed, similar beneficial results are stated.

2.2 Internet of Things:

As mentioned, the continuous expansion of technology and the changes it brings with it, affects many aspects of every person’s life. One of these technological advantages is the use of the internet and particular focus has been given to its ability to connect devices and people that are in a distance. This technology can be named with a single term as the Internet of Things or IoT (Pang, 2013).

10 2.3 Internet of Things in Healthcare:

Internet of things with its applications, is a useful tool also in Healthcare and it offers several advantages that can be promising (Bui & Zorzi, 2011, Li et al., 2015). The European Commission has also given credit to such applications and wants to support them (European Commission site). Through strategies and special grants, the E.C. has set certain goals with the use of systems like IoT. These goals have to do with the improvement of citizen’s health by increasing the healthcare quality and access and to make these tools more effective, user-friendly and widely accepted (European Commission site). The benefits that have been reported from previous studies can be grouped in some categories. IoT is able to create a big and efficient network among the healthcare staff as well as with their patients (Pang, 2013). This network is characterized by flawless sharing of information and creates new possibilities on training and managing of the staff (Ullah et al., 2016), on monitoring and management of the patients (Li et al., 2015), on gathering and storing of information (Pang, 2013), on decreasing the Institution’s expenses (Andreatta, 2017) or on anticipating emergency situations beforehand and on time (Li et al., 2015). Each of these categories result in more advantages that Internet of things is able to offer.

With the inhouse network of the staff in a Healthcare institution, a lot of information and knowledge has the opportunity to be communicated. More experienced members provide their knowledge and help on situations that are not directly responsible for and the training of novice staff increases in quality (Andersen, 2012). The schedule of the staff is becoming easier to get developed through the use of platforms that replace paperwork and manual tables. These platforms are online so they can be reached from everywhere at any time (Wade, 2015). Here it is worth mentioning the vulnerability of internet accessed information and the need for privacy and a well-established security system (Pustisek, 2017). This security is not applied only to the ‘’inhouse’’ but also to the connections and sharing of information between the healthcare staff and their patients. Patients will not be motivated to give away personal and monitored data if they do not feel the safety that they will remain personal. On the other hand, the collection of their personal data and the ability to display their situation and possible progress, works as an enabler and motivation (Pustisek, 2017).

11

patient with the opportunity of a better lifestyle without the need to visit the institution. It is very important that the patients feel independent and comfortable (Ullah et al., 2016). The time they would spend on visiting a healthcare institution can go to their individual preferences and choices of daily life. This last ability of the patients creates benefits for the institutions also as the less the patients visits the easier the scheduling and simultaneously the less the expenses. Furthermore the feedback from the doctors can be updated and more focused on the patient's’ needs (Torres et al., 2017). Doctors even have the ability to examine previous data and compare it with the new ones, so that their advices are on time and with increased prevention towards the wellbeing of their patients (Costa et al., 2017).

The increased level of prevention can be translated as preparedness of the institute itself (Pustisek, 2017). Healthcare staff is able to monitor and control the situation of the institution’s storage, as to be prepared for both emergent and not emergent upcoming events. IoT systems also enable doctors all around the world to share information about individual cases. So, clinicians that are unfamiliar with a new process or situation have the chance to learn from already stated incidents at institutions of other locations (Andreatta, 2017).

2.4 Serious Games:

In order to go to another level, a level that has to do with more concrete applications and devices than just an infrastructure (like IoT), a very interesting direction is the Serious Games. The excessive use of the internet among other technological advances has expanded the use and applications of Serious Games (Baaden et al., 2017). Callies et al. (2017) define Serious Games as ‘’computer-based learning environments, which are built as video games. Their purpose is to train the learner to a specific domain of expertise, while making learning fun’’ (p. 1). ‘’By harnessing the power of entertainment that gaming provides, serious games and gamification have provided a number of research and industrial solutions’’ (Melthis et al., 2016, p.2687).

12

are used today but the number is always increasing with new applications in several industries and with different objectives.

Figure 1: Purposes of Serious Games (Game Accessibility, 2017)

13 2.5 Serious Games in Healthcare:

In healthcare now, there are a lot of applications of Serious Games which are mostly separated in two big categories. The games that are used by the healthcare staff and the games that are used by the patients. In the first category, games are included that enhance the abilities of the members of the Healthcare institutions through training (Goris et al., 2014, Ricciardi and Paolis, 2014) while in the second, there are games that patients use as help during the period of their treatment and rehabilitation (de Wit-Zuurendonk& Oei, 2011, Torres et al., 2017). These types of Serious Games can be really beneficial for the Healthcare sector and below there are some of the advantages that can be offered to that direction and are separated to the above two categories that already have been mentioned.

Considering the first category, the training of the healthcare personnel, there are a lot of reported advantages of Serious Games that are beneficial for the staff of a Healthcare Institution, with most of them related to their training enhancement (Kharrazi et al, 2009, Brox et al., 2011, Howes, 2017). The use of Simulations has created a new avenue for Serious Gaming in Healthcare. Andersen states in his study (2012) that the clinical skills and the proficiency of doctors can be enhanced and accelerated by the use of simulation. Other studies examined the effects of educational games in nursing students (Akl et al., 2013). Results state that simulation training before an actual medical event has positive effects that reflect to better learning outcomes and also decreased anxiety for the nursing students. Other studies mention that there is a difference of the offerings of a Serious Game, which depend on the proficiency of the user. Novice trainees are more enthusiastically using the simulations and this is because with time and experience the portion of benefits is decreasing (Boyd et al., 2006). Another advantage of the Serious Games and simulations is the training environment, as it is artificially designed to give the maximum possible offering to the user. Data from their performance are also saved on the game and are able to be displayed afterwards, something that gives to users the ability to reflect on them and adjust their training. These abilities increase the quality of training and contribute to better achieved results as well as better surgical planning, better awareness of complication as well as error prevention and efficient management (Andersen, 2012).

14

according to the individual’s abilities, proficiency and expertise, something that results in the better effectiveness of their training (Andersen, 2012). This ability of the simulations to adjust to the player’s needs is very important as it increases his engagement level to the training, something that is referred to as ‘’Flow’’ (Cruz and Uresti, 2017). For that to be achieved there should be a balance between the skills of the player and the difficulty of the simulation. Bateman et al. (2012), talk about the Virtual Patients or VPs. VPs are ‘’electronic representation of realistic clinic cases’’ (p.02) and they are used to teach clinical reasoning skills to the user and also improve his/her experience in terms of realism, engagement and learning value.

Another use of simulations is to promote decision-making and team-management skills (Heinrichs et al., 2008). There are such examples in areas like anesthesia (Gaba and DeAnda, 1988), emergency medicine, trauma care (Kizakevich et al., 1998) and obstetrical care (Maslovitz et al., 2007). Moreover, when expert and novice users communicate their knowledge and experiences outside the operating rooms, is beneficial as the experienced ones share their knowledge to the less experienced. This interaction is enhanced by the use of a simulated environment as the level of engagement is higher (Andersen, 2012). With regards to the second category, the patients also gain several advantages by using Serious Games. In the study of Brasil et al. (2013) different games were displayed together with their objectives that target the cognitive abilities of the patients. The ‘’Crossword’’ game was used to stimulate the memory and language of the patient. The ‘’Old’’ game enabled the patients to practice their executive functions like to organize and prioritize information. The “Walking route’’ game that was training the visual and spatial memory. Games like these that are also combined to an IoT system and the internet, have the ability to transfer data from the patients that reach their doctors, creating the ability for constant, accurate, rapid and effective diagnosis (Andersen, 2012). Patients enjoy efficient feedback and they are able to adjust their treatment procedures by distant care, without having to meet with their doctor in person (Meditskos & Kompatsiaris, 2017).

15

their quality of life (Brasil et al., 2013). Furthermore, through the applications of mHealth, assistance and support is provided to patients that suffer from dementia, giving them the opportunity to be less dependent on caregivers, decrease agitation and also decrease passivity (Bateman et al., 2017).

Another Important factor in favor of using Serious Games in Healthcare is the motivation that patients get (Verdaasdonk et al., 2009). Through gaming, patients increase their mobility rates and physical activity, their muscle strength and balance (Soares et al., 2016). ‘’Exercise training has been shown to reduce behavioral and psychological symptoms of dementia, to slow progression of cognitive decline in MCI, and to lead to increased hippocampus size, a region of the brain responsible for short-term memory’’ (Bateman et al., 2017, p. 3).

In the literature a lot of attention of the researchers has been drawn to the question of ‘’How a proper Serious Game should be developed and what characteristics it should have?’’ in order to be able to offer its advantages. These studies offer more information regarding the usefulness of Serious Games and they are ‘’the next step up the ladder’’. It is important for the developers of Serious Games to firstly identify specific needs and problems of the target client or user and afterwards act and try to create a proper Serious Games (Baaden et al., 2017).

16 2.6 Business Models:

All these advantages of the Serious Games and the Internet of Things and provide solutions towards the needs and problems of a Healthcare institution. As in many sectors, in Healthcare also, the ways of operation and making value are changing and with them, the business models are also transforming (Islam et al., 2015).

Teece (2010) explains in his study that a ‘’business model defines how the enterprise creates and delivers value to customers, and then converts payments received to profits’’. Osterwalder and Pigneur (2010) also define a business model as “the rationale of how an organization creates, delivers, and captures value”. As an expected outcome, the IoT and Serious Games usage and their new applications have altered the way companies make business and generate value (Bernsdorf et al., 2016). ‘’Design of IoT architecture involves many factors such as networking, communication, business models and processes, and security’’ (Li et al., 2015, p. 246). The operations that previously were common now are obsolete and different methods have to be used to match the evolving needs and demands. The Business Models have to be altered (Bernsdorf et al., 2016) and the need for openness and collaboration among industries has made their creation complex (Hui, 2014). IoT is linked with the principles of connected devices, continuous information sharing and distant communication. Ju et al. (2016), state that the investigation towards the IoT business models is crucial as there is no ‘’killer application’’ in the market yet and the research has mostly been focused around the technical characteristics. To add on that, ‘’solution with affordable cost and sustainable business models were not in place’’ (Pang, 2013, p. 12). This situation has created an open direction for more investigation on this type of business models.

17

18

3. Theory – Conceptual Model

In the Healthcare sector, the Health-IoT and Serious Games research is scattered without enabling profound directions for applications (Pang, 2013). Pang also states in his article (2013) that ‘’effective device and service integration is crucial for the success of Health-IoT solution’’ (p. 38). A usual mistake during eHealth projects is that research is insufficient, so that the new implementations (like Serious Games applications) do not come with strong proofs of their effectiveness and are not effectively integrated in the development process (Kato, 2012). A big amount of applications are being developed with only a minor amount of them having the potential to reach the market (Kato, 2012). A theoretical base would be really helpful for the developers of Serious Games in order to understand the market they are targeting to, its needs and characteristics (Clark, Osterwalder & Pigneur, 2012, de Boer et al., 2016).

19

proposition block as found by Dijkman et al. are: convenience/usability; ‘’getting the job done’’; performance; possibility for updates; comfort; accessibility; cost reduction; risk reduction; customization; design; price; newness and brand/status, as you can see in figure 2. The target customer for the Serious Games developers in this thesis is a Healthcare institution. Applications for Healthcare, in general is considered a really targeted market, with specific needs and requirements, or a niche market (van Gemert-Pijnen et al., 2011). That is why the applications and devices for this segment should be customized for the targeted audience in order to be accepted (Kotler, 1989).

Figure 2: Value Propositions block (Dijkman et al., 2015) and Healthcare as the target segment

20

that come in touch during the inception of a project of Serious Games in Healthcare. Healthcare sector has a problem or an issue during its operations and is trying to find the possible solutions for it. In the meanwhile the Serious Games developers have to identify these issues and also provide solutions, with specific characteristics, to counter them. This is why these two parties have to work together, exchange valuable information and cooperate in order to accomplish a project of Serious Gaming in Healthcare (Olszewski, 2017). The exact elements/characteristics that a Serious Game should have in order to attract Healthcare institutions as customers are not yet known, and should be specified according to the benefits they offer and also the needs of Healthcare operations that they can cover. During this specification, there may be differences on what the Serious Game developers and Healthcare members think as essential characteristics of these applications. Consequently, in this thesis I will identify these important elements and whether there are differences between these two parties, according to their perceptions on the important characteristics that a Serious Game should have in order to be implemented in Healthcare. In order to do so, the next sub-question will be used:

SQ1:‘’What are the elements of the value proposition block of Serious Games that the Healthcare members and Serious Games developers perceive as important?’’

The process of creating a new application or device for a specific segment of customers, in this situation the Healthcare sector (HC organization, HC workers and patients), has to start with an extensive research. This research should be conducted by the developers with the aim to identify the needs and preferences of their customers (Lowthorpe, 2013). This approach is supported by the theory of the Value Proposition Canvas (Clark, Osterwalder & Pigneur, 2012). According to this theory, the first and most important block when a company is willing to build a Business Model is the Value Propositions block. In order to be able to properly specify the elements of the value proposition, a prerequisite for the developer of a Serious Game (in our situation) is to identify the pains and gains of the client, what are the customer’s operations, what is he/she trying to achieve and what are the issues and needs during these operations. In other words, to make a customer’s profile (of the Healthcare sector) (Osterwalder & Pigneur, 2003, Clark, Osterwalder & Pigneur, 2012). In this thesis, I will figure out what is the perspective of the Healthcare members towards the implementation of Serious Games and what are the conditions in which such games could be used (figure 3). So the first two questions that should be answered are:

21

SQ3:‘’What are the obstacles in the Serious Games implementation in Healthcare?’’

Figure 3: Investigation of the Healthcare sector profile towards the implementation of Serious Games in its operations

For the next step, I will focus on the display of the benefits and obstacles that the implementation of certain Serious Game applications can have. These effects are going to be provided and also discussed by the interviewees. This step is important as many Healthcare members do not really understand the possibilities that Serious Games are able to have in enhancing their operations (Callies et al., 2017). With this final step, I will discover the stated benefits and obstacles of Serious Games in some Applications on Healthcare operations and try to translate their effects on the elements/characteristics of the value proposition block that is used in this research. The conceptual model of this thesis is completed and displayed in figure 4 (page 22).

22

Healthcare with Serious Games and IoT (Ricciardi & Paolis, 2014, Zielke et al., 2015, Bateman et al., 2017, Pustisek, 2017, Vugts et al., 2017). So, the following sub-question should be answered:

SQ4:‘’How Serious Games are able to affect Healthcare in Training of the healthcare staff, Distant care and care for Chronic diseases?’’

A question that has to be separated in the following sub-questions:

SQ4.1:‘’What are the benefits and obstacles that a Serious Game can have on an application for training of the Healthcare staff?’’

SQ4.2:‘’What are the benefits and obstacles that a Serious Game can have on an application for Distant Care?’’

SQ4.3:‘’What are the benefits and obstacles that a Serious Game can have on an application for Chronic Diseases?’’

23

By answering all the above sub questions I will be able to reveal feasible directions that both the Healthcare members and the Serious Games developers are needed and also can follow and improve the creation, acceptance and implementation of Serious Games in Healthcare. This process will be completed by answering the main research question of this thesis which is:

24

4.

Methods

This study is going to analyze the profile (needs and obstacles) of a healthcare institution (sub questions 2, 3), as a possible customer for Serious Games implementation, and the benefits that can be grasped with the use of Serious Games during Healthcare operations. The elements of an Internet of Things business model have been chosen and will be analyzed from both members of the Healthcare sector and Serious Games developers (sub question 1). Through this research and information from the interviews, the specific elements of a Serious Game that tend to offer value to healthcare operations will be identified as well as the possible effects that the use of Serious Games can have in three operations of healthcare: training of the healthcare staff, distant care and care for chronic diseases (sub questions 4, 4.1, 4.2, 4.3). Important stakeholders of such a project are going to give their opinion about the importance of the proposed elements of the value proposition block of a Serious Game. So, while the combination of theory and data methods will help to create the ‘’customer profile’’ of the healthcare sector (Clark, Osterwalder & Pigneur, 2012), search on the internet about previous efforts and initiatives together with information from interviews will provide insights and recommendations on using Serious Games and IoT in this sector. This study will be a list of all this data with the aim to communicate it to the relevant parties. The conceptual model is not going to actually be tested but instead filled with the collected data. All the needs, benefits and new opportunities for application will be listed and form the Profile- Benefits - Applications or PBA framework (figure 4), having in mind that it will be a framework that both the Healthcare members and Game developers can consult, but also a guide for further research and a direction for new applications. .

4.1 Study Population:

25

26

Table 1: Participants of the research

4.2 Method of Measurement:

The sources of information for my thesis will be prior scientific articles as well as collected information that is scattered in Internet sites and Databases. As for the scientific articles, the main keywords that were used were eHealth, Serious Games, Internet of Things and new Business Models. The internet search was based mostly on the site of the European commission where there is much information about projects that combine technology and Serious Gaming in Healthcare. A database found on internet from the European Commission is the “CORDIS’’ database where they list all projects that have been founded in order to research and/or produce beneficial applications. In addition, several sites and web pages of SGs developers and blogs with similar content were searched. In order to find Healthcare institutes in the North of the Netherlands, two databases were used. The ‘’Companyinfo’’ and the ‘’ZorgGids’’ databases. There, filters helped in order to identify Healthcare institutes that are having operations in the regions of Groningen, Drenthe and Friesland. The data and contact details of the possible participants from the Healthcare sector were extracted and used as the database of this thesis. For the contacts regarding the Serious Games developers, search in the web helped me identify the game developers of the North of the Netherlands, and manually fill their details in the database focused on them.

27

operations while the main focus of the second group will be to express the benefits that the implementation of Serious Games in Healthcare is able to offer.

In this thesis I used input of data from semi-structured interviews. The semi-structured interviews were preferred for this research as the interviewees are more willing to share information and by the use of proper and relevant questions, the answers are going to be really useful for the research (Gray, 2014b). Furthermore, the ability for the interviewer to ‘’guide’’ the conversation enhances the probability for optimal collected data (Gray, 2014b). Two pilot interviews were used as a guide to transform the interview plan where needed and be sure of its effectiveness. The feedback on the interview plan and also the pilot interview helped me to optimize the whole interview in terms of content and time. Furthermore, the interview plan was finalized with feedback from some of the stakeholders that examined the process of the interview as well as its outcome. There is an overview of the interview plan in Appendices (Appendix 1). This contributed in more relevance of the data that were gathered from the interviews and increased the validity (Gray, 2014b). Every interview lasted from 48 to 60 minutes. The plan of these interviews was the same for both the categories of stakeholders in order to help me find out the differences between their opinions, something that also decreases bias and increases the validity of the interviews (Verschuren & Doorewaard, 2010).

In addition, the interviews were made at the place of the participant in order to make them feel comfortable and trigger him/her to involve more in the ‘’conversation’’ (Van Diest, 2016, p. 46). All the interviews were conducted by the same researcher, were audio recorded and then transcribed. The transcribed interviews were sent back to the interviewees to validate them and mention any misunderstanding or information that were not precise (Verschuren, & Doorewaard, 2010. The questions were open-ended and the time for each topic was managed by the researcher according to the participation and knowledge of the interviewee.

4.3 Method of Analysis:

28

categorization was named as ‘’Healthcare perspective’’ for the participants from Healthcare (the other first order first order categorization was ‘’Developers’ perspective’’. The second order categorization was more about the general topic that a statement can be grouped into, like ‘’Healthcare operations’’. The third categorization was more specific as for the Healthcare operation in our case, like ‘’distant care’’. Finally, the statement of the interviewee was included at the end of this ‘’path’’. To complete this document, each statement of the participants was written under similar ‘’categorization paths’’ or a new one was created if there was not a similar comment (and consequently a path) from the previous participants. From all the interviewee’s statements, similarities and contradictions between the categories but also between the nine participants of this research were identified (axial coding (Gray, 2014)). The final stage was to integrate the created categories of the excel file into themes and theory, so selective coding was used (Gray, 2014). As a result of these, a coding sheet was created. This coding sheet was organized in the same way the study questions are organized to help me analyze them afterwards and proceed to results (Bloomberg & Volpe, 2012). For the ranking of the elements (sub-question 1), another excel sheet was used. There the detailed rankings of all the participants of the research were displayed. Every line of the excel sheet was a different element when every column was a different respondent. By that I could display the rankings of all the participants according to the elements while ‘excel functions’ (sum, sumIF, countIF, average) helped me create the grouped results for every element. To identify the importance of the elements I measured which ones had the least sum because the respondents had ranked with number one the most important element (in the final counting for example, an element with a score as 10 is more important than another with 15). In order to identify differences between the Healthcare members and the game developers I just programmed the functions to count only the results of the members of the group I wanted. The ‘’count’’ function helped me to see how many times among the participants specific elements were ranked in the higher places of importance. So, the final ranking of the elements was created and then I was able to extract important outcomes that are presented on the results paragraph.

29

Finally, to ensure the validity and reliability of this research in table 2 there is a display of the techniques that were used and were mentioned in the previous paragraphs. These techniques and measures were based on the articles of Verschuren & Doorewaard (2010), van Aken, Berends & van der Bij, (2012), Gray (2014b) and also the tables of Riege and McKinnon (2003) and Voss et al. in Karlsson (2016).

30

5. Results

In the next session, the results of this thesis will be displayed. First of all the Healthcare profile will be analyzed, with the overview of the Healthcare operations, the drivers that trigger the HC members to seek for solutions and the actual needs of their operations but also the obstacles that appear. Secondly, there are the findings towards the elements that were used in the research and their ranking regarding their importance which was identified by the participants. Thirdly, the analysis of the benefits and obstacles of Serious Games in the three chosen operations of Healthcare takes place, together with the influence of these findings on the elements of the value propositions block. Last but not least, the conceptual model of this thesis will be filled in with the findings of the research.

5.1 Healthcare Profile (HC operations and needs):

Through the first part of the interviews, the respondents were asked to discuss about the Healthcare operations, which are these operations, how they try to improve them and what are the needs and obstacles that appear. Having these questions in mind, the main identified Healthcare operations, which were mentioned during the interviews, are the categories of Health (prevention - cure - care), Education, Research and Management [Participants 1, 3, 4]. The Healthcare management and members will try to find solutions when they are triggered by certain events. These drivers can be categorized as: new rules and regulations from outside the institution; internal changes like quality and safety measures (like the need for ISO certification); goals in order to apply for a subsidy; or when there is excessive budget that can be used in several ways (like research or implementation of technological advancements in the HC operations) [Participant 1]. When the Healthcare management is searching for solutions, there are certain directions that they focus on. Some of these directions can be: Training of the HC staff; Healthy Ageing; Prevention; Complex care; Distant care; Rehabilitation; Mental illnesses [P 1, 4, 5, 6, 8].

31

The needs of Healthcare that are identified, according to the respondent’s answers and the coding procedure that I followed on their answers, can be grouped in four categories: Patients; Healthcare staff; Management; Partners. In the category of the Patients, the Healthcare members are trying to find ways that enable their patients to live independently and enhance their self-confidence [P4]. Moreover, there is a need for effective communication between the doctors and the patients [P2]. Here I should add the opinion that patients do not always share important information because they are private data and difficult to communicate (for example information about the fears that a patient has, or what struggles they face while using the toilets of their houses etc.) [P 5, 7]. Another aim of the Healthcare members is towards the comfort of their patients and to reduce the risks they take during the cure/care process [P 1, 3, 6]. In addition, it is beneficial when the cure/care procedures are personalized and are able to be included in the daily lives of the patients [P 2, 3, 4, 7, 8]. Patients need to be engaged to the process, something that will deliver better results [P9].

As for the Healthcare staff, institutions seem to need more employees in order to deliver their operations in better quality and quantity [P 3, 6, 7]. The time of the training of the HC staff should be reduced [P 1, 2, 5] but also there should be certain training sessions of new technologies, their abilities and how they are able to be used [P 2, 3]. Furthermore, there was expressed the need of finding ways to make the HC professional better understand the condition of their patients but also to acknowledge the consequences of their actions on them [P5]. Finally, as far as the training if the HC staff is concerned, safety of the patients during this procedure is very important [P-all] and also ways of evaluating the skills of the professionals are needed [P6].

Considering the Management of a HC institution, the ability to treat more patients is highly valued together with the reduction of the cost of the operations [P 1, 3, 6, 9]. This means that the same number or fewer personnel should be able to cover more needs. In the category of management, the need of financial resources was also stated, in order to make a proper and sustainable operations or projects [P 1, 3, 4, 6]. Finally, a really important matter in Healthcare is the fact that personal data of the patients are stored and transferred, something that creates a great need for their security and privacy [P 1, 4, 5, 7].

32

Technology, including of course Serious Games and IoT applications, is not the solution for every problem [P8]. Logically, there are obstacles that prevent application like Serious Games and IoT to be implemented in the Healthcare operations. These obstacles contribute towards the opinion that an Internet of Things approach is stated to be a bit too far from the reality of nowadays and also a bit demanding on its implementation, due to safety reasons [P1].

So, the first obstacle that was stated is the need for high standards for the privacy and security of the patient’s personal data that are being stored and transferred *P 1, 4, 6+. Secondly, the validation of the Serious Games, or the checks and proof that they work well and as intended, is an important process when the stake is the health and wellbeing of people. That is why these checks are extensive and demand a great amount of time and effort [P 5, 7]. In addition to that, technology is always evolving, so certain delays in the procedure may cause loss of the ‘’technological momentum’’ *P8+. When these delays occur, even if the application is performing well something else will be introduced in the market that does the job better. Thirdly, technological applications as Serious Games and Internet of things target to the efficiency of the HC institutions [P6]. This means that the amount of patients or the time of their treatment will tend to decrease, something that is nowadays a drawback for the Healthcare providers as the money they get from the insurances are also based on the amount of patients that visit the institution [P 1, 6]. Another important difficulty is that the Healthcare institutions do not have the right infrastructure (amount of computers and other devices) to support technological solutions without totally increasing their expenditures to obtain them [P5]. At this point it would be worth mentioning the hardware dependency that could cause problems. If a specific hardware is needed from an application, there may be difficulties in the acquisition of it, in updating it and in its ability to become scalable (or develop many devices to reduce the total cost) [P 5, 7, 8, 9].

33

professional use technology in daily life and how on their working environments [P 2, 3]. They tend to not accept the same devices that they commonly use and they do not understand what can be the advantages of their use at work. Also, the problem of ‘’nurse proof’’ was also stated *P 2, 3+. By that, the new applications and devices that the nurses are asked to implement in their work should be really easy to use. If this is not the case and they need too much time to operate, the nurses and professionals will not accept it.

Finally, there are the psychological matters and concerns. At first there are patients that really value the personal interaction with their doctor and will not accept to communicate and being monitored distantly from a computer or another device [P 2]. Secondly, there is the belief that humans are better than machines when social skills and emotions are involved. Machines find it difficult to fully understand emotions and the way a person may say ‘’yes’’ without meaning it [P 8, 9]. That is why it is mentioned that technological advancements and new applications can be a solution to certain difficulties of the Healthcare institutions but they are not a solution for everything.

5.2 Ranking of the elements:

As there has been identified a problem for Serious Games to reach their market, in this case the Healthcare institutions, it is worthy to know what are the characteristics of the Serious Games that are perceived as important, increasing the possibility for this applications to get accepted and implemented. To discover these elements/characteristics, the final part of the first step of the interviews was used, when the respondents were asked to rank the given elements (or remove or add elements of their own). So, the next step is an overview of several elements that a Serious Game may have, that have been valued and ranked from both the Healthcare members and the Serious Game developers, according to their importance in such a project. These results arise from all the participants in this research, so from both the Healthcare members and the Serious Games developers. In a following paragraph some differences between these two groups’ opinions, are also displayed.

34

be used for [P-all]. Moreover, in this list there were four more elements added by the participants: Identity (fit in the organizations ‘’culture’’) *P7+; Recording of the input data [P8]; Security & Privacy [P1] and Validation [P7]. Validation was mentioned by many interviewees as combined with the elements of ‘’getting the job done’’. These elements, three now, are displayed in figure 05 (page 41) with this mark ‘’*’’.

There are six elements that were mentioned as the most important by the interviewee themselves. Most of the participants expressed the opinion that the elements that they rank as the three to four most important, act also as a threshold for the acceptance of the project. The rest elements are extras to the project. When these important elements are not present, the implementation of the specific Serious Game is likely to be rejected. The elements that create this threshold are chosen as the most important ones and are presented on table 3 (their rank is according to their mentioned importance). The elements were chosen namely by the participants as part of the threshold but also result from the percentages that they were asked to give to the elements. The percentages of importance that were given to these elements were 10% or more (if all the elements were equally important this number would be almost 7.7% for every element), something that also indicates their advanced importance. Cost & Risk reduction are together because they are complementary according to the aim of the projects. The cost reduction is more connected and crucial as an element of a Serious Games that help with the training of the HC staff, while the risk reduction element is the goal of Serious Games that take part in, for example, the rehabilitation process of a patient [P 7, 8]. Also, the element of Security & Privacy of the patient and the patients’ data was ranked by only participant 1 as an important element (ranked as second) but it was indeed mentioned by everyone as an essential factor when we speak about projects of Serious Games and IoT that have the aim to be implemented in Healthcare operations.

35

Table 3: The threshold of the elements of Serious Games projects in Healthcare

There are some interesting outcomes about the ranking of the elements that have to be mentioned. These outcomes arise also from the excel sheet that includes and calculates the final ranking of the elements from all the participants of this research. First of all, the element of ‘’Getting the job done’’ was ranked as number one from the respondents the most times (five) and was often related as part of the validation process [P 1, 6, 7, 8). Secondly, a reason for the limited acceptance of Serious Games in Healthcare may be the differences in the perspective of the client of the Serious Game (HC members) and the Game developers. While checking the rankings of the two different stakeholders we can see that for both the elements of ‘’getting the job done’’ and ‘’convenience/usability’’ are in number one and two, respectively. For the Healthcare members, the element of ‘’Accessibility’’ comes to the 3rd place when ‘’Performance’’ to the 4th. Conversely, the Game Developers think that ‘’Cost/Risk Reduction’’ has to be in the 3rd place and ‘’Accessibility’’ to the 4th. Thirdly, another interesting finding was that none of the Serious Game developers chose to include in their list the element of ‘’price’’. This choice was due to their belief that if their application is able to give the value that a Healthcare institution needs, the price will not be an issue [P 6, 7, 8, 9]. The same element of price, was ranked from all the participating Healthcare members but only from one was mentioned as of high importance (second in ranking), mentioning that the less expensive the device/application is, the bigger amount of them will get purchased by the Healthcare institution and the more patients/HC staff are going to use it [P2].

5.3 Influence of Serious Games on three Healthcare operations:

The second part of the interviews was based on identifying the benefits and obstacles of Serious Games in Healthcare operations. Before the display of the benefits and obstacles that are related with the use of Serious Games in Healthcare, it is worth mentioning the findings that were stated from the

1. Getting the job done 2. Convenience/Usability 3. Accessibility

4. Performance

36

interviewees and are common to all the three selected categories (Training of the HC staff, distant care and Care for chronic diseases). The obstacles were displayed in paragraph 5.1 so here the benefits are going to be shown. These benefits, as the obstacles, are common due to the fact that they are applicable to the new technology itself (Serious Games) and particularly to every application that comes with it [P 4, 9].

The first of these common benefits is the ability of the Serious Games to be used at home together with the possibilities for monitoring of the patients and the easier exchange of feedback [P-all]. Moreover, the elements of fun, engagement and motivation of the patients give them a significant advantage over the traditional methods of treatment and care [P-all]. Among the other advantages that the use of Serious Games can offer in Healthcare is also the ability of the patient (or professional in the case of training) to use it on his/her own and without an instructor after the introductory session of getting to know the game [P-all+. Let’s not forget that Serious Games are still games and thus they can be played multiple times. This is something very useful when there is a need of training the abilities of someone and monitoring of a process [P 2, 3, 4, 9]. In addition, these games can be constructed in a way to target a specific audience of users [P8] something that can be an advantage when targeting personalized care [P-all]. Finally, Serious Games are built; this means that there are various environments that can be created and several levels of difficulty that are adjusted to each user [P-all].

5.3.1 Training of the Healthcare staff:

37

least 50 times cheaper than a similar traditional method of training of the HC staff that is a big simulator.

On the other hand, there are some obstacles that were mentioned towards the first example and the category of Training of the Healthcare staff. This Serious Game was created, among other reasons, to reduce the time of practice of the HC professional, something that did not happen [P 1, 5]. As far as the characteristics of the game itself, there is also a lack of competition elements that give incentive and motivation to the user to practice [P 7, 9]. Furthermore, this specific application was not properly introduced to the market and the project’s budget was concentrated more on the development stage and not the forthcoming ones (e.g. marketing) [P9]. This resulted in limited sales (market introduction and penetration) and also in the discouragement of the doctors to use it [P 4, 9].

5.3.2 Distant care:

Again a Wii application was used to identify the benefits and obstacles towards the implementation of a Serious Game in another operation of Healthcare, the distant care. An important benefit that Serious Games are able to offer in distant care is the ability for personalized care. The game can be customized and adjusted to the needs [P 5, 7] of every patient without several delays during the process [P 5, 8]. Furthermore, the patient can use the device/applications when he/she is willing to [P 3, 7] and also enjoy constant communication and feedback with the doctor [P5]. As for the part of the institution that is using distant care, the doctors will be able to work where they want also and reach more patients at the same time [P 1, 2, 6, 7, 8, 9].

38

Another difficulty created the fact that some Serious Games may include devices which are inconvenient to be installed by everyone [P7]. Moreover, other concerns gather around if such applications can properly monitor the patient, if the evaluation process is indeed correct and last but not least if the ethical standards of the Healthcare institutions are in place and are covered [P6].

5.3.3 Care for chronic diseases:

In order to examine the possible effects of Serious Games in the third operation of Healthcare, the care for chronic diseases, a robot (Personal Assistant for health Lifestyle - PAL) that interacts with diabetic children, was used as an example. Firstly, having a patient to use a Serious Game during the extensive cure/care process is a way to destruct him/her from the problem while simultaneously improving his/her psychological condition and well-being [P6]. Secondly with such an application, there is the possibility of the patient to be monitored in daily life with all the consequential advantages of proper and exact evaluation and adjustment of the process [P 3, 6]. Thirdly, the fact that children have to interact with a robot, will help them open themselves and share more and significant information that they would not reveal to the doctor due to shame or fear [P 5, 3, 7, 9]. It is really helpful that the robot uses the excitement of the children and through playing it gets into the same mindset of a child [P 2, 3, 5] and it is possible to gather data that the doctor may be able to use afterwards [P7].

39

a person (social skills, emotion interpretation, etc.) [P8]. As mentioned before, Serious Games can solve a big variety of issues but they are not the solution for everything.

5.3.4 Translation of the influence on the elements:

During the final part of the interviews, the respondents were asked to ‘’translate’’ the positive or negative effects of the Serious Games that they had already mentioned, into the elements of the value proposition block of step 1. For example, the sixth participant [P6] mentioned that the ability of a Serious Game to identify the emotions of the user, a patient in this case, is beneficial towards the final outcome and as a consequence, it has a positive effect (+) on the ‘’getting the job done’’ element. The positive effect of Serious Games in the three categories that were examined and identified by the interviewees of this research, can be translated into the following elements: Getting the job done; Convenience/Usability; Accessibility; Cost Reduction; Risk Reduction; Customization; Design; Newness; Recording input data; Performance; Comfort.

Conversely, the negative influence of Serious Games in these 3 categories of HC operations affects the following elements: Getting the job done; Convenience/Usability; Accessibility; Cost Reduction; Price; Performance.

40

Benefit Element

Ability for feedback ‘’Getting the job done’’

Evaluation of the skills of the Use Convenience/Usability

Iteration in Use Accessibility

Possibility for experimentation Risk Reduction

Transferable Customization

Recognition of Emotions

Table 4: Translation of some benefits of Serious Games in Healthcare operations and their positive influence on the elements of the Value propositions block

Obstacles Element

Hardware dependency ‘’Getting the job done’’

Interpretation Mistakes Convenience/Usability

Time needed for validation Cost Reduction

Healthcare security/quality standards Price

Table 5: Translation of some obstacles of Serious Games in Healthcare operations and their negative influence on the elements of the Value propositions block

41

finally it is validated on the safety of the private data that are being stored and used. The next elements are considered as extras and less important, than the threshold, for such a project.

42

A

pplications

Figure 5: Complete Conceptual model. P-B-A framework

Training of the Healthcare Staff Increase the abilities of the user

Evaluation of the skills Ability for feedback on

performance

No need to practice on patients Cheaper alternatives to traditional

methods

Distant Care Ability to use at home Personalization of the process

Avoid delays on adjustments of the process

Reach/treat more patients at the same time Better communication among

patient/doctor

Ability to monitor in daily life

Care for Chronic Diseases Psychological support/motivation

Ability to monitor in daily life Input of information of the patient

that is not easily communicated Ability to reach the mind-set

of the patient

43 5.4 Additional findings from the interviews:

The discussions from the semi-structured interviewees, provided more interesting details and information that should be considered in technological projects, like these of Serious Games and IoT, which want to be implemented in Healthcare.

Firstly, all the respondents believe that that the case is not like the following: ‘’a Serious Games is developed and then an effort is made by the developers in order to be sold to a Healthcare institution’’. Instead it is actually the other way around. A Healthcare institution is triggered by the four drivers that were mentioned during the HC profile and is trying to find solutions. In order to cover the needs, Serious Games or/and IoT applications may be examples of a solution, but they are not always the best or not possible to help in every problem [P-all]. So, it is very crucial that from the inception of such a project the stakeholders are able to discuss, share valuable knowledge, understand each other potential and needs and develop a plan for the following steps [P 7, 9]. Of course, the applications that are being created, should be carefully developed, covering the needs and expectations of the ‘’customers’’, the Healthcare institutions in our case [P 4, 6]. What was expressed by an interviewee was that the communication among the stakeholders of such projects is not always the best and several important information about the project are not given to the developers (e.g. the Healthcare institution delays to state the end user of the application) when they are needed [P8]. This may result in continuing the project with certain delays and gaps that will have to be fixed afterwards (causing frustration and increased costs).

44

Thirdly, after the initial stakeholders, more parties should participate in the project to improve it. The available ecosystem that may affect the project should be identified and researched [P4]. Other parties can be members from the government [P1], insurance parties [P1], knowledge institutes [P4], consultant *P9+ or patients. Patient’s feedback for instance is really welcome during the most of the stages of the project as they may be the end users for the new application and their ‘’profile’’ is also needed *P 1, 4, 5, 8].