N EW P RODUCT D EVELOPMENT :

N

EW

P

RODUCT

D

EVELOPMENT

:

D

EVELOPING A CONCEPT TESTING METHOD FOR HEALTHCARE

INFORMATION TECHNOLOGY PLATFORMS

MASTER

’

S THESIS

BY

M

AARTEN VAN

Z

OEST

U

NIVERSITY OF

G

RONINGEN

F

ACULTY OF

E

CONOMICS AND

B

USINESS

MS

C

T

ECHNOLOGY

&

O

PERATIONS

M

ANAGEMENT

AND

N

EWCASTLE

U

NIVERSITY

N

EWCASTLE

U

NIVERSITY

B

USINESS

S

CHOOL

MS

C

O

PERATIONS

&

S

UPPLY

C

HAIN

M

ANAGEMENT

S

UPERVISORS

:

U

NIVERSITY OF

G

RONINGEN

:

P

ROF

.

DR

.

IR

.

J.C.

W

ORTMANN IR

.

A.M.

B

ONVANIE

-L

ENFERINK

N

EWCASTLE

U

NIVERSITY

:

D

R

.

Y

ING

Y

ANG

3

Abstract: To enable self-care management, and with this lowering overall healthcare costs, healthcare information technology platforms (HIT-platforms) are regarded as a suitable means. However, many of these initiatives fail, as they follow an expert-based development process. To break through this process, a more user-centred and co-creation perspective should be taken. In this, concept testing can play a vital role. Therefore, the objective of this research is to develop a concept testing methods for HIT-platforms, which aims at solving the problems identified and takes into account the specifics of the complex healthcare context. By following the design science-approach of Holmström et al. (2009), which entailed an extensive literature study and a single case study, this research resulted in a concept testing method, consisting of several pre-concept testing activities, three concept tests, and post-concept testing activities. The method acknowledges the importance of the incorporation of the end-users and the large variety of stakeholders. However, the constructed method should be further validated and eventually generalised in future projects, to show its applicability in other contexts.

Keywords: Healthcare; Information Technology Platform; Concept Testing; Evaluation; New Product Development

Master’s thesis

New Product Development: Developing a concept testing method for healthcare information technology platforms

Course codes: EBM028A30 & NBS8399 Author

M.J. (Maarten) van Zoest

Akerkhof 10-4, 9711 JB Groningen m.j.van.zoest@student.rug.nl Student number RUG: 2042851 Student number NUBS: 140628063

Word count: 15.529 (Excluding title page, abstract, acknowledgement, references and appendices) December 18, 2015

Universities:

University of Groningen, Faculty of Economics and Business Nettelbosje 2, 9747 AE Groningen

Tel: +31(0)50 3633741

Newcastle University, Newcastle University Business School 5 Barrack Road, Newcastle Upon Tyne, NE1 4SE

5

A

CKNOWLEDGEMENT

Without having any background in neither software engineering and information technology, nor healthcare, this research has been very challenging for me. Nevertheless, even though the research was very challenging to me, I enjoyed to explore the worlds of IT and healthcare. From my perspective, I think I can be very satisfied with the result of the thesis, as it is in my opinion quite a substantial piece of work. However, the end result of this master’s thesis would not have been possible without the help of several people. Therefore, I would like to thank some people for their contribution.

First of all, I want to thank my supervisors from Groningen, prof. dr. ir. Hans Wortmann and ir. Anne Bonvanie-Lenferink, for providing me with constructive feedback and the brainstorm sessions we had. Especially I want to thank Anne for the sparring meetings we had and for always being available to answer urgent questions, which helped me getting new insights on my study and completing my thesis. I wish you the best of luck with your PhD-research coming years. Second, I also would like to thank my co-assessor dr. Ying Yang from Newcastle University for her feedback and supervision during this project. Third, I greatly appreciate the cooperation of Hille Meetsma of VitalinQ and being available to share his experiences and thoughts on the innovation process of HIT-platforms. Last but definitely not least, I owe a big thanks to my family, and in particular my parents, for always supporting me during my studies. Without your endless support, interest and belief in me, finishing my studies and this thesis would never have been possible.

Maarten van Zoest

7

T

ABLE OF

C

ONTENTS

Chapter 1: Introduction ... 9

Chapter 2: Research background ... 11

2.1 Problem framing ... 11

2.2 Context ... 13

2.3 Concept testing ... 14

2.4 Research framework ... 16

Chapter 3: Research Design ... 19

3.1 Research method ... 19

3.2 Research design ... 20

3.3 Research method limitations ... 23

Chapter 4: Solution Design ... 25

4.1 Literature study ... 25

4.2 Initial Solution Design ... 28

4.3 Findings Case Study ... 36

4.4 Refined Solution Design ... 41

4.5 Limitations to solution design ... 45

Chapter 5: Discussion ... 47

5.1 Theoretical contributions... 48

5.2 Implications for practice ... 48

Chapter 6: Conclusion ... 49

6.1 Research questions ... 49

6.2 Further research ... 51

Bibliography ... 52

8

Appendix A: Overview of used keywords for literature study... 61

Appendix B: Interview protocol – Interview Hille Meetsma (VitalinQ), 5-11-2015 ... 63

Appendix C: Interview protocol – Interview Hille Meetsma (VitalinQ), 2-12-2015 ... 69

Appendix D: Case Narrative – Case study VitalinQ... 77

Appendix E: Case Study and Recommendations Report for VitalinQ ... 88

Appendix F: Transcription interview Hille Meetsma (5-11-2015) ... 99

Appendix G: Transcript interview Hille Meetsma (VitalinQ) 2-12-2015 ... 115

T

ABLE OF

F

IGURES

Figure 1. Problem analysis ... 12

Figure 2. Conceptual Framework ... 18

Figure 3. Research Design ... 20

Figure 4. Factors influencing concept testing procedure ... 26

Figure 5. Overview pre-concept testing activities ... 30

Figure 6. Overview activities first concept test ... 32

Figure 7. Overview main activities second concept test ... 33

Figure 8. Overview main activities third concept test ... 34

Figure 9. Overview initial solution design ... 36

Figure 10. Overview refined solution design ... 44

T

ABLE OF

T

ABLES

Table 1. Frequently occurred mismatches with HIT-platforms. ... 12

Table 2. Stages Design Science-approach ... 20

Table 3. Definitions determinants UTAUT2 and confirming sources. ... 27

Table 4. Overview concept testing activities ... 35

Table 5. Articles used for triangulation purposes ... 37

Table 6. Contextual difficulties with complementing way in which it is solved. ... 44

Table 7. Recommended sources per method ... 46

9

C

HAPTER

1:

I

NTRODUCTION

In current society, demographic ageing exerts pressure on the active working force, as it has to carry the increasing healthcare costs of the welfare state. Concluded is that the current healthcare system is unsustainable, which will only become worse with the current and future ageing trends (Galbraith et al. 2008). In order to get a grip on the rising healthcare costs, focus should be on proactive management of health and prevention of diseases, rather than (reactive) management of illness. In this, the development and application of innovative Information Technology (IT)-solutions that stimulate self-care can be seen as an important means. Especially wearable technology that continuously monitors a person’s health situation, linked to an IT-platform, is considered key in establishing proactive self-care management (Milenković et al. 2006).

However, even though many healthcare information technology-platform (henceforth ‘HIT-platform’) initiatives are launched, several sources claim that there is a lack of customer acceptance (e.g. Eysenbach & Jadad 2001; Hesse & Shneiderman 2007; Archer et al. 2011). A major reason driving this, is the expert-driven development process, which results in a medically- or technology-driven design of the HIT-platforms (Esser & Goossens 2009; Verhoeven et al. 2010). The HIT-platforms developed fails to meet the users’ preferences and disregards the contextual difficulties of healthcare (Verhoeven et al. 2010; Nijland et al. 2010). To develop HIT-platforms that better match the users’ needs, a more user-centred and participatory stance towards the development of HIT-platforms is required (Jackson et al. 2006; Demiris et al. 2010). A method that encourages to incorporate the end-user into the process, is concept testing. In literature, concept testing is referred to as “estimating customer reactions to a

product idea before committing substantial funds to it” (Moore 1982, p.279). One of the main purposes

of concept testing, is helping to develop the idea. Even though concept testing methods (henceforth CT-methods) are well analysed in the literature, these techniques do not have a perfect fit with the characteristics of HIT-platforms. Hence, a new CT-method, specifically focussing on HIT-platforms, should be developed to make the technology fit-for-purpose.

10

since a first step to the current missing theory will be developed. By partly following the design science-approach of Holmström et al. (2009), this thesis will answer the following two main questions:

(1) What are the problems currently occurring with HIT-platforms and what difficulties could influence concept testing?

(2) How can developers of HIT-platforms successfully go through the concept testing stage?

These two main questions are interrelated, as the method that answers main question 2 should tackle and take into account the identified problems and difficulties of main question 1. In the second main question, the term ‘successfully’ is specified as helping to develop the concept towards a more user-centred design, aiming at increasing the uptake and acceptance of HIT-platforms. To answer the main research questions, five sub-questions have been formulated. It should be noted that the first three sub-questions relate to main question 1, whereas the last two sub-questions relate to main question 2.  What are specific characteristics of healthcare that could influence the concept testing stage?  What are specific characteristics of IT-platforms and how do these influence the concept testing

stage?

 Why do current HIT-platforms often fail in practice?

 What are relevant concept testing methods, techniques and trends available in literature?  How are HIT-platforms different from general products, and what effect does this have on the

concept testing phase of the platform?

11

C

HAPTER

2:

R

ESEARCH BACKGROUND

2.1 P

ROBLEM FRAMING

2.1.1

B

ACKGROUND INFORMATION

In society, the idea of incorporating IT in daily-life gains popularity. The Quantified Self-movement acknowledges and encourages the proactive stance towards obtaining quantifiable health-related data and acting on it. In this, technology is seen as an important means to gather the data (Swan 2013). By acting on this data, improvements on the health situation can be made (Kim 2014). However, whereas numerous of such HIT-platform innovations are launched, the actual uptake of HIT-platforms is low (Flynn et al. 2009). For example, Archer et al. (2011) state that only 7% of American adults use Personal Health Records (PHRs). Moreover, initiatives such as MiVitals, Revolution Health (Techcrunch.com 2009), and even Google Health (Brandt & Rice 2014; Spil & Klein 2015) did not succeed. This is remarkably within a technology-driven era, in which people are highly used to technologies, through which one would expect a higher acceptance of the technology.

2.1.2

P

ROBLEM ANALYSIS

12

et al. 2011). This mismatch, which has also been acknowledged by the World Health Organisation (WHO 2010), should be tackled by reconsidering the New Product Development (NPD)-process for HIT-platforms.

To overcome the currently perceived mismatch and eventually increase the uptake of HIT-platforms, both the end-users and the HIT-specific context should be taken into account in designing the requirements of the HIT-platforms (Dansky et al. 2006; Hamid & Sarmad 2008). In this, a CT-method that takes into account the specifics of the HIT-platform context can be regarded as an applicable means, since it enables firms to incorporate the end-users in the NPD-process in an early stage (Creusen et al. 2013), and enables the collection of high-quality information about user requirements and feedback to new product concepts (Durmuşoǧlu & Barczak 2011). Conclusively, a method on concept testing specifically focussing on HIT-platforms is highly valuable in order to boost the uptake of the technology.

Figure 1. Problem analysis

Problem Mismatches found Sources

Ease-of-use/user-friendliness

 Too comprehensive set of features  Too difficult terminology

 Poor navigation structures  Missing functionalities  Medical jargon  Unfriendly login-function (Liu et al. 2011) (Nijland et al. 2008) (Nijland et al. 2011) (Spil & Klein 2014)

Aesthetic problems

 Interface problems (e.g. too bright/too grey and depressive)

 Unattractive layout

(Liu et al. 2011) (Nijland et al. 2008)

Privacy & trust  Lack of trust in operating company  Scared of personal information leakages

 Occurring challenge: who to share what data with?

(Archer et al. 2011) (Halamka et al. 2008) (Spil & Klein 2014) (Tang et al. 2006)

Perceived usefulness

 Not seeing the additional benefits of using the technology

 Content provided was not perceived as useful and too difficult to understand (i.e. use of jargon).  Approach of wrong user-group (ceiling effect)

(Nijland et al. 2008) (Nijland et al. 2011) (Spil & Klein 2014) (Spil & Klein 2015)

13

2.2 C

ONTEXT

As already stated, currently the context in which HIT-platforms operate is disregarded, which is one of the causes for the low uptake of the technology. Besides, this specific context complicates the process of concept testing. Therefore, the contextual factors will be elaborated on. In this section, the context will be viewed upon from the two main concepts of HIT-platforms: healthcare and IT-platforms.

2.2.1

H

EALTHCARE

-

SETTING

A characteristic that is inherently related to healthcare combined with technology, are privacy and security issues. Since the data that is stored on these platforms is highly personal from character (e.g. medical and psychological information), several sources stress the importance of a highly secured platform, which encompasses, amongst others, the encryption of data (Schweitzer 2011), and authentication methods (Fernández-Alemán et al. 2013). According to various studies, concerns on security are regarded as one of the main barriers to adopt these platforms (e.g. Sunyaev et al. 2010; Avancha et al. 2012). Besides, as a result of the privacy-issues the technology is concerned with strict legislation (Schweitzer 2011), which reduces the pace of innovation (van Gemert-Pijnen et al. 2011). However, the most important difficulty and reason for failure from a healthcare-context perspective, is the complexity caused by the wide variety of stakeholders involved in HIT-platform projects (Eng 2002; Yusof et al. 2008). The large variety of stakeholders and the dependencies between them leads to a complex situation (van Limburg et al. 2011), since the interests, concerns and preferences of these stakeholders might differ, while all should be taken into account. When looking from a one-sided market perspective (which will be explained in section 2.4.1), important to note is that, from a developer point-of-view, there is a customer – end-user difference, in which one will directly sell to the customer and indirectly to the consumer. Hence, the customer pays for the HIT-platform, but it is the end-user who determines if the technology is going to be accepted or not, and thus should be considered as most important (Hamid & Sarmad 2008). Lastly, as can be seen in table 1, the content of healthcare-related information is often regarded as difficult to understand, having its effect on the perceived usefulness.

2.2.2

I

NFORMATION

T

ECHNOLOGY

P

LATFORMS

In literature, the term ‘platform’ is used within different contexts. Even though there is not one commonly used definition of platforms, Tiwana et al. (2010) provide a clear description of a platform from an IT-perspective. According to them, a platform can be defined as: “the extensible codebase of a

14

and the interfaces through which they interoperate” (Tiwana et al., 2010, p. 675). Within this definition,

the modules can be seen as add-ons that run on the platform and add functionality to it.

Looking at IT-platforms as a product, one can distinguish several other odd characteristics. IT-platforms are characterised by their nonphysical existence or ‘intangibility’ (Meyer & Seliger 1998), whereas it is argued that tangibility to a certain extent is required for a user to say something valuable on a product idea (Rushton & Carson 1989). Consequently, understanding the technology’s potential benefits and its use is hindered, since a certain form of vividness is required (Urban et al. 1996). Besides, similar as healthcare as a standalone element, its combination with IT-platforms can be regarded as an experience

product. Typically, with experience products consumers must be exposed to the product, before they

can value it (Poulsson & Kale 2004).

2.3 C

ONCEPT TESTING

To successfully go through the NPD-process, Crawford and Di Benedetto (2011) list five stages to be executed: Opportunity Identification, Concept Generation, Concept Evaluation, Product Development and Product Launch. Within this NPD-process, concept testing is listed as one of the activities in the Concept Evaluation-phase. As said, concept testing provides the developer of a HIT-platform with a suitable means to integrate the end-user in the NPD-process from an early stage onwards. Concept testing should not be confounded with the testing activities product-use testing and market testing, performed during the product development-phase of the NPD-process (Crawford & Di Benedetto 2011). As listed by Crawford and Di Benedetto (2011), concept testing has several purposes: (1) identify poor concepts; (2) estimating the sales of the concepts; and (3) helping to develop the idea. Several extant sources recognise the importance of concept testing within the NPD-process (e.g. Moore 1982; Page & Rosenbaum 1992), especially in providing guidance in the fuzzy front end of the NPD-process (Trott 2011). Typically, when performing a concept test, a firm tries to assess the likelihood of success of their innovation before committing substantial financial and non-financial means to it (Moore 1982; Page & Rosenbaum 1992). After showing the participants a product concept, they are supposed to answer questions related to, amongst others, product characteristics, price and suggestions for improvement (Krishnan & Ulrich 2001). Based on these tests, the initial concept can be developed further towards the users’ requirements and gradually takes shape. Eventually, the concept test should result in a go/no-go decision (Crawford & Di Benedetto 2011), (dis)allowing a firm or project to enter the development phase.

15

2011). Firstly, stimuli design refers to the aforementioned product concept, that is, the statement that is shown to the participant of a concept test (Klink & Athaide 2006). Secondly, the respondent group refers to the group of prospective users that is involved in the concept test (Klink & Athaide 2006), in which one can roughly make a distinction between experts and real users. Schoormans, Ortt and de Bont (1995) argue that expertise or expert-users are better able to advise on missing aspects, make a distinction between valuable and non-valuable product attributes, and to faster interpret product information. However, the requirements of experts might never reflect needs of the majority of the market (van den Hende et al. 2007). Lastly, response measurement typically involves decisions on how the responses should be measured (Klink & Athaide 2006).

2.3.1

C

ONCEPT TESTING TECHNIQUES

16

Conclusively, it is argued that concept testing can bridge the gap between the mismatch that is currently argued to be present in the HIT-platform world. However, existing methods do not have a perfect fit with the characteristics of HIT-platforms. Therefore, this gap should be filled by constructing a new CT-method that solves the current causes for the low acceptance of HIT-platforms, while tackling the difficulties that are inherently related to the environment of this technology.

2.4 R

ESEARCH FRAMEWORK

2.4.1

S

COPE OF RESEARCH

Before elaborating on the conceptual model, the scope of the research will be provided to fence off the research. Firstly, this thesis will be focussed on firms serving a one-sided market. Often, IT-platforms are regarded as two-sided markets, which facilitate interactions between two sides (Rochet & Tirole 2006): developers of applications, and customers. Nevertheless, firms can also produce the applications as well as the IT-platform themselves. When this is the case, one only serves one side of the market: the

customer – end-user side. Moreover, since the spectrum of technologies that are linked to healthcare

is rather broad, this should be narrowed down as well. For this thesis, the focus will be on HIT-platforms that stimulate self-care management, enabling people to make well-informed decisions on their lifestyle, aiming at improving their health. Hence, some elements coincide with persuasive technology, since in some way it tries to transform the attitudes and behaviour of end-users (Chatterjee & Price 2009). Often, it consists of a monitoring and intervention system, complemented with physical and cognitive sensors to enable data collection. The data will be transferred to a platform, in which it will be analysed and consequently be visualised by means of applications. Lastly, the target group of this research and the eventual method that will be constructed, are developers of HIT-platforms and researchers.

2.4.2

C

ONCEPTUAL FRAMEWORK

As a conclusion of this chapter, a conceptual framework will be constructed. Recall the mismatches from table 1 that, as a result from the expert-driven development process, occurred with several HIT-platforms (e.g. Google Health and Microsoft HealthVault). As is argued before, within concept testing, three important decisions have to be made (i.e. stimuli design, respondent group and response measurement). Below, several relations between these decisions and the problems will be explained, resulting in the conceptual framework (figure 2).

17

visual prototyping is a good means for thorough evaluation. Hence, choosing the right stimuli design to be evaluated will lead to richer information on the ease-of-use and aesthetics, which subsequently can be taken into account in the design, aiming at mitigate the mismatch perceived. Concerning the trust issues, as is argued by Kujala (2003), involving the end-user in the design process of the eventual product leads to a higher level of trust. Hence, it is argued that concept testing in general has a positive effect on the likelihood of HIT-platform acceptance, and thus, arrows from all three decisions are linked to trust issues. In addition, by asking the right questions regarding privacy to the respondents (e.g. who can view what data? Or: what data should be measured and what not?), the privacy issues often perceived can be taken into account, trying to minimise the end-users’ barrier for adoption. Hence, the response measurement is linked with privacy issues.

As can be seen in table 1, there were mismatches in the perceived usefulness as well. For example, Nijland et al. (2011) found that some user-groups perceived that they did not need the technology, as they were already doing well (i.e. the ceiling-effect). Besides, in current evaluation processes with HIT-platforms small convenience samples are used (Pagliari 2007). There is a major deficit to this type of sampling. To explain, this often results in a respondent group highly interested and motivated to use the technology, ignoring people who really need it. This is referred to as the inverse care law (Eysenbach 2007). As a result, the HIT-platform will be developed towards the insights of this group, leading to a low perceived usefulness for other groups. If firms not only want to attract, in terms of Rogers’ diffusion innovation curve (Rogers 1995), not only the early adopters, but also the rest of the market, they should know what triggers this group and what preferences they have. Hence, by systematically selecting respondents, one can adapt the HIT-platform to the needs of different target groups, enhancing the perceived usefulness for those groups.

18

19

C

HAPTER

3:

R

ESEARCH

D

ESIGN

This chapter elaborates on how the research that has been conducted, is designed. Firstly, an overview on the research method will be provided. Hereafter, the research design will be elaborated on. This has been split up in a section on the extensive literature study that has been performed, followed by the description of the case study.

3.1 R

ESEARCH METHOD

Before elaborating on how the research is designed, first the research method should be determined. As stated before, this thesis aims at developing a CT-method that solves the problems currently occurring with HIT-platforms. This can be regarded as establishing a certain form of utility (i.e. the CT-method), trying to solve a practical-knowledge problem. This in contrast to the pure science’s aim of establishing a form of truth (i.e. solving a pure-knowledge problem). Hence, the design science approach is perfectly suited to attain the goal of this thesis.

20 Stage: Description:

1. Solution Incubation In the solution incubation, the objective is to develop an initial solution design. It first starts with understanding the problem (i.e. framing the problem). Hereafter, rudiments of a potential solution design should be developed.

2. Solution Refinement In this stage of the process, the first solution design developed in the former stage is subjected to empirical testing, with the aim of refining and evaluating the solution. It should be noted that this is a highly iterative process, in which the solution is tested on what works and what does not (Hevner et al. 2004). In the refinement phase, design improvements, implementation and evaluation are combined.

3. Explanation I – Substantive theory

Systematically introducing the developed solution in multiple contexts, the practical usefulness of the design is extended beyond the initial context, and thus contributes to the generalisability of the design. 4. Explanation II –

Formal theory

Here, the goal is to demonstrate that the solution is not limited to the empirical context under study, aiming at drawing broader generalisations.

Table 2. Stages Design Science-approach (Holmström et al. 2009)

3.2 R

ESEARCH DESIGN

In line with the first two stages of Holmström et al. (2009), the research is subdivided into two phases (see figure 3). Hence, this section will be split up into two separate subsections to elaborate on these phases.

Figure 3. Research Design

3.2.1

P

HASE

1:

L

ITERATURE STUDY

21

served as a foundation (Hevner et al. 2004). Hence, an extensive literature study can be regarded as an applicable method. In performing the literature study, two main activities can be distinguished: the literature search and the literature analysis.

Literature search and analysis

In order to execute the literature study as rigorously as possible and to limit bias as well as random error, it has been designed as systematically as possible (Cook et al. 1997). In doing so, a set of four databases have been consulted throughout the entire review: Google Scholar, Newcastle University

Library, Web of Sience, and Worldcat. Derived from these databases, articles of several journals have

been used, preferably highly cited ones or articles from high-quality journals. The articles have been sought by using the keywords that can be found in appendix A. The most applied searching technique, is a ‘two-way’ literature search, in which a first source leads to new sources, by means of back- and forth-searching (Karlsson 2009). In order to enable the analysis of the literature collected, the articles found are imported into a software-program (Mendeley), with the aim of organising the research. Besides, Mendeley is used as a reference manager as well as for making annotations. In the ‘Bibliography-chapter’, an overview of the used sources can be seen.

3.2.2

P

HASE

2:

S

INGLE CASE STUDY

22

Case selection

Whereas normally it is opted and advised to select multiple cases, since this leads to more compelling and robust results (Yin 2009), in this research a single case study is performed for several reasons. Firstly, design science, especially the first two stages of Holmström et al. (2009), generally tries to solve a practical-knowledge problem of and with one specific company. Moreover, Dyer and Wilkins (1991) argue that single cases allow the researcher to gather much more in-depth contextual information of the phenomenon under study, which was, according to Hevner et al. (2004), required for the evaluation of the artefact. Hence, a single-case study will be performed, and thus, one case should be selected. Before selecting a case, the Unit of Analysis has been determined, which in this thesis, based on the research questions (Johnston et al. 1999), was the ‘concept testing method’. For the case selection, a non-probability sampling technique was chosen, which is generally applied in qualitative research (Ritchie et al. 2003), indicating that the case company should meet several criteria. First and foremost, the company should meet the problem identified in section 2.1.2 (i.e. expert-driven development process). Based on a course at the University of Groningen in which VitalinQ was involved (i.e. Product and Service Development), research on the internet and advices of this thesis’ supervisors, it was determined that VitalinQ met this criterion. Moreover, the case study company should meet the one-sided market criteria. Furthermore, the technology developed by the firm should be (largely) in line with the definition of HIT-platforms in section 2.4.1. Based on these criteria, VitalinQ is chosen to refine the initial artefact with. Within VitalinQ, the director, which was also the founder and thus had a good impression on the innovation processes performed, was chosen to conduct the interview with. A description of VitalinQ will be given in section 4.3.2.

Data collection

In accordance with the suggestion of Holmström et al. (2009), the solution refinement has been an iterative process. Two iterations have been made, consisting of two semi-structured interviews, which enabled the interviewer to treat the pre-defined topics, but allowing to go into depth when interesting findings occurred (Turner 2010). In the first interview, the purpose was to get an understanding of the company VitalinQ, and their NPD- and concept testing-processes. The second interview aimed at getting a deeper understanding on the concept testing activities, their solutions to (concept testing-related) problems identified in literature, and discussing the solution design.

23

(appendix B) has been constructed according to the funnel model, starting with broad questions first, followed by more detailed questions on the concept testing activities (Voss et al. 2002). The second interview protocol (appendix C) elaborated on multiple topics in-depth, since general ‘introductory’-questions were perceived as unnecessary. To enable triangulation of the data collected during the interviews, the interviewee was asked for additional relevant material, prior as well as after the interview was conducted. Since the case studies had a retrospective focus, observations on their concept testing activities, also aiming at data triangulation, were not possible. In the protocol, it was highlighted that all information obtained were regarded as confidential. Besides, the option was given to be anonymised in the thesis.

During the interviews, mainly ‘open-questions’ were asked, allowing the interviewee to talk uninterruptedly (Jacob & Furgerson 2012), leading to in-depth information and insights (Nonthaleerak & Hendry 2008). Since English is not the native language of the interviewee, conducting the interview in English would hinder the interviewee in talking comfortably, jeopardising the richness and completeness of the data. Hence, the interviews were held in Dutch. Where possible and felt appropriate, the ‘five times why-technique’ has been used, to interrogate and finding root-causes for specific problems or choices. The interview protocol with complemented text-boxes served as the guideline for the interview and for making annotations (i.e. field notes) (Voss et al. 2002). After the interview was conducted, the recording was transcribed immediately, to ensure that the observations during the interview and the field notes were documented and taken into account as well. The transcriptions can be seen in appendix F and G.

Data analysis

To become well familiar with the interviews conducted, a within-case analysis has been performed (Eisenhardt 1989). Firstly, in order to organise and reduce the data (McCutcheon & Meredith 1993), after reading the transcription several times, arrays have been constructed in which (translated) quotes are labelled to main- and sub-categories (see appendix D). Next, a detailed and descriptive write-up has been made (Barratt et al. 2011), in the form of a case narrative (appendix D). In this, there was sought for explanation and causality.

3.3 R

ESEARCH METHOD LIMITATIONS

24

problems that are currently occurring from literature. However, albeit the problem was not company-specific, it was concluded that the case-company followed an expert-driven development process as well. Moreover, whereas Holmström et al. (2009) identify four stages to complete the design science research, this study only performed the first and partly the second stage of this approach. Therefore, another main concern is the generalisability of the findings, since the constructed artefact is not validated and generalised in other contexts.

25

C

HAPTER

4:

S

OLUTION

D

ESIGN

Now the problem has been identified and the general means to solve this problem has been determined (i.e. a CT-method), the solution design should be constructed. Firstly, the results of the literature study will be presented. In line with the problem solving-technique mentioned by Holmström et al. (2009), the problem is fixed and there is sought for interventions to solve this problem. However, to determine which interventions (i.e. CT-methods) are applicable for the HIT-platform situation, firstly the contextual factors that will influence the CT-method should be known. Partly, these have been elaborated in chapter 2 (section 2.2), as part of the problem analysis. However, several other factors that influence the CT-method should be highlighted. Moreover, this section will identify the testing attributes as well. Hereafter, the initial solution will be presented. Subsequently, a section will be devoted to the case study that served as a means to refine the initial solution design. Lastly, the refined solution design will be presented, elaborating on the changes that have been made.

4.1 L

ITERATURE STUDY

4.1.1

C

ONTEXTUAL FACTORS

4.1.1.1

User factors

Taking a user-perspective, some factors that will influence the CT-method should be highlighted. First and foremost, it is argued that users’ requirements change over time, as they acquire more information (Lichtenstein et al. 2007; Bano & Ikram 2010). This implies that the users’ requirements will change, disappear or new ones will be added when users use the HIT-platform. Besides, without having a learning period of the concept, people have difficulties in specifying needs (Crawford & Di Benedetto 2011; Trott 2011). Hence, exposing prospective users short and only once to a concept will be insufficient to thoroughly map all requirements. Therefore, longer use of a product concept is required. In addition, when evaluating a product by means of an interview (e.g. in focus groups) or other in short time collected evaluations, humans tend to use their short-term memory and let recent experiences prevail. This can have a negative effect on the results of such a test, since the short-term memory has only limited capacity. Hence, it is important that external memories (e.g. notebook or piece of paper) are provided during the test (van Vliet 2008).

4.1.1.2

IT-platform factors

26

that is essential to mention, is that in software development the highest costs are incurred in the development phase of the product, not in the production phase. Consequently, the earlier errors within the HIT-platform are discovered, the cheaper to correct these (van Vliet 2008). This implicitly indicates that full-working prototypes should not be developed in the very beginning of the CT-method. Rather, the concept test in the very beginning should test a low-fidelity stimuli design which is very simple and quick to develop, providing the developer with a low-cost guideline for adaptations to the concept. Subsequently, via iterative cycles the fidelity should increase.

Figure 4. Factors influencing concept testing procedure

4.1.2

T

ESTING ATTRIBUTES

Before determining how a HIT-platform should be concept tested, firstly it has to be decided what should be tested (i.e. testing attributes). Since to focus of constructing the CT-method is to increase the acceptance of HIT-platforms, the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) serves as a foundation for the testing attributes (Venkatesh et al. 2012). The seven determinants that eventually lead to use behaviour, and thus acceptance, are listed in table 3. However, the determinants are defined for consumer technology in general, whereas HIT-platforms are characterised by a very specific context. Hence, articles on HIT-platform related products have been reviewed to confirm and complement the list.

27

of the technology (van Gemert-Pijnen et al. 2011). According to Rubin and Chisnell (2008), usability encompasses the usefulness of a system (i.e. the user achieves its goals), its effectiveness (i.e. ease-of-use), its learnability (i.e. easy to learn), and if they are satisfying to use. This underlines the importance of several already identified attributes as well (i.e. performance expectancy, effort expectancy, hedonic motivation, and habit). Moreover, the articles reviewed do not directly mention price value. However, here, price is considered to be essential for the concept test, since this can be the foundation for making financial prognoses.

Besides these initial set of testing attributes, additional testing attributes can differ per project. However, as argued in section 2.4.2, trust and privacy issues should be included as a testing attribute. Moreover, obviously, the functionalities should be evaluated, and elements such as an indication of future adoption could be addressed as well.

Determinant Description Sources confirming attributes Performance

Expectancy

“The degree to which using a technology will provide benefits to consumers in performing certain activities.”

(Ant Ozok et al. 2014) (Delone & McLean 2003) (Holden & Karsh 2010) (Liu et al. 2011) (Maguire 2001b) (Spil & Klein 2014)

(van der Meijden et al. 2003) (Venkatesh & Bala, 2008) Effort

Expectancy

“The degree of ease associated with consumers’ user of technology.”

(Ant Ozok et al. 2014) (Delone & McLean 2003) (Häyrinen et al. 2008) (Holden & Karsh 2010) (Maguire 2001b) (McCullagh et al. 2010) (Venkatesh & Bala, 2008) Social

Influence

“The extent to which consumers perceive that important others (e.g. family and friends) believe they should use a particular technology.”

(Kim & Park 2012) (Spil & Klein 2014) (Spil & Klein 2015) (Venkatesh & Bala, 2008) Facilitating

Conditions

“Consumers’ perceptions of the resources and support available to perform a behaviour.”

(Delone & McLean 2003) (Spil & Klein 2015) (Venkatesh & Bala, 2008) Hedonic

Motivation

“The fun or pleasure derived from using a technology.”

(Brandt & Rice 2014) (Maguire 2001b)

(van der Meijden et al. 2003) (Venkatesh & Bala, 2008) Price Value “Consumers’ cognitive tradeoff between the

perceived benefits of the applications and the monetary cost for using them.”

-

Habit “The extent to which people tend to perform behaviors automatically because of learning.”

(Maguire 2001b)

(van der Meijden et al. 2003)

28

4.2 I

NITIAL

S

OLUTION

D

ESIGN

Now sufficient knowledge is gained by means of the literature study, the initial solution design should be constructed. Again, the purpose is to develop a CT-method for HIT-platforms that solves the current causes for the low uptake of HIT-platforms, while taking into account the contextual difficulties identified (figure 4). Hence, the method to be developed should take into account the factors from the three building blocks IT-platforms, Healthcare context, and Users, through which one can consider this as a holistic view. Firstly, some assumptions should be listed. Hereafter, the initial solution design will be explained.

4.2.1

A

SSUMPTIONS

Firstly, it is assumed that all stages prior to the concept testing stage, as outlined by Crawford and Di Benedetto (2011), are already carried out. This implies that the opportunity has been identified and the concept generation stage is performed. Secondly, it is assumed that a team leading the project is already formed. This team should be a multidisciplinary team of experts (Pagliari 2007). Within this project team, an evaluator should be involved, since the methods used are difficult to perform. A final assumption is that the initiative to develop a HIT-platform is initiated by Small and Medium Enterprises (SMEs). Since SMEs mostly do not have a range of product concepts, the main aim should be to develop the product concept.

4.2.2

I

NITIAL CONCEPT TESTING

-

METHOD

In this section, the initial version of the CT-method will be explained. The method is divided into pre-concept testing activities, pre-concept testing activities and post-pre-concept testing activities. Throughout the sections, overviews on the specific activities that have to be performed are visualised. An overview of the entire solution design is displayed in figure 9.

4.2.2.1

Pre-Concept Testing Activities

Context-of-use analysis

29

elements refer to factors for IT-systems in the working environment. This information can be obtained during the stakeholder meeting opposed in the following section.

Stakeholder identification and analysis

As stated before, the variety of stakeholders involved in the development of HIT-platforms complicates the domain. Stakeholders can be defined as persons or organisations that are impacted by the proposed concept (Crawford & Di Benedetto 2011). To ensure that all relevant stakeholders are involved in the HIT-platform development, and eventually agree on the design of the HIT-platform, a first step is to conduct a stakeholder identification and analysis. After identifying the stakeholders, their main roles, responsibilities, problems and goals should be determined in a stakeholder analysis (Maguire 2001b). The identification of these elements could eventually help to determine what values and features the different parties consider as important for the HIT-platform. Since within co-creation processes continuous dialogues are seen as essential (Prahalad & Ramaswamy 2004), these values should be mapped by means of a stakeholder meeting. This stakeholder meeting creates awareness of each other’s problems and values, and aims at establishing a fit between the values opposed (van Limburg et al. 2011).

The values should be transferred to the business model, which will be explained in the next section. The stakeholder meeting will result in a set of values, which should be translated to requirements. This initial set of requirements should be documented. The IEEE830 (1998) provides a set of guidelines which the documentation should follow. However, in this thesis it is opted to use the Volere-method (Robertson & Robertson 2013), for several reasons. First and foremost, it encourages the software engineer to contemplate about whether the requirement has been successfully integrated in the HIT-platform design or not. Besides, each requirement that is documented is supplemented with the rationale behind it. Lastly, the method forces to link the requirements to customer satisfaction, helping to prioritise the requirements and determine which should be incorporated in the design (van Velsen et al. 2009). Another encouraged method to prioritise the requirements, is by means of a stakeholder salience

analysis. In this, the requirements are prioritised based on the importance of the stakeholders, which is

based on the power, legitimacy and urgency per stakeholder (Mitchell et al. 1997).

Business model

Before the concept testing activities start, a last pre-concept testing activity that should be highlighted is the business model. According to Osterwalder and Pigneur (2010, p. 14) a business model describes

“the rationale of how an organisation creates, delivers and captures value”. A business model is

30

development process is value-driven and stakeholder focussed. Moreover, a business model serves as an input for the business case that should be constructed in the Product Development-phase (van Limburg et al. 2011). Currently, several business models are used in practice, such as the STOF-model (Bouwman et al. 2008) and the Business Model Canvas (Osterwalder & Pigneur 2010). In this thesis, it is opposed to use the latter, as it is the most widely adopted business model in practice. The business model canvas is divided three segments: an organisational segment, users, and a financial segment. Hence, it not only takes into account the user, but also the organisation in the design of the HIT-platform. For a detailed description on the different blocks, I refer to Osterwalder and Pigneur (2010). A start should be made with the business model early in the NPD-process (van Limburg et al. 2011). The stakeholder meeting proposed in the previous section, and the values derived from this meeting, can be seen as a starting point for constructing the first rudiments of the business model. However, the business model is subjected to several changes (e.g. joining or leaving stakeholders). Therefore, the business model should gradually evolve over time.

Figure 5. Overview pre-concept testing activities

4.2.2.2

Concept testing activities

In this section, the concept testing activities will be elaborated. It discusses the three concept tests that should be conducted, and a separate section on the documentation. In accordance with Hall (2001), the stimuli design of the concept tests will evolve from low-fidelity to high-fidelity. A detailed overview of the concept tests can be seen in table 4.

First concept test

31

an expert-based test, to get the most valid results. In the first test, we focus on the latter one, since it is expected that experts can better immerse themselves into products that are in its infancy. Besides, the test will be conducted in a laboratory setting (i.e. in-house).

Creating a stimuli design that makes the (often technical) requirements understandable to the respondent group is difficult (Sutcliffe 1996). In literature, several research methods are opposed to overcome this and conduct the low-fidelity concept test. Examples are: paper prototyping, storyboarding, and Wizard-of-Oz prototyping (Maguire 2001b). However, due to the intangibility of the platform on the one hand, and the relatively easy-to-apply and high degree of realism characteristics of the method on the other hand, this thesis proposes to start with the early concept narrative-approach of Van den Hende and Schoormans (2012). This approach will be conducted in a cognitive walkthrough, since this is argued to be a low-cost method and can be used early in the NPD-process (Sunyaev & Chornyi 2012).

Early concept narratives can be performed in a variety of forms, diverging from a simple plain text to full-fledged movies. It is argued that the more realistic the narrative is, the more valuable for a product evaluation. Therefore, it is stated that translating the concept into a movie is best practice to show a real-life situation of the concept (van den Hende et al. 2007). In this movie, a story (or scenario) based on a (fictive) protagonist should show the use of the HIT-platform in its daily context. It can be either one movie or several shorter movies, constructed around the predetermined requirements and functionalities of the HIT-platform. The protagonist should be based on a persona, a lively characterisation of prospective users of the HIT-platform (Cooper 1999). The context-of-use analysis should be a major input for this. For a detailed description on how to pick a persona and form a story around the protagonist, I refer to Cooper (1999).

32

After the execution phase, the walkthrough should be systematically analysed. Patton (2002, p. 439) highlights four analytical framework approaches, which are applicable for this situation as well: process analysis, issues analysis, questions analysis, and sensitizing concepts analysis. It is advised to use multiple of these. The results of this analysis should be summarised and documented, and, subsequently, the Volere-format should be updated. The designers should incorporate the recommendations into the refinement of the prototype, and construct a higher-fidelity prototype.

Figure 6. Overview activities first concept test

Second concept test

In the second concept test, the higher-fidelity prototype should be tested. To enable low-cost changes after this test, this prototype should not yet have all functions in operation. Rather, some main aspects of the HIT-platform should be developed. For example, a prototypical version of an application with some basic functionalities could be developed. Here, it is opted to use the thinking-aloud method. In literature, this evaluation method is superior to other methods due to several reasons. Firstly, according to Jaspers (2009), this method is important in detecting usability flaws. Van Velsen et al. (2011) even argue that thinking-aloud is a ‘must’, since it is the only method that unmasks critical and serious flaws. In addition to this, Tsakonas and Papatheodorou (2006) argue that thinking-aloud is highly applicable when deciding the usefulness and the degree to which a product will be used. Lastly, as stated by Benbunan-Fich (2001), this thinking-aloud exceeds the performance of normal interviews or questionnaires concerning identification of inadequate functionalities.

The think-aloud method refers to a method in which the evaluator observes the participant, while he/she uses the HIT-platform. The participants are required to talk aloud, through which the evaluator can understand their process of cognition (Jaspers et al. 2004). It is categorised as a usability test, generally entailing two phases: (1) systematically gather the think-aloud protocol, and (2) analysing these protocols (Jaspers 2009). Before the first phase starts, scenarios on what should be performed by the participants have to be prepared. These scenarios are descriptions of tasks that should be performed by the participant. These should not state directly what to do, but rather provide them with a more general described task. For example: “Suppose you want to check the history of your average

heartrate, what would you do?” They should be realistic and representative for the normal use of the

33

at least highlight all essential parts of the HIT-platform. Besides this, the respondents (about 5-8 per target group) should be gathered upfront.

At the start of the test, the evaluator instructs the participant on the procedure and content of the test. During the gathering process, the evaluator guides the participant through the prepared scenarios, instructing him/her to say what he/she is thinking when performing the task on the HIT-platform. In this test, external memory will not be needed, since they can directly express their thoughts. A closing interview can be conducted with, for example, questions regarding their purchasing intentions of this HIT-platform, what the participant (dis)liked about the HIT-platform, and privacy-issues. The whole session should be video and/or audio recorded, to enable a thorough analysis afterwards (Jaspers 2009). By means of a content analysis, the think-aloud sessions should be analysed. For an extensive description on the analysis, see Elo and Kyngäs (2007). What is important to highlight is that the recordings should be transcribed and subsequently coded, which will result in a verbal protocol. The analysis should be performed by evaluation experts, preferably with a background in cognitive ergonomics and IT-platforms. The results of this content analysis should be summarised and documented again, leading to an updated Volere-format. Based on this document and format, the designers should construct a full-working prototype of the HIT-platform.

Figure 7. Overview main activities second concept test

Third concept test

Even though the previous two tests provide the developer with a credible option to discover flaws early, participants need a learning period their needs and requirements change over time. Moreover, testing the HIT-platform merely in a laboratory setting (as with the first two tests) comes with several limitations. Firstly, the participants are only subjected to highly artificial tasks. Besides, it is impossible to control all factors influencing human behaviour (Preece 1994). Therefore, the full-working prototype should be implemented in the daily-life of the user for a prolonged period. In this stage, personal data is gathered and stored on the platform. Hence, due to the privacy-sensitive setting of healthcare, extra attention should be paid to the security of the prototype.

34

predefined list of items that should be asked should be constructed, allowing the user to give their views on the technology that they feel are important (Maguire 2001b). Besides post-experience interviews, one could conduct an evaluation questionnaire on how the respondents liked/disliked several functionalities of the HIT-platform and their opinion on the usability. For the latter, the Software Usability Measurement Instrument (SUMI) could be used (Kirakowski & Corbett 1993). These questionnaires can be easily spread via the application itself.

After the post-experience interviews are conducted, these should be analysed in a same manner as in concept test 2. Findings of the analysis should again be summarised and documented. The Volere-format should be updated again. These documents serve as the input for the designer for refining the prototype.

Figure 8. Overview main activities third concept test

Documentation

35

Concept test 1 Concept test 2 Concept test 3

Method

Early concept narrative approach + Cognitive walkthrough

Thinking aloud method Trial study + post-experience interviews

Stimuli design Full-fledged movie

(low-fidelity prototype)

Partly working prototype (higher-fidelity prototype)

Full working prototype (high-fidelity prototype)

Respondent

group Experts End-users End-users

Setting Laboratory (in-house) Laboratory (in-house) Field (users’ daily-life)

Assessed attributes

- Usability items (i.e. performance expectancy, effort expectancy, hedonic motivation, habit)

- Functionalities - Price value

- Indication on adoption - Trust & privacy issues

- Usability items (i.e. performance expectancy, effort expectancy, hedonic motivation, habit)

- Functionalities - Price value

- Indication on adoption - Trust & privacy issues

- All seven testing attributes UTAUT2

- Functionalities - Price value

- Indication on adoption - Trust & privacy issue

Data analysis method - Process analysis - Issues analysis - Questions analysis - Sensitizing concepts analysis

Content analysis Content analysis

Table 4. Overview concept testing activities

4.2.2.3

Post-concept testing activities

36

Figure 9. Overview initial solution design

4.3 F

INDINGS

C

ASE

S

TUDY

37

in appendix D. An extensive case study and recommendations report has been sent to VitalinQ (appendix E). This section will lead to the refined solution design, explained in section 4.4.

4.3.1

V

ALIDATION

Before elaborating on the results of the case study that is performed, first a small section should be devoted to the validation of the case study. As highlighted in the Research Design-chapter, before and during the interview with Hille Meetsma (director and founder of VitalinQ) additional information has been requested for triangulation purposes. However, due to the competitively sensitive characteristics of the data, this data was not shared with the researcher. Therefore, to enrich the results-section, a second-best option is chosen to enable some form of triangulation: a set of 4 articles from literature, that had similar characteristics as VitalinQ, have been consulted in order to validate the data derived from the interviews to some extent. Even though no ‘perfect matches’ to the VitalinQ-case could be identified, the sources in the table below were believed to be a valuable standard. Table 5 lists the sources used, highlighting the product that is developed and (dis)similarities with the VitalinQ case.

Validation article

Product developed

Description Similarities (+) and dissimilarities (-) with the VitalinQ-case

Stinson et al. (2013)

App for pain assessment

An application developed as an electronic pain diary, which collects data on several pain-related factors. Besides, it evaluates medications and other pain management strategies used.

+ Medical data

+ Data stored in a database, in combination with an application + Similar NPD-process

- Slightly other purpose (collect and store data vs. inform users on their behaviour)

Span et al. (2014) & Span et al. (2015)

DecideGuide Interactive web-tool, helping

people with dementia to make shared decisions with healthcare professionals.

+ Helps in decision making + Medical data

+ Integrates several decision-makers (i.e. users, healthcare professionals etc.)

- Decision-making in other (medical) field

Glasgow et al. (2003)

D-Net An internet-based self-management project aimed at diabetes patients.

+ Medical data

+ Aims at changing health behaviour (investigates attrition rate)

+ Similar attrition pattern

- Not coupled to a app and/or platform

Table 5. Articles used for triangulation purposes

4.3.2

C

OMPANY BACKGROUND

health-38

related information, through which they can make better informed health-related decisions on their daily activities. Hence, it can be concluded that the PHA corresponds with the definition of HIT-platforms provided in section 2.4.1. The PHA was an idea of Hille Meetsma, and was designed with several other experts in the field, which will be explained later. Nevertheless, one can conclude that the criterion of an expert-driven development process, that was stated in the case selection-section (section 3.2.2), is met. Besides the PHA, VitalinQ operates in several other HIT-platform related projects, for which applications are constructed and the content is created. However, their stake in these projects is only small. In section 3.2.2, it was stated that VitalinQ focussed on a one-sided market. This indeed turned out to be true, since VitalinQ makes the platform and the applications themselves. In addition to this, VitalinQ focussed on so-called portals as the sales channels for their PHA. These are firms or organisations which have its own sales-range. Hence, the customer – end-user difference is identified here as well. From these portals, currently several target groups are using the PHA, such as gyms, hospitals and pregnant women.

4.3.3

C

URRENT PROCESSES

4.3.3.1

NPD-Process and Concept Testing Activities

Looking at the overall NPD-process of the PHA, one can conclude that VitalinQ did not follow a highly structured NPD-process, as, for example, is outlined by Crawford and Di Benedetto (2011). As Meetsma argued: “You do not start with an entire investigation and analysis and after four months starting the

project.” When looking at the NPD-process of VitalinQ in terms of Crawford and Di Benedetto (2011),

the Opportunity Identification and Idea Generation came from within the company (i.e. they were based on owned data and own thoughts). The initial concept was derived from an expert-based group, through which one can conclude that their NPD-process was expert-driven. Meetsma explicitly stated: “At a

certain moment you just have to make something that is new-to-the-world. So not something derived from the consumers’ wishes.”

Specifically focussing on the concept testing activities, several things concerning VitalinQ’s concept testing activities became clear. From the beginning onwards, they relied on their own and (potential) partners’ thoughts to develop the initial idea, without integrating the end-user in this. The testing activities performed in the very beginning were conducted with experts (i.e. the project team, potential portals, and experts from healthy-aging), and can be regarded as paper prototyping (i.e. evaluating sketches and drawings) in an iterative manner. Meetsma: “And then it is just a matter of cutting, pasting

and drawing, making a mock-up, again talking with each other…it takes about five or six phases before you have an end-product.” This iterative type of testing is indeed commonly used in the early testing

39

with paper screenshots, based on three iterative cycles. Also Span et al. (2014) evaluated their tool in the first tests based on sketches on paper. Here, the number of iterative cycles is unknown. However, it is very important to highlight that in both studies this stage of low-fidelity prototyping is conducted with end-users, instead of experts.

After several cycles of paper-based prototyping, a working prototypical version of the PHA was constructed. This prototype was tested in a real-life setting. From this point-in-time, the end-user was involved in the evaluation processes of the PHA. In these tests, bus drivers and elderly people tested the PHA in their daily use. The PHA was implemented at this group for a prolonged period, after which it was evaluated. Again, having a trial period for testing and evaluation purposes is an activity that is regularly seen in this sector. Stinson et al. (2013) implemented their application in a group of (fourteen) adolescents for a period of two weeks. In the case of the DecideGuide, a 5-months field study was conducted with nineteen respondents (Span et al. 2015). However, again it should be noted that in both cases irregularities with the approach of VitalinQ are identifiable. To explain, in both approaches at least one extra step was identified between working with paper-based prototypes and implementing a working prototypical version of the HIT-platform in their real-life setting.

4.3.3.2

Problems within VitalinQ

Derived from the interviews, several problems were identified, of which a few will be highlighted here (see the case narrative for an extensive overview). Derived from the first field test, it became clear that there were several severe mistakes in the design of the PHA-concept. For example, in the test with the bus-drivers an application was missing, which was must in the eyes of the bus-drivers. Meetsma: “When

testing it, several things were not functioning well…through which it was not fully appreciated”. In the

(second) field test (with elderly people), he stated: “…it was a bit complex for the target group of 70+.” Besides, the language used in the app, which was medically-driven (i.e. jargon), seemed to be a difficulty. Meetsma: “Later we simplified the language used in the app, a psychologist did that.”

Besides, currently still some problems with the PHA can be identified. The current average usage of the PHA is only two to three months, after which the majority of the users drop-out. Meetsma: “This is

something we have to exaggerate...We’re working with TNO…and we’ve developed several functionalities.” In literature, this is a widely known problem (e.g. Eysenbach 2005; Wangberg et al.

2008). More specifically, similar usage patterns were found in Glasgow et al. (2003), in which the average usage decreased after three months as well. In addition, it was mentioned that VitalinQ does not yet have enough users to reach the break-even point.