Building an information system blueprint to ensure cycle time measurability in the prehospital stroke care pathway Validation of process and data models

(1)

Building an information system blueprint to ensure cycle

time measurability in the prehospital stroke care pathway

Validation of process and data models

Master thesis

Technology and Operations Management

University of Groningen

Author: Natalia Platon

Student number: S2852330

Supervisors: Dr. H. Balsters

Dr. ir. D.J. van der Zee

(2)

2

List of figures

Figure 1.1: Prehospital care pathway ... 10

Figure 2.1: The acute SCP (Lahr et al. 2013b) ... 13

Figure 3.1: The regulative cycle (van Strien 1997) ... 19

Figure 3.2: The regulative cycle adopted by the current study (van Strien 1997) ... 20

Figure 3.3: Overall research’s conceptual model ... 24

Figure 4.1: Developing a descriptive interface ... 29

Figure 4.2: Interrogative user interface ... 30

Figure 4.3: Developing an interrogative interface ... 31

Figure 4.4: Descriptive-interrogative user interface ... 32

Figure 4.5: Interface connection flow ... 33

Figure 4.6: BPMN in Google Forms ... 34

Figure 4.7: ORM diagram ... 36

Figure 4.8: ORM in Google Forms ... 37

Figure 4.9: BPMN model divided in phases ... 39

Figure 4.10: Form Feedback. Usability of the validation method ... 41

Figure 4.11: Form Feedback. Functionality of the validation method... 42

List of tables

Table 4.1: Stakeholder analysis. Goals and CSFs of the validation method ... 27

(5)

5

List of Abbreviations

BPD Business processed diagrams

BPMN Business Process Modelling Notation

CPSS Cincinnati Prehospital Stroke Scale

CSF Critical success factor

CT Computed tomography

DS Design science

ED Emergency department

EDAZ Electronic patient record equipment

EMS Emergency medical service

FAST Face-Arm-Speech-Time

FR Functional requirements

GF Google Forms

GP General practitioner

ICH Intracerebral haemorrhage

IEEE Institute of Electrical and Electronics

Engineers

IS Information system

ISO International Organization for

Standardization

IT Information technology

KII Key improvement indicator

KPI Key performance indicator

LAPSS Los Angeles Prehospital Stroke Screen

MSU Mobile stroke unit

NFR Non-functional requirements

ORM Object-role modelling

PHANTOM-S Prehospital Acute Neurological Treatment

and Optimization of Medical care in Stroke Study

QRM Quick response manufacturing

RAV Regional ambulance service

RHIS Routine Health Information System

SAD System analysis and design

SCP Stroke care pathway

SD System development

SQA Software Quality Assurance

STEMO Stroke emergency mobile unit

tPA Tissue plasminogen activator

UI User interface

UMCGA University Medical Centre Groningen

Ambulancezorg

(6)

6

Abstract

Operations research in healthcare cannot be conducted without data. For this reason, healthcare organizations need to invest in information systems that reflect their business and collect relevant data to support the operational decision making.

A case research was conducted at an emergency healthcare provider in the Netherlands, in order to design an information system blueprint that will illustrate organizations business process in stroke scenario. Respectively, the designed blueprint has the purpose to collect and measure the cycle time performance on the individual patient level. If implemented correctly, the blueprint should support the organization in cycle time analysis and in further operational research to improve the stroke care pathway process.

(7)

7

Preface

This thesis was carried out as a final research project for completing the MSc Technology and Operations Management at the University of Groningen.

I would like to thank UMCG Ambulancezorg organization for welcoming us and for giving us the opportunity to complete this research. Special aknowledgements go to Mr. J. Hatenboer for initiating this project, for taking time to validate the artefact and for making it possible to get in contact with all necessary people to complete this study. Special thanks for Mr. R. Porton and Mr. S. Hazekamp for all the time dedicated to validate and share additional information necessary for understanding the case.

I would also like to thank my thesis advisor Dr. H. Balsters for all his input, guidance and support during this project. Many thanks to Dr. ir. D.J. van der Zee for dedicating his time to provide feedback for the research proposal and the current thesis.

(8)

8

1 Introduction

Stroke is one of the main causes of death among adults (Joutel et al. 1996). Although a successful medically proven treatment exists – intravenous tissue plasminogen activator (tPA), it is administered to less than 8% of patients identified with stroke (Barber et al. 2001, Kleindorfer et al. 2004, Fassbender et al. 2013). The mistreatment is usually caused by delayed hospital arrival (Lahr et al. 2013a, Fassbender et al. 2013, Churilov et al. 2013) and by the narrow time window for tPa administration - less than 4,5 hours (Kessler et al. 2011, Wardlaw et al. 2014, Lahr et al. 2013b). Reducing the time from symptom onset to hospital arrival would be possible in more organized prehospital stroke care processes, meaning: improvements in patients’ stroke awareness (Lacy et al. 2001, Kothari et al. 1997), accurate stroke recognition by emergency medical services (EMS) (Nor et al. 2004, Harbison et al. 2003), the use of standardized prehospital treatment (Kessler et al. 2011), timely prenotification the hospital of arrival (Quain et al. 2008, Casolla et al. 2013) and others. This study is a concurrent research completed by three students and will focus on EMS’s cycle time performance in the prehospital stroke care pathway (SCP). The objective is to enable data collection and monitoring of cycle times on individual stroke patients while the ambulance personnel delivers stroke care. This was achieved by performing a research at an EMS organization (also called ambulance or paramedic service) from the north of the Netherlands - the UMCG Ambulancezorg (UMCGA). An analysis of the prehospital SCP enabled designing the current pathway’s processes in a Business Process Modelling Notation (BPMN). Subsequently, the BPMN model was translated to a data model designed following the Object-Role-Modelling (ORM) language. The data model accommodated the data required to generate cycle times on individual patient level. The last step was to validate, with end-users, the models by means of validation forms. The final validated data models will serve as a blueprint, which if implemented correctly, will enable cycle time data collection. Consequently, this leads to the following research objective:

“An information system blueprint should be designed and validated in order to safeguard cycle time collection and monitoring, on individual patient level, in the prehospital stroke care

pathway.”

(9)

9 design analysis, solution design and design validation (Wieringa 2009) until all end-users’ requirements are integrated in the blueprint. By considering end-users requests towards the final blueprint, a human-engineered artefact is delivered.

The current thesis aims at designing a validation method that will provide relevant and accurate feedback collection from end-users. The validation method shall challenge the designed models by identifying whether it meets user’s requirements. Finally, the validation method should be challenged itself by identifying users comfort when using it. Validation is the final stage in the regulative cycle of this research, but is repeated as many times as necessary. Based on this context, the following research objective is proposed:

“A validation test method should be built in order to safeguard the collection of relevant and accurate feedback, from end-users, about the information system blueprint’s ability to collect

and monitor cycle times in the prehospital stroke care pathway.”

Consequently, the following research question shall be answered to achieve the objective:

“How to design and test a validation method that will gather relevant and accurate feedback on the ability of the designed information system blueprint to collect and monitor cycle times

in the prehospital stroke care pathway?

In order to address this question, the following sub-questions are answered: 1. Who are the stakeholders of the validation method?

2. What are their requirements towards a validation method?

3. How to illustrate the BPMN and ORM models in the validation method? 4. How to collect relevant feedback about the models?

5. How to challenge the users in identifying new requirements, besides the one addressed by the model?

6. How to loop the feedback to process and data design? 7. How to assess the designed validation method?

1.1 Project launching - practical relevance

When this project has been initiated, UMCGA organization (the end-user of the blueprint) presented to us their vision for future ambulance care:

“We see lots of opportunities for Big Data and OR (operations research). In the near future we plan to expand to […] and to models that help us optimize individual workload. And we

(10)

10

“The real challenge for IT systems is not the strategic level, but supporting […] the daily operational level while giving our staff all the necessary information needed to […] that

create happy, competent, motivated, rested and well-equipped Paramedics, EMT’s, Dispatchers, Supervisors and Managers.”

Taking their vision into account, we narrowed down on exploring the ambulance’s SCP. Data that will help assess pathway’s cycle time performance was missing. Precisely, when interviewed, we found that the organization has only one KPI indicator that should be met: 15 minutes from call-to-patient and no KPI is imposed on the rest of the pathway (see figure 1.1 for an illustration).

Figure 1.1: Prehospital care pathway

In the Netherlands (explained in section 2.1.3) an ambulance can usually reach an acute care hospital within 45 minute (Frisian Isles are an exception), for no matter the level of urgency. Therefore, it could not be concluded, how much time does it take for a stroke suspected patient, in particular, to arrive at the hospital. We then have been requested to build a model that will help them collect cycle time data on individual stroke patients.

The objective of this study is in line with ambulance’s vision for three reasons: (1) it facilitates individual workload optimization by enabling data collection and (2) it examines an acute patient flow. As healthcare operations cannot work without data, the implementation of the blueprint will deliver data (3).

The objective of this thesis, however, will ensure that the blueprint conforms to users’ requirements by constantly validating it.

1.2 Academic relevance

Study’s academic relevance lays in the applicability of BPMN-ORM methodology for designing an information system (IS) blueprint in the prehospital setting, which has been recommended as future work by Martena (2015).

(11)

11

(12)

12

2 Theoretical Background

The current chapter will address the identified background that will build understanding of the problem addressed by the objective of the entire project. The content is illustrated using a top-down approach.

Considering the previously identified research objective of the entire study, the first subchapter will focus on the “[..] in the prehospital stroke care pathway” part. It will explain how stroke affects society; how it can be managed by the entire pathway chain; by whom it is managed and then it will move towards the focus of the current research: the prehospital SCP. Lastly it will evaluate EMS’s role in the prehospital SCP - the end-user of the blueprint. The second subchapter will debate upon “An information system blueprint [...] safeguard cycle time collection and monitoring on individual patient level”. It will present how operational research benefits healthcare research and will tackle one operational interest in healthcare - cycle times. Lastly, it will explain how ISs benefit healthcare operations and cycle time management.

The last subchapter will elaborate the “[...] be designed [...]” part. It will explain what and how design science research facilitates IS development.

2.1 Stroke management

In acute stroke management “time is brain” (Saver 2006, Balucani and Levine 2012). For every minute treatment is delayed, an estimated of 1.9 million neurons and 14 billion synapses are lost (Fassbender et al. 2013). Two types of stroke are distinguished: clot (ischemic) and bleeding (intracerebral haemorrhage ICH). tPA is the only medical proven treatment for clot stroke (Lahr et al. 2013b), while recombinant activated factor VIIa (rFVIIa) is the medical treatment for ICH (Mayer et al. 2005). tPA can only be successful when administered in less than 4,5 hours from symptom onset, while rFVIIa must be initiated within the first day (Yang et al. 1994). Most of the researches study the ischemic SCP as it accounts for approximately 87% of all diagnosed strokes (National Stroke Association).

2.1.1 Acute stroke care pathway

(13)

13

Figure 2.1: The acute SCP (Lahr et al. 2013b)

(The current pathway is built by the authors upon Netherland’s healthcare delivery conduct, thus it cannot be generalized to other studies, unless the same sequence of activities and organizational structure are used. Although, the figure displays the ischemic SCP, it is also applicable to ICH diagnosed victims.)

2.1.2 Prehospital stroke care management

As one can clearly notice, there are two way of arriving at the hospital: by self-transport or by EMS transport. Studies that focus on the prehospital phase are conducting attempts to reduce the time to hospital arrival or the onset-to-door time which is the sum of onset-to-call and the call-to-door time. Reductions in onset-to-call stress the importance to raise stroke symptoms awareness among the potential victims (Lacy et al. 2001, Kothari et al. 1997) and educate them on how to behave when one exhibits symptoms (Fussman et al 2010). Patients’ role has been criticized as being one of the main reasons for treatment delay (Pancioli et al. 1998, Rodgers et al. 1999, Slark et al 2012). Studies which investigate the

call-to-door time evaluate the necessity of: accurate diagnose by EMS personnel by means

of various scale instruments (Harbison et al. 2003, Nor et al. 2004); effects of on-time prenotifying the hospital of arrival (Quain et al. 2008, Casolla et al. 2013); increase priority level for patients with stroke (Berglund et al. 2012) and reducing ambulance diversions (Vike et al. 2004, Silka, Geiderman, and Kim 2001, Olshaker and Rathlev 2006).

(14)

14 report improvements in thrombolysis rates (Wendt et al. 2015, Ebinger et al. 2014). Multiple PHANTOM-S studies test the STEMO units and record reduced call-to-needle times (Ebinger et al. 2014, Weber et al. 2013, Ebinger et al. 2012). Despite improvements in thrombolysis rates, these studies lack generalizability as stroke is handled differently across countries. Variations between countries are due to differences in resources, stroke expertise, available technology and followed protocols (Lahr 2013b). Moreover, the lack of generalizability is strengthened by the fact that these studies are experimenting with tPA treatment only. The intrahospital studies concentrate on how the door-to-needle times can be managed. Since this is not the focus of this research, it will not be elaborated further.

2.1.3 EMS’s role in the prehospital stroke care pathway

This research will investigate EMS’s role in the prehospital SCP. In the prehospital stage, the patient is influenced by: the control centre and the EMS. The control centre is represented (figure 2.1) by the “911” activity. The control centre has a coordinator role as it establishes, via phone, the type of emergency, which EMS should be dispatched to localize the victim and monitors the ambulance during the entire journey. The control centre receives calls either from the patient, a by-passer, patient’s relatives or from a GP. However, control centre’s pathway will not be further investigated.

The prehospital phase has received an impressive attention in acute stroke management for the reasons mentioned in 2.1.2. EMS represents a great potential for optimizing the management of stroke care (Fassbender et al. 2013). Although, considering the different EMS structures among studies, applying the results of one study to another setting becomes difficult, especially because of the type of response-to-stroke the EMS is taking. According to Fassebender et al. (2013) two strategies can be employed: paramedic-based and

physician-based. The paramedic-based response is focused on transporting the victim to the hospital

as fast as possible. This concept is known as scoop-and-run and it usually requires less medical expertise from the EMS’s personnel. The physician-based response requires more medical expertise, as it aims at moving hospital’s role earlier in the process (on scene). This concept is known as stay-and-play. Fassebender et al. (2013) point that the physician-based response requires an additional physician while EMS is on scene.

(15)

15 “EMS response” groups the activities performed after the emergency call from the control centre has been received and before the ambulance arrives at victim’s location. These activities include departure from the EMS station and travel to the scene.

“EMS on scene” groups the activities that relate to the need of early symptom recognition by means of scale instruments such as Cincinnati Prehospital Stroke Scale (CPSS), Los Angeles Prehospital Stroke Screen (LAPSS) or Face-Arm-Speech-Time Test (FAST) (Harbison et al. 2003). In most cases, an early symptom diagnose is done prior the ambulance arrives on scene, by a GP or by the control centre via phone. In any case, the EMS should reconfirm or reject stroke and decide whether hospitalization is needed. These require adequate training for EMS’s personnel (Kessler et al. 2011). An accurate early diagnose would decrease overcrowding the SCP by eliminating stroke mimics or other diseases (Olshaker and Rathlev 2006).

“EMS transport” groups the activities which are performed in the ambulance, while the patient is being transported. EMS’s responsibilities at this stage relate to priority transportation to a stroke-expert centre and on-time prenotification of the receiving hospital (Fassbender 2013). EMS transport has been related to a shorter door-to-needle time in comparison to self-transport (Lacy et al. 2001). Prenotification is associated with an improved in-hospital evaluation time, earlier stroke treatment and more patients eligible for treatment (Lin et al. 2012). Additionally, communication between EMS personnel and the receiving hospital is encouraged. The communication should be a means of data exchange about patient’s situation. Kessler et al. (2011) report that it is EMS personnel’s responsibility to hand information gathered in the prehospital phase to the hospital team. The information should report: time of symptom onset, type of complaint, relevant parallel diseases, phone number of relatives, administered medication, symptoms’ evolution and if there are any stroke treatment contradictions (Kessler et al. 2011). All these should start being gathered by the paramedic personnel: on-scene or while transporting the patient.

2.1.4 EMS management in Netherlands

In the Netherlands an EMS can be called at 112 emergency number. According to Volksgezondheidenzorg (2015a) there are 215 ambulance stations spread across 25 regions. In total, 91 hospitals have an accident and emergency department open 24/7 (Volksgezondheidenzorg 2015a). There are 11 trauma medical centres among which 4 poses a helicopter used in complex accidents with multiple victims. Usually, the EMS personnel consist of one driver and one registered nurse.

(16)

16 reach an acute care hospital within 45 minutes (except Frisian Islands). Stroke is being prioritized as level A1 emergency and becomes a high priority for both the EMS and the receiving hospital. Considering Ambulances in-zicht’s report (2013), 92,6% of the A1 patients were reached within 15 minutes. They also report that 47,6 % of the total emergencies required A1 emergency treatment, out of which 7% demanded neurological care (including stroke). Thus, due to the high ED density around the country, Netherlands adopts a scoop-and-run stroke approach, alongside USA, UK and Sweden(Fassbender et al. 2013).

According to the Landelijk Protocol Ambulancezorg (2014), FAST is the scale instrument used by the EMS personnel to detect stroke. The test assesses face weakness, arm weakness, speech disturbance and time of symptom onset. If any of these is abnormal, the probability of stroke is 72% (ACLS Suspected Stroke Algorithm 2016). FAST scale has a 78% positive predictive value and it was designed for simple and quick stroke detection, as compared to the more complex CPSS and the LAPSS(Harbison et al. 2003).

Each Regional Ambulance Service (RAV) is affiliated with an enterprise which safeguards information exchange between EMS and hospitals. Regarding this research, UMCGA is affiliated with Axira. Axira equipped 120 ambulances with an Elektronisch Dossier AmbulanceZorg (EDAZ) server (UMCG Ambulancezrog website). EDAZ is an installed system which enables patient information exchange between the ambulance and the receiving hospital.

2.2 Operations research in healthcare

(17)

17 Operations in healthcare adopted many manufacturing operations practices: lean approach in reducing waste, costs and lead time while improving quality (Mazzocato et al. 2010,

Poksinska 2010, Joosten, Bongers and Janssen 2009, Young et al. 2004); the introduction of IS to monitor, collect and store data (Agarwal et al. 2011, Lenz and Kuhn 2004); performance management, quality assurance and compliance; planning and scheduling; capacity management; optimization of hospital operations using simulations, analytical and descriptive tools (van Sambeek et al. 2010).

2.2.1 Cycle Times in healthcare

In both manufacturing industries(Smith and Reinertsen 1998) and in health care services (Porter 2010), cycle times are a critical outcome for patients and not a secondary outcome as some believe. Cycle times reflect on how fast a service/manufacturing company can respond to customer demands (Harter, Krishnan and Slaughter 2000). Porter (2010) adds that reducing cycle times, in healthcare services, improves functionality and reduces complications (known in manufacturing as quick response manufacturing – QRM philosophy (Suri 1998)). Vice-versa is also applicable: by reducing non-value adding services, reduction in costs and artificial variability, cycle times become shorter, product/service quality improve, the capacity is better used and efforts are channelled in the right direction (Porter 2010)(this is usually known as lean manufacturing philosophy (Suri 1998)). There are multiple costs related to large cycle times in manufacturing literature. In healthcare services, larger cycle times lead directly to larger work in progress (according to Little’s Law); negatively affect the competitiveness and “image” of the organization and can lead to life-threatening delays by displaying poor responsiveness (Karmarkar 1987). Besides the crucial need of reducing time-to-treatment in acute care, cycle times are also a key performance indicator (KPI) and a key improvement indicator (KII) (Setijono and Dahlgaard 2007).

2.2.2 Information systems in healthcare

“We can only manage what we measure” (Santiago 2014). Healthcare operations cannot operate without data. This is why organizations need to invest in ISs. In healthcare, ISs are any system that collects, processes, analyses and transmits data related to patients or activities performed by a healthcare organization (Lippeveld 2001). Data has been defined by

Abdelhak et al. (1996) as “a representation of facts, concepts or instructions”. Information is the “knowledge derived from data” (Kroenke et al. 2008:24).

(18)

18

to help decide, on national level, which improvements have to be made in country’s health system(Hotchkiss, Diana and Foreit 2011).

2.3 Design Science

The process of creating and maintaining an IS is called system development (SD) or system analysis and design (SAD) (Kroenke (2008:234). As the overall study’s objective is to design an IS blueprint, the principles of design science (DS) are elaborated further.

“Design is concerned with how things ought to be in order to attain goals and to function” (Simon 1996:7). DS is a problem solving paradigm that aims at building utility-driven artefacts (Holmström, Ketokivi and Hameri 2009). Simon (1996:6) defines an artefact as something produced by art or people, rather than by nature. Theory, on the other hand, is a pure-knowledge problem that aims at finding the ultimate truth by observing what is known (Balsters 2015a). DS is the research relevant to practice and it adds great value to operations studies, by solving real world problems, rather than testing theory (Holmström, Ketokivi and Hameri 2009).

(19)

19

3 Methodology

This chapter will illustrate the adopted methodology for (1) designing the information system blueprint – the research objective and (2) for designing the validation test method – the individual research objective.

3.1 Overall research project’s methodology

This subchapter examines the methodology behind the entire project. It will first present the principles of group work (concurrent engineering) and the role of the regulative cycle to this group research. The regulative cycle will ensure internal “coordination” while concurrent engineering will justify the principles of individual, but interactive group work.

Next, the methods applied to designing the blueprint are explained. By clarifying how process and data were modelled - the last part of the research objective that was not covered in the theoretical background will be answered. Lastly, before moving to the conceptual model, the requirements engineering of a successful design are described.

3.1.1 Regulative cycle

The design of an IS artefact is a systematic trial-and-error practice. This characteristic is due to the practicality and newness of the problem. Within all design processes, end-users have to be involved in order to assess if the outcome is the one desired. End-users continuous feedback causes the models to be constantly redesigned. In order to keep this repetitive process under control, a methodology needs to be followed. Van Strien (1997) developed a systematic procedure that disciplines practical problem solving. As research aims at exploring and solving a practical problem, van Strien’s regulative cycle was used.

(20)

20 As a remark to van Strien’s regulative cycle, in a repetitive design where the models are continuously changed, the researchers validate the model without implementing it (Balsters 2015b). Implementation, in this case, will be done by the IT developers, if the end-users will consider the design useful. Consider figure 3.2 as the cycle applied to the entire research.

Figure 3.2: The regulative cycle adopted by the current study (van Strien 1997)

3.1.2 Concurrent engineering

As design science has its roots in engineering it borrows a lot from engineering principles. One of them is the concurrent work. The development of complex products usually requires more people to work together. In concurrent/simultaneous engineering the development process is done in parallel rather than in sequence, in order to decrease the cycle time of product development (Sohlenius 1992). However, because of the interdependencies between activities, parallelism causes designers to progress with partial information, incomplete knowledge and subjective interpretations (Prasad, Morenc and Rangan 1993). Loch, Mihm and Huchzermeier (2003) report that the growth rate of the concurrent projects exponentially affects the design to deviate from the goal. As this cannot be eliminated, but only managed, they suggest: limit system’s size; minimize interdependencies and divided the work into smaller parts; cooperate and compromise.

(21)

21

3.1.3 Requirements engineering

Requirements engineering is the first step of product development. Their identification, analysis, modelling and validation safeguard product’s success. Requirements are being collected both at the beginning and throughout the entire design process (via feedback). Because of the multitude of definitions, this research will illustrate the one approved by the IEEE (1990): “Requirement: (1) A condition or capability needed by a user to solve a problem or achieve an objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents. (3) A documented representation of a condition or capability as in (1) or (2)”. Contractual or not, requirements have to be an agreement between the users and the designers.

Finally, it is necessary to distinguish between functional requirements (FR) and non-functional requirements (NFR) (Selvakumar and Rajaram 2011). FRs deal with requirements that affect the functionality of the system and answer the question “what the system should do?” (Balsters 2015a) while NFR are requirements that constraint the system and address “what the system shall be?” (Balsters 2015a). It is commonly accepted (Dabbagh and Lee 2014, Capilla, Babar and Pastor 2012, Selvakumar and Rajaram 2011) that NFR are quality attributes which describe the system: reliable, usable, flexible and so on. As much as the importance of FR are crucial, NFR, however, are harder to incorporate. Many designs have not been implemented, despite their functionality, because the non-functionality was poorly considered (Chung and do Prado Leite 2009). That is why, you can usually hear: “it was not

useful enough”, “it was not easy to operate”, “not user-friendly” etc. These are the reasons why well founded agreements upon the NFR have to be made.

3.1.4 Business Process Modelling – BPMN

Healthcare organizations, as any competitive service delivery organization, need to recognize their business processes in order to manage them. Healthcare processes are the fundament of decision making, which when well operated, help organizations achieve their goals (Rolón et al. 2015).

Process modelling describes how business activities relate and interact with organization’s resources to achieve a goal (Rolón et al. 2008). They serve as the fundament for the creation of appropriate IS to support business operations.

(22)

22 illustrate the business processes (White 2004). In healthcare these are called clinical process models (Fox and Dunlop 2008). The advantage of BPMN is that it provides a BPD which uses simple notation language, but at the same time, complex enough to be used by technical and non-technical experts (White 2004). Chinosi and Trombetta (2012) identified that BPMN is the most used tool for mapping business processes and the preferred BPMN editor is BizAgi (13%). This is the same tool this research used. The reader is kindly advised to refer to Gonçalves (2016) for more details.

3.1.5 Data Modelling – ORM

Practice indicates that often there is no link between the business processes and its business data (Balsters 2013). This often leads to databases that do not fully integrate the business process, thus do not guarantee business data quality. Balsters (2013) has offered a methodology to fill this gap: Fact-Based Business Process Modelling (FB-BPM); and Martena (2015) has validated its successful applicability.

The data model notation used for this project is ORM. ORM is a fact-based modelling approach that was chosen because it uses natural language to model data. As Halpin and Morgan (2010) explain “ORM views the world simply in terms of objects (things) playing roles (parts in relationships)”. In ORM, a role is a part played in a fact-type relationship. This goes in line with the idea than when validating a system “information should be examined in the smallest units possible: “one fact at a time” (Halpin and Morgan 2010). Therefore, ORM’s capability of verbalizing facts makes validation with end-users (non-technical experts) easier (Balsters and Halpin 2008).

The modelled data, based on the modelled processes will serve as the blueprint of the study. Thereby, the modelled data flow is critical to be correctly validated. Due to an ORM feature – Verbalization Browser, which verbalizes the interaction of data, the validation process becomes less ambiguous. Moreover, ORM displays a relational view which also facilitates the validation process.

In order to better understand how the BPMN models were translated to ORM models, by using the ORM2 notation; refer to (Rouby 2016).

3.1.6 Validation of process models and data models

(23)

23 Design validation is a task in which the specified design is assessed whether it would bring stakeholders closer to their goals if implemented correctly (Wieringa 2009). Whenever the design does not conform to end-users’ requirements, the regulative cycle starts again. One way to justify the designed solution would be to implement and test it; however, in a real world problem this is not possible, as stakeholders need sufficient proof that the design implementation will solve the identified problem (Wieringa and Heerkens 2006). As a matter of fact, validation is bridging means and ends, by communicating the design’s technique to end users and by looping back the feedback in the design process cycle.

Although a lot of studies declare validation as a crucial step to legally confirm the research as satisfactory, there is not enough clarity on what tools were used and how the validation process has been conducted. Moreover, van Sambeek et al. (2010) confirm that only 25% of the studies that developed models to optimize hospital processes have been validated. This is a very unfortunate finding as building the model usually becomes more important than using it. Zelkowitz and Wallace (1997)explain that the number of invalid researches wouldn’t be so high if specific and accurate validation techniques would be available, as in other fields which adopted validation successfully.

Although there is a considerable amount of research on testing, verification and validation tools of simulation models (Sargent 2005, Balci 1997, Kleijnen 1995), these tools cannot be adopted to the current research as they do not match with BPMN’s and ORM’s natural language.

Finally, validation in this thesis takes the roles of verbalizing, illustrating and explaining the designed models to end-users and then verifies whatever it meets the specified requirements, by collecting feedback. For this reason, the proper tool that will enable verbalization, illustration and feedback collection, has been considered Google Forms (GF).

3.1.7 The conceptual model of the overall research project

(24)

24

Figure 3.3: Overall research’s conceptual model

The entire study was subject to the regulative cycle; however, for building the validation method a design cycle was applied as well (see section 3.2.3).

3.2 Individual research thesis’s methodology

This section explains how GF will help attain the individual objective; how data was collected and, finally, will illustrate the conceptual model of the individual thesis.

3.2.1 Google Forms Background

GF is an online survey tool provided in the Google Tool Kit account. It is widely used in literature as means of improving learning experiences (de-la-Fuente-Valentín, Pardo and Kloos 2011, Gehringer and Cross 2010), collaborative experiences (Fransen, Kocher and Kempf 2011, de la Fuente-Valentin et al. 2010), e-learning (Roseth, Akcaoglu and Zellner 2013) etc. GF is not only a questionnaire tool, but a tool to deliver questionnaires and manage answers. GF’s questionnaire provides a user interface (UI) for entering data that in the background is directed to a Google spreadsheet (de la Fuente-Valentin et al. 2010). GF are very short cycled as the spreadsheets (which contain the feedback) can be immediately accessed. Therefore, feedback can be evaluated and looped back to process and data models quickly. This is expected to ease and fasten changes to the designed model.

(25)

25 The forms also enable visualization of text, images and videos through a UI display. Basically, by these means, the process models and data models are verbalized and explained, then, by means of the questionnaire, it will check users’ understanding and model’s correctness. Needed to mention is that the UIs built through GF are not IT driven, thus, they are not a potential interface of the final design implemented in software, but are interfaces that will let the user communicate with the designed models.

To sum up, the role of GF in the validation stage is to verbalize, illustrate, build understanding, test understanding, validate correctness and requirements of the models and assess the form itself.

3.2.2 Data and feedback collection

According to Karlsson (2010), instruments for case study research are un/semi-structured interviews, conversations, survey, observation and archival sources. In a survey study, however, data collection can be done via face-to-face, mail, web or telephone interviews. Because this research builds a new product and does not test an existing one, case study was used for investigating the current system and the GF (a survey tool) was used to validate and collect feedback data.

This research used unstructured and semi-structured face-to-face interviews as well as emails for problem investigation. Unstructured interviews allowed the designers to have open debate with the end-users, but without deviating from the subject. Semi-structured interviews were used when the models were under development in order to collect additional information. Email was also a mean of communication as face-to-face meeting were not always possible.

Structured web surveys were used for validating the models. The structure was safeguarded by the validation forms. The online validations were performed by allowing the user to access an URL link of the form.

3.2.3 The conceptual model of the individual thesis research

(26)

26

4 Results

The current section will report on how validation contributed to the entire study by addressing the research objective of the current thesis. It will report the results following the conceptual model of the current thesis and the guideline for reporting practical problem research proposed by Wieringa (2007) and elaborated by Balsters (2015b) (appendix E).

In order to build a reliable validation method, the stakeholders of the validation method were identified. Their goals and requirements towards the validation method were analysed and incorporated in the developed forms. When the forms were implemented, end-users evaluated the BPMN and ORM models. The designed validation method served as a communication bridge between models’ designers and end-users.

4.1 Design problem

The problem identified by this thesis is the lack of consistent method to accurately validate BPMN-ORM models without implementing them. But, as this currently is not possible, the models have to be validated in a manner that will ensure a successful implementation. Lastly, in order to build a human-engineered validation method, the method will require feedback as well.

(27)

27

Table 4.1: Stakeholder analysis. Goals and CSFs of the validation method

As one can notice, there are duplicate CSFs, thus, consider the following full list of CSFs and their interpretation:

1. Accurate – the method should be capable to deliver what has been designed for (the individual research objective). Thus, it should illustrate the models properly and unambiguously.

2. Prompt – the short time available to perform this research requested fast feedback incorporation in the design cycle. Thus, the manager was interested in project’s progress while the designers were concerned about a fast model assessment. 3. Easy to understand – the ability of the validation method to easily explain the BPMN

and ORM models.

4. Ease of use – the ability of the method to be easily operated.

5. Concise – “brief but comprehensive”. As the first validation round turned out to be longer than expected, this CSF was added later.

6. Engaging – the ability of the method to encourage the end-user to be pro-active and come with suggestions.

7. Effective – capable of accomplishing its purpose: to collect accurate and relevant feedback.

4.2 Diagnosis analysis

After a CSF analysis, there is a clear trade-off between the method conciseness and accuracy. Although “conciseness” was requested by the user, it was pondered that validation

Stakeholder Goal

A validation method that:

CSFs

The validation method should be:

Project manager

1. Represents well blueprint’s ability to

measure cycle times

2. Communicates what the blueprint represents in a simple manner 3. Will enable quick model redesign 4. Will challenge judgemental feedback 5. Is not difficult to use

6. Be brief, but comprehensive

1. Accurate 2. Easy to understand 3. Prompt 4. Engaging 5. Ease of use 6. Concise

Data analyst 1. Describes well the data flow in the process

2. Communicates what the blueprint represents in a simple manner 3. Not require much time to evaluate

1. Accurate

2. Easy to understand 3. Concise

Model designers

1. Which accurately represents the models 2. Collects relevant and accurate

feedback

3. Loops the feedback fast

(28)

28 method’s accuracy is important for the blueprint and cannot be compromised. GF unfortunately, does not provide a feature to save the progress in the form and continue later. This disadvantage will be mitigated by introducing a “progress bar” in the forms (explained in subchapter 4.5) which will inform the user of an approximate time required to validate. This trade-off disappears after multiple validations, when the users become familiar with the process.

When analysing the list of CSFs, a solid pattern has been identified among three NFRs. According to SQA, the “ease of use”, “ease of understanding” and “engaging” capture requirements necessary for the development of user interfaces (UI). ISO 9126 comprises them in one term: usability. Usability is the capability of the product to be attractive, understood and operable. These can be assessed by testing user’s satisfaction with the product (Maguire 2001).

However, a system cannot hold usability characteristic unless it functions properly. The functionality can be assessed by the FR. Therefore, the functionality of the designed method is connected to the objective of this thesis: the validation method should collect relevant and accurate feedback. The method will be successful by addressing user’s requirements towards its functionality: be accurate, prompt, concise and effective.

4.3 Design solution

Google Forms has been already used as a validation tool in a BPMN-ORM design approach by Oldenburger (2015). The current’s research forms will be quite different because of two reasons: (1) the activities of only one internal stakeholder are examined – the EMS and (2) some forms had to be more descriptive, and use simpler language than others, because no face-to-face validation sessions were possible to schedule (as opposed to Oldenburger’s validations where the use of only face-to-face sessions enabled guiding the user through the forms). In addition, author’s method of applying GF to BPMN-ORM validation was not available during the development of the forms for this study, thus some aspects will be approached differently.

4.3.1 Definitions, classification and technical aspects of Google Form

development

A form developed in GF is a set of digital “pages”. A form used by a user, however, is a user interface (UI).

(29)

29 1. Descriptive interface - used for guiding the user through the form (1) and explaining,

through text and images, the models (2).

2. Interrogative interface - used to test users’ understanding of the models and to validate model’s accuracy of meeting the specified requirements.

Sometimes a combination of both descriptive and interrogative interfaces was used. This was called: descriptive-interrogative interface.

Any interface will display a part of the pathway that needs validation:

1. Phase: the modelled pathway was divided in some phases. Each division comprised a set of activities which were further displayed one by one.

2. Activity. Each activity within the pathway that was requesting validation.

4.3.1.1 Development of descriptive User Interfaces

Figure 4.1: Developing a descriptive interface

(30)

30 1. Title – displays the name of the point in the pathway the user is validating (either an

activity or a pathway phase).

2. Description - used for detailing through text, the acronyms used in the models Two types of descriptive approaches were used in the descriptive interfaces:

- Text - used for guiding the user through the form, describe the displayed image or illustrate the ORM model (detailed in section 4.3.3).

- Images - print screens of the BPMN model were used in order to explain to the

user which activity from the pathway was described and required feedback.

4.3.1.2 Development of interrogative User Interfaces

Figure 4.2: Interrogative user interface

On a purely interrogative interface, only questions were displayed. Figure 4.2 illustrates an example of an UI (how the user sees it). It displays three questions: one multiple choice and two open questions.

(31)

31

Figure 4.3: Developing an interrogative interface

Figure 4.3 displays how the first question from figure 4.2 was developed. In GF, for each question, the UI designer can complete:

1. Question title - a mandatory field for typing the question itself.

2. Help text - not mandatory, but preferred, especially when the author uses a mix of single and multiple answer questions (informs the user of how many answers they can select).

3. Question type - this is where the designer selects:

- Single choice - in this research there were usually dichotomous questions

(yes/no) or trichotomous (satisfied/dissatisfied/neither).

- Multiple choice - allowed the user to identify more than one mistake in the models. - Open-ended - designed for identifying omissions which are very dangerous and

difficult to discover (Adrion, Branstad, and Cherniavsky 1982). These were also used for collecting explanations about specific aspects which were not accurate.

- Likert 6 point. An even number was selected because research affirms that an

(32)

32 Note that in GF, the names for each question type are different, for example: single choice questions in GF are called “multiple choice”, multiple choice are “checkboxes”, open-ended – “paragraph” and likert scale are “scale” as well.

4. Answer text. This is where all the answer options are written. With this representation the author also ensures answers coding. Answer coding is applied by assigning symbolic codes to open-ended questions (Kaiser, Barghouti and Sokolsky1990). Despite the fact that the forms will use open-ended questions, their answers do not necessitate coding because each open-ended question is preceded by single or multiple choice questions. Hereby, the user gives comments only for selections made above. For example, in figure 4.2, the second open-ended question asks what changes should be made for the answers selected above.

5. Required question. This should be chosen upon necessity.

4.3.1.3 Development of descriptive-interrogative User Interfaces

A combination of a detailed-interrogative UI will be visualized by the user in the following manner (figure 4.4):

(33)

33 Figure 4.4 displays both a description and a question. When developing a form, the digital pages should be connected such that the flow of the interfaces is dependent on users’ answers. Consider a simple example (figure 4.5).

Figure 4.5: Interface connection flow

Depending on the answer to this dichotomous question, the user will be redirected to a different interface. For example, if the user agrees with the representation of the start event in the pathway and answers “yes”, the user will be redirected to the next activity in the pathway: “prepare to leave to the scene”, otherwise, the form will continue to an interrogative interface (page 7) to assess why the user answered “no”.

4.3.2 BPMN validation in Google Forms

(34)

34

Tabel 4.2 BPMN elements (Rolón et al. 2008)

Print screens of the pathway modelled in BPMN were displayed on each interface.

(35)

35 As one can see in figure 4.6, in the description field, a context description was given for each activity, decision point (gateway) and pathway phase. They were used for explaining what the BPMN acronym represents.

4.3.3 ORM validation in Google Forms

By using the sequence of activities provided by BPMN, each activity from the ORM model was validated by providing a detailed description of the data flow designed in ORM.

If one wants to communicate the ORM model to the end-user, a detailed description of the data flow should be presented. This can be done following the next instructions:

1. “is performed by” represents the stakeholder that performs the activity or the swimlaness from BPMN.

2. “leads to” reflects the connections between BPMN flow objects. For example: at-scene-arrival leads to triage-performing.

3. “has” connects flow objects with input, output and the derived cycle time (in this study).

4. “start at” indicates when each activity starts. 5. “end at” indicates when activity ends.

6. “Instant” in this study, enabled cycle time measurability by recording the exact time each activity was performed.

7. “x” and “=” represent a Boolean choice (yes/no) modelled in BPMN as gateways (see

table 4.2). “x” is followed by a “fail to” fact type and “=” by a “success” fact-type.

8. “*” represent when values are derived and “**” – when values are both derived and

stored “**” derivation rule, in this study, was used for cycle time values measurement and collection.

9. “Description” each BPMN event (start and end) have an ORM description.

10. Role is a part played in a fact-type relationship. Relationships can be unary (one role), binary or tertiary. They are indicated in ORM by the single, double or tertiary boxes between associations.

11. Mandatory roles are indicated by the solid dot above relationship boxes. A role that is

not indicated as mandatory is optional.

12. Unique constraints are represented by the line above relationship boxes and they are

performed only once.

(36)

36 Considering these, an ORM can be communicated by indicating for each activity necessary conditions (pre and post), inputs or outputs. These eliminate the chance of the data model to be misinterpreted.

1. Pre-condition is a logical expression (true or false) that describes the state of the

database before an activity is initiated. The pre-condition must be true for the activity to start(Kaiser, Barghouti and Sokolsky 1990).

2. Input is the data the database requires prior to the start of the activity. In this study,

inputs were either a timestamp (see indication 6) or a text annotation (incorporated in artefacts) which provided additional information about the objects.

3. Post-condition is also a logical expression made true by completing the activity

(Kaiser 1989). In this case, it is a characteristic of the results or a description of the status change of the database after the activity was performed.

4. Output is the data the activity will deliver for ensuring process progress.

In the data model built by Rouby (2016) the pre-condition for each activity was the termination of the previous activity. Inputs were necessary when ORM indicated “has”, “start at” and “end at” fact-types and outputs were delivered when CycleTime was derived. Post-conditions were describing the output, for example:

(37)

37 From figure 4.7 - “Leaving (.nr) start Instant” and “leads to AtSceneTravelandArrival(.nr)” which “ends at Intant” and “has CycleTime” which respectively “has TraveToScenelTime” will be reported in the context description as follows:

Figure 4.8: ORM in Google Forms

Note, the activity does not deliver output as no cycle time is being derived at the end of this activity. The cycle time for this phase will be derived when the ambulance arrives at the scene – “AtSceneTravelAndArrival”. See in figure 4.7 the derived “TraveToSceneTime”. By recording as an input the time when the ambulance started driving to the scene “Leaving

(.nr) start at Instant” and when the ambulance arrives at scene “AtSceneTravelandArrival(.nr) end at Instant” the cycle time of travel-to-the-scene time will be recorded. Therefore, the next

formula was used by the ORM model to derive cycle times:

Cycle Time of a phase = last activity end – first activity start.

4.4 Design implementation

This section will report how feedback enabled progress in this entire study.

(38)

38 The final BPMN built by Gonçalves (2016) can be found in figure 4.9, but the detailed elaboration - in his thesis. When communicated to the end-user, the prehospital SCP was divided in four distinct parts (figure 4.9):

1. Preparation Time

2. Travel-to-the-scene Time 3. On site Time

4. Transportation of the patient

These parts are the four cycle times that the implemented blueprint (appendix B) will deliver. Additionally, the total cycle time is derived from the time the first activity started until the time the last activity ended (modelled in BPMN as “the black box”). The ORM model was reported following the data flow modelled by Rouby (2016) (see appendix A for the final ORM model).

4.4.1 Feedback

The feedback gathered during validations enabled Gonçalves (2016) and Rouby (2016) to redesign the models each time new requirements were identified. Each BPMN and ORM model was built in order to meet stakeholder’s requirements towards the blueprint. End-users feedback confirmed when they did or did not.

A separate form was designed to validate blueprint’s stakeholders, their goals and CFSs. Due to the collected feedback, Gonçalves (2016) has identified 1 internal stakeholder and 9 external stakeholders interested in the EMS’s cycle time performance. The internal stakeholder – the end user of the blueprint - is the UMCGA organization, while the external stakeholders are not involved directly in the process, but their goals affect it. Refer to appendix D for a full stakeholders, goals and CSFs lists and to Gonçalves (2016) if more details are needed.

(39)

39

(40)

40 The collected feedback enabled Rouby (2016) to model the final data flow model. Also, due to feedback, Rouby (2016) has recognized the four distinct cycle times that the system shall deliver. The re-designed BPMN models have also affected changes in ORM.

4.4.2 Concurrent engineering

Before any form was sent to the end-user, it was first validated by models’ designers to ensure that it accurately illustrated what they modelled. In this step model designers’ feedback was integrated. Consequently, whenever BPMN was passed to ORM and BPMN-ORM to validation, multiple discussions took place which internally started the regulative cycle without validating the models with the end-user. These discussions ensured that the information collected in the interviews was unanimously interpreted. This was an advantage of concurrent work. The forms approved by models’ designers were forwarded electronically to the end-user.

4.4.3 Assessing solutions integrity in the stroke care chain

In order to take an integrative approach, two more interviews have been performed. One with a medical expert in order to establish the correctness of the medical procedure described in the models and the second with an UMCG research manager of the neurology department to identify possible exceptions that can relate to the current representation of the process (cases of ED overcrowding that will cause ambulance diversion). The medical expert confirmed the identified sequence of activities. The research manager explained that very rare cases of overcrowding at the UMCG ED happen, as A1 emergencies are always prioritized and the facilities reserved for diagnostic and treatment are readily available. Thereby, he confirmed that there is no need to complicate the models as these cases happen very rarely.

4.5 Design validation

This section will report if the designed validation method was able to fulfil stakeholder’s requirements. This section supplements the work of Oldenburger (2015) by suggesting a method to test the usability and functionality of GF validation forms.

4.5.1 Usability of the validation method. Non-functional requirements

(41)

41

Figure 4.10: Form Feedback. Usability of the validation method

(42)

42

4.5.2 Functionality of the validation method. Functional requirements

Figure 4.11: Form Feedback. Functionality of the validation method

This interface (figure 4.11) was developed to test the functionality of the form if the user was dissatisfied with it. Thus, the first and the third questions assessed whether the developed form didn’t meet the “be concise” CSF. By informing the user of an approximate time to complete the form and by introducing the progress bar (see at right-bottom of the interface) this CSF was mitigated. The second assessed user’s opinion about “accuracy” of the form in illustrating the BPMN and ORM models. Also, the second question was an indication that the form was still not easy to understand (a NFR). The forth question tests whether the dissatisfaction is still caused by a NFR (the ease of use) and the fifth question was introduced in order to assess if user’s dissatisfaction is not related to the form, but is caused by the inability of the models to measure cycle times (overall research objective).

(43)

(44)

44

5 Discussion

This research explores the EMS prehospital SCP by delivering to an EMS organization a validated IS blueprint. When implemented, this blueprint will enable data collection of cycle times, on individual patient level, for a set of activities within the identified pathway.

The final approved process model is illustrated in figure 4.9, the final data model – in appendix A, the ORM relational view (the blueprint) – appendix B and examples of validation forms - accompanied by a detailed development procedure - in section 4.3. All stakeholders, their goals and CSFs towards the blueprint are listed in appendix D.

5.1 Reflections on the individual study

The problem encountered by the current thesis was that no consistent validation method was available to validate an IS blueprint built with a BPMN-ORM approach. In order to address this literature gap, a research question was determined followed by seven sub-questions.

1. Who are the stakeholders of the validation method?

Three stakeholders were identified as interested in the validation method: the end-users and the models’ designers. Although the end-user is illustrated as a single stakeholder of the blueprint, it was considered as a “duplicate” stakeholder for the validation method (see section 4.1).

2. What are their requirements towards a validation method?

As illustrated in the results, BPMN-ORM designers are mainly interested in the functionality of the method, opposed to the end-user – whose requirements also address its non-functionality. Their requirements have been detailed in section 4.1.

3. How to illustrate the BPMN and ORM models in the validation method?

(45)

45 simply passed to the next phase within the cycle, but it was assessed at every step repetitively. This approach was of great help since many investigations (literature research, exchange of emails between the designers and the end-user etc) were done before requesting additional feedback through the validation forms.

4. How to collect relevant feedback about the models?

The strategy taken for this study was to anticipate what can go wrong for each activity and write those options in the multitude of answer the user can choose from. Through this, the user is challenged to be engaged and not just provide yes/no answers. Figure 4.2 illustrates an example of this idea. The “relevance” of feedback was the most important for model designers, as mainly this type of feedback enables building robust models.

5. How to challenge the users in identifying new requirements, besides the one addressed by the model?

Because of many limitations, the forms have been validated online only. Face-to-face validations were done at the beginning, following the suggestions published by Wieringa (2007) (appendix E) in order to collect additional information to accurately capture the existing business process, but forms were not used since models were unfinished. Also, at this step, user’s requirements towards the validation method were gathered by understanding their familiarity with BPMN and ORM methodology, by explaining to them what a validation form will look like, by agreeing on the time available for validating the models and their expectation towards it.

Online validations, besides having great advantages, they do not always engage the user. It was observed that open-ended questions were never completed unless they were mandatory. Additionally, the answers just indicated the mistake, without providing explanations. If needed, things were clarified in face-to-face meetings or via emails. Thus, this becomes an unintended limitation of this study.

6. How to loop the feedback to process and data design?

GF’s feature of immediately redirecting the feedback in a background spreadsheet enabled shorter design cycles. The spreadsheet was immediately accessed and structured in a Word document. The feedback was discussed and analyzed whether new interviews were needed to collect additional information for the problem design.

Building an information system blueprint to ensure cycle time measurability in the prehospital stroke care pathway Validation of process and data models