• No results found

mHealth Framework: for the digitalisation of a healthcare facility. Extension of the Labrique et al. (2013) mHealth and ICT framework

N/A
N/A
Protected

Academic year: 2021

Share "mHealth Framework: for the digitalisation of a healthcare facility. Extension of the Labrique et al. (2013) mHealth and ICT framework"

Copied!
50
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

mHealth Framework

for the digitalisation of a

healthcare facility

Extension of the Labrique et al. (2013) mHealth and

ICT framework

Senate Lesaoana

July 2017

(2)
(3)

mHealth Framework

for the digitalisation of a

healthcare facility

Extension of the Labrique et al. (2013) mHealth and ICT framework

Student Mentor

Senate Lesaoana Cees Hesp

Student number: 11077883 PharmAccess Foundation

E-mail:senate.lesaoana@student.uva.nl DirectormHealth Research Labs

Internship Address Tutor

PharmAccess Dr. Robin Langerak

AHTC, Tower 4C Dept. of Medical Informatics

Paasheuvelweg 25 Academic Medical Centre-

1105 BP Amsterdam University of Amsterdam

Internship period

(4)

Dedicated to my brother, Pitso Lesaoana who has carried me on his shoulders since I was a toddler. Although confined to a wheelchair he is the most abled person I know; the workings of his mind are awe inspiring, his indeed is a beautiful mind.

ACKNOWLEDGEMENTS

I would like to express my gratitude to PharmAccess for affording me the opportunity to intern with them. A special thanks to the mHealth Labs, especially Cees Hesp and Hanneke Peeters for all their support, and to the SafeCare offices in Kenya, Ghana, Tanzania, and Nigeria for their assistance in conducting the surveys. I would also like to thank my tutor Dr. Robin Langerak for his great supervision and helpful guidance throughout this research project.

Big thanks to all my family - in all its vastness and spread across the globe- for their consistent support and love. I would not have made it had it not been for the phone calls - some three hours long, the frequent email exchanges, and the WhatsApp messages. Thank you to all my boys and babies, who made me proud in various ways and motivated me to keep on keeping on. Love you guys.

Now thanks be unto God, who always causes us to triumph in Christ, and makes manifest the savour of his knowledge by us in every place. 2 Corinthians 2:14 (KJV)

(5)

Table of Contents

Abstract ... 5 Samenvatting... 6 1 Introduction ... 7 1.1 Research Questions ... 9 1.2 Chapter Organisation ... 9 2 Background ... 10

2.1 Health System Goals ... 12

3 Literature Review ... 14

3.1 Previous frameworks and taxonomies ... 15

3.2 The state of mhealth in Africa ... 16

3.3 Investment decision in mhealth ... 17

3.4 Prioritising different mhealth investments ... 17

4 Methods ... 18

4.1 Framework development ... 19

4.2 Conducting Survey ... 19

5 Results ... 22

5.1 Framework extension ... 23

5.1.1 New mhealth application domains ... 25

5.2 mhealth application domains prioritisation ... 27

5.2.1 Experts ... 27

5.2.2 Providers ... 29

5.2.3 Comparison of experts and providers ... 32

6 Discussion ... 33

7 Conclusion ... 37

References ... 38

Appendix A: Twelve mhealth & ICT applications: Original Labrique et al. framework ... 42

Appendix B: Questionnaires ... 45

Experts ... 46

(6)

ABSTRACT

Background PharmAccess, in the pursuit to improve access to quality health care in

Sub-Saharan Africa, seeks to take advantage of the rapid adoption of mobile technology in health care. The mhealth and ICT framework by Labrique et al. (2013) provided a base for the digitalisation process; however it does not address important functions specific to private health care facilities, a sector within which PharmAccess operates. This research will build on the framework and extend it with additional mhealth applications, especially those relating to private health facilities. This extended framework will be used as a planning tool to prioritise the different mhealth application domains.

Method We identified activities for setting up a private health facility and activities along

the entire care continuum, and paired them with a search for mhealth innovations in Sub-Saharan Africa to identify new possible application areas for mhealth. A survey, using a questionnaire was used to obtain healthcare experts’ & providers’ prioritisation of the different mhealth application domains.

Results Eight new application areas were identified, resulting in a new framework with

20 mhealth application domains. The additional application domains are financial management, legal compliance, marketing, patient safety & satisfaction, provider-patient communication, occupational health & safety, patient planning & transportation, and device & room monitoring. Health care experts (funders and policy makers) preferred supply chain management (10.4%), electronic health records (9.8%), and provider-patient communication (8.9%). Health providers prioritised supply chain management (11.8%), financial transactions & incentives (8.7%), and sensors & point of care diagnostics (8.3%). Both groups perceived quality of care as the most important motivator for any mhealth investment decision.

Conclusion Experts and providers agreed that supply chain management was of highest

priority. There was also agreement that the new mhealth applications contribute more towards transparency and profitability as opposed to patient-centred goals. Their value was perceived to be a contribution towards practice-centred outcomes.

(7)

SAMENVATTING

Achtergrond PharmAccess, in najaging om de toegang tot de kwaliteit van

gezondheidszorg in Sub-Sahara Afrika te verbeteren, tracht om te profiteren van de snelle goedkeuring van mobiele technologie in de gezondheidszorg. De kader van mhealth en ICT van Labrique et al. (2013) biedt een basis voor het digitale proces; echter wendt het zich niet tot belangrijke gebruiken binnen particuliere instellingen voor de gezondheidszorg, een sector waarbinnen PharmAccess opereert. Dit onderzoek zal gebaseerd zijn op deze (bovenstaande) kader en zal uitgebreid worden met extra mhealth toepassingen, vooral die gerelateerd zijn aan particuliere zorginstellingen. Dit uitgebreide kader zal gebruikt worden als planningshulpmiddel om de verschillende mhealth toepassingsdomeinen te prioriteren.

Methode We hebben activiteiten geïdentificeerd om een particuliere gezondheidscentrum

op te zetten en activiteiten in het gehele zorgcontinuüm, en dit gekoppeld aan een zoekopdracht voor mhealth innovaties in Sub-Sahara Afrika om nieuwe toepassingsgebieden te vinden voor de mhealth. Een verslag, dat een vragenlijst bevatte werd ingezet om gezondheidszorg experts en aanbieders priotering te krijgen van de verschillende mhealth toepassingsdomeinen.

Resultaten Acht nieuwe toepassingen werden gekenmerkt, dat resulteerde in een nieuwe

kader met 20 mhealth toepassingsdomeinen. De extra toepassingsdomeinen zijn financieël beheer, wettelijke naleving, marketing, veiligheid en tevredenheid van de patiënt, aanbieder-patiënt communicatie, beroepsmatige zorg & veiligheid, patiënt planning & transport, en apparatuur & toezicht op de kamer. Gezondheidszorg experts (financiers en beleidsmakers) gaven de voorkeur aan integrale ketenbeheer (10,4%), electronische medische dossiers (9,8%), en aanbieder-patiënt communicatie (8,9%). Zorgaanbieders prioriteerden integrale ketenbeheer (11,8%), financiele transacties & prikkels (8,7%), en sensoren & punt van zorgdiagnostiek (8,3%). Beide groepen zijn zich ervan bewust dat de kwaliteit van de zorg de belangrijkste motivatie moet zijn voor elke beslissing voor investering in de mhealth.

.

Conclusie Experts en aanbieders waren het erover eens dat toevoer-keten management

de hoogste prioriteit had. Zij waren het er ook over eens dat de nieuwe mhealth-applicaties meer bijdragen tot openheid en rendement in tegenstelling tot doelen die gefocust zijn op de patient. Hun waarde werd beschouwd als een bijdrage aan

praktijkgerichte uitkomsten.

Trefwoorden: Mobiele gezondheid, mHealth, mHealth kader, digitale gezondheid, gezondheids IT

(8)

1

INTRODUCTION

The burden of disease has placed great pressure on health systems in developing countries and due to a lack of resources, has led to low quality of health care resulting in “ill health systems”.1 These countries face a burden both in communicable and chronic diseases, resulting in what Boutayeb (2005) calls the “dual burden for developing countries”. The advent of mobile technology, especially when applied to health care (mobile health), presents a potential solution to tackle health delivery challenges.

Mobile health or mhealth is health related technological solutions that employ wireless communication devices to enable data exchange and provision of health care.2 It is the use of mobile technologies for processes involving the practice of medicine and delivery of care. Mobile technologies frequently used to facilitate exchange and deliver services include short message service (SMS), voice calling, web, and mobile applications, and connectivity technology such as Bluetooth and WI-FI.

The application of technology in healthcare has become widespread and the adoption of mhealth technology specifically has developed rapidly.4 The reason for the widespread acceptance and interest for mhealth is twofold; firstly, the demonstrated benefit of mobile technology has lead to optimism that there are benefits to be gleaned with its application to healthcare, enabling health information, and services to reach remote communities;5 and secondly, the extensive penetration of mobile devices.6,8 In Africa mobile phone penetration is estimated at 557 million unique mobile service subscribers in 2015 with smartphone adoption reported at 23%.9

There is great enthusiasm from the African continent for mhealth as evidenced by the so called “pilotitis” era which is “the unfettered proliferation of mhealth projects”.10 The enthusiasm and technology advances unlock the possibility to digitalise the healthcare system. However, the realisation of such digitalisation is hindered in part by a lack of a common language to describe and appraise opportunities and challenges of such mhealth interventions6 to synthesize evidence and to guide its scale-up. Such evidence is of

(9)

particular importance for private healthcare providers in African countries who provide more than 50% of healthcare12 yet often have limited resources to invest in technology. PharmAccess, a not-for-profit organization, is dedicated to improving access to quality health care in Africa through building sustainable health systems applying innovative financing mechanisms. It has a private-sector focus and mobilizes public and private resources for the benefit of doctors and patients through clinical standards and quality improvements, loans for healthcare providers, health insurance, mhealth innovations, and operational research.3 PharmAccess believes that mobile technology has the potential to revolutionize the healthcare sector and reach people who until now have remained structurally excluded. To demonstrate the advantages of applying modern technology and business models to the provisioning of healthcare, five clinics will be selected for digitalisation in each of the main countries where PharmAccess is active, viz. Ghana, Kenya, Nigeria, and Tanzania.

To achieve digitalisation a coherent mhealth language is required. To address this issue Labrique, Vasudevan, Kochi, Fabricant, and Mehl (2013) have developed an mhealth and ICT framework that describes the areas of possible digitalisation within the healthcare system. Theframework can serve as a roadmap for the digitalisation of health facilities; however, it was developed with an inclination towards public health facilities. Public and private health facilities differ not only in ownership and financing but also in organization and management.7 A scrutiny of the framework revealed that no provision has been made for financing (fundraising), a function that is very important in privately owned health facilities.

The observation above raised the question of whether there are other functions that have been omitted that are less obvious to observe. The aim of this study is to extend the existing Labrique et al. (2013) framework to include mhealth application domains currently not enumerated, particularly those applicable to private health facilities in low resource settings where the health landscape is often strained. The extended framework will identify digitalisation opportunities across the entire life span of a health facility, clearly indicating the possibilities for every stage of its operation. To our knowledge no studies have been conducted to investigate this matter.

(10)

1.1 Research Questions

1. Which mhealth application domains should be added to the existing mhealth and ICT framework to develop a comprehensive extended framework?

i. What is the completeness of the extended framework? Does it account for all aspects of healthcare with specific attention to the start-up phase, hospital administration & management, patient flows, the practice of medicine, referrals

and reporting, and patient health attitudes (health promotion)? PharmAccess intends to outline a path for the digitalisation process. To achieve this, we

will obtain the opinion of stakeholders on which are the most valued domains, and how each application domain contributes to healthcare system goals. We will make use of the Labrique et al. (2013) framework extended with additional domains if any are identified, as a tool for this planning process. This goal is addressed by the questions below.

2. Which are the top three application domains that healthcare providers and experts respectively, consider to be most important?

i. What are the differences in view between the different health stakeholders on the order of importance of the mhealth application domains?

3. To what extent are the health goals, quality of care, health outcomes, profitability, and transparency, motivating factors in the investment decision in mhealth?

1.2 Chapter Organisation

Chapter 2 provides the background for the study and discusses the base upon which the study is established. Chapter 3 will present existing literature on mhealth frameworks and provide a context for mhealth in Africa. The focus will be specifically on the state of and investment decisions for mhealth. In chapter 4 the methods used for the extension of the Labrique et al. (2013) framework and the prioritisation of mhealth applications are discussed. Chapter 5 then provides the results and presents the extended framework and prioritised list of the mhealth applications. Chapter 6 is the discussion of the research and reflects on the results as well as the implications thereof. Chapter 7 is the conclusion and offers a summary of the research project.

(11)

2

BACKGROUND

This chapter provides a brief summary of the original framework on which the current study is based. The chapter also gives a description of important concepts used in this research.

(12)

This study builds on the mhealth and ICT framework developed by Labrique et al. (2013). The framework enumerates 12 common mhealth and ICT applications that make the components of an mhealth strategy or platform. Following the reproductive, maternal, newborn, and child health (RMNCH) continuum of care, it links the applications to the health system constraints they address. To give context to our study the application areas are summarised in Figure 1 below. The full description is provided in Appendix A.

(13)

2.1 Health systems goals

Any application of mhealth is adopted with the intent that it achieves a certain goal or helps overcome a certain challenge. According to the World Health Organisation (2011), research on technology in healthcare should focus not on computer-based usability but rather on health outcome effect. Guided by this, four health system goals will be highlighted in the current study; quality of care, health outcome, profitability and transparency. These system goals were chosen to reflect the values and objectives of PharmAccess for healthcare in Sub-Saharan Africa.

Quality of care – Quality of care, as used in this study, refers to structures and processes

of care delivery13 and is based on the definition given by the Institute of Medicine. Quality of care is the degree to which health services are delivered effectively and efficiently, ensuring the availability of needed resources, minimising wastage of time and other resources, and capturing patient information in efficient ways. Practitioners should have timely access to information, be effectively educated and practice evidence-based medicine adhering to clinical guidelines in the delivery of care. Interaction between caregivers and patients should be consistent across social and racial lines,be responsive to patient needs and preferences, and should not open patients to risk of injury.14

Transparency – Transparency refers to both administrative and financial transparency.

Administrative transparency refers to transparency in health interactions. This refers to clearly defined administrative, reporting, and clinical pathways, and the voluntary, and mandatory disclosure of health information such as clinical outcomes, physician licensing, malpractice cases, and patient satisfaction.16,23,64 Financial transparency is the availability and disclosure of the financial performance indicators15 of a healthcare facility, ensuring that information for public disclosure is reported and made accessible to patients and other interested groups. This involves access also to comparative information on prices and costs of healthcare products and services.16 Financial transparency refers as well to defining explicit financial paths that help build a sustainable financial system and curb illicit cash flows, making sure that distribution of funds to activities, services or projects is traceable.

(14)

Health Outcomes – Health outcomes represent increased healthy life expectancy which is

the prevalence of good health.26 Included here is quality adjusted life years (QALYs) which is the average number of years added to an individual as a result of a health intervention; it is both a measure of the length of life and the quality of that life.18 Health outcome refers to improvement of health status,21 either in factors known to affect the health and quality of life of the individual or in self-perceived health status.17 It is a reduction in readmissions, and mortality, minimisation of hospital acquired infections, and an improvement in physical functional status, as well as patient satisfaction with care received.19

Profitability – Profitability refers to positively influencing the income of a health care

facility by generating and stimulating revenue while controlling and minimising costs by monitoring cash flows.22 This means increasing the volume of paying customers, improving cash flow and working capital, and generating and retaining funds for future capitalisation.20 Profitability entails improved financial and administrative performance, attaining operating profitability on a sustainable basis, and improving, and maintaining good credit ratings.

(15)

3

LITERATURE REVIEW

This chapter discusses previous research conducted in defining digital health frameworks and taxonomies. It provides context for this study by discussing the current state of mhealth in Africa. It also offers an overview of how investment decisions are made in health and how mhealth fairs in general. A specific consideration is then provided on how investment decisions are made among mhealth solutions specifically.

(16)

3.1 Previous frameworks and taxonomies

A number of studies have been conducted that endeavour to define a structured approach to digital health. Mechael et al. (2010) developed an mhealth framework that consisted of five areas in which mobile technologies are utilised in health; treatment compliance, data collection and disease surveillance, health information systems and point‐of‐care support, health promotion and disease prevention, and emergency medical response. Although not a framework, Lupton (2014) provides a typology for digital health technologies. The typology details eleven areas for digital technology application with a focus on health promotion, medical education, communication and diagnostic devices. Leon et al. (2012) focus on community-based health care and depict the mhealth applications using five categories; data collection, management, clinical service delivery, health promotion activities, and education and training.

Ventola (2014) also presents five broad application areas; administration, health record maintenance and access, communications and consulting, reference and information gathering, and medical education. The author also offers an extended version which has eight application groupings. A framework for identifying optimal mhealth strategies for Sub-Saharan Africa that incorporates health system challenges relating to non-communicable diseases (NCD) and the continuum of NCD development was developed by Bloomfield et al. (2014). The framework divides mhealth applications according to the health system challenge they address and does this along the chronic disease continuum. The authors argue that creating evidence for mhealth along these lines will assist policy makers in defending investments in mhealth.

Magbity et al. (2011) developed an mhealth domains framework which they divided into two domains. The first domain, health service domain comprised of four categories: health education aimed at health promotion, point-of-care support aimed at diagnosis, and clinical reference, client monitoring for appointment tracking and emergency medical response system. The second domain health system domain had five categories; disease and epidemic outbreak surveillance for real-time disease tracking, health management information systems including supply chain management, human resource for health

(17)

management, supervision and professional development, health financing such as mobile payments, and general coordination among health workers, and service providers.

3.2 The state of mHealth in Africa

As the saying goes necessity breeds innovation; that seems to be true throughout the African continent. According to GSM Association (2016) there are a considerable number of mobile innovation projects including mHealth in Africa, with approximately 310 active tech hubs and 180 accelerators or incubators; and Kenya is referred to as the Silicon savannah.30 However, many of these projects are in pilot phase8 and few make it past this stage.35 Leon et al. (2012) suggest that long term project sustainability is a problem because many projects are initiated and funded by donors. The authors argue that ownership, commitment, and leadership from beneficiaries is often lacking and once donors pull out, the projects seize. The lack of a global consistent framework including indicators, and evaluation methods also contributes to the problem and limits scalability of implementations.35 Although evaluation studies are conducted a majority of these focus on feasibility and not on cost-effectiveness and impact of such solutions on improvement, and quality of service delivery processes, health systems, and health outcomes.6

Mecheal et al. (2010) and Kahn et al. (2010) observe that many of the mhealth solutions developed utilise basic or entry level mobile communication technologies, the most common being text messaging – short message service (SMS) and this probably due to the capacity it has to reach large masses particularly remote populations. In a systematic review by Brinkel et al. (2014) in which they assessed the mhealth approaches for public health surveillance, the majority of the studies reviewed used standard low-cost mobile phones and all but one made use of SMS. While SMS is the common technology, Aranda-Jan, Mohutsiwa-Dibe & Loukanova (2014) concluded in their systematic review on what works, what does not work and why of implementation of mobile health (mhealth) projects in Africa, that the most common mhealth implementations are those aiming at improving patient life-style, medication adherence, and treatment follow-up.

(18)

3.3 Investment decisions in mHealth

Investments in mhealth compete with other health system priorities and often the latter get precedence over mhealth.25 A study conducted by PharmAccess (2016) in Kenya on the preferred mode of reward for facilities using MTiba supports this assertion. The results indicated that medical equipment ranked as the most preferred reward while digital basic medical history information of patients and digital management information dashboards providing anonymized information on performance compared to peers, ranked at the bottom. According to WHO (2011) this can be attributed largely to a lack of solid evidence for mhealth on which policy-makers and administrators can base their investment decisions.

3.4 Prioritising different mHealth investments

MEASURE Evaluation SIFSA (2017) suggests that ICT investment priorities should be set within the context of achieving health system goals or alleviating system challenges. Moreover, existing information on health system status should be used to inform this prioritisation process. However, their research indicated that this is often not the case; instead an ad hoc approach is often applied.

Coye and Kell (2006) postulate that the choice of investment, particularly of disruptive technologies is influenced by physician preferences and indirectly by patient demand. The authors argue that this reactive posture of waiting for physician requests or adoption by other hospitals to trigger the planning process is due to difficulties in constructing a business case or conducting a return on investment analysis for mhealth interventions. In Sub-Saharan Africa, as most mhealth initiatives are donor funded and/or subsidised by digital providers priority may be set in accordance with the agenda of these funding bodies.6,39

(19)

4

METHODS

This chapter details the process adopted in the development of the extended framework. Furthermore, it describes the methods used in conducting the survey and provides the details of both the sample choice and instrument employed.

(20)

4.1 Framework Development

To establish all possible areas of application for mhealth and thus develop a more comprehensive framework, activities along the establishment (start and set-up), and entire care continuum of a private healthcare facility were considered. The establishment, and care continuum activities were divided into six stages and each stage was decomposed into its building components. These were identified using books, online resources, and from the results of the literature review performed. The six stages were then grouped according to the main actor; this facilitated further identification of activities and enabled us to determine whether the six stages were realistic and logical. Each building component was then checked against the existing framework to determine whether or not it was accounted for as an area of mhealth application.

To further identify mhealth application domains we searched for mhealth innovations in Sub-Saharan Africa. These could be solutions already in use or start-ups looking for funding. The website Disrupt Africa, and blog Venture capital for Africa served as starting points. These had recommendations to other websites thus providing more resources resulting in the snowball effect. Each mhealth innovation identified was then classified according to the 12 application areas of the Labrique et al. (2013) framework. Any innovations that could not be fitted into the existing categories formed the basis for either new applications or the expansion of the definition of an existing application.

4.2 Conducting survey

A survey in the form of a questionnaire (Appendix B), was conducted among the stakeholders of healthcare. The target group was healthcare providers and a group of healthcare funders, policy makers, and researchers, herein called healthcare experts. These individuals were identified and chosen using the network of PharmAccess. Specifically, the provider group comprised of clinics in Ghana, Nigeria, Kenya, and Tanzania, which have partnered with PharmAccess.

There are cases where owners of private clinics do not practice at these facilities but rather hire another individual to manage the facility. In light of this, the term “providers”

(21)

represented in the survey so as to obtain a holistic view. The owners of the clinics provide a business perspective while the managers give a medical, and day-to-day operations perspective. It seemed particularly important to take this approach as PharmAccess is not only interested in digitalising these facilities but ensuring that such digitalisation is sustainable, addresses real challenges and that there will be ownership of this endeavour from those involved, both to invest in it and to ensure its continuity.

The questionnaire was pretested using two groups; one group consisted of academics in medical informatics and the other of doctors practising in the Netherlands. The Informatics group helped to ensure that examples provided were truly indicative of the mhealth category under which it is listed. The doctors ensured that the language was simple, easily understandable by doctors and that it communicated the benefits that accrued to the doctor and thus facilitated the decision making process. Through consultation with a financial analyst from the PharmAccess group we learned that entry and small loans for clinics in the four African states is on average $8 000. We settled on the amount of $10 000 for arithmetical simplicity.

The questionnaire consisted of a listing of all identified categories for mhealth application. For each category examples were provided to ensure that respondents understood what each mhealth category means. The same questionnaire was used both for providers and experts. Two approaches were used to conduct the providers’ survey; in Nigeria, all providers received the same questionnaire while in Kenya, Ghana, and Tanzania we used two versions of the questionnaire. The additional version presented the application domains in an order different to the original version, the order was chosen randomly. This strategy provides an empirical approach to gauge and minimise any question order effect; therefore providing more validity to the outcomes.

Each respondent was theoretically given an amount in the local currency either equivalent to $10 000 or less depending on the amount that was contextually appropriate. Amounts were adjusted to reflect what is reasonable in each context, which is the limitedness of funds available for digital health investments. This encourages the respondent to think in terms of challenges that need to be addressed and not what is “nice” to have. Respondents were required to indicate how much of this amount they would invest in each category. This investment amount could be anywhere between zero and the entire given amount.

(22)

Additionally respondents had to motivate their investment decision using one or more of the health system goals: quality of care, health outcome, profitability and transparency. Each provider was surveyed individually at their respective practice. We presented the background of the study and explained the purpose of the questionnaire. This was done in person by a PharmAccess representative. The individual was given 20 minutes to complete the questionnaire during which the interviewer was present. This allowed for the provision of more information on categories and health system goals if the interviewee so required. The interviewer could however not discuss the actual prioritisation or offer opinions on such. This strategy was adopted to ensure that the questionnaires were properly filled out and returned.

To obtain the opinion of the industry experts we organised a presentation at the offices of PharmAccess Group in the Netherlands. We invited individuals from Investment Fund for Health in Africa (IFHA), Amsterdam Health Technology Institute (AHTI), Amsterdam Institute for Global Health and Development (AIGHD), and each of the organisations that comprise the PharmAccess Group. That is, individuals from PharmAccess Foundation, SafeCare, Medical Credit Fund (MCF) and Health Insurance Fund (HIF). Following the presentation a questionnaire was distributed among the group. The experts filled the questionnaire and delivered the complete questionnaire to the principal investigator. Descriptive statistics were applied to the data collected. A two-tailed t-test was applied to calculate significance values and the preferred alpha value is 0.05.

(23)

This chapter provides the results. The findings are presented in two sections; framework development, and the prioritisation survey. Each section provides broad detail on all outcomes.

5

RESULTS

(24)

5.1

FRAMEWORK EXTENSION

The six stages that comprise the establishment, and care continuum are: starting up, medical tasks, hospital administration and management, reporting and referrals, patient flow, and health attitudes. The first four stages as listed above have as the main actor the provider. Patient flow is acted on by both the provider and patient whilst health behaviour is the domain of the patient. These are depicted as the two horizontal flows in the framework as shown in Figure 2.

Starting-up is composed of three main activities; funding, legal activities, and marketing. Medical tasks comprise all care provision tasks, both diagnostic, and therapeutic. Hospital administration and management consists of nine functions; supply chain management, human resource management, hospital safety, scheduling and resource allocation, finance, occupational safety, quality management, research and education, and health events tracking. Patient flow follows the movement of the patient through the healthcare encounter, from admission through to post-visit care. Reporting and referrals includes clinical documentation and can be stand-alone or embedded within the other activities. Health attitudes are activities aimed at improving patient health literacy and behaviour. Eight new mhealth application domains were identified and added to the existing framework. These are financial management (fundraising), legal compliance, marketing, provider-to-patient communication, patient planning and transportation, patient safety and satisfaction, occupational health and safety, and device and room scheduling. The activity, insurance claims did not warrant a new category but rather we redefined financial transactions and incentives to accommodate this activity. As shown in Figure 2, the extended framework has twenty domains, listed vertically on the left. The new domains are demarcated by the dotted line in the figure. The relationship between the application domains and set-up and care continuum stages is indicated by the bars, these indicate the mhealth domains that are applicable/ useful per stage.

(25)

FIGURE 2: Extended mHealth and ICT Framework Stages

Client Education &

behaviour change

Sensors & point of

care diagnostics Registries/vital events tracking Data collection and reporting Electronic Health Records Electronic decision Support Provider-to-provider Communication Provider work- planning and scheduling Provider training & education Human Resource Management Supply chain Management Financial transactions & incentives Financial Management Fundraising Legal compliance Marketing Patient safety & satisfaction Provider-patient Communication Occupational health & safety Patient planning &

Starting up Hospital Administration & management Medical tasks & ReferralsReporting Patient flow

Non-Medi

cal tasks Patient flow

Health attitudes

mHealth & ICT Applications 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

(26)

5.1.1 New mHealth application domains

This section presents a description of the new or revised application domains. A summary of the original twelve domains can be found in the background chapter of this paper.

Financial Transactions and Incentives – The initial definition referred to solutions that

facilitate mobile payments and overcome financial barriers to care. The extended definition includes mobile solutions that enable providers to interact with insurance companies particularly during claims filing, and those which give patients access to health insurance or aid them to save towards health care. Examples include Mtiba, a mobile wallet specifically earmarked for healthcare44, and InstaCare which enables real-time verification and authentication of a beneficiary’s health insurance subscription.45

Financial Management (Fundraising) – This refers to mobile-enabled bookkeeping

applications for monitoring cash flows, generating estimates and invoices, and scanning and automatically recording receipts into the bookkeeping system. It also refers to mobile solutions that can be used to obtain extra funds to finance operations. Mobile applications can link providers with money lenders allowing lenders to track the volume of monies flowing in and out of a practice. This information can be used to determine the credit worthiness of and loan amount a provider can receive. Kopo Kopo Grow46 is one such example.

Legal compliance –The establishment and operations of health care facilities is highly

regulated by governments.27 Mobile applications which enable providers to consult with legal professionals on licensing matters can be used to navigate this legal sphere. Patient oriented mobile applications can be used to enable a patient to verify the credentials and legitimacy of providers thus curbing fraudulent practice.

Marketing – Mobile marketing transcends traditional marketing channels and increases

visibility and reach, communicating a healthcare facility’s service offering to a broader audience.42 Providers can use conventional channels such as SMS messaging or novel streams such as social media. An application such as Find-a-Med47 which helps users locate the nearest health facilities and provides directions can be used for marketing purposes.

(27)

Patient safety and satisfaction – Mobile forms and structured SMSes can be used for

reporting unsafe practices that place patients in danger. By identifying and disclosing these often preventable errors, adverse patient events such as medical errors, injuries, accidents, and infections can be minimised.24 Mobile applications such as Maoni48 can be utilised to capture patient feedback, which can be used to improvement the patient experience.

Provider-to-patient communication – Communication between the provider and the patient

can be facilitated by mhealth tools that allow the exchange of information and images to enable remote diagnostics. Through this exchange patients communicate their medical problems, and receive medical advice, and providers can remotely monitor their patients Platforms such as TalktomeDoc allow patients to consult with doctors online.49. This can reduce the lead times between symptoms and treatment, increase a provider’s reach, and reduce the foot traffic at facilities

Occupational health and safety – Healthcare facilities need to ensure the safety of their

workers; mhealth tools including mobile forms or structured SMSes can provide a platform for workers to report occupational hazards and injuries, and exposures to viruses. An example is mPEP which uses SMS to facilitate the reporting of risk factors to exposure to HIV and other infectious disease. 50Dashboards providing health and safety compliance levels, and suggestions for improvement can be used to minimise exposures.

Patient planning and transportation – mHealth tools can be used to schedule patient

appointments, allocate patients to specific services and/or health worker and arrange ambulatory services for them. An ambulatory service, Flare, aggregates available ambulances onto a single system and allows patients or healthcare facilities to request emergency help using their smartphone.51

Device and room scheduling – Healthcare facilities in low income regions have limited

resources and must optimise their use. mHealth tools can be used to plan the optimum schedule for room and device usage. These tools may allow providers to simulate various schedules and help with selecting the best option. Repairs and maintenance of medical equipment can also be tracked to avoid scheduling their use when they are not available.

(28)

5.2

MHEALTH APPLICATION DOMAINS PRIORITISATION

5.2.1 Experts

Two stakeholder groups were surveyed; healthcare experts, and providers. There were 41 expert respondents; however, 4 questionnaires were excluded as the total sum of money of each exceeded the theoretical $10 000 allocated. Another three questionnaires did not contain information on department, and one had neither department nor personal information. These questionnaires were included in the overall analysis however as the lack of this information did not affect the overall outcome of the rankings. Nevertheless, they were excluded in the analysis by department.

The experts group had 20 females, 16 males, and 1 unspecified and an age distribution of 19 (18-34), 14 (35-54), 2 (>=55), and 2 unspecified. The group represented 13 different departments within PharmAccess, and 2 research institutions. These were grouped into seven more general functions, which can be seen in Table 1. Financial transactions & incentives, and provider-to-patient communication had the highest number of selections, 25 and 24 respectively. Based on fund allocation, and segmented by age, provider-to-patient communication (10.2%), and provider training and education, a close second at 10%, were deemed as top priority within the 18-34 age group. Individuals within the 35-54 age group considered electronic health records (13.4%), and financial transactions and incentives (11%) of highest priority. The 55 and above age group placed most importance on data collection and reporting (15%), and supply chain management, financial transactions and incentives, and financial management (fundraising) each receiving 12.5% of the funds. Disaggregated by gender, males placed high priority on client education & behaviour change communication (10.3%), and electronic decision support (10.2%); in contrast their female counterparts prioritised provider-to-patient communication (11.2%), and supply chain management (9.7%).

The top three combined group prioritisations based on fund allocation are supply chain management (10.4%), electronic health records (9.8%), and provider-to-patient communication (8.9%). The least preferred application domains were device & room monitoring (1.5%), human resource management (0.7%), and occupational health and

(29)

department (functions) rankings. The complete ranking of all domains can be seen in Figure 3.

Table 1: Experts’ top three preferred application domains

Business Unit First Second Third

Group consensus (Overall)

Supply chain

management (10.4%) Electronic health records (9.8%) Provider-to-patient communication (8.9%)

Finance Financial Management

(Fundraising) (20%)

Registries/ vital events tracking (16.7%) - Financial transactions & incentives - Provider-to-patient communication (10%) Health intelligence and plans

Provider work planning & scheduling (10.2%)

Provider-to-patient communication (9.8%)

Patient safety & satisfaction (9.5%) Quality

Assurance Provider training & education (20%) Financial transactions & incentives (17.5%) - Electronic decision support - Provider work planning & scheduling - Sensors & point-of-care diagnostics (10%) Product

development (Innovation)

Financial transactions & incentives (20%)

Provider-to-patient communication (16.7%)

Electronic health records (12.5%)

Marketing Electronic health

records (11.7%) Patient planning & transportation (11.6%) Financial transactions & incentives (10.8%)

Research Electronic health

records (13.4%) Client education & behaviour change communication (11.7%)

Electronic decision support (10.8%) Admin and

legal Client education & behaviour change communication (17.5%)

Provider to provider

communication (12.5%) Financial transactions & incentives (11.25%)

The majority of individuals cited the health system goal, quality of care (35.8%), as the reason for making an investment across all application domains. This was followed by health outcomes (29.8%), transparency (20.3%), and profitability (14%). Quality of care weighed heaviest also among the top three application domains; supply chain management (33.3%), electronic health records (37.7%), and provider-patient communication (38.9%). Financial transactions and incentives (6%) is the only domain were this health goal weighed the least among the reasons for making an investment. Figure 4a depicts the perceived contribution of each application to each health system goal.

(30)

Figure 3: Overall Investment Preference: Experts

5.2.2 Providers

A total of 61 providers were approached to participate in the survey across four countries, Ghana, Nigeria, Kenya and Tanzania. However, due to incorrect completion of the questionnaire, and a refusal to participate, only 50 providers are included in the analysis. The distribution of providers per country is; 13 in Ghana, 18 in Nigeria, 13 in Kenya, and 6 in Tanzania. The country specific sample characteristics are described in Table 2.

Of the total 50 providers, 33 received the original questionnaire and 17 the reordered version. The providers represented 18 multi-speciality facilities and 32 primary care providers. Twelve of the respondents were owners and 38 were managers, and the gender composition was 31 males, and 19 females. The age was divided into three categories, 18-34, 35-54, and >55; these were represented by 12, 29, and 9 individuals respectively. The patient catchment representation was high income, 6 providers, middle income, 23 providers, and low income, 21 providers.

0 1 2 3 4 5 6 7 8 9 10 11 12 In ve stme n t p erce n tag e

(31)

Table 2: Sample Characteristics per country

The country specific analysis shows that providers in Kenya prioritised sensors and point of care diagnostics, supply chain management, and patient planning and transportation. Their Tanzanian counterparts however placed more stock on financial management (funding), financial transactions and incentives, and supply chain management. Ghanaian providers preferred provider training and education, financial transactions and incentives, and client education and behaviour change while for Nigerian providers supply chain management, electronic health records, and sensors and point of care diagnostics were regarded as top priority.

The total collective of providers was disaggregated by facility and respondent characteristics, and priority was determined based on percentage of fund allocation. Table 3 and 4 show the top three prioritised applications grouped on facility and respondent characteristics respectively.

Country Speciality Position Gender Age Patient Income Questionnaire

Primary Multi-speciality

Owner Manager Male Female 18-34

35-54

>55 Low Mid High Origin al ReOrd ered Ghana 12 0 4 9 7 6 3 10 0 9 3 0 7 6 Nigeria 5 13 8 10 13 5 2 8 8 7 8 3 18 0 Kenya 8 5 0 13 8 5 5 7 1 2 8 3 6 7 Tanzania 6 0 0 6 3 3 2 4 0 3 3 0 2 4

(32)

Table 3: Priority by facility characteristics

Table 4: Priority by respondent characteristics

Across all providers in the four countries priority was placed on supply chain management which was allocated 11.8% of all funds. This was followed by financial transactions and incentives (8.7%), and sensors and point of care diagnostics (8.3%). The mhealth application that received the least priority based on fund allocation was provider-to-patient communication which received only 1.9% of all funds. In terms of frequency supply chain management, and patient safety & satisfaction were the most recurring at 39 and 37 respectively.

The primary motivation for making an investment across all providers in all countries, which was mentioned more than a third (35.5%) of the time, was quality of care. As shown in Figure 4a providers perceived device and room monitoring, patient safety and satisfaction, and provider training and education to make the most contribution to quality of care. Health outcome was the second most motivating factor with 28.2%; its biggest contributors were perceived to be occupational health and safety, client education and behaviour change, and provider work-planning and scheduling (Figure 4c). Profitability

0 10 20 30 40 50

Multi-speciality Primary care Low income Middle income High income

Perc en ta ge 0 10 20 30 40 50

Owner Manager Male Female Young Middle age Older

Per ce n tag e Provider training and education Financial transactions and incentives Electronic health records

Sensors and point of care diagnostics Supply chain management

(33)

Figure 4: mHealth applications contribution to health system goals

(a) Quality of Care (b) Profitability

(c) Health Outcomes (d) Transparency

Experts: Original New Providers: Original New * The order of applications follows the same as in Figure 2.

5.2.3 Comparison of Experts and Providers

The overall difference observed between the experts and providers was found not to be statistically significant. The t-test yielded a ρ-value of 0.12 which was greater than the set alpha of 0.05. This indicates that the differences observed are likely due to chance. However, significant differences were observed in six specific application areas; provider-patient communication (ρ-value 4.03 x 107); registries/ vital events tracking (ρ-value 0.02); data collection & reporting (ρ-value 0.02); electronic decision support (ρ-value 0.004); provider-to-provider communication (ρ-value 0.0003); and provider work planning and scheduling (ρ-value 0.001).

0 10 20 30 40 50 60 70 1 2 3 4 5 6 7 8 9 1011121314151617181920 mhealth applications* 0 10 20 30 40 50 1 2 3 4 5 6 7 8 9 1011121314151617181920 mhealth applications 0 10 20 30 40 50 60 1 2 3 4 5 6 7 8 9 1011121314151617181920 mhealth applications 0 10 20 30 40 50 1 2 3 4 5 6 7 8 9 1011121314151617181920 mhealth applications

(34)

6

DISCUSSION

Through literature, we identified eight additional mhealth application domains, namely, financial management (fundraising), legal compliance, marketing, provider-to-patient communication, patient safety and satisfaction, occupational health and safety, device and room scheduling, and patient planning and transportation. The extended framework therefore enumerates twenty mhealth applications. Moreover, the definition of the category financial transactions and incentives from the original framework was expanded to include mobile payments and insurance transactions. All these additions provide a broader perspective of the use of mhealth across many of the activities of a healthcare facility, from inception through to patient follow-up. This ‘lifecycle’ wide view offers the possibility of identifying mhealth opportunities at every stage of operation thus presenting a more comprehensive tool in the digitalisation of a healthcare facility. It is noteworthy that the new applications are inclined more towards administrative processes and less towards clinical activities

We utilised a questionnaire to obtain the preferences of healthcare experts and providers for the different mhealth applications. We chose not to include patients in the survey as we wanted to focus exclusively on mhealth application opportunities within the health care facility and this from the perspective of the provider. The focus was on the provider as the main actor or user of mHealth and the aim was for the prioritisation to reflect, and address provider/health facility specific challenges. We did however include the opinion of healthcare experts as donors and policy makers in order to identify inconsistencies in priorities so they can be addressed and funds channelled towards the appropriate projects.

The results show that the foremost priority for both groups was supply chain management; this was also a recurring theme across most sub-classes of respondents. Literature shows that drug stock-outs are among the biggest problems in public health

(35)

a big portion of the cost of primary care. Therefore, supply chain management is very important to private health facilities. Further down the list however, there are differences between the two groups on which applications should be given priority. This difference between individual application areas proved statistically insignificant save for six application areas.

An important observation is that the new application areas received recognition; each received a portion of the investment amount and was chosen by at least four respondents in each group surveyed. The new application that garnered the most attention from the experts group was provider-to-patient communication, it was ranked third most valued application overall. Despite this however, only a little over a quarter of the overall investment amount was allocated to the new application areas. In contrast, providers preferred patient safety and satisfaction which they ranked eighth overall. They allotted a little less than a third of the total investment amount to the new mhealth application areas. The findings indicate that for both experts and providers, the motivation for investing in the different mhealth applications was grounded in all four health system goals. Worth noting however is that patient centred goals were perceived as higher motivators. Quality of care, and health outcomes were more often cited as the reason a specific application was allocated funds. Both groups of stakeholders perceived solutions that promote the wellbeing of the patients of more importance than the practice-centred goals, transparency, and profitability. This is reflected in the high rankings of the new application areas, provider-to-patient communication, and patient safety and satisfaction which were perceived to contribute highly to patient wellbeing.

As Figure 4 depicts, both groups of stakeholders perceived the original mhealth applications to contribute more to health outcomes and quality of care while the new applications were perceived to have more weight on practice centred goals. This may, to some extent, explain why only a small percentage of the total investment amount was allocated to the new applications. There where however differences observed between the two groups with reference to which applications contribute most to which health systems goals.

(36)

incorporate the knowledge of stakeholders or validate the framework with experts who may provide additional applications not necessarily identified in literature. There are also limitations in the design of the survey; the most major limitation is the small sample sizes. As a result the differences reported between the two stakeholder groups were not statistically significant. Therefore extrapolation of the findings to other private health facilities, and settings, should be done with caution. Furthermore, the study was conducted within the PharmAccess network; this introduced a bias as this is a group with a predefined common goal. The opinions expressed are likely to reflect this common goal. Another limitation is that patients were not included in this study. Although this was a deliberate decision it does mean that the opinions of an important stakeholder group are not reflected in the results. However, the study does also have strengths; the foremost being that it was conducted across multiple countries therefore provides a view from four contexts that represent different needs and challenges. Additionally two versions of the questionnaire were used in conducting the survey to minimise the influence of the sequence of application domains on the investment choices.

In relation to other studies and the general body of knowledge this study provides a structured way to think about the use and contribution of mobile technology in the process of creating digital health facilities. Furthermore it provides insight on digital health investment preferences of experts who represent funders, and the providers who are the beneficiaries. The framework presents a broad perspective based on hospital functions and although this is a strength a trade off was made between breadth and depth. This health facility wide view of mhealth came at the expense of depth which could have been achieved by focussing on a specific disease course. Thus there is possibility of the exclusion of relevant applications which could only be identified through following a specific disease continuum.

The framework developed in this study and the prioritised list present valuable tools to various stakeholders in the digitalisation process of a health facility. These can be used as a buyers’ guide to enhance communication between funders and entrepreneurs. Providers of loans such as banks can use the prioritised list as a guide to - on the surface, evaluate the relevance and need of an mhealth innovation proposed, for which financing is requested. In this way banks which may not have a wealth of knowledge in mhealth trends are

(37)

list as a guide for determining start-ups in which to invest. By comparing the mhealth solution a start-up is working on to the list, an investor can tentatively determine if the solution is likely to have a market and be used.

Health care stakeholders such as non-profit organisations and providers can use the framework and list to identify solutions which can assist in the achievement of their actual goal, which is better healthcare. These groups, having determined challenges in achieving this goal can use the framework as a guide to identify the role of mhealth, and identify specific innovations that can address these challenges. Mhealth innovators as well can benefit from this framework in identifying solutions that providers’ value, and gaps in their availability. This helps to bridge the disparity between what innovators develop, and what providers regard valuable and useful towards their end goals.

As mentioned previously this study did not include patients, neither in the identification of mhealth applications nor in the prioritisation thereof. However user acceptance and perceptions are an important factor in the success of mhealth strategies, therefore we recommend that future research should expand the scope to include this aspect. Further studies into this topic may add valuable insight by applying this framework to a specific disease course as this may identify applications that have been overlooked in this study due to the framework development process employed. Furthermore, future studies that focus on the priority of applications should use bigger sample sizes and include more African countries as this would help identify key similarities and differences among private health facilities, and would be of significant value in crafting a continental digital health agenda.

(38)

7

CONCLUSION

This study presents an extended mhealth framework based on the Labrique et al. (2013) mhealth and ICT framework. This extension resulted in a more inclusive framework especially in regards to private health care facilities in low resource settings. There was agreement among all participants that the new mhealth applications identified add more value towards practice-centred outcomes as opposed to patient-centred goals. The extended framework presented can serve as a tool to facilitate communication between funders, mhealth entrepreneurs, and providers in the objective to create digital health care facilities.

Priority of each of the framework elements was determined from both the perspective of the provider and health care experts. Although the differences observed in this research are not statistically significant due to the small sample sizes, the findings offer valuable insight regarding which applications are perceived important. They reveal that at the highest level there is commonality in priority between experts, and providers. However, further along the digitalisation process more dialogue will be required among the groups to bridge the differences in opinion so that mutual benefit can be achieved.

(39)

REFERENCES

1. Boutayeb, A. (2005). The double burden of communicable and non-communicable diseases in developing countries. Royal Society of Tropical Medicine and Hygiene, 100, 191—199.

doi:10.1016/j.trstmh.2005.07.021

2. Driver, M. (2014). mHealth...what does it really mean?. Journal Of Mhealth, 1(2), 4-7. Retrieved from

https://www.joomag.com/magazine/the-journal-of-mhealth-vol-1-issue-2-apr-2014/0345100001397732888

3. About us - PharmAccess Foundation. (2017). PharmAccess Foundation. Retrieved 17 October 2016,

from https://www.pharmaccess.org/about-us/

4. Nilsen, W., Kumar, S., Shar, A., Varoquiers, C., Wiley, T., & Riley, W. et al. (2012). Advancing the Science of mHealth. Journal of Health Communication, 17(sup1), 5-10.

http://dx.doi.org/10.1080/10810730.2012.677394

5. Mechael, P. (2009). The Case for mHealth in Developing Countries. Innovations: Technology,

Governance, Globalization, 4(1), 103-118. http://dx.doi.org/10.1162/itgg.2009.4.1.103

6. Leon, N., Schneider, H., & Daviaud, E. (2012). Applying a framework for assessing the health system challenges to scaling up mHealth in South Africa. BMC Medical Informatics And Decision

Making, 12(1). http://dx.doi.org/10.1186/1472-6947-12-123

7. Woolhandler, S., Himmelstein, D., Silber, R., Harnly, M., Bader, M., & Jones, A. (1983). Public money, private control: a case study of hospital financing in Oakland and Berkeley, California. American

Journal Of Public Health, 73(5), 584-587. http://dx.doi.org/10.2105/ajph.73.5.584

8. Betjeman, T., Soghoian, S., & Foran, M. (2013). mHealth in Sub-Saharan Africa. International Journal

Of Telemedicine And Applications, 2013, 1-7. http://dx.doi.org/10.1155/2013/482324

9. GSMA Intelligence. (2016). The Mobile Economy - Africa 2016. Retrieved 23 March 2017, from

http://www.gsma.com/mobileeconomy/africa/

10. Bloomfield, G., Vedanthan, R., Vasudevan, L., Kithei, A., Were, M., & Velazquez, E. (2014). Mobile health for non-communicable diseases in Sub-Saharan Africa: a systematic review of the literature and strategic framework for research. Globalization And Health, 10(1), 49.

http://dx.doi.org/10.1186/1744-8603-10-49

11. Labrique, A., Vasudevan, L., Kochi, E., Fabricant, R., & Mehl, G. (2013). mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Global Health: Science

And Practice, 1(2), 160-171. http://dx.doi.org/10.9745/ghsp-d-13-00031

12. International Finance Corporation. (2008). The Business of Health in Africa: Partnering with the Private Sector to Improve People’s Lives. Washington, DC: The World Bank Group.

13. Peabody, J., Taguiwalo, M., Robalino, D., & Frenk, J. (2006). Improving the Quality of Care in

Developing Countries. In D. Jamison, J. Breman, A. Measham, G. Alleyne, M. Claeson & D. Evans et al., Disease Control Priorities in Developing Countries (2nd ed., pp. 1293-1307). New York: The World Bank and Oxford University Press.

14. Institute of Medicine (US) Committee on Quality of Health Care in America. (2001). Crossing the quality chasm. (1st ed.). Washington, D.C.: National Academy Press.

15. Honoré, P., Clarke, R., Mead, D., & Menditto, S. (2007). Creating Financial Transparency in Public Health: Examining Best Practices of System Partners. Journal Of Public Health Management And

Practice, 13(2), 121-129. http://dx.doi.org/10.1097/00124784-200703000-00007

16. Yong, P., Saunders, R., & Olsen, L. (2010). The healthcare imperative (1st ed.). Washington, D.C.: National Academies Press.

17. Freidoony, L., Chhabi, R., Kim, C., Park, M., & Kim, C. (2015). The Components of Self-Perceived Health in the Kailali District of Nepal: A Cross-Sectional Survey. International Journal Of

Referenties

GERELATEERDE DOCUMENTEN

ernstig letsel, uitgedrukt in aantal bromfiets-auto-ongevallen per miljard voertuigkilometer over de periode 1985-2004. De expositie is bepaald door het product van het totale

Een verklaring hiervoor zou kunnen zijn dat in de emotionele conditie de deelnemers afgeleid werden door de emotionele zinnen waardoor mogelijk de aandacht meer op de zinnen

To avoid confusion, we use the term system robustness for the ability to remain functioning under a range of possible disturbance magnitudes (see also Mens et al. In

De Rol van Morele Ontwikkeling van Jeugdige Delinquenten bij Desistance Processen 4 Verschillen in Morele Ontwikkeling tussen Delinquenten en Niet-Delinquenten 8 Verandering

De data van in totaal 39 van de 46 deelnemers zijn geanalyseerd voor de resultaten. Geen enkele deelnemer had meer dan 10% van de woordevaluaties fout, maar zeven proefpersonen

Groundwater discharge to surface water bodies was calculated with the use of a numerical groundwater flow model and analytical calculations.. 2.6.1 Numerical