MRSA-net - A tool to provide information about MRSA: A study about the usability of MRSA-net for nurses working in German hospitals

(1)

MRSA-net -A tool to provide information about MRSA

A study about the usability of MRSAnet.nl/de for nurses working in German hospitals

A. Bestert

Supervisors:

Dr. J.E.W.C. van Gemert-Pijnen Dr. J. Karreman

University of Twente Health Science

School of Management and Governance

(2)

2 A study about the usability of MRSA-net.nl/de Title:

MRSA-net.nl/de – A tool provide information about MRSA

A study about the usability of MRSA-net.nl/de for nurses working in German hospitals

Name: A. Bestert

Student number: s1098004

Supervisors: Dr. J.E.W.C. van Gemert-Pijnen Dr. J. Karreman

Institute: University of Twente Health Sciences

School of Management and Governance

Post address: P. O. Box 271 7500 AE Enschede The Netherlands

Date: 30.06.2014

(3)

A study about the usability of MRSA-net.nl/de 3

Abstract

Purpose: Methicillin-resistant staphylococcus aureus (MRSA) is the most common healthcare- associated infection (HAI). More than 5000 people per year die as a result of MRSA-infections and more than one million additional hospital days and additional costs of approximately 380 million euro are the consequences of the rising number of MRSA infections in Europe. In 2011, the rate of MRSA-infections in German hospitals was at 20%, whereas this rate was at almost 2% in Dutch hospitals. These differences in MRSA-rates caused the development of a cross-border project between the Netherlands and Germany: Euregio MRSAnet. This project is aimed on the reduction of high MRSA-rates and sustainability of low MRSA-rates in Germany and the Netherlands. One of the projects started is the development of the webpage MRSA-net.nl/de, a website that provides information about MRSA-care and prevention for the public and for healthcare personnel. This study is focused on investigating the information needs and structure preferences of nurses working in German hospitals concerning systems like MRSA-net.nl/de and evaluating the usability of MRSA- net.nl/de for this target group.

Method: 14 participants working in three different German hospitals participated in this study. All of these hospitals are part of the EurSafety Health-net and have comparable work standards and circumstances. Every participant had to go through two scenario-based think-aloud tests. Before and after these tests, interviews have been made to investigate general participant information and information about the persuasiveness of the website and the experiences that the participants made during searching information on MRSA-net.nl/de. Afterwards, the results have been analyzed using the framework method.

Results: The main problem identified by the participants was the complex structure of MRSA- net.nl/de that makes it difficult to find the information of need in an easy and efficient manner. A clearer composition of the website should be developed to increase its usefulness. The content of the website could be improved by a more user-oriented offer of information of the website and the implementation of specified information for different care givers. Only little improvements have to be made in the visual appearance of the website by the adjustment of the font size and a more explicit use of colors. The overall persuasiveness of the website could be identified as poor.

Conclusion: MRSA-net.nl/de as it currently is cannot be identified as a useful tool for giving information to nurses working in German hospitals in their everyday work. Diverse modifications have to be made in the website structure, content and visual appearance to improve the usefulness of it and increase the persuasiveness for its users. A need for a web-based tool like MRSA-net.nl/de

(4)

4 A study about the usability of MRSA-net.nl/de

as an information-giving tool in the everyday work of nurses working in German hospitals could not be identified, but the function of MRSA-net.nl/de as an additional source of information at work and at home is desired if the website would be improved.

(5)

Preface

This master thesis is the final project to obtain the Master of Science degree in Health Sciences, track Heath Services and Management, of the University of Twente.

I would like to thank my supervisors Lisette van Gemert-Pijnen and Joyce Karreman for their helpful advices and the support during the last 6 months in developing this thesis. Further, a thank to my family and friends for supporting me throughout this time.

(6)

List of abbreviations

HAI Healthcare-associated infections HCW Healthcare worker

ICU Intensive Care Unit

MRSA Methicillin-resistant Staphylococcus aureus PSD Persuasive Systems Design

UCD User-centered Design WHO World Health Organization

(9)

1 Introduction

Health-care associated infections (HAIs) form a major problem in hospitals worldwide and are one of the top ten leading causes of death around the world (Verhoeven, 2009). They are the most common adverse events resulting from a stay in the hospital (Longtin, Sax, Allegranzi, Schneider, & Pittet, 2011). Patients acquire HAIs during the treatment of other diseases or when they get in contact with other infected patients. The most common HAI is the methicillin-resistant Staphylococcus aureus (MRSA), an organism that is characterized by the resistance to all current antibiotic classes, including methicillin¹ (Enright et al., 2002). Because of the multi-resistant nature of this organism, the

possibilities for a successful treatment are limited (Peters et al., 1999). An infection with MRSA can result in a significant increase in morbidity and mortality and leads to higher costs and need of resources in hospitals and other healthcare organizations (Rosner, Becker, Wong, Miller, & Conly, 2004). More than 5000 people per year die as a result of MRSA-infections and more than one million additional hospital days and additional costs of approximately 380 million euro are the consequences of the rising number of MRSA infections per year in Europe (Köck et al., 2011).

In Germany, almost 60% of the population are intermittently carriers of MRSA, and in up to 50% of the patients in German intensive care units (ICUs), MRSA can be detected (Herbst &

Kortmann, 2007). However, the incidence of MRSA in Dutch hospitals is much lower. In 2011 the number of MRSA infections here was at almost 2% (Haenen et al., 2012), whereas in Germany this number was more than ten times higher than in the Netherlands (Köck et al., 2011). Moreover, the rate of new MRSA infections in the past decade in Germany is the highest in Europe (Herbst &

Kortmann, 2007). The differences of MRSA rates per country in Europe are given in figure 1.

1 Methicillin: A semisynthetic penicillin-related antibiotic, also known as Staphcillin, that once was effective against staph lo o i staph esista t to pe i illi e ause the p odu e the e z e pe i illi ase. (Medterms, 2014)

(10)

Figure 1: Proportion of MRSA in Europe in 2008 (RIVM, 2008)

The reasons for these differences are mainly caused by the Dutch focus on prevention. In the Netherlands, specific measures are already used before the patient is admitted on the ward. If there is the suspicion that a patient might be infected with MRSA, he will be isolated from other patients to prevent from infecting other patients, even if MRSA is not diagnosed (Schneider, 2006). Visitors of MRSA-infected patients are only allowed to enter the patients` room when wearing protective clothing. In Germany protective measures only take place when patients are already infected so that the germs have more possibilities for spreading (Schneider, 2006).

To ensure that HCWs do not face barriers in following the right protective measures, organizational circumstances have to be given to make this possible. But this is hard to realize because hospitals and healthcare organizations are facing major changes in the past decades and in the future: a rising number of patients and specialties in the hospitals, more technological and medical possibilities and more bureaucracy are just a few of the changes that the healthcare sector is facing (Schmieg, 2005). New possibilities have to be created to give HCWs the possibilities to get concise information in a fast way to deliver safe care in these continually changing circumstances.

And these possibilities have to fit with the needs and skills of the HCWs to guarantee a proper and correct use. The internet delivers a range of possibilities that could also be useful in the healthcare sector to create new measures for prevention as it continues to grow as delivery medium for health information (Kinzie, Cohn, Julian, & Knaus, 2002). Health-related websites could become an

(11)

A study about the usability of MRSA-net.nl/de 11 important tool for hospitals and other healthcare delivery organizations to reproduce useful information to HCWs about prevention and care processes.

MRSAnet.nl/de is a web-based tool for compliance with safe work practices in healthcare settings, focusing on infection control of MRSA (Verhoeven, van Gemert-Pijnen, & Hendrix, 2009). It has been developed in cooperation between Germany and the Netherlands and is aimed at the reduction of MRSA rates in Germany and the sustainability of the low MRSA rates in the Netherlands.

It gives evidence-based information to HCWs to give them the possibility for enhanced decision making and to deliver safe care to their patients (van Gemert-Pijnen, Karreman, Vonderhorst, Verhoeven, & Wentzel, 2011). The goal of this website is to deliver information about the appropriate care in terms of MRSA for healthcare personnel and the public and to meet the information needs of these user groups.

1.1 Research objective

MRSAnet.nl/de serves as a tool to offer information about MRSA care for HCWs and for the public. It is already used in a number of hospitals in Germany and the Netherlands. The objective of this study is to find out, if MRSAnet.nl/de is a useful tool to give information about MRSA care to nurses working in German hospitals and to investigate in how far information needs and structure preferences of German nurses are met on the website. Moreover the goal is to get to know, which elements of the website have to be modified to improve its usability.

1.2 Research question

Based on the research objective, the main research question is:

To what extend are user-centered web-platforms like MRSAnet.nl/de a useful tool for nurses working in German hospitals to give information in an efficient manner and how does the webpage MRSAnet.nl/de have to be modified?

(12)

In order to answer this question, this research is divided into five sub-questions:

1. How do nurses working in German hospitals search for information on MRSA-net.nl/de and to what extend are they able to retrieve the needed information successfully?

2. To what extend can the structure of MRSAnet.nl/de be described as useful for nurses working in German hospitals?

3. Does the kind and amount of information that is provided on MRSAnet.nl/de depict useful information for nurses working in German hospitals?

4. In what way can the visual appearance of MRSA-net.nl/de satisfy the expectations of nurses working in German hospitals?

5. How does the website MRSAnet.nl/de have to be modified to make it a more useful tool for nurses working in German hospitals?

1.3 Research relevance

As said in the first section of this chapter, the spreading of MRSA germs is an important topic in German hospitals. Since the Netherlands have a significantly lower incidence of MRSA in hospitals, there might be great opportunities for German hospitals to learn from Dutch prevention measures in terms of MRSA care. MRSAnet.nl/de has been developed in a network between German and Dutch health experts and brings together the knowledge and experiences of both countries. From this point, opportunities are created to implement best practices in everyday MRSA care. MRSAnet.nl/de might be an important and helpful tool to deliver information about MRSA in general and useful prevention- and therapy measures.

1.4 Structure of this thesis

In this section, the content of the different chapters of this thesis will be explained. This chapter provided a general introduction of the topic, the research objective, research relevance and the research questions that will be examined. In the second chapter some background information will be given about the Euregio- projects. After that, in the third chapter, the used theories and concepts will be explained in detail and their role in this study will be explained. The fourth chapter gives

(13)

A study about the usability of MRSA-net.nl/de 13 information about the different methods that are used and the way of analyzing the obtained data.

Chapter five is the results chapter and will present the most important evaluations and answers on the five sub-questions. In the subsequent chapter, a general conclusion will be given that gives an answer to the general research question. A discussion of several aspects of this study will be illustrated in chapter seven. The last chapter provides some recommendations for further research.

(14)

2 Background

2.1 Euregio MRSAnet and MRSAnet.nl/de

The treatment of MRSA is a major challenge for hospitals. The multi- resistant character of the germ makes it difficult to stop MRSA from spreading, once the germ has infected patients in a hospital. The ongoing spreading of MRSA has a range of consequences like a greater mortality, a longer duration of hospital stay and thus, a rise in hospital costs (Engemann, 2003). The rate of MRSA infections in the Netherlands is on a constant low level of 2% for many years (Haenen et al., 2012) whereas this rate in Germany is much higher, at 20% (Layer, Cuny, Strommenger, Werner, & Witte, 2012). This significant difference in infections rates between the Netherlands and Germany inspired the University of Twente, the Laboratory of Microbiology Twente Achterhoek, the University Medical Centre in Münster and the Ministry of Public Health Service to build up Euregio MRSAnet, a project between Germany and the Netherlands to create standards, guidelines and best practices to prevent from MRSA-spreading.

Euregio MRSAnet is a cross-border project that has been developed in 2005 in Enschede. The main idea was to develop a Dutch-German network in the border area of Twente (Netherlands) and the region Münsterland (Germany) for better prevention of MRSA. Currently the project is already introduced in the whole Dutch-German cross-border region (Friedrich, 2014a). Important milestones that have been realized with this project are the aggregation of a network consisting of all hospitals in the region, the introduction of comparable standards for screening and hygiene in these hospitals and the standardization of MRSA treatment in medical surgeries (Friedrich, 2014a).

MRSAnet.nl/de is a project that has been introduced by Euregio MRSAnet. It is a website that depicts information about MRSA for the public and for HCWs and is developed as a tool for the education, dissemination and implementation of safe work practices for MRSA prevention and treatment (Verhoeven, van Gemert-Pijnen, et al., 2009) to equalize MRSA- care and standards in the border area of Germany and the Netherlands . By using the internet as platform for this, it is possible to get concise, relevant clinical information at the place and time of need. In figure 2, the welcome- page of MRSA-net.nl/de is given. Before developing the website, research has been conducted about the needs of users and the design of the site (van Gemert-Pijnen, Karreman, et al., 2011). A user- centered design² approach has been followed in the development phase to make the tool as

2User-centered design: User-centered design (UCD) is an approach to design that grounds the process in information about the people who will use the product. UCD processes focus on users through the planning, design and development of a p odu t (UPA, 2014).

(15)

A study about the usability of MRSA-net.nl/de 15 convenient as possible for HCWs and the public (see chapter 3.1). Currently, MRSAnet.nl/de is used in a number of hospitals.

Figure 2: Welcome-page of MRSA-net.nl/de (MRSA-net, 2013)

2.2 The EurSafety Health-Net seal of quality for MRSA prevention

The EurSafety Health-Net has been developed as a follow-up project of MRSAnet and is also realized in the German-Dutch border area project. It has, almost in the same manner as MRSAnet, been founded to overcome the differences in healthcare quality between Germany and the Netherlands to make it possible that patients can make use from an excellent health care in their home country, and across national boundaries. The main goal of this project is to bring prevention quality in Germany and the Netherlands on the same, high quality level and to improve patient safety and infection control (Friedrich, 2014b). The difference between EurSafety Health-Net and MRSAnet is, that EurSafety Health-Net not only focuses on MRSA, but also on the prevention of other inflectional diseases (Friedrich, 2014a). This is realized by a number of projects, for example the development of surveillance-projects, the conduction of research in the domain of infection control and by the certification of hospitals (Friedrich, 2014a).

The certification of hospitals has been introduced in the EurSafety Health-Net project to set standardized quality goals for the hospitals that are joining the project (Friedrich, 2014c). Only hospitals that are actively working on the improvement and success of the predetermined quality

(16)

goals are certificated with a seal of quality. With this, an increase in quality and transparency for patients and between hospitals is ought to be achieved. All in all five seals exist in the EurSafety Health-Net project, one of them is the quality seal for MRSA prevention. Hospitals that are striving to be certificated with this seal have to be active on a range areas in the domains prevention, therapy, rehabilitation, education, regional cooperation and the implementations of guidelines and best practices concerning MRSA-care (Friedrich, 2014c). In this study, only hospitals are included that are certificated with the EurSafety Health-Net seal of quality for MRSA prevention. This is because the hospitals that have these seals of quality might be more comparable to each other in MRSA prevention measures. Also work circumstances and the education level of nurses are on a comparable level. This leads to a higher validity of the study and more powerful conclusions.

(17)

3 Theory and Concepts

3.1 User-centered design

In the development phase of a website or an internet-application, different approaches of designs can be chosen. One approach that is aimed at developing the new technique in a way that takes the needs and wishes of its pote tial use s i to a ou t is the use - e te ed desig . User-centered design (UCD) is defined as `design processes in which end-users influence how a design takes shape´

(Abras, Maloney-Krichmar, & Preece, 2004) and is the cont a of the so alled e pe t-based desig ³. It describes an approach in which users are involved in the development of a system with the goal of an increased identification of the end-user with the product and hence, a higher usefulness and usability. By using this approach, the needs of users can be detected and taken into account in the development of a new system or the improvement of an already existing system (Mao, Vredenburg, Smith, & Carey, 2005). In the health-economic sector, the UCD approach is chosen frequently because of a number of benefits like increased sales and customer satisfaction, competitive advantage or reduced training costs that are affiliated with it (Reed, 2014).

The application of the user-centered design can be conducted in different ways. One way is performing background interviews and questionnaires. With this method, data can be collected concerning the needs and expectations of users or the evaluation of design alternatives can be made.

Another method that can be used in UCD is making sequence of work interviews. Here, the aim is to collect data related to the sequence of work to be performed by the system. Also the creation of focus groups where a wide range of stakeholders are included to discuss issues concerning the system is a frequently used method in UCD and is mostly used in early stages of the design process (Abras et al., 2004). And these are just a few of the methods that are used in UCD.

In the domain of eHealth technology, the use of UCD in its development and evaluation can be an important factor for the adoption of the technology among its potential users. EHealth can be defined as the transfer of health resources and health care by electronic means. It encompasses the delivery of health information, the improvement of public health services with the use of IT and e- commerce and the use of e-commerce and e-business practices in health systems management (WHO, 2014). MRSA-net.nl/de can also be assigned to the domain of eHealth technologies. A variety

3 Expert- ased desig : „E pe t-based methods have the aim of uncovering potential usability problems by having evaluator inspect a user interface with a set of guidelines, heuristics or questions in mind or by

performing a step-wise approach, derived from general k o ledge a out ho hu a s p o ess th ough tasks.

(Jaspers, 2008)

(18)

of stakeholders like health care professionals, patients, relatives or investors can be involved in the development process of an eHealth technology. All of them are facing different needs and wishes that have to be coordinated and considered in the development process for a better adoption of the technology (van Gemert-Pijnen, Nijland, & H.C., 2011). One approach that is developed to fit with the needs of all stakeholders and users of an eHealth technology is the CeHRes roadmap (van Limburg, van Gemert-Pijnen, & Seydel, 2011).

Figure 3: CeHRes Roadmap (van Gemert-Pijnen, Nijland, et al., 2011)

This roadmap is developed because eHealth technologies often disregard the

interdependencies between technology, human characteristics and the socioeconomic environment in the healthcare sector (van Gemert-Pijnen, Nijland, et al., 2011). It is a user-centered approach that can be used to plan, coordinate and evaluate the development process of eHealth technologies. As given in figure 3, this approach consists of 5 phases: The contextual inquiry, value specification, design, operationalization and summative evaluation (Nijland, 2011).

 Contextual inquiry: This is the first phase of the roadmap. Here, the design team gets an understanding of potential users and their context.

 Value specification: In this phase the values of the different stakeholders are

determined. Needs and wishes of the potential users are investigated and translated into user requirements.

 Design: The third phase is aimed at developing the technology based on the requirements that have been investigated in the second phase. The application of the design should be performed in cooperation between the design team and prospective users and stakeholder together.

(19)

 Operationalization: Here, the technology is launched and marketing plans are made.

Organizational working procedures are realized.

 Summative evaluation: In this last phase of the roadmap, the eHealth technology is evaluated. Its effect on the users is detected and potential for improvement is investigated.

The products that are created in each phase of the roadmap should also be subject of a formative evaluation to gather input for improving the product. With this, a continuous improvement of the eHealth application is possible.

In this study, the focus is on the last phase of the CeHRes roadmap, the summative evaluation. The website MRSA-net.nl/de is evaluated with a method called usability testing. Here, qualitative data are collected related to measurable usability criteria. This is done by a specific method, called think aloud testing. This method will be outlined in the methods section. The goals of this method are to improve the products usability, to involve real users in the testing, to give the users real tasks to accomplish, to enable testers to observe and record the actions of the participants and to enable testers to analyze the data obtained and make changes accordingly (Dumas & Redish, 2000). In addition to the usability testing, interviews are made to collect qualitative data related to user satisfaction with the website. This is also a commonly used method in UCD (Abras et al., 2004).

In the development-phase of MRSAnet.nl/de, a user-centered approach has been chosen to get insights in the pote tial use s thinking and acting and to better understand and meet the use s needs (Verhoeven, van Gemert-Pijnen, et al., 2009). This study will evaluate in how far the

development of the website using this approach contributes to the usability of the website for German nurses and which aspects have to be improved.

(20)

20 A study about the usability of MRSA-net.nl/de 3.2 Persuasive Systems design

The term persuasive systems design can be described as the design of `computerized software or information software designed to reinforce change or shape attitudes or behavior or both without using coercion or deception´ (Oinas-Kukkonen & Harjumaa, 2009). This definition underlines the main characteristics a persuasive system has to have: is must change, reinforce or shape user attitudes and behaviors in a way that is desired by the developers of the system.

Figure 4 describes the role of a persuasive technology, like MRSA-net, and its role in the persuasion process. Berdichevski and Neuenschwander compare this role with billboards: U like

ill oa ds, the i te a t d a i all ith the o je ts of pe suasio (Berdichevski &

Neuenschwander, 1999). A persuasive system like MRSA-net.nl/de intermediates actively between the persuader, in this case, the designers of the website, and the persuaded person, the end-user.

With this, a change in behavior and attitudes can be achieved. In the case of MRSA-net.nl/de, another behavior concerning looking-up information about MRSA could be achieved.

Figure 4: Persuasion with technological systems (Berdichevski & Neuenschwander, 1999)

One of the key issues behind the PSD is that persuasive systems should aim at being both, useful and easy to use (Oinas-Kukkonen & Harjumaa, 2009). This means, that the system should serve the needs of the user. The user should profit from the use of the system by a high information quality, lack of errors and attractiveness. In case the system is useless or difficult to use, it would be valueless for the potential user.

A persuasive way of designing a system for its potential users increases the probability that this system will be used significantly. A range of studies demonstrate this and the fact that a

persuasive systems design can have large impact on a systems usability and acceptability (Eysenbach

& Cugelman, 2012). Therefore, it is important to investigate, in how far MRSA-net can be called persuasive and which aspects of the site have to be changed to make it more persuasive and by that increase the usability of the system. This study will examine the persuasiveness of MRSA-net.nl/de

(21)

A study about the usability of MRSA-net.nl/de 21 and possible improvement-factors concerning the website persuasiveness. This will be done by analyzing several criteria of the persuasive systems design by Oinas-Kukkonen that are critical for the usability of MRSA-net.nl/de.

The principles of persuasive systems design can either be used for the development of new software, or for the evaluation of software that already exists. In this study, the PSD-principles serve as evaluation criteria for the evaluation of MRSA-net.nl/de. A range of principles are developed by Oinas-Kukkonen and Harjumaa to judge and develop the software (table 1). All of these all in all 28 principles describe specific requirements a system should have to make it a persuasive system.

(22)

Table 1: Categories and principles of persuasive systems design (Oinas-Kukkonen & Harjumaa, 2009)

Category Principles

Primary Task Support  Reduction*

 Tunneling

 Tailoring*

 Personalization*

 Self-monitoring

 Simulation

 Rehearsal

Dialogue Support  Praise

 Rewards

 Reminders

 Suggestion

 Similarity

 Liking

 Social role

System Credibility Support  Truthworthiness

 Expertise*

 Surface credibility*

 Real-world feel

 Authority

 Third-party endorsements

 Verifiability

Social Support  Social learning

 Social comparison

 Normative influence*

 Social facilitation

 Cooperation*

 Competition

 Recognition

*Principles of the PSD that are examined in this study

The principles are subdivided into the four categories primary task support, dialogue

support, credibility support and social support. The category primary task support contains principles that suppo t the a i g out of the use s p i a task. Dialogue support is a category that deals

(23)

A study about the usability of MRSA-net.nl/de 23 with the feedback offered by the system to guide the user to reach the intended behavior (Torning &

Oinas-Kukkonen, 2009). The design principles in the category system credibility support illustrate how a system should be designed to make it more credible and persuasive. Principles of the last category social support affect the overall persuasiveness of a system and explain motivation guidelines for users for leveraging social influence(Oinas-Kukkonen & Harjumaa, 2009). For a detailed explanation of all the principles, see appendix IA.

These principles are investigated in several questions asked in the post-scenario interviews that are made with the participants after the scenarios. The goal is to evaluate the persuasiveness of MRSAnet.nl/de. Not all principles are investigated in the interview questions because this would obtain an amount of data that would be too huge for a paper like this and would go beyond the scope of a master thesis. Moreover, not all of the principles would be appropriate to test the persuasiveness of MRSA-net.nl/de. Only the principles that are critical for the evaluation of the website are also part of the interview content. The principles that are used for the interview questions are judged and chosen by the researcher and had to meet the following conditions: 1) be appropriate for measuring and judging usability, 2) be appropriate to evaluate the structure of MRSAnet.nl/de and 3) have to give the participants the opportunity to give their opinion about the quality of the information provided on the webpage and the visual appearance of the webpage. The following criteria meet these conditions and are chosen to be evaluated in the post-scenario

interviews: Tailoring, Reduction, Personalization, Expertise, Surface credibility, Cooperation, Normative influence.

These 7 criteria serve as a basis for investigating the persuasiveness of MRSA-net.nl/de. They belong to the categories social support, system credibility support and primary task support. No criteria of the category dialogue support are part of the investigation because this category provides principles to evaluate the interactivity of a system. MRSA-net.nl/de is not an interactive system and thus, this category is not appropriate to measure the persuasiveness of the website. How the chosen criteria are analyzed and judged will be explained in the following chapter.

(24)

4 Methodology

In this chapter, the methods used in this study are explained in detail. In the first part of this chapter the data collection will be given. The second part of the chapter explains the different steps of the research design and procedure. After that, the materials used in this study will be discussed.

Afterwards the case selection and sampling will be explained. At the end of this chapter, the data analysis will be outlined.

4.1 Data collection

The data used in this research are obtained via three instruments:

 Literature study

 Interviews

 Scenario-based tests using the think aloud method

The main objective of the literature study is to get to know more about the topic MRSA itself and its importance in German hospitals. Also more knowledge about the development and implementation of MRSAnet.nl/de and the importance of hand hygiene in infection control is gathered via literature study. The internet sources that are used for this study are Pubmed, Google Scholar, Scopus and the webpage of the online bibliography of the University of Twente. As a result of literature study, two important articles are found that act as frame for the methodological concepts of this research: At first the article A o pa iso of usa ility methods for testing interactive health technologies:

Methodologi al aspe ts a d e pi i al e ide e (Jaspers, 2008). The explanations about the user- ased usa ilit ethod Thi k aloud i the o te t of s e a io testi g a e used as a asis fo the scenarios that are conducted in this research. The article Pe suasi e “ ste s Desig : Ke Issues, P o ess Model a d “ ste Featu es (Oinas-Kukkonen & Harjumaa, 2009) serves as a foundation for the investigation of the persuasiveness and usability of MRSA-net.nl/de.

The interviews made before and after the scenarios are in first place performed to find out more about the participants individual opinions and needs concerning MRSAnet.nl/de. In the interviews participants give personal information and are able to express their experiences with the system and which elements of the webpage have to be modified.

(25)

A study about the usability of MRSA-net.nl/de 25 Besides conducting the interviews, every participant walks through two scenario-based think aloud tests. As already said in the third chapter, one method that is used in a user-centered design approach is this method. It is developed for gathering information on the cognitive behavior of humans performing tasks (Jaspers, 2008). The think aloud method is a very direct way in gathering i fo atio e ause it assesses hu a s og itio s u e tl he the o u . T pi al users perform regular standard tasks in their typical task environment. By doing this, data like the time users need to perform a specific function, the type and rate of errors by users or the subjective user satisfaction can be collected (Abras et al., 2004).

The think aloud method belongs to the verbal report methods and has its origin in the field of cognitive psychology. Participants performing the think aloud method are probable end-users that perform a series of tasks in interaction with the new developed system, or the system that is

evaluated, while verbalizing their thoughts (Jaspers, 2008). That means, that participants are stating directly what they think during performing the task. With this, it is possible to gain insight in the way humans think, act and solve problems. Verbal reports are made of the participants so that afterwards an analysis can be made that gives a detailed insight into the usability problems that have been experienced by the end-user.

The think aloud method is a commonly used method to evaluate the usability of systems and their designs. In this study, the think aloud method is used to investigate the usability of

MRSAnet.nl/de for nurses in German hospitals in the field hand-hygiene. The way in which this method is used in this study will be outlined in the following section

4.2 Research Design & Procedure

This research is conducted in four consecutive steps. In this paragraph, these steps are enlightened more in detail. Figure 5 gives an overview of these steps.

(26)

Figure 5: Research design & procedure

In the first step of this research, pre-scenario interviews are made with all of the participants. The questions asked in these interviews are aimed at obtaining information about the participant himself and about the needs and expectations the participant has concerning a web-based platform like MRSAnet.nl/de. The information acquired about the participant himself consists of individual characteristics like age, work position, experiences with MRSA infection control or with other web- based applications. The i fo atio a out the pa ti ipa ts eeds a d e pe tations concerning MRSAnet.nl/de delivers information about the advantages a system has to have for the participant and which need of information about MRSA prevention and intervention measures exists.

In the second step the scenarios are conducted with the participant. All of the participants have to go through two different scenarios. These scenarios give realistic situations in hospitals everyday care. In all scenarios the care of MRSA infected patients is concerned and thus, specific hand hygiene prevention measures have to be taken by the participants. The goal of the scenario- based testing is to find out, how the participants are searching for the needed information, if they are able to find it and how they evaluate the website.

After walking through the scenarios, another interview is made with the participants. In this interview the objective is to find out, in how far the expectations and needs of the participants are met and in which areas problems occurred when searching for relevant information. The questions that are asked in these post-scenario interviews consist of some general questions, questions about the usability of the website and questions based on the criteria of a persuasive systems design

Pre-test interviews

Scenario-based testing using

"Think aloud"

Post -test interviews

Data transcription and coding

(27)

A study about the usability of MRSA-net.nl/de 27 (Oinas-Kukkonen & Harjumaa, 2009). The aim here is to find out, in how far the system is persuasive for its potential users, to find out where improvements have to be made and in how far these improvements have to be done. This information can provide a basis for the evaluation of MRSAnet.nl/de.

After conducting the interviews and tests, the data analysis is carried out. This will be explained more in detail in section 4.4.

4.3 Materials

As already explained in the previous section, this study is based on conducting interviews and scenario-based tests.

In the pre-scenario interviews contain questions all in all 8 questions are asked concerning age, work position and experiences of the participants with web-based applications like MRSA- net.nl/de. All i te ie s a e e o ded a oi e e o de appli atio alled Eas oi e e o de and serve as a basis for later transcription. All questions are attached in the appendices (see appendix IB).

After the first interview, two scenarios are conducted. These scenarios are focussed on the topic hand hygiene because this is one of the most urgent measures to prevent from infections in

hospitals. A range of studies underline the importance of hand hygiene in hospitals and its major role in infection control (Allegranzi & Pittet, 2009). If hand hygiene measures are not taken in a right way, it is eas fo pathoge s like M‘“A to sp ead. Fo e a ple, i ase that patie t s ski is olo ized ith pathogens, there is a big chance that their environment will be contaminated with these pathogens, too. The environment is in this case the hospital, where HCWs act and work and have contact with diverse patients. This contact is mainly performed by hand contact, and this increases the danger of transfer and microbial colonization in the hospital. If hand hygiene measures are followed in a proper way, this danger is reduced significantly. Therefore, the scenarios in this study are focusing on hand hygiene exemplary for all important aspects of hygiene concerning MRSA-care. All scenario-based think aloud tests are recorded and filmed via a screen-video-p og a e alled Ca “tudio . The scenarios are as follows:

 Scenario 1: Imagine you are on your ward and you have to give an injection to an MRSA patient.

After that, you have to bandage an infected wound of a patient on another ward that is not infected with MRSA. Which hand hygiene measures have to be taken before and after giving the injection? Try to find the relevant information with MRSAnet.nl/de.

(28)

 Scenario 2: Imagine you take sanitary measures on an MRSA patient. Afterwards you recognize that one of your gloves has been damaged. Which hand hygiene measures have to be taken in this situation? Try to find the relevant information with MRSAnet.nl/de.

After conducting the scenarios, the post-scenario interviews are made. The number of questions per interview can vary, dependent from the ability and willingness of the participants to give

information. As a basis, 12 post-scenario questions are developed (see appendix IC).

4.4 Case selection and sampling

All in all 14 participants are recruited for this study. All of these participants are nurses that are working in hospitals in Vreden, Ahaus and Stadtlohn (Germany). Four participants are working in the hospital in Vreden on the geriatric ward, six are working in the hospital in Ahaus on the ICU, and four are working in the hospital in Stadtlohn on the general care ward. These hospitals are all leaders of the EurSafety Health-Net and all have the seal of quality for MRSA prevention. The experiences of the nurses with the internet range from very low experience to very much experience. All in all 13 women and 1 man participated in this research. Their age ranges between 19 and 50 years. The mean age of the participant is 35 years (see analytical framework).

4.5 Data analysis

The last step that is done in the research process is the transcription and coding of the collected data and the analysis of the obtained data. Because of the qualitative nature of these data, a clearly structured coding approach is critical for data analysis. This is done by using a specific approach for data analysis, called the `framework method´. The `framework method´ is becoming increasingly popular in the analysis of qualitative data in health research (Gale, Heath, Cameron, Rashid, & Redwood, 2013). Mostly it is used for the data analysis in multidisciplinary research, but is also useful for the analysis of the qualitative data obtained in this study because the amount of data obtained can be analyzed in a structured manner that no other analyzing approach can deliver. The approach consists of seven steps that are developed to make the analysis and coding process more simple and transparent (figure 6).

(29)

Figure 6: Steps of the "Framework Method" (Gale et al., 2013)

Step 1- Transcription

Before the first phase of data analysis can be started, all interviews and scenario-based think-aloud tests of all nurses are done and recorded. The interviews are recorded and the think aloud tests are recorded and filmed. These recordings are used for the transcription process. The transcripts are characterized by large description field s and adequate line spacing. That makes the coding process less complicated and gives space for some notes. All coded transcripts of the interviews are attached in appendix ID. The coded transcriptions of the scenarios are attached in appendix IE.

Step 2- Familiarisation with the interviews and scenario-based tests

For the interpretation of all collected data, it is important to become familiar with all information that is given. This happens in step 2 of the data analysis. All transcripts are read and all recordings and films of the tests are watched, listened and re-listened to get a full understanding of the

pa ti ipa ts state e ts. In this phase, a first impression is given about the meanings and feelings of the participants and eventually about some antithetic opinions that are expressed by them.

Step 3- Coding:

The third phase is called the coding phase. The main activity that takes place here is the reading of the transcripts line by line. By applying a paraphrase or a label ( code ) that describes what can be

Step1: Transcription

Step 2: Familiarisation with the interviwes

Step 3: Coding

Step 4: Developing a working analytical framework

Step 5:Applying the analytical framework

Step 6: Charting data into the framework matrix

Step 7: Interpreting the data

(30)

interpreted by means of these data, a clear structure is given per transcript about what is important.

Codes in this study can refer to for example the expression of particular behaviours, criticism, emotions or certain ideas for improvement. The codes that are used for the interpretation of the data of this study are gathered from the transcripts and thus developed at the basis of the information that is given by the participants during the interviews and tests. In this research, four main codes are used that are sub-divided into a number of sub-codes (see analytical framework appendix II).

Because this analysis method is frequently used in multidisciplinary health research with several researchers, usually two or more researchers are developing codes. In this research the codes are developed by two different researchers to guarantee that the coding structure is valid. One coder is the researcher himself, having a health science background. The other coder is a social sciences student. All codes are developed by at first coding the transcripts individually and in the second step discussing the different codes and coming together to a set of codes.

Step 4- Developing a working analytical framework

After all codes are developed for all transcripts, codes and sub-codes are defined and grouped together. In this phase some new codes and sub-codes are developed or existing codes are changed.

This takes place in discussion between the two coders and forms the basis for a working analytical framework.

Step 5- Applying the analytical framework

The analytical framework that is developed in step 4 is then applied by cataloguing the different transcripts using the existing codes and sub-codes. All coded transcripts are attached in the appendices (see appendix IE and IF).

Step 6- Charting data into the framework matrix

In this step, the collected data are managed and summarized by charting them into a matrix.

Charting is performed by summarizing the data by sub-code from each transcript. The objective in this step is to reduce the amount of data on the one hand, and to retain the means of the original interviews on the other hand. The framework matrix is attached in the appendices (see appendix II).

Step 7- Interpreting the data

The interpretation of all interviews and tests with all nurses is the final and most important step in data analysis. This interpretation is carried out by answering the predefined five sub questions that

(31)

A study about the usability of MRSA-net.nl/de 31 are given in chapter 1.2. All sub-questions are answered in the results section. In the conclusion an answer is given to the main research question.

(32)

5 Results

In this chapter, answers will be provided on the five sub-questions ´ How do nurses working in German hospitals search for information on MRSA-net.nl/de and to what extend are they able to retrieve the needed information successfully?`, ´To what extend can the structure of MRSA-net.nl/de be described as useful for nurses in their everyday work? `, ´Does the kind and amount of information that is provided on MRSA-net.nl/de depict useful information for German nurses? `, ´In what way can the visual appearance of MRSA-net.nl/de satisfy the expectations of German nurses? ` and ´How does the website MRSA-net.nl/de have to be modified to make it a more useful tool for nurses working in German hospitals?`.

In order to answer all of these questions, in section 5.1 the searching process of the participants will be explained and their success in finding the information needed. In section 5.2, information about the usefulness of the structure of the website and its fit into the everyday work circumstances of German nurses is provided. After that, in section 5.3, the pa ti ipa ts opi io about the kind and amount of information that the website contains will be discussed. Hereafter, in section 5.4 it will be argued, in how far the visual appearance of MRSA-net.nl/de is satisfying for German nurses. Section 5.5, the last section of this paper, is about the modifications that have to be made to make MRSA-net.nl/de a more useful tool for German nurses.

(33)

A study about the usability of MRSA-net.nl/de 33 5.1 How do nurses working in German hospitals search for information on MRSA- net.nl/de and to what extend are they able to retrieve the needed information successfully?

This chapter provides information about the searching process that is used by the participants to find information on MRSA-net.nl/de and the ability to complete the scenarios successfully.

5.1.1 Results searching process

All participants that were part of this study had to go through two scenario-based tests, as described in the previous chapter. The goal was to find the relevant information that is needed to solve the situation described in these scenarios. The way in which this information is searched has individually been chosen by the participants. In the screen videos, the way in which the participants clicked through the website of MRSA- et. l/de a d the thi k –aloud o e ts ade the have been recorded.

On MRSA-net different possibilities exist do gather the information of need. One Method is the use of the task bar (see figure 7).

Figure 7: Task bar on MRSA-net.nl/de (MRSA-net, 2014b)

Here different menu items with topics concerning MRSA are listed. With a click on one of the menu items, a range of questions is given per topic. The second possibility to look for information on the website is the look-up field (see figure 8).

(34)

Figure 8: Look-up field on MRSA-net.nl/de (MRSA-net, 2014b)

With the loo-up field the user has the possibility to tick in a word or sentence he/she is willing to look for. After ticking in these words, a number of hits are given concerning these search items. Table 2 and table 3 give an overview of the ways in which the participants clicked through the website in scenario 1 and 2 respectively, whether in the task bar or look-up field.

Table 2: Search process scenario 1

Participant Start task bar

Start look- up field

Keep searching in task bar

Keep searching in look-up field

Search in task bar and look- up field

Click back

1 x x x

2 x x x

3 x x

4 x x x

5 x x

6 x x x

7 x x

8 x x

9 x x x

10 x x x

11 x x

12 x x x

13 x x x

14 x x x

Total 11 3 8 2 4 9

(35)

Table 3: Search process scenario 2

Participant Start task bar

Start look- up field

Keep searching in task bar

Keep searching in look-up field

Search in task bar and look- up field

Click back

1 not done

2 x x x

3 x x

4 x x x

5 x x x

6 x x x

7 x x x

8 x x x

9 x x x

10 x x x

11 x x x

12 x x

13 x x x

14 x x x

Total 11 2 8 0 5 11

´Start task bar` in this case means that the first action the participant takes to look for information in the website is clicking on a menu item in the task bar. ´Keep searching in task bar`

means that the participant keeps looking for the information in the task bar during the entire scenario. Corresponding to that ´Start look-up field` means that the participant starts to look for information by using the look-up field and ´Keep searching on look-up field `means that the

participant keeps on using this method for gathering information during the entire scenario. ´Search in task bar and look-up field` denotes that the participant makes use of both of these methods during the scenario. The item ´Click back` describes that the participant clicks one or more sites back during the searching process on MRSA-net.nl/de.

The results given in the two tables above show, that the majority of the participants starts searching for the needed information by making use of the task bar. This was the case in 22 of the 27 scenarios that have been conducted (81%). In 16 of these 22 scenarios (73%) the task bar has been used during the whole scenario. No use has been made of the look-up field during the scenarios to look for the needed information here. In the remaining 6 scenarios (13%), the participants started by using the task bar made use of both, the task bar and the look-up field during the scenarios.

Only a small number of the participants started with the look-up field in their search process.

This was the case in 5 of the 27 scenarios (19%). Only in 2 of these scenarios (40%), the participants kept on looking for the information with this method. In the remaining 3 scenarios (60%) participants chose to keep on searching for the information with the task bar rather than with the look-up field.

MRSA-net - A tool to provide information about MRSA: A study about the usability of MRSA-net for nurses working in German hospitals