Scientific Research Project
K.F. Driest, BSc.
February 2015
Measuring quality of
teleconsultation services,
the patients’ perspective
The development of a concept questionnaire according to
the first phase of the Consumer Quality Index
Measuring quality of teleconsultation services, the
patients’ perspective
The development of a concept questionnaire according to the first phase of the
Consumer Quality Index
Trainee
Keiko F. Driest Student number: 6174280 E-mail: keiko.driest@gmail.comMentor
MSc, L. ThijssingAcademic Medical Center – University of Amsterdam Department of Medical Informatics
Meibergdreef 9
1105 AZ Amsterdam, the Netherlands KSYOS TeleMedisch Centrum
Professor J.H. Bavincklaan 2 - 4 1183 AT Amstelveen, the Netherlands Email: l.thijssing@amc.uva.nl
Tutor
Prof. Dr. M.W.M. Jaspers
Academic Medical Center – University of Amsterdam Department of Medical Informatics
Meibergdreef 9
1105 AZ Amsterdam, the Netherlands Email: m.w.jaspers@amc.uva.nl
Location Scientific Research Project
Academic Medical Center – University of Amsterdam Department of Medical InformaticsMeibergdreef 9 1105 AZ Amsterdam
KSYOS TeleMedisch Centrum Professor. J.H. Bavincklaan 2 - 4 1183 AT Amstelveen
Practice Teaching Period
December 2013 – February 2015Preface
This thesis is the final assignment of the Master Medical Informatics of the University of Amsterdam. It describes my scientific research project (SRP). I performed my SRP at the department of Medical Informatics of the Academic Medical Center (AMC) in Amsterdam and KSYOS TeleMedical Center in Amstelveen. The aim of this thesis is to develop a standardized concept questionnaire according to the first phase of the Consumer Quality Index to assess patients’ experiences with teleconsultation services. This thesis reports on the first step of the development and has as result a concept questionnaire.
I express my deep gratitude to my mentors Leonie Thijssing, Linda Dusseljee-Peute (the first period) and my supervisor Monique Jaspers for their guidance, feedback, support and supervision throughout this research project.
Furthermore I would like to thank Job van der Heijde and Leonard Witkamp for the opportunity to perform my SRP at KSYOS TeleMedical Center. I have learned a lot in the past year.
Without the patients, the GPs, the dermatologist, Job van der Heijde and Arda van Breda (as teleconsultation providers) for the focus group and workgroup, there would not be a final concept questionnaire available. Therefore I thank them for their time and feedback.
I wish to express my gratitude to my family, boyfriend and friends for supporting me and making it possible to finalize my study. Special thanks to all the people that reviewed my work during the different stages of progress.
Finally, I thank my fellow students in the room J1b-213.1 at the Medical Informatics department for the support and the great time. Konec Hry!
Keiko Driest
Table of contents
Samenvatting ... 5
Abstract ... 7
1
General introduction ... 9
Bibliography ... 11
2
Literature review: Quality aspects of teleconsultation from the patients’ perspective 12
2.1 Introduction ... 12 2.2 Method ... 13 2.2.1 Search strategy ... 13 2.2.2 Study selection ... 13 2.3 Results ... 15 2.3.1 Quality aspects ... 17 2.4 Discussion ... 19 2.5 Conclusion ... 21Bibliography ... 22
3
Development of the CQ-index for Teleconsultation services ... 24
3.1 Introduction ... 24
3.2 Background CQ-index ... 25
3.2.1 Development phases CQ/index ... 25
3.3 Methods... 26
3.3.1 Development of the concept CQ-index of Teleconsultation Services ... 26
3.3.2 Study participants of the focus groups ... 26
3.3.3 Focus groups ... 27
3.3.4 Mapping the quality aspects of the literature review with the quality aspects of the focus groups 28 3.3.5 Validation Workgroup ... 28 3.4 Results ... 28 3.4.1 Focus groups ... 28 3.4.2 Quality aspects ... 28 3.4.3 Clustering ... 29 3.4.4 Importance rating ... 29 3.4.5 Mapping ... 35 3.4.6 Validation workgroup ... 35 3.4.7 Final Questionnaire ... 36 3.5 Discussion ... 37 3.6 Conclusion ... 40
Bibliography ... 41
List of abbreviations ... 43
Appendix A: Approval of the Medical Ethical Commission ... 44
Appendix B: Invitation letter ... 46
Appendix C: Confirmation letter ... 49
Appendix D: Background characteristics ... 52
Appendix E: From quality aspect to question ... 53
Appendix F: Origin of questions and the changed made by the Workgroup ... 60
Samenvatting
Doel:
De vraag van de gezondheidszorg in Nederland groeit. In de afgelopen tien jaar groeide de uitgaven in de gezondheidszorg met 4,4% per jaar. De verwachting is dat deze groei zich zal voortzetten. Telemedicine fungeert als een potentieel middel om de kosten van de gezondheidszorg te verlagen, de toegang tot diensten te verbeteren, de juiste expertise overal beschikbaar te maken, in een vroeg stadium te zorgen voor de juiste behandeling en het verbeteren van de kwaliteit van de zorg. De zorg die wordt verstrekt via teleconsultatie diensten heeft bewezen zo efficiënt en effectief te zijn als fysieke consult met een hoge diagnostische nauwkeurigheid. Echter, slaagt het beschikbare onderzoek niet in om een bevredigende uitleg van de onderliggende redenen voor de tevredenheid van de patiënt (of ontevredenheid) met Telemedicine te bieden. Bovendien laat bestaande literatuur beperkingen zien van patiënt tevredenheid vragenlijsten met betrekking tot Telemedicine diensten en ook beperkingen in het ontwikkelingsproces van deze vragenlijsten. Specifieker, bij deze studies ontbreken vaak informatie over de betrouwbaarheid, factoranalyse en de geldigheid van de vragenlijsten gebruikt om de tevredenheid van de patiënt met Telemedicine diensten te beoordelen. Bij KSYOS TeleMedisch Centrum is er behoefte om ervaringen van patiënten te meten met de Teleconsultatie diensten die zij leveren. Er is daarom behoefte naar een gestandaardiseerde en gevalideerde vragenlijst. De antwoorden op dergelijke vragenlijst kan worden gebruikt om de kwaliteit van zorg geleverd door teleconsultatie diensten te verbeteren. Het KSYOS Teleconsultatie Systeem (KSYOS TeleMedisch Centrum) biedt huisartsen de mogelijkheid om te communiceren met andere zorgverleners (specialisten). Deze mededeling is gebaseerd op de ‘store and forward’ modaliteit. In deze scriptie willen we een gestandaardiseerde concept vragenlijst ontwikkelen.Methoden:
De concept vragenlijst is ontwikkeld op basis van de Consumer Quality Index (CQ-index). De CQ-index is een gestandaardiseerde methode voor het ontwikkelen van vragenlijsten en het meten van kwaliteit van zorg vanuit het perspectief van de patiënt. De CQ-index bestaat uit drie fases. De eerste fase bestaat uit het maken van een concept vragenlijst. De volgende stappen werden in de eerste fase uitgevoerd: (i) een systematische literatuuronderzoek dat gericht is op het identificeren van kwaliteitsaspecten met betrekking tot teleconsultatie diensten die patiënten ervaren als relevant of belangrijk, (ii) twee focusgroepen met patiënten gericht op het identificeren van kwaliteitsaspecten met betrekking tot de levering van store-and-forward Teleconsultatie diensten, (iii) het in kaart brengen van de resultaten van de systematische literatuurstudie en de resultaten van de focusgroepen om een unieke lijst te creëren van de kwaliteitsaspecten, (iv) het ontwikkelen van vragen aan de hand van de kwaliteitsaspecten en (v) het pre-valideren van het concept vragenlijst met een werkgroep bestaande uit stakeholders van Teleconsultatie diensten (patiënten, huisartsen, specialisten, leveraars van de teleconsultatie diensten). De eerste fase van de CQ-index is uitgevoerd in deze studie.Resultaten:
Uit de literatuur studie zijn er 22 kwaliteitsaspecten gevonden. Uit de focus groepen zijn er ook 22 kwaliteitsaspecten gevonden. Vijf van de kwaliteitsaspecten uit de focus groepen kwamen overeen met kwaliteitsaspecten van de literatuur studie. Twee kwaliteitsaspecten uit de literatuur studie zijn niet gebruikt in het concept vragenlijst. 19 verplichte vragen, geleverd door de CQ-index zijn toegevoegd aan de concept vragenlijst. De ontwikkelde concept vragenlijst bestaat in totaal uit 56 vragen, gegroepeerd in de thema’s die geleverd zijn door de CQ-index. De concept vragenlijst bestaat uit: (i) twee introductie vragen (ii) drie vragen betreffende het thema toegang tot zorg, (iii) vijf vragen betreffende het thema communicatie en informatie, (iv) vijf vragen betreffende hetthema bejegening, (v) acht vragen betreffende het thema patiënt regierol, (vi) drie vragen betreffende het thema competentie, (vii) zes vragen betreffende het thema organisatie van de zorg, (viii) drie vragen betreffende het thema continuïteit van de zorg, (ix) twee vragen betreffende het thema effectieve en veilige zorg, (x) zes vragen betreffende het thema kosten en vergoedingen, (xi) twee vragen betreffende algemeen oordeel van teleconsultatie diensten en (xii) elf vragen betreffende de demografie van de patiënten.
Discussie:
De validiteit en de generaliseerbaarheid van de opgenomen artikelen in de systematische literatuurstudie zijn laag, omdat de artikelen een lage steekproefomvang hadden, vaak het aantal respons niet vermeld was en vaak geen achtergrond informatie over hun deelnemers beschikbaar stelden. Tijdens de literatuur studie zijn er ook geen studies geincludeerd met het type store-and-forward Teleconsultatie diensten. Kwaliteitsaspecten van Teleconsultatie diensten die patiënten belangrijk achten kunnen hierdoor worden gemist. In de focusgroepen is er een potentieel selectiebias opgetreden. Patiënten in de regio van het AMC werden benaderd om vrijwillig deel te nemen. Één van de focusgroepen voldeed niet aan het vereiste aantal deelnemers. Geen van de deelnemers in de focus groepen hebben de Teleconsultatie dienst telepulmonology ervaren.Conclusie:
Het belangrijkste voordeel van de ontwikkelde concept vragenlijst om kwaliteit te meten van de geleverde zorg van teleconsultatie is dat deze is ontwikkeld is vanuit het perspectief van de patiënt. Een ander voordeel is dat vanwege het brede toepassingsgebied van de systematische literatuurstudie en focusgroepen met patiënten kan deze toegepast worden op verschillende vormen van teleconsultatie diensten. De concept vragenlijst kan na validatie (tweede en derde fase van de CQ-index) worden gebruikt om vanuit het perspectief van de patiënt de kwaliteit van de geleverde zorg te beoordelen.Toekomstig onderzoek:
Toekomstig onderzoek is gericht op het testen en ontwikkelen van de validatie en psychometrische eigenschappen van de concept vragenlijst. De tweede (constructie) fase en de derde (psychometrische) fase van de ontwikkeling van de CQ-index moet worden uitgevoerd om de ontwikkeling van de CQ-index van Teleconsultatie dienst vanuit de patiënt af te ronden.Sleutelwoorden:
Patiënten perspectief, CQ-index, Teleconsultatie diensten, Kwaliteit van zorg, Kwaliteit aspectenAbstract
Objective:
The demand of healthcare delivery in the Netherlands is growing. In the past ten years the healthcare expenses grew with 4.4% per year. The expectations are that this growth will continue. Telemedicine acts as a potential source to reduce health care expense, improve access to services, make the right expertise available anywhere, early provide the appropriate treatment and improve quality of care. The healthcare provided through teleconsultation services has proven to be as efficient and effective as face-to-face consultation with high diagnostics accuracy. However, the available research fails to provide satisfactory explanations of the underlying reasons for patient satisfaction (or dissatisfaction) with Telemedicine. Furthermore, the literature shows limitations of patient satisfaction questionnaires concerning Telemedicine services and in the development process of those questionnaires. More specific, these studies lack details on reliability, factor analysis and validity of the questionnaires used to assess patient satisfaction with Telemedicine services. There is therefore a need of a standardized and validated questionnaire with quality aspects of Telemedicine services that patients perceive as important can be measured. The responses on such a questionnaire can presumably be used to enhance the quality of care delivered by Telemedicine. At KSYOS TeleMedical Center there is a need to measure patients’ experiences with the Teleconsultation services they provide. The KSYOS Teleconsultation System provides General Practitioners (GPs) the ability to communicate with other healthcare providers (specialists). This communication is based on store-and-forward modality. In this thesis we aim to develop a standardized concept questionnaire.Methods:
The concept questionnaire was developed according to the Consumer Quality Index (CQ-index) methodology. The CQ-index is a standardized method for developing surveys and measuring healthcare quality from the patient’s perspective. The CQ-index consists of three phases. The first phase is used to develop the concept questionnaire. The following steps were performed: (i) a systematic literature review that aimed to identify quality aspects concerning the delivery of Teleconsultation services that patients perceived as relevant or important, (ii) two focus groups with patients aimed to identify quality aspects concerning the delivery of store-and-forward Teleconsultation services, (iii) the mapping of the results from the systematic literature review and the two focus groups to create a unique list of quality aspects, (iv) the development of questions from the quality aspects and (v) the pre-validation the concept questionnaire with a workgroup consisting of stakeholders from the Teleconsultation services (patients, GPs, specialist, teleconsultation provider). The first phase is conducted in this study and reported in this thesis.Results:
From the systematic literature review 22 quality aspects were revealed. From the focus groups also 22 quality aspects were revealed. Five of the quality aspects of the focus groups matched with the quality aspects of the literature study. Two quality aspects of the literature review were not included into the concept questionnaire. 19 required questions provided by the CQ-index were included. The final concept questionnaire consisted of 56 questions, clustered according to the themes of the CQ-index. The concept questionnaire consist of (i) two introduction questions (ii) three questions concerned the theme access to care, (iii) five questions concerned the theme communication and information, (iv) five questions concerned the theme interpersonal conduct, (v) eight questions concerned the theme patient management role, (vi) three questions concerned the theme competence, (vii) six questions concerned the theme organization of care, (viii) three questions concerned the theme continuity of care, (ix) two questions concerned the theme effectiveand safe care, (x) six questions concerned the theme costs and compensation, (xi) two questions concerned patients general judgment of teleconsultation and (xii) eleven questions concerned the demographics of the patients.
Discussion:
The validity and the generalizability of the included articles in the systematic literature review are low. The included articles had low sample sizes, often no response rate was reported and often no descriptive information about their patient sample was reported. We also did not encounter studies on store-and-forward Teleconsultation services. Quality aspects of Teleconsultation services that patients deem important can be missed. In the focus groups there is a potential selection bias for the participants of the focus groups. Patients in the region of the AMC were contacted to participate voluntarily. One focus group did not meet the requirement of six participants. None of the participants in the focus groups experienced the Teleconsultation services telepulmonology and is therefore not represented.Conclusion:
The main advantage of this concept questionnaire to measure quality aspects of teleconsultation services is that it is developed from the patients’ perspective. Another advantage is that after validation (second and third phase of the CQ-index) the questionnaire can be used to assess patients perspectives on quality of care delivered through any kind of teleconsultation service, because of the broad scope of the systematic literature study and the focus groups conducted with patients familiar with different kind of teleconsultation services.Future research:
Future research is focussed on the validation and psychometric properties of the concept questionnaire. The second (constructive) phase and third (psychometric) phase of the development of the CQ-index needs to be performed to complete the development of the CQ-index of teleconsultation services from the patients’ perspective.Keywords:
Patients’ perspective, CQ-index, Teleconsultation services, Quality of care, Quality aspects1 General introduction
The demand of healthcare delivery in the Netherlands is growing. In the past ten years the healthcare expenses grew with 4.4% per year. The expectations are that this growth will continue. The reasons for the rising costs are ageing of the population, higher welfare, changing society and new medical technology and treatment options. The number of older people is increasing and the elderly live longer, but often with chronic diseases putting a burden on the healthcare system. Higher welfare results in changes in the society, resulting in higher expectations on the healthcare delivery with less discomfort and a need of patients to be well informed about the risks or treatment of their diseases. New treatment techniques, drugs and devices contribute to a better healthcare. They are often cheaper than the older alternatives. The number of treatments increase with more patients being helped and cured (1).
Taking advantage of telecommunication and information technologies, telemedicine acts as a potential source to reduce health care expense, improve access to services, make the right expertise available anywhere, early provide the appropriate treatment and improve quality of care (2). Recent works in communication technologies have inspired the development of telemedicine to a large extent.
One example of Telemedicine, Teleconsultation has been used more often as a new means of health care delivery in the past few years (3,4). The healthcare provided through Teleconsultation services has proven to be as efficient, and effective as face-to-face consultation with high diagnostics accuracy. Teleconsultation services in the field of dermatology (teledermatology) prevented 78% of unnecessary physical referrals, yielding an 18% reduction in conventional costs (5). In the field of pulmonology (telepulmonology), Teleconsultation services prevented 68% of unnecessary physical referrals (6). In the field of dermatology (teledermatoscopy), diagnostic accuracy was determined between teledermatoscopy and face-to-face services. The agreement between the specialist (that answered the teledermatoscopy) and face-to-face was 66% (7).
Brejaart (2008) shows that perceived usefulness, perceived ease of use and compatibility (degree to which an innovation like teleconsultation is consisted with existing values and needs of stakeholders) are factors that significantly influence the acceptance of teledermatology among physicians (8). Telemedicine literature concerning publications about patient satisfaction with Telemedicine shows a generally positive trend. As a consequence there is a tendency to presume that further research into patients’ experiences with Telemedicine is no longer needed. However, the available research fails both to provide satisfactory explanations of the underlying reasons for patient satisfaction or dissatisfaction with Telemedicine and to explore factors that contribute or impede patients’ satisfaction with of telemedicine services in depth (9). Patients may for example have concerns about the technology, their privacy, the impersonal character of the consult, the accuracy of the Teleconsultation services as opposed to face-to face consultations, communication patterns between the involved healthcare providers, and whether they can trust the information provided (10).
Therefore, there is a need of a standardized and validated questionnaire with quality aspects of Teleconsultation services that patients perceive as important can be measured. The responses on such a questionnaire can presumably be used to enhance the quality of care delivered by teleconsultation, give choice information to healthcare consumers, advocacy information for patients and patient organizations to inform their members about the quality of care of telemedicine services,
healthcare purchasing information for healthcare insurers, monitoring and policy information for the government, supervisory information for the healthcare inspectorate and quality information for institutions and professionals in healthcare with respect Telemedicine services (11).
In this thesis we aim to develop a standardized concept questionnaire. In the Netherlands, the Consumer Quality Index (CQ-index), a standardized method for developing surveys and measuring healthcare quality from the patient’s perspective, was introduced in 2006 in order to promote patient-centred care. In order to obtain reliable and valid questionnaires, the development process is structured by guidelines. The content validity of questionnaires is ascertained during a qualitative phase which includes a literature search, patient focus groups and interviews with stakeholders. After this phase a pilot study on the CQ-index questionnaire should be performed to determine internal consistency. The CQ-index is characterized by combining patients’ experiences with the relative importance of each experience item resulting in a list of priorities for improvement of quality of care. The questionnaire for measuring quality aspects of Telemedicine services from the patients’ perspective will be developed according to the CQ-index guidelines.
Chapter 2 describes the literature review to gain insight in quality aspects perceived as relevant by patients. The following research question is answered in chapter 2, the literature study:
Which quality aspects concerning the delivery of Teleconsultation services are reported in scientific literature that patients’ perceive as relevant and important?
Chapter 3 describes (i) the elicitation of patients’ experiences and perceptions of quality aspects for Teleconsultation during focus groups, (ii) the creation of a complete list of quality aspects of Teleconsultation services using the mapping method to combine the quality aspects found in the literature review and focus groups, (iii) the development of questions for the concept questionnaire derived from the quality aspects of the complete list, (iv) the pre-validation of the concept questionnaire by stakeholders/workgroup (consisting of patients, GPs and a specialist), (v) the development of the final concept questionnaire. The following research questions are answered in chapter 3, describing the method of the development of the standardized concept questionnaire:
What are the positive and negative experiences of patients who experienced Teleconsultation services?
Which quality aspects of Teleconsultation services, based on these patients’ experiences, should be incorporated into the validated and standardized questionnaire to assess patient experiences regarding Teleconsultation services?
Are the questions from the concept questionnaire perceived relevant and clear by the stakeholders/workgroup (patients, GPs, specialist and Teleconsultation providers) and does the workgroup find the concept questionnaire complete or is it missing quality aspects?
Bibliography
1. Ministerie van Volksgezondheid,, Welzijn en Sport. De zorg: hoeveel extra is het ons waard? 2012. 2. Hjelm NM. Benefits and drawbacks of telemedicine. J Telemed Telecare. 2005;11(2):60–70. 3. Deshpande, MD MBA A, Khoja, MD, PhD S, McKibbon, BSc MLS PhD A, Jadad, MD DPhil FRCP(C) AR. Real-Time (Synchronous) Telehealth in Primary Care: Systematic Review of Systematic Reviews. 2008 Jan [cited 2015 Jan 13]; Available from: http://www.ncbi.nlm.nih.gov/books/NBK168923/ 4. Hersh, MD WR, Hickam, MD, MPH DH, Severance, MPH SM, Dana, MLS TL, Kragers, AMLS, MA KP, Helfand, MD, MS, EPC Director M. Telemedicine for the Medicare Population: Update [Internet]. 2006 [cited 2014 Jul 15]. Available from:
http://archive.ahrq.gov/downloads/pub/evidence/pdf/telemedup/telemedup.pdf
5. Van der Heijden JP, de Keizer NF, Bos JD, Spuls PI, Witkamp L. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br J Dermatol. 2011 Nov;165(5):1058–65.
6. Thijssing L, van der Heijden JP, Chavannes NH, Melissant CF, Jaspers MWM, Witkamp L. Telepulmonology: Effect on quality and efficiency of care. Respir Med. 2014 Feb;108(2):314–8. 7. Van der Heijden JP, Thijssing L, Witkamp L, Spuls PI, de Keizer NF. Accuracy and reliability of teledermatoscopy with images taken by general practitioners during everyday practice. J Telemed Telecare. 2013 Sep;19(6):320–5.
8. Brejaart J. The adoption of teledermatology; Factors affecting the adoption by physicians. 2008. 9. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. Bmj. 2000;320(7248):1517–20.
10. Zilliacus EM, Meiser B, Lobb EA, Kirk J, Warwick L, Tucker K. Women’s Experience of Telehealth Cancer Genetic Counseling. J Genet Couns. 2010 Apr 22;19(5):463–72.
2 Literature review: Quality aspects of teleconsultation from the
patients’ perspective
2.1 Introduction
In recent years, patient satisfaction gained increasing attention in assessing the quality of healthcare delivery (1). Patient satisfaction refers to a balanced evaluation of patients’ experiences of the health care they received, in relation to their personal preferences or expectations (2), regarding multiple indicators concerning the structure, process, and outcome of care (3). Structure of care concerns all factors that affect the context in which care is delivered (e.g., number of doctors per patient, presence of equipment), whereas process of care indicators refer to all the activities performed to deliver health care (e.g., average patient waiting times, percentage of patients compliant with treatment). Outcome of care indicators measure the impact of health care on patients’ wellbeing (e.g., health status, quality of life) (4). The extend to which patients are satisfied with certain health care delivery modes can influence patients’ willingness to play an active role in their own care (4,5), can increase their desire or willingness to continue using certain health care services (5–7) and can lead to higher treatment compliance (5,7). Furthermore a higher patient satisfaction with quality of care has a positive influence on patients’ physical and mental well-being (7).
A rather new mode of healthcare delivery is Telemedicine. Teleconsultation services is a specific type of Telemedicine and can be defined as the consultation of a healthcare professional by another healthcare professional using information technology to bridge the spatial distance between the participants (8). The information that is communicated between these professionals can either be synchronously transmitted (i.e. interaction is performed real-time such as video conferencing) or asynchronously (i.e. pre-recorded or ‘store-and-forward’) (9). Teleconsultation helps eliminate distance barriers, can improve access to medical services that would often not be consistently available and decrease waiting time for appointments (10). Patient satisfaction questionnaires are often used to assess healthcare quality and enhance quality of care (2). However, available research fails to provide explanations and reasons for patient satisfaction or dissatisfaction with Teleconsultation services (11). Therefore there is no insight in whether the quality of the healthcare service is high or whether the quality of the healthcare service should be enhanced.
In order to improve patient satisfaction with this new mode of health care delivery, it is essential to gain insight in patients’ experiences and expectations regarding the structure, process, and outcome of health care services delivered through Teleconsultation services. Besides showing what quality aspects of health care services provided through Teleconsultation services patients perceive as important, insight in those aspects may guide health care professionals and policy makers in their focus on improving aspects of the care they provide based on patients’ quality perceptions (2). This is specifically relevant for the indicators concerning the structure and process of Teleconsultation services, as these tend to be more actionable than health care outcomes, and because outcomes of care can be seen as the product of structures and processes (4).
Although there are already questionnaires available that aim to measure patient satisfaction with general healthcare services and Telemedicine service, Crow et al. (2002) show limitations in satisfaction studies in general healthcare services (12). Williams et al. (2001) and Mair et al. (2000) show limitations in satisfaction studies in Telemedicine services applying these questionnaires (10,11). The studies included in these literature reviews applied questionnaires that were brief,
quantitative, lacked validation, lacked standardization and did not report on non-responses. Further, these studies did not provide sufficient details to determine the type of questions asked to patients (11,13). Additionally, these studies did not report whether patients valued the questions posed to assess their satisfaction with the delivery of health care services as important or relevant.
Therefore, there is a need of a standardized and validated questionnaire with which quality aspects of Teleconsultation services that patients perceive as important can be measured. Before we can evaluate what specific questions concerning quality aspects of Teleconsultation services should be incorporated into this standardized and validated questionnaire, insight is needed into which quality aspects of Teleconsultation services patients find important and relevant. In this chapter, we aim to answer the following research question:
Which quality aspects concerning the delivery of Teleconsultation services are reported in scientific literature that patients’ perceive as relevant and important?
To answer this research question we conducted a systematic literature review.
2.2 Method
2.2.1 Search strategy
PubMed and PsycINFO were searched for relevant studies. The search terms we used were related to “Telemedicine”, “Patient experience”, “Patient satisfaction” and “Quality”. The literature searches were restricted to English language articles. Figure 1 shows the keywords used and Figure 2 shows the search strategy used to identify relevant articles.
Searches were constructed using three groups. Group A consisted of terms related to telemedicine, group B consisted of terms related to patients’ perspective and group C consisted of terms related to quality of healthcare. Searches in PubMed were constructed using all groups. Searches in PsycINFO were constructed using group A and from B only the keywords patient satisfaction and patient experience. The keyterms within a group were searched with “ OR” and between the groups “AND” was used. A librarian was consulted to help with the search terms and strategy.
2.2.2 Study selection
Two researchers (KD, LDP) independently reviewed and assessed titles and abstracts of the resulting papers against predefined inclusion criteria. Articles that were included focused on exploring patients’ satisfaction or patients’ experiences with Teleconsultation services, specifically those who focused on quality of care delivered through Teleconsultation services. Papers that only reported an overall satisfaction rate of patients with teleconsultation services were excluded, since these articles would not provide information on the specific quality aspects of telemedicine/teleconsultation services that patients perceive as important and relevant.
Inter-reliability of the two reviewers was determined by calculating the Cohen’s kappa. The kappa coefficient is a chance-corrected measurement of the agreement between reviews. A Cohen’s kappa of zero means that the agreement between ratings was based entirely on chance and a Cohen’s kappa of one means that there is a perfect agreement between the reviewers. Kappa values can be interpreted in terms k <0 is poor, 0 to 0,20 is slight, 0,21 to 0,4 is fair, 0,41 to 0,60 is moderate, 0,61 to 0,8 is substantial, and above is 0,81 almost perfect. (14). For the calculation of the Cohen’s kappa SPSSv20 was used.
After a first exclusion of studies based on the title and abstract, a full-text analysis was performed on the remaining articles. An article was included by the two reviewers (KD, LT) if: (i) the focus of the study was on assessing the Teleconsultation service on quality aspects that patients deem important; (ii) the study provided insights in patients’ perspectives on quality aspects of teleconsultation services.
The included articles were screened on reports on quality aspects perceived as important and relevant by the patients. Data on quality aspects was derived from these articles, if the quality aspect referred to one of the indicators concerning the structure and process of healthcare delivery. Data on quality aspects was not derived from these articles if: (i) predefined, standardized questionnaires which had been constructed with no direct input from the patients were used in the study (e.g. with no possibility for the patients to comment if they find the question important or relevant), and (ii) the study concerned comparison of face-to-face (traditional) healthcare delivery with Teleconsultation services, as these studies gave no insight into quality aspects of Teleconsultation services itself that patients rate as important. The two reviewers (KD, LT) extracted all relevant quality aspects from the selected articles. Any disagreements were mediated by a third person and were resolved by consensus.
Quality aspects derived from the selected articles were clustered according to themes of the Consumer Quality Index (CQ-index) methodology. The CQ-Index is a standardized methodology for measuring, analysing and reporting customer experiences in healthcare. The themes were: (i) access
1. Telemedicine 2. Telehealthcare 3. Telehealth 4. Telediagnostic 5. Remote consultation 6. Teleconsultation 7. Teledermatology 8. Telecardiology 9. Telepulmonology 10. Teleophthalmology 11. teleradiology 12. Telenursing 13. Telepharmacy 14. Telerehabilitation 15. Telepsychiatry 16. Teleneurology 17. Patient acceptance of healthcare 18. Consumer satisfaction 19. Patient perspective 20. Patient satisfaction 21. Patient experience 22. Quality assurance, health care 23. Delivery of health care 24. Quality of healthcare
Figure 1 Groups of keywords and MeSH terms used in the search strategy. MeSH Terms are in bold. Tiab (title or abstract) are in italic
A: Teleconsultation services
to care, (ii) communication and information, (iii) interpersonal conduct, (iv) patient management role, (v) competence, (vi) organization of care, (vii) continuity of care, (viii) effective and safe care and (ix) costs and compensation (15).
The following information was extracted from the articles: author, year of publication, study location, type of telemedicine system, medical field (speciality) in which the Teleconsultation service is used, methodology used and sample size.
2.3 Results
The initial search identified 1474 articles. First 88 articles were removed, because they concerned duplicates. From the remaining 1474 articles, 1349 were excluded based on the title and abstract screening. Reasons for exclusion of these articles were: (i) the abstract was not available (n=126), (ii) article was not written in English (n=48), (iii) patient satisfaction nor patient experiences with Teleconsultation services were reported (n=610), (iv) no quality assessment or quality aspects concerning teleconsultation services were reported (n=427), (v) the subject of the articles was not Teleconsultation services (n=95) and (vi) if a patients’ satisfaction rate was reported, but no specifications on which quality aspects of Teleconsultation services patients were satisfied on (n=42). The Cohen’s kappa of the first study selection based on title and abstract was moderate (Kappa= 0,508).
For the remaining set of 37 articles, the reasons for exclusion of articles were: (i) the quality aspects concerning Teleconsultation services were not reported by the patients themselves (n=25), (ii) the focus was on tele-monitoring instead of Teleconsultation services (n=1) and (iii) the non-availability of the full text article (n=4). Ultimately seven articles were included in the review. The Cohen’s kappa of the inclusion of the full-text analysis of these seven articles was substantial (Kappa = 0,637). The search flow is shown in figure 2. The included articles were read to extract quality aspects. The inclusion of each quality aspect was based on consensus between the two reviewers.
The studies were performed in four different countries: two in the USA (16,17), three in Australia (18–20), one in Colombia (21) and one in Canada (22). The stakeholders using the Teleconsultation services of these studies were patients and clinical pharmacists (16), patients and nurses (18,22), patients and specialists (dermatologists or surgeons) (21), patients and physicians (17), patients (together with the genetic counsellor) and the specialist genetic clinicians (19). The last study concerned a ‘’willingness to use’’ study of Teleconsultation services by audiology patients and their healthcare professional (20). The main characteristics of the articles are summarized in table 1. Four of the seven studies performed face-to-face interviews using a semi structured questionnaire (16,18,19,22), one study performed the patient interviews over the telephone (21), one study sent the questionnaires on paper to patients (17) and one article did not describe how the survey among patients was performed (20).
Figure 2 Search Flow
Excluded based on:
Abstract not available (n=126)
language not English (n=48) No teleconsultation services (n=95) No patient satisfaction/patient experience (n=610) No quality assessment (n=427) No quality aspects (n=42)
Not original article (n=1)
Excluded based on:
Not from a patients’ perspective (n=25)
Telemonitoring (n=1)
Full articles not available (n=4)
Abstract/title screening for relevance (n=1386) Included articles (n=7) Initial search strategy in: PubMed (n=1351) I Removal of duplicates (n=88) Initial search strategy in: PsycINFO (n=123) I
Full articles retrieved(n=37) Total articles
2.3.1 Quality aspects
A review of the seven articles gave insight in quality aspects of Teleconsultation services that were reported by the patients themselves as being important and relevant. Table 2-7 summarizes the quality aspects derived from these articles clustered in to the themes of the CQ-index. There are themes were no quality aspects were clustered. These themes were: (i) Interpersonal conduct, (ii) continuity of care and (iii) effective and safe care.
Table 1 Descriptive table of included articles Author Year Study
location Telemedicine System Telemedicine specialty Methodology used Sample size Saberi et al (16) 2013 USA Real-time video
teleconsultation
HIV Semi structured interviews
14 Wade et al (18) 2012 Australia Real-time video
teleconsultation
Tuberculosis Semi structured interviews
11 Lopez et al (21) 2011 Colombia Real-time video
teleconsultation Dermatology and surgery Telephone questionnaires 121 Zilliacus et al (19)
2010 Australia Real-time video teleconsultation Cancer Genetic Counselling Semi structured interviews 12 Sevean et al (22)
2008 Canada Real-time video teleconsultation General healthcare Semi structured interviews 10 Dixon et al (17) 2008 USA Real-time video
teleconsultation General healthcare Questionnaires on paper 30 Eikelboom et al (20)
2005 Australia Teleconsultation Ear and hearing Questionnaires 116
Table 2 Access to care
Quality aspects Description Teleconsultation provided
convenient appointment times
bf
Patients may feel that because of video teleconsultation services, they could arrange an appointment at a time that was more convenient to them (17,18).
Patients do not need to travel to see a healthcare provider eg
Because of the use of teleconsultation services, patients could appreciate that they may not need to travel to see the specialist physically (20,22). Teleconsultation provides a
reduction of travel time adf
Patients may feel that because of the teleconsultation service the travel time to the healthcare provider could be reduced (16,17,19).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
Table 3 Communication and information
Quality aspects Description The healthcare provider was
informative d
Patients may feel that the healthcare provider can be informative about the patient’s condition. The healthcare provider could be yet more informative rather than giving emotional support to the patient (19).
All question asked by the patients were addressed properly d
Patients may feel that all the questions they could ask can be addressed properly by their healthcare provider with the use of teleconsultation services (19).
Explanation given by the healthcare provider was clear d
Patients may feel that the explanation they could receive about their condition from the healthcare provider through the teleconsultation service can be clear and understandable (19).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
Table 4 Patient management role
Quality aspects Description The healthcare provider
provided sufficient privacy abe
Patients may feel that the healthcare provider provided more privacy through teleconsultation in comparison with face-to-face visits, because for example there was no chance of encountering acquaintances (16,18,22). Patients are less reliant on
others for transport to the healthcare provider g
Patients may rely on others to transport them to see the healthcare provider. By making use of teleconsultation services the patients may feel they need to rely less on transport of others (20).
Patients felt more at ease discussing issues a
Patients may feel more at ease about discussing issues which they might never bring up in the office, because they are not talking in-person with someone (16).
Teleconsultation diminished the level of trust in the information provided d
Patients may feel that the nature of teleconsultation services can
diminished the level of trust they have in the information provided by the healthcare provider (clinician) (19).
Teleconsultation was not a disruption of patients daily life
e
Patients may feel that teleconsultation services will not be an intrusion into their daily lives. The patients could start the video teleconsultation
whenever they want and when the teleconsultation is finished they could carry on with their previous activities(22).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
Table 5 Competence
Quality aspects Description Healthcare providers or
patients experienced technical difficulties during
teleconsultation acd
Patients could experience technical difficulties during the teleconsultation services. This could for example be difficulties with the camera, video or microphone (16,19,21).
Patients felt like physical exam was not possible through teleconsultation f
Patients may feel that a video exam cannot be thorough enough, compared to an examination performed physical (17).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
Table 6 Organization of care
Quality aspects Description There is a smaller queue at the
GP’s office than to go to the specialist physically g
Patients could experience and appreciate that the time to wait for an appointment to see a specialist can be reduced (20).
Teleconsultation saves time for the patients e
Teleconsultation services could save time for the patients, because the patients do not need to travel to see the specialist physically (22). Patients do not have to wait in
clinical waiting rooms a
By using teleconsultation services the patients’ appreciate that they may not have to wait in the clinic waiting room (16).
There is less time devoted to the appointment of the consult
g
Patients may feel that time devoted to the appointment could be reduced (20).
Convenient for minor problems
abf Teleconsultation services can be viewed as convenient by the participant for handling minor medical problems. Patients can find teleconsultation a
good screening method for the healthcare providers that might alleviate their workload and that gives patients direct contact with the healthcare provider for minor, but significant medical problems (17).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
Table 7 Costs and compensations
Quality aspects Description Patients experienced cost
saving through teleconsultation
adef
Patients could experience cost savings, because of the use of teleconsultation services (16,17,19,22).
Patients have less travel expenses e
Patients appreciated that they had less travel expenses to contact the healthcare provider (22).
Patients have less accommodation costs de
Patients often come from long distances to see their healthcare provider. Therefore some patients arranged accommodation close to the hospital. Patients could experience and appreciate a reduction of accommodation costs (22).
Patients loses less time off work hours eg
Patients could experience and appreciate that they could be less time off of work, because of the teleconsultation services. Lost wages and time were not of a great concern to them (20,22). Patients who did not live nearby the hospital appreciated that long driving to or arranging accommodation nearby the health care center was no longer needed.
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and g=Eikelboom et al(20)
2.4 Discussion
The overall aim of this research project is to develop a concept questionnaire for Teleconsultation services from the patients’ perspective that is standardized and validated. With a systematic review of the literature, we aimed to identify quality aspects concerning the delivery of Teleconsultation services that patients perceived as relevant or important in scientific literature. The literature review revealed 22 quality aspects which were clustered into nine themes of the CQ-index. The theme access to care concerns the accessibility and availability of the health care institution or healthcare provider. The theme communication and information concerns the degree of transparency of information by the healthcare institution or healthcare provider and how it is communicated. The theme interpersonal conduct concerns the aspects that have to do with the way the customer is treated by the institution or healthcare provider. The theme patient management role concerns the management role of the patients, whether the patients are able to make choices about the direction and progress of their healthcare problems. The theme competence concerns how the professional skills of the healthcare provider are experienced. The theme organization of care concerns how well the care process as well as the physical environment is supportive of the need for care of the patients. The theme continuity of care concerns how the cooperation between institutions and healthcare providers is arranged. The theme effective and safe care concerns whether the care or treatment contributed to improving the health or quality of life. The theme cost and compensation concerns whether the patients understand what costs are involved in care and treatment.
Three quality aspects concerned access to care. Three quality aspects concerned the communication and information. Five quality aspects concerned patients’ management role. Two quality aspects concerned competence. Five quality aspects concerned organization of care. Finally, four quality aspects concerned cost and compensation. Our systematic literature study missed quality aspects in the themes interpersonal conduct, continuity of care and effective and safe care.
Yip et al. 2003 developed a questionnaire for assessing patient satisfaction with Telemedicine videoconference services. This questionnaire was developed by a literature review and by consultation of an expert panel. The final questionnaire consisted of fourteen questions (23). According to the nine themes of the CQ-index, one question concerned access to care, two questions
concerned communication and information, two questions concerned interpersonal conduct, one question concerned the patients management role, two questions concerned competence and four questions concerned organization of care. Finally, two questions concerned the general judgment of patients with videoconference services. One theme that our systematic literature study did cover from the CQ-index and was not covered by the questionnaire from Yip et al., was effective and safe care. Yip et al. covered the theme interpersonal conduct whereas our systematic literature study did not cover this theme. Both Yip et al. and our systematic literature review failed to provide quality aspects with the themes continuity of care and effective and safe care.
Yip et al reported on the validity and consistency of the questionnaire, but not on the origin of each question. Yip et al. did not report on how they conducted the literature review. Therefore it is difficult to explain the differences in quality aspects we could derive from the literature and those quality aspects related to the questions that Yip et al included in their questionnaire. Moreover, their expert panel consisted of doctors, nurses and experts in telemedicine that ensured the relevance, clarity and coverage of the questions. The patients as one of the stakeholder group were not involved in the development of this questionnaire.
Our literature review provides insight into which quality aspects concerning the delivery of Teleconsultation services are reported relevant and important from the patients’ perspective. It therefore gives insight in which quality aspects should be covered by a validated and standardized questionnaire aiming at Teleconsultation services from the patients’ perspective. Our literature review provides 22 quality aspects of Teleconsultation services from the patients perspective whereas Yip et al. only used fourteen quality aspects that was developed by an literature review that is not reported in detail and by an expert panel where other stakeholders than the patients were consulted.
However, this literature has its own limitations. Four of the seven studies included in the literature review had a sample size smaller than twenty patients. These patients may not be representative for the overall population and as a consequence, quality aspects of Teleconsultation services that patients deem important can be missed in these studies. Further, two of the seven articles did not provide descriptive information about their patient sample included in their study (17,22). Patients with different backgrounds can have other needs and preferences concerning Teleconsultation services. For example patients that are younger tend to be more comfortable with telemedicine technology (10). Moreover, five of the seven articles did not mention the response rate to the questionnaires sent (16–18,20,22). Whitten et al. shows that results of patient satisfaction questionnaires may be biased due to self-selection. Patients who are less satisfied with the healthcare services are more likely to drop out of the study. Satisfaction studies are limited to those patients who accept the healthcare service and are willing to fill in a satisfaction questionnaire (10). Due to these limitations the generalizability and validity these studies are limited.
Another limitation of the literature review is that we did not encounter studies on store-and-forward Teleconsultation services. Teleconsultation services provide different kinds of modes of interaction. This can be synchronously transmitted Teleconsultation systems (e.g. videoconferencing) or asynchronously services (e.g. store-and-forward). Studies on store-and-forward Teleconsultation services are limited. Also Teleconsultation services can concern different stakeholders, like healthcare providers and their patients, or healthcare providers and clinical specialists
communicating about a patient health status. Six of the seven studies concerned healthcare providers interacting with their patients through Teleconsultation (16–18,20–22), the remaining study concerned clinical specialists communicating with their patients (together with their counsellor) through Teleconsultation services (19). Studies on types of Teleconsultation services concerning communication of healthcare providers with clinical specialists are limited.
Due to these limitations, the generalizability and validity of the results of these studies on Teleconsultation services included in this systematic literature review is low. As a consequence, we may not have gained full insight in the complete set of quality aspects of Teleconsultation services. Hence, more patient satisfaction studies on store-and-forward interaction modes of Teleconsultation services concerning the interaction of healthcare providers and their patients should be performed. A strength of our literature review is that the Cohen’s kappa’s concerning the agreement of reviewers on inclusion of articles based on their title and abstract was moderate and the kappa concerning the inclusion of articles based on full text review was substantial.
Recommendations for future research would be to complete the quality aspects with the themes of the CQ-index where we did not find quality aspects: (i) interpersonal conduct, (ii) continuity of care and (iii) effective and safe care. Also we recommend to development a standardized and validated questionnaire for measuring the quality aspects of Teleconsultation services from the patient’s perspective. The results of this literature review – providing insight in the quality aspects of Teleconsultation services that patients perceive as important - can be used as a stepping stone for the establishment of such a validated and standardized questionnaire.
2.5 Conclusion
This systematic literature review revealed 22 quality aspects concerning the delivery of Teleconsultation services that patients’ perceive as relevant and important. The quality aspects were clustered in nine themes of the CQ-index. Three quality aspects concerned access to care. Three quality aspects concerned the communication and information. Five quality aspects concerned patients’ management role. Two quality aspects concerned competence. Five quality aspects concerned organization of care. Finally, four quality aspects concerned cost and compensation. However, the included articles have a small sample size, misses descriptive tables of the patients, misses response rates, the focus is on video conferencing Teleconsultation services, and the focus is on one type of stakeholders (healthcare providers communicating with patients through Teleconsultation services). Due to these limitations, the generalizability and validity of the results of the studies on Teleconsultation services included in this literature review is low. This may have led to an uncompleted set of quality aspects of all the types of Teleconsultation services. More studies are therefore needed with store-and-forward Teleconsultation services and Teleconsultation services between the healthcare provider and specialist to gain fully insight in all the quality aspects of different types of Teleconsultation services from the patients’ perspective.
Bibliography
1. Bleich S, Özaltin E, Murray C. How does satisfaction with the health-care system relate to patient experience? Bull World Health Organ. 2009 Apr 1;87(4):271–8.
2. Sixma HJ, Kerssens JJ, Campen C van, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expect. 1998;1(2):82–95.
3. Lebow JL. Research assessing consumer satisfaction with mental health treatment: a review of findings. Eval Program Plann. 1983;6(3):211–36.
4. Donabedian A. The quality of care. How can it be assessed? JAMA J Am Med Assoc. 1988 Sep 23;260(12):1743–8.
5. Asadi-Lari M, Tamburini M, Gray D. Patients’ needs, satisfaction, and health related quality of life: towards a comprehensive model. Health Qual Life Outcomes. 2004 Jun 29;2:32.
6. Marquis MS, Davies AR, Ware JE. Patient satisfaction and change in medical care provider: a longitudinal study. Med Care. 1983 Aug;21(8):821–9.
7. Guldvog B. Can patient satisfaction improve health among patients with angina pectoris? Int J Qual Health Care J Int Soc Qual Health Care ISQua. 1999 Jun;11(3):233–40.
8. Asbach P, Nerlich M. A telemedicine guideline for the practice of teleconsultation. Stud Health Technol Inform. 2003;97:1–14.
9. Craig J, Patterson V. Introduction to the practice of telemedicine. J Telemed Telecare. 2005;11(1):3–9.
10. Whitten P, Love B, others. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4):294.
11. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. Bmj. 2000;320(7248):1517–20.
12. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature [Internet]. Core Research; 2002 [cited 2014 Jul 15]. Available from:
http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0009/59895/ExecutiveSummary-hta6320.pdf
13. Williams TL, May CR, Esmail A. Limitations of patient satisfaction studies in telehealthcare: a systematic review of the literature. Telemed J E Health. 2001;7(4):293–316.
14. De Haan RJ. Cohen’s kappa [Internet]. [cited 2014 Sep 23]. Available from: The Cohen’s Kappa determines the chance measure of agreement between reviewers. A Kappa of zero means that the agreement between ratings was based entirely on chance and kappa one is a full agreement. http://os1.amc.nl/wikistatistiek/index.php?title=Cohen%27s_kappa
15. Centrum Klantervaring Zorg. Ontwikkeltraject CQ-index: Informatie voor patiënt/cliëntvertegenwoordigers in de begeleidende werkgroep. 2010.
16. Saberi P, Yuan P, John M, Sheon N, Johnson MO. A Pilot Study to Engage and Counsel HIV-Positive African American Youth Via Telehealth Technology. AIDS Patient Care STDs. 2013 Sep;27(9):529–32. 17. Dixon RF, Stahl JE. Virtual visits in a general medicine practice: A pilot study. Telemed E-Health. 2008;14(6):525–30.
18. Wade VA, Karnon J, Eliott JA, Hiller JE. Home Videophones Improve Direct Observation in Tuberculosis Treatment: A Mixed Methods Evaluation. Neyrolles O, editor. PLoS ONE. 2012 Nov 30;7(11):e50155.
19. Zilliacus EM, Meiser B, Lobb EA, Kirk J, Warwick L, Tucker K. Women’s Experience of Telehealth Cancer Genetic Counseling. J Genet Couns. 2010 Apr 22;19(5):463–72.
20. Eikelboom RH, Atlas MD. Attitude to telemedicine, and willingness to use it, in audiology patients. J Telemed Telecare. 2005 Dec 1;11(8):22–5.
21. Lopez C, Valenzuela JI, Calderon JE, Velasco AF, Fajardo R. A telephone survey of patient
satisfaction with realtime telemedicine in a rural community in Colombia. J Telemed Telecare. 2011 Mar 1;17(2):83–7.
22. Sevean P, Dampier S, Spadoni M, Strickland S, Pilatzke S. Patients and families experiences with video telehealth in rural/remote communities in Northern Canada. J Clin Nurs. 2008
Sep;18(18):2573–9.
23. Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study. J Telemed Telecare. 2003 Feb 1;9(1):46–50.
3 Development of the CQ-index for Teleconsultation services
3.1 Introduction
The use of Teleconsultation services for healthcare delivery has increased in the past few years (1,2). To assess healthcare quality and enhance the quality of care, patients’ satisfaction questionnaires are often used (3). Validated and standardized patient satisfaction questionnaires on traditional healthcare services are not yet applicable to Teleconsultation services, since the regular workflows of the general practitioner in comparison to the workflows of the general practitioner during a teleconsultation with a patient or another healthcare provider differs. It may therefore be relevant to measure which aspects concerning the quality of healthcare delivered through teleconsultation patients deem relevant as opposed to aspects of healthcare quality delivered through traditional healthcare.
Due to the broad definition of telemedicine, patient satisfaction questionnaires of telemedicine services cannot be applied directly to Teleconsultation services. For example the “Telemedicine satisfaction and usefulness questionnaire” is a questionnaire that is used to assess patient satisfaction and usefulness of telemedicine services. The questionnaire consists of various items that are not applicable to teleconsultation e.g. “The telemedicine equipment is easy to use” and “The telemedicine system helps me to better manage my health and medical needs” (4). These two questions refer to patients who use a (monitoring) device at home and are not aimed to assess quality aspects of Teleconsultation services at e.g. the GP’s office.
There are questionnaires that have been developed and used specifically to assess patient satisfaction with Teleconsultation services. However Blozik et al. showed limitations of these questionnaires and in the development process of the questionnaires. More specific, the studies in this systematic review lack details on reliability, factor analysis (the identification on the relevant underlying construct) and validity of the questionnaires used to assess patient satisfaction with Teleconsultation services. Because of these limitations no comparisons between different Teleconsultation services, quality of care and patient satisfaction can be made adequately over time (5).
Also, the measurements of patients’ experiences concerning the quality of care they received tend to generate more objective information than assessments of patients satisfaction about the quality of care. Patient satisfaction refers to a balance between patients’ experiences and preferences or expectations (6). Williams et al. showed that patients’ satisfaction findings should be interpreted cautiously, because it is difficult to extract data of patients’ reflections on the evaluation of the healthcare quality only, which can be influenced by the study settings. For example, patients may feel treated special because they receive both telemedicine as face-to-face care (7). In a study on patients’ perceptions of a Teleconsultation services clinic, 50% of the patients approved to participate in the study if they saw the specialist from time to time and the patients perceived the telemedicine clinic as providing mainly a 'monitoring' function (8). Therefore questionnaires focussing on patient experiences instead of patients’ satisfaction tend to be more adequate to assess quality of care (6).
A validated and standardized questionnaire focussing on patients’ experiences with Teleconsultation services is still lacking. In this chapter we describe to the development a first concept questionnaire
to assess quality aspects regarding Teleconsultation services from the perspective of the patient. The first version of the questionnaire was developed using the Consumer Quality Index (CQ-index) guidelines. The CQ-Index is a standardized methodology for measuring, analysing and reporting customer experiences in healthcare. This chapter describes the first phase of the development process towards a validated and standardized questionnaire. To this end we defined the following research questions:
1. What are the positive and negative experiences of patients who experienced Teleconsultation services?
2. Which quality aspects of Teleconsultation services, based on these patients’ experiences, should be incorporated into the validated and standardized questionnaire to assess patient experiences regarding Teleconsultation services?
3. Are the questions from the concept questionnaire perceived relevant and clear by the stakeholders/workgroup (patients, GPs, specialist and Teleconsultation providers) and does the workgroup find the concept questionnaire complete or is it missing quality aspects?
3.2 Background CQ-index
The CQ-index methodology was developed by the Dutch Center for Consumer Experience in Health Care (Centrum Klantervaring Zorg, CKZ). The CQ-index combines the inventory of patients’ experiences of a specific healthcare dlivery with an assessment of their priority. The primary aim of the CQ-index is to promote patient-centered care. The CQ-index is based on CAHPS1 (Consumer Assessment of Healthcare Providers and Systems) and QUOTE (QUality Of care Through the patient’s Eyes) developed by NIVEL (Nederlands instituut voor onderzoek van de gezondheidszorg)2. The CQ-index has already been developed for several conditions (e.g. chronic dialysis questionnaire(9) and heart failure (10)), specific patient groups (e.g. diabetes(11)), community series and care settings (e.g. emergency department questionnaire(12) and hospital care(13)).
The CQ-index offers a framework to develop valid and reliable questionnaires to measure patients’ experiences. These questionnaires can be applied to gain insight into the patients’ perspective on the evaluated healthcare services.
3.2.1 Development phases CQ/index
The development of a CQ-index consists of three phases. The first phase is the preparation/qualitative phase. The aim of this phase is to elicit the relevant quality aspects of a certain healthcare service by conducting a systematic literature review and conducting patient focus groups. The systematic literature study and the results from the focus groups are used as input to the discussion with the stakeholders. In this study, the stakeholders concerned patients who experienced the teleconsult, general practitioners who perform teleconsultations, specialists who receive and answer the teleconsultations and the provider of the Teleconsultation services. In this chapter we describe the first phase of the CQ-index.
1
https://www.cahps.ahrq.gov/
2
