Teledermoscopy consultations: Data analysis and pre-literature review focused on early skin cancer detectionand prevention

Teledermoscopy consultations

Data analysis and pre-literature review focused on early skin cancer detection

and prevention

Master’s Thesis

Supervisor

Mentor

3

Teledermoscopy consultations

Data analysis and pre-literature review focused on early skin cancer detection

and prevention

Tutor

Prof. Dr. M.W.M. Jaspers

Adjunct head of Medical Informatics department University of Amsterdam

m.w.jaspers@amc.uva.nl SRP address

KSYOS TeleMedical Center Professor J.H. Bavincklaan 2-4 1183 AT, Amstelveen

SRP period

4

Table of content

Chapter 1 – General introduction on skin cancer and teledermoscopy ...10

Chapter 2 – Follow-up of suspected melanoma cases and early development of study protocol ...12

Introduction ... 12

Methods ... 14

Results ... 15

Discussion ... 19

Strengths and weaknesses ...20

Future research ...20

Conclusion ... 21

Chapter 3 – Effect of training on TDsc and diagnostic skills of clinicians: A pre-literature review ...22

Introduction ... 22

Methods ... 23

Results ... 24

Discussion ... 29

Strengths and weaknesses ...30

Future research ...30

Conclusion ... 31

Chapter 4 – Diagnostic trend analysis of teledermoscopy consultations over time: A retrospective data analysis ...32

Introduction ... 32

Methods ... 35

Results ... 36

Discussion ... 44

Strengths and weaknesses ...46

Future research ...47

Conclusion ... 47

Chapter 5 - General discussion ...48

References ...50

5 Appendix A – Study protocol ... 54 Appendix B - Embase search ... 60 Appendix C - Pubmed search ... 61 Appendix D – Distinction between relatively complex and easy to diagnose skin diseases and their ICD-10 code... 62 Appendix E – Syntax of the data analysis ... 63

6

Preface

I have conducted my Scientific Research Project (SRP) at KSYOS TeleMedical Center as part of my master Medical Informatics at the University of Amsterdam. The SRP resulted in this master’s thesis.

Chapter 1 serves as an overall introduction of the teledermoscopy consultation process and provides background information about skin cancer and its detection. Chapter 2 consists of two parts. The first part is a follow-up on previous research which was focussed on

requesting histopathology of patients that were indicated with a malignant form of skin cancer in that previous research. The results of this follow-up led to more questions that needed to be answered with future research for which a first draft of a study protocol is developed.

Chapter 3 is a pre-literature review on the effects and benefits of educating healthcare providers involved in the teledermoscopy process. Chapter 4 of this thesis is a diagnostic trend analysis with retrospective data on teledermoscopy teleconsultations performed with KSYOS TeleDermoscopy applications. Chapter 5 is a general discussion which intertwines the results of the previous chapters together.

Acknowledgements

Writing a master’s thesis and performing the corresponding research does not go without serious struggle and lack of motivation along the way. However, support and guidance from the right people helped me finishing my master’s thesis. Therefore, I would like to express my gratitude to some people and thank them for supporting me during the SRP.

First of all I would like to thank Monique for her guidance and feedback during the project. Words cannot express enough how much I have learned from you, and how you challenged me to keep striving for the best. I would also like to thank you for your support and social chitchat, you helped me become the person I am today.

I would like to thank Esmée for taking the time shaping the project and work plan. It was a pleasure working with you and your feedback was always very welcome. In addition, I would like to thank Leonard for giving me the opportunity to do my internship at KYSOS and

provide me with practical insights about how to tackle and prevent bottlenecks.

Thank you mom and dad for always being there for me and supporting me emotionally and financially, without you guys I would have never been able to pull this off. A special shout out to my sweetest sister who always knows how to make me laugh and support me when times were rough.

My dearest friends, Wies and Nawa, I wouldn’t have survived 5 years of university without you guys. We have been there for each other during the good times and bad times, I will never forget the countless memories we have with us three. I love you two.

8

Abstract

Introduction: Each year in the Netherlands, over 55,000 people receive a diagnosis of skin cancer and 767 die from melanoma, making melanoma the deadliest of all skin cancers [3]. Teledermoscopy consultation applications are a proficient advisory tool for GPs for detecting patients with melanoma and other skin diseases and studies report that GPs experience a learning effect after using TDsc tools for a while. However, concise evidence on the reported GP learning effect and histological information confirming melanoma diagnosis of eleven patients indicated in previous research are not available.

Aim: To assess whether GPs truly experience a learning effect from conducting teledermoscopy consultations throughout the years. In addition, this thesis aimed to develop a first draft of a study protocol, focussed on assessing the false negative rates of melanoma patients missed when using TDsc, for approval of the Medical Ethics Committee.

Methods: Teledermatologists and GPs that were involved in diagnosing a group of 11 patients with a malignant tumour, that would have been sent home if TDsc was unavailable, were approached with the request for additional patient information regarding the final diagnosis after a dermatologist had seen the patient. Further, a first draft for a new prospective cohort study protocol, regarding the false negative rates of TDsc, based on the WHO recommended study format was constructed.

A pre-literature review was conducted aimed at gathering evidence of a learning effect among TDsc users. Two databases (PubMed and Embase) were searched for articles. That search was filtered on training, education, experience and confidence of a GP or dermatologist in their diagnosis. Data about learning effects and educational effects, among GPs as well as dermatologists, were extracted from articles that had a focus on (early) skin cancer detection using TDsc applications. Furthermore, TDsc consultations (from 1 July 2015 till 19 December 2017) conducted by GPs and teledermatologists using KSYOS TeleDermascopy services were analysed. Linear regression trend analysis of the ICD-10 codes entered in the teleconsultations by the GPs and teledermatologists resulted in graphs with linear trend lines plotted over time.

Results: Two of the eleven patients turned out to actually have a malignant melanoma and three patients had a nevus naevocellularis (oncologic). The first draft of study protocol based on the WHO study format should be further developed with information regarding study group inclusion.

Eleven articles were included in the literature review. Most of the articles came more or less to the same conclusion; increasing diagnostic dermatological knowledge and appropriate use of TDsc applications of GPs and dermatologists through educational classes and trainings may lead to more accurate diagnoses and subsequently to better patient care. In addition, 5,770 TDsc consultations were analysed. The results showed that the GPs left less consultations without a diagnosis (p-value of 0.002, -0.374% (95% CI -0.593 – -0.156)) and the number of easy-to-diagnose skin lesions diagnosed by the GPs themselves increased (p-value of 0.001, +0.245% (95% CI 0.111 – 0.378)) over time. However, diagnostic agreement was not statistically significant (p-value of 0.314).

Discussion: The propriety of malignant melanoma tumour diagnosed by the teledermatologists is questionable and the study protocol for future research regarding the false negative rates of

melanoma diagnosed through TDsc should be developed further in order to provide a clear overview of the efficiency of TDsc applications. Literature did not provide much objective evidence on a learning effect among the GPs using teledermoscopy applications. Based on the retrospective trend analysis, this thesis indicated that GPs may experience a learning effect. However, the diagnostic agreement between GPs and teledermatologists did not increase over time.

9

Samenvatting

Introductie: Elk jaar krijgen meer dan 55.000 mensen in Nederland de diagnose huidkanker, waarvan er 767 sterven aan een melanoom. Dit maakt melanoom de dodelijkste vorm van huidkanker [3]. Teledermatoscopische applicaties zijn handige tools voor huisartsen voor het (vroegtijdig) opsporen van melanomen en andere huid aandoeningen. Diverse studies rapporteren dat de huisartsen die teledermatoscopische applicaties een tijd gebruiken een leereffect ervaren. Echter, er is geen sluitend bewijs met betrekking tot dit leereffect. Daarnaast is er geen histologische informatie beschikbaar met betrekking tot de elf melanoma patiënten die zijn aangeduid in vorig onderzoek. Doel: Bewijs uit de literatuur verzamelen en bewijzen of huisartsen daadwerkelijk een leer effect ervaren door het uitvoeren van teledermatoscopie consulten over de jaren heen. Bovendien, is er een eerste versie van een studie protocol ontwikkeld met als doel het berekenen van de vals negatieve waardes die betrekking hebben op patiënten die zijn niet zijn gediagnosticeerd met huidkanker maar het wel hebben.

Methoden: Teledermatologen en huisartsen betrokken bij het diagnosticeren van de elf patiënten met een kwaadaardige tumor, zijn benaderd met het verzoek om patiënt informatie te verstrekken over de uiteindelijke diagnose van de dermatoloog. Bovendien is er een eerste versie van een studie protocol voor een nieuwe prospectieve cohort studie die focust op het berekenen van de vals negatieve waardes van TDsc op basis van het WHO format, ontwikkeld. Om wetenschappelijke bewijs te verzamelen zijn twee databases (PubMed en Embase) doorzocht met behulp van een uitgebreide zoekopdracht. De zoekresultaten zijn gefilterd op training, educatie, ervaring, en

vertrouwen van een TDsc gebruiker in zijn of haar diagnose. Data met betrekking tot leereffecten bij huisartsen en dermatologen werd verkregen uit de artikelen gericht op (vroegtijdig) huidkanker detectie met behulp van TDsc. Daarnaast werden TDsc consulten van huisartsen en dermatologen die uitgevoerd zijn met KSYSOS TeleDermatoscopie van 1 juli 2015 tot 19 december 2017

geanalyseerd. Trendanalyse met behulp van lineaire regressie analyse van de ICD-10 codes resulteerde in grafieken met lineaire trends eroverheen geplot.

Resultaten: Uit de patiëntinformatie bleek dat van de elf patiënten er twee een kwaadaardig melanoom hadden en drie een oncologische nevus naevocellularis. De eerste opzet van het studieprotocol voor een nieuwe prospectieve studie moet nog verder ontwikkeld worden. Elf artikelen zijn geïncludeerd in het literatuur onderzoek. De meeste artikelen kwamen min of meer tot dezelfde conclusie; het verhogen van de dermatologische kennis en juist gebruik van

teledermatoscopische toepassingen bij huisartsen en dermatologen door middel van informatieve lessen en trainingen, leidt tot een accuratere diagnose en daaruit volgend wellicht betere

gezondheidszorg. Bovendien zijn er 5,770 teledermatoscopische consulten geanalyseerd. Uit de resultaten bleek dat huisartsen minder teleconsulten zonder diagnose insturen (p-waarde van 0.002, -0.374% (95% CI -0.593 – -0.156)) en dat het aantal patiënten met een makkelijk-te-diagnosticeren huidaandoening gediagnosticeerd door de huisarts zelf gaande weg stijgt (p-waarde van 0.001, +0.245% (95% CI 0.111 – 0.378)). De diagnostische overeenkomst tussen GPs en teledermatologen bleek niet statisch significant (p-waarde van 0.314).

Discussie: De diagnostische juistheid van een kwaadaardige melanoom door de teledermatoloog is betwistbaar. In de literatuur is beperkt objectief wetenschappelijk bewijs gevonden met betrekking tot een leereffect bij huisartsen die teledermatocopie toepassingen gebruiken in hun praktijk. Deze retrospectieve trendanalyse heeft aangetoond dat huisartsen die TDsc toepassingen gebruiken een leereffect kunnen ervaren. Hoewel de diagnostische overeenkomst tussen huisarts en

teledermatoloog niet steeg.

10

Chapter 1 – General introduction on skin cancer and teledermoscopy

and encompasses both surgical and medical aspects [1]. The most common diseases in dermatology are skin cancer, warts, fungal infections, dermatitis, psoriasis and acne [2]. Skin cancer is the most common disease and there are several types of skin cancer: basal cell carcinoma, squamous cell carcinoma, melanoma and some rare forms of skin cancer. Treatment differs for each type [3].

Each year over 55.000 people in The Netherlands get the diagnosis skin cancer and 767 people in the Netherlands die from a melanoma, this makes melanoma the deadliest of all skin cancers [3, 4]. Those high mortality rates are a result of the ability of spreading of melanoma which is directly related to the survival rate [5]. Over 90% of very early stage (localized, Stage 0 or I) melanoma for example is curable with treatment, while patients with melanoma that has progressed to stage 4 have a median life expectancy of less than one year. The substantial increase in survival with early detection is therefore undeniable [5]. There is a fairly lengthy period between occurrence and spreading of the melanoma which allows time for screening, early detection and treatment before the melanoma has spread to other body parts [6].

eHealth and telemedicine services have emerged in the dermatology branch/domain. Those services are offered by KSYOS TeleMedical Center, which is a Dutch virtual hospital offering several other telemedicine and eHealth services (TDsc) [7]. TDsc is a sub specialism of teledermatology and is a combination of teledermatology and dermotoscopy [8]. TDsc a non-invasive intervention for detecting skin malignancies. TDsc provides teledermoscopy services through a secured internet connection and is particularly suitable for the judgement of solitary skin lesions and early detection of malignant skin lesions such as melanoma. In the Dutch health care system, the General Practitioner (GP) is the gatekeeper of all non-emergency treatments, which means that patients can only access secondary care, such as the specialism dermatology, when referred by a GP. Costs can be reduced when a GP only refers patients to a dermatologist when needed. The study of Tensen et al. study states that 70 percent of Dutch GPs indicated that they want to contribute to skin cancer care [9]. However the GPs lack the knowledge to do so [10]. Therefore, it is important that the GP possesses sufficient knowledge about skin diseases to prevent unnecessary referrals to a dermatologist [11].

The teledermoscopy tool is used by the GP. The GP takes an overview picture, a detailed picture and a dermoscopic picture of the suspicious lesion and uploads it to the

teleconsultation record. In addition, the GP fills in basic patient characteristics (e.g. medical history) and their own diagnosis with corresponding International Classification of Diseases (ICD-10) code. However, the GP is not obligated to fill in a diagnosis and ICD-10 code.

The patient comes to the GP who decides to start a TDsc, see figure 1 below. When the GP starts the TDsc, the system asks the GP whether he or she would have referred the patient if TDsc was not available. The GP takes pictures of a patient’s skin and fills in basic patient information and relevant medical history (e.g. skin cancer frequency in the family), and the

11 consult is then sent to the specialist. The teledermatologist, examines the skin images from the teleconsultation and determines whether a physical visit to the dermatologist is necessary. The teledermatologists is obligated to classify the diagnosis with an ICD-10 code. The teleconsultation is then sent back to the GP who discusses the results of the TDsc with the patient and recommends a treatment plan. The GP then ascertains whether to refer the patient to a dermatologist. Thereafter, the GP answers two more questions: question 4 asks the GP if he or she and the patient are helped with the teledermatologist’s answer, while question 5 asks the GP if he or she has learned from the teleconsultation. The TDsc is then closed. All questions are directed to the GP, except for question 2 that is intended for the teledermatologist. Notably, question 2 is the only question, which cannot be skipped.

The GP starts the TDsc when he/she deems it necessary, 82.4% of the dermatologists answer the TDsc within one working day [7]. TDsc has several useful advantages. First of all,

unnecessary referral from the GP to the dermatologist can be prevented, invoked by diagnostic uncertainties. Since in most cases the photo’s that the GP takes may provide the teledermatologist with enough information about the severity of the patients complaint, referral of non-serious illnesses to the dermatologist can be prevented. Those cases can be treated by the GP or the patient can be sent home if the patient does not have a serious skin diseases. This prevents an unnecessary referral and its corresponding costs. Secondly, TDsc provides the patients with relatively fast diagnosis (average 8,7 hour). Lastly, GP’s seem to learn from the TDsc regarding faster and better recognition of skin melanoma's and other skin diseases [7]. A study of Ford et al. reported this as the greatest benefit of TDsc. The clinicians using teledermatology services stated that the service improved their knowledge and confidence in diagnosing skin lesions and other skin diseases over time [12]. The GP can use the service in order to get a second diagnosis from a teledermatologist for their own diagnostic uncertainties.

12

Chapter 2 – Follow-up of suspected melanoma cases and early

development of study protocol

Introduction

The first aim of this study is to confirm the histopathology of the eleven patients that were indicated with a malignant melanoma according to the ICD-10 code filled in by the

teledermatologist, in previous research. The second aim of this study is to develop a first draft study protocol to prepare a new prospective cohort study focussed on calculating false negative rates of melanoma diagnosed with TDsc by GPs using TDsc applications by KSYOS. The study will focus on the group of patients where the GP states that they would have referred the patients without TDsc but do not refer these patients after use of TDsc. This study is divided into two parts

Part 1 – Follow-up

Van Sinderen et al. performed a data analysis on available TDsc data from the KSYOS database. This data analysis was updated and resulted in 8,713 teledermoscopy consults deducted from the KSYOS database, from July 2015 till December 2017. An clear overview of the analysis is shown in figure 2 below. The skin images of the 216 cases in the green box at the bottom (follow the red arrows) were reviewed by a teledermatologist. According to that review 11 patients had a malignant tumour. Those 11 people would not have been

(immediately) referred to the dermatologist without the use of teledermoscopy services. The 11 patients can be found at the bottom of the tree by answering “Yes” to Q1: would you refer this patient to a dermatologist if TDsc was not available and “No” to Q3: do you refer this patient to the dermatologist, were not seen by a dermatologist.

13 However, the only indication for a melanoma is the ICD-10 code C43.9 “Malignant

melanoma’’ that the dermatologist had entered in the KSYOS system, which was not histopathologically confirmed.

The clinical diagnosis of a dermatologist might be different from the diagnosis entered by the teledermatologist, which is based on skin images only. This would result in a false positive rate, that indicates that a given condition is present when it is not [13]. The

histopathological confirmation of the 11 patients with a malignant tumour were outside the scope of the study of van Sinderen et al. However, it is important that a follow-up is

performed since the definite diagnosis of those 11 patients might differ from the ICD-10 code entered by the teledermatologist.

Part 2 – Study protocol

To provide a complete overview of the results of the updated data analysis it is important to not only provide false positive rates but also false negative rates, results that appear

negative when it should not [14]. In context of the data analysis, this entails that the group where Q1 (Would you refer this patient to a dermatologist if TDsc was not available) was answered with ‘Yes’ and Q3 (Do you refer this patient to the dermatologist?) with ‘No’ would be examined further in real life by a dermatologist.

This study should grant further insight into how many patients in that group might have a malignant melanoma but were missed by the teledermatologist. The false negative rates can then be compared to the false positive rates. To examine the false negative rate of the 1,706 cases a study protocol is required. The importance of a study protocol lies in enhancing the integrity of the research, in prescribing and initiating the documentation process, in

increasing work efficiency, and such a protocol serves as a communication link [15].

To prepare and initially start up the study regarding false negative rates of teledermoscopy consultations labelled as melanoma discussed above, a first draft of a study protocol for a prospective cohort study among GPs using TDsc by KSYOS was developed. The format for international study protocols of the World Health Organization (WHO) was followed [16]. The study protocol focusses on the group of patients where the GP states that the patient is not referred after TDsc. Participants of the study will be subjected for further examination by a dermatologist in real life in order to assess the false negative rate concerning

melanoma, in situ (C43.9) of that group.

A study protocol for the prospective research should comply to all regulations regarding data collection and analysis. The protocol was reviewed by the Clinical Research Unit (CRU). The Clinical Research Unit was established in 2006 and supports the clinical research in the Academic Medical Center (AMC). The CRU aims to improve the quality of clinical research and ultimately patient care. The CRU operates according to the Guidelines for Good Clinical Practice (GCP) and in compliance with current regulatory requirement [17]. Approval of the study protocol by the Medical Ethics Committee will be the next step.

14

Methods

Part 1 – Follow-up

First of all the data analysis from previous research of van Sinderen et al. had was updated to include the most recent data for this study. The period over which the data has been extracted was from 01-07-2015 till 19-12-2017. The data source is the KSYOS database. The export of the database was cleaned; all test teleconsultations (consultations with fake patients) were deleted, all unfinished teleconsultations (consultations that were still up for review) were deleted and only completed teleconsultations were included in the dataset. A teleconsultation was considered complete when the first and third evaluation question of a TDsc was answered. The first evaluation question was “Would you refer this patient if TDsc was not available?” and had to be answered with ‘No’. The third evaluation question was “Do you refer this patient to the dermatologist?” and had to be answered with ‘Yes’. The remaining dataset was filtered on the ICD-10 code ‘C43.9’ (malignant melanoma of skin, unspecified) entered by the teledermatologist.

To confirm those supposedly malignant melanoma cases, the GPs who treated the patients were contacted for patient data from their electronic patient record. The teledermatologists that judged the images of the TDsc were approached through email by Prof. Dr. L. Witkamp to ask for permission to send them letters, which contained the request for additional clinical information from GPs. All five teledermatologists gave permission to send them letters.

Subsequently, two letters were sent to the involved teledermatologists The first letter referred to the previous contact through email, and again explained what the research purpose was and why there is a need for additional clinical information of patients from their GP.

The second letter the teledermatologist received, was a letter addressed to the GPs who treated the patients; the teledermatologist had to sign the request of patient information at the bottom of the letter and then had to send the letter to the GP. This was done because of contractual obligations of the teledermatologists and GPs whom use TDsc applications from KSYOS.

Only GPs who treated patients where the teledermatologist had filled in a diagnosis malignant melanoma of skin, unspecified (ICD-10 code C43.9) in the teleconsultation were included. A self-addressed envelope to KSYOS headquarter was attached. The GP was requested to send the additional clinical patient information, that was requested by the teledermatologist, to KSYOS, using the retour envelope. GPs that did not respond after three months were approached again in the manner described above.

Part 2 – Study protocol

The first draft of the study protocol followed the criteria and regulations which the World Health Organization (WHO) strongly suggests to comply to in their recommended format for a research protocol worldwide [16]. The activity diagram that we developed, aimed at mapping the complete teledermoscopy process and serving as a window of understanding for the study protocol, has been made online with www.draw.io. In addition, the activity

15 diagram was used to develop images to support and visualize the data analysis which

underlies the study protocol.

Results

Part 1 – Follow-up

After two months, eight requests for additional clinical information were fulfilled. Two cases remained unanswered after two months, after sending the letters to the teledermatologists. The teledermatologists and GPs regarding those cases were approached again. The GPs fulfilled the requests for additional clinical information within one month after the second approach with a self-addressed envelope.

One teledermatologist that diagnosed one teleconsultation with a melanoma was no longer working in the medical field, resulting in one teleconsultation without additional clinical information.

Table 1 shows the final diagnoses after a body examination by a dermatologist was performed. Those consultations were performed by another dermatologist than the

teledermatologist. Two malignant melanoma diagnosis were confirmed by histological data and three malignant melanoma diagnosis turned out to be of a oncological form of nevus naevocellularis. These aforementioned cases are highlighted in red in table 1.

Table 1. – Results from the additional clinical information. The final diagnosis of the patient after examination of the dermatologist can be seen on the left side of the table. The frequency of those diagnosis’s is reflected in the right column of the table. Cases of significance are highlighted in red.

Final diagnosis described in patient record Frequency Verruca seborrheic 1

Nevus naevocellularis (oncologic) 3 Junctional nevus cellularis 1

Hemangioma 1 Blue nevus 1 Laesion disappeared 1 Malignant melanoma of other part of trunk 1 Malignant melanoma of right lower limb, including hip 1

The GP diagnosed four patients with a melanocytic nevi, unspecified (ICD-10 code D22.9) and two patients with illness, unspecified (R69.0). Table 2 below shows the final diagnoses of those patients. It can be seen that the GP would have diagnosed patients with a melanocytic nevi while one patient had malignant melanoma and another patient had a oncologic nevus naevocellularis.

16 Table 2. – Comparison of the diagnosis and ICD-10 code entered by the GP to the final diagnosis. The ICD-10 code and the corresponding diagnosis that the GP entered in the teleconsultation before physical examination of a dermatologist is shown in the left column. The final diagnosis of the patient after examination of the dermatologist is reflected in the right column.

ICD-10 code entered by GP in teleconsultation Final diagnosis after additional examination

4 (D22.9 melanocytic nevi, unspecified) Nevus naevocellularis (oncologic)

Laesion disappeared Blue nevus

Malignant melanoma of other part of trunk 2 R69.0 (Illness, unspecified) Nevus naevocellularis (oncologic)

17

Figure 3.- Activity diagram of diagnostic skin diseases process. This activity diagram illustrates all activity from all stakeholders involved in the diagnostic skin process. This involves the workflow of the use of TDsc by the GP and all decision (numbered) during this process. Red boxes cannot be skipped and have to be completed before continuing. Yellow boxes are sticky notes regarding the textbox it is connected to.

18 Part 2 – Study protocol

Figure 3 on the previous page shows the activity diagram of the TDsc process. The diagram displays four activity lanes. Each activity lane, patient, GP, teledermatologist and the dermatologist, represents an actor in the TDsc process. All textboxes that are shaped in a diamond style represent decision points of the diagnostic process.

The decision nodes of the TDsc process are elaborated in greater detail in table 3 below. The evaluation questions that are asked throughout the TDsc, to the GP and teledermatologist, are represented in figure 3 and table 3. Table 3 is an addition to figure 3 to clarify the decision nodes represented by the numbers in figure 3 to make the figure more comprehendible for laypeople.

Table 3. – Decision nodes activity diagram. The numbers in the table on the left side relate to the numbers in figure 3 on the previous page. The decision and the decision maker of each decision node can be seen in the table below.

Number Decision node Decision maker

1 Start TDsc GP

2 Fill in an ICD-10 code GP

3 Answer Q1: Would you refer this patient if TDsc was not available? GP

4 Fill in an ICD-10 code Teledermatologist

5 Answer Q2: Is a visit to the specialist neccesarry? Teledermatologist 6 Answer Q3: Do you refer this patient to the dermatologist? GP

7 Answer Q4: Are you and your patient helped with the answer of the

specialist? GP

8 Answer Q5: Have you learned from this teleconsult? GP 9 Ask for supplementary advice from teledermatologist GP

10 Refer patient GP

11 Go to the dermatologist Patient

This activity diagram was used to construct the flowchart which was subsequently used for construction of the study protocol. An alternative version of figure 2 is given to give a clear overview of the study group of interest discussed in the study protocol which can be found in Appendix A.

19

Discussion

Part 1 – Follow-up

The first aim of this part of the study was to seek confirmation of the diagnosis of patients possibly suffering from melanoma as indicated by the ICD-10 code entered by

teledermatologists, which would have been missed if TDsc was not available for the GP by histopathological information.

The results showed that most of the patients indicated with a malignant melanoma had another skin disease than a malignant melanoma. This might implicate that the diagnosis of the teledermatologist is not conclusive. Reasons for this might be the that the correctness of the diagnosis is dependent on the image quality, user experience and overall difficulty of diagnosing melanoma properly [18,19].

Studies showed that user experience is a major influence on the ability of the

teledermatologist to diagnose melanoma properly. Teledermatologists with few experience misdiagnose patients more often compared to teledermatologists with several years of experience [20]. In addition, a teledermatologist cannot perform a full body examination of the patient by which other melanoma or another serious skin disease on the skin might be missed and not diagnosed in a timely manner since the teleconsultation only contains images from a specific lesion of interest.

The addition of dermoscopic images to teledermatology enhances the sensitivity and specificity of the diagnosis of those teleconsultations [21]. Quality of skin images greatly influences the ability to properly diagnose a skin disease. However, studies state that the quality of the skin images can improve over time with appropriate training of users in using up-to-date soft and hardware, and therefore lead to more precise diagnosis [22]. Patients whom are clearly instructed on how to take proper photos of skin lesions might even start a TDsc for themselves in the future [23]. Those developments might contribute to less

unnecessary patients being referred to the dermatologists office and increases the value of TDsc to healthcare system.

At last, malignant melanoma is from itself, already a difficult skin lesion to diagnose. The symptoms of melanoma mimic the symptoms of several other skin diseases which makes it hard to ascertain a malignant melanoma diagnosis based on skin images from a

teleconsultation [24]. Part 2 - Study protocol

The aim of this part of the study was to prepare and develop a first draft for a new

prospective cohort study that focusses on determining false negative rates of the group of patients that initially would be referred to the dermatologist but ultimately were sent home after evaluation of skin images by the teledermatologist. The patient cases concerning the false negative rates of TDsc are patients that might have a melanoma (or another serious skin disease) but the teledermatologists may have missed or misdiagnosed the melanoma based on the skin images of the teleconsultation. Therefore, the teledermatologist

recommends sending the patient home since he or she did not see anything alarming on the skin images.

20 The study protocol for the new study is needed to get approval from the Medical Ethics Committee to start the research and to have a clear overview of all the facets of the study. The study protocol (Appendix A) constructed in this study is a first draft for a future

prospective cohort study. It should be revised and updated with more recent data and a more elaborated overview of the inclusion and exclusion of the study groups and

interventions.

To make the activity diagram fit for the definite study protocol and study selection several changes have to be made. First, It should be defined which patients to include in which study group. The diagram should have a concise classification as to in which study group the

patients from each branch of figure 2 end up, this means that the study group classification should be clear at the bottom of the figure. Second, the protocol still lacks information about the randomization process and the patient referral process. The study protocol could not be further developed in the timeframe of this study. The prospective cohort study will start once the further developed study protocol has been approved by the Medical Research Ethics Committee.

Strengths and weaknesses

A strength of the study is the completeness of the information provided by the GPs involved in this study. The GPs fully cooperated and provided information about the patients follow-up within the timeframe of this study. Therefore, a detailed review of the actual diagnosis could be performed, encompassing all the clinical considerations and final outcomes. This gives insight in the correctness of the diagnosis determined by the teledermatologist based on the skin images of the teleconsultation and allows for a more detailed analysis of the diagnostic TDsc process in the future. Another strength of this study is that the study protocol is based on the recommended format of a study protocol by the WHO, which is adhered to worldwide.

The weakness of this study is its limited timeframe which prevented further development of the first draft of the study protocol. The work has been given to other researchers who took the development of the study protocol to the next level, and they will oversee the execution of the study. Another minor weakness of this study is, that the histopathological information of one patient is not provided since the teledermatologist involved retired.

Future research

Future research focussing on calculating the false negative rate, of melanoma diagnosed through TDsc performed with KSYOS applications, is most urgent in order to draw a concise conclusion regarding the false negative rate. The first draft of the study protocol aims to provide input to the final study protocol that needs approval of the Medical Ethics Committee of the AMC before the prospective study can start.

Studies evaluating the false positive and false negative rates of the diagnoses of TDsc

applications with built-in computer-aided diagnosis options could also be interesting. Studies showed that some clinicians expressed computer-aided diagnosis as one of the reasons to use TDsc [25]. Perhaps, KSYOS can build a diagnostic advisory tool trained with their own database consisting of thousands of teleconsultations where the patient outcomes is already available and therefore suitable for training such tool.

21 Furthermore, research is needed to identify the bottlenecks of TDsc use and asses what hinders the uptake of dermoscopy applications in various other healthcare settings and where bottlenecks occur. The study of Breton et al. for instance takes notice a correlation between the age of clinicians and the use teledermoscopy applications but does not elaborate much further on it [26]. KSYOS could analyse the use of their teledermatology applications and correlate it to the age of the users. Perhaps, an age related trend can be seen and action can be taken to stimulate the use of their applications among possibly older physicians.

Conclusion

This study aimed to serve as a foundation for future research focussed on assessing the false negative (results that appear negative but should be positive) and false positive (results that appear positive but should be negative) rates of melanoma diagnosed when using

teledermoscopy consultation service from KSYOS. The histopathological information that was requested from GPs who saw patients with a supposed malignant melanoma showed that most of these suspect malignant melanoma were actually other skin lesions or diseases. This might implicate that the diagnostic accuracy of the teledermatologist is not as

conclusive as thought, since most of the patients indicated with melanoma in a

teleconsultation did not actually have melanoma. Several causes such as image quality, user experience and difficulty in diagnosing a malignant melanoma have been reported as major influence on the accuracy of the diagnosis of a teledermoscopy consultation. It is important to calculate the false positive rates since only false negative rates limit the insight in the results of the data analysis. This means that a new prospective cohort study assessing the false negative rate of melanoma missed when using TDsc should be performed based on an elaborate study protocol approved by the Medical Ethics Committee. This study aimed to develop a first draft for a future prospective cohort study focussed on the aforementioned false negative rates, which hopefully will extent the scientific knowledge about the accuracy and efficiency of teledermoscopy applications.

22

Chapter 3 – Effect of training on TDsc and diagnostic skills of clinicians:

A pre-literature review

Introduction

TDsc has several advantages. First and foremost the advantage of preventing unnecessary referralsfrom primary care professionals to dermatologists and prevent the corresponding costs. Secondly, TDsc provides patients with a relatively fast diagnosis with an of average 8,7 hour compared to a waiting time of about 6 weeks for physical referral [7]. Thirdly, TDsc shows a learning curve of GPS; they become better and faster in recognizing skin

melanoma's and other skin diseases, due to the option they have of asking questions to the teledermatologist online and the feedback given by the teledermatologist to them[8]. Lastly, TDsc might reduce the travel time for patients [27]. The systematic review of Wootton et al. states that teledermatology applications can reduces the travel time with 18-94% [28]. However, some users lack the practical skills for appropriate TDsc use correlated to the advantages mentioned above. One fifth of the dermatologists in France who use

teledermoscopy have never received any training for it [29]. Which might lead to incorrect use of teledermoscopy applications. However, the study of Stratton et al. indicated that TDsc users were open to learning more about the possibilities and appropriate use of TDsc [30]. Further, the buy-in from the GPs providing care through TDsc is still low, the satisfaction of the GPs using TDsc applications is an important factor influencing the buy-in [31]. Besides, the image quality of some TDs consults are too vague to use for diagnosing. Moreover, a lack of consensus on guidelines in the definition of some skin disease was reported [32]. Those issues might be reasons of the limited use of TDsc on a day-to-day basis. This reported limited use of TDsc was the reason to perform this pre-literature review.

To provide the best healthcare possible, it is important that patients receive the most

accurate diagnosis from healthcare providers as possible. With an appropriate diagnosis, the patient can receive the right treatment. Therefore, the diagnostic abilities of the GPs and dermatologist have to be of an appropriate level. It remains unclear whether diagnostic abilities increase when clinicians take up education or training focused on their diagnostic skills and using TDsc applications in the right way [33]. Training of GPs about how to take clear pictures with teledermoscopy applications might reduce the occurrence of vague teledermoscopy images, while ongoing courses focussed on the diagnostic skills of the GPs and dermatologists may improve their performance and diagnostic agreement [32]. There is no aggregated insight in the effects of educating healthcare providers using TDsc with special classes or other educational methods on their TDsc or diagnostic skills. Therefore the first aim of this pre-literature review is to assess if the TDsc skills and

diagnostic abilities of healthcare providers increases with education and training. The second aim of the review focusses on the direct and indirect (e.g. less referrals) educational effects of using TDsc on GPs and dermatologists user experiences.

23

Methods

The systematic literature search was conducted on 10th _{of January 2018. The PubMed}

database and the Embase database were searched for relevant studies. Different search queries were designed for each database to yield relevant studies. The search strategies for Embase and Pubmed are given in Appendix B and Appendix C respectively.

The Pubmed search resulted in 269 articles, and the Embase search in 445 articles. After the removal of 216 duplicates and 36 non English or Dutch articles, 462 articles remained for further examination. Studies on educational effects of teledermatoscopy, teledermatology or dermatology were included.

As a pre-study of the full literature review, a smaller subset was taken from the 462 articles. The following words were used to filter this subset from the larger set: training, education,

experience or confidence.

This search resulted in 130 articles. Before screening of the abstracts, a second manual duplicate check was performed to prevent articles from appearing more than once in the subset. This resulted in the removal of one article which was published in different journals with a different title. The initial duplicate check did not exclude this article because of the slightly different title and minor changes and additions in the abstract.

Initial exclusion of articles was based on the following criteria:

1. The subject of the article did not contribute to the research aim of this study àThe article had a medical focus (e.g. research that focussed on casual relations of melanoma)

àThe article had a pharmaceutical focus (e.g. research that focussed on medical outcomes of specific medication)

2. The article was focussed on risk factors of skin diseases (e.g. is immunosuppression an independent risk factor for melanoma)

3. Collection of abstracts from meetings (e.g. Abstracts for the British Association of Dermatologists 90th Annual Meeting)

4. The article focussed on reasons to use TDsc

After the abstract screening of the remaining articles, a second exclusion round was conducted. The second exclusion was performed based on the following criteria:

1. Article with no full text available online or in the AMC database. 2. Conference abstract.

3. Article which focused solely on willingness to pay, telepathology, patient satisfaction or cost-effectiveness of TDsc.

24 Figure 4. – Exclusion flowchart of the literature review.

Results

Eventually 11 articles remained in this pre-literature review of which six studies showed significant evidence for educational and training effects on the performance (diagnosing, referring) of healthcare providers. Five other articles lack the objective research and evidence of training effects and express the need of training and education, on diagnosing, referring, and use of TDsc, of healthcare providers.

25 Table 4 contains all information about articles with significant evidence and table 5

regarding articles with suggestive evidence about training effects on their TDsc practical skills or diagnostic knowledge and skills. Table 4 and table 5 provide per article the authors, year, country, title and study type of the research.

Table 4. – General information about included articles with significant evidence

ID Authors Year Country Title Study type Focus

1 Boespflug, Guerra, Dalle, & Thomas [29]

2015 France Enhancement of

customary dermoscopy education with spaced education e-Learning

Prospective

controlled trial Dermoscopy

2 Lynne V. McFarland, Gregory J. Raugi, & Gayle E. Reiber [31]

2012 United

States Primary Care Provider and Imaging Technician Satisfaction with a Teledermatology Project in Rural Veterans Health Administration Clinics Cross-sectional (questionnaire) Teledermatology 3 E. Caumes, V. Le Bris, C. Couzigou, A. Menard,† M. Janier and A. Flahault [33]

2003 France Dermatoses associated with travel to Burkina Faso and diagnosed by means of

teledermatology

Prospective

cohort study Teledermatology

4 See, A. C. Lim, K. Le, J.A. See & S.P.

Shumack [34]

2005 Australia Operational

teledermatology in Broken Hill, rural Australia Observational study Teledermatology 5 K. Chen, A. Lim & S. Shumack [35] 2001 Australia Teledermatology: Influence of zoning and education on a clinician’s ability to observe

peripheral lesions

Pre- and

post-education studies Teledermatology

6 J. D.

Gyllencreutz, E.J. Backman, K. Terstappen & J. Paoli [36]

2017 Sweden Teledermoscopy images acquired in primary health care and hospital settings – a comparative study of image quality

Retrospective comparative study

26 Table 5. – General information about included articles with suggestive evidence

The first study of Boespflug et al. shows evidence for a significant increase in diagnostic dermoscopy skills of dermatologists. The dermatologists in the intervention group were enrolled in an e-learning spaced education dermoscopy program. In a pretest-posttest design, the intervention group of dermatologists that had followed an e-Learning module designed on image-based dermoscopic questions, had better results at the post-test than the control group of dermatologists that did not follow the e-Learning module. Beside the learning effect, 92% of the participants were extremely or very satisfied with the e-Learning module. This study shows that spaced education in combination with in-class training improve dermatologists diagnostic performances in dermoscopy.

In the cross-sectional study of McFarland et al. 68% of the teledermatology consults did not require additional follow-up after education of the GPs. The education consisted of weekly classes of conference calls and shared slideshows covering a variety of dermatology topics. Furthermore, the article reports that 71% of the care providers and 94% of the imaging technicians were extremely satisfied with the training project. The training project made healthcare providers using TDsc, more secure when diagnosing patients. Almost all

healthcare providers (95%) found the continuing education classes on dermatology diagnosis and treatment topics useful. The article concludes that although the satisfaction among users is high, and the referral follow-up decreases because of the education, challenges remain in the increased workload for GPs and adequate technology support.

ID Authors Year Country Title Study type Focus

7 John A. Ford & Augustine Pereira [37]

2015 United

Kingdom Does teledermatology reduces secondary care referrals and is it acceptable to patients and doctors?:
a service evaluation
 Controlled trial/ time series analysis Teledermatology 8 Julie C. Philp, Ilona J. Frieden & Kelly M. Cordoro [38] 2013 United

States Pediatric Teledermatology Consultations: Relationship Between Provided Data and Diagnosis

Retrospective

cohort study Teledermatology

9 E. Dahl [39] 2014 Norway Briefing notes on maritime

teledermatology - Teledermatology 10 K. Beroukhim, C. Nguyen, M.J. Danesh, J. J. Wu & J. Koo [40] 2015 United

States The evolving role of Primary care Practitioners in Dermatology - Dermatology 11 D. Moreno-Ramírez & G. Argenziano [21]

2017 Spain Teledermatology and Mobile applications in the Management of Patients with Skin Lesions

Observational study

27 A prospective cohort study of E. Caumes et al. reports that after short education of GPs and continuing feedback of dermatologists the diagnostic agreement between the local GPs and the remote dermatologists, was 49% overall with a 95% CI between 41 – 58. Agreement between the GP and the dermatologists on diagnoses improved significantly over time (p < 0.5). The GP was trained in most common skin diseases. The GPs received continuing

extensive feedback on the patient cases which resulted in the learning effect. The GPs had to report the most likely diagnosis and treatment recommendation, the teledermatologist would then report their findings and a provide a short explanation and learning tips to the GP. Because of the ongoing educational effect, more accurate diagnoses were reported and better treatments given as a consequence of the more accurate diagnoses. This study illustrates how teledermatology directly benefits both patients and GPs.

The study of A. See et al. reports that GPs experience a learning effect when they have to retake pictures, as a result of the feedback of the teledermatologist, in order to provide the teledermatologists with correct skin pictures. As a consequence, the proportion of poor-quality images decreased in the last 8 months of the study (5%: 1/20). The better poor-quality of photos appeared to be directly related to how good the teledermatologist could judge the photos and diagnose the patient accordingly. In addition, the majority of teledermatology stakeholders like GPs, teledermatologists and even patients indicated in the questionnaire that they had confidence in the telediagnosis (3.1 on scale of 0 – 4, with a range between 2 – 4). The article concludes with a recommendation of more extensive education in order to improve diagnostic performances and image quality of GPs even more.

The study of K. Chen et al. reports a positive trend regarding educational effect of the dermatologists in the intervention group. The intervention group was educated in

recognizing specific lesions through teledermoscopy images. The education took place at two annual meetings. However, the educational effect on recognition of these lesions of this study was not significant (p-value of 0.08).

The article from D. Gyllencreutz et al. states that there is very little to no difference in teledermoscopy images taken by trained GPs and untrained GPs. The images acquired by untrained GPs were compared with images taken by trained GPs. The images of the

untrained GPs were rated, by dermatologists, with a slightly lower quality, but this was not statistically significant. In addition, there was no difference in agreement between the TDcs diagnosis based on the images of untrained clinicians in comparison with the final clinical or histopathological diagnosis. In conclusion, the article shows that the teledermoscopy image quality is not negatively affected when taken by untrained GPs.

A time-series analysis study of Ford et al. aimed to assess if the use of teledermatology reduces referrals because of the learning curve that GPs experience when introducing teledermatology applications in practice. Referral data for 12 months before and after the introduction of teledermatology was compared in practices with and without

teledermatology. The study did not find any evidence that implementation of

teledermatology reduced referrals. The authors state that this is due to the nature of the small pilot study, and highlighted the educational benefits that clinicians reported through structured user dialogues. The article urges that there is more research needed to measure if these educational benefits translate into better patient outcomes.

28 The article of Philp et al. argues that training of clinician’s photograph skills and

standardization of teledermatology images and standard historical data templates is essential for proper diagnosis of skin diseases. The retrospective study showed that badly taken photos are associated with the lack of a diagnosis which subsequently is correlated with an unnecessary biopsy or in-person consultation. Basic training of GPs in how to take appropriate teledermatology pictures is arguably more important than medical education about specific teledermatology diagnoses according to the authors.

The briefing notes of E. Dahl et al. underline the need to educate medical staff on ships confronted with skin lesions of passengers. These lesions may be difficult to describe for seafarers without extensive medical training and background. Training can prevent imprecise descriptions and therefore benefit patients on ships according to Dahl.

The article of K. Beroukhim et al. states that GPs face a shortage of dermoscopy educational resources that fit in their busy schedules. Beroukhim sees opportunities in internet-based medical modules. The article cites research (Brochez et al. 2001) which illustrated that a lecture focussed on GPs diagnosing malignant melanoma with teledermoscopy, resulted in an increase in sensitivity of recognizing malignant melanoma from 72% to 84%. The author suggests that training GPs to diagnose more accurate with teledermoscopy applications, results in a more cost-effective healthcare.

The study of Moreno-Ramírez et al. states that the teledermatology applications themselves can be used as training tools to improve diagnostic skills of GPs and teledermatologists. Such learning effects are due to the regular feedback from teledermatologist to GPs according to the authors. Further, the teledermatologists may learn from rare cases and might even use them for educational purposes in medical school.

Generally speaking, the first six articles discussed above present evidence for a significant increase in diagnostic accuracy when a healthcare provider is educated in diagnosing patient with teledermoscopy images. The last five articles do not prove statistically evidence for learning effects but present suggestive evidence for a learning effect, or express the urge for education of healthcare providers using teledermoscopy applications. In addition, the GPs using TDsc report to have more confidence in their diagnostic abilities when they are trained in using teledermoscopy applications in daily practice.

29

Discussion

The aim of this pre-literature review was to evaluate whether diagnostic abilities and TDsc skills of healthcare providers increase through education. In addition, indirect educational effects (e.g. less referrals) were reviewed.

This pre-literature review indicates that there is still very little objective research which focusses on the educational value of using TDsc and the training of healthcare providers using TDsc. There are multiple studies that suggest that educating healthcare providers in using TDsc is beneficial to diagnosing and treating patients, but lack objective proof. The studies in the review differed enormously in their study designs, but all authors came more or less to the same conclusion; increasing diagnostic dermatological knowledge and appropriate use of TDsc applications of GPs and dermatologists through educational classes and trainings leads to more accurate diagnoses and subsequently to better patient care. Most authors provided some sort of recommendation about the need of extra education for healthcare providers in dermatology diseases and/or using teledermatology or

teledermoscopy to ensure their appropriate application in daily practice. All studies tried to correlate a decrease in physical referrals of patients to a learning effect due to GPs using TDsc, but no significant evidence was found.

The outcomes among the studies were not comparable, since the participant population differed greatly per study and the inclusion of skin diseases assessed varied per study. In addition, the image quality acquired by the clinicians using TDsc in the studies varied which influenced the diagnostic outcomes. Several authors reported that the image quality depends on the years of experience with TDsc, the function of the healthcare provider and age of the user.

Remarkably, nine articles in total show a high user satisfaction of healthcare providers using teledermatology or teledermoscopy but few studies show real increases in performance of the healthcare providers diagnostic ability. However, articles reporting statistically

significant effects of education on TDsc and diagnostic skills of clinicians and those that do not, state that it is reasonable to assume that diagnostic accuracy improves over time, because GPs using TDsc become more experienced and receive continuous from the teledermatologist on their proposed diagnoses.

Several studies showed non-significant results. This may be due to a small number of participants involved in the second educational meetings. The authors state that it is therefore presumable that future studies with more participants will result in a significant learning effect [35]. Other reasons for the non-significant results may be due to the

experienced participants in the studies. Some of the participants already had experience in using teledermoscopy applications in daily practice. In addition, the equipment used in this study was reported as user-friendly, easy-to-use and required little technical know-how which could contribute to the non-significant differences[36].

Results from this pre-literature review are comparable to the results of other systematic reviews such as the one from of Sinderen et al. or the research of Boesflug et al. [7, 29].

30 Both articles concluded that larger studies should be performed focussing on the

educational effects of training clinicians, in order to make the results more generalizable. Those studies state that a meta-analysis could not be performed due to the heterogeneity of the included studies, which is also the case with the included articles of this study since the participants characteristics of each included articled differed greatly.

The study of van Sinderen et al. had a slightly different scope; primary focus of inclusion on diagnostic accuracy which resulted in different inclusion and exclusion criteria. Therefore, this study found more evidence for training of GPs and dermatologists in comparison with the study of van Sinderen et al. However, it is underlined in several included articles of this study that training of GPs and dermatologists improves their diagnostic accuracy and as a result greater diagnostic agreement between GPs and teledermatologists might be achieved over time.

Those studies did not find much evidence on a possible diagnostic shift in easy-to-diagnose skin lesions and complex-to diagnose skin lesions. This study did not find much evidence on that shift as well, however this study included more articles that take note of that shift and recommend future research to examine it further.

Strengths and weaknesses

A strength of this pre-literature reviews is that all abstracts have been screened twice. The first time to make the initial exclusion of the articles that appeared in both databases. The second screening was to make sure no important articles were missed and that the included articles align with the aim of this review. Articles that were focussed on teledermatology were not excluded. This is because teledermatology has been in use longer than

teledermoscopy and has been researched more in comparison to teledermoscopy. A weakness of this review is that some of the included articles do not have a strong study design which results in the lack of statistical evidence for educational effects. Most of those articles however conclude with recommendations for educational lectures and suggestions for future research with stronger study designs (i.e. prospective cohort studies or random clinical trials).

Future research

Future research should be conducted with more robust study designs (e.g. random

controlled trials, cohort studies) and larger sample sizes. Those studies may perhaps result in stronger significant evidence providing significantly evidence on educating GPs and

dermatologists on their diagnostic abilities and practical TDsc skills. Some of the articles from this review have a relatively weak study design (e.g. review articles, briefing notes), those articles are not useless in terms of the recommendations provided but, lack

statistically significant evidence for educational effects of TDsc on improvement of the

diagnostic skills of GPs and dermatologists and perhaps resulting in better patient treatment. Further studies are necessary to evaluate the benefits of educating healthcare providers whom use TDsc. No study to date has been performed on the long-term memory effects of education and training on the diagnostic abilities of the clinicians. It could be interesting to research how to make the educational effects from the lectures last. Perhaps the lectures

31 may have to be repeated yearly or once every two years at special dermatology conventions in order to keep the medical knowledge up to date with new innovations and diagnostic techniques.

Furthermore, one article stated that the clinicians were trained using a combination of in-class lectures and an e-learning spaced education module online. It might be interesting for future studies to research which method is more successful, or that perhaps combination of multiple learning methods is more successful[29].

Research with the focus on educational effects of TDsc on patient outcomes is important and has not yet been performed. Future research on patient outcomes could focus on timeliness and the effectiveness of care or perhaps even mortality rate. The timeliness could be analysed through waiting times, and the effectiveness of care could focus on guideline adherence. Guideline adherence should be measured with retrospective data. The outcomes of those patient cases are available and one could retrospectively check if the GPs and dermatologists adhered to the clinical guidelines during the diagnostic process. The mortality rate is more difficult to correlate to the use of TDsc, but the prevented cases of serious skin cancer might be an interesting variable to analyse in future research.

Conclusion

Based on the results of this pre-literature review it can be concluded that the evidence of the learning effects of TDsc and training effects of healthcare providers whom use TDsc is still limited but, the results show promising potential on the diagnostic abilities of GPs and dermatologists. Most of the studies lack the objective research to support the claims they set out to prove, and therefore end up merely giving recommendations. However, the research with significant effects of educating clinicians through special lectures and online modules show a positive increase on the diagnostic abilities of clinicians. The lack of significant results is mainly due to small sample sizes of the studies and the lack of

homogenous participants population. Future research should consist of more robust study designs with larger sample sizes and more homogenous participants in order to allow a meta-analysis in the future. The suspected shift in diagnosis and treatment of relatively easy-to-diagnose skin diseases by the GPs and immediate referral of complex-easy-to-diagnose skin diseases to the dermatologist should be objectively measured. Some of the articles suspect that this shift will occur in the future because of the learning effect that comes with TDsc use, but lack the scientific evidence.

32

Chapter 4 – Diagnostic trend analysis of teledermoscopy consultations

over time: A retrospective data analysis

Introduction

Figure 5 below shows what the actual diagnostic section of the TDsc service from KSYOS software system looks like and how the International Classification of Diseases (ICD-10) code is filled in by a teledermatologist when he or she sends the teleconsultation back to the GP. It illustrates how the teledermatologist can give advice to the GP. In addition, the

teledermatologist evaluates if a visit to a dermatologist is necessary. Over time, the feedback from the teledermatologist might have increased the GP’s diagnostic abilities regarding specific or common skin diseases, which could eventually decrease unnecessary referrals to dermatologists. This decrease would add to the results of van der Heijden et al.’s study, which stated that 74% of the teleconsultations were conducted in order to prevent a referral to the dermatologists; this decrease confirms, therefore, that TDsc is an effective diagnostic tool for GPs [41].

Figure 5. Example of a teleconsultation

The teledermatologist is obliged to fill in a diagnosis code (ICD-10) in the teleconsultation when he or she examines the teleconsultation with the corresponding skin images of a patient. The GP is not obliged to fill in an ICD-10 code; this is optional when a GP sends the teleconsultation to the teledermatologist. If the GP chooses to do so, he or she has to fill in the ICD-10 code before sending the TDsc to the teledermatologist.

The literature review of Chanier et al., which focused on the diagnostic learning effect of teleconsultations in the practice of GPs and dermatologists, concluded that future research should examine whether a diagnostic shift of GPs takes place as a results of the reported learning effect. Since, some of the included articles of that review posited that the diagnostic behaviour of TDsc users might change over time but lack the scientific evidence. This shift indicates that after using TDsc for a while, GPs diagnose fairly ‘easy’ skin diseases more often and more accurately. As a result, GPs might treat those patients with easy-to-diagnose skin diseases themselves without the use of TDsc, which was also reported in the pre-literature review (see articles in Table 5) [42]. However, no study to date has objectively examined this suspected shift.

These diagnostic trends possibly occur because the GPs might experience a learning effect over the years through the continuous feedback from the teledermatologist. The GPs may learn to recognise patients with similar symptoms and to diagnose them based on former feedback from the teledermatologist about similar patient cases. As a result, GPs may remember that more complicated cases are to be referred to the dermatologist and that they may treat relatively easy cases themselves based on what they have learned from previous feedbacks given by the teledermatologist [41,43,44]. Studies reported that GPs

33 using TDsc experience a learning effect in 95.1% of the teledermoscopy consultations

[27,41]. Possibly, after having used teledermoscopy for a certain time period and having seen the same patient cases multiple times, GPs might have learned how to treat those patients without having to consult or seek advice from the teledermatologist. Nonetheless, this phenomenon has not been proven to date by objective research.

This phenomenon could in turn, decrease physical referrals of patients from the GP to the dermatologist’s office. Notably, less physical referrals of patients might not be an immediate result of learning effects after GPs have used teledermoscopy services [45,46]. Other factors may explain the reduction in physical referrals of patients by GPs over time. For instance, high costs could be an incentive for patients not to go to the dermatologists office. Especially in countries where a patient does not get reimbursed for their consult at the dermatologists office.

Accordingly, GPs report the ongoing learning effect of using TDsc, and multiple literature reviews conclude that offering educational classes to GPs is even more beneficial to all stakeholders involved as the knowledge acquired from these classes can lessen unnecessary referrals to the dermatologist’s office and can result in better treatment of easy skin

diseases by the GPs themselves [9,10,41,44].

Thus far, most studies regarding the possible learning effects of using teledermoscopy among clinicians focus on training and educational outcomes but diagnostic shifts have not been researched. In this study a retrospective data analysis will be performed on the ICD-10 code data from teledermoscopy consultations conducted by GPs and teledermatologists. This data analysis aims to assess if GPs experience a learning effect of the continuous

feedback from the teledermatologist. This learning effect is assessed through analyses of the ICD-10 codes of the teleconsultations filled in by the GPs. The hypothesized learning effect is measured in terms of changes in the diagnostic behaviour of GPs in terms of a shift in the ICD-10 codes filled in by GPs and teledermatologists. Since GPs are not obligated to fill in ICD-10 codes and the teledermatologists are, the ICD-10 codes entered by the

teledermatologists are also analysed.

This shift might be correlated to a learning effect of GPs using teledermoscopy consultation services offered by KSYOS, and would assert that over time, GPs would label less

teleconsultations for relatively easy-to-diagnose patient cases and use teleconsultations for more complex cases. In addition, the diagnostic agreement based on corresponding ICD-10 codes filled in by a GP and dermatologist for similar cases, might increase since the GPs might learn how to identify skin lesions and other skin diseases. Therefore, as a result of the learning effect among GPs using TDsc, their diagnoses and ICD-10 codes would correspond more often with the ICD-10 codes entered by the teledermatologists. Lastly, the GPs might have become more confident in their diagnostic abilities and perhaps as a consequence, would fill in more teleconsultations with a diagnosis code over time.