RESEARCH ARTICLE
First international external quality assessment
scheme of nucleic acid amplification tests for
the detection of Schistosoma and
soil-transmitted helminths, including
Strongyloides: A pilot study
Piet CoolsID1, Lisette van Lieshout2,3, Rob Koelewijn3,4, David Addiss5, Sitara S.
R. Ajjampur6, Mio Ayana7, Richard S. BradburyID8, Jason L. Cantera9, Daniel Dana7,
Kerstin FischerID10, Rubina Imtiaz11, Joyce Kabagenyi12, James Lok13, James McCarthy14,
Rojelio Mejia15, Zeleke Mekonnen7, Sammy M. Njenga16, Nurulhasanah Othman17, Hongguang Shao13, Rebecca Traub18, Marjan Van EsbroeckID19, Jozef Vercruysse1,
Johnny Vlaminck1, Steven A. WilliamsID20,21, Jaco J. Verweij22, Jaap J. van Hellemond3,4,
Bruno Levecke1*
1 Laboratory of Parasitology, Ghent University, Merelbeke, Belgium, 2 Leiden University Medical Center, Leiden, The Netherlands, 3 Dutch Foundation for Quality Assessment in Medical Laboratories, Nijmegen, The Netherlands, 4 Erasmus University Medical Center, Rotterdam, The Netherlands, 5 The Task Force for Global Health, Decatur, Georgia, United States of America, 6 Christian Medical College, Vellore, India, 7 Jimma University, Jimma, Ethiopia, 8 Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America, 9 PATH, Seattle, Washington, United States of America, 10 Washington University in St. Louis, St. Louis, Missouri, United States of America, 11 Children Without Worms, Decatur, Georgia, United States of America, 12 Medical Research Council/Uganda Virus Research Institute and London School of Hygiene & Tropical Medicine Uganda Research Unit, Entebbe, Uganda, 13 University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, 14 QIMR Berghofer Medical Research Institute, Herston, QLD, Australia, 15 Baylor College of Medicine, Houston, Texas, United States of America, 16 Kenya Medical Research Institute, Nairobi, Kenya, 17 Universiti Sains Malaysia, Penang, Malaysia, 18 The University of Melbourne, Melbourne, Australia, 19 Institute of Tropical Medicine, Antwerp, Belgium, 20 Smith College, Northampton, Massachusetts, United States of America, 21 University of Massachusetts, Amherst, Massachusetts, United States of America, 22 Elisabeth-TweeSteden Hospital, Tilburg, The Netherlands
*bruno.levecke@ugent.be
Abstract
Background
Nucleic acid amplification tests (NAATs) are increasingly being used as diagnostic tools for soil-transmitted helminths (STHs; Ascaris lumbricoides, Trichuris trichiura, Necator ameri-canus, Ancylostoma duodenale and A. ceylanicum), Strongyloides stercoralis and Schisto-soma in human stool. Currently, there is a large diversity of NAATs being applied, but an external quality assessment scheme (EQAS) for these diagnostics is lacking. An EQAS involves a blinded process where test results reported by a laboratory are compared to those reported by reference or expert laboratories, allowing for an objective assessment of the diagnostic performance of a laboratory. In the current study, we piloted an international EQAS for these helminths (i) to investigate the feasibility of designing and delivering an
a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS
Citation: Cools P, van Lieshout L, Koelewijn R, Addiss D, Ajjampur SSR, Ayana M, et al. (2020) First international external quality assessment scheme of nucleic acid amplification tests for the detection of Schistosoma and soil-transmitted helminths, including Strongyloides: A pilot study. PLoS Negl Trop Dis 14(6): e0008231.https://doi. org/10.1371/journal.pntd.0008231
Editor: Maria Victoria Periago, Consejo Nacional de Investigaciones Cientificas y Tecnicas, Fundacio´n Mundo Sano, ARGENTINA
Received: August 26, 2019
Accepted: March 17, 2020
Published: June 16, 2020
Copyright:© 2020 Cools et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability Statement: All relevant data are within the manuscript and its Supporting Information Files.
Funding: This study was funded by a grant from the Bill and Melinda Gates foundation (OPP1120972,www.starworms.org), Dutch Foundation for Quality Assessment in Medical Laboratories (SKML) and Children Without
EQAS; (ii) to assess the diagnostic performance of laboratories; and (iii) to gain insights into the different NAAT protocols used.
Methods and principal findings
A panel of twelve stool samples and eight DNA samples was validated by six expert labora-tories for the presence of six helminths (Ascaris, Trichuris, N. americanus, Ancylostoma, Strongyloides and Schistosoma). Subsequently this panel was sent to 15 globally dispersed laboratories. We found a high degree of diversity among the different DNA extraction and NAAT protocols. Although most laboratories performed well, we could clearly identify the laboratories that were poorly performing.
Conclusions/Significance
We showed the technical feasibility of an international EQAS for the NAAT of STHs, Stron-gyloides and Schistosoma. In addition, we documented that there are clear benefits for par-ticipating laboratories, as they can confirm and/or improve the diagnostic performance of their NAATs. Further research should aim to identify factors that explain poor performance of NAATs.
Author summary
Tests that detect parasite DNA in human stool are increasingly being used for the diagno-sis of infections with intestinal worms, including schistosomiadiagno-sis. To ensure the quality in diagnostic testing of these parasitic worms, it is important that laboratories evaluate the diagnostic performance of their DNA-based tests. This can best be achieved by participat-ing in an external quality assessment scheme (EQAS). An EQAS involves a blinded pro-cess where test results reported by a laboratory are compared to those reported by reference or expert laboratories, allowing for an objective assessment of the diagnostic performance of a laboratory. Currently, such an EQAS for parasitic worms is lacking. We therefore piloted an international EQAS for the diagnosis of parasitic worms involving 15 laboratories in Africa, Asia, Australia, Europe, and North America. Although most labora-tories performed well, we could clearly identify those laboralabora-tories that may need to improve their test protocol. We found that laboratories were using many different test protocols, and further research should aim to verify whether this has an impact on the per-formance of the diagnostic outcomes.
Introduction
Soil-transmitted helminths (STHs, i.e.Ascaris lumbricoides, Trichuris trichiura and the
hook-wormsAncylostoma duodenale, Ancylostoma ceylanicum and Necator americanus), Strongy-loides stercoralis and Schistosoma (S. mansoni, S. haematobium and S. japonicum) are parasitic
worms that still impose a burden on global health. The STHs affect one-fifth of the world’s population and cause 1.9 million disability adjusted life-years (DALYs) [1]. Schistosomiasis affects 230 million people [2] and has been estimated to cause 1.4 million DALYs [3].S. stercor-alis has an estimated global prevalence of 370 million [4], but the global burden is still to be determined. Although these infections mainly affect the poorest populations in low- and Worms. PC was financially supported the Bill and
Melinda Gates foundation (OPP1120972,www. starworms.org). BL was a postdoctoral fellow of the Research Foundation (www.FWO.be). JV was financially supported through an International Coordination Action of the Flemish Research Foundation. LvL is funded in part by the EDCTP2 program supported by the European Union (grant number RIA2017NCT-1845 STOP;www. stoptheworm.org). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist.
middle-income countries, hookworms andS. stercoralis are also found in low-income regions
of the United States [5], marginalized communities in Europe [6] and in Indigenous popula-tions in Australia [7,8]. In addition, strongyloidiasis and schistosomiasis are important travel-related diseases, causing substantial morbidity, both in travelers and migrants moving from endemic to non-endemic countries [9,10].
Diagnosis of these parasites is still largely based on the detection of worm eggs or larvae in stool or, in the case ofS. haematobium, eggs in urine samples. However, nucleic acid
amplifica-tion tests (NAATs) are increasingly being used as diagnostic tools for soil-transmitted helmin-thiasis, strongyloidiasis and schistosomiasis [11]. For example, NAATs are already considered the standard diagnostics in some clinical laboratories in high-income countries [12,13]. In the public health field, NAATs are being explored as diagnostic tools to assess the distribution of prevalence and infection intensity at the population level in endemic areas [14]. These parasi-tological indicators are used to guide both the implementation and evaluation of large-scale deworming programs that aim to control the morbidity attributable to soil-transmitted hel-minthiasis and schistosomiasis [15].
Currently, a large diversity of available NAATs are being applied, but there is no external quality assessment scheme (EQAS) to support the use of these diagnostics [16,17]. However, such EQAS are essential to attain the highest achievable diagnostic quality and to improve health outcomes in terms of care and treatment at the individual level, as well as monitoring of disease burden and response to control at the population level. An EQAS involves a blinded process in which test results reported by a laboratory are compared to those reported by refer-ence or expert laboratories testing the same panel of samples, allowing for an objective assess-ment of the diagnostic performance of a laboratory [18]. EQAS have already proven to be essential to highlight weaknesses in NAATs used for the diagnosis of other parasitic and tropi-cal diseases such as diarrhea-causing protozoa [19], malaria [20], yellow fever [21] and Zika [22].
In the current study, we piloted an EQAS for the DNA-based detection ofSchistosoma and
STHs, includingStrongyloides, the so-called Helminth External Molecular Quality Assessment
Scheme (HEMQAS). The objectives of this HEMQAS pilot were (i) to investigate the feasibility of designing and delivering a technically and logistically challenging EQAS for the detection of
Schistosoma and STHs, including Strongyloides in both preserved stool and DNA samples, (ii)
to evaluate and compare the diagnostic performance of laboratories, and (iii) to gain insights in the diversity of the different NAAT protocols used.
Methodology
Ethics
This study was part of work package 1 of the Starworms study (www.starworms.org). The Star-worms study was registered (Belgian registration number B670201627755) and approved by the Ethics Committee of the Faculty of Medicine and Health Sciences of Ghent University and the Ghent University Hospital, Belgium (reference number 2016/0266). Local approval was obtained at the Institutional Review Board of Jimma University, Ethiopia (reference number RPGC/547/2016). The school director, teachers, parents or legal guardians, and children were informed about the objectives and procedures of the study. Written informed consent was obtained from the parents or legal guardians of all participating children. Participation was voluntary. In addition to samples obtained from these children, two stool samples were col-lected from a person living in a non-endemic region, after obtaining a written consent.
S. stercoralis infective third-stage larvae used in this study were from a canine strain
(Philadelphia, USA). This vivarium is accredited by the International Association for Assessment and Accreditation of Laboratory Animal Care. Dogs were infected with
S. stercoralis and maintained thereafter in accordance with protocols 804798 and 804893
approved by the University of Pennsylvania Institutional Animal Care and Use Committee. All protocols, as well as routine husbandry care of the animals, were carried out in strict accordance with the Guide for the Care and Use of Laboratory Animals of the US National Institutes of Health.
Nomenclature
In the context of collecting samples for the HEMQAS study, we use the species names of the helminths where appropriate. In the context of the HEMQAS panel, given that not all experts laboratories used species specific assays (see further), we refer to the respective HEMQAS sam-ples as being validated forAscaris, Ancylostoma, Schistosoma, Strongyloides and Trichuris
(even though microscopic screening and/or epidemiology allowed speciation). We used the termN. americanus throughout the manuscript, given that all expert laboratories used species
specific assays forN. americanus.
Designing and delivering HEMQAS
We designed and delivered HEMQAS through three consecutive steps (Fig 1). In the prepara-tory phase, we identified the expert and participating laboratories, collected the samples and prepared the panel. Expert laboratories were identified and defined based on their scientific track record on molecular diagnostics for STHs andSchistosoma and authority in this field. In
addition, we developed a questionnaire to gain insights into the diversity in NAAT protocols. Next, the HEMQAS panel was distributed to the expert laboratories, which validated the pro-cedures of homogenizing and aliquoting samples, and the targets in the panel. In a final step, the HEMQAS panel was distributed to the participating laboratories, which reported their findings and received feedback on their diagnostic performance. In the following paragraphs we will discuss each of the three steps in more detail.
Preparatory phase
(i) Coordination and identification of laboratories. The study was coordinated by the
Laboratory of Parasitology of Ghent University and the Department of Medical Microbiology and Infectious diseases (Erasmus University Medical Center), which is the coordinating center for parasitological EQAS of the Dutch Foundation for Quality Assessment in Medical Labora-tories (SKML;www.skml.nl). The SKML has long-term experience in setting up EQAS and
Fig 1. Overview of the different steps, the expert and participating laboratories of the HEMQAS pilot study.
has already established an international EQAS for the DNA-based diagnosis of intestinal pro-tozoa [19].
We aimed to include expert laboratories for the validation of the HEMQAS panel in such way that each target (Ascaris, Trichuris, Ancylostoma, N. americanus, Strongyloides and Schisto-soma) was validated by a minimum of five expert laboratories geographically dispersed around
the world. Expert laboratories were considered if their NAAT protocols had been published or when their expertise on NAATs was internationally acknowledged. Based on these criteria, two clinical and four research laboratories across Australia (n = 1), Europe (n = 3) and the USA (n = 2) were identified as expert laboratories (Table 1). Each of these laboratories com-mitted to examine the HEMQAS panel in a blinded way and to discuss the overall obtained results applying a priori defined criteria explained below.
For the actual HEMQAS pilot study, nine laboratories were identified within the network of the HEMQAS coordinators (Table 1). Here, we only considered laboratories for participa-tion if they had an operaparticipa-tional NAAT for at least two of the six targets. We also aimed to include participants from both endemic and non-endemic countries. Furthermore, all six expert laboratories that were involved in the validation of the HEMQAS samples were also
Table 1. The expert and participating laboratories of the HEMQAS pilot study. The order of the list does not correspond with the order of the laboratories inTable 5.
Country Institution/Organization Laboratory/Department Laboratory type
Expert laboratories
Australia QIMR Berghofer Medical Research Institute The Clinical Tropical Medicine Laboratory Research laboratory The
Netherlands
Erasmus Medical Center Department of Medical Microbiology and Infectious Diseases
Clinical laboratory The
Netherlands
Elisabeth TweeSteden Hospital Laboratory of Medical Microbiology and Immunology Clinical laboratory The
Netherlands
Leiden University Medical Center Department of Parasitology Research laboratory USA Baylor College of Medicine Laboratory of Clinical Parasitology and Diagnostics Research
laboratory USA Smith College and the University of Massachusetts Department of Biological Sciences and Biochemistry;
Program in Molecular and Cellular Biology
Research laboratory Participating laboratories
Australia University Melbourne Melbourne Veterinary School Research laboratory Belgium Institute of Tropical Medicine Central Laboratory for Clinical Biology Clinical
laboratory India Christian Medical College, Vellore The Wellcome Trust Research Laboratory Research
laboratory Kenya Kenya Medical Research Institute Eastern and Southern Africa Centre of International
Parasite Control
Research laboratory
Malaysia Universiti Sains Malaysia Institute for Research in Molecular Medicine Research laboratory Uganda Medical Research Council/Uganda Virus Research Unit & London School
of Hygiene and Tropical Medicine Uganda Research Unit
Immunity and Vaccines Programme Research laboratory USA Centers for Disease Control and Prevention Parasitic Diseases Branch Research
laboratory
USA PATH Diagnostics Program Research
laboratory USA Washington University in St. Louis Infectious Diseases Division Research
laboratory https://doi.org/10.1371/journal.pntd.0008231.t001
included as participating laboratories, resulting in a total of 15 laboratories. Eleven laboratories were research laboratories and four of the 15 laboratories were clinical laboratories. Two labo-ratories were from Africa, two from Asia, two labolabo-ratories from Australia, four from Europe, and five from the USA. Participating in the study was free of charge.
(ii) Collection of specimens and preparation of the panel. The HEMQAS panel
included both a stool panel and a DNA panel. For the stool panel, a total of 12 stool samples were collected. Ten of these clinical stool samples were collected in Jimma (Ethiopia) as part of the Starworms study [23]. Stool samples from school-aged children in which one of the STH species orS. mansoni had been detected using the Kato-Katz technique [23] were preserved in ethanol by filling half of a 50 ml Falcon tube with stool and suspending this in 25 ml of 96% ethanol. These ten stool samples were selected, as much as possible, to include mixed infec-tions in order to maximize the total number of targets while minimizing the number of sam-ples. The samples were kept refrigerated until shipment at ambient temperature to the Laboratory of Parasitology (Ghent University, Belgium). Two stool samples were collected from one person in Belgium. One of these stool samples was spiked withS. stercoralis
third-stage larvae at a concentration of 200 larvae per gram (Table 2) before being further diluted circa 40 times, the other stool sample was used as a negative control. All stool samples were homogenized, further diluted as needed, and aliquoted according to the in-house method of SKML applied for its protozoa EQAS panel [19].
For the HEMQAS DNA panel, worm materials were obtained from different partners (Table 2). DNA from adultA. lumbricoides worms (only the heads were used), N. americanus
third-stage larvae,S. stercoralis infective third-stage larvae and S. mansoni adult worms was
extracted by one expert laboratory (Elisabeth-TweeSteden Hospital, Tilburg, The Nether-lands). For this, the QIAamp DNA Mini Kit (Qiagen) was used according to the manufactur-er’s guidelines. The quantity of the DNA was measured using the Qubit fluorometer. From each DNA extract, a tenfold dilution was made and included in the DNA panel, resulting in a total of eight DNA extracts.
(iii) Development of QBase and questionnaire. In order to gain insights into the
differ-ent NAAT protocols used by the participants, we developed a questionnaire that was incorpo-rated into QBase. QBase is an online results submission tool developed and used by SKML for other EQAS. It was adapted to fit the specific needs of this pilot study. A summary manual and an extended manual were developed to introduce all participants to this submission tool (S1 File).
Validation of the HEMQAS panel
Following the distribution of the panel to the expert laboratories, the panel was validated. The validation of the panel was two-fold and was based on the procedures previously described by Schuurs and coworkers [19]. First, we validated the procedures for homogenizing and aliquot-ing stool samples. Second, we validated the targets in the panel. In the followaliquot-ing paragraphs we will discuss each of the steps in more detail.
Table 2. Origin and type of worm material used for the HEMQAS panel.
Species Origin Host Type of material
Ascaris lumbricoides Jimma, Ethiopia Human Adult worm
Necator americanus Leiden, The Netherlands Human Third-stage larvae
Schistosoma mansoni Rotterdam, The Netherlands Hamster Adult worms
Strongyloides stercoralis Philadelphia, United States Dog Third-stage larvae
(i) Distribution of the panel. The HEMQAS panel for validation consisted of twenty
samples (twelve ethanol-preserved stool samples and eight DNA samples) and was sent out to the six expert laboratories who received the samples blinded. For validation, the panels were organized in such a way that each expert laboratory received three or four samples as quintu-plicates (i.e., five aliquots) and the remaining 16 to 17 samples as single aliquots, and that the different quintuplicates were equally dispersed over the expert laboratories. Given that there were six laboratories and a panel of twenty samples, some expert laboratories received three samples as quintuplicates and 17 samples as single aliquots, and the other expert laboratories received four samples as quintuplicates and 16 samples as single aliquots. Expert laboratories were asked to remove the ethanol from the ethanol-preserved stool samples according to a standard SKML protocol that was sent along with the samples (seeS2 File). Further, expert laboratories were asked to process and analyze all samples according to their in-house DNA extraction and NAAT protocol.
(ii) Validation of stool homogenization and aliquoting methods. Eggs of STHs and
Schistosoma and larvae of S. stercoralis can be unequally dispersed in a stool sample. In order
to validate the homogenization and aliquoting procedures, quintuplicates of each sample were analyzed by a single laboratory. We concluded that homogenization and aliquoting procedures were appropriate when the standard deviation of the Cq-values of the targets with the lowest Cq-values (i.e., the highest DNA concentration) across the quintuplicates did not exceed two. We did not consider this criterion for targets for which the Cq-values were high (i.e., the lowest DNA concentration), as a large variation in values is already expected (the variation in Cq-values between the aliquots of the same samples increases as the DNA concentration drops [24]).
(iii) Validation of the targets. At least five expert laboratories assessed all samples for all
targets, except forSchistosoma, which was assessed by four expert laboratories. All laboratories
were blinded to the microscopy data and the outcome of the other laboratories. Expert labora-tories were asked to report their results to SKML within one month using a standard Excel file. Reports were compiled by SKML and predefined criteria were applied to validate samples as positive, negative or educational for a specific target (i.e.,Ascaris, Trichuris, N. americanus, Ancylostoma, Strongyloides and Schistosoma). A sample was validated as positive for a
particu-lar target when the following three criteria were fulfilled: (i) all quintuplicates analyzed by one of the expert laboratories was positive for that target, (ii) the standard deviation of the Cq-val-ues of the quintuplicates was less than two cycles, and (iii) all other expert laboratories con-firmed the presence of that particular target. These criteria guarantee that DNA of the target can be detected by state-of-the-art NAAT variants in all aliquots of the sample, and therefore, a positive result by the participating laboratories can be demanded. A sample was classified as negative for particular target when the two following criteria were fulfilled: (i) all quintupli-cates analyzed by the expert laboratory were negative for that particular target and (ii) all other expert laboratories confirmed the absence of that particular target. Samples that were not validated as positive or negative for a specific target were considered as educational for this particular target, meaning that the samples contained such a low parasite load that it was not consistently detected by all the expert laboratories. Therefore, a positive result cannot be demanded from each participating laboratory. These criteria are commonly applied by variety of EQAS organizations, including but not limited to SKML. The results of this validation were compiled, shared between expert laboratories (seeS3 File) and discussed during a teleconfer-ence call. Based on the results, it was decided to keep all 12 stool and 8 DNA samples in the final HEMQAS panel. The composition of the HEMQAS stool and DNA panel is summarized inTable 3. The 12 stool and 8 DNA samples included targets for five helminth targets (Ascaris, Trichuris, N. americanus, Strongyloides and Schistosoma). Six of the 20 samples were validated
as positive forAscaris, two as positive for Trichuris, seven as positive for N. americanus, five as
positive forSchistosoma and one as positive for Strongyloides. Ten of the 20 samples were
vali-dated negative forAscaris, twelve for Trichuris, sixteen for Strongyloides and fifteen for Schisto-soma. The panel did not include Ancylostoma, and all samples were validated as negative for
this genus. The educational samples included four forAscaris, six for Trichiura, two for N. americanus and three for Strongyloides. For Schistosoma, there were no educational samples.
HEMQAS pilot study
(i) Distribution of the panel. The HEMQAS panels were sent to all participating
labora-tories at ambient temperature using an express courier. For each laboratory, the stool and DNA panel samples were placed in protective sealed envelopes. Per stool and DNA sample, we only sent one aliquot of 250μl and 50 μl, respectively. No quintuplicates were sent in the pilot study. Each package also contained instructions on how to remove the ethanol from stool, prior to commencing DNA extraction procedures (seeS2 File). No information on the out-come of the validation was shared to the participating laboratories.
(ii) Reporting of the test results and completion of the questionnaire. All participating
laboratories were asked to process the panel according to their in-house DNA extraction and NAAT protocols, and to report the results using the QBase submission tool. They were also questioned about the main characteristics of both their DNA extraction procedures and NAAT protocols by means of a questionnaire. Participants were allowed two months to com-plete the analysis, submit their results and comcom-plete the questionnaire. This is longer than what is normally giving during an EQAS by SKML, but the extra time was given to anticipate logistical issues such as packages delayed at customs, and the need of laboratories to become acquainted with the online QBase submission software.
Table 3. Composition of the HEMQAS stool (ST) and the DNA panel. Red cells represent samples that are classified as negative for that particular target; green cells represent samples that are classified as positive for the target; orange cells represent samples that are classified as educational for the target.
Sample ID Ascaris Trichuris Necator
americanus
Ancylostoma Strongyloides Schistosoma ST1 ST2 ST3 ST4 ST5 ST6 ST7 ST8 ST9 ST10 ST11 ST12 DNA1 DNA2 DNA3 DNA4 DNA5 DNA6 DNA7 DNA8 https://doi.org/10.1371/journal.pntd.0008231.t003
(iii) Feedback on the diagnostic performance. After receiving the results from all
partici-pants, results were compiled and reported back to all participants by SKML. In these reports, all participants received a complete anonymized list of all submitted results with only their own laboratory recognizable. An example of such a report is provided inS4 File. For the cur-rent study, the raw data were analyzed as described below.
Statistical analysis
We analyzed the diagnostic performance of the different NAATs across laboratories and tar-gets by assessing the number of false positive (FP) and false negative (FN) test results. Educa-tional samples were not considered in these analyses because interpretation of these results requires more observations. We assessed the variation in DNA extraction and NAAT proto-cols across the different participants. We did not explore associations between DNA extraction or NAAT protocols and diagnostic performance. This is because the pilot study was not designed nor powered to identify possible factors that may explain the variation in diagnostic performance across laboratories. To gain insights into the variation in quantitative test results, we assessed the variation in the reported Cq-values across the different targets and individual samples. The stability of the DNA and stool panel was analyzed by comparing the Cq-values obtained by the expert laboratories during the panel validation with those reported approxi-mately six months later in the HEMQAS pilot study.
Results
Description of the DNA extraction and NAAT protocols
All participating laboratories completed the online questionnaire. A summary of the different DNA extraction and NAAT protocols used by the 15 laboratories is provided inTable 4. All laboratories used a commercial DNA extraction kit. Twelve laboratories applied a procedure in which the helminth eggs were mechanically broken by bead beating, five included a freeze-thaw cycle in the DNA extraction protocol, and two included a protease treatment, all in order to increase DNA yield from the helminth eggs. There was a great deal of variation in the type of beads (n = 11), bead beaters (n = 10) and DNA extraction kits (n = 10) used. Both the vol-ume of stool suspensions used for DNA extraction and the elution volvol-ume differed substan-tially between laboratories, ranging from 50 to 500μl and from 50 to 200 μl, respectively. All laboratories used quantitative PCR (qPCR) as NAAT, but in different formats. Six laboratories used a multiplex qPCR and nine laboratories used different singleplex qPCRs. All qPCR assays used hydrolysis probes to detect amplicons. There was a large diversity in master mixes used, with almost every laboratory using a different master mix. Five of the fifteen laboratories ana-lyzed the panels for the presence of all targets. Five laboratories had NAATs for all targets exceptSchistosoma, two laboratories had all targets except Schistosoma and Strongyloides, two
laboratories only had NAAT available forStrongyloides and Schistosoma, and one laboratory
had qPCR protocols forSchistosoma, Strongyloides and N. americanus. The DNA template
vol-ume used in the NAATs ranged from 1μl to 10 μl and the corresponding stool suspension used per NAAT reaction ranged from 1.5μl to 36.4 μl.
Stability of the targets over time
The mean difference of the Cq-values (± standard deviation) for all stool samples reported as positive by the expert laboratories during the validation of the panel and those reported approximately six months later during the actual HEMQAS pilot study by the same laborato-ries was 1.8± 0.4 for Ascaris, -1.6 ± 1.7 for N. americanus, 0.8 ± 0.9 for Schistosoma and
0.4± 0.6 for Trichuris. However, one stool sample that was validated as positive for Trichuris by the five expert laboratories was found negative by three of these laboratories during the pilot study. For the DNA samples, the mean difference (± standard deviation) was 0.6 ± 0.1 for
Ascaris, 0.5 ± 0.0 for N. americanus, 0.1 ± 0.1 for Schistosoma and 0.0 ± 0.6 for Strongyloides.
Given the minor variation over time of all targets, except for oneTrichuris-positive stool
sam-ple, we concluded that the targets in both the stool and DNA panel were stable over time.
Diagnostic performance
The diagnostic performance assessed by the number of false positives (FPs) and false negatives (FNs) across targets and/or laboratories is shown inTable 5. Of the total of 975 negative tar-gets that were analyzed by the different laboratories, thirteen were FP test results. All of the FP test results were reported by only two of the laboratories (laboratory B and N). Eight of these were FPStrongyloides results reported by laboratory B and three were FP Schistosoma results
reported by laboratory N. Laboratory N also reported a FPN. americanus result. Across the
240 positive targets, there were a total of 30 FN test results. Of these, 23 FN test results were Table 4. Overview of the different DNA extraction and NAAT protocols used in the HEMQAS pilot study. The number between brackets corresponds to the number of laboratories. ITS-1, intergenic transcribed spacer-1; ITS-2, intergenic transcribed spacer-2; 18S rRNA, 18S ribosomal ribonucleic acid; 28S rRNA, 28S ribosomal ribonucleic acid. DNA extraction
Bead beating: Yes (12), No (3)
Type of beads: MagNA Lyser Green Beads (Roche) (1); Garnet beads 0.7 mm (SanBio) (1); Zirconia/silica beads 0.5 mm (Daintree Scientific) (1); Silica beads (1); Included in FastDNA SPIN kit for Soil (MP Biomedicals) (4); Mixture of ceramic and silica particles (1); Included in ISOLATE II Faecal DNA extraction kit (Bioline) (1); Zirconium beads 1.5 mm (Benchmark Scientific) (1); Included in Precellys Lysing Kit (Bertin technologies) (1)
Bead beater: TissueLyser LT (Qiagen) (1); FastPrep (MP Biomedicals) (6); Mini-Beadbeater (Biospec Products) (1); Beadbug (Benchmark Scientific) (1); Precellys 24 Tissue Homogenizer (Bertin technologies) (1); Vortex (2) Proteinase K pretreatment: Yes (4), No (11)
Freeze-thaw cycle: Yes (5), No (10)
DNA extraction kit: MagnaPure (Roche) (1); Isolate II Faecal DNA Kit (Bioline) (1); Maxwell (Promega) (1); Maxwell RSC PureFood GMO and Authentication kit (Promega) (1); FastDNA SPIN Kit for Soil (MP-Biomedicals) (5); QIAamp DNA Mini Kit (Qiagen) (1); DNeasy Blood and Tissue Kit (Qiagen) (2); QIAamp Stool Kit (Qiagen) (1); DNeasy PowerSoil Kit (formerly Powersoil kit sold by MO BIO) (3); QIA Symphony (Qiagen) (1)
NAAT
NAAT technology: qPCR (15)
qPCR format: Singleplex (9), Multiplex (6) Detection: Hydrolysis probes (15)
Master mix: Qiagen (1); Quantitect (Qiagen) (2); HotStarTaq MasterMix (Qiagen) (2); Applied Biosystems (4); VWR (1); Roche (1); Promega (1); Platinum qPCR Supermix-UDG with ROX (Thermofisher) (1); TaqPath ProAmp Master (Thermofisher) (1); Technologies Brilliant II QPCR Master Mix (Agilent) (1)
Stool volume used for DNA extraction (mean, range): 223μl, 50–500 μl Elution volume (mean, range): 109μl, 50–200 μl
DNA used for NAAT (mean, range): 3.2μl, 1–10 μl
Stool volume used per NAAT reaction (mean, range): 8.6μl, 1.5–36.4 μl
Ascaris primer and probe target: ITS-1 (10), non-coding repetitive DNA sequence (2)
Trichuris primer and probe target: 18S rRNA (4), 28S rRNA (1), ITS-1 (1), non-coding repetitive DNA sequence (6)
Necator primer and probe target: ITS-1 (1), ITS-2 (6), non-coding repetitive sequence (5)
Schistosoma primer and probe target: 28S rRNA (1), cytochrome c oxidase (1), non-coding repetitive DNA sequence (1), ITS-2 (4)
Strongyloides primer and probe target: 18S rRNA (8), non-coding repetitive sequence (5)
reported by three laboratories, including laboratory M (n = 16), laboratory F (n = 4) and labo-ratory H (n = 3). The other seven FNs were reported by seven different laboratories. Three lab-oratories reported no FP and no FN test results. The number of educational samples found positive for each target and laboratory are shown inTable 5.
Quantification
All laboratories reported results in relative quantification units, i.e. using a unit that reports the cycle at which the fluorescence from the qPCR reaction exceeds the background fluores-cence. None of the laboratories reported in absolute quantification units.Fig 2illustrates the variation in Cq-values reported across the different laboratories for each target and each sam-ple separately. Overall, there was a large variation in reported Cq values. For examsam-ple, across laboratories, the variation for each of the sevenN. americanus positive samples exceeded ten
Cq-values. The number of FN and FP results per laboratory and target as a function of the reported Cq values are shown inFig 3. Overall, there was no clear link between FP or FN and the Cq-values. Because of the limited number of positive targets forTrichuris and Strongy-loides, plots are not shown for these helminth species.
Discussion
NAATs are increasingly used for the detection and quantification ofSchistosoma and STHs,
includingStrongyloides, but an EQAS, a key component for an overall quality management
Table 5. The performance of 15 laboratories for the detection of six helminths in stool and DNA. The numbers in the second row represent the total number of true negatives (TN), true positives (TP) and educational samples (Ed) per target. The laboratories are shown in the left column. The body of the table shows the number of false-positive (FP), false-negative (FN) and educational (Ed) test results per laboratory and per target. Blue cells indicate FP results, red cells indicate FN results, green cells indicate the absence of FP and FN results, black cells indicate that a NAAT has not been performed by a laboratory. The numbers in the blue, red and white cells indicate the number of FP, FN or Ed results, respectively. The ‘All targets’ column, represents per laboratory across all targets the ratio of the totals of FP, FN and educational posi-tives (EP) over the total number of TN, TP and Ed.
Ascaris Trichuris N. americanus Ancylostoma Strongyloides Schistosoma All targets TN TP Ed TN TP Ed TN TP Ed TN TP Ed TN TP Ed TN TP Ed 10 6 4 12 2 6 11 7 2 20 0 0 16 1 3 15 5 0 Lab FP FN EP FP FN EP FP FN EP FP FN EP FP FN EP FP FN EP FP/TN FN/TP EP/Ed A 3 6 1 3 1 0/84 1/21 13/15 B 8 3 8/31 0/6 3/3 C 2 0/31 0/6 2/3 D 3 6 0 2 0/84 0/21 11/15 E 3 1 0 1 2 0/69 1/16 6/15 F 2 1 3 3 0 2 0/84 4/21 7/15 G 3 1 0 0 2 0/84 1/21 5/15 H 2 2 1 0 1 2 0/69 3/16 5/15 I 3 0 1 1 1/53 0/15 4/12 J 3 1 0 1 2 0/69 1/16 6/15 K 2 1 4 0 0/53 1/15 6/12 L 2 1 0 0 2 0/69 1/16 4/15 M 6 0 2 0 7 0 1 0 0/69 16/16 0/15 N 1 1 0 2 3 4/42 1/13 2/5 O 2 0 0 1 0/84 0/21 3/15 Total 0 8 28 0 9 19 1 11 5 1 0 8 1 25 3 1 13/975 30/240 77/185 https://doi.org/10.1371/journal.pntd.0008231.t005
system, is lacking. To our knowledge, we piloted the first international EQAS for the DNA-based detection of these parasitic worms. In the current study, a total of 15 clinical and research laboratories, representing both non-endemic and endemic countries, participated in this EQAS for NAATs targetingSchistosoma and STH, including Strongyloides.
The HEMQAS panel was stable over the course of the study
For both the DNA and the stool panels, there were only small differences in quantitative results reported among the expert laboratories during the validation of the panel and the pilot study, which was completed approximately 6 months later. These findings demonstrate that both panels were stable over the course of the study, and that discrepancies were not due to degra-dation of DNA during long-term storage. These results correspond with previous reports that showed that eggs from STHs in stool could be well preserved in ethanol over a one-year period for subsequent examination by NAATs [25]. However, for one stool sample, there were dis-crepant results between the validation round and the pilot study. This sample was validated as aTrichuris positive sample, i.e., found positive by all expert laboratories during the validation
of the panel, but found negative by three of the expert laboratories in the pilot study. Further-more, this sample was also found negative by five of the other participating laboratories. Retro-spectively, the amount ofTrichuris DNA in this stool sample was rather low (median reported
Cq-value of 34.8), comparable to that of the educationalTrichuris samples (range median
reported Cq-values 30.8 to 35.4) and probably close to the limit of detection (LOD) of most Fig 2. The variation in the reported Cq-values across the different targets and samples. The Cq-values reported by the different participants for the different targets is shown on the Y-axis. The boxes represent the interquartile range; the lines in the boxes represent the median values; the upper and lower whiskers represent the 95thand the 5th percentile, respectively; and the dots represent outliers. The X-axis represents the different samples of the HEMQAS panel (seeTable 3).
NAATs. In general, the probability of reporting a FN increases when the target DNA concen-tration approaches the LOD, which probably explains these FN results, although the LODs of the different NAATs are unknown. In cases where fewer than 50% of the participating labora-tories report the expected positive result, it is common practice in EQAS organized by SKML to change the status of that target from positive into educational. However, this was not done during current pilot study. Knowledge of the absolute target concentrations of each sample and the LODs of each NAAT would allow for better interpretation of such cases (see also below).
Participation in HEMQAS enables identification of flaws in diagnostic
performance
Overall, the diagnostic performance of most laboratories was good. However, a substantial number of FPs and FNs were reported. The majority of reported FPs in our study involved
Strongyloides reported by a single laboratory. Remarkably, all FP Strongyloides results reported
by this single laboratory wereAscaris positive stool and DNA samples. Hence, one of the
possi-ble explanations is that there was a cross-reaction betweenStrongyloides and Ascaris in the
qPCR protocols of this laboratory, e.g. due to unspecific primers and/or fluorescence cross-over between channels. Another possibility is thatStrongyloides negative samples were
con-taminated during laboratory manipulations and/or non-specific curves were considered as specific DNA amplification, e.g. because of fluorescent crosstalk from other channels. The fact thatStrongyloides FP test results were reported both in the stool and the DNA panel
empha-sizes the need for including both stool and DNA samples in an EQAS for helminths. Indeed, without including DNA samples, the laboratory reporting these FP results could have counter-argued that their FP test results in the stool panel were actually true positive samples that were missed by the expert laboratories due to a difference in DNA extraction protocols. By includ-ing a panel of DNA samples derived from parasites, this possible counterargument can there-fore be rejected. Three FPSchistosoma test results were reported by a single laboratory and
included both stool and DNA samples (the FP reported DNA samples were a true positiveN. americanus and Strongyloides sample). These were likely due to contamination during
labora-tory manipulations or cross-reactivity with other targets.
Over half of the FN test results were reported by a single laboratory (laboratory M), which did not report a single positive result for any of the targets in the pilot EQAS. The reasons for this remain unclear, as we validated the stability of the panel and this laboratory confirmed the use of positive and negative controls.
Although educational samples are more challenging to interpret, additional information on diagnostic performance can be obtained from them. For example, the second educational
Strongyloides DNA sample (DNA2, seeTable 3) which was a hundred-fold dilution of the pos-itiveStrongyloides DNA sample (DNA1, seeTable 3) was validated as educational because it tested positive in only 4 of 5 expert laboratories. Nevertheless, in the pilot study, it was tested positive in the majority of participating laboratories with a median Cq-value of 34.6, indicating that this was probably close to the LOD of most assays used. One of the educational
Fig 3. The variation in the reported Cq-values across the different laboratories and targets. Each dot represents the difference in Cq-value for one sample reported by one laboratory in comparison with the median Cq-values reported by the expert laboratories for that sample. These differences are shown on the Y-axis, with negative values meaning lower Cq-values compared to the median Cq-values reported across the experts, and positive values meaning higher Cq-values. The laboratories, represented by letters, are shown on the X-axis. The number of negative and false-positive results reported by each laboratory are shown by means of red stars and blue triangles, respectively. Laboratory M was omitted from this analysis because no positive results were reported by this laboratory. https://doi.org/10.1371/journal.pntd.0008231.g003
Strongyloides stool samples tested positive by the majority of laboratories as well. The inclusion
of more of these challenging samples and more participating laboratories in future HEMQAS will reveal further details about the association between used test protocols and test outcome.
The lack of information on both the absolute quantity of targets in the
HEMQAS panel and LOD of NAATs leads to ambiguous interpretations
Besides above-mentioned FP and FN test results, FN test results were reported forSchistosoma(n = 1),Ascaris (n = 2) and N. americanus (n = 3). It remains for the laboratories to interpret
these results, but this currently remains a challenge. These FNs might be explained by less effi-cient DNA extraction procedures and/or a poorly performing NAAT. Knowledge of both the absolute target concentrations in the panel and of the LOD of the NAAT would allow for better interpretation of such cases, as well as the interpretation of the educational samples. In other words, it would allow to decide that a FN is missed because of the LOD of a NAAT or because of another error. However, this strategy is currently complicated by the absence of a unit that expresses qPCR results as absolute DNA concentration of the targets and that allows these absolute concentrations to be compared if results are obtained from different assays/laborato-ries [16,17]. Such a unit is required to compare LODs and quantitative HEMQAS values among laboratories [17].
The high diversity in DNA extraction and NAAT protocols calls for
standardization
The current study highlights the diversity of both DNA extraction and the NAAT protocols used for the detection and quantification of helminths in stool. Due to the design of the pilot study we did not explore differences in protocols that may explain the variation in diagnostic performance across laboratories. For this, a larger number of participating laboratories and samples will be needed to address these issues. In addition, given the plethora of protocols, more standardization in both DNA extraction and NAAT protocols should be encouraged. However, it is unlikely that laboratories will completely relinquish their in-house protocols, which required important investments to develop. A level of standardization that is feasible and increases overall diagnostic performance will include (i) methodological steps that have been proven to improve the performance of NAATs, e.g. by improving DNA extraction effi-ciency through mechanical lysis of samples (bead beating) [25,26], (ii) the use of positive, nega-tive and inhibition controls, (iii) regular participation in an EQAS, (iv) assessment of the LOD of the NAATs and (v) reporting the quantity of DNA in absolute units.
Challenges for the use of NAAT in a programmatic setting
In clinical laboratories, qualitative NAAT test results for helminth infections, which indicate presence or absence of infection in an individual patient, are sufficient to make decisions on the initiation or success of therapy. However, there are still important obstacles that impede implementation of NAATs in large-scale deworming programs [16,17]. In these programs, both prevalence and intensity of infections at the population level are important indicators that guide decision making [27]. This information is currently provided by assessing the fecal egg counts by means of microscopic methods, of which the Kato-Katz is the most widely used. The intensity of infection is often further classified as low, moderate or heavy [28]. NAATs have been considered and suggested as potential alternatives to these microscopic tools, due to their ability to detect very low levels of egg excretion [17]. However, the public health commu-nity is currently not ready to use NAATs in control programs. The obstacles that are currently
hampering the use of qPCR are low throughput, high cost, lack of standardization and the absence of a quality assurance scheme. Furthermore, qPCR outcomes will need to be readily translated to program decision algorithms.
Limitations and recommendations for future HEMQAS
Although they also cause significant health burdens in some areas, we did not include targets such asAncylostoma (including A. duodenale and the zoonotic A. ceylanicum) and other Schis-tosoma species such as S. haematobium and S. japonicum, although these helminths also cause
significant health burdens in some areas [2,8]. Future HEMQAS rounds should include these species. Moreover, an important improvement would be the inclusion ofTrichuris DNA in the
DNA panel to better understand the somewhat poor results in our study for this target in stool samples. In the present study, clinical samples were obtained from a single site (Jimma, Ethio-pia). In order to capture (possible) more diverse species and strains, samples from geographi-cally more dispersed regions should be included in future HEMQAS. To gain more insights into discrepancies in diagnostic performance across laboratories, it would also be informative to ask participating laboratories to develop a standard curve on a common target DNA mate-rial. From 2019 onwards, SKML provides a HEMQAS open to laboratories from all over the world.
Conclusion
We showed the technical feasibility of preparing homogeneous and stable stool and DNA sam-ples for an international EQAS for the NAAT ofSchistosoma and STHs, including Strongy-loides. We documented a clear benefit of participating in a HEMQAS, as laboratories were able
to identify weaknesses and/or to confirm the performance of their NAATs. In contrast to other organizations providing panels for the molecular detection of pathogens in an EQAS, which commonly distribute only purified DNA in an artificial matrix, we demonstrated the necessity of sending both stool and DNA samples, enabling laboratories to evaluate the entire laboratory procedure including the important DNA extraction step. Furthermore, we showed a high technological diversity in both DNA extraction and NAATs protocols, which under-scores the urgent need for such an EQAS. HEMQAS should be implemented as an integral part of the overall standardization and quality management in the field of molecular detection of helminths. There is a need for a minimal set of recommendations to improve both the diag-nostic performance of protocols, including but not limited to the assessment of the LOD, reporting DNA concentration in absolute and universal units, and participating in an EQAS. Further research should aim to identify factors that explain poor performance of NAATs and how the NAATs can best be implemented in large-scale deworming programs.
Supporting information
S1 File. Summary manual and extended manual of the QBase submission tool.
(PDF)
S2 File. Instructions accompanying the panel on how to remove ethanol from the stool samples.
(PDF)
S3 File. The target validation of the stool and DNA panel by expert laboratories.
S4 File. Example of a HEMQAS report.
(PDF)
Acknowledgments
The authors would like to express their gratitude to Eric Brienen, Allison Elliott, Meredith Lane, Stacey Llewelyn, Manuel Malathi, Noorizan Miswan, Rahmah Noordin, Nils Pilotte, Gongyi Ren and Nicolette van der Ham for their contributions in preparing and analyzing the HEMQAS samples.
Disclaimer
The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the Agency for Toxic Substances and Disease Registry and/or the U.S. Department of Health and Human Services.
Author Contributions
Conceptualization: Piet Cools, Lisette van Lieshout, Rob Koelewijn, Sitara S. R. Ajjampur,
Rubina Imtiaz, Zeleke Mekonnen, Jozef Vercruysse, Johnny Vlaminck, Jaco J. Verweij, Jaap J. van Hellemond, Bruno Levecke.
Data curation: Piet Cools, Rob Koelewijn, Jaap J. van Hellemond, Bruno Levecke. Formal analysis: Piet Cools, Lisette van Lieshout, Rob Koelewijn, Jaap J. van Hellemond,
Bruno Levecke.
Funding acquisition: David Addiss, Rubina Imtiaz, Jaap J. van Hellemond, Bruno Levecke. Investigation: Lisette van Lieshout, Rob Koelewijn, Sitara S. R. Ajjampur, Mio Ayana, Richard
S. Bradbury, Jason L. Cantera, Daniel Dana, Kerstin Fischer, Joyce Kabagenyi, James Lok, James McCarthy, Rojelio Mejia, Zeleke Mekonnen, Sammy M. Njenga, Nurulhasanah Oth-man, Hongguang Shao, Rebecca Traub, Marjan Van Esbroeck, Steven A. Williams, Jaco J. Verweij, Jaap J. van Hellemond.
Methodology: Lisette van Lieshout, Rob Koelewijn, Sitara S. R. Ajjampur, Mio Ayana, Richard
S. Bradbury, Jason L. Cantera, Daniel Dana, Kerstin Fischer, Joyce Kabagenyi, James McCarthy, Rojelio Mejia, Zeleke Mekonnen, Sammy M. Njenga, Nurulhasanah Othman, Hongguang Shao, Rebecca Traub, Marjan Van Esbroeck, Steven A. Williams, Jaco J. Ver-weij, Jaap J. van Hellemond.
Project administration: Piet Cools, Rob Koelewijn, Jaap J. van Hellemond, Bruno Levecke. Resources: Lisette van Lieshout, Mio Ayana, Daniel Dana, James Lok, Zeleke Mekonnen,
Hongguang Shao.
Software: Rob Koelewijn, Jaap J. van Hellemond.
Supervision: Piet Cools, Jaap J. van Hellemond, Bruno Levecke.
Validation: Lisette van Lieshout, Rob Koelewijn, Jaco J. Verweij, Jaap J. van Hellemond,
Bruno Levecke.
Visualization: Piet Cools, Lisette van Lieshout, Rob Koelewijn, Jaap J. van Hellemond, Bruno
Levecke.
Writing – review & editing: Piet Cools, Lisette van Lieshout, Rob Koelewijn, David Addiss,
Sitara S. R. Ajjampur, Jason L. Cantera, Daniel Dana, Kerstin Fischer, Rubina Imtiaz, James Lok, James McCarthy, Rojelio Mejia, Zeleke Mekonnen, Sammy M. Njenga, Nurulhasanah Othman, Hongguang Shao, Rebecca Traub, Marjan Van Esbroeck, Jozef Vercruysse, Johnny Vlaminck, Steven A. Williams, Jaco J. Verweij, Jaap J. van Hellemond, Bruno Levecke.
References
1. Kyu HH, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national dis-ability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018; 392(10159):1859–922.https://doi.org/10.1016/S0140-6736(18)32335-3PMID:
30415748
2. Colley DG, Bustinduy AL, Secor WE, King CH. Human schistosomiasis. Lancet. 2014; 383 (9936):2253–64.https://doi.org/10.1016/S0140-6736(13)61949-2PMID:24698483
3. Hay SI, Abajobir AA, Abate KH, Abbafati C, Abbas KM, Abd-Allah F, et al. Collaborators H. Global, regional, and national disability-adjusted life-years (DALYs) for 333 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2017; 390(10100):1260–344.https://doi.org/10.1016/ S0140-6736(17)32130-XPMID:28919118
4. Bisoffi Z, Buonfrate D, Montresor A, Requena-Mendez A, Munoz J, Krolewiecki AJ, et al. Strongyloides stercoralis: a plea for action. PLoS Negl Trop Dis. 2013; 7(5):e2214.https://doi.org/10.1371/journal. pntd.0002214PMID:23675546
5. McKenna ML, McAtee S, Bryan PE, Jeun R, Ward T, Kraus J, et al. Human Intestinal Parasite Burden and Poor Sanitation in Rural Alabama. Am J Trop Med Hyg. 2017; 97(5):1623–8.https://doi.org/10. 4269/ajtmh.17-0396PMID:29016326
6. Strkolcova G, Goldova M, Bockova E, Mojzisova J. The roundworm Strongyloides stercoralis in chil-dren, dogs, and soil inside and outside a segregated settlement in Eastern Slovakia: frequent but hardly detectable parasite. Parasitol Res. 2017; 116(3):891–900.https://doi.org/10.1007/s00436-016-5362-1
PMID:28074315
7. Beknazarova M, Whiley H, Judd JA, Shield J, Page W, Miller A, et al. Argument for Inclusion of Strongy-loidiasis in the Australian National Notifiable Disease List. Trop Med Infect Dis. 2018; 3(2).
8. Smout FA, Skerratt LF, Butler JRA, Johnson CN, Congdon BC, Thompson RCA. The hookworm Ancy-lostoma ceylanicum: An emerging public health risk in Australian tropical rainforests and Indigenous communities. One Health. 2017; 3:66–9.https://doi.org/10.1016/j.onehlt.2017.04.002PMID:28616506
9. Barrett J, Warrell CE, Macpherson L, Watson J, Lowe P, Armstrong M, et al. The changing aetiology of eosinophilia in migrants and returning travellers in the Hospital for Tropical Diseases, London 2002– 2015: An observational study. J Infect. 2017; 75(4):301–8.https://doi.org/10.1016/j.jinf.2017.08.007
PMID:28842188
10. Asundi A, Beliavsky A, Liu XJ, Akaberi A, Schwarzer G, Bisoffi Z, et al. Prevalence of strongyloidiasis and schistosomiasis among migrants: a systematic review and meta-analysis. Lancet Glob Health. 2019; 7(2):e236–e48.https://doi.org/10.1016/S2214-109X(18)30490-XPMID:30683241
11. Verweij JJ, Stensvold CR. Molecular testing for clinical diagnosis and epidemiological investigations of intestinal parasitic infections. Clin Microbiol Rev. 2014; 27(2):371–418.https://doi.org/10.1128/CMR. 00122-13PMID:24696439
12. van Lieshout L, Roestenberg M. Clinical consequences of new diagnostic tools for intestinal parasites. Clin Microbiol Infect. 2015; 21(6):520–8.https://doi.org/10.1016/j.cmi.2015.03.015PMID:25843505
13. O’Connell EM, Nutman TB. Molecular Diagnostics for Soil-Transmitted Helminths. Am J Trop Med Hyg. 2016; 95(3):508–13.https://doi.org/10.4269/ajtmh.16-0266PMID:27481053
14. Llewellyn S, Inpankaew T, Nery SV, Gray DJ, Verweij JJ, Clements AC, et al. Application of a Multiplex Quantitative PCR to Assess Prevalence and Intensity Of Intestinal Parasite Infections in a Controlled Clinical Trial. PLoS Negl Trop Dis. 2016; 10(1):e0004380.https://doi.org/10.1371/journal.pntd.0004380
PMID:26820626
15. Asbjornsdottir KH, Ajjampur SSR, Anderson RM, Bailey R, Gardiner I, Halliday KE, et al. Assessing the feasibility of interrupting the transmission of soil-transmitted helminths through mass drug administra-tion: The DeWorm3 cluster randomized trial protocol. PLoS Negl Trop Dis. 2018; 12(1):e0006166.
16. Papaiakovou M, Gasser RB, Littlewood DTJ. Quantitative PCR-Based Diagnosis of Soil-Transmitted Helminth Infections: Faecal or Fickle? Trends Parasitol. 2019.
17. Cools P, Vlaminck J, Albonico M, Ame S, Ayana M, Jose Antonio BP, et al. Diagnostic performance of a single and duplicate Kato-Katz, Mini-FLOTAC, FECPAKG2 and qPCR for the detection and quantifica-tion of soil-transmitted helminths in three endemic countries. PLoS Negl Trop Dis. 2019; 13(8): e0007446.https://doi.org/10.1371/journal.pntd.0007446PMID:31369558
18. WHO. WHO manual for organizing a national external quality assessment programme for health labora-tories and other testing sites. 2016.
19. Schuurs TA, Koelewijn R, Brienen EAT, Kortbeek T, Mank TG, Mulder B, et al. Harmonization of PCR-based detection of intestinal pathogens: experiences from the Dutch external quality assessment scheme on molecular diagnosis of protozoa in stool samples. Clin Chem Lab Med. 2018; 56(10):1722– 7.https://doi.org/10.1515/cclm-2017-1057PMID:29451859
20. Murphy SC, Hermsen CC, Douglas AD, Edwards NJ, Petersen I, Fahle GA, et al. External quality assur-ance of malaria nucleic acid testing for clinical trials and eradication surveillassur-ance. PLoS One. 2014; 9(5): e97398.https://doi.org/10.1371/journal.pone.0097398PMID:24838112
21. Domingo C, Escadafal C, Rumer L, Mendez JA, Garcia P, Sall AA, et al. First international external quality assessment study on molecular and serological methods for yellow fever diagnosis. PLoS One. 2012; 7(5):e36291.https://doi.org/10.1371/journal.pone.0036291PMID:22570700
22. Donoso Mantke O, McCulloch E, Wallace PS, Yue C, Baylis SA, Niedrig M. External Quality Assess-ment (EQA) for Molecular Diagnostics of Zika Virus: Experiences from an International EQA Pro-gramme, 2016(-)2018. Viruses. 2018; 10(9).
23. Vlaminck J, Cools P, Albonico M, Ame S, Ayana M, Bethony J, et al. Comprehensive evaluation of stool-based diagnostic methods and benzimidazole resistance markers to assess drug efficacy and detect the emergence of anthelmintic resistance: A Starworms study protocol. PLoS Negl Trop Dis. 2018; 12(11):e0006912.https://doi.org/10.1371/journal.pntd.0006912PMID:30388108
24. Forootan A, Sjoback R, Bjorkman J, Sjogreen B, Linz L, Kubista M. Methods to determine limit of detec-tion and limit of quantificadetec-tion in quantitative real-time PCR (qPCR). Biomol Detect Quantif. 2017; 12:1– 6.https://doi.org/10.1016/j.bdq.2017.04.001PMID:28702366
25. Ayana M, Cools P, Mekonnen Z, Biruksew A, Dana D, Rashwan N, Prichard R, Vlaminck J, Verweij J, Levecke B. Comparison of four DNA extraction and three preservation protocols for the molecular detection and quantification of soil-transmitted helminths in stool. PLoS Negl Trop Dis. 2019; 13(10): e0007778.https://doi.org/10.1371/journal.pntd.0007778PMID:31658264
26. Kaisar MMM, Brienen EAT, Djuardi Y, Sartono E, Yazdanbakhsh M, Verweij JJ, et al. Improved diagno-sis of Trichuris trichiura by using a bead-beating procedure on ethanol preserved stool samples prior to DNA isolation and the performance of multiplex real-time PCR for intestinal parasites. Parasitology. 2017; 144(7):965–74.https://doi.org/10.1017/S0031182017000129PMID:28290266
27. Lim MD, Brooker SJ, Belizario VY Jr., Gay-Andrieu F, Gilleard J, Levecke B, et al. Diagnostic tools for soil-transmitted helminths control and elimination programs: A pathway for diagnostic product develop-ment. PLoS Negl Trop Dis. 2018; 12(3):e0006213.https://doi.org/10.1371/journal.pntd.0006213PMID:
29494581
28. World Health Organization. Guideline: Preventive chemotherapy to control soil-transmitted helminth infections in at-risk population groups. Geneva; 2017.
