Standardization in fetal growth restriction
Beune, Irene
DOI:
10.33612/diss.156487314
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2021
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Beune, I. (2021). Standardization in fetal growth restriction: Progression by consensus. University of Groningen. https://doi.org/10.33612/diss.156487314
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Chapter
10
Main findings of this thesis
This thesis aids in standardization of research and health care in fetal growth restriction (FGR). Heterogeneity in the definition of the condition itself, of the reporting of the study population and the reporting of outcomes hampers the comparison of cohorts and thereby the improvement of healthcare. The studies described in this thesis aim to overcome these issues.
Part 1: What are the effects of fetal growth restriction?
Chapter 2, reviews the research questions on fetal and neonatal mortality, morbidity and long-term (neuro-)development of pregnancies complicated by early-onset FGR (diagnosed before 32 weeks of gestation). The overall survival rate from published studies was 81%, neurodevelopmental impairment was assessed in only a minority of surviving children. The quality of evidence was generally rated as very low to moderate, except for 3 large well-designed randomized controlled trials. Between the included patient groups in these RCT’s however, FGR was defined differently and also patient characteristics and pregnancy characteristics differed. For most outcomes one large well-designed RCT(1) provides high quality of evidence on the mortality and morbidity outcomes and neurodevelopmental out-comes at two years of age(2). Because of the heterogeneity in defining FGR and in patient- and pregnancy characteristics, the results of the included studies are difficult to compare. Therefore, it remains unknown what the risk percentages are for a significant proportion of possible sequelae associated with FGR (for example neuro- and motor development at school age). This also hampers the counseling in couples with a pregnancy complicated by FGR.
Part 2: Why is there a need for standardization in fetal growth restriction studies and medical care?
Standardization
Standardization of the definition for FGR, the reporting on study population variables and the reporting of outcomes is essential for improvement of all aspects in future management of FGR as described in Chapter 3.
In the process of choosing between different treatments for a condition, patients, the public, health professionals and other stakeholders need evidence. To be able to set up a study, to analyze data and to compare study results and thus being able to choose the best treatment for their particular patient, there has to be consensus on the basic principles. This starts with how to define the condition. In this thesis, a standardization exercise for FGR was performed. Previously, there was no consensus on how to define FGR which led to
10
a wide spectrum of used definitions, often even defined by essentially an outcome of the condition: birth weight (after all, birthweight is measured once the child is born and the fetal period is over). Thus, until now it was difficult to compare study results of FGR cohorts, as each cohort included different definitions for their study-populations. For example, for FGR during pregnancy, frequently used definitions were estimated fetal weight (EFW) <10th percentile and EFW <10th percentile combined with abnormal Doppler in the umbilical artery. These population subsets have some overlap, but they are principally different and have different expected outcomes beforehand. Furthermore, to be able to compare study results and pool these results, there has to be similarity in the measured outcomes and these outcomes have to be relevant and fundamental to the condition. Outcome reporting bias is a frequent problem in randomized controlled trials in FGR as well as in other condi-tions. This is defined as the results-based selection for publication of a subset of the original measured outcome variables, and affects the conclusions in a substantial proportion of Cochrane reviews.(3) For example, the most accessed and the top cited Cochrane Reviews in 2009 all describe problems due to inconsistencies in the outcomes reported in trials.(4) These issues are addressed by developing a core outcome set (COS).
Present heterogeneity
As described in Chapter 4, there is substantial heterogeneity in defining FGR in existing literature. Over the years 1994, 2004 and 2014, arbitrarily chosen, 31, 33 and 44 different definitions for FGR were reported respectively. Historically, FGR was used interchangeably with very low birthweight. It used to be defined as an exact weight (below 1500 grams for example), not differentiating between preterm or growth restricted babies. This definition does make sense because both preterm and small babies have the same risks and the lower the weight the higher chance of problems. The preferred definition in our scoping review among the decades from 1994 onwards was still a definition that missed the opportunity to properly monitor the fetus during pregnancy and make a just decision on the optimal timing for delivery: birth weight (measured after birth by definition) below the 10th percen-tile. Over time, more different definitions for FGR with antenatal variables were used. The heterogeneity in definitions underlines the need for standardization in FGR studies, both in terms of definitions as well as in outcomes.
Part 3: Is there a possibility to come to consensus how to diagnose fetal growth restric-tion antenatal, postnatal and after fetal demise?
In the absence of a gold standard, in Chapter 5, 6, 7 & 8 consensus definitions for FGR were developed for early and late FGR in singleton pregnancies, (s)FGR in monochorionic and dichorionic twin pregnancies, for growth restriction in the newborn and for FGR in intra-uterine fetal death. In the procedure of twin-pregnancies, the most important vari-ables for management and monitoring were also defined. For these procedures, the Delphi methodology was used.
The aim of the consensus definitions was, next to standardization in defining FGR, to be able to better identify fetuses and neonates at risk for poor outcome. By developing a defi-nition that more closely follows the pathological process, interventions including increased surveillance or timely birth can be better directed at the fetuses who are likely to benefit from it. Moreover, fetuses who have a lower risk profile can be monitored less frequently and receive less (unnecessary) interventions.
Another aim was to be able to better compare true FGR cohorts with true appropriately grown cohorts, independent of size.
Early FGR singleton pregnancy: Late FGR singleton pregnancy:
GA <32 weeks, in absence of congenital anomalies
GA ≥32 weeks, in absence of congenital anomalies
AC/EFW <3rd percentile or UA-AEDF AC/EFW <3rd percentile
Or Or at least 2 out of 3 of the following
1. AC/EFW <10th percentile combined with 1. AC/EFW <10th percentile
2. UtA-PI >95th percentile and/or 2. AC/EFW crossing percentiles >2 quartiles 3. UA-PI >95th percentile 3. CPR < 5th centile or UA-PI >95th percentile
FGR in dichorionic twins: FGR in monochorionic twins:
EFW <3rd percentile EFW <3rd percentile
Or at least 2 out of 3 of the following: Or at least 2 out of 4 of the following:
1. EFW <10th percentile 1. EFW <10th percentile
2. EFW discordance >=25% 2. EFW discordance >=25%
3. UA PI >95th percentile 3. UA PI >95th percentile
4. AC <10th percentile
10
FGR in the newborn FGR in intra-uterine fetal death
Birth weight <3rd percentile Evident antenatal clinical diagnosis of FGR/ Birth weight <3rd percentile
Or at least 3 out of 5 of the following: Or at least 5 out of 10 of the following:
1. Birth weight <10th percentile 1. Birth weight <10th percentile
2. HC <10th percentile 2. Body weight at time of autopsy <10th
percen-tile
3. Length <10th percentile 3. Brain weight <10th percentile
4. Prenatal diagnosis of fetal growth restriction 4. Foot length <10th percentile 5. Maternal pregnancy information
(eg, pre-eclampsia)
5. Liver weight <10th percentile 6. Placental weight <10th percentile 7. Brain weight to liver weight ratio >4
8. Placental weight to birth weight ratio >90th percentile
9. Risk factors in the clinical antenatal history including scan results and Doppler studies sug-gestive for FGR
10. Histological or gross placental features of placental insufficiency / vascular malperfusion
*Growth centiles are non-customized centiles in FGR in singleton or twin pregnancy and are either population based or customized centiles in FGR in the newborn and in intra-uterine fetal death. AC fetal abdominal circumference; AEDF absent end-diastolic flow; CPR cerebroplacental ratio; EFW estimated fetal weight; GA gestational age; PI pulsatility index; UA umbilical artery; UtA uterine artery; HC head circumference
Changes in defining FGR
In all definitions, the consensus exists of biometric variables as well as functional variables and the historically used definition of FGR only defined as small for gestational age (SGA) was abandoned. The PORTO-study team emphasized the need for functional variables in defining FGR in singleton pregnancies. In this study, abnormal Doppler velocimetry in the umbilical artery (threshold not specified) was identified as a factor in the diagnosis of FGR by questioning 200 obstetricians.(5) The lower cut-off for absolute size measurements in all definitions as solitary variables, reflect the fact that even in the absence of abnormal func-tional parameters, long-term outcomes for severe SGA fetuses/neonates are unfavorable and are therefore more likely to be growth restricted.(6)
A systematic review showed that an abnormal cerebro/placental ratio (CPR) as included in the definition of late FGR in singleton pregnancy, and middle cerebral artery (MCA) Dop-pler assessment, separately, can add value to UA DopDop-pler assessment in the prediction of adverse perinatal outcome in women with a singleton pregnancy. However, there were not enough data to support the theory that the CPR or MCA Doppler is of particular clinical val-ue in late FGR.(7) From this follows the observation that research does not stop when con-sensus is reached but that it can set the stage for future research, and the use of concon-sensus definitions should be evaluated prospectively.
Delphi methodology; strengths and weaknesses
The Delphi methodology is widely used to reach consensus on definitions and in the de-velopment of core outcome sets and is part of the COMET (Core Outcome Measures in Effectiveness Trials) methodology. The Delphi method consist of iterative rounds of voting with anonymized feedback of results of previous rounds on group level to converge to consensus.(8) The feedback of results is reported at group level, with guaranteed anonymi-ty of subjects’ individual responses to others. The equal weighing of votes and anonymous approach minimizes peer pressure from authoritative individuals.(9-11)
The Delphi process encourages panel members to (re)consider their opinion in response to the group answers in every subsequent round. Panel members are more likely to change their opinion in line with the group’s majority of votes in an on-line Delphi procedure than in a face-to-face meeting. The fact that panel members are more tended to change their opinion is both an advantage and a risk of bias.(10) The advantage is that it is more likely that consensus is reached and the anonymous votes ensure that the fear of loss of face is not an argument to adhere to the first opinion. On the other hand, it is undesirable if the experts are too easily inclined to change their opinion, because that could lead to a consen-sus that does not reflect a true underlying opinion. For example, in the semi-anonymous panel, a participant can still experience peer pressure in the knowledge that the fellow panel-members include authorities in the field.
10
The strength of the Delphi procedure is highly influenced by the selection of genuine ex-perts in the field.(9) This ensures that the collective knowledge of the expert panel is of the best available quality. Also, the uptake of the outcome will be better if the included experts are generally accepted as authorities on the subject. When the expert panel is composed, there is a risk of bias if only experts are selected that share the same opinions and interests on beforehand.
The Delphi procedure drains from the collective knowledge that is available among the experts. In that sense, it is a distillation of the ‘wisdom of the crowd’ (with the crowd being the selected experts). However, it is important that the experts provide their particular individual opinion on the topic. Although these interpretations are more likely to be close to the ‘truth’ than the interpretations of non-experts, it remains important to be sensitive to the fact that experts also live within a contemporary paradigm with, hopefully, knowl-edge based on the available evidence at that time, and as such the outcomes of consensus procedures may also be the ‘foolishness of the crowd’ (selected experts). As an example, if we would have performed a Delphi procedure on headache in the year 1779, it may be very likely that cupping or bloodletting would have been designated the optimal management strategy, however with the increased understanding on the subject, in the present this management strategy would be voted out immediately (or would not even be suggested as an option at all).
In a Delphi procedure, it is only possible to use the collective knowledge optimally if the experts are critical and engaged. This commitment is also based on their interest with the question being examined. Moreover, if the experts gain profit directly by the decision to be made, they are more likely to be committed in the Delphi process.(10)
The questions or statements should be written carefully by the study group to proactively avoid any potential misinterpretation. The consequence of unclear procedures is that extra rounds may become necessary if it becomes apparent that participants interpreted the questions differently.
Furthermore, to be able to extrapolate the results as genuine results of the expert panel, drop-out should be minimized. A response rate of 80% for each stakeholder group in each Delphi round is advised by the COMET-initiative.(12) A higher number of rounds increases the total drop-out rate.
A pitfall of Delphi procedures is that fatigue to the procedure can occur if the process is too long.(11) The ideal number of rounds is two to four.(10, 12) This gives opportunity to reach a level of consensus without losing the attention of panel members.
In the original Delphi method, the first round start with open questions to investigate all possible answers to the research question to avoid selection bias by the researches.(13) Typically however, first a review of literature is conducted to give input to the first round.
Delphi procedures in this thesis
In all Delphi procedures, we were able to include a high level of expertise in the expert-panels, with 68% and 75% of the panel described themselves as professors or associate professors in the FGR- and GRN-procedure respectively. In the FGR in IUFD procedure 96% of the panel were pathologists specialized in the field of perinatology. And, in the (s)FGR in twin pregnancy procedure 70% worked in a clinic that performed fetoscopic laser photoco-agulation.
In all procedures, there was an aim for global participation, to ensure that the new con-sensus definitions could be globally applicable. However, in the final panels there was an under-representation of members participating from Africa and South America. This also reflects the geographical distribution of research reports on the topic, since the experts were often approached because of them being author of a paper on the topic.
The procedures were in general longer than previously anticipated with a maximum of 6 rounds in the procedure of FGR in IUFD. However, the panel members showed to be com-mitted and engaged, as the lowest overall response rate was 71%. Also in all procedures, the panel members were keen to give input into the procedures. In all procedures, free-text comments were asked to expand on the panel member’s answers, and in all procedures new variables were advised and comments were provided.
The relevance of the consensus definitions in research and clinical practice is reflected by the great number of citations and that the definition for FGR is the most frequently cited article of the past years in the Journal of Ultrasound in Obstetrics and Gynecology therefore awarded the Stuart Campbell award lecture at the ISUOG conference in Berlin 2019.
Part 4: How to improve the methodology in consensus procedures?
To ensure standardization and to aid in comparison there has to be consensus on the core outcomes. Core outcomes should, as the name implies, be fundamental to the condition. A core outcome set (COS) is the minimum list of the most important outcomes that every clinical trial on that specific topic should report. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) group (http://www.gradeworkinggroup.org) recognize the need to identify a relevant set of core outcomes, and recommends that up to seven patient-important outcomes are listed in the ‘Summary of Findings’ tables in sys-tematic reviews. This is supported by Cochrane Reviews of the effects of healthcare inter-ventions(14, 15) and by the World Health Organization (WHO) in developing guideline rec-ommendations (WHO guideline for handbook). The COMET initiative has issued a number of documents to aid in the methodology for developing a core outcome set (COS).(12) The final step of this methodology is a face-to-face consensus meeting, of representatives of the
10
stakeholder groups, in which the prioritized list of outcomes is discussed and is reached on the final COS. It is assumed that the representatives of the stakeholder groups validly make a valid selection of the outcomes, without losing the strength of the much larger initial pan-el. This assumption has not been tested before.
In Chapter 9, in two COS procedures, an additional consultation round showed that the on-line expert panel agreed with the participants of the face-to-face consensus meeting. This new underlying evidence further progressed the existing COS methodology.
Future work
Outcomes of Fetal growth restriction
The ultimate goal of future FGR studies should be to allow individualized risk assessments, management and counseling of couples with a pregnancy complicated by FGR. Therefore, the knowledge of prognostic characteristics on adverse outcomes has to be improved. We have set and standardized the basics for a better quality of studies on FGR with the pos-sibility of data synthesis and data comparison. An individual patient data (IPD) systematic review is currently performed to combine the individual data of outcomes of pregnancies complicated by early FGR from several trials.
Definitions
A Delphi procedure is a method to reach opinion-based consensus on a specific topic. Therefore, it is not the absolute “truth” and inferior to empirical evidence. It is a dynamic process and in the future new insights and evidence will arise. In that case, the procedure could be repeated to establish consensus again. The consensus based definitions that are found in this thesis should be tested for their discriminating ability for adverse outcomes in comparison to currently used definitions.
In the definition of late FGR in singleton pregnancy, the CPR is included as a contributo-ry variable (more than one abnormal variable is needed to come to the diagnosis) and is currently tested in a Dutch multicenter randomized controlled trial: the DRIGITAT (Doppler Ratio In Growth restriction Intervention Trial At (near) Term) study. In this nested RCT study, women at 32-37 weeks of gestation with SGA without Doppler abnormalities are recruited for the cohort study and prospectively followed. If the CPR becomes abnormal between 34-37 weeks of gestation the women are randomized between induction of labour either at 34 weeks of gestation (with an estimated fetal weight (EFW) or abdominal circumference (AC) below p3) or at 36-37 weeks of gestation (EFW or AC p3-p10) of gestation and standard management (induction of labour at 38 weeks of gestation). The prospectively followed group with normal Dopplers can remain pregnant with regular fetal monitoring until 40
ment in late FGR, and will help in validation of the consensus definition.
In the Delphi procedure of FGR in singleton or twin pregnancy all proposed biomarkers were excluded by the expert panel. Currently there is no known biomarker that has a high predictive accuracy for FGR.(16) However, with ongoing research and development in biomarkers, in the future these could be more important and potentially used in prediction and defining of FGR.
In the procedure for FGR in twin pregnancy there was little difference in the definition for monochorionic and dichorionic pregnancy with only the AC <10th centile that was extra included as a contributory variable for monochorionic pregnancy. Therefore, in the mono-chorionic twins probably more fetuses are diagnosed with (s)FGR. The two definitions can be tested and validated in a future trial for poor outcomes both in dichorionic and mono-chorionic pregnancy and it can be evaluated if the two different twin pregnancies need a different definition for FGR.
The consensus definition for growth restriction in the newborn has two major differen ces with the antenatal definitions. Firstly, it adhered more than the other definitions to the 10th percentile threshold of biometric variables. This is confusing for both caregivers and patients, because a fetus that is considered SGA and not FGR based on the antenatal definition, can be diagnosed as growth restricted after birth based on the same biometric deviations measured during pregnancy. Secondly, in contrary to the antenatal definition of FGR, customized percentiles were included into the definition next to population based percentiles. The importance of use of the 10th percentile selected for biometric variables (apart from birth weight less than the third percentile as a solitary variable) and the use of customized percentile charts in the newborn should be tested for their discriminating value to predict poor outcome.
In the Delphi procedure for defining FGR in intra-uterine fetal death, it proved to be im-possible to develop algorithms for calculating the intra-uterine retention interval (the time between fetal demise and delivery) and how correction of biometric variables should be applied. Future research should be focused on these issues to make the consensus based definition better applicable in clinical practice. Furthermore, the variable ‘foot length below the 10th percentile’ is included into this definition. However, the variable foot length is in general little influenced in FGR and is one of the items recognized to be a relatively accurate measure to determine gestational age at time of death.(17)
Delphi/COMET methodology
Several aspects of the methodology of consensus building procedures need further investi-gation, for example, the COMET and the Delphi methodology.
It remai ns unknown whether it is necessary to perform a systematic review as advised by COMET to give input to the Delphi procedure, or if a scoping review is sufficient. It would
10
not be very logic to perform a systematic analysis to encounter all possible outcomes, because chances are low that an outcome that is hardly ever used is considered fundamen-tal to the condition and will ultimately be included as a core outcome. Alternatively, would even an open first Delphi round will provide the least risk of bias? In an open first Delphi round, the participants are asked to name the most important outcomes in their opinion, without proposing outcomes based on literature by the steering group.(13) If the panel is composed adequately, one would expect that all important outcomes are mentioned in this open round. This may even lead to more relevant outcomes, because tunnel vision is avoided due to the absence of the ready-made literature-based outcome list. Most scien-tific articles are written by professionals and possibly lack outcomes that are considered relevant by patients. The patients are the ones who experience the effect of the condition on their daily quality of life and can therefore have another opinion on the most important outcomes. Thus, patients and/or caregivers should be involved in the COS-procedure. Another item of the Delphi procedure that lacks evidence is the use of the Likert-scale. The advised Likert scale is a 9-point Likert scale in the COMET methodology although no under-lying evidence is provided for this advice,(12) also 3, 5 and 7-point Likert scales are used. Enlarging the Likert-scale will result in less pronounced voting with regression to the mean, whereas a small Likert-scale will have less discriminating value. The used threshold for inclusion also affects the results. For example, if the threshold for inclusion is 7 on a 9-point Likert scale, more variables are included than if the threshold for inclusion is 5 on a 5-point Likert scale.
Furthermore, the ideal number of panel-members in a Delphi procedure is unknown. The original method developed by RAND, consisted of 9 panel-members.(13) Depending on the number of known experts in the research topic, the size of the expert panel can vary significantly.
A response rate of 80% for each stakeholder group is advised by COMET, but it is unknown how drop-outs affect the final results. What is an acceptable response rate in the Delphi procedure to avoid losing the strength of the panel? Also, attrition bias will occur when non-responders drop-out because their (possibly) different views related to their stakehold-er pestakehold-er-group are undstakehold-errepresented and thstakehold-erefore they may not be able to make a diffstakehold-er- differ-ence. This would falsely result in a stronger consensus.
COMET recognizes the expertise and crucial contribution of patients and caregivers in de-veloping COS’s, but what is the ideal percentage of patients versus clinicians to participate? The COMET methodology does not provide in an advice for patient-saturation, in previous COS procedures the percentage of patients/caregivers varied greatly from 4-50%.(12) The original Delphi methodology specifically stated that the individual panel-members should not meet in face-to-face meetings to prevent unwanted influence from socially
consensus round to finalize the consensus procedure in developing a core outcome set. (12) Despite the previously mentioned risks of bias, the advantages of this meeting are that it facilitates the possibility to speed up the consensus building and to make fast decisions, also direct verbal explanation of opinions can be given. Currently a live face-to-face meet-ing is advised, but maybe an electronic meetmeet-ing can serve the same needs. Also, the ideal timing of the consensus meeting lacks underlying evidence. Currently it is held at the very end and therefore has a very big impact on the final COS results, as the final selection for the COS is made in this meeting. If there have been misinterpretations in the procedure the timing at the very end is too late to adjust the Delphi procedure. It could be debated that a face-to-face consensus meeting (if it is chosen to be held), should be held in between the Delphi rounds, or maybe even at the very beginning to give input to the first Delphi round and refine formulation of questions. Furthermore, there is no advise what the best struc-ture or methodology is to organize the consensus meeting. Is it necessary to include all stakeholders in this meeting? Is it better to separate patients, to avoid that they lose out to the professionals? What is the ideal percentage of participants of the Delphi procedure to join the consensus meeting?
Future research is needed to answer these questions and further improve the COMET and Delphi methodology.
10
Box 1: Aspects of the COMET methodology that need further investigation
Is the advised initial systematic review (to give input to the Delphi procedure) the best method to collect information on all relevant outcomes?
What is the optimal Likert-scale and threshold to be used, to have enough discriminating power without regression to the mean?
How should patient’s/caregiver’s participation be managed to avoid that they lose out to the professionals?
What is the ideal percentage of patients/caregivers versus professionals to participate in the Delphi procedure? And in the consensus meeting?
What is the optimal response-rate in the Delphi procedure to avoid selection bias? Is the timing (at the very end of the procedure) of the consensus meeting adequate? Is a face-to-face consensus meeting mandatory, or could it be electronic?
References
1. Lees C, Marlow N, Arabin B, Bilardo CM, Brezinka C, Derks JB, Duvekot J, Frusca T, Diemert A, Ferrazzi E, Ganzevoort W, Hecher K, Martinelli P, Ostermayer E, Papageorghiou AT, Schlembach D, Schneider KT, Thilaganathan B, Todros T, van Wassenaer-Leemhuis A, Valcamonico A, Visser GH, Wolf H, Group T. Perinatal morbidity and mortality in early-onset fetal growth restriction: cohort outcomes of the trial of randomized umbilical and fetal flow in Europe (TRUFFLE). Ultrasound Obstet Gynecol. 2013;42(4):400-8.
2. Lees CC, Marlow N, van Wassenaer-Leemhuis A, Arabin B, Bilardo CM, Brezinka C, Calvert S, Derks JB, Diemert A, Duvekot JJ, Ferrazzi E, Frusca T, Ganzevoort W, Hecher K, Martinelli P, Ostermayer E, Papageorghiou AT, Schlembach D, Schneider KT, Thilaganathan B, Todros T, Valcamonico A, Visser GH, Wolf H, group Ts. 2 year neurodevelopmental and intermediate perinatal outcomes in infants with very preterm fetal growth restriction (TRUFFLE): a randomised trial. Lancet. 2015;385(9983):2162-72.
3. Kirkham JJ, Altman DG, Chan AW, Gamble C, Dwan KM, Williamson PR. Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic re views. BMJ. 2018;362:k3802.
4. Tovey D. The impact of Cochrane Reviews. Cochrane Database Syst Rev. 2010;2011:ED000007.
5. Unterscheider J, Daly S, Geary MP, Kennelly MM, McAuliffe FM, O’Donoghue K, Hunter A, Morrison JJ, Burke G, Dicker P, Tully EC, Malone FD. Definition and management
of fetal growth restriction: a survey of contemporary attitudes. Eur J Obstet Gynecol Reprod Biol. 2014;174:41-5.
6. Figueras F, Eixarch E, Meler E, Iraola A, Figueras J, Puerto B, Gratacos E. Small-for-gestation al-age fetuses with normal umbilical artery Doppler have suboptimal perinatal and neurode velopmental outcome. Eur J Obstet Gynecol Reprod Biol. 2008;136(1):34-8.
7. Vollgraff Heidweiller-Schreurs CA, De Boer MA, Heymans MW, Schoonmade LJ, Bossuyt PMM, Mol BWJ, De Groot CJM, Bax CJ. Prognostic accuracy of cerebroplacental ratio and middle cerebral artery Doppler for adverse perinatal outcome: systematic
review and meta-analysis. Ultrasound Obstet Gynecol. 2018;51(3):313-22.
8. McKenna HP. The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs. 1994;19(6):1221-5.
9. Helmer O. Analysis of the future: the Delphi method. http://wwwrandorg/content/dam /rand/pubs/papers/2008/P3558pdf. 1967.
10. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008-15.
11. Schmidt RC. Managing Delphi surveys using nonparametric statistical techniques. Decision sciences. 1997;28(3):763-74.
12. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, Prinsen CAC, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280.
13. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. The RAND/UCLA Appropriateness Method User’s Manual. 2001.
14. Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010;63(6):607-19.
10
2010;63(6):620-6.
16. Conde-Agudelo A, Papageorghiou AT, Kennedy SH, Villar J. Novel biomarkers for predicting intrauterine growth restriction: a systematic review and meta-analysis. BJOG.
2013;120(6):681-94.
17. Conway DL, Hansen NI, Dudley DJ, Parker CB, Reddy UM, Silver RM, Bukowski R, Pinar H, Stoll BJ, Varner MW, Saade GR, Hogue C, Willinger M, Coustan D, Koch MA, Goldenberg RL, Eunice Kennedy Shriver National Institute of Child H, Human Development Stillbirth Col laborative Research N. An algorithm for the estimation of gestational age at the time of fetal death. Paediatr Perinat Epidemiol. 2013;27(2):145-57.
18. Loughlin KG, Moore LF. Using Delphi to achieve congruent objectives and activities in a pedi atrics department. J Med Educ. 1979;54(2):101-6.
