Innovation in neurosurgery: less than IDEAL? A systematic review

(1)

REVIEW ARTICLE - NEUROSURGICAL TECHNIQUES

Innovation in neurosurgery: less than IDEAL? A systematic review

I. S. Muskens^1,2&S. J. H. Diederen¹&J. T. Senders^1,2&A. H. Zamanipoor Najafabadi³&

W. R. van Furth³_&A. M. May⁴_&T. R. Smith²_&A. L. Bredenoord⁵_&M. L. D. Broekman^1,2

Received: 16 February 2017 / Accepted: 19 July 2017 / Published online: 6 August 2017

# The Author(s) 2017. This article is an open access publication

Abstract

Background Surgical innovation is different from the introduction of novel pharmaceuticals. To help address this, in 2009 the IDEAL Collaboration (Idea, Development, Exploration, Assessment, Long-term follow-up) introduced the five-stage framework for surgical innovation. To evaluate the framework feasibility for novel neurosurgical procedure introduction, two innovative surgical procedures were examined: the endoscopic endonasal approach for skull base meningiomas (EEMS) and the WovenEndobridge (WEB device) for endovascular treatment of intracranial aneurysms.

Methods The published literature on EEMS and WEB devices was systematically reviewed. Identified studies were classified according to the IDEAL framework stage. Next,

studies were evaluated for possible categorization according to the IDEAL framework.

Results Five hundred seventy-six papers describing EEMS were identified of which 26 papers were included. No prospective studies were identified, and no studies reported on ethical approval or patient informed consent for the innovative procedure. Therefore, no clinical studies could be categorized according to the IDEAL Framework. For WEB devices, 6229 articles were screened of which 21 were included. In contrast to EEMS, two studies were categorized as 2a and two as 2b.

Conclusion The results of this systematic review demonstrate that both EEMS and WEB devices were not introduced according to the (later developed in the case of EEMS) IDEAL framework. Elements of the framework such as informed consent, ethical approval, and rigorous outcomes reporting are important and could serve to improve the quality of neurosurgical research. Alternative study designs and the use of big data could be useful modifications of the IDEAL framework for innovation in neurosurgery.

Keywords IDEAL framework . Innovation . Neurosurgery . Meningioma . Ethics . Intracranial aneurysm . WEB device

Introduction

Today, it is unusual to perform neurosurgical procedures in most countries without access to an operative microscope, state-of-the-art neuro-navigational systems, or even hemostat- ic agents such as a bipolar electrocautery devices. In fact, technological innovation has been the hallmark of neurosurgery, and the vast majority of procedures that are currently considered routine would not be possible at all without innovation. However, not all innovation is an improvement over the technology it seeks to supplant. Evidence of patient Electronic supplementary material The online version of this article

(doi:10.1007/s00701-017-3280-3) contains supplementary material, which is available to authorized users.

* M. L. D. Broekman

M.L.D.Broekman-4@umcutrecht.nl

1 Department of Neurosurgery, University Medical Center Utrecht, HP G03.124, PO Box 85500, 3508 GA Utrecht, The Netherlands

2 Cushing Neurosurgery Outcomes Center (CNOC), Department of Neurosurgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

3 Department of Neurosurgery, Leiden University Medical Center, Leiden, The Netherlands

4 Department of Epidemiology, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht,

Utrecht, The Netherlands

5 Department of Medical Humanities, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands

(2)

outcome superiority is often lacking or non-existent in the real-time of innovation. In neurosurgical disease, low incidence and high burden may further hinder systematic evaluation of any new technique. Regardless of these difficulties, it is vital that new technologies and procedures undergo a strategic and ethical clinical introduction [9].

As surgical innovation does not typically follow the same introductory path as novel pharmaceuticals, the IDEAL Collaboration, formed by surgeons and methodologists, introduced the IDEAL (Idea, Development, Exploration, Assessment, Long-term follow-up) framework in 2009 and have published several updates since [16,23,47,48,65]. The goal of the collaboration is to improve surgical research, especially research surrounding innovation, and to overcome obstacles and methodological problems inherent to surgery [29,47].

The IDEAL framework describes five stages through which interventional therapeutic innovations typically pass, together with the characteristics and study design of each stage (Table1, adapted from McCulloch et al.) [16,23,47,48,65].

Any study involving non-human pre-clinical assessment of a novel technique, including simulator or animal studies, is regarded as stage 0. Stage one describes a proof-of-concept study in the first human patient. Stage 2a consists of a prospective study in up to 30 patients conducted by surgeons responsible for the earlier stage(s). Involving surgeons with no prior experience in a larger prospective study usually takes place in stage 2b to assess the learning curve and further de- velop the procedure. In stage 3, the procedure should be stable and is investigated in a randomized controlled trial (RCT) that compares outcomes of the innovative procedure with the gold standard. Assessment of rare and long-term outcomes takes place in stage 4 (Table1) [29,47].

To assess whether the IDEAL framework has been used two different neurosurgical procedures were evaluated: an endoscopic endonasal approach for skull base meningiomas (EEMS) and the use of the Woven Endobridge (WEB device,

©Sequent Medical) for endovascular treatment of intracranial aneurysms. Traditionally, skull base meningiomas are resected using an open transcranial microscopic approach [36].

However, recently, EEMS has been introduced and has gained some traction in neurosurgical literature [36]. The WEB device is a new option for intracranial aneurysm treatment, consisting of an unfoldable, detachable metallic mesh that is placed into the aneurysm neck leading to flow disruption [35].

The WEB device was especially developed for bifurcation and wide neck aneurysms as an alternative to traditional clipping or coiling [35]. Since the two innovations, one a device and the other a procedure, are used in different fields of neurosurgery and were recently introduced, we chose these two as examples for neurosurgical innovations in general.

In this review, published literature on these two procedures was evaluated to assess whether they were introduced according to the stages of the IDEAL framework.

Methods

Search strategy and paper selection

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta- Analyses (PRISMA) statement [50]. The literature search for EEMS was conducted in PubMed and Embase up to 26 November 2015, using the following keywords: endoscopy, neurosurgery, endo- and transnasal and meningioma. The search strategy is provided in Supplemental Digital Content Table1a. This search strategy resulted in 576 unique papers.

In addition, bibliographies of the included papers were screened for relevant papers. For WEB devices, a search was conducted in the same search engines on 29 May 2016 using the keywords: WEB device, endovascular treatment, and intracranial aneurysm as depicted in Supplemental Digital Content Table1b. This resulted in 6229 articles. These papers were supplemented by hand searching of the bibliographies of the papers retrieved by the electronic search. This review was restricted to published data. Only papers written in English, Dutch, French, or German were considered for this review.

The search was not limited by date of publication. Titles and abstracts of retrieved citations were screened by two authors, and potentially suitable studies for EEMS were read in full by IM and SD and for WEB devices by IM and JS. We included papers that solely focused on EEMS as depicted in Fig.1a [50]. For WEB devices we included papers reporting outcomes of treated aneurysms as described in Fig. 1b [50].

Disagreements were solved by reviewer consensus.

Study assessment

Relevant studies were reviewed in full text to determine whether they could be classified according to an IDEAL stage by two authors (IM and SD for EEMS and IM and JS for WEB devices) [47]. The following criteria were used to clas- sify studies according to the five stages. Pre-clinical studies were classified as stage 0, and proof of principal in one patient was regarded as stage 1 if informed consent had been obtained [47]. Studies were classified as stage 2 if ethical approval for a prospective study and informed consent for an innovative procedure from included patients had been obtained. Studies with up to 20 patients were classified as stage 2a and those with more than 20 patients as stage 2b. Studies that compared EEMS or WEB devices with the current gold standard in a prospective fashion were regarded as stage 3. As an RCT may not have been feasible for ethical or pragmatic reasons, we also evaluated studies with different designs [16,23]. Long- term follow-up studies were categorized as stage 4. In addition to study design, ethical approval and informed consent, all studies were evaluated for reporting surgical or radiological

(3)

outcomes for EEMS and WEB devices studies, respectively.

Disagreements were solved by consensus discussion.

Results

For EEMS, 576 abstracts and titles were screened, 110 were examined full text, and 26 papers were included (Fig.1a) [1,8, 12,13,17,19–21,24–26,30,32,34,37,38,52–55,62,68, 70–73]. Two cadaveric studies were categorized as stage 0 [12,30]. No studies were categorized as stage 2, as none of the included studies reported outcomes of a prospective study with adequate informed consent [1,8,13,17,19–21,24–26, 32,34,37,38,52–55,62,68,70–73]. Even though four studies compared EEMS with an open transcranial approach, they did not do this in a prospective fashion, and no RCTs could be identified [8,19,20,52]. Furthermore, there were no studies that examined long-term outcomes, and therefore no studies were categorized as stage 4 (Table2). All other studies could not be categorized into an IDEAL stage.

For WEB devices, 6229 abstracts and titles were screened, 88 articles were examined full text, and 21 papers were included (Fig.1b) [2,5,10,11,14,15,22,27,33,35,41–43,45, 56–60,64,69]. Preclinical studies using rabbit models were classified as stage 0 [22, 64]. One study that acquired informed consent for treatment of two patients was categorized as stage 1, but did not describe the clinical problem that need- ed a solution [35]. Two studies with ethical approval for a prospective study and informed consent of included patients

were categorized as stage 2a [2,44]. The studies with larger populations that reported the outcomes of the WEBCAST trial and the French observatory trial were categorized as stage 2b [57,60]. All other studies could not be categorized into an IDEAL stage, and no studies were categorized as stage 3 or 4 as no comparison was made with other treatment modalities and no long-term outcomes were evaluated (Table3).

Discussion

The results of this systematic review demonstrate that both the endoscopic endonasal transsphenoidal approach for resection of skull base meningiomas and WEB devices were not introduced according to the IDEAL Framework. Not only could not all IDEAL framework stages be identified, some of the early pre-clinical studies (stage 0) were performed long after the description of the first-in-man studies (for EEMS) or after publication of prospective studies (WEB devices) [12,22,30, 64]. Perhaps unsurprisingly, only five clinical studies could be categorized into an IDEAL stage. WEB device studies followed the IDEAL Framework more closely than EEMS, but only up to stage 2b [2,35,45,57,60]. In addition, only six WEB device studies acquired ethical approval for a prospective study in line with the IDEAL framework [2,27,45,56,60, 61]. No study reported patient selection for EEMS compared to five WEB device studies [27,41,45,57,60]. Furthermore, no studies were categorized as stage 3 as no clinical study (of Table 1 Overview of the IDEAL framework, adapted from McColloch [47]

Purpose 1: Idea 2a: Development 2b: Exploration 3: Assessment 4: Long-term study

Proof of concept Development Learning Assessment Surveillance

Number and types of patients

Single digit, highly selected

Few; selected Many; may expand to mixed; broadening indication

Many; expanded indications

All eligible

Number and types of surgeons

Very few;

innovators

Few; innovators and some early adaptors

Many; innovators, early adaptors, early majority

Many; early majority All eligible

Output Description Description Measurement; comparison Comparison; Complete information for non RCT participants

Description; audit; regional variation; quality assurance; risk adjustment Intervention Evolving: procedure

inception

Evolving;

procedure development

Evolving; procedure refinement; community learning

Stable Stable

Method Structured case reports

Prospective development studies

Research database;

explanatory or feasibility RCT

RCT with or without additions/modifications;

alternative designs

Registry; routine

database; rare case reports Outcomes Proof of concept;

technical achievement;

Disasters; dramatic successes

Mainly safety;

technical and procedural success

Safety; clinical outcomes;

short-term outcomes;

patient centered outcomes feasibility outcomes

Clinical outcomes;

middle-term

and long-term outcomes;

patient-centered outcomes;

cost-effectiveness

Rare events; long-term outcomes; quality assurance

Ethical approval

Sometimes Yes Yes Yes No

(4)

either procedure) was a prospective comparison with the gold standard or was an RCT.

We believe that this is not unique to these two procedures specifically or to neurosurgery in general. For instance, a study investigating literature on laparoscopic colonic polyp resection found that its introduction into widespread use also did not follow the stages and recommendations of the IDEAL framework [18].

The introduction of novel neurosurgical techniques that result in a paradigm change, i.e., the first endovascular treatment of aneurysms, could be introduced according to some predefined framework such as IDEAL. However, in reality, novel surgical techniques are often the result of small stepwise changes to existing approaches (e.g., EEMS and the transcranial approach to pituitary adenomas). This makes it challenging to introduce innovations as EEMS according to all require- ments of the IDEAL framework. Adherence to the IDEAL framework might not only be challenging because of small stepwise changes of existing approaches but also because of a lack of a universally accepted definition of neurosurgical innovation in general.

A major change in endonasal surgery was the introduction of the endoscope, in particular for pituitary adenomas [31].

With expansion of endoscopic technique and experience, a wider spectrum of tumors became resectable through the endonasal approach. However, in retrospect, one could argue that EEMS is indeed a valuable alternative to a classic craniotomy for specific indications.

The WEB device is also example of expanding endovascular experience, and because of new endovascular devices a wider array of pathologies is treatable. Compared to EEMS, WEB devices were studied in a prospective fashion with patient informed consent [2,45,57,60]. However, the WEB device is already used clinically despite lack of comparison with other treatment options (a stage 3 study) [11,14,69].

The important question is whether this new technique could have been rigorously compared to established techniques prior to wide-spread adoption.

Overall, this review suggests that neurosurgical innovation (at least for the two procedures evaluated here) has not histor- ically followed the IDEAL framework. On the one hand, this could simply be caused by a lack of awareness of the

IdenﬁcaonScreeningIncludedEligibility

Records idenﬁed through Pubmed on 11-26-2015 (n = 190)

Addional records idenﬁed through PUBMED on 11-26-2015 (560)

Records aer duplicates removed in Endnote® X7.5 (n = 576)

Title/abstract screen (n = 576)

Records excluded (n =466)

Full-text arcles assessed for eligibility

(n = 110)

84 Full-text arcles excluded, with reasons

Congress abstract (n

= 23)

Not anterior skull base (36) Not purely endoscopic (n =4) Reply/commentary (n = 7)

Duplicate in populaon (n = 3) Not clinical study (n

= 11) Studies included in

qualitave analysis (n = 26) Inclusion criteria:

Clinical studies reporng outcomes or experience of meningiomas resected endoscopically and endonasally Anterior skull base meningiomas Exclusively meningioma Exclusion criteria:

Full text availability in English or Dutch Case-report Congress abstract Review

Reply/Commentary Endoscopically

Clinical arcles on outcomes of endoscopic endonasal meningioma resecon

Fig. 1 a Flowchart of the study selection process of included articles on endoscopic endonasal meningioma resection. b Flowchart of the study selection process of included articles on WEB devices

(5)

framework. On the other hand, a different distinct possibility for this could be related to feasibility. The IDEAL collaboration recognizes that, to improve the quantity and quality of surgical research, these proposals/recommendations would have to be practical and adapted to the process of innovation [47]. Indeed, the IDEAL Collaboration supports several recommendations for specific (alternative) study designs and reporting standards at different stages of the framework [16, 23,47]. These alternatives could contribute to the quantity and quality of neurosurgical research.

At the innovation stage (stage 1), the recommendations include online registries for first-in-man innovations. No reports on the entry of a study in a registry were found in our review. Often in neurosurgery innovations take place in an acute setting, and only in retrospect is there clarity with regards to the innovation itself. However, it is possible that future innovations could be entered in a registry, especially in the case of new devices like the WEB device. Registries could help reduce positive reporting bias inherent to new innovations. Reports of both successes and failures of new technology are useful for ethical innovation [51].

At the second development stage recommendations include: prospective development studies, protocol and study

registries for prospective development studies in surgery and development of agreed reporting standards and definitions for key outcomes [29,47]. These recommendations were not met for the introduction of EEMS and by only four studies for WEB devices [2,45,57,60].

Again, not all of these recommendations may be possible in neurosurgery. However, protocol and prospective study registries are feasible in the neurosurgical field and could help ensure that clinical results of all patients are transparent and methodologically sound. Furthermore, novel techniques could be reported using professionally accepted reporting guidelines for prospective (and if inapplicable, retrospective) studies that favor clear interpretation of the study design and study results.

Also, open comparison of individual studies and applicability of the reported outcomes would be useful. Key patient- centered outcomes for various pathologies result in research with comparable and clinically meaningful results.

All studies described the surgical outcomes, and this is outstanding. One next step could be to unify informed consent and outcomes reporting, which should include both positive and negative findings, for emerging innovative procedures.

Furthermore, one could argue this process should be done in

IdenﬁcaonScreeningIncludedEligibility

Records idenﬁed through Pubmed on 29-05-2016 (published from

2011) (n = 3512)

Addional records idenﬁed through EMBASE on 29-05-2016 (published from 2011) (n = 5254)

Records aer duplicates removed in Endnote® X7.5 (n = 6229)

Title/abstract screen (n = 6229)

Records excluded (n = 6141)

Full-text arcles assessed for eligibility

(n = 88)

67 Full-text arcles excluded, with reasons

Congress abstract (n

= 15)

No WEB device used (n = 23)

Review (n = 11) No clinical outcomes reported (n = 5) Reply/commentary (n = 7)

Duplicate in populaon (n = 6) Studies included in

qualitave analysis (n = 21) Inclusion criteria:

Arcles reporng data on outcomes of intracranial aneurysms Treatment with solely WEB device Clinical and pre- clinical studies Exclusion criteria:

Full-text unavailability in English or Dutch Case-report Congress abstract Review

Reply/Commentary

Clinical arcles on outcomes of intracranial aneurysms treated with a WEB device

Fig. 1 (continued)

(6)

a more uniform manner across the neurosurgical field. One method might be the use of centralized regulation as seen with m e d i c a l d e v i c e a p p r o v a l b y t h e F o o d a n d D r u g Administration (FDA) [39,40]. Alternatively, institutions or neurosurgical societies could create guidelines for reporting of trial registration, prospective design and patient registries, ef- fectively following the IDEAL framework to a certain extent [48]. Nevertheless, informed consent and ethical approval for a prospective study are, we believe, something that should always be feasible when evaluating a new neurosurgical procedure.

No prospective randomized studies or RCTs, the‘default option’ at the third or exploration stage of the IDEAL framework, were identified [47]. This may be one area of the IDEAL framework that is not completely feasible in all types of neurosurgical innovation. As discussed, innovation occurs by incremental but gradual changes over a prolonged period

of time, and an RCT may not be the preferred study design for numerous reasons: (1) It is ethically challenging and practical- ly impossible to compare EEMS to an open approach as the endonasal approach is not applicable to all patients; (2) the number of patients with skull base meningiomas is relatively small, which makes it difficult to recruit enough patients for proper statistical analyses; (3) the difference in outcomes be- tween an open and endonasal approach might be small and therefore difficult to prove, especially with point 2 in mind; (4) there could be a lack of clinical equipoise; (5) surgeons might not be willing to participate because of personal treatment preference or experience [46]; (6) surgeons have different skill levels; (7) the location, extent and size of meningiomas vary, complicating inter-patient comparability and randomization, again complicated by point 3; (8) concomitant factors can change during the trial, e.g., innovation in anesthesiology and perioperative care [6,49]; (9) improvement of endoscopic Table 2 IDEAL Framework recommendations and endoscopic endonasal meningioma surgery

Author

(year of publication)

Participants (N=)

Ethical approval for prospective study

Informed consent for innovative procedure

Described surgical outcome

Randomized controlled trial

Awarded IDEAL stage

Cavallo et al. (2005) [12] 0 NA NA NA NA 0

Jacquesson et al. (2015) [30] 0 NA NA NA NA 0

Alexander et al. (2010) [1] 1 N N Y N None

Bowers et al. (2011) [8] 27 N N Y N None

Chowdhury et al. (2012) [13] 6 N N Y N None

Cook et al. (2004) [17] 3 N N Y N None

De Almeida et al. (2015) [19] 20 N N Y N None

De Divitiis et al. (2008) [20] 51 N N Y N None

De Divitiis et al. (2008) [21] 11 N N Y N None

Fernadez-Miranda et al. (2012) [24]

1 N N Y N None

Gadgil et al. (2013) [25] 5 N N Y N None

Gardner et al. (2008) [26] 35 N N Y N None

Julian et al. (2014) [32] 1 N N Y N None

Khan et al. (2014) [34] 46 N N Y N None

Koutourousiou (2014) [37] 75 N N Y N None

Koutourousiou (2014) [38] 50 N N Y N None

Mortazavi et al. (2015) [52] 27 N N Y N None

Ogawa et al. (2012) [53] 19 N N Y N None

Ottenhausen et al. (2014) [54] 20 N N Y N None

Padhye et al. (2012) [55] 15 N N Y N None

Prevedello et al. (2007) [62] 1 N N Y N None

Van Gompel et al. (2011) [68] 13 N N Y N None

Wang et al. (2009) [71] 7 N N Y N None

Webb-Myers et al. (2008) [73] 1 N N Y N None

The Y (Yes) means the study meets the IDEAL framework recommendations. The N (No) means the study did not meet the IDEAL framework recommendations. NA Not applicable

(7)

endonasal meningioma surgery is a constantly evolving process with differences in every center, which contributes to the often reported difficulty in standardization for innovative surgical procedures, it is inefficient to conduct an RCT for every incremental technological advance, and the incidence of these lesions is quite low [6]. For these reasons, aBclassical^ RCT in low-volume, highly complex cases as with skull base meningioma resections or similar procedures might not be feasible. However, the IDEAL collaboration endorses various alternatives to this trial design at the third stage. These include case-matching studies and controlled interrupted-time series designs, but also modified RCTs with Baysean modifications to recruitment, randomization or analysis [47]. These study designs might be useful in neurosurgical innovation.

Especially the introduction of prospective research databases and collaborative studies, endorsed by the IDEAL collaboration, seem valuable for low-volume, highly complex surgeries such as skull base meningioma resections. Also, the recom- mended additions to the RCTs that include learning curve evaluation, quality control and compliance measures could be feasible and helpful for innovations as EEMS.

Even though an RCT for WEB devices could be challenging, especially because of the above-mentioned reasons 3–9,

an RCT is possible and could have been conducted prior to wide-spread European adoption [35]. However, in the absence of a traditional RCT, a Baysian RCT or registry could have also been helpful to establish its efficacy and safety. In fact, application of all stages of the IDEAL framework in a more strategic fashion could be possible in technological innovations like the WEB device. To date, the WEB device appears to be efficacious and safe, but a more rigorous and transparent process for the introduction of this type of technology could potentially help prevent deleterious outcomes, as seen with the Poly Implant Prothèse (PIP) breast implants and metal-on- metal hip prostheses [4,28,67]. Currently, proof of safety and efficacy is required by the FDA for class III devices (the most invasive devices), but this is not standardized [39,40].

Therefore, a change in regulation that results in a closer adherence to the IDEAL framework could lead to a more uniform implementation [48].

At the fourth or long-term study stage, the emphasis is on rare and long-term outcomes. We did not identify any (stage 4) studies reporting long-term outcomes of EEMS or WEB devices. We believe that in addition to a closer adherence to the‘IDEA’ part of the IDEAL framework, attention to the long-term outcomes of innovations such as EEMS or WEB devices would greatly Table 3 IDEAL framework recommendations and the WEB device

Author

(year of publication)

Participants (N=)

Ethical approval for prospective study

Informed consent for innovative procedure

Described radiological outcome

Randomized controlled trial

Awarded IDEAL stage

Ding et al. (2011) [22] 24 (rabbits) NA NA NA N 0

Rouchaud et al. (2016) [64] 80 (rabbits) NA NA NA N 0

Ambrosi et al. (2015) [2] 10 Y Y Y N 2a

Behme et al. (2015) [5] 52 N N Y N None

Caroff et al. (2014) [10] 6 N N Y N None

Caroff et al. (2015) [11] 98 N N Y N None

Clajus et al. (2016) [14] 108 N Y^a Y N None

Colla et al. (2013) [15] 4 N N Y N None

Gherasim et al. (2015) [27] 10 Y N Y N None

Kabbasch et al. (2016) [33] 43 N N Y N None

Klisch et al. (2011) [35] 2 N Y Y N 1

Lawson et al. (2016) [41] 23 N N Y N None

Lescher et al. (2016) [42] 22 N N Y N None

Liebig et al. (2015) [43] 47 N N Y N None

Lubicz et al. (2013) [44] 19 Y Y Y N 2a

Papagiannaki et al. (2014) [56] 83 Y N Y N None

Pierot et al. (2013) [58] 33 N N Y N None

Pierot et al. (2015) [59] 45 N N Y N None

Pierot et al. (2016) [57] 51 Y Y Y N 2b

Pierot et al. (2016) [60] 62 Y Y Y N 2b

Van Rooij et al. (2016) [69] 32 N N Y N None

The Y (Yes) symbol means the study met the IDEAL framework recommendations. The N (No) symbol means the study did not meet the IDEAL framework recommendations

aInformed consent was only obtained in cognitively intact patients

(8)

benefit innovation in neurosurgery. Registries are an appropriate study design for this purpose, although representativeness of the data is a potential limitation. Efforts made to ensure that data entry is complete help strengthen the representativeness of the registry [47]. Reporting fatigue can compromise comprehensive data collection; therefore, the development of concentrated, outcome relevant registries are optimal. Also, the use of registries with patient informed consent for surveillance of specific established techniques in neurosurgery is desirable, especially for use of new materials like the WEB device.

In general, innovation in low-volume, highly complex (neuro)surgical cases might benefit from alternatives to traditional RCTs. For example, in aBcohort multiple RCT,^ some, but not all, patients are randomly assigned to a specific treatment and are followed up regularly over time, blending a RCT with an observational study with some of their respective ben- efits [7,63,66]. A potential stage 3 study on a low-volume, highly complex surgical innovation could include the following: (1) patient informed consent; (2) ethical approval; (3) strict definition (and registration) of indications for treatment;

(4) prospective observational design; (5) registration in a trial registry; (6) random allocation of a standard treatment group or the well-defined innovative procedure; (7) regular follow- up on relevant outcomes for patients; (8) reporting of all outcomes; (9) collaboration of multiple centers.

This, however, does not address the issue of which innovative procedures merit such a study.BBig data^ could fill the gap with regard to identification of trial-worthy innovations.

The use of the electronic medical record, the digitization of patient outcomes and the computational capacity now available to the typical researcher have opened the door to detailed and comprehensive analysis of pre-trial data. Indeed, these types of large data sets could become a new level of evidence in and of itself if an RCT is not feasible [3].

Conclusion

The introduction of EEMS and WEB devices did not follow the stages as described by the IDEAL framework. The introduction of WEB devices followed the IDEAL Framework more closely, but only up to stage 2b. We believe this is not unique to neurosurgery or to these techniques, and it simply may not be feasible to follow this framework in its current iteration for all types of innovation. Despite this, informed consent, ethical approval and rigorous outcomes reporting are important elements of the IDEAL framework, which could serve to improve the quality of both experimental and alternative neurosurgical study designs. Alternatives to traditional RCTs and the use ofBbig data^ could be useful modifications of the IDEAL framework. We believe that neurosurgical innovation and research could be improved by following a framework such as (a modified version of) IDEAL. This

would improve evidence-based practice and potentially patient outcomes. After all, methodologically sound prospective studies, which require informed consent, ethical approval and equipoise, are feasible in neurosurgery.

Acknowledgements The authors would like to thank Paulien Wiersma for help with drafting the search strategy.

Compliance with ethical standards

Conflict of interest The authors declare that they have no competing interests.

Disclosures IM received funding from the KNAW to present the content of this manuscript at the 2017 AANS meeting in Los Angeles.

Funding No funding.

Human and animal consent This article does not contain any studies with human participants or animals performed by any of the authors.

Open Access This article is distributed under the terms of the Creative C o m m o n s A t t r i b u t i o n 4 . 0 I n t e r n a t i o n a l L i c e n s e ( h t t p : / / creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

References

1. Alexander H, Robinson S, Wickremesekera A, Wormald PJ (2010) Endoscopic transsphenoidal resection of a mid-clival meningioma.

J Clin Neurosci 17:374–376

2. Ambrosi PB, Gory B, Sivan-Hoffmann R, Riva R, Signorelli F, Labeyrie PE, Eldesouky I, Sadeh-Gonike U, Armoiry X, Turjman F (2015) Endovascular treatment of bifurcation intracranial aneurysms with the WEB SL/SLS: 6-month clinical and angiographic results. Interv Neuroradiol 21:462–469

3. Angus DC (2015) Fusing randomized trials with big data: the key to self-learning health care systems? JAMA 314:767–768

4. Ardaugh BM, Graves SE, Redberg RF (2013) The 510(k) ancestry of a metal-on-metal hip implant. N Engl J Med 368:97–100 5. Behme D, Berlis A, Weber W (2015) Woven endo bridge

intrasaccular flow disrupter for the treatment of ruptured and unruptured wide-neck cerebral aneurysms: report of 55 cases. Am J Neuroradiol 36:1501–1506

6. Blencowe NS, Brown JM, Cook JA, Metcalfe C, Morton DG, Nicholl J, Sharples LD, Treweek S, Blazeby JM, Members of the MRCHfTMRNW (2015) Interventions in randomised controlled trials in surgery: issues to consider during trial design. Trials 16:392 7. Booth CM, Tannock IF (2014) Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. Br J Cancer 110:551–555

8. Bowers CA, Altay T, Couldwell WT (2011) Surgical decision- making strategies in tuberculum sellae meningioma resection.

Neurosurg Focus 30:E1

9. Broekman ML, Carriere ME, Bredenoord AL (2016) Surgical innovation: the ethical agenda: a systematic review. Medicine (Baltimore) 95:e3790

10. Caroff J, Mihalea C, Dargento F, Neki H, Ikka L, Benachour N, Moret J, Spelle L (2014) Woven Endobridge (WEB) device for

(9)

endovascular treatment of ruptured intracranial wide-neck aneurysms: a single-center experience. Neuroradiology 56:755–761 11. Caroff J, Mihalea C, Klisch J, Strasilla C, Berlis A, Patankar T,

Weber W, Behme D, Jacobsen EA, Liebig T, Prothmann S, Cognard C, Finkenzeller T, Moret J, Spelle L (2015) Single-layer webs: Intrasaccular flow disrupters for aneurysm treatment- feasibility results from a european study. Am J Neuroradiol 36:

1942–1946

12. Cavallo LM, Messina A, Cappabianca P, Esposito F, de Divitiis E, Gardner P, Tschabitscher M (2005) Endoscopic endonasal surgery of the midline skull base: anatomical study and clinical consider- ations. Neurosurg Focus 19:E2

13. Chowdhury FH, Haque MR, Goel AH, Kawsar KA (2012) Endoscopic endonasal extended transsphenoidal removal of tuberculum sellae meningioma (TSM): an experience of six cases.

Br J Neurosurg 26:692–699

14. Clajus C, Strasilla C, Fiebig T, Sychra V, Fiorella D, Klisch J (2016) Initial and mid-term results from 108 consecutive patients with cerebral aneurysms treated with the WEB device. J Neurointerv Surg 9:411–417

15. Colla R, Cirillo L, Princiotta C, Dall'Olio M, Menetti F, Vallone S, Leonardi M (2013) Treatment of wide-neck basilar tip aneurysms using the web II device. Neuroradiol J 26:669–677

16. Cook JA, McCulloch P, Blazeby JM, Beard DJ, Marinac-Dabic D, Sedrakyan A, Group I (2013) IDEAL framework for surgical innovation 3: randomised controlled trials in the assessment stage and evaluations in the long term study stage. BMJ 346:f2820 17. Cook SW, Smith Z, Kelly DF (2004) Endonasal transsphenoidal

removal of tuberculum sellae meningiomas: technical note.

Neurosurgery 55:239–244

18. Currie A, Brigic A, Blencowe NS, Potter S, Faiz OD, Kennedy RH, Blazeby JM (2015) Systematic review of surgical innovation reporting in laparoendoscopic colonic polyp resection. Br J Surg 102:e108–e116

19. De Almeida JR, Carvalho F, Vaz Guimaraes Filho F, Kiehl TR, Koutourousiou M, Su S, Vescan AD, Witterick IJ, Zadeh G, Wang EW, Fernandez-Miranda JC, Gardner PA, Gentili F, Snyderman CH (2015) Comparison of endoscopic endonasal and bifrontal craniotomy approaches for olfactory groove meningiomas: a matched pair analysis of outcomes and frontal lobe changes on MRI. J Clin Neurosci 22:1733–1741

20. de Divitiis E, Esposito F, Cappabianca P, Cavallo LM, de Divitiis O (2008) Tuberculum sellae meningiomas: high route or low route? A series of 51 consecutive cases. Neurosurgery 62:556–563 discussion 556-563

21. de Divitiis E, Esposito F, Cappabianca P, Cavallo LM, de Divitiis O, Esposito I (2008) Endoscopic transnasal resection of anterior cranial fossa meningiomas. Neurosurg Focus 25:E8

22. Ding YH, Lewis DA, Kadirvel R, Dai D, Kallmes DF (2011) The woven EndoBridge: a new aneurysm occlusion device. AJNR Am J Neuroradiol 32:607–611

23. Ergina PL, Barkun JS, McCulloch P, Cook JA, Altman DG, Group I (2013) IDEAL framework for surgical innovation 2: observational studies in the exploration and assessment stages. BMJ 346:f3011 24. Fernandez-Miranda JC, Morera VA, Snyderman CH, Gardner P

(2012) Endoscopic endonasal transclival approach to the jugular tubercle. Neurosurgery 71:146–158 discussion 158-149

25. Gadgil N, Thomas J, Takashima M, Yoshor D (2013) Endoscopic resection of tuberculum sellae meningiomas. J Neurol Surg Part B:

Skull Base 74:201–210

26. Gardner PA, Kassam AB, Thomas A, Snyderman CH, Carrau RL, Mintz AH, Prevedello DM (2008) Endoscopic endonasal resection of anterior cranial base meningiomas. Neurosurgery 63:36–52 discussion 52-34

27. Gherasim DN, Gory B, Sivan-Hoffmann R, Pierot L, Raoult H, Gauvrit JY, Desal H, Barreau X, Herbreteau D, Riva R,

Impiombato FA, Armoiry X, Turjman F (2015) Endovascular treatment of wide-neck anterior communicating artery aneurysms using WEB-DL and WEB-SL: short-term results in a multicenter study.

Am J Neuroradiol 36:1150–1154

28. Greco C (2015) The Poly Implant Prothese breast prostheses scan- dal: embodied risk and social suffering. Soc Sci Med 147:150–157 29. Hirst A, Agha RA, Rosin D, McCulloch P (2013) How can we improve surgical research and innovation?: the IDEAL framework for action. Int J Surg 11:1038–1042

30. Jacquesson T, Simon E, Berhouma M, Jouanneau E (2015) Anatomic comparison of anterior petrosectomy versus the expanded endoscopic endonasal approach: interest in petroclival tumors surgery. Surg Radiol Anat 37:1199–1207

31. Jankowski R, Auque J, Simon C, Marchal JC, Hepner H, Wayoff M (1992) Endoscopic pituitary tumor surgery. Laryngoscope 102:

198–202

32. Julian JAS, Álvarez PS, Lloret PM, Ramirez EP, Borreda PP, Asunción CB (2014) Full endoscopic endonasal transclival approach: Meningioma attached to the ventral surface of the brainstem. Neurocirugia 25:140–144

33. Kabbasch C, Mpotsaris A, Reiner M, Liebig T (2016) WEB as part of a multimodality treatment in complex, large, and partially thrombosed intracranial aneurysms: a single-center observational study of technical success, safety, and recurrence. J Neurointerv Surg. doi:10.1136/neurintsurg-2015-012126

34. Khan OH, Anand VK, Schwartz TH (2014) Endoscopic endonasal resection of skull base meningiomas: the significance of a "cortical cuff" and brain edema compared with careful case selection and surgical experience in predicting morbidity and extent of resection.

35. Klisch J, Sychra V, Strasilla C, Liebig T, Fiorella D (2011) The woven endobridge cerebral aneurysm embolization device (WEB II): initial clinical experience. Neuroradiology 53:599– 607

36. Komotar RJ, Starke RM, Raper DMS, Anand VK, Schwartz TH (2012) Endoscopic endonasal versus open transcranial resection of anterior midline skull base meningiomas. World Neurosurg 77:

713–724

37. Koutourousiou M, Fernandez-Miranda JC, Stefko ST, Wang EW, Snyderman CH, Gardner PA (2014) Endoscopic endonasal surgery for suprasellar meningiomas: experience with 75 patients. J Neurosurg 120:1326–1339

38. Koutourousiou M, Fernandez-Miranda JC, Wang EW, Snyderman CH, Gardner PA (2014) Endoscopic endonasal surgery for olfactory groove meningiomas: outcomes and limitations in 50 patients.

39. Kramer DB, Xu S, Kesselheim AS (2012) How does medical device regulation perform in the United States and the European union? A systematic review. PLoS Med 9:e1001276

40. Kramer DB, Xu S, Kesselheim AS (2012) Regulation of medical devices in the United States and European Union. N Engl J Med 366:848–855

41. Lawson A, Goddard T, Ross S, Tyagi A, Deniz K, Patankar T (2017) Endovascular treatment of cerebral aneurysms using the Woven EndoBridge technique in a single center: preliminary results. J Neurosurg 126:17–28

42. Lescher S, du Mesnil de Rochemont R, Berkefeld J (2016) Woven Endobridge (WEB) device for endovascular treatment of complex unruptured aneurysms—a single center experience. Neuroradiology 58:383–390

43. Liebig T, Kabbasch C, Strasilla C, Berlis A, Weber W, Pierot L, Patankar T, Barreau X, Dervin J, Kursumovic A, Rath S, Lubicz B, Klisch J (2015) Intrasaccular flow disruption in acutely ruptured aneurysms: a multicenter retrospective review of the use of the WEB. AJNR Am J Neuroradiol 36:1721–1727

(10)

44. Lubicz B, Klisch J, Gauvrit JY, Szikora I, Leonardi M, Liebig T, Nuzzi NP, Boccardi E, Paola FD, Holtmannspotter M, Weber W, Calgliari E, Sychra V, Mine B, Pierot L (2014) WEB-DL endovascular treatment of wide-neck bifurcation aneurysms:

short- and midterm results in a European study. AJNR Am J Neuroradiol 35:432–438

45. Lubicz B, Mine B, Collignon L, Brisbois D, Duckwiler G, Strother C (2013) WEB device for endovascular treatment of wide-neck bifurcation aneurysms. Am J Neuroradiol 34:1209–1214 46. Macefield RC, Boulind CE, Blazeby JM (2014) Selecting and mea-

suring optimal outcomes for randomised controlled trials in surgery.

Langenbeck's Arch Surg 399:263–272

47. McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J (2009) No surgical innovation without evaluation: the IDEAL recommendations. Lancet 374:1105–1112 48. McCulloch P, Cook JA, Altman DG, Heneghan C, Diener MK,

Group I (2013) IDEAL framework for surgical innovation 1: the idea and development stages. BMJ 346:f3012

49. Meakins JL (2008) The rules of evidence-based medicine: can they be generalized to improve GI surgical practice? J Gastrointest Surg 12:620–623

50. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P (2009) Preferred reporting items for systematic reviews and meta-analyses:

the PRISMA statement. PLoS Med 6:e1000097

51. Moojen WA, Bredenoord AL, Viergever RF, Peul WC (2014) Scientific evaluation of spinal implants: an ethical necessity.

Spine (Phila Pa 1976) 39:2115–2118

52. Mortazavi MM, da Silva HB, Ferreira M Jr, Barber JK, Pridgeon JS, Sekhar LN (2015) Planum sphenoidale and tuberculum sellae meningiomas: operative nuances of a modern surgical technique with outcome and proposal of a new classification system. World Neurosurg 86:270–286

53. Ogawa Y, Tominaga T (2012) Extended transsphenoidal approach for tuberculum sellae meningioma—what are the optimum and crit- ical indications? Acta Neurochir 154:621–626

54. Ottenhausen M, Banu MA, Placantonakis DG, Tsiouris AJ, Khan OH, Anand VK, Schwartz TH (2014) Endoscopic endonasal resection of suprasellar meningiomas: the importance of case selection and experience in determining extent of resection, visual improvement, and complications. World Neurosurg 82:442–449

55. Padhye V, Naidoo Y, Alexander H, Floreani S, Robinson S, Santoreneos S, Wickremesekera A, Brophy B, Harding M, Vrodos N, Wormald PJ (2012) Endoscopic endonasal resection of anterior skull base meningiomas. Otolaryngol Head Neck Surg (United States) 147:575–582

56. Papagiannaki C, Spelle L, Januel AC, Benaissa A, Gauvrit JY, Costalat V, Desal H, Turjman F, Velasco S, Barreau X, Courtheoux P, Cognard C, Herbreteau D, Moret J, Pierot L (2014) WEB intrasaccular flow disruptor-prospective, multicenter experience in 83 patients with 85 aneurysms. AJNR Am J Neuroradiol 35:2106–2111

57. Pierot L, Costalat V, Moret J, Szikora I, Klisch J, Herbreteau D, Holtmannspotter M, Weber W, Januel AC, Liebig T, Sychra V, Strasilla C, Cognard C, Bonafe A, Molyneux A, Byrne JV, Spelle L (2016) Safety and efficacy of aneurysm treatment with WEB:

results of the WEBCAST study. J Neurosurg 124:1250–1256 58. Pierot L, Klisch J, Cognard C, Szikora I, Mine B, Kadziolka K,

Sychra V, Gubucz I, Januel AC, Lubicz B (2013) Endovascular WEB flow disruption in middle cerebral artery aneurysms:

preliminary feasibility, clinical, and anatomical results in a multicenter study. Neurosurgery 73:27–34

59. Pierot L, Klisch J, Liebig T, Gauvrit JY, Leonardi M, Nuzzi NP, Paola FD, Sychra V, Mine B, Lubicz B (2015) WEB-DL endovascular treatment of wide-neck bifurcation aneurysms: long- term results in a European series. Am J Neuroradiol 36:2314–2319 60. Pierot L, Moret J, Turjman F, Herbreteau D, Raoult H, Barreau X, Velasco S, Desal H, Januel AC, Courtheoux P, Gauvrit JY, Cognard C, Molyneux A, Byrne J, Spelle L (2016) WEB treatment of intracranial aneurysms: clinical and anatomic results in the French observatory. AJNR Am J Neuroradiol 37:655–659

61. Pierot L, Spelle L, Molyneux A, Byrne J, Webcast, French Observatory I (2016) Clinical and anatomical follow-up in patients with aneurysms treated with the WEB device: 1-year follow-up report in the cumulated population of 2 prospective, multicenter series (WEBCAST and French observatory). Neurosurgery 78:

133–141

62. Prevedello DM, Thomas A, Gardner P, Snyderman CH, Carrau RL, Kassam AB (2007) Endoscopic endonasal resection of a synchro- nous pituitary adenoma and a tuberculum sellae meningioma: technical case report. Neurosurgery 60:E401 discussion E401 63. Relton C, Torgerson D, O'Cathain A, Nicholl J (2010) Rethinking

pragmatic randomised controlled trials: introducing the "cohort multiple randomised controlled trial" design. BMJ 340:c1066 64. Rouchaud A, Brinjikji W, Ding YH, Dai D, Zhu YQ, Cloft HJ,

Kallmes DF, Kadirvel R (2016) Evaluation of the angiographic grading scale in aneurysms treated with the WEB device in 80 rabbits: correlation with Histologic evaluation. AJNR Am J Neuroradiol 37:324–329

65. Sedrakyan A, Campbell B, Merino JG, Kuntz R, Hirst A, McCulloch P (2016) IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ 353:i2372 66. Silverman SL (2009) From randomized controlled trials to obser-

vational studies. Am J Med 122:114–120

67. Smith AJ, Dieppe P, Vernon K, Porter M, Blom AW, National Joint Registry of E, Wales (2012) Failure rates of stemmed metal-on- metal hip replacements: analysis of data from the National Joint Registry of England and Wales. Lancet 379:1199–1204

68. Van Gompel JJ, Frank G, Pasquini E, Zoli M, Hoover J, Lanzino G (2011) Expanded endonasal endoscopic resection of anterior fossa meningiomas: report of 13 cases and meta-analysis of the literature.

69. van Rooij WJ, Peluso JP, Bechan RS, Sluzewski M (2016) WEB treatment of ruptured intracranial aneurysms. AJNR Am J Neuroradiol 37:1679–1683

70. Wang Q, Lu XJ, Ji WY, Yan ZC, Xu J, Ding YS, Zhang J (2010) Visual outcome after extend ed endoscopi c endonasal transsphenoidal surgery for tuberculum sellae meningiomas.

World Neurosurg 73:694–700

71. Wang Q, Lu XJ, Li B, Ji WY, Chen KL (2009) Extended endoscopic endonasal transsphenoidal removal of tuberculum sellae meningiomas: a preliminary report. J Clin Neurosci 16:889–893 72. Wang WH, Abhinav K, Wang E, Snyderman C, Gardner PA,

Fernandez-Miranda JC (2015) Endoscopic endonasal transclival transcondylar approach for foramen magnum meningiomas: surgical anatomy and technical note. Oper Neurosurg 12:153–162 73. Webb-Myers R, Wormald PJ, Brophy B (2008) An endoscopic

endonasal technique for resection of olfactory groove meningioma.

J Clin Neurosci 15:451–455