Renée Janne Van Brummelen
Clinical Practice Guideline Adherence in the
Treatment of Atrial Fibrillation
1 Renée Janne Van Brummelen
Clinical Practice Guideline Adherence in the
Treatment of Atrial Fibrillation
Student R.J. Van Brummelen Student number: 10576444 E-mail: r.j.vanbrummelen@amc.uva.nl Mentor R.J. van Woersem MD Chipsoft B.V. Tt. Melissaweg 25 1033 SP Amsterdam Tutor Drs. D.L. Arts MD AMC Amsterdam Faculty of Medicine
Department of Medical Informatics & General Practice/Family Medicine Location of Scientific Research Project
Chipsoft B.V. Tt. Melissaweg 25 1033 SP Amsterdam Practice Teaching Period November 2014 – June 2015
2
FOREWORD & ACKNOWLEDGMENTS
This thesis is written as completion of the master program Medical Informatics at the University of Amsterdam, during which much attention was paid to clinical practice guidelines. Although in the beginning, I was considering them as a cumbersome attempt to improve quality of care, I became more enthusiastic about the topic when I learned about how guidelines could be used for clinical decision support systems. In my opinion, making guidelines accessible to clinicians as integrated content of decision support systems could actually lead to improved quality of care. When I had to make a decision regarding the topic of my scientific research project, clinical decision support was therefore high on my wish list. Fortunately, ChipSoft was interested in exploring the possibilities of decision support for atrial fibrillation as integrated part of their electronic health record software, and the idea of conducting a randomized controlled trial to assess its effectiveness arose. Almost eight months later, my thesis is finally completed and now lies in front of you.
However, I could not have completed this project without the help and support of others. Therefore, I would like to thank my tutor Derk Arts for his encouragement during my internship and his ability to come up with Plan B, and my mentor Ruurd van Woersem for his never-ending positivism. Furthermore, I would like to thank my mother (you just always seem to know what to say), and my father (the master of relativism) for supporting me in everything I am trying to achieve. A big thanks to all my fellow students, and especially to Jorrit for letting me complain all the time and making it possible to listen to music synchronously while working on our theses (you made my internship days more bearable), to Dennis for making me feel less lonely because we had to deal with the same internship-related difficulties and could whine about it together, to my friends and family for not asking about my thesis all the time (especially in the end) and for reviewing my drafts, and to Tim for accepting all the sleepless nights and my crankiness, and for your positive altitude and advices even when I thought that apocalypse was near. Thank you all, I could not have done it without you!
3
CONTENT
Summary... 6 Samenvatting ... 7 Keywords ... 8 Abbreviations ... 8 1 Introduction ... 9 2 Preliminaries ... 132.1 Clinical Practice Guidelines ... 13
2.2 Clinical Decision Support Systems ... 13
2.3 Atrial Fibrillation ... 14
2.4 Related work ... 17
2.4.1 Classification of CDSS ... 19
2.4.2 Characteristics and Results of Performed Studies ... 21
3 Intervention ... 24
3.1 Classification of Intervention ... 26
4 Data and Methods ... 28
4.1 Data and Data Collection ... 28
4.2 Guideline Adherence Measurement... 29
4.3 Randomized Controlled Trial... 30
4.3.1 Trial Design ... 30
4.3.2 Participants and Facilities ... 30
4.3.3 Intervention ... 31
4.3.4 Outcome ... 31
4.3.5 Sample Size ... 32
4.3.6 Randomization & Blinding ... 32
4
4.4 Logistic Regression Analysis ... 33
4.4.1 Hypothesis ... 33
4.4.2 Measures ... 33
4.4.3 Statistical Methods ... 34
5 Results ... 35
5.1 Baseline Guideline Adherence ... 35
5.1.1 Descriptive Statistics & Overall Adherence ... 35
5.1.2 Adherence per Cardiologist ... 36
5.1.3 Adherence per Hospital ... 38
5.2 Guideline Adherence Improvement ... 39
5.2.1 Performance at Baseline ... 39
5.2.2 Descriptive Statistics ... 39
5.2.3 Contingency Table ... 40
5.2.4 Chi-Square Test Results ... 40
5.3 Logistic Regression Analysis ... 41
5.3.1 Univariate Analysis: Gender ... 41
5.3.2 Univariate Analysis: Age ... 41
5.3.3 Univariate Analysis: Location of Employment ... 41
5.3.4 Univariate Analysis: Years Since Graduation ... 42
5.3.5 Mulitvariate Analysis ... 42
5.3.6 Significance of the Overall Model ... 42
6 Discussion ... 43
6.1 Main Findings ... 43
6.2 Baseline Guideline Adherence ... 43
6.3 Guideline Adherence Improvement ... 45
6.4 Physician Characteristics ... 47
5 7 Conclusion ... 51 References ... 52 Appendix A ... 62 Appendix B ... 63 Appendix C ... 64 Appendix D ... 68 Appendix E ... 71 Appendix F ... 72 Appendix G ... 73 Appendix H ... 78 Appendix I ... 79
6
SUMMARY
Atrial fibrillation is a cardiac arrhythmia, characterized by rapid and irregular heart beating, and is associated with an increased risk of stroke. To reduce the risk of stroke, oral anticoagulants are often prescribed. However, oral anticoagulants increase the risk of major bleeding. Therefore, cardiologists need to balance the risks and benefits of anticoagulant therapy for every patient individually. The European Society of Cardiology recommends in its clinical practice guideline for the management of atrial fibrillation to perform the risk-benefit analysis by using the CHA2DS2-VASc and HAS-BLED risk stratification schemes. These risk stratification schemes indicate stroke and bleeding risk scores, based on risk factors present in atrial fibrillation patients, supporting the decision regarding anticoagulant therapy.
A clinical decision support system, automating the calculation of both risk scores and advising on anticoagulant medication choices, was developed and integrated into the electronic health record software used by cardiologists during patient consultations. A randomized controlled study design was chosen to evaluate the effectiveness of the clinical decision support system in terms of guideline adherence. Guideline adherence was defined as a Boolean variable, and assessed based on appropriate anticoagulant therapy. Since the trial had not been finalized at the time of writing, simulated data were used for the statistical analyses. Results indicated a significant improvement in guideline adherence when comparing the control and intervention group.
Furthermore, characteristics of cardiologists associated with guideline adherence were examined using logistic regression analysis. In univariate analysis, age, gender and location of the cardiologist showed a significant univariate association with guideline adherence. The multivariate analysis revealed that guideline adherence was associated with the location of employment of the cardiologist and the gender of the cardiologist.
7
SAMENVATTING
Atriumfibrilleren is een hartritestoornis, waarbij het hartritme onregelmatig en meestal versneld is. Atriumfibrilleren verhoogt het risico op een cerebrovasculair accident, en om dit risico te verlagen worden vaak antistollingsmiddelen voorgeschreven. Door het gebruik van antistollingsmiddelen is de kans op een cerebrovasculair accident lager, maar lopen patiënten een verhoogd bloedingsrisico. Hierdoor is het van belang dat cardiologen de voor- en nadelen van het gebruik van antistollingsmiddelen per patiënt zorgvuldig afwegen. De European Society of Cardiology adviseert hiervoor het gebruik van de risicoanalyse schema’s CHA2DS2-VASc en HAS-BLED. Deze schema’s berekenen het risico op een cerebrovasculair accident en op een bloeding aan de hand van risicofactoren aanwezig bij een bepaalde patiënt.
Een klinisch beslissingsondersteunend systeem werd ontwikkeld en geïntegreerd in de software van het elektronisch patiëntendossier. Het systeem berekent de risico scores en geeft een advies betreffend antistollingsmiddelen wanneer een patiënt met atriumfibrilleren door de cardioloog behandeld wordt. Om te onderzoeken in hoeverre het gebruik van een klinisch beslissingsondersteunend systeem de naleving van de richtlijn voor atriumfibrilleren bevordert, werd voor een gerandomiseerd onderzoek met controlegroep als onderzoeksopzet gekozen. De naleving van de richtlijn werd gemeten in gepastheid van het voorschrijven van antistollingsmiddelen. Aangezien het onderzoek ten tijde van schrijven nog niet afgerond was, zijn de statistische analyses uitgevoerd op basis van gesimuleerde data. De resultaten tonen een verbetering in naleving van de richtlijn in de vergelijking tussen controle en interventie groep.
Tevens zijn kenmerken van cardiologen die van invloed kunnen zijn op het naleven van klinische richtlijnen onderzocht door middel van logistische regressie. Leeftijd van de cardioloog, geslacht en locatie waren significant van invloed in univariabele logistische regressie analyse. In multivariabele logistische regressie analyse was dit het geval voor locatie en geslacht van de cardioloog.
8
KEYWORDS
Clinical Practice Guideline Adherence, Clinical Decision Support System, Atrial Fibrillation, Randomized Controlled Trial, Physician Characteristics
ABBREVIATIONS
AbbreviationAF Atrial Fibrillation
CDSS Clinical Decision Support System CI Confidence Interval
CPG Clinical Practice Guideline
CVZ College voor Zorgverzekeringen (Dutch Healthcare Insurance Board) DBC Diagnose Behandel Combinatie (Diagnosis Treatment Combination) DHD Dutch Hospital Data
DM Diabetes Mellitus DSS Decision Support Systems EHR Electronic Health Record ESC European Society of Cardiology GP General Practitioner
HSMR Hospital Standardized Mortality Ratio INR International Normalized Ratio LVEF Left Ventricular Ejection Fraction MeSH Medical Subject Heading
NHG Nederlands Huisartsen Genootschap (Dutch General Practitioner Association) NOAC New Oral Anticoagulant
NVVC Nederlandse Vereniging voor Cardiologie (Dutch Society of Cardiology) OAC Oral Anticoagulant
OR Odds Ratio
RCT Randomized Controlled Trial TIA Transient Ischemic Attack VKA Vitamin K Antagonist
9
1 INTRODUCTION
Atrial fibrillation (AF) is the most common cardiac arrhythmia worldwide1, characterized by uncoordinated atrial activation with impairment of the atrial mechanical function as a consequence2. Furthermore, AF is often associated with burdening symptoms and socioeconomic problems such as permanent disability, cognitive impairments, hospitalization and absence from work2. Additionally, AF is related to substantial morbidity and mortality3 and has several independent risk factors like gender, hypertension, diabetes, heart failure and valvular heart disease1.
The incidence of AF increases with age, rising above 5% in people older than 651, which may be explained by population ageing due to rising life expectancy, the increasing amount of comorbidities with age, and other factors such as lifestyle changes4. AF is seen as a significant public health burden5 and research predicts that the number of patients with AF is likely to increase 2.5-fold during the next 50 years in the Unites States6.
Overall prevalence of AF in the Dutch population over 55 years is 5.5%, corresponding to about 250,000 AF patients with approximately 45,085 new AF patients each year7. According to estimates by Heemstra et al.7, the total cost of AF in 2009 in the Netherlands was € 583 million, the majority (70%) of which was caused by hospitalizations and in-hospital procedures. In 2009, the pharmacotherapeutic management of AF in the Netherlands was estimated at € 17 million7. Heemstra et al.7 conclude that improved medicinal treatment of risk factors, AF itself, and of stroke in patients with AF, will lead to a minor increase in the cost of treatment, but at the same time to a significant reduction in serious events and related hospitalizations. Therefore, prescribing appropriate medication for AF patients is an important step to take in order to reduce the burden of AF.
Clinical Practice Guidelines (CPGs) represent a meaningful effort to offer healthcare professionals guidance in the management of clinical conditions8, and following CPGs should lead to improved quality of care and patient outcomes9. However, some CPGs are better adhered to in practice than others10. The differences in adherence could be due to several factors such as the type of health problem addressed, the method of guideline development used, the content of the recommendations, the source of dissemination of the guideline, or the format and layout11. Furthermore, research suggests that physician, patient and facility factors may be associated with differences in guideline adherence as well12,13.
10
CPG adherence in the Netherlands varies widely, as identified by Bloemendaal et al.14 in a systematic review of 91 relevant studies concerning the adherence to Dutch CPGs. The identified adherence variation can be illustrated by the example of patients suffering from anxiety disorders or depression, of which only 38% received care as specified by the CPG, and breast cancer patients receiving a diagnostic sonography, which was in accordance with the CPG in 94% of the cases14. It is important to notice that due to differences in measuring methods of guideline adherence and the ambiguity of CPGs themselves, the comparison of adherence percentages cannot be made one-to-one14.
One of the reviewed studies, the EXAMINE-AF study15, compared the adherence to antithrombotic treatment prescription guidelines in patients with AF between general practitioners (GPs), internists and cardiologists in the Netherlands, and identified a rate of appropriate antithrombotic treatment of 67–72% patients. Dinh et al.15 found that guideline adherence was best for cardiologists when compared to GPs and internists (70% vs. 58% and 55% respectively). Furthermore, the study revealed an underutilization of oral anticoagulants in patients with AF, since only 22–62% of eligible AF patients were prescribed with oral anticoagulants15. A similar trend was identified in Canada, where a gap between the usage of evidence-based dosing methods of warfarin and clinical practice was observed16. Especially GPs often managed warfarin treatment solely based on their own experience16, neglecting guideline recommendations.
Underutilization of oral anticoagulation medication in patients with AF is a common problem17. According to Dinh et al.15 important reasons for physicians to withhold oral anticoagulation for certain patients, although indicated by the CPG, are advanced age, residing in rural areas, expected non-compliance of the patient, (presumed) increased risk of hemorrhage, patient preferences and patient health believes15.
Further research regarding general barriers to adherence of CPGs was performed by Cabana et al.18. They identified barriers to physician adherence to CPGs and created a framework based on these barriers according to their effect on physician knowledge, attitudes, or behavior. Barriers to adherence concerning the knowledge of the physician can be related to a lack of familiarity or a lack of awareness of the CPG, caused by guideline length, time needed to stay informed, or guideline accessibility. Attitude barriers may be caused by different attitudes of the physician, such as a lack of agreement with specific guidelines or guidelines in general, a lack of outcome expectancy, self-efficacy, or motivation, all causing poor guideline
11
adherence by the physician. Barriers concerning the behavior are external barriers, which can be guideline related, patient related or environmental18.
Lugtenberg et al.19 used the framework of Cabana et al.18 to assess the influence of barriers to guideline adherence in the Netherlands by performing a qualitative study with focus groups. The study identified that the barriers most perceived by clinicians were lack of agreement with recommendations, environmental factors such as organizational constraints, lack of knowledge regarding the guideline recommendations, and guideline factors such as unclear or ambiguous guideline recommendations19. One of the assessed CPGs was the AF guideline, where barriers concerning environmental factors, guideline factors and lack of agreement were applicable to 50-100% of its key recommendations19. Goud et al.20 demonstrated the effect of computerized decision support on barriers to guideline adherence by conducting a qualitative study in outpatient cardiac rehabilitation. The results showed that the implemented decision support system increased the familiarity of CPGs by reducing guideline complexity and inertia to previous practice20. The studies by Lugtenberg et al.19 and Goud et al.20 suggest that the implementation of a decision support system for the treatment of AF could reduce barriers to guideline adherence, especially those related to guideline factors. Furthermore, improved guideline adherence could lead to an improved utilization of oral anticoagulants in patients with AF.
The effectiveness of clinical decision support systems (CDSSs) in terms of improved guideline adherence has been investigated in various attempts. Hicks et al.21 examined the effectiveness of a CDSS for blood pressure management, and identified that the CDSS significantly improved Joint National Committee guideline adherent medication prescribing compared to usual care. Nilasena et al.22 evaluated a decision support system which reminded physicians of recommendations for diabetes care. The study found significantly improved guideline adherence rates. However, research by Wilson et al.23 found no significant improvement. The decision support system in the form of electronic alerts for acute kidney injury did not improve guideline adherence.
The main goal of this thesis is therefore to investigate whether the implementation of a CDSS for AF improves guideline adherence in terms of anticoagulant medication prescription after determining the risk of stroke (CHA2DS2-VASc score) and bleeding (HAS-BLED score). Based on the above the following main research question is formulated: “To what extent does the usage of a CDSS improves the adherence to the ESC Guidelines for the management of AF in terms of anticoagulant medication prescription?”. A secondary goal of this thesis is to obtain insights in
12
factors associated with guideline adherence related to the treating cardiologist, leading to the secondary research question: “What cardiologist characteristics are associated with adherence to the guideline for AF in terms of anticoagulant medication prescription?”
In order to answer both research questions, a multi-center randomized controlled trial (RCT) was executed in three nonacademic hospitals in the Netherlands, where cardiologists used the developed CDSS in daily practice. A baseline adherence measurement was performed using retrospective data of patients with AF treated in one of the three hospitals. These data were used to determine physician characteristics associated with guideline adherence. To answer the secondary research question multivariate logistic regression analysis was used.
In the first chapter background information is provided regarding the relevant concepts of this thesis and on related research performed on the topic of AF, CDSSs and guideline adherence. The second chapter describes the CDSS used to investigate the effect on guideline adherence. In the third chapter the methodology for the RCT and the multivariate logistic regression analysis is established. Results are reported in the fourth and discussed in the fifth chapter. Furthermore, a conclusion is drawn, references are provided, and supporting information and evidence is provided in the appendices.
13
2 PRELIMINARIES
In the following sections, background information for the key concepts clinical practice guidelines, clinical decision support systems and atrial fibrillation is given. Furthermore, a literature study was performed to extract information on related work, which is reported here.
2.1 CLINICAL PRACTICE GUIDELINES
Clinical Practice Guidelines, abbreviated as CPGs, are defined by Field et al.24 as “systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances.” Twaddle25 extends this definition by stating that CPGs are designed to help practitioners gather, evaluate and implement evidence regarding best current practice in medicine.
According to Field et al.24 and Berg et al.26 CPGs can improve the quality, appropriateness, and cost-effectiveness of healthcare, and can serve as valuable educational tools. A systematic review on the effect of Dutch CPGs on the process and structure of care and on patient outcomes was conducted by Lugtenberg et al.27, aimed at assessing the effectiveness of Dutch CPGs at improving quality of care. The study indicated that Dutch CPGs can be effective at improving the process and structure of care27. However, the effects of CPGs on patient outcomes were less studied and results were less convincing27. Therefore, although overall the reviewed studies demonstrated that there is evidence for the effectiveness of Dutch CPGs, the relation between quality of care and guideline adherence needs further investigation.
In the Netherlands, GPs use the CPGs developed by the Dutch College of General Practitioners (Nederlands Huisartsen Genootschap, NHG), whereas specialists are expected to use the CPGs endorsed by the association of their specialism. In the case of AF, the Netherlands Society of Cardiology (Nederlandse Vereniging voor Cardiologie, NVVC) endorses the guidelines developed by the European Society of Cardiology (ESC).
2.2 CLINICAL DECISION SUPPORT SYSTEMS
The term clinical decision support system (CDSS) is defined as software that is designed to be a direct aid to clinical decision-making28. CDSSs are a subgroup of decision support systems (DSSs), which can be classified based on the relationship between the user and the system29 and the mode of assistance30.
14
The relationship between the user and the DSS can be passive, active or cooperative29. Passive systems aid the user without providing explicit suggestions or solutions for the decision to be made, whereas active systems provide solutions or suggestions to the user29. Cooperative systems aim at a consolidated solution through interactive refinement of the solution between the DSS and its user29.
The different modes of assistance, as identified by Power30, are communication-driven, data-driven, document-driven, knowledge-driven, and model-driven. Communication-driven DSSs support multiple users on a shared task, and therefore support the communication to reach a decision31. Data-driven DSSs focus on providing access to and manipulation of data. Document-driven DSSs are aimed at managing, retrieving, and manipulating information in different electronic formats. Knowledge-driven DSSs provide problem-solving expertise in the form of facts, rules, procedures, or similar structures. The last category, model-driven DSSs, emphasizes access to and manipulation of statistical, financial, optimization, or simulation models30.
According to Musen et al.32 CDSSs can be subdivided into three different types, ranging from generalized to patient specific, as illustrated by Figure 1. The three types are tools for information management, for focusing attention, and for providing patient-specific recommendations. Musen et al.32 define information-management tools as CDSSs providing the data and knowledge needed by clinicians for making a clinical decision. However, these tools do not help clinicians to apply the information to a certain decision task, the interpretation of the information has to be done by the user32. Tools for focusing attention are CDSSs that support the attention of the user, e.g. by reminding clinicians of diagnoses, problems, or interactions32. Tools for providing patient-specific recommendations are CDSSs that propose recommendations or advices based on patient-specific data32.
Figure 1: Different types of CDSSs according to Musen et al.32
2.3 ATRIAL FIBRILLATION
Atrial fibrillation (AF), is the most common Cardiac arrhythmia, and the current estimate regarding the prevalence of AF in the developed world is approximately 1.5–2% of the general
15
population3. AF is an important risk factor for ischemic stroke, increasing the risk of stroke five-fold and accounting for approximately 15% of all strokes33. The average age of patients with AF is steadily rising, and now averages between 75 and 85 years3. The costs of AF are high and driven by the consequences of AF-related complications such as strokes, cost of hospitalizations of AF patients, and loss of productivity34.
Several treatment options for AF have been developed in recent years. In general, there are two main approaches: cardioversion and treatment with antiarrhythmic drugs to maintain sinus rhythm, and the use of rate-controlling drugs, allowing AF to persist35. With both approaches, anticoagulant drugs are recommended3. Anticoagulants are a class of drugs that prevent coagulation of blood3. An example of anticoagulants are vitamin K antagonists (VKAs), with a prominent member being warfarin36. Newer attempts are direct factor Xa inhibitors and direct thrombin inhibitors, belonging to a group of medications called new oral anticoagulants (NOACs)37.
Figure 2: Choice of anticoagulant medication based on ESC guideline of 20123
Anticoagulation is determined based on the stroke and bleeding risk assessment of patients with AF by using the CHA2DS2-VASc and HAS-BLED schemes. The CHA2DS2-VASc scheme is a refinement of the CHAD2 scheme (congestive heart failure, hypertension, age over
16
75, diabetes, previous stroke or trans ischemic attack (TIA)), introduced by Gage et al. in 200138. For the CHA2DS2-VASc scheme, additional stroke risk factors (vascular disease, age 65-74, and female gender) are included. In a systematic review and meta-analysis performed by Chen et al.39, similar clinical utility of the CHADS2 and CHA2DS2-VASc scores in predicting stroke and thromboembolism was identified. However, the CHA2DS2-VASc scheme has the important advantage of identifying extremely low-risk patients with AF39.
For every risk factor of the CHA2DS2-VASc scheme, the patient receives zero, one or two points, depending on whether the risk factor is present40. The risk factors and the associated risk score are presented in Table 1. The ESC recommends prescribing anticoagulant medication whenever the CHA2DS2-VASc score is greater than or equal to 1, with the exception of women with lone AF. The process of selecting anticoagulant medication based on the CHA2DS2-VASc and HAS-BLED scores are presented in Figure 2.
CHA2DS2-VASc Risk Score
Congestive Heart Failure or Left Ventricular Dysfunction, LVEF ≤ 40% 1
Hypertension 1
Age ≥ 75 2
Diabetes mellitus (DM) 1
Stroke / transient ischemic attack / thromboembolism 2
Vascular Disease 1
Age 65-74 1
Sex Category (female) 1
Table 1: CHA2DS2-VASc risk stratification scheme37
HAS-BLED Risk Score
Hypertension 1
Abnormal renal and liver function (1 point each) 1 or 2
Stroke 1
Bleeding 1
Labile International Normalized Ratio (INR) 1
Elderly (e.g. age > 65) 1
Drugs or alcohol (1 point each) 1 or 2
Table 2: HAS-BLED risk stratification scheme37
The HAS-BLED bleeding risk–prediction scheme is used to estimate the 1-year risk for major bleeding of patients on anticoagulant medication. The HAS-BLED scheme represents common bleeding risk factors and assigns one or two points for the presence of each41. In a study conducted by Apostolakis et al.42 the predictive performance of bleeding risk–estimation schemes was assessed. The risk-estimation schemes compared were HAS-BLED, HEMORR2HAGES and ATRIA. The study found that the HAS-BLED scheme demonstrates
17
superior performance compared to ATRIA and HEMORR2HAGES42, and represents the recommended way to assess the bleeding risk of patients by the AF guideline3. The included bleeding risk factors are presented in Table 2.
2.4 RELATED WORK
To provide an overview of the research performed on CDSSs for AF, a search strategy was developed to identify relevant research articles published. The search strategy was executed using PubMed, and consisted of two search domains: AF and CDSSs. Both domains included several text word search terms and a Medical Subject Heading (MeSH), and were connected with an AND operator. The search query is included in Appendix A.
The goal of the literature study was twofold. Firstly, characteristics of developed CDSSs had to be determined. This was conducted using the Framework for Classifying Decision Support Systems introduced by Sim et al.43. The second goal was to identify characteristics of related studies to get an overview of relevant research performed on the subject of CDSSs and AF. This overview included characteristics such as methods, study type, participants, and results.
The executed search resulted in thirty-five articles, of which four were inaccessible or not in English. The remaining articles were analyzed based on title and abstract information, and excluded if considered irrelevant. Excluded articles and the reasons for their exclusion can be found in Appendix B.
For the first goal of the literature study all articles describing a CDSS, regardless of the study type, were included, resulting in eleven unique systems described in nineteen articles. For the second goal articles were only included when describing a study evaluating a CDSS, resulting in nine unique systems described in fifteen articles. The article selection procedure is illustrated in Figure 3.
Table 3 provides a short summary of all CDSSs resulting from the literature search, along with the author of the respective article. Furthermore, the table indicates whether an article was only used for the classification of the CDSS, or for the study overview as well.
18
Figure 3: Selection process of articles resulting from systematic literature search for determination of CDSS characteristics (left) and identification of related research (right)
Study# Author (Year) CDSS Description C la ss ifi ca ti o n St udy O v erv ie w 1 Abidi et al.44 (2013)
COMET is a CDSS which is used to help gather relevant clinical data and to provide evidence-based recommendations for the management of patients with co-morbid congestive heart failure and AF.
X X
2 Arts et al.45
(2013)
The developed CDSS is used by GPs to assess the risk and the benefits of antithrombotic therapy in patients with AF. The system displays a list with pending messages for the on-screen medical record regarding medication advices.
X X
3 Bajorek et
al.46 (2012)
CARAT is a decision support tool used to support the decision-making process and the risk benefit assessment performed by physicians for an individual patient. The application flags medication management issues, and quantifies the risk of a patient for stroke and bleeding.
X X
4 Bajorek et
al.47 (2014) See Bajorek et al.46 X X
5
Borgman et al.48 (2012)
PerMIT is a CDSS that calculates estimated loading and maintenance doses of warfarin based on a patient's genetic and clinical characteristics to improve warfarin management.
X X
6 Eckman et
al.49 (2014)
AFDST is a CDSS which is used retrospectively. For every AF patient the QALE is calculated for three strategies: no antithrombotic therapy, aspirin, and oral anticoagulation therapy. The recommendation of the CDSS is based on the strategy resulting in the largest expected QALYs.
X X
7 Fraenkel et
al.50 (2011)
The purpose of the developed CDSS is to inform the patient regarding their individual stroke and bleeding risk, to assist them in clarifying priorities, and to promote communication between physician and patient.
X X
8 Fraenkel et
al.51 (2012 See Fraenkel et al.50 X X
9 Heaven et
al.52 (2006) See Thomson et al.53 X
19 Study# Author (Year) CDSS Description C la ss ifi ca ti o n St udy O v erv ie w
al.54 (2012) improve shared-decision making for AF (among other diseases). The tool helps to define the problem, discuss the individual risks, discuss the treatment options, and choose a treatment.
11 Johnston et
al.55 (2003)
The developed web CDSS uses a regression model to predict the gain in QALYs for an individual patient following a certain antithrombotic treatment, based on the entered values in the tool.
X
12 Jones et
al.56 (2005)
INRStar is a CDSS focusing on antithrombotic treatment using the INR values of the patient. Physicians can access patient data and get recommendations regarding antithrombotic treatment based on the INR of the patient. Furthermore, the CDSS determines the INR measurement appointments needed for a patient.
X
13 Kaner, et
al.57 (2007) See Thomson et al.53 X X
14 Murtagh et
al.58 (2007) See Thomson et al.53 X
15
Robinson et al.59 (2000)
See Thomson et al.53 X
16
Thomson et al.53 (2002)
The DARTS tool uses the standard gamble method to derivate patient’s values for relevant health states associated with stroke and treatment with warfarin. Furthermore, for individual patients the risk information using the Framingham stroke risk equation is shown. Additionally, the Markov decision model component of the tool helps clinicians and patients to reach a shared decision.
X X
17
Thomson et al.60 (2007)
See Thomson et al.53 X X
18 Wess et
al.61 (2011) See Wess et al.62 X X
19 Wess et
al.62 (2008)
AF-DST provides patient-specific information on the risk-benefit tradeoff of anticoagulation. The decision model estimates the risk for both ischemic stroke and major bleeding event. The result of the decision module is the number of quality adjusted life years (QALTs) gained or lost through treatment of warfarin.
X X
Table 3: Summary of CDSSs described in articles resulting from systematic literature search
2.4.1 CLASSIFICATION OF CDSS
The Framework for Classifying Decision Support Systems, introduced by Sim et al.43 and applied by Berlin et al.63 was used to categorize the CDSSs according to their specific characteristics. The framework of Sim et al.43 consists of twenty-four axes, grouped into five different categories: Context, knowledge and data source, decision support, information delivery, and workflow.
For every selected CDSS, the twenty-four axes were evaluated based on the information provided by the articles. Some axes had identical values for all selected CDSSs such as the unit
20
of optimization, update mechanism, clinical urgency, delivery mode, and output intermediary. An explanation of these axes, along with the assigned values is presented in Table 4.
The values for the other axes of the classification by Sim et al.43 varied among the selected CDSSs. Some significant differences between CDSSs are mentioned in this section. The complete classification is included in Appendix C.
Axe Explanation based on Sim et al.43 Value for all CDSSs
Unit of optimization
The CDSS may be designed to support patient outcomes, or
physician/organizational outcomes. Patient outcomes
Update mechanism The evidence base of a CDSS should be kept up-to-date. This can be
done automated, manual, or not at all. Unknown
Clinical urgency
CDSSs may provide decision support for decisions that need to be made imminently, where other systems support less urgent decisions.
Not urgent
Delivery mode
Delivery can be in a pull manner (target decision maker requests a recommendation), or in a push manner (the system automatically delivers recommendations).
Pull
Output Intermediary
An individual who handles or manipulates the information generated by the CDSS before it is viewed by the target decision maker.
None
Table 4: Axes of the framework for classifying CDSSs by Sim et al.43 where identical values were found
Clinical Setting
Only the CDSS described by Eckman et al.49 did not focus on an outpatient setting, which includes GP practices, and had no direct relationship to a healthcare entity. This is caused by the fact that the CDSS was developed for research purposes, and not for daily use in healthcare. The other CDSSs all focused on an outpatient setting.
Clinical Task
An important division of the selected CDSSs is the clinical task: Seven of the selected systems focused on antithrombotic management45–49,55,56,61,62, three focused on informing the patient as a part of shared decision-making50–54,57–60, and one of the systems was designed to support the management of comorbidity44.
Clinical Knowledge Source
All selected CDSSs were evidence-based, meaning that the knowledge base of the system was created with clinical evidence, and was not primarily based on clinician-developers or on participation of the system its eventual users. However, some of the systems explicitly stated
21
that the knowledge source were CPGs44,45,49, whereas others only specified that the knowledge was evidence-based.
Data Source & Data Source Intermediary
The data source varied widely among the selected CDSSs. Some of the systems used patient consultation data, whereas others imported the data directly from the electronic health record (EHR) of the patient, or used a patient database to obtain the data needed. Input was mainly given by the physician or clinician44–47,50,51,54–56,61,62. However, some of the CDSSs received their input from non-clinicians or patients48,49,52,53,57–60.
Response Requirement
Some systems required a response of the user of the system regarding whether the recommendation was followed by the user48,54,56, or which action was taken after the recommendation was made46,47,52,53,58–62. Other systems did not require a response at all49–51,55, whereas one system only required a response in a certain control group of the study45.
2.4.2 CHARACTERISTICS AND RESULTS OF PERFORMED STUDIES
In order to give an overview of research performed on the subject of AF and CDSSs, Table 5 is included. The table presents the study type, provides information about key methodological points, shows the number and kind of participants of the study, and sums up the main findings of the study. Two studies45,47 included in Table 5 are study protocols, therefore no results were available at the time of writing.
As shown in Table 5 different aspects of CDSSs for AF were evaluated such as the usability of the system, the effectiveness of the system in terms of improving guideline adherence, correct choices in AF treatment, and shared decision-making. This study focused on the evaluation of the CDSS in terms of effectiveness regarding guideline adherence by performing an RCT, and is comparable to the study protocol of Arts et al.45. However, no results of similar research are available yet.
Study# Study Type Methods Participants Results
1 Usability
Evaluation
Non-experimental post-interaction think aloud sessions and a survey (Likert-scale post study questionnaire)
10 GPs
Common usability problems encountered are related to inadequate information content, navigation, and time and effort for data entry
2 Cluster RCT
3 study groups were established: A control group, an intervention group 1 (a message can be ignored without
500 patients within 18 clusters of 35
22
Study# Study Type Methods Participants Results
justification), and an intervention group 2 (a message can only be ignored with justification). The study focused on the calculation of CPG adherence rates.
GPs
3 Usability
Evaluation Usability questionnaire 27 clinicians
Overall, clinicians were satisfied with CARAT’s format, and most clinicians agreed with CARAT recommendations. 63% responded that CARAT was at least ‘somewhat useful’ for clinical practice.
4 Cluster RCT
(prospective)
There were 2 study groups, an intervention group and a control group. The GPs in the intervention arm use CARAT during routine patient consultations. Outcomes measured are changes in therapy, difference in proportion of patients receiving no therapy, and differences in proportion of patients receiving antithrombotic therapy
25 GP’s in the intervention arm, 25 GP’s in control arm, treating 500 patients in total
None (Study Protocol)
5
Randomized Pilot Trial (prospective)
There were 2 study groups: A PerMIT guided treatment group, and a routine anticoagulation service management group. Measures were the time to first stable therapeutic INR, stable maintenance dose and time to stable therapy, time in range, frequency of dosage adjustments 13 clinicians in PerMIT study arm, 13 clinicians in control group
A decrease in the time to reach a stabilized INR within the therapeutic range was measured, along with an increase in time spent within the therapeutic interval over the first 25 days of therapy and a decrease in the frequency of warfarin dose adjustments per INR measurement
6 Retrospective
Cohort Study
Projection for quality-adjusted life expectancy reported as quality-adjusted life years are calculated for different treatment options and compared to actual treatment
1876 adults with AF
Current treatment was different in 931 patients than suggested by the decision support tool. A significant gain in quality-adjusted life expectancy was projected in 931 patients.
7 Pilot test
A Questionnaire was administered regarding ease of completion of the CDSS, the amount of information provided, and if the user would recommend the tool.
11
participants
In pilot testing with the participants, 8 rated ease of completion as “very easy,” 9 participants rate amount of information as “just right.”
8 Pilot RCT
The 100-point Informed and Values Clarity subscales of the Decisional Conflict Scale were used. Furthermore, knowledge, patient-clinical
communication and change in treatment were assessed. 69 patients in the intervention group, 66 patients in the control group
The intervention group had lower scores on the Informed and Values clarity subscales. Knowledge on reducing stroke risk and side risks increased. Stroke and bleeding risk was more frequently discussed in the intervention group.
10 Qualitative
Evaluation Patient interviews and focus groups
29 GPs treating 192 patients
A positive association to the decision-making process in patients and physicians was
23
Study# Study Type Methods Participants Results
identified. Physicians seem to adapt their counseling strategy to different patient groups. Furthermore, prior experience with the use of decision aids has an influence on the acceptance of the CDSS
13 Video-based
process study
The study was executed within an efficacy trial53. A detailed content analysis using interaction analysis protocols for verbal behavior and ethological techniques for non-verbal behavior were conducted.
29
consultations
The consultation based on paper-based guidelines tool took 21 minutes to work through compared to 31 minutes for the implicit CDSS tool. GP’s dominated the conversation, and information giving was at higher rates in consultations involving computerized decision aids.
12 Feasibility
study
Consultations were audio/video taped. Furthermore, semi-structured interviews were conducted to identify how well nurses and physicians had executed the consultation according to patients. Patient’s understanding of the tool was assessed as well. As well as the confidence of the users in using the tool, and potential improvements of the tool.
4 GP’s, 3 practice nurses, treating 10 patients No patient expressed a dislike of the tool, most were very positive. 8 patients understood it well and reported no important problems with the tool.
16 Exploratory
RCT
The study group used the implicit computerized decision aid, the control group used evidence-based paper guidelines. The study measures decision conflict (decision conflict scale), anxiety, knowledge, decision-making preference, treatment decision, use of primary and secondary care services and health outcomes. 109 patients from 40 general practices (56 in control group, 53 in decision aid group)
Decision conflict was lower in de computerized decision aid group immediately after the clinic. Participants not already on warfarin were less likely to start warfarin than those in the control arm
18 Pilot Usability
Testing Study
Positive and negative critical incidents of simulations were recorded, and the CSUQ was used.
4 medical house officers, 4 attending physicians
6 positive critical incidents and 14 negative critical incidents were identified. Furthermore, high CSQU scores were measured.
19
Retrospective, observational cohort analysis
Cox proportional hazards models were used to compare the group of patients who received warfarin treatment with those who did not receive warfarin treatment stratified by the decision support tool’s recommendation.
6’123 patients
The tool recommended warfarin for 49% of the patients, but only 9.9% received warfarin. In patients for whom the tool
recommended warfarin and it was administered, a decreased hazard for stroke was identified.
24
3 INTERVENTION
In order to determine the effect of a CDSS on guideline adherence, a CDSS was developed and integrated into EHR software used by cardiologists during patient consultations. The system is provided in Dutch and the content base of the system is the guideline for the management of AF by the ESC of 201037 and its focused update of 20123. The CDSS focuses on the stroke risk stratification scheme CHA2DS2-VASc and the bleeding risk stratification scheme HAS-BLED, as explained in Section 2.3. The CDSS consists of the calculation of both risk scores, triggered by the entry of the diagnosis AF, and a medication advice based on the CHA2DS2-VASc score. Figure 4 illustrates the process flow of the CDSS.
Figure 4: Process flow of the CDSS for AF
Whenever the diagnosis AF or atrial flutter is added to the EHR of the patient by the cardiologist, the question is posed whether the CHA2DS2-VASc and HAS-BLED scores should be calculated. If the decision is made in favor of calculating the scores, both risk stratification schemes are shown and prefilled as much as possible, based on existing patient data in the EHR. The CDSS uses expressions with ICD-10 codes and codes of the thesaurus for diagnoses to
25
gather patient history information to prefill the schemes. Appendix D presents all the codes used to determine risk factors present in a patient.
If necessary, prefilled answers can be altered and unfilled questions can be answered by the cardiologists. The module calculates both risk scores automatically and provides the user with the adjusted stroke and bleeding rate associated with the specific patient. A screenshot of the system is added as Figure 5, presenting the two risk schemes.
Figure 5: Screenshot of the CDSS showing the risk schemes
Dependent on the CHA2DS2-VASc score calculated, a medication prescription recommendation is given. The system is able to display three different medication advices. The first advice is shown when the CHA2DS2-VASc is less than one and no medication should be prescribed. The second advice is shown in case of a CHA2DS2-VASc of one in women with lone AF. According to Olesen et al.64 and Friberg et al.65 female patients with gender alone as risk factor do not need anticoagulation as long as they fulfil the criteria of age < 65 and lone AF3. The third medication advice is triggered as soon as the CHA2DS2-VASc is greater than or equal to one. The advice indicates the prescription of a NOAC or OAC and to lower the dosage in case of renal insufficiency or advanced age. The medication advices and their associated conditions are presented in Appendix E.
All available NOACs and OACs are listed below the medication advice, and are irrespective of the medication advice given. The medications are listed as predefined
26
medication orders for standard and adjusted dosages. The medication advice and the predefined medication orders can be seen in the screenshot added as Figure 6.
When cardiologists choose a predefined medication order, if applicable, a screen with possible interactions with other medications and other concerns associated with the chosen medication is displayed. Additionally, the medication is added to the current medication list of a patient. After the process of calculating both risk stratification scores and prescribing the chosen medication, the CDSS returns to the EHR of the patient.
Figure 6: Screenshot of the CDSS for AF showing the medication advice and the predefined medication orders
3.1 CLASSIFICATION OF INTERVENTION
The system can be described using the classifications of Haettenschwiler29, Power30, and Musen et al.32, as introduced in Section 2.2. The relationship between the user and the CDSS is an active relationship since the system provides a medication advice based on the CHA2DS2-VASc risk stratification scheme, as shown to and answered by the cardiologist. The mode of assistance of the system is knowledge-driven due to the fact that the system provides an advice regarding medication prescriptions. It therefore provides problem-solving expertise in the form of a rule for anticoagulant medication prescriptions. Furthermore, the system is patient-specific, since the advices are based on patient-specific data and are tailored to the specific history of the patient.
The system was categorized using the classification introduced by Sim et al.43, as described in Section 2.4.1. Tables 6-10 provide the characteristics of the CDSS for all of the listed axes.
Context Axes Characteristics of CDSS
Clinical Setting Outpatient Setting (Patients visiting the cardiology outpatient clinic)
Clinical Task Antithrombotic therapy (The calculation of the risk scores and the
anticoagulation medication advice based on these scores)
Unit op Optimization Patient outcomes (the tool is designed to optimize the treatment of AF
27
Context Axes Characteristics of CDSS
Relation to Point of Care Synchronous (the tool supports the cardiologist during patient consultations)
Potential Barriers to Action -
Table 6: Context axes
Knowledge and Data Source Axes Characteristics of CDSS
Clinical Knowledge Source Guideline (CPG developed by the ESC for AF)
Data Source EHR and input during consult (If possible, data is retrieved from the EHR,
input during consultation is possible as well)
Data Source Intermediary Physician (the cardiologist is the data source)
Data coding Coded (data is entered in a coded way, free text is not possible)
Degree of Customization Customized (the advice is given based on patient data of the EHR and during
patient consultations)
Update Mechanism Manual (if new evidence is available, the tool has to be adapted manually)
Table 7: Knowledge and data source axes
Decision Support Axes Characteristics of CDSS
Reasoning Method Rules (conditions are integrated into the system, evaluated using EHR data
or patient consultation data)
Clinical Urgency Not urgent (the system is used during outpatient consultations, not in
emergency situations)
Recommendation Explicitness Explicit (the system indicates whether NOACs/OACs need to be prescribed)
Logistical Complexity of Recommended Action
Simple (medications can be added in a simple, direct way, based on the medication advice)
Response Requirement No response required
Table 8: Decision support axes
Information Delivery Axes Characteristics of CDSS
Delivery Format On screen CDSS (the recommendation is made within the EHR)
Delivery Mode Pull (the user is asked whether the calculations should be performed when
the AF diagnosis is entered)
Action Integration Integrated (the medication prescription is performed within the EHR)
Delivery Interactivity/Explanation
Availability No (additional information on the recommendation cannot be requested)
Table 9: Information delivery axes
Workflow Axes Characteristics of CDSS
System User Physician (the cardiologists uses the system)
Target Decision Maker Physician (the cardiologists is the target decision maker)
Output Intermediary None (nobody reviews the recommendation before the physician receives the
advice)
Degree of Workflow Integration Integrated (the system is integrated within the EHR, only a few single extra
steps are needed)
28
4 DATA AND METHODS
A multi-center RCT was conducted in three nonacademic hospitals in the Netherlands to assess the effect of a CDSS on guideline adherence in terms of appropriate anticoagulant medication prescription for AF patients. Baseline adherence of the hospitals was determined and physician characteristics associated with guideline adherence were investigated. This chapter describes the methodological aspects of the analyses.
4.1 DATA AND DATA COLLECTION
The databases of the selected hospitals were used to access the data needed on patients treated for AF. The data regarding medication prescriptions of patients for certain diagnoses were saved using a casemix system, which is called the Diagnose Behandel Combinatie (DBC) in the Netherlands. A DBC is a code to describe a care product for billing purposes66. A DBC is valid for one year, meaning that after the duration of a year, a new DBC is created for the patient for the same care product, if needed. When adding a diagnosis to the EHR of a patient, a DBC is created and the prescribed medication for the DBC is recorded.
The required data were collected by combining data columns from different database tables by using Structured Query Language (SQL). For every risk factor of the CHA2DS2-VASc score, its presence was determined, resulting in a table consisting of data columns indicating which risk factors were present for an AF patient. Several DBC diagnosis codes were used to determine the presence of a risk factor, which can be found in Appendix F.
Additionally, the query determined if anticoagulants were prescribed for the respective DBC before the end date of the DBC. Since DBCs can be valid for one year, not all registered DBCs had an end date recorded in the database. Therefore, for DBCs where no end date was available, medication prescribed within a year after creation of the DBC was analyzed regarding the presence of prescribed anticoagulants. The query used to extract the data can be found in Appendix G.
The CHA2DS2-VASc score was calculated based on the columns indicating the presence of risk factors and saved as a new column. Additionally, a column was added to the table, indicating whether the patient was female with lone AF. Based on the CHA2DS2-VASc score, taking into account female patients with lone AF, the recommended treatment in terms of anticoagulants was determined. In a different column, the actual anticoagulant treatment and
29
the recommended anticoagulant treatment were compared, resulting in a verdict regarding the appropriateness of treatment. A description of all data columns can be found in Appendix H.
For the measurement of baseline adherence only data from April 1, 2013 up to December 31, 2014 were extracted from the databases. The starting date was determined based on the publication date of the focused update of the ESC guideline for AF of 20123, which was published on the website of the Dutch Society of Cardiology (NVVC) on 21 March 2013. The same data were used to investigate cardiologist characteristics associated with guideline adherence.
For the RCT simulated data were used, since the trial had not been finalized at the time of writing. When the RCT will be completed, analyses will be conducted with the new dataset, and will replace results based on the simulated data.
4.2 GUIDELINE ADHERENCE MEASUREMENT
Descriptive statistics were used to outline the characteristics of the cardiologists and patients at baseline. Guideline adherence measurements were performed to determine the proportion of AF DBCs that were handled according to the ESC guideline.
The ESC guideline recommends anticoagulant therapy whenever the CHA2DS2-VASc is greater than one. For women younger than 65 with no other risk factors present, and for AF patients with a CHA2DS2-VASc score less than one, no anticoagulants should be prescribed. For every patient, the CHA2DS2-VASc score was determined, which was used to decide whether anticoagulants should have been prescribed. Whenever the system indicated that anticoagulants should be prescribed by the cardiologist, and the data indicated that NOACs or OACs were given to the patient accordingly, the treatment was judged as being appropriate. Treatment was also judged as being appropriate when no medication was given for patients with a CHA2DS2-VASc score less than one and for female patients with lone AF.
As a result of the baseline guideline adherence measurements, baseline guideline adherence for AF was determined. Baseline adherence was reported per hospital, per cardiologist, and as an overall adherence for all selected hospitals.
It was determined whether cardiologists and hospitals performed significantly different from each other by using the Chi-squared Test of Independence. If this was the case, pairwise comparisons were performed to identify the combinations of cardiologists or hospitals performing significantly different.
30
A required minimum of fifty AF DBCs treated was established for cardiologists to reduce the imbalance of the number of observations per cardiologist. Cardiologists not reaching the required minimum were excluded from the comparison between cardiologists, but included in the comparison of hospitals. The significance level of the Chi-squared test of Independence and the pairwise comparisons was determined at 0.05.
4.3 RANDOMIZED CONTROLLED TRIAL
The methodology used for the multi-center RCT is described in the following sections. To ensure that all important aspects of the RCT are clarified, the CONSORT statement for reporting randomized trials67 was used as a guideline. The RCT was chosen as study design since it is often considered as the gold standard for clinical trials. Although its position as gold standard is questioned by some68, it is still seen as a good experimental design, especially because RCTs are least affected by bias compared to other study designs69.
4.3.1 TRIAL DESIGN
A multi-center, unblinded, parallel group study was performed in three nonacademic hospitals in the Netherlands. Physicians were assigned to one of the two parallel groups per hospital with allocation ratio 1:1. Figure 7 shows the design of the trial.
Figure 7: RCT trial design
4.3.2 PARTICIPANTS AND FACILITIES
Subjects who were eligible to participate in the RCT were cardiologists working in one of the selected hospitals, treating patients at the cardiology outpatient department. All cardiologists work with identical EHR software. Patients visiting the cardiologist for AF were included in the
31
study and were either treated by cardiologists using the CDSS or by cardiologists to whom the system was not available.
In the Netherlands, GPs function as gatekeepers, which means that hospital care and specialist care are only accessible upon referral from the GP70. The cardiology outpatient departments are part of secondary care, and patients have been seen by the GP before visiting the cardiologist.
Table 11 provides information regarding capacity and resources of the selected hospitals. Furthermore, the Hospital Standardized Mortality Ratio (HSMR) is included. The HSMR serves as a tool for Dutch hospitals to analyze their death rates71. The HSMR is the ratio of the observed and expected mortality in a hospital. A 95% confidence interval (CI) is calculated for each HSMR, which is presented in the table.
Hospital A Hospital B Hospital C
Number of beds 323 (2011)72 385 (2013)73 472 (2013)73
Medical specialists 104 (2015)74 121 (2013)73 149 (2013)73
Number of cardiologists 5 5 5
HSMR 2013 (95% CI) 114 (97-133)75 103 (89-119)76 Unknown77
Table 11: Capacity and resources information of the participating hospitals
4.3.3 INTERVENTION
Cardiologists belonging to the intervention group were presented with the CDSS whenever the physician entered the diagnosis AF in the EHR. As explained in Chapter 3, the CDSS presented the risk stratification schemes for stroke and bleeding, a medication advice, and OACs and NOACs as predefined medication orders.
4.3.4 OUTCOME
The primary outcome of interest of the multi-center RCT was guideline adherence in terms of appropriate medication prescriptions after calculation of the CHA2DS2-VASc score, which was determined for each DBC created for AF. The primary outcome was used to answer the main research question. Improved guideline adherence in this context means a higher proportion of AF cases is treated appropriately in terms of anticoagulant medication prescriptions when comparing intervention and control group. To determine the appropriateness, the actual prescription was compared with the medication prescription recommended by the CPG for AF of the ESC3, as described in Section 4.2.
32 4.3.5 SAMPLE SIZE
To detect guideline adherence improvement, a sample size of 650 DBCs per group was necessary. To recruit this number of DBCs a three-month inclusion period was anticipated, based on baseline data indicating that approximately 325 DBCs for AF are registered per month in the three selected hospitals. Since the RCT was not finalized at time of writing, for the statistical analyses a simulated dataset was created with a sample of 7474 DBCs.
Power analysis for a Chi-square test was conducted in R to determine a sufficient sample size using an alpha of 0.05, a power of 0.95, an effect size of w = 0.1 and one degree of freedom. The power was determined at 0.95 in order to reduce the influence of type II error, which means to increase the probability of finding an effect that is actually there. The effect size was determined based on baseline adherence data and comparable research, as proposed by Chan78. Baseline data suggested guideline adherence in 27% of the registered DBCs. Observing an absolute guideline adherence improvement of 10% in the intervention group was determined as being clinically relevant based on comparable study design of Fitzmaurice et al.79 and Arts et al.45
4.3.6 RANDOMIZATION & BLINDING
Cardiologists were randomly assigned to the control or the intervention group by using an R script for randomization. The randomization was performed per hospital. Blocked randomization was used to ensure that comparison groups were generated according to the predetermined ratio of 1:1. Cardiologists and the researchers were aware of the allocated group, therefore the RCT was unblinded.
4.3.7 STATISTICAL METHODS
The primary outcome of the RCT was guideline adherence in terms of appropriate anticoagulant medication prescription. The control group and the intervention group had to perform approximately equal at baseline regarding the primary outcome, which was tested using the Pearson's Chi-squared test with Yates' continuity correction (𝜒𝑌𝑎𝑡𝑒𝑠2 = ∑ (|𝑓0−𝑓𝑒|−0.5)
2
𝑓𝑒
𝑘 )
with a significance level of 0.05.
The statistical analysis used to identify the effect of the CDSS on the primary outcome was the Pearson's Chi-squared test with Yates' continuity. The null hypothesis was that the occurrence of the appropriate medication prescription outcome for the control group and the
33
intervention group was equal, meaning that appropriateness of treatment was independent of the allocated group.
As effect size measures, the odds ratio (OR) and the Phi coefficient (φ) were used. The OR was calculated to show how the allocated group affected the probability of the outcome. An OR of one means that using the CDSS does not affect odds of positive outcome, and OR greater than one indicates that the exposure to the CDSS is associated with higher odds of positive outcome, whereas an OR of less than one means the opposite80. The 95% CI of the OR was determined. The CI was calculated from the log OR and backtransformed. For the effect size φ, the results are judged based on the following general rule: A value of 0.1 is considered a small effect, 0.3 a medium effect and 0.5 a large effect81.
4.4 LOGISTIC REGRESSION ANALYSIS
To determine physician characteristics associated with guideline adherence, logistic regression analysis was performed with baseline data from the selected hospitals. The included variables were analyzed in univariate and multivariate logistic regression analysis. The specifics of the analysis are described in the following sections.
4.4.1 HYPOTHESIS
The main null hypothesis of the multivariate logistic regression analysis was that there were no relationships between the dependent variable, which is appropriate medication prescription, and the independent variables. This means that the values which are predicted for the dependent variable from the multivariate logistic regression equation are no closer to the actual values of the dependent variable than one would expect by chance.
4.4.2 MEASURES
The dependent variable used for the regression analysis was guideline adherence in terms of appropriate medication prescription after the calculation of the CHA2DS2-VASc score. The independent variables investigated were age and gender of the cardiologist, the time since graduation from medical school, and the location at which the cardiologist was employed. The dependent and independent variables are included in Table 12 along with the related measurement level.
