clinical and methodological issues in a cost-utility analysis alongside a
randomized clinical trial in patients with rectal cancer undergoing
total mesorectal excision
Brink, Mandy van den
Citation
Brink, M. van den. (2005, June 28). Economic evaluation of preoperative radiotherapy in
rectal cancer : clinical and methodological issues in a cost-utility analysis alongside a randomized clinical trial in patients with rectal cancer undergoing total mesorectal excision. Retrieved from https://hdl.handle.net/1887/4273
Version: Corrected Publisher’s Version
License: Licence agreement concerning inclusion of doctoral thesis in theInstitutional Repository of the University of Leiden Downloaded from: https://hdl.handle.net/1887/4273
CHAPTER 9
CONCLUSI
ONS AND DI
SCUSSI
ON
CONCLUSIONS AND DISCUSSION
Cost-utility analysis
The purpose of this study was to compare the societal costs and the (quality-adjusted) life expectancy of patients with rectal cancer undergoing total mesorectal excision (TME) with or without 5 × 5 Gy preoperative radiotherapy (PRT). A Markov model was developed to project data on local recurrence rates, quality of life and costs, obtained from patients participating in a randomized clinical trial, to a lifelong perspective. The base case model estimated that PRT reduces local recurrence rates by 5.2%, thereby improving life expectancy and quality adjusted life years (QALYs) by 0.67 and 0.39 years respectively, but also increases costs by $9,800, resulting in a cost-effectiveness ratio of $25,100 per QALY. Sensitivity analyses showed that the cost-effectiveness ratio remained acceptable under a wide range of assumptions.
Comparability to other economic evaluations in colorectal cancer
In the literature review presented in chapter 2, 24 cost-effectiveness analyses (CEAs) on treatment and follow-up of colorectal cancer were found until 2001. Three CEAs had been conducted on primary surgery, five on adjuvant chemotherapy, eight on follow-up and eight on treatment of advanced diseases. No study investigated the cost-effectiveness of adjuvant radiotherapy. Ideally, adherence to methodology guidelines would allow for comparison of cost-effectiveness ratios over different health care interventions and patient groups (1;2). However, methodology ranged widely over studies, and also when compared to our cost-utility analysis of PRT. Most studies did not use the societal perspective and did not include health valuations from the general public or patients. For example, the estimated cost-effectiveness of PRT would have been much more favorable at $6,700 per QALY if we would have considered only the costs of initial treatment, and down to $4,300 per LY if we would also not have incorporated health valuations, thereby neglecting several adverse effects of PRT (e.g. hospital re-admissions, district nursing, side-effects).
aware of the resulting interpretation problems and try to more closely follow recommended guidelines (4).
Credibility of the survival difference
An important assumption in our Markov model is that a reduction in local recurrence rates by PRT will lead to a gain in survival. For the base case analysis, the model estimates that a reduction in the long-term probability of local recurrence by 5.2% will translate into a survival benefit of 2.8% and 3.2% after 5 and 10 years respectively. At a median follow-up time of 38 months the observed survival difference between randomization groups in the TME-study was 1% (p=0.75). The difference between modeled and observed survival is not only attributable to the different follow-up times, but also to the combined nonsignificant adverse effects of PRT on some of the observed mortality and recurrence rates, which were not included in our model to avoid type I errors.
Before judging the modeled survival benefit as incredible given the difference from observed survival, several other considerations need to be taken into account. First, the TME-study was designed and powered to detect a reduction of local recurrences from 10% to 5% by PRT. To detect the estimated 2.8% and 3.2% survival benefits at 5 and 10 years, sample sizes of 5,700 and 5,100 analyzable patients per arm would have been required (alpha 0.05, two-sided, power 0.90) (5). Therefore, longer follow-up of the TME-study may still not allow for more definite conclusions on whether there truly is a survival benefit. Second, the assumption that a reduction in local recurrence rates will result in a survival benefit was recently confirmed in two meta-analyses of PRT (6;7). Trials that found no or only small benefits in survival studied different treatment regimens and used small sample sizes. In addition, even when we assumed no survival benefit by PRT in our model, the gain in quality of life resulting from prevented local recurrences outweighed the loss in quality of life due to PRT, albeit less cost-effectively so than in the basecase analysis.
Generalizability of estimated cost-effectiveness
Our cost-utility analysis of PRT was performed as a piggyback study alongside a randomized clinical trial. Although this provides the best guarantee of internal validity for evaluating costs and effectiveness, clinical trials may have limited external validity or generalizability (8;9). External validity refers to the extent to which the results of the trial apply to other settings, e.g. other patient groups, other hospitals where quality of treatment may differ, and other countries.
CONCLUSIONS AND DISCUSSION showed acceptable cost-effectiveness. More likely are scenarios in which patients with worse prognosis undergo TME-surgery or in which the quality of surgery is less high than in the TME-study. This would result in higher local recurrence rates, which would increase the effectiveness of PRT without increasing the costs, thereby leading to more favorable cost-effectiveness.
For the generalizability of estimated cost-effectiveness to other countries, differential cost prices between countries are an important topic, apart from differences in patient groups and treatment procedures. There is no uniform standard for comparing cost prices between countries. An alternative to the exchange rate is the purchasing power parity (PPP). The PPP is a rate of currency conversion that aims to eliminate the difference in price levels between countries, but also has drawbacks, e.g. the PPP differs between economic sectors, and for health care, it is difficult to calculate a reliable PPP (12). In 2002, the general PPP of health care for conversion of Dutch to American price levels was 1.05 (13). The resulting CE-ratio based on PPPs would have been $26,400/QALY, only a slight change as compared to the reported CE-ratio of $25,100/QALY based on exchange rates. Since there is no direct solution for the problem of international comparability, researchers should also report the volumes of health care. This allows the interested readers to label their own national price tag to those volumes and determine the cost-effectiveness for their specific situation.
Implications for patients, doctors and policy makers
Economic evaluations provide information on the costs and consequences of health care interventions. This information may not only help policy makers in deciding how to allocate health care resources, but may also be valuable to doctors and patients, since all available data on clinical outcomes, e.g. quality of life, recurrence rates and survival are integrated as well, and, if relevant, extrapolated to a lifelong time horizon. Our results showed that PRT has a beneficial effect on local recurrence rates and associated survival, but also has disadvantages, e.g. quality of life, health care consumption, out-of-pocket costs. For the patient population as a whole, these disadvantages were too small to outweigh the benefits of PRT, but for an individual patient, the benefits may not outweigh the disadvantages, which may turn the scales towards a preference for no radiotherapy; consider for example an elderly patient who wishes to avoid the side-effects of PRT. The doctor need not feel obliged to offer PRT based on the estimated cost-effectiveness ratio. Although the cost-effectiveness ratio of $25,100 per QALY is generally considered acceptable, this does not mean that PRT is cost saving, but only that PRT on average provides good value for money.
CE-ratio would be more than $50,000 per QALY and $100,000 per QALY if the 5-year local recurrence risk without PRT is less than about 5% and 3% respectively. Thus, for subgroups of patients with a 5-year local recurrence risk without PRT of less than 5%, the cost-effectiveness of PRT is debatable, for subgroups of patients with a 5-year local recurrence risk without PRT of less than 3%, PRT may not be considered cost-effective.
For policy makers, the cost-effectiveness ratio is one of many decision criteria. Treatment with PRT provides good value for money, but also has an impact on the health care budget of $17,150,000 per year (assuming 1750 Dutch eligible patients with rectal cancer per year). With this investment 680 QALYs are gained. Some authors argue that it is the task of the researcher to determine not only the costs and benefits of treatments per patient, but also the opportunity costs incurred by other patient groups (14-17), in order to help the policy maker in redirecting resources. This presumes that cost-effectiveness data are available for all potential health care interventions, which is a rather idealized picture. Cost-effectiveness analyses are mostly performed for selected interventions that are controversial and may be potentially expensive. Data on cost-effectiveness are then one of the many pieces of information among for example legal considerations, considerations of distributive justice, and budgetary implications, which all must be weighed by policy makers. The ultimate responsibility for the distribution of health care resources lies with policy makers, who have received society’s mandate to do so. The task of the researcher is to properly inform the policy maker whether a particular treatment provides good value for money.
Clinical issues
Two clinical issues were addressed in this thesis. The purpose of the study presented in chapter 4 was to investigate the impact of PRT on paid and unpaid labour. Data were assessed prospectively in two samples of patients participating in the TME-study. From 3 to 18 months after surgery, paid labour resumption increased from 19% to 63%. Irradiated patients tended to resume paid labour later than non-irradiated patients. From 18 months on, paid labour resumption did not differ between randomization groups. The average amount of unpaid labour increased from 17.3 hours per week at 3 months to 21 hours per week at 12 months after TME-surgery, and did not differ by randomization group.
The purpose of the study presented in chapter 5 was to document the clinical nature and prognosis of locally recurrent rectal cancer. For all Dutch patients with a local recurrence in the TME trial, data on treatments and follow-up were gathered from medical specialists. Patients with a local recurrence who had previously been treated with PRT for the primary tumor had distant metastases more often (74%) than patients without PRT (40%). Median survival after local recurrence in the PRT+TME group was only 6.1 months as compared with 15.9 months in the TME group.
Productivity losses in the cost-utility analysis
CONCLUSIONS AND DISCUSSION losses were included only for a ‘friction’ period of 4 months, which is the estimated time needed to find a replacement (18). The observed difference in paid labour between randomization groups mainly occurred between 3 and 18 months after surgery, which was thus not included in our base case model. The cost-effectiveness of PRT would have been less favorable, especially for younger ages, if we would have included productivity losses for longer periods of time, because of the lower paid labour participation in irradiated patients. In the cost-utility analysis presented in chapter 3 we reported a trend towards less unpaid labour in irradiated patients. In chapter 4 we concluded that there was no significant difference in unpaid labour between randomization groups. This discrepancy illustrates some of the difficulties in the assessment and integration of cost data from a clinical trial. Several years after the start of the TME-trial, the cost-utility analysis was added on as a piggyback study. Many of the relevant costs could therefore only be assessed in small samples and with limited follow-up. In addition, costs typically have large variations (19). Therefore, the decision on the inclusion of differences in costs between randomization groups in the Markov model was not only based on statistical significance, but also on observed general trends and the plausibility of long-term adverse effects of PRT (20). Several costs, including those of unpaid labour, were lifelong extrapolated, at the expense of PRT, based on observed (nonsignificant) differences at one year of follow-up. Sensitivity analyses showed that the cost-effectiveness ratio would be much more favorable, at $14,500 and $9,500 per QALY, if differences in costs and transition rates between randomization groups were assumed to end after 10 and 5 years respectively. Longer follow-up of cost data alongside the TME-trial, preferably in larger samples, could have provided more information on the credibility of the assumptions used in the Markov model.
Follow-up after local recurrence
Methodological issues
The purpose of three studies in this thesis was to investigate the feasibility and convergent validity of different measurement methods for the assessment of costs and utilities: (1) a within-subjects study was carried out on the comparability of patients’and providers’reports of health care utilitization, (2) patients’reports of health care utilization assessed by means of a cost diary or a cost questionnaire were prospectively compared in two separate samples of patients, and (3) the comparability of face-to-face and telephone follow-up interviews for the assessment of costs, quality of life and the elicitation of preferences was studied in a randomized design. For most types of health care utilization and for patients’ preferences we found no or only small differences between measurement methods with respect to response rates, missing values and estimated volumes of care, costs, quality of life and utility values. Health care utilization in the cost-utility analysis
Concerning costs of continuing care in the Markov model, reports from providers of care were used to estimate costs of hospitalizations, medications and stoma care products. The costs of other types of health care utilization were estimated based on the patients’ diary. Patients reported less types of medications and stoma care products than providers of care. Thus, the cost-effectiveness ratio would have been somewhat more favorable if we would have used the volumes reported in the cost diary, because we would have underestimated the incremental costs by PRT. Although patients’ and providers’ reports of hospital days and outpatient visits did not differ significantly, other types of health care utilization (e.g. district nursing, visits to general practitioners) may also have been underreported by patients. The cost-effectiveness ratio of PRT might be least favorable if we would have obtained all data on health care utilization from providers of care, because of higher estimated incremental costs by PRT. The cost-effectiveness ratio might be most favorable if all data would have been obtained from cost questionnaires, filled out by patients, because significantly less care was reported in the questionnaire sample with respect to open questions than in the diary sample, and thus the estimated incremental costs by PRT would have been lowest.
In general, providers’reports may be more valid and most costly, but our results indicate that the importance of using providers’reports depends on the contents of the question and on the type of question asked. Patients’ responses to questions on major events and closed questions were more valid than questions on less important events and open questions. Also here, the skill is to match the level of required precision to the level of importance of the item (1). For example, in a study in which costs of medication comprise the main determinant of all costs, it is important to obtain providers’ reports of medication, whereas in a study where costs of medication are only a minor category, patients’ reports may suffice.
Utility values in the cost-utility analysis
CONCLUSIONS AND DISCUSSION affected if utility estimates would have been obtained from face-to-face or telephone follow-up interviews only.
Validity of the measurement instruments
There are few standardized and validated self-report instruments available for the measurements of costs and patients’ preferences. We developed our own measurement instruments, which were tested in a pilot study and adapted accordingly. For most questions results did not differ significantly between patients’ and providers’ reports, different recall periods, diary and a questionnaire method, and face-to-face and telephone follow-up interviews, which provides evidence for the validity of our instruments. However, we also found indications that the results for certain types of care or certain types of questions vary between measurement methods. In addition, results may differ outside the setting of a clinical trial and for other patient groups. The availability of standardized validated questionnaires for the assessment of costs and utilities would not only enhance the feasibility of collecting data, but would also promote the validity and comparability of the results of cost-effectiveness studies.
Concluding remarks
The studies presented in this thesis were all carried out alongside the TME-trial, a randomized clinical trial of TME-surgery with or without PRT. This allowed for prospective and extensive assessment of costs and utilities, as well as for the investigation of some other clinical and methodological issues. The cost-utility analysis of PRT followed recommended methodology, and it was concluded that PRT can be considered a cost-effective treatment. The findings of the clinical and methodological studies highlighted some of the opportunities and pitfalls when conducting a cost-utility analysis alongside a multi-center clinical trial. A number of issues need clarification by future research.
Areas for future research
An important assumption in the model used for the cost-effectiveness analysis of PRT is the relation between reduced local recurrence rates and improved survival. We believe that evidence is accumulating that this is indeed so, but clinical findings from future studies would be useful to finally resolve the debate. Also, there is an increasing call for preoperative identification of subgroups of patients for whom PRT may not be considered cost-effective. Two issues are important to answer this question: the development of accurate preoperative staging techniques and prospective confirmation of results for subgroups of patients. Ultimately this may allow for more individualized treatment decision making based on the patient’s profile, consisting of both patient and disease characteristics.
estimated cost-effectiveness. Concerning the clinical nature and prognosis of locally recurrent rectal cancer, research into the pathway by which local recurrences develop after PRT may provide knowledge for further optimization of the treatment of rectal cancer. Concerning methods for the assessment of costs and utilities, there is foremost a need for the development of standardized, validated, and patient and researcher friendly instruments. The availability of such instruments would promote adherence to methodology guidelines for cost-effectiveness analysis, and thus the comparability of results over studies.
References
(1) Drummond MF, Richardson WS, O’Brien BJ, et al: Users’ guides to the medical literature, XIII: How to use an article on economic analysis of clinical practice, A—Are the results of the study valid? Evidence-Based Medicine Working Group JAMA 277:1552-1557, 1997
(2) O’Brien BJ, Heyland D, Richardson WS, et al: Users’ guides to the medical literature, XIII: How to use an article on economic analysis of clinical practice, B—What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group [published erratum appears in JAMA 278:1064, 1997]. JAMA 277:1802-1806, 1997
(3) Dahlberg M, Stenborg A, Pahlman L, et al: Cost-effectiveness of preoperative radiotherapy in rectal cancer: results from the Swedish Rectal Cancer Trial. Int J Radiat Oncol Biol Phys 54(3):654, 2002 (4) Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on
Cost-effectiveness in Health and Medicine. JAMA 1996; 276(15):1253-1258.
(5) Armitage P, Berry G: Statistical Methods in Medical Research (2nd ed). Blackwell Scientific Publications, 1987, pp 183
(6) Camma C, Giunta M, Fiorica F, et al: Preoperative radiotherapy for resectable rectal cancer: A meta-analysis. JAMA 284(8):1008-1015, 2000
(7) Colorectal cancer collaborative group: Adjuvant radiotherapy for rectal cancer: a systematic overview of 8,507 patients from 22 randomised trials. Lancet 358(9290):1291-1304, 2001
(8) Brown J. Economic evaluation of cancer treatments: a review of the methods. Clin Oncol (R Coll Radiol ) 1999; 11(2):78-83.
(9) Schulman KA, Boyko WL, Jr. Evaluating cancer costs in NCI trials. Cancer Treat Res 1998; 97:37-52. (10) Kapiteijn E, van de Velde CJH: European trials with total mesorectal excision. Sem Surg Oncol
19:350-57, 2000
(11) Marijnen CA, Nagtegaal ID, Klein Kranenbarg E, Hermans J, van de Velde CJ, Leer JW, van Krieken JH; Pathology Review Committee and the Cooperative Clinical Investigators. No downstaging after short-term preoperative radiotherapy in rectal cancer patients. J Clin Oncol. 2001 Apr 1;19(7):1976-84. (12) Organisation for Economic Co-operation and Development: Castle's report : Review of the
Eurostat-OECD PPP Programme 1997. http://www.oecd.org/dataoecd/33/15/ 2405262.pdf
(13) Organisation for Economic Co-operation and Development: Purchasing Power Parities (PPPs) for OECD countries 1980-2003, 2/04 update. http://www.oecd.org/statisticsdata/
(14) Simunovic M, Gafni A, Levine M: Economics of preoperative radiotherapy with total mesorectal excision: What can we learn from the Dutch experience? J Clin Oncol 22(2): 217-219, 2004
(15) Birch S, Gafni A: Cost effectiveness/utility analyses: Do current decision rules lead us to where we want to be? J Health Econ 11:279-296, 1992
(16) Birch S, Gafni A: Changing the problem to fit the solution: Johannesson and Weinstein’s (mis)application of economics to real world problems. J Health Econ 12:469-476, 1993
(17) Sendi P, Gafni A, Birch S: Opportunity costs and uncertainty in the economic evaluation of health care interventions. Health Econ 11:23-31, 2002
(18) Koopmanschap MA, Rutten FF: A practical guide for calculating indirect costs of disease. Pharmacoeconomics 10(5):460-466, 1996
(19) Barber JA, Thompson SG. Analysis and interpretation of cost data in randomised controlled trials: review of published studies. BMJ 1998; 317(7167):1195-1200
