Editorial: Is molecular testing cost effective? It depends Carrie C. Lubitz, MD, MPH, Boston, MA
From the Department of Surgery, Massachusetts General Hospital, and the Institute for Technology Assessment, Boston, MA
THYROID NODULES are common and the majority of nodules (#85%) are benign.1-4 Fine-needle aspiration is the mainstay of evaluating thyroid nodules for malignancy. Moreover, with the increasing incidence of thyroid cancer diagnosis in recent decades, it is likely that this increased recognition will result in more fine-needle aspiration biopsies.5 Although fine-needle aspiration is accurate at determining benign and malignant nodules, there remain a large group of biopsies that are ‘‘indeterminate,’’ about 22% in a recent, large multicenter study.4 In 2009, the Bethesda System was introduced, establishing distinct categories for indeterminate lesions: (1) Bethesda III lesions are termed atypia/follicular lesion of undetermined significance (5–15% malignancy rate) and (2) Bethesda IV include follicular neoplasms (15–30% malignancy rate).6 Conventional management for indeterminate nodules includes diagnostic thyroid lobectomy, even though the majority of patients have benign disease. In an effort to avoid surgery in patients with benign disease, molecular diagnostic tests have been developed. Alexander et al. published the sentinel report of the gene-expression classifier (GEC) in 265 indeterminate nodules with corresponding surgical pathology. This study, funded by the developer, Veracyte, reports a negativepredictive value of 94–95% for indeterminate nodules (Bethesda III–IV).7 A cost-utility analysis of the GEC compared with diagnostic lobectomy, also funded by Veracyte, found GEC to be cost effective over a 5-year time horizon.8
Disclosures: None. Accepted for publication July 11, 2015. Reprint requests: Carrie C. Lubitz, MD, MPH, Assistant Professor of Surgery, Division of General Surgery, Harvard Medical School, Massachusetts General Hospital, 55 Fruit Street, Yawkey 7B, Boston, MA 02114-3117. E-mail: [email protected]. Surgery 2016;159:130-1. 0039-6060/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2015.07.047
130 SURGERY
In this issue of Surgery, Wu et al9 analyze the cost effectiveness of GEC as compared with conventional diagnostic lobectomy in patients with cytologically indeterminate thyroid nodules (ie, Bethesda categories III and IV). This study differs from prior cost-effectiveness studies in that they utilize primary institutional data on GEC test performance, use a lifetime time horizon (likely to favor surgery), and perform extensive sensitivity analyses. The authors highlight the dependence of negative predictive value on prevalence of disease (in this case malignancy rate) and they conclude that cost effectiveness of GEC compared with conventional management is inversely related to malignancy rate. The strength of the conclusions drawn from a decision-analytic model is based on the assumptions made, the design of the model, and the quality of the input data. Performing sensitivity analyses assesses the robustness of the results. In this article, the authors present a well-constructed model and perform extensive and appropriate 1-way and probabilistic sensitivity analyses to test parameter uncertainty. They conclude that at a willingness-to-pay (WTP) threshold of $100,000 per quality-adjusted life-year ($/QALY), routine GEC testing is not cost effective. However, what this study really highlights is that with varying assumptions, inputs, and WTP thresholds, the preferred approach differs. This is underscored by the result that conventional management was only preferred in 53% of the simulations in the probabilistic sensitivity analyses. A key assumption for which there is a paucity of data, and which was not addressed by this study, is the long-term outcome of those patients with falsenegative GEC. We currently do not know if the delay in treatment for these patients has an effect on length or quality of life. Longitudinal follow-up will be a vital area for further research. Moreover, the assumption that all patients with suspicious GEC testing underwent surgery while using an approximation of clinical practice (80% undergo surgery) for cytologically indeterminate nodules is problematic.
Lubitz 131
Surgery Volume 159, Number 1
It is important to note that determining which strategy is preferred or ‘‘cost effective’’ depends on the WTP threshold, or the highest price that a society is willing to pay for a given health benefit. This is theoretically based on a fixed budget with all interventions listed on the same scale, with the cost-effective ($/QALY) intervention last to be covered determining a society’s WTP ratio.10 In reality, no fixed budget exists and research indicates that, based on increasing health care expenditures and survey data on patients’ willingness to pay, true thresholds may be as high as $200,000–300,000/QALY.11 The World Health Organization promotes that use of 3 times per capita annual income as the threshold ($140,000/QALY in the United States).12 If this threshold were used by Wu et al,9 this would have changed their conclusions. So is GEC cost effective? It depends. If we accept a higher WTP threshold, yes. If only atypia of undetermined significance/follicular lesion of undetermined significance nodules ($94,000/QALY) are sent for GEC, yes. If only Bethesda category IV nodules ($241,500/QALY) are sent for GEC, no. Varying time horizons also affect outcomes. Li et al8 performed a similar study with a different conclusion, but only looked at the cost effectiveness over a 5-year period. This lack of certainty in the preferred strategy illustrated by this work underscores that the decision to utilize GEC should be based on a number of factors, including the malignancy rate at one’s institution and patient preference. It is imperative that we collect quality of life data and longitudinal data on outcomes for observed patients.
REFERENCES 1. Hegedus L. Clinical practice. The thyroid nodule. N Engl J Med 2004;351:1764-71. 2. Reiners C, Wegscheider K, Schicha H, Theissen P, Vaupel R, Wrbitzky R, et al. Prevalence of thyroid disorders in the working population of Germany: ultrasonography screening in 96,278 unselected employees. Thyroid 2004; 14:926-32. 3. Tan GH, Gharib H. Thyroid incidentalomas: management approaches to nonpalpable nodules discovered incidentally on thyroid imaging. Ann Intern Med 1997;126: 226-31. 4. Wang CC, Friedman L, Kennedy GC, Wang H, Kebebew E, Steward DL, et al. A large multicenter correlation study of thyroid nodule cytopathology and histopathology. Thyroid 2011;21:243-51. 5. Chen AY, Jemal A, Ward EM. Increasing incidence of differentiated thyroid cancer in the United States, 1988-2005. Cancer 2009;115:3801-7. 6. Cibas ES, Ali SZ. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid 2009;19:1159-65. 7. Alexander EK, Kennedy GC, Baloch ZW, Cibas ES, Chudova D, Diggans J, et al. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med 2012;367:705-15. 8. Li H, Robinson KA, Anton B, Saldanha IJ, Ladenson PW. Cost-effectiveness of a novel molecular test for cytologically indeterminate thyroid nodules. J Clin Endocrinol Metab 2011;96:E1719-26. 9. Wu JX, Lam R, Levin M, Rao J, Sullivan PS, Yeh MW. Impact of malignancy rates on cost-effectiveness of routine gene expression classifier testing for indeterminate thyroid nodules. Surgery 2015; XX: xx-xx. 10. Neumann PJ, Cohen JT, Weinstein MC. Updating costeffectiveness–the curious resilience of the $50,000-perQALY threshold. N Engl J Med 2014;371:796-7. 11. Braithwaite RS, Meltzer DO, King JT Jr, Leslie D, Roberts MS. What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care 2008;46:349-56. 12. World Health Organization (WHO). Choosing interventions that are cost-effective; 2014. Available from: www. who.int/choice/en/ [Accessed 15 May 2015].
From the Department of Surgery, Massachusetts General Hospital, and the Institute for Technology Assessment, Boston, MA
THYROID NODULES are common and the majority of nodules (#85%) are benign.1-4 Fine-needle aspiration is the mainstay of evaluating thyroid nodules for malignancy. Moreover, with the increasing incidence of thyroid cancer diagnosis in recent decades, it is likely that this increased recognition will result in more fine-needle aspiration biopsies.5 Although fine-needle aspiration is accurate at determining benign and malignant nodules, there remain a large group of biopsies that are ‘‘indeterminate,’’ about 22% in a recent, large multicenter study.4 In 2009, the Bethesda System was introduced, establishing distinct categories for indeterminate lesions: (1) Bethesda III lesions are termed atypia/follicular lesion of undetermined significance (5–15% malignancy rate) and (2) Bethesda IV include follicular neoplasms (15–30% malignancy rate).6 Conventional management for indeterminate nodules includes diagnostic thyroid lobectomy, even though the majority of patients have benign disease. In an effort to avoid surgery in patients with benign disease, molecular diagnostic tests have been developed. Alexander et al. published the sentinel report of the gene-expression classifier (GEC) in 265 indeterminate nodules with corresponding surgical pathology. This study, funded by the developer, Veracyte, reports a negativepredictive value of 94–95% for indeterminate nodules (Bethesda III–IV).7 A cost-utility analysis of the GEC compared with diagnostic lobectomy, also funded by Veracyte, found GEC to be cost effective over a 5-year time horizon.8
Disclosures: None. Accepted for publication July 11, 2015. Reprint requests: Carrie C. Lubitz, MD, MPH, Assistant Professor of Surgery, Division of General Surgery, Harvard Medical School, Massachusetts General Hospital, 55 Fruit Street, Yawkey 7B, Boston, MA 02114-3117. E-mail: [email protected]. Surgery 2016;159:130-1. 0039-6060/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2015.07.047
130 SURGERY
In this issue of Surgery, Wu et al9 analyze the cost effectiveness of GEC as compared with conventional diagnostic lobectomy in patients with cytologically indeterminate thyroid nodules (ie, Bethesda categories III and IV). This study differs from prior cost-effectiveness studies in that they utilize primary institutional data on GEC test performance, use a lifetime time horizon (likely to favor surgery), and perform extensive sensitivity analyses. The authors highlight the dependence of negative predictive value on prevalence of disease (in this case malignancy rate) and they conclude that cost effectiveness of GEC compared with conventional management is inversely related to malignancy rate. The strength of the conclusions drawn from a decision-analytic model is based on the assumptions made, the design of the model, and the quality of the input data. Performing sensitivity analyses assesses the robustness of the results. In this article, the authors present a well-constructed model and perform extensive and appropriate 1-way and probabilistic sensitivity analyses to test parameter uncertainty. They conclude that at a willingness-to-pay (WTP) threshold of $100,000 per quality-adjusted life-year ($/QALY), routine GEC testing is not cost effective. However, what this study really highlights is that with varying assumptions, inputs, and WTP thresholds, the preferred approach differs. This is underscored by the result that conventional management was only preferred in 53% of the simulations in the probabilistic sensitivity analyses. A key assumption for which there is a paucity of data, and which was not addressed by this study, is the long-term outcome of those patients with falsenegative GEC. We currently do not know if the delay in treatment for these patients has an effect on length or quality of life. Longitudinal follow-up will be a vital area for further research. Moreover, the assumption that all patients with suspicious GEC testing underwent surgery while using an approximation of clinical practice (80% undergo surgery) for cytologically indeterminate nodules is problematic.
Lubitz 131
Surgery Volume 159, Number 1
It is important to note that determining which strategy is preferred or ‘‘cost effective’’ depends on the WTP threshold, or the highest price that a society is willing to pay for a given health benefit. This is theoretically based on a fixed budget with all interventions listed on the same scale, with the cost-effective ($/QALY) intervention last to be covered determining a society’s WTP ratio.10 In reality, no fixed budget exists and research indicates that, based on increasing health care expenditures and survey data on patients’ willingness to pay, true thresholds may be as high as $200,000–300,000/QALY.11 The World Health Organization promotes that use of 3 times per capita annual income as the threshold ($140,000/QALY in the United States).12 If this threshold were used by Wu et al,9 this would have changed their conclusions. So is GEC cost effective? It depends. If we accept a higher WTP threshold, yes. If only atypia of undetermined significance/follicular lesion of undetermined significance nodules ($94,000/QALY) are sent for GEC, yes. If only Bethesda category IV nodules ($241,500/QALY) are sent for GEC, no. Varying time horizons also affect outcomes. Li et al8 performed a similar study with a different conclusion, but only looked at the cost effectiveness over a 5-year period. This lack of certainty in the preferred strategy illustrated by this work underscores that the decision to utilize GEC should be based on a number of factors, including the malignancy rate at one’s institution and patient preference. It is imperative that we collect quality of life data and longitudinal data on outcomes for observed patients.
REFERENCES 1. Hegedus L. Clinical practice. The thyroid nodule. N Engl J Med 2004;351:1764-71. 2. Reiners C, Wegscheider K, Schicha H, Theissen P, Vaupel R, Wrbitzky R, et al. Prevalence of thyroid disorders in the working population of Germany: ultrasonography screening in 96,278 unselected employees. Thyroid 2004; 14:926-32. 3. Tan GH, Gharib H. Thyroid incidentalomas: management approaches to nonpalpable nodules discovered incidentally on thyroid imaging. Ann Intern Med 1997;126: 226-31. 4. Wang CC, Friedman L, Kennedy GC, Wang H, Kebebew E, Steward DL, et al. A large multicenter correlation study of thyroid nodule cytopathology and histopathology. Thyroid 2011;21:243-51. 5. Chen AY, Jemal A, Ward EM. Increasing incidence of differentiated thyroid cancer in the United States, 1988-2005. Cancer 2009;115:3801-7. 6. Cibas ES, Ali SZ. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid 2009;19:1159-65. 7. Alexander EK, Kennedy GC, Baloch ZW, Cibas ES, Chudova D, Diggans J, et al. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med 2012;367:705-15. 8. Li H, Robinson KA, Anton B, Saldanha IJ, Ladenson PW. Cost-effectiveness of a novel molecular test for cytologically indeterminate thyroid nodules. J Clin Endocrinol Metab 2011;96:E1719-26. 9. Wu JX, Lam R, Levin M, Rao J, Sullivan PS, Yeh MW. Impact of malignancy rates on cost-effectiveness of routine gene expression classifier testing for indeterminate thyroid nodules. Surgery 2015; XX: xx-xx. 10. Neumann PJ, Cohen JT, Weinstein MC. Updating costeffectiveness–the curious resilience of the $50,000-perQALY threshold. N Engl J Med 2014;371:796-7. 11. Braithwaite RS, Meltzer DO, King JT Jr, Leslie D, Roberts MS. What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care 2008;46:349-56. 12. World Health Organization (WHO). Choosing interventions that are cost-effective; 2014. Available from: www. who.int/choice/en/ [Accessed 15 May 2015].
