Letters
4. Freedland KE, Mohr DC, Davidson KW, Schwartz JE. Usual and unusual care: existing practice control groups in randomized controlled trials of behavioral interventions. Psychosom Med. 2011;73(4):323-335.
In Reply Walach et al suggest that randomized clinical trials (RCTs) should not be used to evaluate meditation programs. While we agree that cohort studies provide valuable information, we disagree with the rationale that conscious choice and active engagement are eliminated in RCTs. Individuals make a conscious choice to join meditation trials, and since meditation requires engagement by the participant, it cannot become a passive activity merely by the act of randomization any more than exercise could. Both cohort studies and RCTs have inherent strengths and weaknesses in terms of generalizability and the extent to which “causal” conclusions can be drawn. Cohort studies are useful early in establishing short- and long-term outcomes of a risk or risk reduction factor. Randomized clinical trials then provide evidence as to the “causal” role of this factor through manipulating, ideally, only the factor of interest. The Institute of Medicine report cited by Dr Loucks makes a point of this value of RCTs.1 Following high-quality RCTs, cohort studies can further establish the circumstances that optimize and promote engagement over long periods. Our review only included trials with an “active control.” The main purpose of nonspecific active controls (Figure 1A in our article2) is to control for nonspecific effects. Specific active controls (Figure 1B in our article2) are comparisons with a known therapy. Schmidt et al,3(p362) while using some progressive muscle relaxation, described the control as “an active control intervention aimed at equating the nonspecific features of MBSR [mindfulness-based stress reduction],” rather than as a specific comparison with progressive muscle relaxation itself. Therefore we categorized it as a nonspecific (but active) control. A clarification: Allocation concealment refers to the protection of the randomization process, can be accomplished in unblinded trials, and should not be confused with blinding.4,5 Our rating criteria did not assess double blinding but did assess single blinding of outcome assessors as a quality criterion. Regarding usual care controls, many of the outcomes of interest were symptoms such as anxiety that are not routinely addressed in usual care in the same manner that blood pressure or cholesterol are managed. Therefore, relying on usual care as a benchmark may not provide a rigorous test of an intervention, particularly when the outcomes are selfreported. The choice of control also depends on the question one is trying to answer. Numerous reviews have already shown the positive effects of meditation programs compared with usual care, and we did not wish to replicate these. Instead, we sought to understand the effects of these programs beyond nonspecific effects such as time and attention. These are important questions to answer if we wish to know the extent and mechanisms through which meditation operates. While attention and group support may be important components of any behavioral intervention, they have not been advocated as a primary means by which meditation programs work.6
We agree with Rutledge et al that different meta-analyses with different inclusion criteria can lead to different conclusions. We also agree it is important to perform a systematic review of biological outcomes of meditation programs, which would require much additional effort beyond what was possible within the scope of our funded project. Madhav Goyal, MD, MPH Eric B. Bass, MD, MPH Jennifer A. Haythornthwaite, PhD Author Affiliations: Department of Medicine, Johns Hopkins University, Baltimore, Maryland (Goyal, Bass); Department of Health Policy and Management, Johns Hopkins School of Public Health, Baltimore, Maryland (Bass); Department of Psychiatry and Behavioral Services, Johns Hopkins University, Baltimore, Maryland (Haythornthwaite). Corresponding Author: Madhav Goyal, MD, MPH, Department of Medicine, Johns Hopkins University, 2024 E Monument St, Ste 1-500W, Baltimore, MD 21287 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Institute of Medicine, Committee on Comparative Effectiveness Research Prioritization. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: National Academies Press; 2009. 2. Goyal M, Singh S, Sibinga EM, et al. Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Intern Med. 2014;174(3):357-368. 3. Schmidt S, Grossman P, Schwarzer B, Jena S, Naumann J, Walach H. Treating fibromyalgia with mindfulness-based stress reduction: results from a 3-armed randomized controlled trial. Pain. 2011;152(2):361-369. 4. Sedgwick P. Allocation concealment. BMJ. 2012;344:e156. 5. Viera AJ, Bangdiwala SI. Eliminating bias in randomized controlled trials: importance of allocation concealment and masking. Fam Med. 2007;39(2):132137. 6. MacCoon DG, Imel ZE, Rosenkranz MA, et al. The validation of an active control intervention for mindfulness based stress reduction (MBSR). Behav Res Ther. 2012;50(1):3-12.
Assessing the Clinical Impact of Appropriate Echocardiograms To the Editor Appropriate use criteria (AUC) were created by the American College of Cardiology and subspecialty societies with the goal of providing health care practitioners and reimbursement agencies a rational approach to the use of diagnostic imaging in the delivery of high-quality care.1 In a recent issue of JAMA Internal Medicine, Matulevicius and colleagues2 present novel data regarding the association of AUC classification and clinical impact of transthoracic echocardiography (TTE). They conclude that despite a high prevalence of appropriate TTEs (approximately 92%), only one-third of TTEs resulted in an active change in clinical care. Findings from this study have generated discussion in the echocardiography community and resulted in a response statement from the American Society of Echocardiography.3 We agree that attempting to identify the clinical impact of appropriate vs inappropriate TTEs is of great importance and may help validate—or fail to validate—the use of AUC. However, a retrospective review of the electronic medical record is limited by the challenge of distinguishing whether TTE results did not affect care or whether a lack of determination in the electronic medical record made that appear to be the case.
jamainternalmedicine.com
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
1195
Letters
Prospective studies will help to address this issue, although other more objective measures of the utility of appropriate vs inappropriate TTE are needed. We previously studied the 2011 AUC for echocardiography in 3 distinct practice environments.4 As a part of this analysis, we assessed whether new major TTE abnormalities were more likely to be present on appropriate or inappropriate TTEs. In studies classified as appropriate, TTEs were in fact more likely to have new, major abnormalities compared with uncertain and inappropriate studies. Of note, in the outpatient community environment, only 3% of inappropriate TTEs had such an abnormality. The study by Matulevicius et al2 has important implications for the echocardiographic and health care community as a whole. The aim of studying the “usefulness” of the AUC is commendable, although it is a challenging scientific endeavor. Objective measures of the utility of a TTE are needed, and we propose that incorporating the presence or absence of a TTE abnormality may be a helpful adjunct. In addition, studies in various clinical settings are necessary, since the practice environment has an impact on the rate of TTE appropriateness. Prospective studies that incorporate numerous outcome measures will help define the use of AUC in clinical practice, provide guidance for future AUC revisions, and assist in optimizing the use of valuable health care resources. David M. Dudzinski, MD, JD R. Sacha Bhatia, MD, MBA Rory B. Weiner, MD
Level 1: Technical efficacy (spatial resolution and imaging reproducibility);
Author Affiliations: Cardiac Ultrasound Laboratory, Massachusetts General Hospital, Boston.
Level 2: Diagnostic efficacy (sensitivity, specificity, and accuracy);
Corresponding Author: David M. Dudzinski, MD, JD, Cardiac Ultrasound Laboratory, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114 ([email protected]).
Level 3: Diagnostic thinking efficacy (incremental value of testing information on diagnosis);
Conflict of Interest Disclosures: Dr Weiner served as a writing committee member for the 2011 Appropriate Use Criteria for Echocardiography.1 No other disclosures are reported.
Level 4: Therapeutic efficacy (changes in patient management related to testing);
1. Douglas PS, Garcia MJ, Haines DE, et al; American College of Cardiology Foundation Appropriate Use Criteria Task Force; American Society of Echocardiography; American Heart Association; American Society of Nuclear Cardiology; Heart Failure Society of America; Heart Rhythm Society; Society for Cardiovascular Angiography and Interventions; Society of Critical Care Medicine; Society of Cardiovascular Computed Tomography; Society for Cardiovascular Magnetic Resonance. ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/ SCCM/SCCT/SCMR 2011 appropriate use criteria for echocardiography. J Am Coll Cardiol. 2011;57(9):1126-1166.
Level 5: Patient outcome efficacy (safety concerns, improved patient outcomes, including quality of life, morbidity, and mortality as a result of testing);
2. Matulevicius SA, Rohatgi A, Das SR, Price AL, DeLuna A, Reimold SC. Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med. 2013;173(17):1600-1607. 3. American Society of Echocardiography. Echo under attack in the press. http: //www.asecho.org/auc. Accessed August 15, 2013. 4. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr. 2012;25(11):1162-1169.
In Reply We appreciate Dudzinski and colleagues’ thoughtful letter and generally agree with the points they raise. Appropriate use criteria (AUC) are a strong step by multiple societies to address the clinical use of transthoracic echocardiography (TTE). However, we caution against using the relative 1196
prevalence of major new abnormalities on appropriate vs inappropriate TTEs as a surrogate for their relative clinical utility. Appropriate TTEs are expected to have a higher prevalence of major abnormal findings owing to the inherent bias present in the definition of appropriate vs inappropriate AUCs. An appropriate indication, such as “reevaluation of known heart failure to guide therapy,” will by construction have a high prevalence of major TTE abnormalities. In contrast, an inappropriate indication, such as “initial evaluation of ventricular function with no symptoms or signs of cardiovascular disease,” will yield few abnormal findings. Furthermore, the clinical impact of TTE does not solely depend on the detection of TTE abnormalities. A normal TTE for an indication like “hypotension or hemodynamic instability of uncertain or suspected cardiac etiology” may result in the pursuit of noncardiac diagnoses and therapies. Similarly, an indication like “initial evaluation when there is a reasonable suspicion of valvular or structural heart disease” may detect moderate aortic stenosis, but if the patient has a terminal illness and/or refuses or is not a candidate for future surgical or percutaneous valve interventions, care may not change. Recently, the American College of Cardiology, in collaboration with multiple subspecialty societies, released a health policy statement on noninvasive imaging use.1 They discuss a 6-level hierarchical technology efficacy assessment framework:
Level 6: Society efficacy (cost vs overall societal benefit of testing).1,2 Appropriate use criteria and their association with abnormal TTE findings are consistent with level 2 of this scheme. Our study, although with limitations, attempts to address level 4. Although our study was retrospective, prior prospective studies from the 1990s that assessed the clinical impact of TTE found active change in care in 36% to 38% of cases, consistent with the 32% active change we saw in our study.3-5 A prime tenet of the RAND process was that the assignment of appropriateness would need to be formally tested to see if it led to improved outcomes and modified if it did not.6 For TTE AUC, this would require validating if criteria rated as appropriate accomplish the ultimate objective of impacting patient care. Despite significant growth in TTE volume over the past 2 decades and the development of AUC, clinical impact has remained constant. Better metrics that reflect the incremen-
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
jamainternalmedicine.com
Letters
tal benefit of testing on diagnostic thinking, therapy, and patient outcomes must be pursued to maximize the societal benefit of TTE. Susan A. Matulevicius, MD, MSCS Sandeep R. Das, MD, MPH Sharon C. Reimold, MD Author Affiliations: Department of Medicine, University of Texas Southwestern Medical Center, Dallas. Corresponding Author: Susan A. Matulevicius, MD, MSCS, Department of Medicine, University of Texas Southwestern, 5909 Harry Hines Blvd, Dallas, TX 75390-9047 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Mark DB, Anderson JL, Brinker JA, et al. ACC/AHA/ASE/ASNC/HRS/IAC/ Mended Hearts/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR/SNMMI 2014 health policy statement on use of noninvasive cardiovascular imaging: a report of the American College of Cardiology Clinical Quality Committee. J Am Coll Cardiol. 2014;63(7):698-721. 2. Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making. 1991;11(2):88-94. 3. Matulevicius SA, Rohatgi A, Das SR, Price AL, DeLuna A, Reimold SC. Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med. 2013;173(17):1600-1607. 4. Tam JW, Nichol J, MacDiarmid AL, Lazarow N, Wolfe K. What is the real clinical utility of echocardiography? a prospective observational study. J Am Soc Echocardiogr. 1999;12(9):689-697. 5. Waggoner AD, Harris KM, Braverman AC, Barzilai B, Geltman EM. The role of transthoracic echocardiography in the management of patients seen in an outpatient cardiology clinic. J Am Soc Echocardiogr. 1996;9(6):761-768. 6. Fitch K, Bernstein SJ, Aguilar MS, et al. The RAND/UCLA Appropriateness Method User's Manual. Santa Monica, CA: RAND Corporation; 2001.
Estimating Overdiagnosis in Lung Cancer Screening To the Editor We read with interest the article by Patz et al1 investigating overdiagnosis in the National Lung Screening Trial (NLST). In their investigation, the authors found the upper bound for probability of overdiagnosis to be 11.0% to 18.5% for all lung cancers and even higher for bronchioloalveolar carcinoma (BAC) (67.6% to 78.9%). However, this risk assessment did not consider the lead- and the length-time biases.2 If data for non-BAC non–small cell lung cancer (NSCLC) (from Table 1 of the article by Patz et al1) had been plotted, we would have observed that the data actually distributed into 2 phases. First, in the active screening period, more non-BAC NSCLCs are diagnosed in the low-dose computed tomography (LDCT) arm than in the chest radiography (CXR) arm, emphasizing that some tumors are detected early by LDCT but not by CXR (overdiagnosis). During the second period beginning at the end of screening, the number of non-BAC NSCLCs became lower in the LDCT arm than in the CXR arm. This suggests the tumors not previously diagnosed by CXR were diagnosed later (lead-time bias) or grew slowly (length-time bias) but ultimately became symptomatic. Therefore, an accurate way to estimate actual overdiagnosis risk would be to use areas between the 2 curves in these 2 phases. We calculated a 5.3% larger area in the first phase than in the second one. We believe this proportion is a better estimator of overdiagnosis because it integrates time biases. Moreover, some previous lung cancer screening trials demonstrated
that follow-up have to be extended up to 9 or 10 years to completely take into account both overdiagnosis and timebased biases.3,4 Conversely, numbers of diagnosed BACs in the LDCT arm are much higher than in the CXR arm during but overlap after the active screening period, up to study year 6. This illustrates that BAC are at high-risk of overdiagnosis during the screening period without effects of time-based biases. Indeed, BACs are mainly ground glass opacities with very mild growth. Using a volume-doubling time approach for nodule management (as done by the NELSON [Nederlands Leuvens Longkanker Screenings Onderzoek] trialists), would have resulted in ruling out these lesions as positive screening results.5 In summary, lung cancer screening highlights 2 different clusters. Non-BAC NSCLC could be overdiagnosed. Such a risk should be investigated and explained to patients, but it is overestimated by Patz et al.1 Otherwise, BACs are at high risk for overdiagnosis, but this risk may be easily managed by using a volume-monitoring approach and should not hide the true benefit for lung cancer screening on lung cancer mortality. Sébastien Couraud, MD, MSc Laurent Greillier, MD, PhD Bernard Milleron, MD; for the IFCT Lung Cancer Screening Group Author Affiliations: Service de Pneumologie Aiguë Spécialisée et Cancérologie Thoracique, CH Lyon Sud, Hospices Civils de Lyon, Pierre Bénite, France (Couraud); Faculté de Médecine Lyon-Sud, Université Lyon 1, Oullins, France (Couraud); Aix Marseille Univ–Assistance Publique–Hôpitaux de Marseille, Multidisciplinary Oncology & Therapeutic Innovations Department, Marseille, France (Greillier); Respiratory Disease Department, Tenon Hospital, Assistance Publique–Hôpitaux de Paris, Paris, France (Milleron); Intergroupe Francophone de Cancérologie Thoracique (IFCT), Paris, France (Milleron). Corresponding Author: Sébastien Couraud, MD, MSc, Service de Pneumologie Aiguë Spécialisée et Cancérologie Thoracique, CH Lyon Sud, Hospices Civils de Lyon, 165 Chemin du Grand Revoyet, 69495 Pierre Bénite CEDEX, France ([email protected]). Conflict of Interest Disclosures: None reported. Group Information: Members of the IFCT Lung Cancer Screening Group are listed in Ann Oncol. 2013;24(3):586-597. 1. Patz EF Jr, Pinsky P, Gatsonis C, et al; NLST Overdiagnosis Manuscript Writing Team. Overdiagnosis in low-dose computed tomography screening for lung cancer. JAMA Intern Med. 2014;174(2):269-274. 2. Patz EF Jr, Goodman PC, Bepler G. Screening for lung cancer. N Engl J Med. 2000;343(22):1627-1633. 3. Marcus PM, Bergstralh EJ, Zweig MH, Harris A, Offord KP, Fontana RS. Extended lung cancer incidence follow-up in the Mayo Lung Project and overdiagnosis. J Natl Cancer Inst. 2006;98(11):748-756. 4. Oken MM, Hocking WG, Kvale PA, et al; PLCO Project Team. Screening by chest radiograph and lung cancer mortality: the Prostate, Lung, Colorectal, and Ovarian (PLCO) randomized trial. JAMA. 2011;306(17):1865-1873. 5. Horeweg N, van der Aalst CM, Vliegenthart R, et al. Volumetric computed tomography screening for lung cancer: three rounds of the NELSON trial. Eur Respir J. 2013;42(6):1659-1667.
To the Editor Patz et al1 attempt to report on the rate of overdiagnosis that occurred as part of the National Lung Cancer Screening Trial (NLST), however their conclusion that “the probability is 18.5%…that any lung cancer detected by LDCT
jamainternalmedicine.com
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
1197
4. Freedland KE, Mohr DC, Davidson KW, Schwartz JE. Usual and unusual care: existing practice control groups in randomized controlled trials of behavioral interventions. Psychosom Med. 2011;73(4):323-335.
In Reply Walach et al suggest that randomized clinical trials (RCTs) should not be used to evaluate meditation programs. While we agree that cohort studies provide valuable information, we disagree with the rationale that conscious choice and active engagement are eliminated in RCTs. Individuals make a conscious choice to join meditation trials, and since meditation requires engagement by the participant, it cannot become a passive activity merely by the act of randomization any more than exercise could. Both cohort studies and RCTs have inherent strengths and weaknesses in terms of generalizability and the extent to which “causal” conclusions can be drawn. Cohort studies are useful early in establishing short- and long-term outcomes of a risk or risk reduction factor. Randomized clinical trials then provide evidence as to the “causal” role of this factor through manipulating, ideally, only the factor of interest. The Institute of Medicine report cited by Dr Loucks makes a point of this value of RCTs.1 Following high-quality RCTs, cohort studies can further establish the circumstances that optimize and promote engagement over long periods. Our review only included trials with an “active control.” The main purpose of nonspecific active controls (Figure 1A in our article2) is to control for nonspecific effects. Specific active controls (Figure 1B in our article2) are comparisons with a known therapy. Schmidt et al,3(p362) while using some progressive muscle relaxation, described the control as “an active control intervention aimed at equating the nonspecific features of MBSR [mindfulness-based stress reduction],” rather than as a specific comparison with progressive muscle relaxation itself. Therefore we categorized it as a nonspecific (but active) control. A clarification: Allocation concealment refers to the protection of the randomization process, can be accomplished in unblinded trials, and should not be confused with blinding.4,5 Our rating criteria did not assess double blinding but did assess single blinding of outcome assessors as a quality criterion. Regarding usual care controls, many of the outcomes of interest were symptoms such as anxiety that are not routinely addressed in usual care in the same manner that blood pressure or cholesterol are managed. Therefore, relying on usual care as a benchmark may not provide a rigorous test of an intervention, particularly when the outcomes are selfreported. The choice of control also depends on the question one is trying to answer. Numerous reviews have already shown the positive effects of meditation programs compared with usual care, and we did not wish to replicate these. Instead, we sought to understand the effects of these programs beyond nonspecific effects such as time and attention. These are important questions to answer if we wish to know the extent and mechanisms through which meditation operates. While attention and group support may be important components of any behavioral intervention, they have not been advocated as a primary means by which meditation programs work.6
We agree with Rutledge et al that different meta-analyses with different inclusion criteria can lead to different conclusions. We also agree it is important to perform a systematic review of biological outcomes of meditation programs, which would require much additional effort beyond what was possible within the scope of our funded project. Madhav Goyal, MD, MPH Eric B. Bass, MD, MPH Jennifer A. Haythornthwaite, PhD Author Affiliations: Department of Medicine, Johns Hopkins University, Baltimore, Maryland (Goyal, Bass); Department of Health Policy and Management, Johns Hopkins School of Public Health, Baltimore, Maryland (Bass); Department of Psychiatry and Behavioral Services, Johns Hopkins University, Baltimore, Maryland (Haythornthwaite). Corresponding Author: Madhav Goyal, MD, MPH, Department of Medicine, Johns Hopkins University, 2024 E Monument St, Ste 1-500W, Baltimore, MD 21287 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Institute of Medicine, Committee on Comparative Effectiveness Research Prioritization. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: National Academies Press; 2009. 2. Goyal M, Singh S, Sibinga EM, et al. Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Intern Med. 2014;174(3):357-368. 3. Schmidt S, Grossman P, Schwarzer B, Jena S, Naumann J, Walach H. Treating fibromyalgia with mindfulness-based stress reduction: results from a 3-armed randomized controlled trial. Pain. 2011;152(2):361-369. 4. Sedgwick P. Allocation concealment. BMJ. 2012;344:e156. 5. Viera AJ, Bangdiwala SI. Eliminating bias in randomized controlled trials: importance of allocation concealment and masking. Fam Med. 2007;39(2):132137. 6. MacCoon DG, Imel ZE, Rosenkranz MA, et al. The validation of an active control intervention for mindfulness based stress reduction (MBSR). Behav Res Ther. 2012;50(1):3-12.
Assessing the Clinical Impact of Appropriate Echocardiograms To the Editor Appropriate use criteria (AUC) were created by the American College of Cardiology and subspecialty societies with the goal of providing health care practitioners and reimbursement agencies a rational approach to the use of diagnostic imaging in the delivery of high-quality care.1 In a recent issue of JAMA Internal Medicine, Matulevicius and colleagues2 present novel data regarding the association of AUC classification and clinical impact of transthoracic echocardiography (TTE). They conclude that despite a high prevalence of appropriate TTEs (approximately 92%), only one-third of TTEs resulted in an active change in clinical care. Findings from this study have generated discussion in the echocardiography community and resulted in a response statement from the American Society of Echocardiography.3 We agree that attempting to identify the clinical impact of appropriate vs inappropriate TTEs is of great importance and may help validate—or fail to validate—the use of AUC. However, a retrospective review of the electronic medical record is limited by the challenge of distinguishing whether TTE results did not affect care or whether a lack of determination in the electronic medical record made that appear to be the case.
jamainternalmedicine.com
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
1195
Letters
Prospective studies will help to address this issue, although other more objective measures of the utility of appropriate vs inappropriate TTE are needed. We previously studied the 2011 AUC for echocardiography in 3 distinct practice environments.4 As a part of this analysis, we assessed whether new major TTE abnormalities were more likely to be present on appropriate or inappropriate TTEs. In studies classified as appropriate, TTEs were in fact more likely to have new, major abnormalities compared with uncertain and inappropriate studies. Of note, in the outpatient community environment, only 3% of inappropriate TTEs had such an abnormality. The study by Matulevicius et al2 has important implications for the echocardiographic and health care community as a whole. The aim of studying the “usefulness” of the AUC is commendable, although it is a challenging scientific endeavor. Objective measures of the utility of a TTE are needed, and we propose that incorporating the presence or absence of a TTE abnormality may be a helpful adjunct. In addition, studies in various clinical settings are necessary, since the practice environment has an impact on the rate of TTE appropriateness. Prospective studies that incorporate numerous outcome measures will help define the use of AUC in clinical practice, provide guidance for future AUC revisions, and assist in optimizing the use of valuable health care resources. David M. Dudzinski, MD, JD R. Sacha Bhatia, MD, MBA Rory B. Weiner, MD
Level 1: Technical efficacy (spatial resolution and imaging reproducibility);
Author Affiliations: Cardiac Ultrasound Laboratory, Massachusetts General Hospital, Boston.
Level 2: Diagnostic efficacy (sensitivity, specificity, and accuracy);
Corresponding Author: David M. Dudzinski, MD, JD, Cardiac Ultrasound Laboratory, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114 ([email protected]).
Level 3: Diagnostic thinking efficacy (incremental value of testing information on diagnosis);
Conflict of Interest Disclosures: Dr Weiner served as a writing committee member for the 2011 Appropriate Use Criteria for Echocardiography.1 No other disclosures are reported.
Level 4: Therapeutic efficacy (changes in patient management related to testing);
1. Douglas PS, Garcia MJ, Haines DE, et al; American College of Cardiology Foundation Appropriate Use Criteria Task Force; American Society of Echocardiography; American Heart Association; American Society of Nuclear Cardiology; Heart Failure Society of America; Heart Rhythm Society; Society for Cardiovascular Angiography and Interventions; Society of Critical Care Medicine; Society of Cardiovascular Computed Tomography; Society for Cardiovascular Magnetic Resonance. ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/ SCCM/SCCT/SCMR 2011 appropriate use criteria for echocardiography. J Am Coll Cardiol. 2011;57(9):1126-1166.
Level 5: Patient outcome efficacy (safety concerns, improved patient outcomes, including quality of life, morbidity, and mortality as a result of testing);
2. Matulevicius SA, Rohatgi A, Das SR, Price AL, DeLuna A, Reimold SC. Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med. 2013;173(17):1600-1607. 3. American Society of Echocardiography. Echo under attack in the press. http: //www.asecho.org/auc. Accessed August 15, 2013. 4. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr. 2012;25(11):1162-1169.
In Reply We appreciate Dudzinski and colleagues’ thoughtful letter and generally agree with the points they raise. Appropriate use criteria (AUC) are a strong step by multiple societies to address the clinical use of transthoracic echocardiography (TTE). However, we caution against using the relative 1196
prevalence of major new abnormalities on appropriate vs inappropriate TTEs as a surrogate for their relative clinical utility. Appropriate TTEs are expected to have a higher prevalence of major abnormal findings owing to the inherent bias present in the definition of appropriate vs inappropriate AUCs. An appropriate indication, such as “reevaluation of known heart failure to guide therapy,” will by construction have a high prevalence of major TTE abnormalities. In contrast, an inappropriate indication, such as “initial evaluation of ventricular function with no symptoms or signs of cardiovascular disease,” will yield few abnormal findings. Furthermore, the clinical impact of TTE does not solely depend on the detection of TTE abnormalities. A normal TTE for an indication like “hypotension or hemodynamic instability of uncertain or suspected cardiac etiology” may result in the pursuit of noncardiac diagnoses and therapies. Similarly, an indication like “initial evaluation when there is a reasonable suspicion of valvular or structural heart disease” may detect moderate aortic stenosis, but if the patient has a terminal illness and/or refuses or is not a candidate for future surgical or percutaneous valve interventions, care may not change. Recently, the American College of Cardiology, in collaboration with multiple subspecialty societies, released a health policy statement on noninvasive imaging use.1 They discuss a 6-level hierarchical technology efficacy assessment framework:
Level 6: Society efficacy (cost vs overall societal benefit of testing).1,2 Appropriate use criteria and their association with abnormal TTE findings are consistent with level 2 of this scheme. Our study, although with limitations, attempts to address level 4. Although our study was retrospective, prior prospective studies from the 1990s that assessed the clinical impact of TTE found active change in care in 36% to 38% of cases, consistent with the 32% active change we saw in our study.3-5 A prime tenet of the RAND process was that the assignment of appropriateness would need to be formally tested to see if it led to improved outcomes and modified if it did not.6 For TTE AUC, this would require validating if criteria rated as appropriate accomplish the ultimate objective of impacting patient care. Despite significant growth in TTE volume over the past 2 decades and the development of AUC, clinical impact has remained constant. Better metrics that reflect the incremen-
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
jamainternalmedicine.com
Letters
tal benefit of testing on diagnostic thinking, therapy, and patient outcomes must be pursued to maximize the societal benefit of TTE. Susan A. Matulevicius, MD, MSCS Sandeep R. Das, MD, MPH Sharon C. Reimold, MD Author Affiliations: Department of Medicine, University of Texas Southwestern Medical Center, Dallas. Corresponding Author: Susan A. Matulevicius, MD, MSCS, Department of Medicine, University of Texas Southwestern, 5909 Harry Hines Blvd, Dallas, TX 75390-9047 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Mark DB, Anderson JL, Brinker JA, et al. ACC/AHA/ASE/ASNC/HRS/IAC/ Mended Hearts/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR/SNMMI 2014 health policy statement on use of noninvasive cardiovascular imaging: a report of the American College of Cardiology Clinical Quality Committee. J Am Coll Cardiol. 2014;63(7):698-721. 2. Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making. 1991;11(2):88-94. 3. Matulevicius SA, Rohatgi A, Das SR, Price AL, DeLuna A, Reimold SC. Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med. 2013;173(17):1600-1607. 4. Tam JW, Nichol J, MacDiarmid AL, Lazarow N, Wolfe K. What is the real clinical utility of echocardiography? a prospective observational study. J Am Soc Echocardiogr. 1999;12(9):689-697. 5. Waggoner AD, Harris KM, Braverman AC, Barzilai B, Geltman EM. The role of transthoracic echocardiography in the management of patients seen in an outpatient cardiology clinic. J Am Soc Echocardiogr. 1996;9(6):761-768. 6. Fitch K, Bernstein SJ, Aguilar MS, et al. The RAND/UCLA Appropriateness Method User's Manual. Santa Monica, CA: RAND Corporation; 2001.
Estimating Overdiagnosis in Lung Cancer Screening To the Editor We read with interest the article by Patz et al1 investigating overdiagnosis in the National Lung Screening Trial (NLST). In their investigation, the authors found the upper bound for probability of overdiagnosis to be 11.0% to 18.5% for all lung cancers and even higher for bronchioloalveolar carcinoma (BAC) (67.6% to 78.9%). However, this risk assessment did not consider the lead- and the length-time biases.2 If data for non-BAC non–small cell lung cancer (NSCLC) (from Table 1 of the article by Patz et al1) had been plotted, we would have observed that the data actually distributed into 2 phases. First, in the active screening period, more non-BAC NSCLCs are diagnosed in the low-dose computed tomography (LDCT) arm than in the chest radiography (CXR) arm, emphasizing that some tumors are detected early by LDCT but not by CXR (overdiagnosis). During the second period beginning at the end of screening, the number of non-BAC NSCLCs became lower in the LDCT arm than in the CXR arm. This suggests the tumors not previously diagnosed by CXR were diagnosed later (lead-time bias) or grew slowly (length-time bias) but ultimately became symptomatic. Therefore, an accurate way to estimate actual overdiagnosis risk would be to use areas between the 2 curves in these 2 phases. We calculated a 5.3% larger area in the first phase than in the second one. We believe this proportion is a better estimator of overdiagnosis because it integrates time biases. Moreover, some previous lung cancer screening trials demonstrated
that follow-up have to be extended up to 9 or 10 years to completely take into account both overdiagnosis and timebased biases.3,4 Conversely, numbers of diagnosed BACs in the LDCT arm are much higher than in the CXR arm during but overlap after the active screening period, up to study year 6. This illustrates that BAC are at high-risk of overdiagnosis during the screening period without effects of time-based biases. Indeed, BACs are mainly ground glass opacities with very mild growth. Using a volume-doubling time approach for nodule management (as done by the NELSON [Nederlands Leuvens Longkanker Screenings Onderzoek] trialists), would have resulted in ruling out these lesions as positive screening results.5 In summary, lung cancer screening highlights 2 different clusters. Non-BAC NSCLC could be overdiagnosed. Such a risk should be investigated and explained to patients, but it is overestimated by Patz et al.1 Otherwise, BACs are at high risk for overdiagnosis, but this risk may be easily managed by using a volume-monitoring approach and should not hide the true benefit for lung cancer screening on lung cancer mortality. Sébastien Couraud, MD, MSc Laurent Greillier, MD, PhD Bernard Milleron, MD; for the IFCT Lung Cancer Screening Group Author Affiliations: Service de Pneumologie Aiguë Spécialisée et Cancérologie Thoracique, CH Lyon Sud, Hospices Civils de Lyon, Pierre Bénite, France (Couraud); Faculté de Médecine Lyon-Sud, Université Lyon 1, Oullins, France (Couraud); Aix Marseille Univ–Assistance Publique–Hôpitaux de Marseille, Multidisciplinary Oncology & Therapeutic Innovations Department, Marseille, France (Greillier); Respiratory Disease Department, Tenon Hospital, Assistance Publique–Hôpitaux de Paris, Paris, France (Milleron); Intergroupe Francophone de Cancérologie Thoracique (IFCT), Paris, France (Milleron). Corresponding Author: Sébastien Couraud, MD, MSc, Service de Pneumologie Aiguë Spécialisée et Cancérologie Thoracique, CH Lyon Sud, Hospices Civils de Lyon, 165 Chemin du Grand Revoyet, 69495 Pierre Bénite CEDEX, France ([email protected]). Conflict of Interest Disclosures: None reported. Group Information: Members of the IFCT Lung Cancer Screening Group are listed in Ann Oncol. 2013;24(3):586-597. 1. Patz EF Jr, Pinsky P, Gatsonis C, et al; NLST Overdiagnosis Manuscript Writing Team. Overdiagnosis in low-dose computed tomography screening for lung cancer. JAMA Intern Med. 2014;174(2):269-274. 2. Patz EF Jr, Goodman PC, Bepler G. Screening for lung cancer. N Engl J Med. 2000;343(22):1627-1633. 3. Marcus PM, Bergstralh EJ, Zweig MH, Harris A, Offord KP, Fontana RS. Extended lung cancer incidence follow-up in the Mayo Lung Project and overdiagnosis. J Natl Cancer Inst. 2006;98(11):748-756. 4. Oken MM, Hocking WG, Kvale PA, et al; PLCO Project Team. Screening by chest radiograph and lung cancer mortality: the Prostate, Lung, Colorectal, and Ovarian (PLCO) randomized trial. JAMA. 2011;306(17):1865-1873. 5. Horeweg N, van der Aalst CM, Vliegenthart R, et al. Volumetric computed tomography screening for lung cancer: three rounds of the NELSON trial. Eur Respir J. 2013;42(6):1659-1667.
To the Editor Patz et al1 attempt to report on the rate of overdiagnosis that occurred as part of the National Lung Cancer Screening Trial (NLST), however their conclusion that “the probability is 18.5%…that any lung cancer detected by LDCT
jamainternalmedicine.com
JAMA Internal Medicine July 2014 Volume 174, Number 7
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Manitoba User on 06/05/2015
1197
