Letters COMMENTS
AND
Annals of Internal Medicine RESPONSES
The Seductress TO THE EDITOR: Johnson (1) passionately shares his experience
working with a heroin addict. What his essay lacks is acknowledgment that addiction is a disease that, like cancer, is best left to professionals who know how to treat it. Far too often, physicians and other helpers impede recovery by their kindness and enabling behavior. The goal in treating patients with addiction is to form a therapeutic relationship to support them in a process of engaging in behaviors that they do not yet believe in, understand, or think will work in their particular case. Neither an authoritarian, angry approach nor an overtly codependent stance is beneficial. Addiction is cunning, baffling, and powerful. It is a disease that takes hostages, runs on fear, and results in a set of skills in manipulation and denial that humbles those of us who live or have lived with the active disease. I have experienced these aspects of addiction while living with my husband, a physician, when he was in the throes of his disease. We have 24 years of recovery and a long history with this illness. Professionals trained in addiction medicine skillfully intervened and supported my husband as he entered recovery. In the wake of his early attempts at recovery, he left an assortment of internists, therapists, and psychiatrists. Recovery became accessible through a combination of compassion and tough love offered by the physicians trained in addiction medicine. My husband, the addict, became willing to follow directions, and I, the codependent, learned to detach with love and allow him the dignity of recovering without my loving interference. Lynn Malinoff, EdD Eastern Michigan University Ypsilanti, Michigan Acknowledgment: Dr. Malinoff wrote this letter in collaboration with her husband, Herbert Malinoff, MD. He helped edit and revise the letter and contributed to the content. Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0181.
Reference 1. Johnson RA. The seductress. Ann Intern Med. 2014;160:209. [PMID: 24658699] doi:10.7326/M13-0962
IN RESPONSE: Dr. Malinoff provides an engaging and highly personal perspective on addiction from a unique vantage point: that of the spouse of a recovering addict and physician. She rightly points out that treatment of addiction is best left to professionals trained in addiction medicine and reiterates the power that addiction holds over those trapped by it. I know all too well from my family’s experience with addiction that it is a disease as abhorrent as cancer or any other chronic medical condition that requires daily attention to survive.
In my essay, I chose to focus on the dilemma faced by physicians not trained in addiction medicine who regularly take care of the medical complications of addiction and the other health care needs of addicts on the wards and in the clinic. In our current system, specialists trained in addiction medicine cannot feasibly fill all of these roles. Physicians with training in all backgrounds will inevitably encounter addicts in various stages of recovery throughout our careers. The challenge in these settings is to walk the thin line between enabling through naivete while not acting judgmental or condescending to this patient population when they present to our care or fall short of our expectations. Russell Andrew Johnson, MD, MSc University of Utah School of Medicine Salt Lake City, Utah Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽M13-0962.
Treatment of Primary Sjo¨gren Syndrome With Rituximab TO THE EDITOR: Devauchelle-Pensec and colleagues (1) conclude that their data do not support the use of rituximab in most patients with recent-onset or systemic primary Sjo¨gren syndrome (pSS). We agree that the best outcome measure for assessing treatment efficacy in pSS is debatable. The European League Against Rheumatism Sjo¨gren’s Syndrome Disease Activity Index (ESSDAI) is a major step toward the development of standardized outcome measures for pSS but does not incorporate assessments of dryness. The ESSDAI was responsive to disease activity in patients with pSS treated with rituximab (2) when examined in a prospective single-center trial. However, this index has not been extensively validated in controlled trials. The primary outcome measure reported by Devauchelle-Pensec and colleagues is a 30-mm improvement from baseline by week 24 on a visual analogue scale (VAS) for 2 of 4 measures derived from expert opinion. If a participant had a baseline VAS score of 51 mm, a 30-mm improvement would amount to a 60% response. As the editors caution, outcome measurements may have been insensitive for detecting improvement. A VAS dryness score may not capture improvement in moisture production or subsequent alleviation of tissue damage. In a smaller randomized, controlled trial of rituximab in pSS, such functional measurements as oral salivary flow and ocular lissamine green staining did show significant improvement (3). Devauchelle-Pensec and colleagues reported that individual outcome measures significantly improved in their study, which used 2 rituximab infusions 14 days apart. Figure 2 of their article shows that VAS dryness scores improved (global P ⫽ 0.022) when analyzed longitudinally after adjustment for baseline characteristics. The optimal rituximab regimen required to show improvement in pSS is unclear. Salivary gland B cells deplete by 16 weeks after 2 rituximab infusions and repopulate by 52 weeks (4). However, exactly when during this interval repopulation occurs is unknown. Pers and associates (4) suggest that increasing the number of rituximab infusions may prolong tissue B-cell depletion. Carubbi and coworkers (5) showed statistically significant improvement by
376 © 2014 American College of Physicians
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters administering rituximab at 24-week intervals, resulting in participants receiving approximately 5 rituximab cycles over 120 weeks. This cohort had a disease duration of 1 year compared with 5 to 6 years reported by Devauchelle-Pensec and colleagues and Meijer and associates (3) and a higher baseline ESSDAI score (19.8) than rituximab recipients in Devauchelle-Pensec and colleagues’ study (10.0). We believe that B-cell depletion may be considered a therapeutic option for certain patients with pSS. The issues that we mention underscore the need for continued development of sensitive and reproducible outcome measures for pSS and further study of B-cell depletion regimens. Denise L. Faustman, MD, PhD Harvard Medical School Boston, Massachusetts Frederick B. Vivino, MD Penn Presbyterian Medical Center, University of Pennsylvania Health System Philadelphia, Pennsylvania Steven E. Carsons, MD Winthrop-University Hospital Campus, Stony Brook University School of Medicine Mineola, New York Note: The authors are members of the Sjo¨gren’s Syndrome Foundation Medical and Scientific Advisory Board. Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽L14-0206. References 1. Devauchelle-Pensec V, Mariette X, Jousse-Joulin S, Berthelot JM, Perdriger A, Pue´chal X, et al. Treatment of primary Sjo¨gren syndrome with rituximab: a randomized trial. Ann Intern Med. 2014;160:233-42. [PMID: 24727841] 2. Meiners PM, Arends S, Brouwer E, Spijkervet FK, Vissink A, Bootsma H. Responsiveness of disease activity indices ESSPRI and ESSDAI in patients with primary Sjo¨gren’s syndrome treated with rituximab. Ann Rheum Dis. 2012;71:1297-302. [PMID: 22258489] doi:10.1136/annrheumdis-2011-200460 3. Meijer JM, Meiners PM, Vissink A, Spijkervet FK, Abdulahad W, Kamminga N, et al. Effectiveness of rituximab treatment in primary Sjo¨gren’s syndrome: a randomized, double-blind, placebo-controlled trial. Arthritis Rheum. 2010;62:960-8. [PMID: 20131246] doi:10.1002/art.27314 4. Pers JO, Devauchelle V, Daridon C, Bendaoud B, Le Berre R, Bordron A, et al. BAFF-modulated repopulation of B lymphocytes in the blood and salivary glands of rituximab-treated patients with Sjo¨gren’s syndrome. Arthritis Rheum. 2007;56:146477. [PMID: 17469105] 5. Carubbi F, Cipriani P, Marrelli A, Benedetto P, Ruscitti P, Berardicurti O, et al. Efficacy and safety of rituximab treatment in early primary Sjo¨gren’s syndrome: a prospective, multi-center, follow-up study. Arthritis Res Ther. 2013;15:R172. [PMID: 24286296] doi:10.1186/ar4359
IN RESPONSE: We fully agree that the ideal primary end point for assessing treatment efficacy in pSS remains unknown and that the best option among those available to date (the VAS, European League Against Rheumatism Sjo¨gren’s Syndrome Patient Reported Index [1], the ESSDAI [2], and physician opinion) is undecided. We also concur that the best cutoff for defining an improvement in the VAS score in millimeters or a percentage cannot be reliably deterwww.annals.org
mined because we have no data on the minimal clinically important differences for these measures. Therefore, there is no sound basis on which to select patients for biologic treatment or to choose the best treatment regimen (rituximab dose, concomitant medications, or retreatment) or the primary efficacy end point for assessing treatments. To our knowledge, our study is the first large randomized trial showing a small but rapid effect of rituximab on fatigue and a delayed and substantially variable response on dryness (Appendix Figure, available at www.annals.org). Post hoc analyses of the results will be of use to define variables for future studies. The effects of rituximab in our study invite a discussion about the usefulness of new trials of anti–B-cell treatments in pSS. We believe that such studies are needed to obtain definite conclusions and should benefit from the experience of our trial. Our data indicate that using fatigue and/or dryness at week 6, or perhaps later in the event of re-treatment, as the primary objective would be expected to show an effect of rituximab. However, we need evidence on which to base cutoff values. New validated efficacy criteria, including objective signs, and an evaluation of the relevance of these criteria for patients, clinicians, and payers also are needed. We anticipate that the ongoing TRACTISS (Trial of Anti– B-Cell Therapy in Patients With Primary Sjo¨gren’s Syndrome) protocol (3) will show the efficacy of rituximab, given the patient inclusion criteria and the use of a 30% improvement in the VAS score for fatigue or oral dryness as the primary end point. Our objectives now, which we are pursuing in collaboration with physicians and researchers previously involved in metrological assessments in pSS, are to determine the minimal clinically important differences for the ESSDAI and European League Against Rheumatism Sjo¨gren’s Syndrome Patient Reported Index and allow an assessment of the relevance of the TRACTISS protocol results before the end of TRACTISS, to suggest a new primary efficacy end point based on a post hoc analysis of data from our study and then apply this primary end point to TRACTISS results, and to use data from the ASSESS (Assessment of Systemic Signs and Evolution of Sjo¨gren’s Syndrome) trial cohort (4) and our study to estimate the required sample size for future studies on pSS according to the chosen primary end point. Alain Saraux, MD, PhD Hoˆpital de la Cavale Blanche Brest, France Emmanuel Nowak, PhD Institut National de la Sante´ et de la Recherche Me´dicale, Centre d’Investigation Clinique 0502, Centre Hospitalier Universitaire de la Cavale Blanche Brest, France Vale´rie Devauchelle-Pensec, MD, PhD Hoˆpital de la Cavale Blanche Brest, France Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-1085. References 1. Seror R, Ravaud P, Mariette X, Bootsma H, Theander E, Hansen A, et al; EULAR Sjo¨gren’s Task Force. EULAR Sjogren’s Syndrome Patient Reported Index (ESSPRI): development of a consensus patient index for primary Sjogren’s syndrome. 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5 377
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters Ann Rheum Dis. 2011;70:968-72. [PMID: 21345815] doi:10.1136/ard.2010 .143743 2. Seror R, Ravaud P, Bowman SJ, Baron G, Tzioufas A, Theander E, et al; EULAR Sjo¨gren’s Task Force. EULAR Sjogren’s syndrome disease activity index: development of a consensus systemic disease activity index for primary Sjogren’s syndrome. Ann Rheum Dis. 2010;69:1103-9. [PMID: 19561361] doi:10.1136/ard.2009.110619 3. Brown S, Navarro Coy N, Pitzalis C, Emery P, Pavitt S, Gray J, et al; TRACTISS trial team. The TRACTISS protocol: a randomised double blind placebo controlled clinical trial of anti-B-cell therapy in patients with primary Sjo¨gren’s Syndrome. BMC Musculoskelet Disord. 2014;15:21. [PMID: 24438039] doi:10.1186/1471-247415-21 4. Tobo´n GJ, Saraux A, Gottenberg JE, Quartuccio L, Fabris M, Seror R, et al. Role of Fms-like tyrosine kinase 3 ligand as a potential biologic marker of lymphoma in primary Sjo¨gren’s syndrome. Arthritis Rheum. 2013;65:3218-27. [PMID: 23982978] doi:10.1002/art.38129
Cost-Effectiveness of Genotype-Guided and Dual Antiplatelet Therapies TO THE EDITOR: Kazi and colleagues’ (1) cost-effectiveness analysis
of antiplatelet therapies in the acute coronary syndrome (ACS) is an interesting attempt that I consider misleading because it neglects 2 factors: the uncertainty in the results of the clinical trials of prasugrel and ticagrelor and the differences between ACS with and without ST-segment elevation myocardial infarction (STEMI). The authors may have neglected these factors because they relied on the published literature and apparently did not incorporate the U.S. Food and Drug Administration (FDA) reviews of prasugrel (2) and ticagrelor (3) that discuss these factors in detail. The FDA reviews suggest that the most striking result in TRITON-TIMI (Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition With Prasugrel— Thrombolysis in Myocardial Infarction) was an early mortality benefit in patients with STEMI, that mortality equalized later in TRITON, and that late site-reported event rates also did not significantly differ between the prasugrel and clopidogrel groups. The reviews also suggest that the results for ticagrelor from the PLATO (Platelet Inhibition and Patient Outcomes) study are questionable because of data quality issues discussed extensively in the FDA review materials; that short-term results in patients managed with angioplasty, particularly those with STEMI, were worse with ticagrelor than clopidogrel; and that a significant interaction for mortality between ticagrelor and baseline statin use may explain the mortality benefit of ticagrelor, probably because ticagrelor substantially increases the exposures of simvastatin and atorvastatin. The most effective approach may be to use prasugrel for the initial treatment of ACS, then switch to clopidogrel for long-term use (and scrupulously administer statins). One FDA advisory committee member recently commented that some clinicians had adopted this approach, switching patients with ACS to prasugrel after 30 days. This approach is not supported directly by a randomized trial; however, neither are Kazi and colleagues’ cross-trial comparisons, which they state in their limitations. I project that switching from prasugrel to clopidogrel is also more cost-effective than the approaches that Kazi and colleagues analyzed. This discussion has many implications beyond costeffectiveness. While we slavishly calculate P values and qualityadjusted life-years, we neglect data quality issues that make these
calculations meaningless. We do not know how best to characterize data quality or how to incorporate it quantitatively into journal articles, drug labels, and clinical conclusions. Because we ignore or minimize data quality issues in journal articles and drug labels, drug companies are not motivated to address these issues with additional studies. Thomas A. Marciniak, MD U.S. Food and Drug Administration Silver Spring, Maryland Disclaimer: This letter reflects the views of the author and should not be
construed to represent the views or policies of the FDA. Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0208. References 1. Kazi DS, Garber AM, Shah RU, Dudley RA, Mell MW, Rhee C, et al. Costeffectiveness of genotype-guided and dual antiplatelet therapies in acute coronary syndrome. Ann Intern Med. 2014;160:221-32. [PMID: 24727840] 2. U. S. Food and Drug Administration. Drug approval package: Effient (prasugrel) tablets: medical review(s). 2009. Accessed at www.accessdata.fda.gov/drugsatfda_docs /nda/2009/022307s000TOC.cfm on 30 July 2014. 3. U. S. Food and Drug Administration. Drug approval package: Brilinta (ticagrelor) tablets: medical review(s). 2011. Accessed at www.accessdata.fda.gov/drugsatfda_docs /nda/2011/022433Orig1s000TOC.cfm on 30 July 2014.
IN RESPONSE: Dr. Marciniak highlights some of the many sources of uncertainty about the choice of an antiplatelet agent for patients receiving percutaneous coronary intervention after ACS. Uncertainty in results of clinical research is common, which is precisely why a systematically and transparently designed simulation model is useful. We reviewed more than 100 publications, including publicly available FDA documents, to identify the most credible estimates of each parameter in the model. A key feature of the model is that, beyond the use of the single best estimate of key parameters (the “base case”), it explicitly assesses the effect of uncertainty in the input parameters. We reported extensive sensitivity analyses and interpreted our results cautiously, noting that genotype-guided use of prasugrel seemed to consistently outperform the prasugrel-for-all strategy even as the role of genotyping with ticagrelor was less certain. Although we acknowledge the limitations inherent in such analyses, we disagree that mathematical modeling is equivalent to “slavishly [calculating] P values” and that data quality issues “make . . . calculations meaningless.” Indeed, we counter that a fundamental strength of our approach is the systematic use of sensitivity analyses to determine whether the inevitable uncertainty in parameters matters enough to change conclusions (1). We agree that patients’ responses to antiplatelet agents may differ. However, like the FDA advisory committees that evaluated these drugs, we were concerned about overinterpreting subgroup analyses. We did not separately model patients presenting with and without STEMI because this factor did not modify the effectiveness of ticagrelor or prasugrel in randomized trials. The tests for interaction of ticagrelor with doses of statins or aspirin were post hoc analyses and not prespecified, and we were concerned that making multiple comparisons would result in an
378 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
www.annals.org
Letters unacceptably high type I error (2). For example, the test for interaction between ticagrelor and statin dose was one of at least 45 analyses; at an ␣ of 0.05, the probability of finding a spurious association would be greater than 90%. The strategy of switching antiplatelet agents suggested by Dr. Marciniak is interesting but, as he acknowledges, untested. It overlooks what we know about genotype-specific responses to antiplatelet agents and minimizes the risk in switching between drugs with distinctly different pharmacokinetics. Instead of relying on anecdotal clinical experience or a back-of-the-envelope cost-effectiveness analysis, we suggest a formal simulation model of this strategy, which would provide useful insights and could guide the design of an empirical test. Dhruv S. Kazi, MD, MSc, MS University of California, San Francisco San Francisco, California Douglas K. Owens, MD, MS Veterans Affairs Palo Alto Healthcare System Palo Alto, California Mark A. Hlatky, MD Stanford University School of Medicine Stanford, California Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-1999. References 1. Basu S, Andrews J. Complexity in mathematical models of public health policies: a guide for consumers of models. PLoS Med. 2013;10:e1001540. [PMID: 24204214] doi:10.1371/journal.pmed.1001540 2. Sainani KL. The problem of multiple testing. PM R. 2009;1:1098-103. [PMID: 20006317] doi:10.1016/j.pmrj.2009.10.004
Random-Effects Meta-analysis of Inconsistent Effects TO THE EDITOR: Cornell and colleagues (1) have presented a particularly cogent discussion of the technical problems of calculating the “final answer” in meta-analysis. I have struggled to understand and teach these concepts for years, and their explanations are extremely helpful. My goal in teaching young physicians about meta-analysis is modest: I want them to be savvy—and skeptical—readers. At the risk of oversimplification, I seek useful generalizations. In critically reading a meta-analysis, I am a proponent of using common sense and suggest that readers can, to some extent, apply their own judgment, particularly on the issue of qualitative heterogeneity. As for quantitative heterogeneity and the calculation of the summary numbers, I am frankly happy if students understand the basic concepts and merely appreciate that the particular choice of fixed- or randomeffects modeling technique in a given meta-analysis is a technical issue beyond the understanding of most readers. Cornell and colleagues’ examples suggest that the various random-effects models produce similar point estimates of the combined effect and mostly differ in the magnitude of the CI. Would it be fair to consider this concept as a general principle? Namely, is it www.annals.org
probable that the point estimate of the summary number will not vary greatly depending on the method chosen but that the CI will? In other words, should the (nontechnical) reader of a meta-analysis generally be more skeptical about the CI and less about the estimated effect value? Such a principle implies that, when a meta-analysis concludes that an intervention is valuable but the CI is fairly close to the “no-effect” value, we should be skeptical if the older calculation methods, such as the DerSimonian–Laird estimator, have been used. David Lander, MD Edward Via Virginia College of Osteopathic Medicine Blacksburg, Virginia Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0310. Reference 1. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160:267-70. [PMID: 24727843] doi:10.7326/M13-2886
TO THE EDITOR: We read Cornell and colleagues’ (1) article with
great interest. Meta-analysis, a statistical synthesis of independent but similar studies, has been widely used to derive a quantitative summary of the available evidence. Currently, the random-effects model based on the DerSimonian–Laird estimator (2) is generally the standard approach unless there is a strong a priori scientific belief about the homogeneous nature of the study effect. Because results from meta-analyses have great implications for guiding clinical practice and making public health recommendations, there are important considerations for revisiting meta-analysis methodology. Cornell and colleagues highlight 1 problem with this method: The DerSimonian–Laird random-effects model does not account for uncertainty in estimating (that is, between-study heterogeneity), resulting in summary 95% CIs that are too narrow. Thus, the authors recommend alternative methods to construct more accurate 95% CIs. Another criticism of the DerSimonian–Laird method is that the point estimate from the random-effects model gives disproportionately large weights to small, less reliable studies that are probably of lower quality (3). With the point and interval estimates of this current “gold standard” approach being called into question, how should we redirect the standard practice in this field? The method presented by Shore and associates (4) provides some direction. It adopts a point estimate directly from the fixedeffects model, weighting studies according to precision, but adjusts its 95% CI by inflating the variance by the ratio of chi-square statistics to degrees of freedom from the heterogeneity test, taking into account the between-study variation. An additional alternative is a hybrid of fixed- and random-effects models that presents a point estimate from the fixed-effects model and a 95% CI accounting for the uncertainty in estimating as in the Knapp–Hartung approach or other methods (1). Simulation studies comparing Shore and associates’ method with such new hybrid approaches are warranted to advance the field of meta-analysis. Above all, it is important to recognize that the ultimate purpose of meta-analysis is not to derive a single quantitative summary but to identify sources of heterogeneity, if they exist. In the presence of true 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5 379
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters between-study variation beyond random sampling variation, heterogeneity is not something to be muffled statistically but rather to be discovered, like a pearl in the mud. Thus, heterogeneity should not be considered an obstacle to obtaining a single quantitative summary but rather an opportunity to find a subgroup of persons who might benefit most from the findings. NaNa Keum, MS Chung-Cheng Hsieh, ScD Nancy Cook, ScD Harvard School of Public Health Boston, Massachusetts Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0309. References 1. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160:267-70. [PMID: 24727843] doi:10.7326/M13-2886 2. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177-88. [PMID: 3802833] 3. Peto R, Awasthi S, Read S, Clark S, Bundy D. Vitamin A supplementation in Indian children—authors’ reply [Letter]. Lancet. 2013;382:594-6. [PMID: 23953381] doi:10.1016/S0140-6736(13)61741-9 4. Shore RE, Gardner MJ, Pannett B. Ethylene oxide: an assessment of the epidemiological evidence on carcinogenicity. Br J Ind Med. 1993;50:971-97. [PMID: 8280635]
IN RESPONSE: Statistical heterogeneity increases our uncertainty
about the magnitude, expected clinical variation, and clinical informativeness of the available evidence for the relative benefits and harms of a medical intervention. Random-effects models estimate the additional uncertainty that we have about the expected clinical variation in the relative effectiveness of a treatment. Quantitatively, the added uncertainty increases the variance and produces a wider CI. Dr. Lander and Ms. Keum and colleagues focus on a less discussed issue in meta-analysis: differences in the point estimate for the pooled treatment effect between the fixed- and random-effects models. Random-effects models reweight the individual study estimates so that the results from smaller studies have a greater influence on the overall estimate. In general, the various random-effects models produce fairly similar estimates for the overall odds ratio that are closer to the estimates from the smaller, less precise studies. As Ms. Keum and colleagues note, random-effects models are susceptible to small-
study effects. Thus, the less precise and perhaps lower-quality trials unduly influence the pooled treatment effect. Ms. Keum and colleagues suggest that we consider a hybrid 2-step approach proposed by Armitage (1). A fixed-effects model is used to estimate the overall odds ratio. A heterogeneity-adjusted variance is used to estimate the 95% CI. It is an interesting, ad hoc method that shares some features of the Knapp–Hartung approach. When this method is applied to the preeclampsia data, the point and interval estimates for the overall odds ratio are 0.67 (95% CI, 0.49 to 0.92). The CI estimates clearly fall somewhere between the DerSimonian–Laird estimate and other random-effects CIs. Small-study effects on the point estimate are amplified when the smaller, less precise studies yield odds ratios that are farther from the null. The greater the asymmetry in funnel plots, the greater the difference between the fixed- and random-effects estimates. There is some evidence of asymmetry in the funnel plot for the preeclampsia example, although little statistical evidence for “publication bias” is available (Harbord regression test; P ⫽ 0.56). Nevertheless, giving proportionally more weight to smaller studies can clearly produce odds ratios that suggest a greater benefit for diuretics than the fixed-effects estimate does. As such, the randomeffects estimate seems less conservative than the fixed-effects estimate. However, in this case, the 95% CI is considerably wider and includes 1.0. The simplicity of Armitage’s hybrid approach is appealing, but we clearly need additional simulation studies to assess whether it adequately accounts for the uncertainty. It is also important to consider the conditions that produce discrepant estimates for the odds ratio. Risk of bias, degree of statistical heterogeneity, and small-study effects all need to be carefully weighed when selecting the most appropriate statistical model to combine the evidence. However, the best decision may be to refrain from pooling, critically evaluate the limitations in the available evidence, and focus on the best available evidence produced by the few larger clinically and methodologically robust trials. John E. Cornell, PhD University of Texas Health Science Center at San Antonio San Antonio, Texas Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-2886. Reference 1. Armitage P. Statistical considerations: conclusion. In: Wald NJ, Doll R, eds. Interpretation of Negative Epidemiological Evidence for Carcinogenicity. Lyon, France: International Agency for Research on Cancer; 1985:190.
380 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
www.annals.org
Annals of Internal Medicine Appendix Figure. VAS scores for fatigue and dryness for each rituximab-treated patient with baseline scores >50 mm.
VAS ⫽ visual analogue scale.
www.annals.org
2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
AND
Annals of Internal Medicine RESPONSES
The Seductress TO THE EDITOR: Johnson (1) passionately shares his experience
working with a heroin addict. What his essay lacks is acknowledgment that addiction is a disease that, like cancer, is best left to professionals who know how to treat it. Far too often, physicians and other helpers impede recovery by their kindness and enabling behavior. The goal in treating patients with addiction is to form a therapeutic relationship to support them in a process of engaging in behaviors that they do not yet believe in, understand, or think will work in their particular case. Neither an authoritarian, angry approach nor an overtly codependent stance is beneficial. Addiction is cunning, baffling, and powerful. It is a disease that takes hostages, runs on fear, and results in a set of skills in manipulation and denial that humbles those of us who live or have lived with the active disease. I have experienced these aspects of addiction while living with my husband, a physician, when he was in the throes of his disease. We have 24 years of recovery and a long history with this illness. Professionals trained in addiction medicine skillfully intervened and supported my husband as he entered recovery. In the wake of his early attempts at recovery, he left an assortment of internists, therapists, and psychiatrists. Recovery became accessible through a combination of compassion and tough love offered by the physicians trained in addiction medicine. My husband, the addict, became willing to follow directions, and I, the codependent, learned to detach with love and allow him the dignity of recovering without my loving interference. Lynn Malinoff, EdD Eastern Michigan University Ypsilanti, Michigan Acknowledgment: Dr. Malinoff wrote this letter in collaboration with her husband, Herbert Malinoff, MD. He helped edit and revise the letter and contributed to the content. Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0181.
Reference 1. Johnson RA. The seductress. Ann Intern Med. 2014;160:209. [PMID: 24658699] doi:10.7326/M13-0962
IN RESPONSE: Dr. Malinoff provides an engaging and highly personal perspective on addiction from a unique vantage point: that of the spouse of a recovering addict and physician. She rightly points out that treatment of addiction is best left to professionals trained in addiction medicine and reiterates the power that addiction holds over those trapped by it. I know all too well from my family’s experience with addiction that it is a disease as abhorrent as cancer or any other chronic medical condition that requires daily attention to survive.
In my essay, I chose to focus on the dilemma faced by physicians not trained in addiction medicine who regularly take care of the medical complications of addiction and the other health care needs of addicts on the wards and in the clinic. In our current system, specialists trained in addiction medicine cannot feasibly fill all of these roles. Physicians with training in all backgrounds will inevitably encounter addicts in various stages of recovery throughout our careers. The challenge in these settings is to walk the thin line between enabling through naivete while not acting judgmental or condescending to this patient population when they present to our care or fall short of our expectations. Russell Andrew Johnson, MD, MSc University of Utah School of Medicine Salt Lake City, Utah Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽M13-0962.
Treatment of Primary Sjo¨gren Syndrome With Rituximab TO THE EDITOR: Devauchelle-Pensec and colleagues (1) conclude that their data do not support the use of rituximab in most patients with recent-onset or systemic primary Sjo¨gren syndrome (pSS). We agree that the best outcome measure for assessing treatment efficacy in pSS is debatable. The European League Against Rheumatism Sjo¨gren’s Syndrome Disease Activity Index (ESSDAI) is a major step toward the development of standardized outcome measures for pSS but does not incorporate assessments of dryness. The ESSDAI was responsive to disease activity in patients with pSS treated with rituximab (2) when examined in a prospective single-center trial. However, this index has not been extensively validated in controlled trials. The primary outcome measure reported by Devauchelle-Pensec and colleagues is a 30-mm improvement from baseline by week 24 on a visual analogue scale (VAS) for 2 of 4 measures derived from expert opinion. If a participant had a baseline VAS score of 51 mm, a 30-mm improvement would amount to a 60% response. As the editors caution, outcome measurements may have been insensitive for detecting improvement. A VAS dryness score may not capture improvement in moisture production or subsequent alleviation of tissue damage. In a smaller randomized, controlled trial of rituximab in pSS, such functional measurements as oral salivary flow and ocular lissamine green staining did show significant improvement (3). Devauchelle-Pensec and colleagues reported that individual outcome measures significantly improved in their study, which used 2 rituximab infusions 14 days apart. Figure 2 of their article shows that VAS dryness scores improved (global P ⫽ 0.022) when analyzed longitudinally after adjustment for baseline characteristics. The optimal rituximab regimen required to show improvement in pSS is unclear. Salivary gland B cells deplete by 16 weeks after 2 rituximab infusions and repopulate by 52 weeks (4). However, exactly when during this interval repopulation occurs is unknown. Pers and associates (4) suggest that increasing the number of rituximab infusions may prolong tissue B-cell depletion. Carubbi and coworkers (5) showed statistically significant improvement by
376 © 2014 American College of Physicians
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters administering rituximab at 24-week intervals, resulting in participants receiving approximately 5 rituximab cycles over 120 weeks. This cohort had a disease duration of 1 year compared with 5 to 6 years reported by Devauchelle-Pensec and colleagues and Meijer and associates (3) and a higher baseline ESSDAI score (19.8) than rituximab recipients in Devauchelle-Pensec and colleagues’ study (10.0). We believe that B-cell depletion may be considered a therapeutic option for certain patients with pSS. The issues that we mention underscore the need for continued development of sensitive and reproducible outcome measures for pSS and further study of B-cell depletion regimens. Denise L. Faustman, MD, PhD Harvard Medical School Boston, Massachusetts Frederick B. Vivino, MD Penn Presbyterian Medical Center, University of Pennsylvania Health System Philadelphia, Pennsylvania Steven E. Carsons, MD Winthrop-University Hospital Campus, Stony Brook University School of Medicine Mineola, New York Note: The authors are members of the Sjo¨gren’s Syndrome Foundation Medical and Scientific Advisory Board. Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽L14-0206. References 1. Devauchelle-Pensec V, Mariette X, Jousse-Joulin S, Berthelot JM, Perdriger A, Pue´chal X, et al. Treatment of primary Sjo¨gren syndrome with rituximab: a randomized trial. Ann Intern Med. 2014;160:233-42. [PMID: 24727841] 2. Meiners PM, Arends S, Brouwer E, Spijkervet FK, Vissink A, Bootsma H. Responsiveness of disease activity indices ESSPRI and ESSDAI in patients with primary Sjo¨gren’s syndrome treated with rituximab. Ann Rheum Dis. 2012;71:1297-302. [PMID: 22258489] doi:10.1136/annrheumdis-2011-200460 3. Meijer JM, Meiners PM, Vissink A, Spijkervet FK, Abdulahad W, Kamminga N, et al. Effectiveness of rituximab treatment in primary Sjo¨gren’s syndrome: a randomized, double-blind, placebo-controlled trial. Arthritis Rheum. 2010;62:960-8. [PMID: 20131246] doi:10.1002/art.27314 4. Pers JO, Devauchelle V, Daridon C, Bendaoud B, Le Berre R, Bordron A, et al. BAFF-modulated repopulation of B lymphocytes in the blood and salivary glands of rituximab-treated patients with Sjo¨gren’s syndrome. Arthritis Rheum. 2007;56:146477. [PMID: 17469105] 5. Carubbi F, Cipriani P, Marrelli A, Benedetto P, Ruscitti P, Berardicurti O, et al. Efficacy and safety of rituximab treatment in early primary Sjo¨gren’s syndrome: a prospective, multi-center, follow-up study. Arthritis Res Ther. 2013;15:R172. [PMID: 24286296] doi:10.1186/ar4359
IN RESPONSE: We fully agree that the ideal primary end point for assessing treatment efficacy in pSS remains unknown and that the best option among those available to date (the VAS, European League Against Rheumatism Sjo¨gren’s Syndrome Patient Reported Index [1], the ESSDAI [2], and physician opinion) is undecided. We also concur that the best cutoff for defining an improvement in the VAS score in millimeters or a percentage cannot be reliably deterwww.annals.org
mined because we have no data on the minimal clinically important differences for these measures. Therefore, there is no sound basis on which to select patients for biologic treatment or to choose the best treatment regimen (rituximab dose, concomitant medications, or retreatment) or the primary efficacy end point for assessing treatments. To our knowledge, our study is the first large randomized trial showing a small but rapid effect of rituximab on fatigue and a delayed and substantially variable response on dryness (Appendix Figure, available at www.annals.org). Post hoc analyses of the results will be of use to define variables for future studies. The effects of rituximab in our study invite a discussion about the usefulness of new trials of anti–B-cell treatments in pSS. We believe that such studies are needed to obtain definite conclusions and should benefit from the experience of our trial. Our data indicate that using fatigue and/or dryness at week 6, or perhaps later in the event of re-treatment, as the primary objective would be expected to show an effect of rituximab. However, we need evidence on which to base cutoff values. New validated efficacy criteria, including objective signs, and an evaluation of the relevance of these criteria for patients, clinicians, and payers also are needed. We anticipate that the ongoing TRACTISS (Trial of Anti– B-Cell Therapy in Patients With Primary Sjo¨gren’s Syndrome) protocol (3) will show the efficacy of rituximab, given the patient inclusion criteria and the use of a 30% improvement in the VAS score for fatigue or oral dryness as the primary end point. Our objectives now, which we are pursuing in collaboration with physicians and researchers previously involved in metrological assessments in pSS, are to determine the minimal clinically important differences for the ESSDAI and European League Against Rheumatism Sjo¨gren’s Syndrome Patient Reported Index and allow an assessment of the relevance of the TRACTISS protocol results before the end of TRACTISS, to suggest a new primary efficacy end point based on a post hoc analysis of data from our study and then apply this primary end point to TRACTISS results, and to use data from the ASSESS (Assessment of Systemic Signs and Evolution of Sjo¨gren’s Syndrome) trial cohort (4) and our study to estimate the required sample size for future studies on pSS according to the chosen primary end point. Alain Saraux, MD, PhD Hoˆpital de la Cavale Blanche Brest, France Emmanuel Nowak, PhD Institut National de la Sante´ et de la Recherche Me´dicale, Centre d’Investigation Clinique 0502, Centre Hospitalier Universitaire de la Cavale Blanche Brest, France Vale´rie Devauchelle-Pensec, MD, PhD Hoˆpital de la Cavale Blanche Brest, France Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-1085. References 1. Seror R, Ravaud P, Mariette X, Bootsma H, Theander E, Hansen A, et al; EULAR Sjo¨gren’s Task Force. EULAR Sjogren’s Syndrome Patient Reported Index (ESSPRI): development of a consensus patient index for primary Sjogren’s syndrome. 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5 377
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters Ann Rheum Dis. 2011;70:968-72. [PMID: 21345815] doi:10.1136/ard.2010 .143743 2. Seror R, Ravaud P, Bowman SJ, Baron G, Tzioufas A, Theander E, et al; EULAR Sjo¨gren’s Task Force. EULAR Sjogren’s syndrome disease activity index: development of a consensus systemic disease activity index for primary Sjogren’s syndrome. Ann Rheum Dis. 2010;69:1103-9. [PMID: 19561361] doi:10.1136/ard.2009.110619 3. Brown S, Navarro Coy N, Pitzalis C, Emery P, Pavitt S, Gray J, et al; TRACTISS trial team. The TRACTISS protocol: a randomised double blind placebo controlled clinical trial of anti-B-cell therapy in patients with primary Sjo¨gren’s Syndrome. BMC Musculoskelet Disord. 2014;15:21. [PMID: 24438039] doi:10.1186/1471-247415-21 4. Tobo´n GJ, Saraux A, Gottenberg JE, Quartuccio L, Fabris M, Seror R, et al. Role of Fms-like tyrosine kinase 3 ligand as a potential biologic marker of lymphoma in primary Sjo¨gren’s syndrome. Arthritis Rheum. 2013;65:3218-27. [PMID: 23982978] doi:10.1002/art.38129
Cost-Effectiveness of Genotype-Guided and Dual Antiplatelet Therapies TO THE EDITOR: Kazi and colleagues’ (1) cost-effectiveness analysis
of antiplatelet therapies in the acute coronary syndrome (ACS) is an interesting attempt that I consider misleading because it neglects 2 factors: the uncertainty in the results of the clinical trials of prasugrel and ticagrelor and the differences between ACS with and without ST-segment elevation myocardial infarction (STEMI). The authors may have neglected these factors because they relied on the published literature and apparently did not incorporate the U.S. Food and Drug Administration (FDA) reviews of prasugrel (2) and ticagrelor (3) that discuss these factors in detail. The FDA reviews suggest that the most striking result in TRITON-TIMI (Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition With Prasugrel— Thrombolysis in Myocardial Infarction) was an early mortality benefit in patients with STEMI, that mortality equalized later in TRITON, and that late site-reported event rates also did not significantly differ between the prasugrel and clopidogrel groups. The reviews also suggest that the results for ticagrelor from the PLATO (Platelet Inhibition and Patient Outcomes) study are questionable because of data quality issues discussed extensively in the FDA review materials; that short-term results in patients managed with angioplasty, particularly those with STEMI, were worse with ticagrelor than clopidogrel; and that a significant interaction for mortality between ticagrelor and baseline statin use may explain the mortality benefit of ticagrelor, probably because ticagrelor substantially increases the exposures of simvastatin and atorvastatin. The most effective approach may be to use prasugrel for the initial treatment of ACS, then switch to clopidogrel for long-term use (and scrupulously administer statins). One FDA advisory committee member recently commented that some clinicians had adopted this approach, switching patients with ACS to prasugrel after 30 days. This approach is not supported directly by a randomized trial; however, neither are Kazi and colleagues’ cross-trial comparisons, which they state in their limitations. I project that switching from prasugrel to clopidogrel is also more cost-effective than the approaches that Kazi and colleagues analyzed. This discussion has many implications beyond costeffectiveness. While we slavishly calculate P values and qualityadjusted life-years, we neglect data quality issues that make these
calculations meaningless. We do not know how best to characterize data quality or how to incorporate it quantitatively into journal articles, drug labels, and clinical conclusions. Because we ignore or minimize data quality issues in journal articles and drug labels, drug companies are not motivated to address these issues with additional studies. Thomas A. Marciniak, MD U.S. Food and Drug Administration Silver Spring, Maryland Disclaimer: This letter reflects the views of the author and should not be
construed to represent the views or policies of the FDA. Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0208. References 1. Kazi DS, Garber AM, Shah RU, Dudley RA, Mell MW, Rhee C, et al. Costeffectiveness of genotype-guided and dual antiplatelet therapies in acute coronary syndrome. Ann Intern Med. 2014;160:221-32. [PMID: 24727840] 2. U. S. Food and Drug Administration. Drug approval package: Effient (prasugrel) tablets: medical review(s). 2009. Accessed at www.accessdata.fda.gov/drugsatfda_docs /nda/2009/022307s000TOC.cfm on 30 July 2014. 3. U. S. Food and Drug Administration. Drug approval package: Brilinta (ticagrelor) tablets: medical review(s). 2011. Accessed at www.accessdata.fda.gov/drugsatfda_docs /nda/2011/022433Orig1s000TOC.cfm on 30 July 2014.
IN RESPONSE: Dr. Marciniak highlights some of the many sources of uncertainty about the choice of an antiplatelet agent for patients receiving percutaneous coronary intervention after ACS. Uncertainty in results of clinical research is common, which is precisely why a systematically and transparently designed simulation model is useful. We reviewed more than 100 publications, including publicly available FDA documents, to identify the most credible estimates of each parameter in the model. A key feature of the model is that, beyond the use of the single best estimate of key parameters (the “base case”), it explicitly assesses the effect of uncertainty in the input parameters. We reported extensive sensitivity analyses and interpreted our results cautiously, noting that genotype-guided use of prasugrel seemed to consistently outperform the prasugrel-for-all strategy even as the role of genotyping with ticagrelor was less certain. Although we acknowledge the limitations inherent in such analyses, we disagree that mathematical modeling is equivalent to “slavishly [calculating] P values” and that data quality issues “make . . . calculations meaningless.” Indeed, we counter that a fundamental strength of our approach is the systematic use of sensitivity analyses to determine whether the inevitable uncertainty in parameters matters enough to change conclusions (1). We agree that patients’ responses to antiplatelet agents may differ. However, like the FDA advisory committees that evaluated these drugs, we were concerned about overinterpreting subgroup analyses. We did not separately model patients presenting with and without STEMI because this factor did not modify the effectiveness of ticagrelor or prasugrel in randomized trials. The tests for interaction of ticagrelor with doses of statins or aspirin were post hoc analyses and not prespecified, and we were concerned that making multiple comparisons would result in an
378 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
www.annals.org
Letters unacceptably high type I error (2). For example, the test for interaction between ticagrelor and statin dose was one of at least 45 analyses; at an ␣ of 0.05, the probability of finding a spurious association would be greater than 90%. The strategy of switching antiplatelet agents suggested by Dr. Marciniak is interesting but, as he acknowledges, untested. It overlooks what we know about genotype-specific responses to antiplatelet agents and minimizes the risk in switching between drugs with distinctly different pharmacokinetics. Instead of relying on anecdotal clinical experience or a back-of-the-envelope cost-effectiveness analysis, we suggest a formal simulation model of this strategy, which would provide useful insights and could guide the design of an empirical test. Dhruv S. Kazi, MD, MSc, MS University of California, San Francisco San Francisco, California Douglas K. Owens, MD, MS Veterans Affairs Palo Alto Healthcare System Palo Alto, California Mark A. Hlatky, MD Stanford University School of Medicine Stanford, California Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-1999. References 1. Basu S, Andrews J. Complexity in mathematical models of public health policies: a guide for consumers of models. PLoS Med. 2013;10:e1001540. [PMID: 24204214] doi:10.1371/journal.pmed.1001540 2. Sainani KL. The problem of multiple testing. PM R. 2009;1:1098-103. [PMID: 20006317] doi:10.1016/j.pmrj.2009.10.004
Random-Effects Meta-analysis of Inconsistent Effects TO THE EDITOR: Cornell and colleagues (1) have presented a particularly cogent discussion of the technical problems of calculating the “final answer” in meta-analysis. I have struggled to understand and teach these concepts for years, and their explanations are extremely helpful. My goal in teaching young physicians about meta-analysis is modest: I want them to be savvy—and skeptical—readers. At the risk of oversimplification, I seek useful generalizations. In critically reading a meta-analysis, I am a proponent of using common sense and suggest that readers can, to some extent, apply their own judgment, particularly on the issue of qualitative heterogeneity. As for quantitative heterogeneity and the calculation of the summary numbers, I am frankly happy if students understand the basic concepts and merely appreciate that the particular choice of fixed- or randomeffects modeling technique in a given meta-analysis is a technical issue beyond the understanding of most readers. Cornell and colleagues’ examples suggest that the various random-effects models produce similar point estimates of the combined effect and mostly differ in the magnitude of the CI. Would it be fair to consider this concept as a general principle? Namely, is it www.annals.org
probable that the point estimate of the summary number will not vary greatly depending on the method chosen but that the CI will? In other words, should the (nontechnical) reader of a meta-analysis generally be more skeptical about the CI and less about the estimated effect value? Such a principle implies that, when a meta-analysis concludes that an intervention is valuable but the CI is fairly close to the “no-effect” value, we should be skeptical if the older calculation methods, such as the DerSimonian–Laird estimator, have been used. David Lander, MD Edward Via Virginia College of Osteopathic Medicine Blacksburg, Virginia Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0310. Reference 1. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160:267-70. [PMID: 24727843] doi:10.7326/M13-2886
TO THE EDITOR: We read Cornell and colleagues’ (1) article with
great interest. Meta-analysis, a statistical synthesis of independent but similar studies, has been widely used to derive a quantitative summary of the available evidence. Currently, the random-effects model based on the DerSimonian–Laird estimator (2) is generally the standard approach unless there is a strong a priori scientific belief about the homogeneous nature of the study effect. Because results from meta-analyses have great implications for guiding clinical practice and making public health recommendations, there are important considerations for revisiting meta-analysis methodology. Cornell and colleagues highlight 1 problem with this method: The DerSimonian–Laird random-effects model does not account for uncertainty in estimating (that is, between-study heterogeneity), resulting in summary 95% CIs that are too narrow. Thus, the authors recommend alternative methods to construct more accurate 95% CIs. Another criticism of the DerSimonian–Laird method is that the point estimate from the random-effects model gives disproportionately large weights to small, less reliable studies that are probably of lower quality (3). With the point and interval estimates of this current “gold standard” approach being called into question, how should we redirect the standard practice in this field? The method presented by Shore and associates (4) provides some direction. It adopts a point estimate directly from the fixedeffects model, weighting studies according to precision, but adjusts its 95% CI by inflating the variance by the ratio of chi-square statistics to degrees of freedom from the heterogeneity test, taking into account the between-study variation. An additional alternative is a hybrid of fixed- and random-effects models that presents a point estimate from the fixed-effects model and a 95% CI accounting for the uncertainty in estimating as in the Knapp–Hartung approach or other methods (1). Simulation studies comparing Shore and associates’ method with such new hybrid approaches are warranted to advance the field of meta-analysis. Above all, it is important to recognize that the ultimate purpose of meta-analysis is not to derive a single quantitative summary but to identify sources of heterogeneity, if they exist. In the presence of true 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5 379
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
Letters between-study variation beyond random sampling variation, heterogeneity is not something to be muffled statistically but rather to be discovered, like a pearl in the mud. Thus, heterogeneity should not be considered an obstacle to obtaining a single quantitative summary but rather an opportunity to find a subgroup of persons who might benefit most from the findings. NaNa Keum, MS Chung-Cheng Hsieh, ScD Nancy Cook, ScD Harvard School of Public Health Boston, Massachusetts Disclosures: Authors have disclosed no conflicts of interest. Forms can
be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms .do?msNum⫽L14-0309. References 1. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160:267-70. [PMID: 24727843] doi:10.7326/M13-2886 2. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177-88. [PMID: 3802833] 3. Peto R, Awasthi S, Read S, Clark S, Bundy D. Vitamin A supplementation in Indian children—authors’ reply [Letter]. Lancet. 2013;382:594-6. [PMID: 23953381] doi:10.1016/S0140-6736(13)61741-9 4. Shore RE, Gardner MJ, Pannett B. Ethylene oxide: an assessment of the epidemiological evidence on carcinogenicity. Br J Ind Med. 1993;50:971-97. [PMID: 8280635]
IN RESPONSE: Statistical heterogeneity increases our uncertainty
about the magnitude, expected clinical variation, and clinical informativeness of the available evidence for the relative benefits and harms of a medical intervention. Random-effects models estimate the additional uncertainty that we have about the expected clinical variation in the relative effectiveness of a treatment. Quantitatively, the added uncertainty increases the variance and produces a wider CI. Dr. Lander and Ms. Keum and colleagues focus on a less discussed issue in meta-analysis: differences in the point estimate for the pooled treatment effect between the fixed- and random-effects models. Random-effects models reweight the individual study estimates so that the results from smaller studies have a greater influence on the overall estimate. In general, the various random-effects models produce fairly similar estimates for the overall odds ratio that are closer to the estimates from the smaller, less precise studies. As Ms. Keum and colleagues note, random-effects models are susceptible to small-
study effects. Thus, the less precise and perhaps lower-quality trials unduly influence the pooled treatment effect. Ms. Keum and colleagues suggest that we consider a hybrid 2-step approach proposed by Armitage (1). A fixed-effects model is used to estimate the overall odds ratio. A heterogeneity-adjusted variance is used to estimate the 95% CI. It is an interesting, ad hoc method that shares some features of the Knapp–Hartung approach. When this method is applied to the preeclampsia data, the point and interval estimates for the overall odds ratio are 0.67 (95% CI, 0.49 to 0.92). The CI estimates clearly fall somewhere between the DerSimonian–Laird estimate and other random-effects CIs. Small-study effects on the point estimate are amplified when the smaller, less precise studies yield odds ratios that are farther from the null. The greater the asymmetry in funnel plots, the greater the difference between the fixed- and random-effects estimates. There is some evidence of asymmetry in the funnel plot for the preeclampsia example, although little statistical evidence for “publication bias” is available (Harbord regression test; P ⫽ 0.56). Nevertheless, giving proportionally more weight to smaller studies can clearly produce odds ratios that suggest a greater benefit for diuretics than the fixed-effects estimate does. As such, the randomeffects estimate seems less conservative than the fixed-effects estimate. However, in this case, the 95% CI is considerably wider and includes 1.0. The simplicity of Armitage’s hybrid approach is appealing, but we clearly need additional simulation studies to assess whether it adequately accounts for the uncertainty. It is also important to consider the conditions that produce discrepant estimates for the odds ratio. Risk of bias, degree of statistical heterogeneity, and small-study effects all need to be carefully weighed when selecting the most appropriate statistical model to combine the evidence. However, the best decision may be to refrain from pooling, critically evaluate the limitations in the available evidence, and focus on the best available evidence produced by the few larger clinically and methodologically robust trials. John E. Cornell, PhD University of Texas Health Science Center at San Antonio San Antonio, Texas Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M13-2886. Reference 1. Armitage P. Statistical considerations: conclusion. In: Wald NJ, Doll R, eds. Interpretation of Negative Epidemiological Evidence for Carcinogenicity. Lyon, France: International Agency for Research on Cancer; 1985:190.
380 2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
www.annals.org
Annals of Internal Medicine Appendix Figure. VAS scores for fatigue and dryness for each rituximab-treated patient with baseline scores >50 mm.
VAS ⫽ visual analogue scale.
www.annals.org
2 September 2014 Annals of Internal Medicine Volume 161 • Number 5
Downloaded From: https://annals.org/pdfaccess.ashx?url=/data/journals/aim/930714/ by a Universite Laval Biblioteque User on 07/30/2017
