Letters

5. Huesch MD. External adjustment sensitivity analysis for unmeasured confounding: an application to coronary stent outcomes, Pennsylvania 2004-2008. Health Serv Res. 2013;48(3):1191-1214.

In Reply The purpose of our study was to demonstrate that previous population-based studies reporting a patient-level association between intraoperative cholangiography and reduced common duct injury were subject to unmeasured confounding.1-3 The causal direction is not clear when using administrative data because the intent of intraoperative cholangiography (to delineate anatomy or to detect injury) is unknown, and cholangiograms that are attempted—but unsuccessful—are not captured. The instrumental variable analysis eliminated this unmeasured confounding by predicting exposure to intraoperative cholangiography based on an instrument (percentage of hospital intraoperative cholangiography use) and examining its association with common duct injury, rather than actual receipt at the patient level, in which the reason for intraoperative cholangiography use (or lack of use) is unclear. Both letters question the exogeneity of the instrument. As Dr Flum points out, hospitals routinely using intraoperative cholangiography may also have implemented other safetyfirst measures. Even if this were true, it would not change our conclusions for 2 reasons. First, if routine intraoperative cholangiography use were related to other hospital quality measures, our instrumental variable analysis results would be biased in favor of intraoperative cholangiography. Second, if such hospitals had lower rates of common duct injury, which was not observed, the association may be due to any of these quality measures rather than intraoperative cholangiography use itself. Our instrumental variable analysis results apply to the marginal population (ie, patients whose receipt of intraoperative cholangiography depends on the instrument).4 Drs Huesch and Romley correctly state that there are patients who would or would not receive intraoperative cholangiography regardless of the hospital where they are treated. In these patients, the assumption of quasi randomization is violated. We have previously demonstrated significant variation in the use of intraoperative cholangiography, ranging from 2% to 90% of cholecystectomies across surgeons and 4% to 95% across hospitals. Nearly half of the variance in intraoperative cholangiography use was attributable to the surgeon or hospital to which the patient presented.5 Given this variation, a large proportion of patients in our sample would be expected to fall within the marginal population. Another valid concern is the broader applicability of our results if local practices are correlated in small areas such that patients do not have similar chances of being seen by surgeons or hospitals with high or low intraoperative cholangiography use. However, we observed significant variation even within small geographic regions. There were 42 hospitals in the greater Houston area. Despite the geographic proximity, intraoperative cholangiography rates ranged from 0% to 86.3% across hospitals. We agree that the study was potentially underpowered to observe a statistically significant association. Nevertheless, the 2674

point estimate for intraoperative cholangiography was significantly attenuated using instrumental variable analysis. Our study highlights the unmeasured confounding in previous studies and addresses the policy-level question of “Does routine hospital-level intraoperative cholangiography use prevent common duct injury?” Our study does not advocate for intraoperative cholangiography never to be used, and it does not address the benefit of intraoperative cholangiography at the individual patient level (ie, individual patients who may in fact benefit). However, the data do not support mandatory use of intraoperative cholangiography during cholecystectomy as the standard of care. Taylor S. Riall, MD, PhD Kristin M. Sheffield, PhD Yong-Fang Kuo, PhD Author Affiliations: Department of Surgery, University of Texas Medical Branch, Galveston. Corresponding Author: Taylor S. Riall, MD, PhD, University of Texas Medical Branch, 301 University Blvd, JSA 6.110c, Galveston, TX 77555 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Riall and Sheffield reported receiving grants from the National Institutes of Health. Dr Kuo reported a pending grant with the Agency for Healthcare Research and Quality. 1. Fletcher DR, Hobbs MS, Tan P, et al. Complications of cholecystectomy: risks of the laparoscopic approach and protective effects of operative cholangiography: a population-based study. Ann Surg. 1999;229(4):449-457. 2. Flum DR, Dellinger EP, Cheadle A, Chan L, Koepsell T. Intraoperative cholangiography and risk of common bile duct injury during cholecystectomy. JAMA. 2003;289(13):1639-1644. 3. Waage A, Nilsson M. Iatrogenic bile duct injury: a population-based study of 152 776 cholecystectomies in the Swedish Inpatient Registry. Arch Surg. 2006;141(12):1207-1213. 4. Harris KM, Remler DK. Who is the marginal patient? understanding instrumental variables estimates of treatment effects. Health Serv Res. 1998;33(5 pt 1):1337-1360. 5. Sheffield KM, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS, Riall TS. Variation in the use of intraoperative cholangiography during cholecystectomy. J Am Coll Surg. 2012;214(4):668-681.

Spending and Quality of Care for Medicare Beneficiaries in Massachusetts To the Editor The study by Dr McWilliams and colleagues1 on spending and quality of care for Medicare beneficiaries associated with the Alternative Quality Contract (AQC) in Massachusetts contains important threats to validity that raise questions about the reliability of the results. The most important is volunteer bias. This bias was identified as an issue with an earlier evaluation of the AQC,2 but the limitation is not mentioned in the article by McWilliams et al.1 Only 11 medical practices volunteered for participation and these practices had higher Medicare spending per beneficiary than the control practices, which raises the possibility that the practices chose to participate because they knew they could achieve savings and earn bonuses, possibly for activities that were already planned or under way. The practices that declined to participate, which became the control group, may have known that they could

JAMA December 25, 2013 Volume 310, Number 24

Copyright 2013 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by a University of Georgia User on 05/28/2015

jama.com

Letters

not achieve the required savings, further exaggerating the results. Controlling for patient differences between practices does not address the underlying selection bias. In addition, the unit of analysis should have been the medical practice and not the patient because the intervention targeted only the practices. Left unknown is whether the modest savings reported were due to use of lower cost services, changes in price, or changes in billing practices. These important distinctions are needed to assess the effect of the policy. McWilliams et al1 reported a modest association with medical spending (nonmedical spending such as the cost of the intervention was not reported) that cannot be generalized to other settings or practices or disentangled from the study bias, which would tend to exaggerate the effect of the intervention. The study does not provide strong evidence that a broad application of this intervention will achieve savings, improvements in quality measures, or improved quality of care. Rigorous study designs are needed to assess the effect of global payment strategies before these approaches are broadly implemented. Jeffrey Brown, PhD Author Affiliation: Department of Population Medicine, Harvard Medical School, Boston, Massachusetts. Corresponding Author: Jeffrey Brown, PhD, Harvard Pilgrim Health Care Institute, 133 Brookline Ave, Sixth Floor, Boston, MA 02215 ([email protected]). Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported. 1. McWilliams JM, Landon BE, Chernew ME. Changes in health care spending and quality for Medicare beneficiaries associated with a commercial ACO contract. JAMA. 2013;310(8):829-836. 2. Song Z, Safran DG, Landon BE, et al. The ‘Alternative Quality Contract,’ based on a global budget, lowered medical spending and improved quality. Health Aff (Millwood). 2012;31(8):1885-1894.

In Reply We agree with Dr Brown that rigorous research is needed to inform provider payment policy. Because physicians and health care provider organizations are not randomized to different payment systems, however, researchers must turn to quasi-experimental methods to evaluate innovations in health care markets. Contrary to Brown’s assertion, not only did we recognize selection bias as a potential limitation of the quasi-experimental design we used, we tested for it. Specifically, we compared preintervention spending trends in Medicare between the intervention group of organizations participating in the AQC and the control group of nonparticipating organizations. These trends were very similar (as were preintervention trends in Blue Cross Blue Shield spending),1 supporting the identifying assumption of our difference-in-differences approach that spending differences between comparison groups would have remained constant in the absence of intervention. A differential change in spending in the intervention group would not be expected without a differential change in payment incentives. Although there were differences in organi-

zational traits and baseline spending levels between the intervention and control groups, which could reflect a number of factors, these differences in levels do not pose a major threat to the internal validity of our study. We agree with Brown that self-selection of organizations into the AQC limits the external validity or generalizability of our findings. The apparent spillover effects we identified may generalize only to similar organizations implementing similar strategies in response to similar risk contracts. As we acknowledged, our findings also may not generalize beyond Massachusetts. Accordingly, we concluded only that our study suggests the potential for payment models like the AQC to foster systemic changes in care delivery within organizations, and that spillovers should be considered in evaluations of similar initiatives. These conclusions may be overly cautious because spillovers across payers have been identified in other settings,2,3 and findings from AQC evaluations may generalize to more providers as the delivery system integrates in response to demands for cost-effective care. Although the provider group determines exposure to the AQC, we disagree that it is the appropriate unit for statistical analysis. We did not contend that controlling for patient differences between the intervention and control groups would address provider-level selection bias. Rather, we conducted analyses at the beneficiary level to adjust for potential changes in the characteristics of patients served by AQC participants (we found no evidence that these organizations selected healthier patients in response to global budgets). We adjusted standard errors for correlation of per-beneficiary spending within provider groups. As we wrote in the article, changes in spending largely reflected changes in use because prices in Medicare are set administratively. In addition, the modest nature of the savings we identified does not diminish the importance of our findings. The magnitude of spillover effects should be judged relative to the direct effects of payment reforms, which were expectedly modest in years 1 and 2 of the AQC.1,4 Because organizations in the AQC did not receive bonuses for quality of care in Medicare, the full spillover effect suggested by our study would contribute to systemwide savings. J. Michael McWilliams, MD, PhD Bruce E. Landon, MD, MBA, MSc Michael E. Chernew, PhD Author Affiliations: Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts. Corresponding Author: J. Michael McWilliams, MD, PhD, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115 ([email protected] .edu). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Landon reported serving as a consultant to Active Networks, Navigenics, Massachusetts Medical Society, United Biosource, and Research Triangle Institute. Dr Chernew reported board membership on the Medicare Payment Advisory Commission and the Congressional Budget Office. No other disclosures were reported. 1. Song Z, Safran DG, Landon BE, et al. Health care spending and quality in year 1 of the alternative quality contract. N Engl J Med. 2011;365(10):909-918.

jama.com

JAMA December 25, 2013 Volume 310, Number 24

Copyright 2013 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by a University of Georgia User on 05/28/2015

2675

Letters

2. Baker LC. Association of managed care market share and health expenditures for fee-for-service Medicare patients. JAMA. 1999;281(5):432-437. 3. Chernew M, Decicca P, Town R. Managed care and medical expenditures of Medicare beneficiaries. J Health Econ. 2008;27(6):1451-1461. 4. Song Z, Safran DG, Landon BE, et al. The ‘Alternative Quality Contract,’ based on a global budget, lowered medical spending and improved quality. Health Aff (Millwood). 2012;31(8):1885-1894.

Adherence to Diets for Weight Loss To the Editor: In a recent Viewpoint, Drs Pagoto and Appelhans1 made a convincing argument for researching new strategies to improve adherence to lifestyle modifications intended for weight management. We disagree, however, that further study of diets with varying macronutrient composition is no longer helpful. Controversy remains regarding the safety of these diets and their effectiveness for improving health outcomes.2 The various diets that are commonly studied (eg, low-fat, Mediterranean, low-carbohydrate, low-glycemic index, vegetarian) can have widely different metabolic and health effects, particularly when adherence is high and even if weight loss is ultimately similar.3 More importantly, despite voluminous research on diet and health, there is scant high-quality evidence (ie, randomized controlled trials) to support any diet’s beneficial effects on clinical events. Until questions about safety and effectiveness are settled, it may be difficult to convince practitioners to “counsel patients to choose a dietary plan they find easiest to adhere to in the long term.”1 Furthermore, whereas selection of a diet based on an individual’s preference is compelling, it is unclear that this approach will produce maximal weight loss or health benefits; one study found inferior weight loss when patients were assigned to a preferred weight loss diet.4 Controversy also remains regarding how adherable these different diets are, meaning that any research done on adherence to one particular diet may not be generalizable to other approaches. There is no doubt that depictions of research on various diets, both accurate and inaccurate, have led to confusion among patients, clinicians, and experts alike. However, a lack of scientific research on a diet approach is unlikely to reduce confusion or stem the popularity of new dietary approaches, which can become a focus of popular media attention and commercialization regardless of the existence of research. Ignoring approaches that may have different appeal, adherence, and metabolic effects will diminish continued progress toward increased treatment options, targeted therapy, and personalized nutrition. Considering only one diet approach in research can lead to overlooking the shortcomings of that particular approach. In other words, we agree the pursuit of the one ideal diet for everyone should end, but a better understanding of the spectrum of potential ideal diets for every individual should continue to be pursued. William S. Yancy Jr, MD, MHSc Megan A. McVay, PhD Grant D. Brinkworth, PhD 2676

Author Affiliations: Department of Medicine, Duke University Medical Center, Durham, North Carolina (Yancy, McVay); Commonwealth Scientific and Industrial Research Organisation, Adelaide, South Australia, Australia (Brinkworth). Corresponding Author: William S. Yancy Jr, MD, MHSc, VA Medical Center, 508 Fulton St, Durham, NC 27705 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr McVay reported receiving a grant from the Agency for Healthcare Research and Quality. No other disclosures were reported. 1. Pagoto SL, Appelhans BM. A call for an end to the diet debates. JAMA. 2013;310(7):687-688. 2. Noto H, Goto A, Tsujimoto T, Noda M. Low-carbohydrate diets and all-cause mortality: a systematic review and meta-analysis of observational studies. PLoS One. 2013;8(1):e55030. 3. Nordmann AJ, Nordmann A, Briel M, et al. Effects of low-carbohydrate vs low-fat diets on weight loss and cardiovascular risk factors: a meta-analysis of randomized controlled trials. Arch Intern Med. 2006;166(3):285-293. 4. Borradaile KE, Halpern SD, Wyatt HR, et al. Relationship between treatment preference and weight loss in the context of a randomized controlled trial. Obesity (Silver Spring). 2012;20(6):1218-1222.

In Reply Even though they agree with our emphasis on behavioral adherence, Dr Yancy and colleagues advocate for additional diet comparison studies given the controversy regarding their effectiveness for improving health outcomes. Specifically, Yancy and colleagues describe scant highquality evidence to support any diet’s beneficial effects on clinical events. We agree with the importance of demonstrating effects of diet on clinical events (eg, incident diabetes, stroke) rather than on weight and cardiometabolic risk factors alone. However, to the extent that the dietary effects on clinical events are driven by weight loss, the 4 meta-analyses1-4 cited in our Viewpoint, and a fifth published afterward,5 indicate that differences between individual diets are likely to be trivial. Although individual studies sometimes show small to moderate differences between diets on various cardiometabolic risk factors, meta-analyses indicate either negligible or inconsistent effects on fasting glucose and lipid profiles.1-5 Such findings do not support the value of long-term studies (eg, 5-10 years of follow-up) comparing different macronutrientbased diets on clinical events and safety, particularly when behavioral adherence declines greatly during a year.6 The difference in efficacy between any 2 diets is likely explained by macronutrient composition, caloric restriction, and adherence. Unfortunately, research has focused almost exclusively on macronutrients and caloric restriction, which sends a message of false hope to the public that weight management is as simple as striking the perfect macronutrient balance. We are not suggesting total cessation of all diet comparison research, but rather that the perspective be broadened to acknowledge that diet is not merely a biochemical process but instead heavily relies on human behavior. This means prioritizing behavioral adherence as a topic deserving intensive study and as a primary intervention target in lifestyle interventions. In addition, we do not share the sentiment of Yancy and colleagues that scaling back on diet comparison studies will do little to stop the proliferation of fad diets or reduce public confusion. The scientific community has a responsibility to

JAMA December 25, 2013 Volume 310, Number 24

Copyright 2013 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by a University of Georgia User on 05/28/2015

jama.com

Spending and quality of care for Medicare beneficiaries in Massachusetts.

Spending and quality of care for Medicare beneficiaries in Massachusetts. - PDF Download Free
68KB Sizes 0 Downloads 0 Views