Letters
Figure 2. For-Profit, Nonprofit, and Publicly Owned US Opioid Treatment Programs Offering On-Site Human Immunodeficiency Virus (HIV) Testing From 2000 to 2011 Opt-out HIV testinga
US Opioid Treatment Programs Offering Testing, %
Public 80
Nonprofit 70
60
For-profit 2002
2004
2006
2008
2010
323 324 102
288 338 93
2012
Year
261 369 130
240 340 115
337 373 112
a
Refers to revised recommendations from the US Centers for Disease Control and Prevention in 2006 to screen all adults, adolescents, and pregnant women for HIV infection in all health care settings, unless the patient declines.4
to syphilis, which is often required by health departments. The survey is cross-sectional with no identifiers linking responses over time. Because patient-level data are not available, patient requests, use, and opting out of testing cannot be determined. Opioid treatment programs are important venues for offering testing to high-risk individuals. As the number of forprofit opioid treatment programs increases, and the opioid, HIV, and HCV epidemics continue to intersect, further work is needed to understand and reverse declines in offering onsite testing. Marcus A. Bachhuber, MD Chinazo O. Cunningham, MD, MS Author Affiliations: Division of General Internal Medicine, Albert Einstein College of Medicine, Bronx, New York. Corresponding Author: Chinazo O. Cunningham, MD, MS, Albert Einstein College of Medicine, 111 E 210th St, Bronx, NY 10467 ([email protected]). Author Contributions: Dr Bachhuber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bachhuber. Acquisition of data: Bachhuber. Analysis and interpretation of data: Bachhuber, Cunningham. Drafting of the manuscript: Bachhuber, Cunningham. Critical revision of the manuscript for important intellectual content: Bachhuber, Cunningham. Statistical analysis: Bachhuber. Obtained funding: Cunningham. Study supervision: Cunningham. Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Cunningham’s husband was recently employed by Pfizer Pharmaceuticals and is currently employed by Quest Diagnostics. No other disclosures were reported. 2672
2. Substance Abuse and Mental Health Services Administration. The N-SSATS Report: Trends in the Use of Methadone and Buprenorphine at Substance Abuse Treatment Facilities: 2003 to 2011. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2012. 3. Pollack HA, D’Aunno T. HIV testing and counseling in the nation’s outpatient substance abuse treatment system, 1995-2005. J Subst Abuse Treat. 2010;38(4):307-316.
50
No. of treatment programs offering HIV testing For-profit 209 Nonprofit 314 Public 104
Role of the Sponsor: The National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. 1. Office of Applied Studies, Substance Abuse and Mental Health Services Administration. The N-SSATS Report: infectious disease screening. http://www.oas.samhsa.gov/2k10/227/227DiseaseScreen2k10.htm. Accessibility verified November 15, 2013.
90
40 2000
Funding/Support: This study was supported by grants R34DA031066, R01DA032110, R25DA023021, and AI-51519 from the National Institutes of Health.
4. Branson BM, Handsfield HH, Lampe MA, et al; Centers for Disease Control and Prevention (CDC). Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Recomm Rep. 2006;55(RR-14):1-17. 5. Office of Applied Studies, Substance Abuse and Mental Health Services Administration. The DASIS report: the National Survey of Substance Abuse Treatment Services (N-SSATS). http://www.samhsa.gov/data/2k3/NSSATS /NSSATS.pdf. Accessibility verified November 15, 2013. 6. Metsch LR, Feaster DJ, Gooden L, et al. Implementing rapid HIV testing with or without risk-reduction counseling in drug treatment centers: results of a randomized trial. Am J Public Health. 2012;102(6):1160-1167.
COMMENT & RESPONSE
Intraoperative Cholangiography During Cholecystectomy To the Editor Dr Sheffield and colleagues1 found that on a patient level, intraoperative cholangiography was associated with reduced risk of common bile duct injury during cholecystectomy, presumably by laying out a roadmap for the surgeon. This is consistent with other appropriately powered studies of the topic. A secondary, instrumental variable analysis was performed to control for the unmeasured confounding in this study based on administrative data, but it is unclear whether this was appropriate. To justify this approach, the authors cited 3 studies using clinical data that have not shown a benefit with intraoperative cholangiography, but these studies were either underpowered or included minor common bile duct injuries that are expected when intraoperative cholangiography helps to avoid a major common bile duct injury. More recently, Törnqvist et al2 used only clinical data and found the same protective relationship of intraoperative cholangiography, including a survival benefit. The instrumental variable analysis was also problematic. Intraoperative cholangiography is used for different reasons by routine vs selective users, the former using it for a roadmap or safety-first approach and the latter using it to look for an injury once it is suspected. As the Editorial3 pointed out, hospitals where surgeons are routinely using intraoperative cholangiography may be hospitals where surgeons are doing other safety-first activities, including the use of trained assistants and better visualization technology that may be associated with lower risk of common bile duct injury.
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
jama.com
Letters
The instrumental variable analysis resulted in an attenuated protective association with widened confidence intervals, but Sheffield et al1 incorrectly interpreted this as no significant association. When looking at a rare event like common bile duct injury, the numbers of patients needed for a valid analysis are in the thousands to avoid a type 2 error. In the instrumental variable analysis, the sample size was too small to render valid conclusions. Lastly, decisions about intraoperative cholangiography use are not made at the hospital level but by surgeons for individual patients. Given the seriousness of common bile duct injury, and because no other intervention with the riskreducing effect of intraoperative cholangiography is available, the burden of proof for those trying to negate its benefit should be much higher. David Reed Flum, MD, MPH Author Affiliation: Department of Surgery, University of Washington School of Medicine, Seattle. Corresponding Author: David Reed Flum, MD, MPH, University of Washington, 1959 NE Pacific St, Seattle, WA 98195 ([email protected]). Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported being a consultant for and receiving grant support from the Patient-Centered Outcomes Research Institute; and providing expert testimony for Fain Anderson VanDerhoef. 1. Sheffield KM, Riall TS, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS. Association between cholecystectomy with vs without intraoperative cholangiography and risk of common duct injury. JAMA. 2013;310(8):812-820. 2. Törnqvist B, Strömberg C, Persson G, Nilsson M. Effect of intended intraoperative cholangiography and early detection of bile duct injury on survival after cholecystectomy: population based cohort study. BMJ. 2012;345:e6457. 3. Bilimoria KY, Chung J, Soper NJ. Laparoscopic cholecystectomy, intraoperative cholangiograms, and common duct injuries. JAMA. 2013;310(8):801-802.
To the Editor Dr Sheffield and colleagues1 reported an instrumental variable analysis of the association between cholecystectomy with vs without intraoperative cholangiography and the risk of common duct injury among elderly Texans. Sheffield et al1 concluded intraoperative cholangiography “is not effective as a preventive strategy against common duct injury during cholecystectomy.” We argue that these results favor a more nuanced conclusion. Concerns about threats to the exogeneity of the instrument are raised and expanded on in the Editorial,2 which also questioned the causal direction between intraoperative cholangiography and common duct injuries. However, our main concern is that these instrumental variable results apply to a much smaller marginal set of patients than “… patients who would receive intraoperative cholangiography in high-use hospitals but not in low-use hospitals….”1 Hence the questions that can be answered are far narrower. To clarify, the true marginal population excludes patients who will not receive the focal investigation or who will receive it regardless of who performs the procedure or where their treatment is performed.3 In those patients, the assumption of
quasi-random assignment via the instrumental variable as the randomizer is violated. That is, the marginal patients are ones for whom sufficient clinical uncertainty exists surrounding intraoperative cholangiography use. Moreover, the instrumental variable analysis will downweight those patients whose hospital or surgeon options are similar in terms of propensity to perform the investigation.4 For broader applicability, patients in a region should have similar chances of being treated in a hospital with a high investigation propensity or a hospital with low propensity, and analogously for patient choice of physician.5 Sufficient variation in propensity was found in aggregate in Texas. But, to the extent that local practices are correlated within small areas, the estimated treatment effects apply to an unknown and possibly very small marginal set of patients. In addition, the confidence interval for the instrumental variable estimate included all of the observational analyses’ point estimates. This raises doubt that the observational results are greatly biased. For this and the other reasons listed above, a more conservative conclusion would be that intraoperative cholangiography appears to be less effective than commonly believed in those marginal patients in whom the evidence base is ambiguous about intraoperative cholangiography, and in those who also reside in areas with large-scale local variation in intraoperative cholangiography use. Increasing intraoperative cholangiography rates among these clinically and geographically specific sets of patients may not be warranted as a strategy to reduce common duct injuries. Marco D. Huesch, MBBS, PhD John Romley, PhD Author Affiliations: University of Southern California Sol Price School of Public Policy, Los Angeles (Huesch); University of Southern California Schaeffer Center for Health Policy and Economics, Los Angeles (Romley). Corresponding Author: Marco D. Huesch, MBBS, PhD, Sol Price School of Public Policy, 3335 S Figueroa St, USC Gateway Unit A, Los Angeles, CA 90089 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Huesch reported receiving support from grants from the Agency for Healthcare Research and Quality and Lockheed Martin; serving as a consultant to the Institute of Medicine, Precision Health Economics, and Parkland Center for Clinical Innovation; receiving payment for manuscript preparation from Precision Health Economics and Parkland Center for Clinical Innovation; holding a provisional patent for a predictive clinical algorithm; and receiving payment for educational presentations from Lockheed Martin and the University of Southern California. Dr Romley reported serving as a consultant to Precision Health Economics; and receiving travel expenses from Bristol-Myers Squibb and AstraZeneca. 1. Sheffield KM, Riall TS, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS. Association between cholecystectomy with vs without intraoperative cholangiography and risk of common duct injury. JAMA. 2013;310(8):812-820. 2. Bilimoria KY, Chung J, Soper NJ. Laparoscopic cholecystectomy, intraoperative cholangiograms, and common duct injuries. JAMA. 2013;310(8):801-802. 3. Harris KM, Remler DK. Who is the marginal patient? understanding instrumental variables estimates of treatment effects. Health Serv Res. 1998;33(5 pt 1):1337-1360. 4. Brookhart MA, Schneeweiss S. Preference-based instrumental variable methods for the estimation of treatment effects: assessing validity and interpreting results. Int J Biostat. 2007;3(1):Article 14.
jama.com
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
2673
Letters
5. Huesch MD. External adjustment sensitivity analysis for unmeasured confounding: an application to coronary stent outcomes, Pennsylvania 2004-2008. Health Serv Res. 2013;48(3):1191-1214.
In Reply The purpose of our study was to demonstrate that previous population-based studies reporting a patient-level association between intraoperative cholangiography and reduced common duct injury were subject to unmeasured confounding.1-3 The causal direction is not clear when using administrative data because the intent of intraoperative cholangiography (to delineate anatomy or to detect injury) is unknown, and cholangiograms that are attempted—but unsuccessful—are not captured. The instrumental variable analysis eliminated this unmeasured confounding by predicting exposure to intraoperative cholangiography based on an instrument (percentage of hospital intraoperative cholangiography use) and examining its association with common duct injury, rather than actual receipt at the patient level, in which the reason for intraoperative cholangiography use (or lack of use) is unclear. Both letters question the exogeneity of the instrument. As Dr Flum points out, hospitals routinely using intraoperative cholangiography may also have implemented other safetyfirst measures. Even if this were true, it would not change our conclusions for 2 reasons. First, if routine intraoperative cholangiography use were related to other hospital quality measures, our instrumental variable analysis results would be biased in favor of intraoperative cholangiography. Second, if such hospitals had lower rates of common duct injury, which was not observed, the association may be due to any of these quality measures rather than intraoperative cholangiography use itself. Our instrumental variable analysis results apply to the marginal population (ie, patients whose receipt of intraoperative cholangiography depends on the instrument).4 Drs Huesch and Romley correctly state that there are patients who would or would not receive intraoperative cholangiography regardless of the hospital where they are treated. In these patients, the assumption of quasi randomization is violated. We have previously demonstrated significant variation in the use of intraoperative cholangiography, ranging from 2% to 90% of cholecystectomies across surgeons and 4% to 95% across hospitals. Nearly half of the variance in intraoperative cholangiography use was attributable to the surgeon or hospital to which the patient presented.5 Given this variation, a large proportion of patients in our sample would be expected to fall within the marginal population. Another valid concern is the broader applicability of our results if local practices are correlated in small areas such that patients do not have similar chances of being seen by surgeons or hospitals with high or low intraoperative cholangiography use. However, we observed significant variation even within small geographic regions. There were 42 hospitals in the greater Houston area. Despite the geographic proximity, intraoperative cholangiography rates ranged from 0% to 86.3% across hospitals. We agree that the study was potentially underpowered to observe a statistically significant association. Nevertheless, the 2674
point estimate for intraoperative cholangiography was significantly attenuated using instrumental variable analysis. Our study highlights the unmeasured confounding in previous studies and addresses the policy-level question of “Does routine hospital-level intraoperative cholangiography use prevent common duct injury?” Our study does not advocate for intraoperative cholangiography never to be used, and it does not address the benefit of intraoperative cholangiography at the individual patient level (ie, individual patients who may in fact benefit). However, the data do not support mandatory use of intraoperative cholangiography during cholecystectomy as the standard of care. Taylor S. Riall, MD, PhD Kristin M. Sheffield, PhD Yong-Fang Kuo, PhD Author Affiliations: Department of Surgery, University of Texas Medical Branch, Galveston. Corresponding Author: Taylor S. Riall, MD, PhD, University of Texas Medical Branch, 301 University Blvd, JSA 6.110c, Galveston, TX 77555 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Riall and Sheffield reported receiving grants from the National Institutes of Health. Dr Kuo reported a pending grant with the Agency for Healthcare Research and Quality. 1. Fletcher DR, Hobbs MS, Tan P, et al. Complications of cholecystectomy: risks of the laparoscopic approach and protective effects of operative cholangiography: a population-based study. Ann Surg. 1999;229(4):449-457. 2. Flum DR, Dellinger EP, Cheadle A, Chan L, Koepsell T. Intraoperative cholangiography and risk of common bile duct injury during cholecystectomy. JAMA. 2003;289(13):1639-1644. 3. Waage A, Nilsson M. Iatrogenic bile duct injury: a population-based study of 152 776 cholecystectomies in the Swedish Inpatient Registry. Arch Surg. 2006;141(12):1207-1213. 4. Harris KM, Remler DK. Who is the marginal patient? understanding instrumental variables estimates of treatment effects. Health Serv Res. 1998;33(5 pt 1):1337-1360. 5. Sheffield KM, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS, Riall TS. Variation in the use of intraoperative cholangiography during cholecystectomy. J Am Coll Surg. 2012;214(4):668-681.
Spending and Quality of Care for Medicare Beneficiaries in Massachusetts To the Editor The study by Dr McWilliams and colleagues1 on spending and quality of care for Medicare beneficiaries associated with the Alternative Quality Contract (AQC) in Massachusetts contains important threats to validity that raise questions about the reliability of the results. The most important is volunteer bias. This bias was identified as an issue with an earlier evaluation of the AQC,2 but the limitation is not mentioned in the article by McWilliams et al.1 Only 11 medical practices volunteered for participation and these practices had higher Medicare spending per beneficiary than the control practices, which raises the possibility that the practices chose to participate because they knew they could achieve savings and earn bonuses, possibly for activities that were already planned or under way. The practices that declined to participate, which became the control group, may have known that they could
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
jama.com
Figure 2. For-Profit, Nonprofit, and Publicly Owned US Opioid Treatment Programs Offering On-Site Human Immunodeficiency Virus (HIV) Testing From 2000 to 2011 Opt-out HIV testinga
US Opioid Treatment Programs Offering Testing, %
Public 80
Nonprofit 70
60
For-profit 2002
2004
2006
2008
2010
323 324 102
288 338 93
2012
Year
261 369 130
240 340 115
337 373 112
a
Refers to revised recommendations from the US Centers for Disease Control and Prevention in 2006 to screen all adults, adolescents, and pregnant women for HIV infection in all health care settings, unless the patient declines.4
to syphilis, which is often required by health departments. The survey is cross-sectional with no identifiers linking responses over time. Because patient-level data are not available, patient requests, use, and opting out of testing cannot be determined. Opioid treatment programs are important venues for offering testing to high-risk individuals. As the number of forprofit opioid treatment programs increases, and the opioid, HIV, and HCV epidemics continue to intersect, further work is needed to understand and reverse declines in offering onsite testing. Marcus A. Bachhuber, MD Chinazo O. Cunningham, MD, MS Author Affiliations: Division of General Internal Medicine, Albert Einstein College of Medicine, Bronx, New York. Corresponding Author: Chinazo O. Cunningham, MD, MS, Albert Einstein College of Medicine, 111 E 210th St, Bronx, NY 10467 ([email protected]). Author Contributions: Dr Bachhuber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bachhuber. Acquisition of data: Bachhuber. Analysis and interpretation of data: Bachhuber, Cunningham. Drafting of the manuscript: Bachhuber, Cunningham. Critical revision of the manuscript for important intellectual content: Bachhuber, Cunningham. Statistical analysis: Bachhuber. Obtained funding: Cunningham. Study supervision: Cunningham. Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Cunningham’s husband was recently employed by Pfizer Pharmaceuticals and is currently employed by Quest Diagnostics. No other disclosures were reported. 2672
2. Substance Abuse and Mental Health Services Administration. The N-SSATS Report: Trends in the Use of Methadone and Buprenorphine at Substance Abuse Treatment Facilities: 2003 to 2011. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2012. 3. Pollack HA, D’Aunno T. HIV testing and counseling in the nation’s outpatient substance abuse treatment system, 1995-2005. J Subst Abuse Treat. 2010;38(4):307-316.
50
No. of treatment programs offering HIV testing For-profit 209 Nonprofit 314 Public 104
Role of the Sponsor: The National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. 1. Office of Applied Studies, Substance Abuse and Mental Health Services Administration. The N-SSATS Report: infectious disease screening. http://www.oas.samhsa.gov/2k10/227/227DiseaseScreen2k10.htm. Accessibility verified November 15, 2013.
90
40 2000
Funding/Support: This study was supported by grants R34DA031066, R01DA032110, R25DA023021, and AI-51519 from the National Institutes of Health.
4. Branson BM, Handsfield HH, Lampe MA, et al; Centers for Disease Control and Prevention (CDC). Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Recomm Rep. 2006;55(RR-14):1-17. 5. Office of Applied Studies, Substance Abuse and Mental Health Services Administration. The DASIS report: the National Survey of Substance Abuse Treatment Services (N-SSATS). http://www.samhsa.gov/data/2k3/NSSATS /NSSATS.pdf. Accessibility verified November 15, 2013. 6. Metsch LR, Feaster DJ, Gooden L, et al. Implementing rapid HIV testing with or without risk-reduction counseling in drug treatment centers: results of a randomized trial. Am J Public Health. 2012;102(6):1160-1167.
COMMENT & RESPONSE
Intraoperative Cholangiography During Cholecystectomy To the Editor Dr Sheffield and colleagues1 found that on a patient level, intraoperative cholangiography was associated with reduced risk of common bile duct injury during cholecystectomy, presumably by laying out a roadmap for the surgeon. This is consistent with other appropriately powered studies of the topic. A secondary, instrumental variable analysis was performed to control for the unmeasured confounding in this study based on administrative data, but it is unclear whether this was appropriate. To justify this approach, the authors cited 3 studies using clinical data that have not shown a benefit with intraoperative cholangiography, but these studies were either underpowered or included minor common bile duct injuries that are expected when intraoperative cholangiography helps to avoid a major common bile duct injury. More recently, Törnqvist et al2 used only clinical data and found the same protective relationship of intraoperative cholangiography, including a survival benefit. The instrumental variable analysis was also problematic. Intraoperative cholangiography is used for different reasons by routine vs selective users, the former using it for a roadmap or safety-first approach and the latter using it to look for an injury once it is suspected. As the Editorial3 pointed out, hospitals where surgeons are routinely using intraoperative cholangiography may be hospitals where surgeons are doing other safety-first activities, including the use of trained assistants and better visualization technology that may be associated with lower risk of common bile duct injury.
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
jama.com
Letters
The instrumental variable analysis resulted in an attenuated protective association with widened confidence intervals, but Sheffield et al1 incorrectly interpreted this as no significant association. When looking at a rare event like common bile duct injury, the numbers of patients needed for a valid analysis are in the thousands to avoid a type 2 error. In the instrumental variable analysis, the sample size was too small to render valid conclusions. Lastly, decisions about intraoperative cholangiography use are not made at the hospital level but by surgeons for individual patients. Given the seriousness of common bile duct injury, and because no other intervention with the riskreducing effect of intraoperative cholangiography is available, the burden of proof for those trying to negate its benefit should be much higher. David Reed Flum, MD, MPH Author Affiliation: Department of Surgery, University of Washington School of Medicine, Seattle. Corresponding Author: David Reed Flum, MD, MPH, University of Washington, 1959 NE Pacific St, Seattle, WA 98195 ([email protected]). Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported being a consultant for and receiving grant support from the Patient-Centered Outcomes Research Institute; and providing expert testimony for Fain Anderson VanDerhoef. 1. Sheffield KM, Riall TS, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS. Association between cholecystectomy with vs without intraoperative cholangiography and risk of common duct injury. JAMA. 2013;310(8):812-820. 2. Törnqvist B, Strömberg C, Persson G, Nilsson M. Effect of intended intraoperative cholangiography and early detection of bile duct injury on survival after cholecystectomy: population based cohort study. BMJ. 2012;345:e6457. 3. Bilimoria KY, Chung J, Soper NJ. Laparoscopic cholecystectomy, intraoperative cholangiograms, and common duct injuries. JAMA. 2013;310(8):801-802.
To the Editor Dr Sheffield and colleagues1 reported an instrumental variable analysis of the association between cholecystectomy with vs without intraoperative cholangiography and the risk of common duct injury among elderly Texans. Sheffield et al1 concluded intraoperative cholangiography “is not effective as a preventive strategy against common duct injury during cholecystectomy.” We argue that these results favor a more nuanced conclusion. Concerns about threats to the exogeneity of the instrument are raised and expanded on in the Editorial,2 which also questioned the causal direction between intraoperative cholangiography and common duct injuries. However, our main concern is that these instrumental variable results apply to a much smaller marginal set of patients than “… patients who would receive intraoperative cholangiography in high-use hospitals but not in low-use hospitals….”1 Hence the questions that can be answered are far narrower. To clarify, the true marginal population excludes patients who will not receive the focal investigation or who will receive it regardless of who performs the procedure or where their treatment is performed.3 In those patients, the assumption of
quasi-random assignment via the instrumental variable as the randomizer is violated. That is, the marginal patients are ones for whom sufficient clinical uncertainty exists surrounding intraoperative cholangiography use. Moreover, the instrumental variable analysis will downweight those patients whose hospital or surgeon options are similar in terms of propensity to perform the investigation.4 For broader applicability, patients in a region should have similar chances of being treated in a hospital with a high investigation propensity or a hospital with low propensity, and analogously for patient choice of physician.5 Sufficient variation in propensity was found in aggregate in Texas. But, to the extent that local practices are correlated within small areas, the estimated treatment effects apply to an unknown and possibly very small marginal set of patients. In addition, the confidence interval for the instrumental variable estimate included all of the observational analyses’ point estimates. This raises doubt that the observational results are greatly biased. For this and the other reasons listed above, a more conservative conclusion would be that intraoperative cholangiography appears to be less effective than commonly believed in those marginal patients in whom the evidence base is ambiguous about intraoperative cholangiography, and in those who also reside in areas with large-scale local variation in intraoperative cholangiography use. Increasing intraoperative cholangiography rates among these clinically and geographically specific sets of patients may not be warranted as a strategy to reduce common duct injuries. Marco D. Huesch, MBBS, PhD John Romley, PhD Author Affiliations: University of Southern California Sol Price School of Public Policy, Los Angeles (Huesch); University of Southern California Schaeffer Center for Health Policy and Economics, Los Angeles (Romley). Corresponding Author: Marco D. Huesch, MBBS, PhD, Sol Price School of Public Policy, 3335 S Figueroa St, USC Gateway Unit A, Los Angeles, CA 90089 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Huesch reported receiving support from grants from the Agency for Healthcare Research and Quality and Lockheed Martin; serving as a consultant to the Institute of Medicine, Precision Health Economics, and Parkland Center for Clinical Innovation; receiving payment for manuscript preparation from Precision Health Economics and Parkland Center for Clinical Innovation; holding a provisional patent for a predictive clinical algorithm; and receiving payment for educational presentations from Lockheed Martin and the University of Southern California. Dr Romley reported serving as a consultant to Precision Health Economics; and receiving travel expenses from Bristol-Myers Squibb and AstraZeneca. 1. Sheffield KM, Riall TS, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS. Association between cholecystectomy with vs without intraoperative cholangiography and risk of common duct injury. JAMA. 2013;310(8):812-820. 2. Bilimoria KY, Chung J, Soper NJ. Laparoscopic cholecystectomy, intraoperative cholangiograms, and common duct injuries. JAMA. 2013;310(8):801-802. 3. Harris KM, Remler DK. Who is the marginal patient? understanding instrumental variables estimates of treatment effects. Health Serv Res. 1998;33(5 pt 1):1337-1360. 4. Brookhart MA, Schneeweiss S. Preference-based instrumental variable methods for the estimation of treatment effects: assessing validity and interpreting results. Int J Biostat. 2007;3(1):Article 14.
jama.com
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
2673
Letters
5. Huesch MD. External adjustment sensitivity analysis for unmeasured confounding: an application to coronary stent outcomes, Pennsylvania 2004-2008. Health Serv Res. 2013;48(3):1191-1214.
In Reply The purpose of our study was to demonstrate that previous population-based studies reporting a patient-level association between intraoperative cholangiography and reduced common duct injury were subject to unmeasured confounding.1-3 The causal direction is not clear when using administrative data because the intent of intraoperative cholangiography (to delineate anatomy or to detect injury) is unknown, and cholangiograms that are attempted—but unsuccessful—are not captured. The instrumental variable analysis eliminated this unmeasured confounding by predicting exposure to intraoperative cholangiography based on an instrument (percentage of hospital intraoperative cholangiography use) and examining its association with common duct injury, rather than actual receipt at the patient level, in which the reason for intraoperative cholangiography use (or lack of use) is unclear. Both letters question the exogeneity of the instrument. As Dr Flum points out, hospitals routinely using intraoperative cholangiography may also have implemented other safetyfirst measures. Even if this were true, it would not change our conclusions for 2 reasons. First, if routine intraoperative cholangiography use were related to other hospital quality measures, our instrumental variable analysis results would be biased in favor of intraoperative cholangiography. Second, if such hospitals had lower rates of common duct injury, which was not observed, the association may be due to any of these quality measures rather than intraoperative cholangiography use itself. Our instrumental variable analysis results apply to the marginal population (ie, patients whose receipt of intraoperative cholangiography depends on the instrument).4 Drs Huesch and Romley correctly state that there are patients who would or would not receive intraoperative cholangiography regardless of the hospital where they are treated. In these patients, the assumption of quasi randomization is violated. We have previously demonstrated significant variation in the use of intraoperative cholangiography, ranging from 2% to 90% of cholecystectomies across surgeons and 4% to 95% across hospitals. Nearly half of the variance in intraoperative cholangiography use was attributable to the surgeon or hospital to which the patient presented.5 Given this variation, a large proportion of patients in our sample would be expected to fall within the marginal population. Another valid concern is the broader applicability of our results if local practices are correlated in small areas such that patients do not have similar chances of being seen by surgeons or hospitals with high or low intraoperative cholangiography use. However, we observed significant variation even within small geographic regions. There were 42 hospitals in the greater Houston area. Despite the geographic proximity, intraoperative cholangiography rates ranged from 0% to 86.3% across hospitals. We agree that the study was potentially underpowered to observe a statistically significant association. Nevertheless, the 2674
point estimate for intraoperative cholangiography was significantly attenuated using instrumental variable analysis. Our study highlights the unmeasured confounding in previous studies and addresses the policy-level question of “Does routine hospital-level intraoperative cholangiography use prevent common duct injury?” Our study does not advocate for intraoperative cholangiography never to be used, and it does not address the benefit of intraoperative cholangiography at the individual patient level (ie, individual patients who may in fact benefit). However, the data do not support mandatory use of intraoperative cholangiography during cholecystectomy as the standard of care. Taylor S. Riall, MD, PhD Kristin M. Sheffield, PhD Yong-Fang Kuo, PhD Author Affiliations: Department of Surgery, University of Texas Medical Branch, Galveston. Corresponding Author: Taylor S. Riall, MD, PhD, University of Texas Medical Branch, 301 University Blvd, JSA 6.110c, Galveston, TX 77555 ([email protected]). Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Riall and Sheffield reported receiving grants from the National Institutes of Health. Dr Kuo reported a pending grant with the Agency for Healthcare Research and Quality. 1. Fletcher DR, Hobbs MS, Tan P, et al. Complications of cholecystectomy: risks of the laparoscopic approach and protective effects of operative cholangiography: a population-based study. Ann Surg. 1999;229(4):449-457. 2. Flum DR, Dellinger EP, Cheadle A, Chan L, Koepsell T. Intraoperative cholangiography and risk of common bile duct injury during cholecystectomy. JAMA. 2003;289(13):1639-1644. 3. Waage A, Nilsson M. Iatrogenic bile duct injury: a population-based study of 152 776 cholecystectomies in the Swedish Inpatient Registry. Arch Surg. 2006;141(12):1207-1213. 4. Harris KM, Remler DK. Who is the marginal patient? understanding instrumental variables estimates of treatment effects. Health Serv Res. 1998;33(5 pt 1):1337-1360. 5. Sheffield KM, Han Y, Kuo YF, Townsend CM Jr, Goodwin JS, Riall TS. Variation in the use of intraoperative cholangiography during cholecystectomy. J Am Coll Surg. 2012;214(4):668-681.
Spending and Quality of Care for Medicare Beneficiaries in Massachusetts To the Editor The study by Dr McWilliams and colleagues1 on spending and quality of care for Medicare beneficiaries associated with the Alternative Quality Contract (AQC) in Massachusetts contains important threats to validity that raise questions about the reliability of the results. The most important is volunteer bias. This bias was identified as an issue with an earlier evaluation of the AQC,2 but the limitation is not mentioned in the article by McWilliams et al.1 Only 11 medical practices volunteered for participation and these practices had higher Medicare spending per beneficiary than the control practices, which raises the possibility that the practices chose to participate because they knew they could achieve savings and earn bonuses, possibly for activities that were already planned or under way. The practices that declined to participate, which became the control group, may have known that they could
JAMA December 25, 2013 Volume 310, Number 24
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Eastern Kentucky University User on 05/25/2015
jama.com
