ORIGINAL ARTICLE
The Effect of Surgical Care Improvement Project (SCIP) Compliance on Surgical Site Infections (SSI) Guido Cataife, PhD,* Daniel A. Weinberg, PhD,* Hui-Hsing Wong, MD, JD,w and Katherine L. Kahn, MDz
Background: The Surgical Care Improvement Project (SCIP) has developed a set of process compliance measures in an attempt to reduce the incidence of surgical site infections (SSIs). Previous research has been inconclusive on whether compliance with these measures is associated with lower SSI rates. Objectives: To determine whether hospitals with higher levels of compliance with SCIP measures have lower incidence of SSIs and to identify the measures that are most likely to drive this association.
Key Words: surgical care improvement project (SCIP), surgical site infections, antibiotics (Med Care 2014;52: S66–S73)
BACKGROUND AND MOTIVATION
From the *IMPAQ International, LLC, Columbia, MD; wUS Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Washington, DC; and zRAND Corporation, Santa Monica, CA. The authors declare no conflict of interest. The findings and conclusions of this article are those of the authors and do not necessarily represent the views of the Office of the Assistant Secretary for Planning and Evaluation or the U.S. Department of Health and Human Services. Reprints: Guido Cataife, PhD, IMPAQ International, LLC, 10420 Little Patuxent Parkway, Suite 300, Columbia, MD 21044. E-mail: gcataife@ impaqint.com. Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Website, www.lww-medical care.com. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5202-0S66
The Surgical Care Improvement Project (SCIP) program was introduced in 2002 by the Centers for Medicare & Medicaid Services (CMS) in partnership with national organizations, including the American Hospital Association, Centers for Disease Control and Prevention (CDC), the Institute for Healthcare Improvement, and The Joint Commission. The goal of the program is to reduce the rates of postoperative surgical infections by promoting the adoption of publicly reported individual SCIP measures selected by a technical expert panel.1 The 3 core preventive measures that relate to healthcare–associated infections (HAIs) include: (1) the initiation of prophylactic antibiotics within 1 hour before surgical incision (or within 2 if the patient is receiving vancomycin or fluoroquinolones), (2) the use of prophylactic antibiotics appropriate for the specific procedure of the patient, and (3) the discontinuation of prophylactic antibiotics within 24 hours of surgery completion (within 48 h for cardiothoracic surgery).2,3 The frequently cited Compendium of Strategies to Prevent HAIs in Acute Care Hospitals4,5 cites grade A-1 evidence to support a recommendation6 to administer antimicrobial prophylaxis in accordance with the evidence-based standards and guidelines in association with surgical procedures for all 3 of these measures.7–9 In 2009, these measures were endorsed for use by the National Quality Forum.10 Three studies have provided evidence for the effectiveness of prophylactic antibiotic use.11–13 However, it is unclear whether these findings translate into actual benefits in routine clinical care for a national sample. It is also unknown whether hospital characteristics (eg, hospital type, size, and location) contribute to a substantial reduction of surgical site infections (SSIs) through a tighter enforcement of SCIP compliance. Previous studies that attempted to examine these issues have had mixed results. A study of 491 patients undergoing colorectal surgery found no significant decrease in SSIs in the period of high SCIP compliance rates compared with the period of low compliance rates.14 Another study15 found that, for a similar group of patients, the implementation of SCIP guidelines in a 500-bed tertiary
S66 | www.lww-medicalcare.com
Medical Care
Data and Methods: Analysis of linked SCIP compliance rates and SSIs on 295 hospital groups observed annually over the study period 2007–2010. A hospital group comprises all hospitals sharing identical categories for location by state, teaching status, bed size, and urban/rural location. We used a generalized linear model regression with logistic link and binomial family to estimate the association between 3 SCIP measures and SSI rates. Results: Hospital groups with higher compliance rates had significantly lower SSI rates for 2 SCIP measures: antibiotic timing and appropriate antibiotic selection. For a hospital group of median characteristics, a 10% improvement in the measure provision of antibiotic 1 hour before intervention led to a 5.3% decrease in the SSI rates (P < 0.05). Rural hospitals had effect sizes several times larger than urban hospitals (P < 0.05). A third-core measure, Timely Antibiotic Stop, showed no robust association. Conclusions: This analysis supports a clinically and statistically meaningful relationship between adherence to 2 SCIP measures and SSI rates, supporting the validity of the 2 publicly available healthcare–associated infection metrics.
Volume 52, Number 2 Suppl 1, February 2014
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
hospital was followed by a statistically significant reduction in SSI rates. In a retrospective cohort study of over 4 million patients from 398 hospitals, SCIP adherence measured through a global all-or-none composite infection prevention score was associated with a lower probability of developing a postoperative infection; however, none of the individual SCIP measures was significantly associated with a lower probability of infection.2 A systematic review of the literature concluded that “the overall success of SCIP has been decidedly mixed,” with results varying across SSI rates and research methodology.16 The purpose of this paper is 3 fold: to determine whether hospitals with higher levels of compliance to 3 SCIP core preventive measures have lower incidence of SSIs, to assess the magnitude of the effects, and to identify the group characteristics associated with the largest effects.
DATA AND METHODS We constructed an analytic dataset with SSI and SCIP compliance rates paired at the hospital group level. A hospital group is an analytic construct we defined based on the unique combination of the following 4 characteristics: (1) hospital teaching status (any/none), (2) state in which the hospital is located, (3) bed size (large/medium/small), and (4) location (urban if in metropolitan area by US Census Bureau, rural otherwise). For instance, all large teaching hospitals in an urban location within Massachusetts define a group; medium-size nonteaching hospitals in an urban location within Missouri define another group. We identified 14.5 unique hospitals, on an average, per group (median = 9, range 1–80), and 295 groups. Although hospital-level data would be preferable, such data are not available because of privacy concerns. The group construct provided the maximum possible disaggregation for the relevant study variables attainable with the available national data. Group-level SSI rates for the period January 1, 2005 through December 31, 2010 were obtained from the census of Medicare fee-for-service inpatient discharges, which draws from Medicare claims data. The specifications, based on the work of Stevenson et al,17 can be found in the appendix, Supplemental Digital Content 1, http://links. lww.com/MLR/A596. There is a follow-up period of 30 days if the procedure did not involve an implant, and 1 year if the procedure involved an implant.18 The follow-up is partial for the last year of analysis due to data lags; the year indicators and sensitivity analyses addressed this issue. SCIP rates for the same years were extracted from the publicly available Hospital Compare database19 and aggregated up from the hospital-level to the group-level according to hospital teaching status, state, bed size, and location (urban or rural). SCIP data are available for essentially all general, acute-care, short-stay hospitals, which are incentivized by CMS to report SCIP compliance rates and other quality information through the Inpatient Quality Reporting program. The SCIP rates at the group level were calculated as the average SCIP rates of the hospitals included in the group, using the sample sizes reported by hospitals for SCIP rates as weights. Our analytic dataset resulted from r
2014 Lippincott Williams & Wilkins
SCIP Compliance and Surgical Infections
merging these SSI and SCIP measures for calendar years 2005 through 2010. We studied the association between SSI rates and SCIP compliance rates using a generalized linear model (GLM) with logistic link and binomial family. Because this regression analysis technique was designed for dependent variables that are expressed as rates, it is preferable to others, including ordinary least squares (OLS). It does not require ad hoc adjustments for SSI rates taking a value of 0 and yields predicted values that are straightforward to generate and interpret. Unlike OLS, marginal effects are not assumed to be constant through the range of the independent variable (SCIP rate), making nonlinearities easier to capture.20 Our model includes controls for all characteristics defining the groups, namely each of the 49 participating states using indicators, and for teaching hospital status, urban status, and bed size category. We also included year indicators to capture time trends. Throughout the study, we used heteroskedasticity-robust standard errors clustered at the group level. We did not include hospital group fixed effects to avoid the incidental parameter problem.21 The strong correlation between SCIP measures—that is, hospitals that comply with at least one SCIP measure also tend to comply with the remainder—posed a methodological challenge for our study. Including highly collinear dependent variables would produce highly imprecise estimates. To address this concern, we ran a separate set of regressions for each SCIP measure. In a first set of regressions, the SCIP measure Timely Antibiotic (ATB) Start, assessing the proportion of patients who receive antibiotics within 1 hour before surgery, was the main independent variable. In a second set of regressions, the SCIP measure Appropriate ATB was the main independent variable. The 2 sets of regressions were identical in all other respects. Each set of regressions consisted of 3 different model specifications. Model 1 had the main dependent variable only. Model 2 had the main dependent variable and a set of controls including state indicators, year indicators, group size, teaching status, and urban status. Model 3 included the interaction between urban status and the SCIP measure (Timely ATB Start or Appropriate ATB) in addition to the controls. Hospitals in urban settings are different from hospitals in rural settings in many major respects, such as size, complexity of procedures, and the underlying health conditions of their populations. Although the covariates included in model 2 would capture differences in SSI rates between urban and rural hospitals, differences in the marginal effect of SCIP scores on SSI rates could only be studied by including this interaction term.
RESULTS Descriptive Analyses After excluding 30 observations whose SSI rates were based on fewer than 10 records, we obtained an analytic sample consisting of 1430 year-groups. The sample included a total of 295 groups observed repeatedly over the study period 2007–2010, including 288 groups (comprising 4105 hospitals) for year 2007, 285 groups (4136 hospitals) for year www.lww-medicalcare.com |
S67
Medical Care
Cataife et al
Volume 52, Number 2 Suppl 1, February 2014
TABLE 1. Hospital Group Characteristics Variable Names
Variable Description
Mean; Median; SD
Range
Timely ATB Start
SCIP score 1: proportion of patients who were given an antibiotic within 1 hour before surgery SCIP score 2: proportion of patients whose preventive antibiotics were stopped within 24 h after surgery SCIP score 3: proportion of patients who received the right kind of antibiotic to help prevent infection SSI rate Member Council Teaching Hospitals Hospitals residing in counties with a Metropolitan Statistical Area (MSA)type of metropolitan area Between 50 and 199 beds 200 beds or more
0.90; 0.92; 0.08
0.22–1.00
0.86; 0.89; 0.11
0.22–1.00
0.94;0.96; 0.06
0.29–1.00
0.08; 0.08; 0.03 0.18; 0; 0.39 0.62; 1; 0.48
0–0.37 0–1 0–1
0.32; 0; 0.47 0.41; 0; 0.49
0–1 0–1
Timely ATB Stop Appropriate ATB SSI rate Teaching Urban Medium Large
The number of unique group-year units is 1430. ATB is the abbreviation for antibiotic; SCIP, Surgical Care Improvement Project; SSI, surgical site infection.
2008, 287 groups (4194 hospitals) for year 2009, and 284 groups (4123 hospitals) for year 2010. Table 1 displays summary statistics and descriptions for all variables. Compliance rates across the 3 SCIP measures ranged between 86% and 100%. The proportion of patients who were given an antibiotic within 1 hour before surgery had a mean of 90% (median 92%, SD 8%, range 22%–100%). Similarly, the proportion of patients who were given the appropriate antibiotics had a mean of 94% (median 96%, SD 6%, range 29%–100%). The mean SSI rate across groups was 8% (median 8%, SD 3.0%, range 0%–100%). Our estimates (not reported in Table 1) showed that SCIP measure compliance rates in urban locations were consistently higher compared with rural locations. In urban areas, ATB Start, ATB Stop, and ATB Appropriate rates were 90.9% (median 92.9%, range 21.9%–100%), 87.4% (median 90.0%, range 28.6%– 100%), 95.1% (median 96.4%, range 29.0%–100%), respectively, whereas in rural settings these same rates were 88.1% (median 90.2%, range 33.0%–100%), 84.7% (median 88.2%, range 22%–100%), and 93.3% (median 95.5%, range 35%–100%). SSI rates were also higher in urban locations (mean 8.22%, median 8.26%, range 0%–23.8%) compared with rural locations (mean 6.7%, median 6.0%, range 0%–36.7%). We first report the findings for the 2 core measures for which we found a robust association with SSI rates. We postpone to the end of this section a brief discussion of the findings for the remaining core measure.
Core Measure 1: Use of Antibiotic 1 Hour Before Intervention Table 2 shows 2 sets of regressions with SSI rates as the dependent variable. We first discuss the results of the regression of SSI rates on the use of antibiotic 1 hour before intervention (columns 1–3). The regression in model 1, which included a SCIP linear effect only, shows that SSI rates decreased as SCIP scores increased. The regression in model 2 shows that the coefficient for SCIP score remained negative and statistically significant after controlling for state, size, year, teaching status, and urban status. The regression in model 3 shows that rural hospitals had a coefficient ( 1.060) of much larger magnitude compared with
S68 | www.lww-medicalcare.com
urban hospitals ( 1.060 + 1.020 = 0.040). The model with the interaction term fit the data better than the model without the interaction term. The point estimates in interacted model were more precise and the coefficient for urban status switched from positive to negative.
Effect of Improving Compliance for ATB Starts To measure the size of the effect that an improvement in Timely ATB Start produces on SSI rates, we selected 1 state (Illinois) and year (2008) (arbitrarily) and the median values of our sample for the remaining covariates: nonteaching hospital, urban location, medium size. Table 3 provides the marginal effects calculated based on 1% increases from the 10%, 25%, 50%, 75%, and 90% percentiles of Timely ATB Start. All the coefficients are statistically significant; the P-values are those of the marginal effects of their associated Timely ATB Start coefficients (reported in Table 2). The marginal effects for model 2 were approximately 30% larger than those for model 1. The effect sizes consistently increased with compliance rates for both models, reflecting the fact that a 1% increase represents a higher percentage point increase as the level of the variable increases. A 1% increase in the compliance SCIP measure rate Timely ATB Start led to a 0.028–0.032 percentage point decrease in SSI rates in model 1 (0.038–0.043 in model 2). For instance, the 0.031 percentage point decrease associated to the median Timely ATB Start value in model 1 implies that a 10% improvement in the compliance rate would reduce the SSI rate from 7.6% to 7.3% (SSI numbers not reported in Table 3). Note that this represents a considerable percentage reduction of 3.9% from the baseline rate 7.6%. The effect was even larger according to model 2, which would imply a decrease from 7.6% to 7.2% (5.3% reduction). Model 3 disaggregated marginal effects by urban/rural location. The magnitude of the effect for urban locations, although still negative, was very small (approximately one tenth of that of model 1). However, it was surprisingly large for rural locations. For instance, starting from the median Timely ATB Start rate (0.92), a 10% increase in SCIP compliance would lead to a 0.7 percentage point decrease in the SSI rate, from 8.0% to 7.3% (that amounts to a 8.7% reduction from the baseline rate 8.0%). Finally, we ran r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
SCIP Compliance and Surgical Infections
TABLE 2. Compliance With 2 SCIP Measures and SSI: GLM Regression With SSI Rate as the Dependent Variable Timely ATB Start Coefficient (SE) [95% CI] (1) Model 1 Score effect
Appropriate ATB Coefficient (SE) [95% CI]
(2) Model 2
Score effect urban
0.478*** (0.161) [ 0.794, 0.163] Not included
0.552* (0.217) [ 0.977, 0.127] Not included
State dummies Year 2007 dummy
Not included Not included
Year 2008 dummy
Not included
Year 2009 dummy
Not included
Year 2010 dummy
Not included
Teaching dummy
Not included
Urban dummy
Not included
Medium size
Not included
Large size
Not included
Constant
2.060 (0.145)*** [ 2.344,1.776]
6w (out of 49) 0.023 (0.020) [ 0.016,0.064] 0.010 (0.026) [ 0.040,0.060] 0.052 (0.031) [ 0.010,0.114] 0.129*** (0.033) [ 0.194, 0.064] 0.220*** (0.034) [0.154,0.287] 0.095* (0.039) [0.020,0.171] 0.120* (0.048) [0.024,0.215] 0.196*** (0.043) [0.112,0.281] 2.039*** (0.252) [ 2.631, 1.446]
(3)
(4)
(5)
Model 3
Model 1
1.060*** (0.278) [ 1.604, 0.515] 1.020** (0.324) [0.384,1.655] 7w (out of 49) 0.026 (0.021) [ 0.015,0.066] 0.006 (0.028) [ 0.048,0.061] 0.039 (0.035) [ 0.030,0.108] 0.145*** (0.037) [ 0.216, 0.073] 0.218*** (0.033) [0.153,0.283] 0.811* (0.291) [ 1.381, 0.241] 0.124* (0.048) [0.030,0.218] 0.200*** (0.042) [0.117,0.283] 1.602*** (0.348) [ 2.285, 0.918]
1.023*** (0.226) [ 1.465, 0.580] Not included
0.625** (0.253) [ 1.121,0.130] Not included
Not included Not included
7w (out of 49) 0.005 (0.022) [ 0.039,0.048] 0.012 (0.026) [ 0.063,0.040] 0.018 (0.027) [ 0.034,0.070] 0.173*** (0.028) [ 0.229,0.117] 0.225*** (0.033) [0.159,0.290] 0.091 (0.039)* [0.015,0.168] 0.113 (0.047)* [0.021,0.206] 0.186 (0.040)*** [0.107,0.265] 1.890 (0.354)*** [ 2.582,1.196]
Not included Not included Not included Not included Not included Not included Not included 1.526*** (0.214) [ 1.945, 1.108]
n = 1429
(6)
Model 2
Model 3 1.127** (0.401) [ 1.914,0.340] 0.955* (0.434) [0.105,1.804] 8w (out of 49) 0.005 (0.022) [ 0.037,0.048] 0.017 (0.028) [ 0.072,0.037] 0.010 (0.029) [ 0.047,0.068] 0.180*** (0.031) [ 0.240,0.119] 0.226*** (0.033) [0.161,0.292] 0.801 (0.412) [ 1.609,0.006] 0.113* (0.047) [0.021,0.205] 0.181*** (0.040) [0.102,0.261] 1.429*** (0.479) [ 2.368,0.490]
n = 1425
GLM regression with logistic link and binomial family. Reference categories: score range 97.6–100, year 2006, nonteaching hospital, rural location, small size. SE are heteroskedasticity-robust and clustered at the group level. *Significant at 5% level. **Significant at 1% level. ***Significant at 0.1% level. w Number of statistical significant state dummies at 5% or lower level. ATB indicates antibiotic; CI, confidence interval; GLM, generalized linear model; SCIP, Surgical Care Improvement Project; SSI, surgical site infection.
similar effect size calculations assuming a flat 1 percentage point increase from baseline SCIP rate (instead of a 1% increase). This gave the effect of complying with an additional 1% of patients at different levels of compliance. These results (not reported) indicated that size effects consistently decreased with the level of SCIP compliance. For Timely ATB Start, the effect size at the 75% SCIP percentile was 7% and 9% lower compared with the 25% SCIP percentile for
models 1 and 2, respectively. For Appropriate ATB, the equivalent numbers were 8% and 5%.
Sensitivity Analysis We first estimated a regression using all control covariates but, instead of linear SCIP scores effects, we used dummies for 20 SCIP ranges of identical density. We found (results not shown) that 8 of the 19 SCIP ranges showed
TABLE 3. Magnitude of Effects Associated to GLM Regression—ATB Start Timely ATB Start Starting Value (Percentile), Final Value 0.790 0.858 0.920 0.960 0.981
(10%), (25%), (50%), (75%), (90%),
0.797 0.866 0.929 0.969 0.991
Marginal Effect (Percentage Points) Change (%)
Model 1
Model 2
Model 3 (Urban)
Model 3 (Rural)
1 1 1 1 1
0.028 0.029 0.031 0.032 0.032
0.038 0.040 0.041 0.042 0.043
0.003 0.003 0.003 0.003 0.003
0.066 0.068 0.069 0.069 0.069
Marginal effects computed as the difference in surgical site infection rates predicted based on the coefficients provided in Table 2 (statistical significance provided in Table 2). All predictions assume the state of Illinois, the year 2008, and nonteaching hospitals of medium size. Models 1–2 assume urban setting. For Model 3 we provide urban and rural estimates in separate columns.
r
2014 Lippincott Williams & Wilkins
www.lww-medicalcare.com |
S69
Medical Care
Cataife et al
statistical significance at the 5% level or lower, and the coefficients showed a downward (although nonmonotonic) trend, suggesting that as SCIP measures increased, SSI rates decreased. Next, we ran a similar exercise using 30 range dummies instead of 20. We found that 20 of the 29 dummies were statistically significant at the 5% level or lower. Again, there was a clear but nonmonotonic downward trend. Second, OLS is known to produce consistent estimates (and effect sizes) even in the presence of substantial measurement error in the dependent variable. The consequence of inaccuracies in claims data-based SSI rates on OLS estimates is wider confidence levels. We addressed concerns related to the quality of claims-based SSI rates reestimating model 2 using OLS. Using all covariates (same as model 2), we obtained a negative and statistically significant coefficient at the 5% level for Timely ATB Start. The OLS estimate had a coefficient of 0.038, implying that a 1 percentage point increase in compliance leads to an approximately 0.04 percentage point reduction in the SSI rate. This implies that a 0.92% increase such as that assumed in Table 3 for the 50% Timely ATB Start percentile (from 0.920 to 0.929) would lead to a 0.037 percentage point decrease in SSI— remarkably similar to the 0.041 GLM size effect. Finally, we reestimated all models dropping all 2010 data (that had limited follow-up); the results (see the online appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/ A596) were substantially the same.
Core Measure 2: Use of Appropriate Antibiotics We next turn to the results of the regressions of SSI rates on the SCIP measure related to the use of appropriate antibiotics. The results were very similar to those for timely use of antibiotics.
Effect of Improving Compliance for Appropriate Use of ATBs With minor exceptions, the results for this measure were similar to those for timely ATB (the sign and statistical significance of the variables in columns 4, 5, and 6 (Table 2), are similar to those in columns 1, 2, and 3, respectively). However, the effect sizes (Table 4) were substantially higher. The marginal effect at the median (50th percentile) for model 3 assuming a rural location is 0.078 percentage points in Table 4 versus 0.068 percentage points in Table 3.This implies that a 10% increase in SCIP compliance at the me-
Volume 52, Number 2 Suppl 1, February 2014
dian value of Appropriate ATB for a nonteaching, rurally located, medium-sized hospital would reduce the SSI rate from 8.2% to 7.4%.
Sensitivity Analysis Finally, we conducted a similar sensitivity analysis. Using dummies for SCIP ranges instead of linear scores, we found similar results (not shown). The number of statistically significant range dummies was 7 of 19 with 20 SCIP ranges and 16 of 29 with 30 SCIP ranges. Using OLS we observed a coefficient 0.046 (P < 0.05), implying that a 1% increase in SCIP compliance leads to an approximate 0.05 reduction in the SSI rate. Estimates excluding all 2010 data showed no substantial changes (see online appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/A596).
Core Measure 3: Timely ATB Stop We conducted similar analyses for the SCIP measure Timely ATB Stop (Table 5). We found a statistically significant and negative effect on SSI rates in model 1 only. However, the correlation disappeared when controls were introduced, suggesting that the initial correlation was driven by confounders.
DISCUSSION Three of the 9 publicly reported SCIP measures related to HAIs are considered core.2,3 We showed that 2 of the latter, Timely ATB Start and Appropriate ATB, have a high level of association with SSI rates. A third core measure, Timely ATB Stop, showed no robust association. SCIP measures are publicly reported measures of quality that may inform users’ decisions about where to get healthcare. Also, hospitals invest significant resources in complying with these measures and collecting and reporting their performance. Hence, it is critical to understand whether these measures adequately correlate to the underlying quality of surgical care. Our study provides robust evidence of this correlation for a subset of SCIP measures related to HAIs. Our results suggest that a tighter enforcement of SCIP measures in hospitals that are not complying consistently3 can lead to a substantial improvement in SSI rates and hence largely reduce the current societal burden22 of the estimated 750,000–1,000,000 annual SSIs,16 which includes an economic annual excess cost of $1.6 billion.23
TABLE 4. Magnitude of Effects Associated to GLSM Regression—Appropriate ATB Appropriate ATB Starting Value (Percentile), Final Value 0.888 0.930 0.960 0.977 0.985
(10%), (25%), (50%), (75%), (90%),
0.897 0.939 0.970 0.987 0.995
Marginal Effect (in Percentage Points) Change (%)
Model 1
Model 2
Model 3 (Urban)
Model 3 (Rural)
1 1 1 1 1
0.067 0.068 0.068 0.068 0.068
0.048 0.049 0.050 0.050 0.050
0.013 0.013 0.014 0.014 0.014
0.077 0.078 0.078 0.078 0.078
Marginal effects computed as the difference in surgical site infection rates predicted based on the coefficients provided in Table 2 (statistical significance provided in Table 2). All predictions assume the state of Illinois, the year 2008, and nonteaching hospitals of medium size. Models 1–2 assume urban setting. For Model 3 we provide urban and rural estimates in separate columns.
S70 | www.lww-medicalcare.com
r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
SCIP Compliance and Surgical Infections
TABLE 5. Compliance With Timely ATB Stop and SSI: GLM Regression With SSI as Dependent Variable Timely ATB Stop Coefficient (SE) [95% CI] (1) Model 1 Score effect
(2) Model 2
Score effect urban
0.467** (0.147) [ 0.755, 0.180] Not included
0.123 (0.177) [ 0.470,0.224] Not included
State dummies Year 2007 dummy
Not included Not included
Year 2008 dummy
Not included
Year 2009 dummy
Not included
Year 2010 dummy
Not included
Teaching dummy
Not included
Urban dummy
Not included
Medium size
Not included
Large size
Not included
Constant
2.086 (0.145)*** [ 2.340, 1.833]
7w (out of 49) 0.010 (0.024) [ 0.038,0.058] 0.018 (0.036) [ 0.088,0.053] 0.005 (0.041) [ 0.075.0.085] 0.184*** (0.046) [ 0.274, 0.094] 0.226*** (0.034) [0.160,0.293] 0.084* (0.039) [0.007,0.161] 0.098* (0.048) [0.004,0.192] 0.173*** (0.041) [0.092,0.253] 2.343*** (0.266) [ 2.864, 1.822] n = 1429
(3) Model 3 0.182 (0.229) [ 0.630,0.267] 0.126 (0.234) [ 0.332,0.584] 6w (out of 49) 0.008 (0.025) [ 0.040,0.057] 0.021 (0.037) [ 0.094,0.053] 0.001 (0.043) [ 0.083,0.085] 0.189*** (0.048) [ 0.283,0.095] 0.218*** (0.033) [0.160,0.292] 0.023 (0.205) [ 0.425,0.378] 0.099* (0.048) [0.005,0.192] 0.172*** (0.041) [0.092,0.253] 2.292*** (0.308) [ 2.896,1.687]
GLM regression with logistic link and binomial family. Reference categories: score range 97.6–100, year 2006, nonteaching hospital, rural location, small size. SE are heteroskedasticity-robust and clustered at the group level. ATB is the abbreviation for antibiotic; CI, confidence interval; GLM, generalized linear model; SSI, surgical site infection. *Significant at 5% level. **Significant at 1% level. ***Significant at 0.1% level. w Number of statistical significant state dummies at 5% or lower level.
For the SCIP measures Timely ATB Start and Appropriate ATB, the association is statistically significant and robust across regression models (GLM vs. OLS), across models that differ in terms of covariates included, and across models that vary with respect to the construction of the SCIP adherence score. The effect sizes are of considerable magnitude even in the lowest end of the range of estimates across models. On the basis of the available evidence from this and other studies,11–13 we believe that these findings support the ongoing use of Timely ATB Start and Appropriate ATB Use as performance measures that can provide valuable feedback to healthcare practices and stimulate quality improvement interventions. This analysis does not provide evidence to support the use of the Timely ATB Stop measure. The latter might be explained by the fact that the benefits of antibiotic discontinuation are dispersed over the general population and over time rather than concentrating on the specific patient whose antibiotic is discontinued at the time of surgery. This is because antibiotic discontinuation operates by minimizing the antibiotic resistance (rather than minimizing the chances of infection on the specific patient at the time of surgery). This dispersed effect would be difficult to capture in the regressions. r
2014 Lippincott Williams & Wilkins
The fact that urban hospitals had both higher SCIP and SSI rates compared with rural hospitals seems counterintuitive. However, this fact is explained by the results of model 3 (for both Timely ATB Start and Appropriate ATB measures), which showed that holding all other observed factors constant, rural hospitals had higher average SSI rates than urban hospitals. Model 3 showed that when the SCIP scores of rural hospitals improved, their SSI rates dropped more pronouncedly compared with urban hospitals. This is likely due to the finding of existence of decreasing size effects because, as already discussed, rural hospitals have lower average SCIP levels than urban hospitals. Previous studies found that SSI based on claims data are either subject to low sensitivity or poor positive predictive values.24–27 A plausible concern is that this would bias our estimates. Because OLS has been proven to give consistent estimates in the presence of measurement error,28 we addressed this concern by confirming that the results also hold when we use OLS. The fact that OLS results were very similar to GLM results is strong evidence that our results are not contaminated by measurement error. Finally, any measurement error in the SCIP measures would lead to attenuation bias, making our estimates of effect sizes conservative. www.lww-medicalcare.com |
S71
Medical Care
Cataife et al
Three major limitations are that we proved association but not causality, we used grouped data, and we did not control for patient level covariates such as demographics or comorbidities. Despite the use of a robust study design, this analysis was observational and could be subject to unknown confounders and secular trends. Major strengths are the use of a nationally representative sample, inclusion of a broad set of surgeries and a posthospital follow-up period, and the robustness of the results.
CONCLUSIONS We provided clear evidence that 2 SCIP compliance measures (timely use of antibiotic and appropriate use of antibiotics) were associated with lower SSI rates and the effect sizes are large. A third measure (timely stop of antibiotic) was not associated with lower SSI rates. This might be because its effects operate through antibiotic resistance and hence are dispersed over time and over a larger population. The combination of the magnitude of the annual levels of SSIs and the annual cost associated with the considerable effect sizes for the first 2 measures suggests the potential for large societal savings if tighter enforcement of these 2 SCIP measures is applied. The size of the effect is substantially larger in rural locations, suggesting that the enforcement of SCIP compliance in rural hospitals should be a policy priority. REFERENCES 1. CMS. Hospital inpatient quality reporting and hospital outpatient quality reporting surgical care improvement project technical expert panel. Available at: http://www.ofmq.com/Websites/ofmq/images/Measures/ IP_TEP_Lists/September_2012/SCIP_TEP_List_9.pdf. Accessed April 25, 2013. 2. Stulberg JJ, Delaney CP, Neuhauser DV, et al. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303:2479–2485. 3. Salkind AR, Kavitha CR. Antibiotic prophylaxis to prevent surgical site infections. Am Fam Physician. 2011;83:585–590. 4. Yokoe DS, Mermel LA, Anderson DJ, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29:S12–S21. 5. Andersen DJ, Kaye KS, Classen D, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29:S51–S61. 6. Canadian Task Force on the periodic health examination. The periodic health examination. Can Med Assoc J. 1979;121:1193–1254. 7. Mangram AJ, Horan TC, Pearson ML, et al. Guideline for prevention of surgical site infection, 1999. Hospital Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol. 1999;20:250–278. quiz 279-280. 8. Bratzler DW, Houck PM. Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project. Clin Infect Dis. 2004;38:1706–1715. 9. Bratzler DW, Hunt DR. The surgical infection prevention and surgical care improvement projects: national initiatives to improve outcomes for patients having surgery. Clin Infect Dis. 2006;43:322–330. 10. Quality Forum. QualityForum.org. May 2013. Available at: http:// www.qualityforum.org. Accessed May 25, 2013. 11. Bratzler DW, Houck PM, Richards C, et al. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical infection Prevention Project. Archives of Surgery. 2005; 140:174–182. 12. Classen DC, Evans RS, Pestotnik SL, et al. The timing of prophylactic administration of antibiotics and the risk of surgical-wound infection. NEJM. 1992;326:281–286.
S72 | www.lww-medicalcare.com
Volume 52, Number 2 Suppl 1, February 2014
13. Steinberg JP, Braun BI, Hellinger WC, et al. Timing of antimicrobial prophylaxis and the risk of surgical site infections: results from the trial to reduce antimicrobial prophylaxis errors. Annals of Surgery. 2009;250:10–16. 14. Pastor C, Artinyan A, Varma MG, et al. An Increase in compliance with the Surgical Care Improvement Project measures does not prevent surgical site infection in colorectal surgery. Dis Colon Rectum. 2010;53:24–20. 15. Barenguer CM, Gage Ochsner M, Lord A, et al. Improving surgical site infections: using National Surgical Quality Improvement Program Data to Institute Surgical Care Improvement Project Protocols in Improving Surgical Outcomes. J Am Coll Surg. 2010;5:737–741. 16. Edmiston CE, Spencer M, Lewis BD, et al. Reducing the risk of surgical site infections: did we really think SCIP was going to lead us to the promised land? Surg Infect. 2011;12:169–177. 17. Stevenson KB, Khan Y, Dickman J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health care– associated infections. Am J Infect Control. 2008;36:155–164. 18. CDC/NSHN. Surgical Site Infection Event. Available at: http:// www.cdc.gov/nhsn/PDFs/pscManual/9pscSSIcurrent.pdf?agree = yes& next = Accept. Accessed May 25 2013. 19. CMS. Hospital compare. Available at: http://www.medicare.gov/hospital compare. Accessed September 5, 2013. 20. Papke LE, Wooldridge JF. Econometric methods for fractional response variables with an application to 401(K) plan participation rates. J Appl Econometrics. 1996;11:619–632. 21. Lancaster T. The incidental parameter problem since 1948. J Econometrics. 2000;95:391–413. 22. Anderson DJ, Kaye KS, Classen D, et al. Strategies to prevent surgical site infections in acute care hospitals. Infect Control Hosp Epidemiol. 2013;29:S51–S61. 23. Lissovoy G, Fraeman K, Hutchins V, et al. Surgical site infection: incidence and impact on hospital and treatment costs. Infect Control Hosp Epidemiol. 2009;37:387–397. 24. Best WR, Khuri SF, Phelan M, et al. Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. J Am Coll Surg. 2002; 194:257–266. 25. Romano PS, Chan BK, Schembri ME, et al. Can administrative data be used to compare postoperative complication rates across hospitals? Med Care. 2010;40:856–867. 26. Cima RR, Cassivi SD, VanSuch M. How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACSNSQIP) Postoperative Adverse Events at a Single Institution. Surgery. 2011;150:943–949. 27. Lawson EH, Louie R, Zingmond DS, et al. A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications. Ann Surg. 2012;256:973–981. 28. Cameron AC, Trivedy PK. Microeconometrics. Methods and Applications. Ner York: Cambridge University Press; 2005. 29. Stevenson KB, Khan Y, Dickman J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health care– associated infections. Am J Infect Control. 2008;36:155–164. 30. CDC/NSHN. Available at: http://www.cdc.gov/nhsn/PDFs/pscManual/ 9pscSSIcurrent.pdf?agree = yes&next = Accept. Accessed May 25, 2013. 31. Centers for Disease Control and Prevention (CDC). 2012. NHSN Members Meeting at APIC—San Antonio. Available at: http:// www.cdc.gov/nhsn/PDFs/MemberMeetings/NHSN-Members-MeetingAPIC-2012.pdf. Accessed August 29, 2012.
APPENDIX The Effect of Surgical Care Improvement Project (SCIP) Compliance on SSIs The specifications for SSI rates are based on the work of Stevenson et al.29 We amended the specifications by r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
“grouping” the denominator conditions so that inpatient stays with more than 1 denominator code for a particular type of infection (eg, 2 craniotomy codes) are counted only once. We added a limited number of numerator infection codes. The SSI numerator includes the following infections: meningitis, central nervous system abscess, pericarditis, endocarditis, mediastinits, cellulitis of various sites, myositis, osteomyelitis, periostitis, bone infection, open wounds, device reactions, and postoperative infections. There is a follow-up period of 30 days if the procedure did not involve an implant, and 1-year if the procedure involved an implant.30 The follow-up periods capture any inpatient stays and are consistent with NHSN’s surveillance methodology until that system’s recent change. As of 2013, the 1-year followup period was replaced with a 90-day follow-up period for some procedures, and the use of a surgical implant will no longer be recorded.31 The follow-up is partial for the last year of analysis, given that after discharge data are
r
2014 Lippincott Williams & Wilkins
SCIP Compliance and Surgical Infections
updated with considerable lag. This issue is addressed in the analysis through the inclusion of year dummies. The SSI denominator represents discharges associated with any of the following surgeries: coronary artery bypass graft, peripheral vascular, colorectal, craniotomy, head and neck, hysterectomy, spinal, ventricular shunt, and total knee and hip replacement. The categories small (fewer than 50 beds), medium (50–200), and large (200+) were used in the SCIP data file to approximate the bed size variable in the SSI rate. Urban was defined as located in a metropolitan area by the US Census Bureau. Acumen LLC performed the programming and calculations of the SSI rates using the Medicare claims data under contract with the Centers for Medicare and Medicaid. This analysis was funded by the Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
www.lww-medicalcare.com |
S73
The Effect of Surgical Care Improvement Project (SCIP) Compliance on Surgical Site Infections (SSI) Guido Cataife, PhD,* Daniel A. Weinberg, PhD,* Hui-Hsing Wong, MD, JD,w and Katherine L. Kahn, MDz
Background: The Surgical Care Improvement Project (SCIP) has developed a set of process compliance measures in an attempt to reduce the incidence of surgical site infections (SSIs). Previous research has been inconclusive on whether compliance with these measures is associated with lower SSI rates. Objectives: To determine whether hospitals with higher levels of compliance with SCIP measures have lower incidence of SSIs and to identify the measures that are most likely to drive this association.
Key Words: surgical care improvement project (SCIP), surgical site infections, antibiotics (Med Care 2014;52: S66–S73)
BACKGROUND AND MOTIVATION
From the *IMPAQ International, LLC, Columbia, MD; wUS Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Washington, DC; and zRAND Corporation, Santa Monica, CA. The authors declare no conflict of interest. The findings and conclusions of this article are those of the authors and do not necessarily represent the views of the Office of the Assistant Secretary for Planning and Evaluation or the U.S. Department of Health and Human Services. Reprints: Guido Cataife, PhD, IMPAQ International, LLC, 10420 Little Patuxent Parkway, Suite 300, Columbia, MD 21044. E-mail: gcataife@ impaqint.com. Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Website, www.lww-medical care.com. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5202-0S66
The Surgical Care Improvement Project (SCIP) program was introduced in 2002 by the Centers for Medicare & Medicaid Services (CMS) in partnership with national organizations, including the American Hospital Association, Centers for Disease Control and Prevention (CDC), the Institute for Healthcare Improvement, and The Joint Commission. The goal of the program is to reduce the rates of postoperative surgical infections by promoting the adoption of publicly reported individual SCIP measures selected by a technical expert panel.1 The 3 core preventive measures that relate to healthcare–associated infections (HAIs) include: (1) the initiation of prophylactic antibiotics within 1 hour before surgical incision (or within 2 if the patient is receiving vancomycin or fluoroquinolones), (2) the use of prophylactic antibiotics appropriate for the specific procedure of the patient, and (3) the discontinuation of prophylactic antibiotics within 24 hours of surgery completion (within 48 h for cardiothoracic surgery).2,3 The frequently cited Compendium of Strategies to Prevent HAIs in Acute Care Hospitals4,5 cites grade A-1 evidence to support a recommendation6 to administer antimicrobial prophylaxis in accordance with the evidence-based standards and guidelines in association with surgical procedures for all 3 of these measures.7–9 In 2009, these measures were endorsed for use by the National Quality Forum.10 Three studies have provided evidence for the effectiveness of prophylactic antibiotic use.11–13 However, it is unclear whether these findings translate into actual benefits in routine clinical care for a national sample. It is also unknown whether hospital characteristics (eg, hospital type, size, and location) contribute to a substantial reduction of surgical site infections (SSIs) through a tighter enforcement of SCIP compliance. Previous studies that attempted to examine these issues have had mixed results. A study of 491 patients undergoing colorectal surgery found no significant decrease in SSIs in the period of high SCIP compliance rates compared with the period of low compliance rates.14 Another study15 found that, for a similar group of patients, the implementation of SCIP guidelines in a 500-bed tertiary
S66 | www.lww-medicalcare.com
Medical Care
Data and Methods: Analysis of linked SCIP compliance rates and SSIs on 295 hospital groups observed annually over the study period 2007–2010. A hospital group comprises all hospitals sharing identical categories for location by state, teaching status, bed size, and urban/rural location. We used a generalized linear model regression with logistic link and binomial family to estimate the association between 3 SCIP measures and SSI rates. Results: Hospital groups with higher compliance rates had significantly lower SSI rates for 2 SCIP measures: antibiotic timing and appropriate antibiotic selection. For a hospital group of median characteristics, a 10% improvement in the measure provision of antibiotic 1 hour before intervention led to a 5.3% decrease in the SSI rates (P < 0.05). Rural hospitals had effect sizes several times larger than urban hospitals (P < 0.05). A third-core measure, Timely Antibiotic Stop, showed no robust association. Conclusions: This analysis supports a clinically and statistically meaningful relationship between adherence to 2 SCIP measures and SSI rates, supporting the validity of the 2 publicly available healthcare–associated infection metrics.
Volume 52, Number 2 Suppl 1, February 2014
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
hospital was followed by a statistically significant reduction in SSI rates. In a retrospective cohort study of over 4 million patients from 398 hospitals, SCIP adherence measured through a global all-or-none composite infection prevention score was associated with a lower probability of developing a postoperative infection; however, none of the individual SCIP measures was significantly associated with a lower probability of infection.2 A systematic review of the literature concluded that “the overall success of SCIP has been decidedly mixed,” with results varying across SSI rates and research methodology.16 The purpose of this paper is 3 fold: to determine whether hospitals with higher levels of compliance to 3 SCIP core preventive measures have lower incidence of SSIs, to assess the magnitude of the effects, and to identify the group characteristics associated with the largest effects.
DATA AND METHODS We constructed an analytic dataset with SSI and SCIP compliance rates paired at the hospital group level. A hospital group is an analytic construct we defined based on the unique combination of the following 4 characteristics: (1) hospital teaching status (any/none), (2) state in which the hospital is located, (3) bed size (large/medium/small), and (4) location (urban if in metropolitan area by US Census Bureau, rural otherwise). For instance, all large teaching hospitals in an urban location within Massachusetts define a group; medium-size nonteaching hospitals in an urban location within Missouri define another group. We identified 14.5 unique hospitals, on an average, per group (median = 9, range 1–80), and 295 groups. Although hospital-level data would be preferable, such data are not available because of privacy concerns. The group construct provided the maximum possible disaggregation for the relevant study variables attainable with the available national data. Group-level SSI rates for the period January 1, 2005 through December 31, 2010 were obtained from the census of Medicare fee-for-service inpatient discharges, which draws from Medicare claims data. The specifications, based on the work of Stevenson et al,17 can be found in the appendix, Supplemental Digital Content 1, http://links. lww.com/MLR/A596. There is a follow-up period of 30 days if the procedure did not involve an implant, and 1 year if the procedure involved an implant.18 The follow-up is partial for the last year of analysis due to data lags; the year indicators and sensitivity analyses addressed this issue. SCIP rates for the same years were extracted from the publicly available Hospital Compare database19 and aggregated up from the hospital-level to the group-level according to hospital teaching status, state, bed size, and location (urban or rural). SCIP data are available for essentially all general, acute-care, short-stay hospitals, which are incentivized by CMS to report SCIP compliance rates and other quality information through the Inpatient Quality Reporting program. The SCIP rates at the group level were calculated as the average SCIP rates of the hospitals included in the group, using the sample sizes reported by hospitals for SCIP rates as weights. Our analytic dataset resulted from r
2014 Lippincott Williams & Wilkins
SCIP Compliance and Surgical Infections
merging these SSI and SCIP measures for calendar years 2005 through 2010. We studied the association between SSI rates and SCIP compliance rates using a generalized linear model (GLM) with logistic link and binomial family. Because this regression analysis technique was designed for dependent variables that are expressed as rates, it is preferable to others, including ordinary least squares (OLS). It does not require ad hoc adjustments for SSI rates taking a value of 0 and yields predicted values that are straightforward to generate and interpret. Unlike OLS, marginal effects are not assumed to be constant through the range of the independent variable (SCIP rate), making nonlinearities easier to capture.20 Our model includes controls for all characteristics defining the groups, namely each of the 49 participating states using indicators, and for teaching hospital status, urban status, and bed size category. We also included year indicators to capture time trends. Throughout the study, we used heteroskedasticity-robust standard errors clustered at the group level. We did not include hospital group fixed effects to avoid the incidental parameter problem.21 The strong correlation between SCIP measures—that is, hospitals that comply with at least one SCIP measure also tend to comply with the remainder—posed a methodological challenge for our study. Including highly collinear dependent variables would produce highly imprecise estimates. To address this concern, we ran a separate set of regressions for each SCIP measure. In a first set of regressions, the SCIP measure Timely Antibiotic (ATB) Start, assessing the proportion of patients who receive antibiotics within 1 hour before surgery, was the main independent variable. In a second set of regressions, the SCIP measure Appropriate ATB was the main independent variable. The 2 sets of regressions were identical in all other respects. Each set of regressions consisted of 3 different model specifications. Model 1 had the main dependent variable only. Model 2 had the main dependent variable and a set of controls including state indicators, year indicators, group size, teaching status, and urban status. Model 3 included the interaction between urban status and the SCIP measure (Timely ATB Start or Appropriate ATB) in addition to the controls. Hospitals in urban settings are different from hospitals in rural settings in many major respects, such as size, complexity of procedures, and the underlying health conditions of their populations. Although the covariates included in model 2 would capture differences in SSI rates between urban and rural hospitals, differences in the marginal effect of SCIP scores on SSI rates could only be studied by including this interaction term.
RESULTS Descriptive Analyses After excluding 30 observations whose SSI rates were based on fewer than 10 records, we obtained an analytic sample consisting of 1430 year-groups. The sample included a total of 295 groups observed repeatedly over the study period 2007–2010, including 288 groups (comprising 4105 hospitals) for year 2007, 285 groups (4136 hospitals) for year www.lww-medicalcare.com |
S67
Medical Care
Cataife et al
Volume 52, Number 2 Suppl 1, February 2014
TABLE 1. Hospital Group Characteristics Variable Names
Variable Description
Mean; Median; SD
Range
Timely ATB Start
SCIP score 1: proportion of patients who were given an antibiotic within 1 hour before surgery SCIP score 2: proportion of patients whose preventive antibiotics were stopped within 24 h after surgery SCIP score 3: proportion of patients who received the right kind of antibiotic to help prevent infection SSI rate Member Council Teaching Hospitals Hospitals residing in counties with a Metropolitan Statistical Area (MSA)type of metropolitan area Between 50 and 199 beds 200 beds or more
0.90; 0.92; 0.08
0.22–1.00
0.86; 0.89; 0.11
0.22–1.00
0.94;0.96; 0.06
0.29–1.00
0.08; 0.08; 0.03 0.18; 0; 0.39 0.62; 1; 0.48
0–0.37 0–1 0–1
0.32; 0; 0.47 0.41; 0; 0.49
0–1 0–1
Timely ATB Stop Appropriate ATB SSI rate Teaching Urban Medium Large
The number of unique group-year units is 1430. ATB is the abbreviation for antibiotic; SCIP, Surgical Care Improvement Project; SSI, surgical site infection.
2008, 287 groups (4194 hospitals) for year 2009, and 284 groups (4123 hospitals) for year 2010. Table 1 displays summary statistics and descriptions for all variables. Compliance rates across the 3 SCIP measures ranged between 86% and 100%. The proportion of patients who were given an antibiotic within 1 hour before surgery had a mean of 90% (median 92%, SD 8%, range 22%–100%). Similarly, the proportion of patients who were given the appropriate antibiotics had a mean of 94% (median 96%, SD 6%, range 29%–100%). The mean SSI rate across groups was 8% (median 8%, SD 3.0%, range 0%–100%). Our estimates (not reported in Table 1) showed that SCIP measure compliance rates in urban locations were consistently higher compared with rural locations. In urban areas, ATB Start, ATB Stop, and ATB Appropriate rates were 90.9% (median 92.9%, range 21.9%–100%), 87.4% (median 90.0%, range 28.6%– 100%), 95.1% (median 96.4%, range 29.0%–100%), respectively, whereas in rural settings these same rates were 88.1% (median 90.2%, range 33.0%–100%), 84.7% (median 88.2%, range 22%–100%), and 93.3% (median 95.5%, range 35%–100%). SSI rates were also higher in urban locations (mean 8.22%, median 8.26%, range 0%–23.8%) compared with rural locations (mean 6.7%, median 6.0%, range 0%–36.7%). We first report the findings for the 2 core measures for which we found a robust association with SSI rates. We postpone to the end of this section a brief discussion of the findings for the remaining core measure.
Core Measure 1: Use of Antibiotic 1 Hour Before Intervention Table 2 shows 2 sets of regressions with SSI rates as the dependent variable. We first discuss the results of the regression of SSI rates on the use of antibiotic 1 hour before intervention (columns 1–3). The regression in model 1, which included a SCIP linear effect only, shows that SSI rates decreased as SCIP scores increased. The regression in model 2 shows that the coefficient for SCIP score remained negative and statistically significant after controlling for state, size, year, teaching status, and urban status. The regression in model 3 shows that rural hospitals had a coefficient ( 1.060) of much larger magnitude compared with
S68 | www.lww-medicalcare.com
urban hospitals ( 1.060 + 1.020 = 0.040). The model with the interaction term fit the data better than the model without the interaction term. The point estimates in interacted model were more precise and the coefficient for urban status switched from positive to negative.
Effect of Improving Compliance for ATB Starts To measure the size of the effect that an improvement in Timely ATB Start produces on SSI rates, we selected 1 state (Illinois) and year (2008) (arbitrarily) and the median values of our sample for the remaining covariates: nonteaching hospital, urban location, medium size. Table 3 provides the marginal effects calculated based on 1% increases from the 10%, 25%, 50%, 75%, and 90% percentiles of Timely ATB Start. All the coefficients are statistically significant; the P-values are those of the marginal effects of their associated Timely ATB Start coefficients (reported in Table 2). The marginal effects for model 2 were approximately 30% larger than those for model 1. The effect sizes consistently increased with compliance rates for both models, reflecting the fact that a 1% increase represents a higher percentage point increase as the level of the variable increases. A 1% increase in the compliance SCIP measure rate Timely ATB Start led to a 0.028–0.032 percentage point decrease in SSI rates in model 1 (0.038–0.043 in model 2). For instance, the 0.031 percentage point decrease associated to the median Timely ATB Start value in model 1 implies that a 10% improvement in the compliance rate would reduce the SSI rate from 7.6% to 7.3% (SSI numbers not reported in Table 3). Note that this represents a considerable percentage reduction of 3.9% from the baseline rate 7.6%. The effect was even larger according to model 2, which would imply a decrease from 7.6% to 7.2% (5.3% reduction). Model 3 disaggregated marginal effects by urban/rural location. The magnitude of the effect for urban locations, although still negative, was very small (approximately one tenth of that of model 1). However, it was surprisingly large for rural locations. For instance, starting from the median Timely ATB Start rate (0.92), a 10% increase in SCIP compliance would lead to a 0.7 percentage point decrease in the SSI rate, from 8.0% to 7.3% (that amounts to a 8.7% reduction from the baseline rate 8.0%). Finally, we ran r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
SCIP Compliance and Surgical Infections
TABLE 2. Compliance With 2 SCIP Measures and SSI: GLM Regression With SSI Rate as the Dependent Variable Timely ATB Start Coefficient (SE) [95% CI] (1) Model 1 Score effect
Appropriate ATB Coefficient (SE) [95% CI]
(2) Model 2
Score effect urban
0.478*** (0.161) [ 0.794, 0.163] Not included
0.552* (0.217) [ 0.977, 0.127] Not included
State dummies Year 2007 dummy
Not included Not included
Year 2008 dummy
Not included
Year 2009 dummy
Not included
Year 2010 dummy
Not included
Teaching dummy
Not included
Urban dummy
Not included
Medium size
Not included
Large size
Not included
Constant
2.060 (0.145)*** [ 2.344,1.776]
6w (out of 49) 0.023 (0.020) [ 0.016,0.064] 0.010 (0.026) [ 0.040,0.060] 0.052 (0.031) [ 0.010,0.114] 0.129*** (0.033) [ 0.194, 0.064] 0.220*** (0.034) [0.154,0.287] 0.095* (0.039) [0.020,0.171] 0.120* (0.048) [0.024,0.215] 0.196*** (0.043) [0.112,0.281] 2.039*** (0.252) [ 2.631, 1.446]
(3)
(4)
(5)
Model 3
Model 1
1.060*** (0.278) [ 1.604, 0.515] 1.020** (0.324) [0.384,1.655] 7w (out of 49) 0.026 (0.021) [ 0.015,0.066] 0.006 (0.028) [ 0.048,0.061] 0.039 (0.035) [ 0.030,0.108] 0.145*** (0.037) [ 0.216, 0.073] 0.218*** (0.033) [0.153,0.283] 0.811* (0.291) [ 1.381, 0.241] 0.124* (0.048) [0.030,0.218] 0.200*** (0.042) [0.117,0.283] 1.602*** (0.348) [ 2.285, 0.918]
1.023*** (0.226) [ 1.465, 0.580] Not included
0.625** (0.253) [ 1.121,0.130] Not included
Not included Not included
7w (out of 49) 0.005 (0.022) [ 0.039,0.048] 0.012 (0.026) [ 0.063,0.040] 0.018 (0.027) [ 0.034,0.070] 0.173*** (0.028) [ 0.229,0.117] 0.225*** (0.033) [0.159,0.290] 0.091 (0.039)* [0.015,0.168] 0.113 (0.047)* [0.021,0.206] 0.186 (0.040)*** [0.107,0.265] 1.890 (0.354)*** [ 2.582,1.196]
Not included Not included Not included Not included Not included Not included Not included 1.526*** (0.214) [ 1.945, 1.108]
n = 1429
(6)
Model 2
Model 3 1.127** (0.401) [ 1.914,0.340] 0.955* (0.434) [0.105,1.804] 8w (out of 49) 0.005 (0.022) [ 0.037,0.048] 0.017 (0.028) [ 0.072,0.037] 0.010 (0.029) [ 0.047,0.068] 0.180*** (0.031) [ 0.240,0.119] 0.226*** (0.033) [0.161,0.292] 0.801 (0.412) [ 1.609,0.006] 0.113* (0.047) [0.021,0.205] 0.181*** (0.040) [0.102,0.261] 1.429*** (0.479) [ 2.368,0.490]
n = 1425
GLM regression with logistic link and binomial family. Reference categories: score range 97.6–100, year 2006, nonteaching hospital, rural location, small size. SE are heteroskedasticity-robust and clustered at the group level. *Significant at 5% level. **Significant at 1% level. ***Significant at 0.1% level. w Number of statistical significant state dummies at 5% or lower level. ATB indicates antibiotic; CI, confidence interval; GLM, generalized linear model; SCIP, Surgical Care Improvement Project; SSI, surgical site infection.
similar effect size calculations assuming a flat 1 percentage point increase from baseline SCIP rate (instead of a 1% increase). This gave the effect of complying with an additional 1% of patients at different levels of compliance. These results (not reported) indicated that size effects consistently decreased with the level of SCIP compliance. For Timely ATB Start, the effect size at the 75% SCIP percentile was 7% and 9% lower compared with the 25% SCIP percentile for
models 1 and 2, respectively. For Appropriate ATB, the equivalent numbers were 8% and 5%.
Sensitivity Analysis We first estimated a regression using all control covariates but, instead of linear SCIP scores effects, we used dummies for 20 SCIP ranges of identical density. We found (results not shown) that 8 of the 19 SCIP ranges showed
TABLE 3. Magnitude of Effects Associated to GLM Regression—ATB Start Timely ATB Start Starting Value (Percentile), Final Value 0.790 0.858 0.920 0.960 0.981
(10%), (25%), (50%), (75%), (90%),
0.797 0.866 0.929 0.969 0.991
Marginal Effect (Percentage Points) Change (%)
Model 1
Model 2
Model 3 (Urban)
Model 3 (Rural)
1 1 1 1 1
0.028 0.029 0.031 0.032 0.032
0.038 0.040 0.041 0.042 0.043
0.003 0.003 0.003 0.003 0.003
0.066 0.068 0.069 0.069 0.069
Marginal effects computed as the difference in surgical site infection rates predicted based on the coefficients provided in Table 2 (statistical significance provided in Table 2). All predictions assume the state of Illinois, the year 2008, and nonteaching hospitals of medium size. Models 1–2 assume urban setting. For Model 3 we provide urban and rural estimates in separate columns.
r
2014 Lippincott Williams & Wilkins
www.lww-medicalcare.com |
S69
Medical Care
Cataife et al
statistical significance at the 5% level or lower, and the coefficients showed a downward (although nonmonotonic) trend, suggesting that as SCIP measures increased, SSI rates decreased. Next, we ran a similar exercise using 30 range dummies instead of 20. We found that 20 of the 29 dummies were statistically significant at the 5% level or lower. Again, there was a clear but nonmonotonic downward trend. Second, OLS is known to produce consistent estimates (and effect sizes) even in the presence of substantial measurement error in the dependent variable. The consequence of inaccuracies in claims data-based SSI rates on OLS estimates is wider confidence levels. We addressed concerns related to the quality of claims-based SSI rates reestimating model 2 using OLS. Using all covariates (same as model 2), we obtained a negative and statistically significant coefficient at the 5% level for Timely ATB Start. The OLS estimate had a coefficient of 0.038, implying that a 1 percentage point increase in compliance leads to an approximately 0.04 percentage point reduction in the SSI rate. This implies that a 0.92% increase such as that assumed in Table 3 for the 50% Timely ATB Start percentile (from 0.920 to 0.929) would lead to a 0.037 percentage point decrease in SSI— remarkably similar to the 0.041 GLM size effect. Finally, we reestimated all models dropping all 2010 data (that had limited follow-up); the results (see the online appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/ A596) were substantially the same.
Core Measure 2: Use of Appropriate Antibiotics We next turn to the results of the regressions of SSI rates on the SCIP measure related to the use of appropriate antibiotics. The results were very similar to those for timely use of antibiotics.
Effect of Improving Compliance for Appropriate Use of ATBs With minor exceptions, the results for this measure were similar to those for timely ATB (the sign and statistical significance of the variables in columns 4, 5, and 6 (Table 2), are similar to those in columns 1, 2, and 3, respectively). However, the effect sizes (Table 4) were substantially higher. The marginal effect at the median (50th percentile) for model 3 assuming a rural location is 0.078 percentage points in Table 4 versus 0.068 percentage points in Table 3.This implies that a 10% increase in SCIP compliance at the me-
Volume 52, Number 2 Suppl 1, February 2014
dian value of Appropriate ATB for a nonteaching, rurally located, medium-sized hospital would reduce the SSI rate from 8.2% to 7.4%.
Sensitivity Analysis Finally, we conducted a similar sensitivity analysis. Using dummies for SCIP ranges instead of linear scores, we found similar results (not shown). The number of statistically significant range dummies was 7 of 19 with 20 SCIP ranges and 16 of 29 with 30 SCIP ranges. Using OLS we observed a coefficient 0.046 (P < 0.05), implying that a 1% increase in SCIP compliance leads to an approximate 0.05 reduction in the SSI rate. Estimates excluding all 2010 data showed no substantial changes (see online appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/A596).
Core Measure 3: Timely ATB Stop We conducted similar analyses for the SCIP measure Timely ATB Stop (Table 5). We found a statistically significant and negative effect on SSI rates in model 1 only. However, the correlation disappeared when controls were introduced, suggesting that the initial correlation was driven by confounders.
DISCUSSION Three of the 9 publicly reported SCIP measures related to HAIs are considered core.2,3 We showed that 2 of the latter, Timely ATB Start and Appropriate ATB, have a high level of association with SSI rates. A third core measure, Timely ATB Stop, showed no robust association. SCIP measures are publicly reported measures of quality that may inform users’ decisions about where to get healthcare. Also, hospitals invest significant resources in complying with these measures and collecting and reporting their performance. Hence, it is critical to understand whether these measures adequately correlate to the underlying quality of surgical care. Our study provides robust evidence of this correlation for a subset of SCIP measures related to HAIs. Our results suggest that a tighter enforcement of SCIP measures in hospitals that are not complying consistently3 can lead to a substantial improvement in SSI rates and hence largely reduce the current societal burden22 of the estimated 750,000–1,000,000 annual SSIs,16 which includes an economic annual excess cost of $1.6 billion.23
TABLE 4. Magnitude of Effects Associated to GLSM Regression—Appropriate ATB Appropriate ATB Starting Value (Percentile), Final Value 0.888 0.930 0.960 0.977 0.985
(10%), (25%), (50%), (75%), (90%),
0.897 0.939 0.970 0.987 0.995
Marginal Effect (in Percentage Points) Change (%)
Model 1
Model 2
Model 3 (Urban)
Model 3 (Rural)
1 1 1 1 1
0.067 0.068 0.068 0.068 0.068
0.048 0.049 0.050 0.050 0.050
0.013 0.013 0.014 0.014 0.014
0.077 0.078 0.078 0.078 0.078
Marginal effects computed as the difference in surgical site infection rates predicted based on the coefficients provided in Table 2 (statistical significance provided in Table 2). All predictions assume the state of Illinois, the year 2008, and nonteaching hospitals of medium size. Models 1–2 assume urban setting. For Model 3 we provide urban and rural estimates in separate columns.
S70 | www.lww-medicalcare.com
r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
SCIP Compliance and Surgical Infections
TABLE 5. Compliance With Timely ATB Stop and SSI: GLM Regression With SSI as Dependent Variable Timely ATB Stop Coefficient (SE) [95% CI] (1) Model 1 Score effect
(2) Model 2
Score effect urban
0.467** (0.147) [ 0.755, 0.180] Not included
0.123 (0.177) [ 0.470,0.224] Not included
State dummies Year 2007 dummy
Not included Not included
Year 2008 dummy
Not included
Year 2009 dummy
Not included
Year 2010 dummy
Not included
Teaching dummy
Not included
Urban dummy
Not included
Medium size
Not included
Large size
Not included
Constant
2.086 (0.145)*** [ 2.340, 1.833]
7w (out of 49) 0.010 (0.024) [ 0.038,0.058] 0.018 (0.036) [ 0.088,0.053] 0.005 (0.041) [ 0.075.0.085] 0.184*** (0.046) [ 0.274, 0.094] 0.226*** (0.034) [0.160,0.293] 0.084* (0.039) [0.007,0.161] 0.098* (0.048) [0.004,0.192] 0.173*** (0.041) [0.092,0.253] 2.343*** (0.266) [ 2.864, 1.822] n = 1429
(3) Model 3 0.182 (0.229) [ 0.630,0.267] 0.126 (0.234) [ 0.332,0.584] 6w (out of 49) 0.008 (0.025) [ 0.040,0.057] 0.021 (0.037) [ 0.094,0.053] 0.001 (0.043) [ 0.083,0.085] 0.189*** (0.048) [ 0.283,0.095] 0.218*** (0.033) [0.160,0.292] 0.023 (0.205) [ 0.425,0.378] 0.099* (0.048) [0.005,0.192] 0.172*** (0.041) [0.092,0.253] 2.292*** (0.308) [ 2.896,1.687]
GLM regression with logistic link and binomial family. Reference categories: score range 97.6–100, year 2006, nonteaching hospital, rural location, small size. SE are heteroskedasticity-robust and clustered at the group level. ATB is the abbreviation for antibiotic; CI, confidence interval; GLM, generalized linear model; SSI, surgical site infection. *Significant at 5% level. **Significant at 1% level. ***Significant at 0.1% level. w Number of statistical significant state dummies at 5% or lower level.
For the SCIP measures Timely ATB Start and Appropriate ATB, the association is statistically significant and robust across regression models (GLM vs. OLS), across models that differ in terms of covariates included, and across models that vary with respect to the construction of the SCIP adherence score. The effect sizes are of considerable magnitude even in the lowest end of the range of estimates across models. On the basis of the available evidence from this and other studies,11–13 we believe that these findings support the ongoing use of Timely ATB Start and Appropriate ATB Use as performance measures that can provide valuable feedback to healthcare practices and stimulate quality improvement interventions. This analysis does not provide evidence to support the use of the Timely ATB Stop measure. The latter might be explained by the fact that the benefits of antibiotic discontinuation are dispersed over the general population and over time rather than concentrating on the specific patient whose antibiotic is discontinued at the time of surgery. This is because antibiotic discontinuation operates by minimizing the antibiotic resistance (rather than minimizing the chances of infection on the specific patient at the time of surgery). This dispersed effect would be difficult to capture in the regressions. r
2014 Lippincott Williams & Wilkins
The fact that urban hospitals had both higher SCIP and SSI rates compared with rural hospitals seems counterintuitive. However, this fact is explained by the results of model 3 (for both Timely ATB Start and Appropriate ATB measures), which showed that holding all other observed factors constant, rural hospitals had higher average SSI rates than urban hospitals. Model 3 showed that when the SCIP scores of rural hospitals improved, their SSI rates dropped more pronouncedly compared with urban hospitals. This is likely due to the finding of existence of decreasing size effects because, as already discussed, rural hospitals have lower average SCIP levels than urban hospitals. Previous studies found that SSI based on claims data are either subject to low sensitivity or poor positive predictive values.24–27 A plausible concern is that this would bias our estimates. Because OLS has been proven to give consistent estimates in the presence of measurement error,28 we addressed this concern by confirming that the results also hold when we use OLS. The fact that OLS results were very similar to GLM results is strong evidence that our results are not contaminated by measurement error. Finally, any measurement error in the SCIP measures would lead to attenuation bias, making our estimates of effect sizes conservative. www.lww-medicalcare.com |
S71
Medical Care
Cataife et al
Three major limitations are that we proved association but not causality, we used grouped data, and we did not control for patient level covariates such as demographics or comorbidities. Despite the use of a robust study design, this analysis was observational and could be subject to unknown confounders and secular trends. Major strengths are the use of a nationally representative sample, inclusion of a broad set of surgeries and a posthospital follow-up period, and the robustness of the results.
CONCLUSIONS We provided clear evidence that 2 SCIP compliance measures (timely use of antibiotic and appropriate use of antibiotics) were associated with lower SSI rates and the effect sizes are large. A third measure (timely stop of antibiotic) was not associated with lower SSI rates. This might be because its effects operate through antibiotic resistance and hence are dispersed over time and over a larger population. The combination of the magnitude of the annual levels of SSIs and the annual cost associated with the considerable effect sizes for the first 2 measures suggests the potential for large societal savings if tighter enforcement of these 2 SCIP measures is applied. The size of the effect is substantially larger in rural locations, suggesting that the enforcement of SCIP compliance in rural hospitals should be a policy priority. REFERENCES 1. CMS. Hospital inpatient quality reporting and hospital outpatient quality reporting surgical care improvement project technical expert panel. Available at: http://www.ofmq.com/Websites/ofmq/images/Measures/ IP_TEP_Lists/September_2012/SCIP_TEP_List_9.pdf. Accessed April 25, 2013. 2. Stulberg JJ, Delaney CP, Neuhauser DV, et al. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303:2479–2485. 3. Salkind AR, Kavitha CR. Antibiotic prophylaxis to prevent surgical site infections. Am Fam Physician. 2011;83:585–590. 4. Yokoe DS, Mermel LA, Anderson DJ, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29:S12–S21. 5. Andersen DJ, Kaye KS, Classen D, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29:S51–S61. 6. Canadian Task Force on the periodic health examination. The periodic health examination. Can Med Assoc J. 1979;121:1193–1254. 7. Mangram AJ, Horan TC, Pearson ML, et al. Guideline for prevention of surgical site infection, 1999. Hospital Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol. 1999;20:250–278. quiz 279-280. 8. Bratzler DW, Houck PM. Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project. Clin Infect Dis. 2004;38:1706–1715. 9. Bratzler DW, Hunt DR. The surgical infection prevention and surgical care improvement projects: national initiatives to improve outcomes for patients having surgery. Clin Infect Dis. 2006;43:322–330. 10. Quality Forum. QualityForum.org. May 2013. Available at: http:// www.qualityforum.org. Accessed May 25, 2013. 11. Bratzler DW, Houck PM, Richards C, et al. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical infection Prevention Project. Archives of Surgery. 2005; 140:174–182. 12. Classen DC, Evans RS, Pestotnik SL, et al. The timing of prophylactic administration of antibiotics and the risk of surgical-wound infection. NEJM. 1992;326:281–286.
S72 | www.lww-medicalcare.com
Volume 52, Number 2 Suppl 1, February 2014
13. Steinberg JP, Braun BI, Hellinger WC, et al. Timing of antimicrobial prophylaxis and the risk of surgical site infections: results from the trial to reduce antimicrobial prophylaxis errors. Annals of Surgery. 2009;250:10–16. 14. Pastor C, Artinyan A, Varma MG, et al. An Increase in compliance with the Surgical Care Improvement Project measures does not prevent surgical site infection in colorectal surgery. Dis Colon Rectum. 2010;53:24–20. 15. Barenguer CM, Gage Ochsner M, Lord A, et al. Improving surgical site infections: using National Surgical Quality Improvement Program Data to Institute Surgical Care Improvement Project Protocols in Improving Surgical Outcomes. J Am Coll Surg. 2010;5:737–741. 16. Edmiston CE, Spencer M, Lewis BD, et al. Reducing the risk of surgical site infections: did we really think SCIP was going to lead us to the promised land? Surg Infect. 2011;12:169–177. 17. Stevenson KB, Khan Y, Dickman J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health care– associated infections. Am J Infect Control. 2008;36:155–164. 18. CDC/NSHN. Surgical Site Infection Event. Available at: http:// www.cdc.gov/nhsn/PDFs/pscManual/9pscSSIcurrent.pdf?agree = yes& next = Accept. Accessed May 25 2013. 19. CMS. Hospital compare. Available at: http://www.medicare.gov/hospital compare. Accessed September 5, 2013. 20. Papke LE, Wooldridge JF. Econometric methods for fractional response variables with an application to 401(K) plan participation rates. J Appl Econometrics. 1996;11:619–632. 21. Lancaster T. The incidental parameter problem since 1948. J Econometrics. 2000;95:391–413. 22. Anderson DJ, Kaye KS, Classen D, et al. Strategies to prevent surgical site infections in acute care hospitals. Infect Control Hosp Epidemiol. 2013;29:S51–S61. 23. Lissovoy G, Fraeman K, Hutchins V, et al. Surgical site infection: incidence and impact on hospital and treatment costs. Infect Control Hosp Epidemiol. 2009;37:387–397. 24. Best WR, Khuri SF, Phelan M, et al. Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. J Am Coll Surg. 2002; 194:257–266. 25. Romano PS, Chan BK, Schembri ME, et al. Can administrative data be used to compare postoperative complication rates across hospitals? Med Care. 2010;40:856–867. 26. Cima RR, Cassivi SD, VanSuch M. How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACSNSQIP) Postoperative Adverse Events at a Single Institution. Surgery. 2011;150:943–949. 27. Lawson EH, Louie R, Zingmond DS, et al. A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications. Ann Surg. 2012;256:973–981. 28. Cameron AC, Trivedy PK. Microeconometrics. Methods and Applications. Ner York: Cambridge University Press; 2005. 29. Stevenson KB, Khan Y, Dickman J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of health care– associated infections. Am J Infect Control. 2008;36:155–164. 30. CDC/NSHN. Available at: http://www.cdc.gov/nhsn/PDFs/pscManual/ 9pscSSIcurrent.pdf?agree = yes&next = Accept. Accessed May 25, 2013. 31. Centers for Disease Control and Prevention (CDC). 2012. NHSN Members Meeting at APIC—San Antonio. Available at: http:// www.cdc.gov/nhsn/PDFs/MemberMeetings/NHSN-Members-MeetingAPIC-2012.pdf. Accessed August 29, 2012.
APPENDIX The Effect of Surgical Care Improvement Project (SCIP) Compliance on SSIs The specifications for SSI rates are based on the work of Stevenson et al.29 We amended the specifications by r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 2 Suppl 1, February 2014
“grouping” the denominator conditions so that inpatient stays with more than 1 denominator code for a particular type of infection (eg, 2 craniotomy codes) are counted only once. We added a limited number of numerator infection codes. The SSI numerator includes the following infections: meningitis, central nervous system abscess, pericarditis, endocarditis, mediastinits, cellulitis of various sites, myositis, osteomyelitis, periostitis, bone infection, open wounds, device reactions, and postoperative infections. There is a follow-up period of 30 days if the procedure did not involve an implant, and 1-year if the procedure involved an implant.30 The follow-up periods capture any inpatient stays and are consistent with NHSN’s surveillance methodology until that system’s recent change. As of 2013, the 1-year followup period was replaced with a 90-day follow-up period for some procedures, and the use of a surgical implant will no longer be recorded.31 The follow-up is partial for the last year of analysis, given that after discharge data are
r
2014 Lippincott Williams & Wilkins
SCIP Compliance and Surgical Infections
updated with considerable lag. This issue is addressed in the analysis through the inclusion of year dummies. The SSI denominator represents discharges associated with any of the following surgeries: coronary artery bypass graft, peripheral vascular, colorectal, craniotomy, head and neck, hysterectomy, spinal, ventricular shunt, and total knee and hip replacement. The categories small (fewer than 50 beds), medium (50–200), and large (200+) were used in the SCIP data file to approximate the bed size variable in the SSI rate. Urban was defined as located in a metropolitan area by the US Census Bureau. Acumen LLC performed the programming and calculations of the SSI rates using the Medicare claims data under contract with the Centers for Medicare and Medicaid. This analysis was funded by the Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
www.lww-medicalcare.com |
S73
