Original Research

Annals of Internal Medicine

Effect of Clinical and Social Risk Factors on Hospital Profiling for Stroke Readmission A Cohort Study Salomeh Keyhani, MD, MPH; Laura J. Myers, PhD; Eric Cheng, MD; Paul Hebert, PhD; Linda S. Williams, MD; and Dawn M. Bravata, MD

Background: The Centers for Medicare & Medicaid Services (CMS) and Veterans Health Administration (VA) will report 30-day stroke readmission rates as a measure of hospital quality. A national debate on whether social risk factors should be included in models developed for hospital profiling is ongoing. Objective: To compare a CMS-based model of 30-day readmission with a more comprehensive model that includes measures of social risk (such as homelessness) or clinical factors (such as stroke severity and functional status). Design: Data from a retrospective cohort study were used to develop a CMS-based 30-day readmission model that included age and comorbid conditions based on codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (model 1). This model was then compared with one that included administrative social risk factors (model 2). Finally, the CMS model (model 1) was compared with a model that included social risk and clinical factors from chart review (model 3). These 3 models were used to rank hospitals by 30-day risk-standardized readmission rates and examine facility rankings among the models. Setting: Hospitals in the VA.

S

troke is the fourth leading cause of death and a leading cause of disability among U.S. adults and the second most common cause of hospitalization in elderly persons (1). In the Veterans Health Administration (VA), more than 6000 veterans are hospitalized annually for acute ischemic stroke in a VA medical center (2). Within the VA, stroke is common and costly; understanding factors predictive of readmission is important to reducing 30-day readmission rates and improving outcomes. In some mortality models, differences in disease severity explain much of the variation in mortality rates among patients. However, most published 30-day readmission models, even those with detailed clinical data, explain little of the variation in readmission rates among patients (3). Recent research has shown that the predictive ability of 30-day readmission models for patients with congestive heart failure improves with the addition of clinically detailed information, such as severity of disease indices and “social risk factors” that represent the degree of chaos and social risk in a patient’s life (4). Identifying the social and clinical factors associated with 30-day stroke readmission would allow targeted interventions, such as supported discharge transitions, care coordination, home visits, physical therapy, earlier appointments, and greater education efforts, to be implemented to prevent readmission and potentially intervene on behalf of patients who are at the highest risk for readmission. A

www.annals.org

Participants: Patients hospitalized with stroke in 2007. Measurements: 30-day readmission rates. Results: The 30-day readmission rate was 12.8%. The c-statistics for the 3 models were 0.636, 0.646, and 0.661, respectively. All hospitals were classified as performing “as expected” using all 3 models (that is, performance did not differ from the VA national average); therefore, the addition of detailed clinical information or social risk factors did not alter assessment of facility performance. Limitation: A predominantly male veteran cohort limits the generalizability of these findings. Conclusion: In the VA, more comprehensive models that included social risk and clinical factors did not affect hospital comparisons based on 30-day readmission rates. Primary Funding Source: U.S. Department of Veterans Affairs.

Ann Intern Med. 2014;161:775-784. doi:10.7326/M14-0361 For author affiliations, see end of text.

www.annals.org

recent systematic review of stroke readmission showed limited literature on 30-day stroke readmission. An improved understanding of factors predictive of this outcome is warranted (5). In addition, the Centers for Medicare & Medicaid Services (CMS) has selected 30-day stroke readmission as a measure of hospital quality. It will also be used in the VA and reported on Hospital Compare; however, the proposed CMS stroke readmission model, like all other prediction models developed by the CMS for Hospital Compare, includes only age, sex, and comorbid conditions. The CMS has taken the position that including race in the model holds hospitals that care for minority populations to a different standard (6). Similarly, other variables of social risk (for example, low income and homelessness) are not included. However, current readmission models might penalize hospitals caring for disadvantaged populations with more needs. In response to this concern, the National Quality Forum invited public comment on whether sociodemographic variables should be included in hospital profiling and has published a draft report suggest-

See also: Related article. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 Editorial comment. . . . . . . . . . . . . . . . . . . . . . . . . . 833 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11 775

Original Research

Hospital Profiling for Stroke Readmission

Context Whether adjustment for medical and social risk factors available from administrative and electronic health records improves models to predict hospital readmission after stroke is not known.

Contribution The addition of detailed clinical information and social risk factors to a Centers for Medicare & Medicaid Services model that includes only age, sex, and comorbid conditions did not substantially alter the evaluation of 30-day readmission performance of Veterans Health Administration hospitals caring for patients with stroke.

Implication More comprehensive models to evaluate hospital readmission after stroke might not improve upon simpler models. —The Editors

ing that adjustment for these factors may be warranted (7). Nevertheless, given the lack of availability of such variables in Medicare data, information is limited on how they may affect hospital profiling. Because the VA shares outcome data with the Hospital Compare program for other conditions, understanding the effects of including the best clinical and social risk factors available on hospital-level comparisons in the VA is informative to policymakers. In this article, we examine the effect of including these factors on hospital profiling based on 30-day readmission rates.

METHODS Overview

We used data from the 2007 VA Office of Quality and Performance Stroke Special Project to construct 3 patientlevel models that examined predictors of 30-day readmission. First, we developed a 30-day readmission model using methods outlined by the CMS (model 1) (8). Second, we compared this model with one including some measures of social risk (for example, homelessness and substance abuse) from VA administrative data (model 2). Finally, we compared model 1 with a model that included social risk and clinical factors using data from medical record review (such as stroke severity and Acute Physiology and Chronic Health Evaluation [APACHE] and Morse Fall Scale scores) (model 3). We then ranked hospitals by their 30-day risk-standardized readmission rates (RSRRs) for each model and examined facility rankings among the 3 models to determine whether a more comprehensive model (models 2 or 3) classified hospital performance differently from the CMS-based readmission model (model 1). Data Source and Sample

A sample of 5000 veterans admitted to a VA hospital in fiscal year 2007 (9) with a primary discharge diagnosis 776 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

of ischemic stroke was identified from VA administrative data by using a modified high-specificity algorithm of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM), codes (10). Abstractors from the West Virginia Medical Institute who were specially trained for this study collected data through retrospective chart review of medical records. A total of 1013 patients were excluded because they had carotid endarterectomy during the stroke hospitalization, the stroke occurred after admission, or they were admitted only for poststroke rehabilitation. Patients were further excluded if they were transferred and ultimately discharged from a non-VA hospital (because we were interested only in VA hospitals), were discharged against medical advice (because providers did not have the opportunity to deliver full care and prepare the patient for discharge), died during the index hospitalization (because they were not eligible for readmission), or were enrolled in a Medicare HMO in the month before or after the index stroke admission (because readmission information may not be available). The final analytic sample included 3436 patients from 114 hospitals. Dependent Variable

Our main dependent variable was unplanned, allcause, 30-day readmission rates. We adhered to the principles outlined in CMS technical documents to define hospital readmissions (6, 8, 11). Specifically, readmissions were defined as a subsequent inpatient admission to any acute care facility in the VA or a CMS facility within 30 days of discharge from the index stroke hospitalization. We used the recently updated CMS Planned Readmission Algorithm, version 3.0, for stroke to identify planned readmissions (8), which determines hospitalizations that may be planned for follow-up stroke care (for example, carotid endarterectomy) and other commonly planned hospitalizations (such as elective cholecystectomy). Per the updated CMS Planned Readmission Algorithm, if the first readmission after discharge is planned, any subsequent unplanned readmissions are not counted as an outcome because the unplanned readmission may be related to care received during the planned readmission. Thus, patients with a planned readmission followed by an unplanned readmission within 30 days of discharge were considered not to have had a readmission. Readmissions were assessed from several data sources, including VA medical SAS inpatient data (SAS Institute), the VA Non-VA Medical Care (formerly Fee Basis) inpatient files (which include care at non-VA facilities that is paid for by the VA), and CMS Medical Provider Analysis and Review files for hospitalizations in the Medicare program. Independent Variables

Independent variables included demographic, clinical, and social risk factors associated with 30-day readmission. These domains were built on previous work that identified domains of health associated with 30-day readmission (4, 12). www.annals.org

Hospital Profiling for Stroke Readmission Demographic Characteristics

Demographic variables included age, sex, and race data available in VA national data sets for each veteran. Data on race were 98% complete. Clinical Characteristics

For clinical characteristics, we included data on disease severity, functional status, and utilization. Disease Severity. The 25 clinical covariates outlined in the CMS technical documents for creating the 30-day stroke readmission measure for the CMS-based model (model 1) were identified in VA and CMS administrative data in the 12 months before the stroke admission and were included as separate dichotomous variables in the CMS risk-standardized readmission models. For the other 2 models, we included additional clinical measures based on data from the medical record review. Trained abstractors retrospectively constructed the National Institutes of Health Stroke Scale, a measure of stroke severity, from physician notes within 24 hours of admission (13). Disease severity was represented by a modified APACHE score (14) calculated using admission data for each veteran. Data on hypoxia, dysphagia, and code status were collected during medical record abstraction. We calculated a Charlson Comorbidity Index score (15) based on data on each patient’s medical history. Finally, as another measure of health status, we created a dichotomous variable that indicated whether a veteran’s copayment was waived for VA health care services because of a military service– connected disability. Functional Status. We used 3 measures to categorize functional status. First, the Morse Fall Scale is a validated measure documented in the medical record for each veteran during hospitalization (16 –18). The scale ranges from 0 to 150, and a score greater than 50 indicates a high risk for falls. Second, chart abstractors classified patients as ambulatory before the stroke, nonambulatory, or of unknown status. Third, using administrative data, we categorized patients on the basis of whether they were receiving VA home-based primary care or were eligible to receive such services in the year before hospital admission. Patients are typically referred to home-based primary care when families and providers deem them unable to travel for primary care services because of illness. Patients are eligible for such services if they live within a 60-mile radius of the VA facility. Utilization. We determined the total number of acute care hospitalizations and outpatient visits in the year before admission for each veteran by using VA and CMS administrative data. Social Risk Factors

We used the ZIP code of residence at the time of stroke admission to estimate the percentage of persons in a neighborhood below the poverty threshold, which was obwww.annals.org

Original Research

tained from the 2008 to 2012 American Community Survey (19). We also classified patients as “low income” if their VA copayment was waived on the basis of their means test evaluation. Dichotomous variables were created using administrative data to indicate whether patients had an active substance abuse disorder based on ICD-9-CM codes or utilized substance abuse services based on outpatient clinic codes and inpatient hospital bed section codes in the year before the stroke admission. Similarly, we created dichotomous variables indicating a mental health disorder or homelessness by using a combination of ICD-9-CM codes and data on outpatient and inpatient utilization of homeless patient services. We included data on whether a patient had a health care visit with a social worker by using outpatient clinic codes or a diagnosis of a sexually transmitted infection in the 2 years before stroke admission based on ICD-9-CM codes. We used the diagnosis of sexually transmitted infection as 1 possible indication of a patient with “high-risk behavior.” Statistical Analysis Readmission Models

We developed 3 readmission models using the SAS statistical code that the CMS has made publicly available (Appendix and Appendix Table 1, available at www.annals .org). Model 1 was constructed similarly to the CMS model with 1 difference: We did not include sex in our model because so few veterans with stroke admission were women (2%). Otherwise, model 1 was a “CMS-based” model that included measures of age and 25 cardiovascular, cerebrovascular, and other comorbid conditions. Model 2 included CMS variables and social risk factors. Model 3 included CMS variables, social risk factors, and clinical covariates (model 2 plus clinical covariates). Although we reported the relationship between race and 30-day readmission in our univariate analyses, we adhered to CMS principles and did not include it in the multivariate models.

Patient-Level Analysis

The patient-level analyses included all patients with an eligible stroke admission from all VA facilities, regardless of the number of patients hospitalized at each facility. We calculated means and distributions of patient characteristics included in each of the 3 risk-standardized readmission models. Adjusted odds ratios (ORs) and 95% CIs were reported to identify individual factors that were predictive of 30-day readmission. Model performance was assessed using various summary statistics, including area under the receiver-operating characteristic curve, percentage of variation explained by the risk covariates, predictive ability, and Hosmer–Lemeshow goodness-of-fit statistics. The cstatistic from the logistic regression models was used to 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11 777

Original Research

Hospital Profiling for Stroke Readmission

Table 1. Patient-Level 30-d Readmission Frequencies and Univariate Analyses Variable

P Value

Readmitted No (n ⴝ 2996; 87.2%)

Yes (n ⴝ 440; 12.8%)

66.9 (11.4)

69.5 (11.4)

66.6 24.1 7.1 2.2

68.2 24.1 7.3 0.5

CMS comorbid conditions, % CHF Hypertensive heart disease Cerebral hemorrhage Ischemic or unspecified stroke Cerebrovascular disease Hemiplegia, paraplegia, paralysis, or functional disability Vascular or circulatory disease Metastatic cancer or acute leukemia Cancer DM and related complications Protein–calorie malnutrition Fluid, electrolyte, acid, or base disorders Obesity/disorders of the thyroid or cholesterol or lipid levels Severe hematologic disorders Iron-deficiency and other/unspecified anemias Dementia and senility Quadriplegia, paraplegia, or functional disability Seizure disorders and convulsions COPD Other lung disorders ESRD or dialysis Renal failure Other urinary tract disorders Decubitus or chronic skin ulcer Major symptoms or abnormalities

15.8 1.5 0.70 28.8 14.9 10.4 25.2 1.0 17.9 40.6 0.9 14.7 59.6 0.5 17.1 10.7 3.4 4.8 19.3 10.2 0.6 13.5 11.1 4.1 43.6

22.7 2.5 1.4 31.4 15.7 13.0 36.6 3.4 25.5 46.8 2.3 23.4 64.3 1.6 28.0 15.7 4.3 5.9 24.6 15.5 2.7 23.2 17.3 10.5 58.0

⬍0.001 0.139 0.146 0.27 0.68 0.103 ⬍0.001 ⬍0.001 ⬍0.001 0.013 0.023 ⬍0.001 0.056 0.013 ⬍0.001 0.002 0.35 0.30 0.011 0.001 ⬍0.001 ⬍0.001 ⬍0.001 ⬍0.001 ⬍0.001

Social risk factors, % Below the poverty threshold (SD)* Low income Substance abuse Homeless Mental health disorder Social work visit in prior year STI in past 2 y

18.9 (10.6) 48.0 28.4 7.4 39.8 22.7 5.5

19.9 (11.3) 54.1 29.1 9.1 45.9 33.0 7.1

0.079 0.018 0.77 0.22 0.015 ⬍0.001 0.183

12.1 (7.1) 1.5 (1.5)

14.1 (7.8) 1.9 (1.6)

⬍0.001 ⬍0.001 0.001

50.7 40.1 9.3 1.4 9.0 10.7 36.3

43.9 41.8 14.3 2.5 11.8 13.9 33.0

41.4 34.1 24.6

37.3 38.2 24.6

Demographic characteristics Mean age (SD), y Race, % White Black Other Unknown

Clinical factors Disease severity APACHE score (SD) Charlson Comorbidity Index score (SD) NIHSS score, % 0–2 3–9 ⱖ10 Hypoxia, % Dysphagia, % DNR/DNI, % Disability, %† Functional status Morse Fall Scale score, % 0–50 (low) ⬎50 (high) Unknown

⬍0.001 0.115

0.070 0.059 0.050 0.174 0.177

Continued on following page

778 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

www.annals.org

Hospital Profiling for Stroke Readmission

Original Research

Table 1—Continued Variable No (n ⴝ 2996; 87.2%) Nonambulatory before admission HBPC, % Eligible and using HBPC Eligible and not using HBPC Not eligible for HBPC Utilization‡ ⱖ1 hospital visit in prior year, % Outpatient visits in prior year (SD), n

P Value

Readmitted Yes (n ⴝ 440; 12.8%)

5.8

7.5

2.3 86.2 11.5

5.0 85.0 10.0

27.9 18.1 (21.7)

44.3 24.3 (23.7)

0.164 0.004

⬍0.001 ⬍0.001

APACHE ⫽ Acute Physiology and Chronic Health Evaluation; CHF ⫽ congestive heart failure; CMS ⫽ Centers for Medicare & Medicaid Services; COPD ⫽ chronic obstructive pulmonary disease; DM ⫽ diabetes mellitus; DNI ⫽ do not intubate; DNR ⫽ do not resuscitate; ESRD ⫽ end-stage renal disease; HBPC ⫽ home-based primary care; NIHSS ⫽ National Institutes of Health Stroke Scale; STI ⫽ sexually transmitted infection. * According to the U.S. Census Bureau. † Based on priority score. ‡ Visits in the Veterans Health Administration include those to physicians, the laboratory, radiology, the pharmacy, and prosthetics departments and other services available to veterans.

examine model performance and assess the incremental effect of adding other variables (such as social risk factors) to the model. Discrimination of model 1 was assessed by determining the c-statistic and was compared with the discrimination of the model with the addition of social risk factors (model 2) and then social and clinical risk factors (model 3). We also used the net reclassification improvement (NRI) index, which compares the shifts in reclassified categories by readmission and results from the addition of the social risk and clinical covariates to the model. A higher NRI index indicates greater improvement in risk discrimination and better reclassification. Hospital-Level Analysis

We first reported the observed unadjusted readmission rate across the 114 hospitals. Then, using the approach developed by the CMS, we calculated hospital-specific RSRRs for each hospital and reported mean, median, and interquartile ranges. Bootstrapping techniques were used to calculate 95% CIs. In brief, the CMS method estimates hospital-level, 30day, all-cause RSRRs by using a hierarchical generalized logistic regression model to account for the clustering of patients within hospitals and sample size variations among hospitals. This model calculates the RSRR by producing a ratio of the number of “predicted” readmissions to the number of “expected” readmissions and multiplying that figure by the national unadjusted readmission rate (20, 21). Hospitals with at least 25 stroke admissions were then ranked by their 30-day RSRRs and classified as better than, worse than, or no different from the national average on the basis of whether the 95% CI was higher, lower, or overlapped the mean national unadjusted rate. Per current CMS methodology, hospitals with fewer than 25 stroke admissions are not reported because the cases are too few to reliably tell how well the hospital is performing. We calcuwww.annals.org

lated hospital-level distributions of demographic and clinical characteristics across hospitals (Appendix Table 2, available at www.annals.org). All analyses were done using SAS software, version 9.2. Institutional review board approval was obtained. Role of the Funding Source

This study was funded by the VA Health Services Research and Development Service Quality Enhancement Research Initiative and the National Heart, Lung, and Blood Institute. The funding sources had no role in the design, analysis, or decision to submit the manuscript for publication.

RESULTS Patient-Level Characteristics

The average age in our sample was 67.2 years, and most of the 3436 patients were white (66.8%). Overall, the 30-day readmission rate was 12.8%. Univariate analyses showed that patients who were readmitted within 30 days of discharge had higher rates of comorbid conditions— including congestive heart failure, renal failure, chronic obstructive pulmonary disease, and metastatic cancer—than those who were not readmitted (Table 1). Patients who were readmitted also seemed to have a higher rate of some social risk measures. For example, they were more likely to have had a visit with a social worker, a mental health disorder, or a waived VA copayment because of low income. Patients who were readmitted also had more severe illness as shown by higher APACHE and Charlson Comorbidity Index scores, had worse stroke severity, were more likely to use home-based primary care, and had significantly higher rates of health care utilization. Patient-Level Models and Multivariate Analysis Model 1 (CMS-Based Model)

Multivariate analyses showed that older patients with a higher prevalence of comorbid conditions were more likely 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11 779

Original Research

Hospital Profiling for Stroke Readmission

Table 2. Patient-Level Adjusted ORs* Variable

Adjusted OR (95% CI) Model 1†

Model 2‡

Model 3§

Age

1.013 (1.003–1.022)

1.015 (1.004–1.026)

1.014 (1.003–1.025)

CMS comorbid conditions CHF Hypertensive heart disease Cerebral hemorrhage Ischemic or unspecified stroke Cerebrovascular disease Hemiplegia, paraplegia, paralysis, or functional disability Vascular or circulatory disease Metastatic cancer or acute leukemia Cancer DM and related complications Protein–calorie malnutrition Fluid, electrolyte, acid, or base disorders Obesity/disorders of the thyroid or cholesterol or lipid levels Severe hematologic disorders Iron-deficiency and other/unspecified anemias Dementia and senility Quadriplegia, paraplegia, or functional disability Seizure disorders and convulsions COPD Other lung disorders ESRD or dialysis Renal failure Other urinary tract disorders Decubitus or chronic skin ulcer Major symptoms or abnormalities

0.987 (0.750–1.301) 1.210 (0.596–2.459) 1.452 (0.564–3.737) 0.957 (0.744–1.233) 0.856 (0.635–1.155) 1.002 (0.706–1.423) 1.205 (0.945–1.535) 2.312 (1.171–4.565) 1.171 (0.903–1.518) 1.106 (0.888–1.377) 1.458 (0.664–3.203) 1.031 (0.771–1.378) 0.964 (0.765–1.214) 2.006 (0.761–5.291) 1.135 (0.864–1.492) 1.077 (0.787–1.474) 0.767 (0.445–1.320) 1.095 (0.695–1.726) 1.039 (0.804–1.344) 1.133 (0.830–1.547) 2.257 (0.995–5.121) 1.260 (0.936–1.696) 1.063 (0.782–1.446) 1.912 (1.292–2.830) 1.337 (1.055–1.694)

0.958 (0.725–1.264) 1.238 (0.608–2.517) 1.287 (0.498–3.323) 0.937 (0.727–1.208) 0.889 (0.658–1.200) 0.989 (0.696–1.405) 1.192 (0.934–1.522) 2.168 (1.091–4.308) 1.166 (0.898–1.513) 1.105 (0.886–1.377) 1.417 (0.641–3.131) 1.013 (0.757–1.355) 0.997 (0.789–1.260) 2.105 (0.796–5.563) 1.112 (0.845–1.463) 1.047 (0.745–1.470) 0.739 (0.428–1.276) 1.085 (0.686–1.714) 1.004 (0.773–1.304) 1.157 (0.845–1.581) 2.230 (0.976–5.096) 1.226 (0.909–1.654) 1.065 (0.782–1.450) 1.899 (1.280–2.819) 1.289 (1.013–1.639)

0.882 (0.664–1.171) 1.234 (0.606–2.515) 1.239 (0.476–3.224) 0.893 (0.690–1.156) 0.870 (0.643–1.178) 0.943 (0.660–1.347) 1.161 (0.907–1.485) 1.791 (0.880–3.644) 1.111 (0.849–1.454) 1.012 (0.804–1.274) 1.359 (0.615–3.005) 0.969 (0.721–1.303) 1.004 (0.792–1.272) 1.968 (0.734–5.277) 1.058 (0.801–1.397) 1.013 (0.716–1.435) 0.755 (0.433–1.314) 1.061 (0.669–1.683) 0.939 (0.717–1.228) 1.139 (0.831–1.563) 1.782 (0.758–4.189) 1.040 (0.758–1.427) 1.060 (0.778–1.445) 1.818 (1.214–2.721) 1.223 (0.952–1.571)

– – – – – – –

1.008 (0.999–1.018) 1.253 (1.016–1.545) 1.070 (0.834–1.374) 0.865 (0.581–1.288) 1.012 (0.794–1.289) 1.284 (0.990–1.666) 1.234 (0.802–1.900)

1.007 (0.998–1.017) 1.200 (0.871–1.654) 1.060 (0.824–1.364) 0.862 (0.577–1.288) 1.030 (0.804–1.319) 1.195 (0.911–1.567) 1.109 (0.710–1.730)

– –

– –

1.018 (1.003–1.033) 1.074 (0.994–1.159)

– – – – – – –

– – – – – – –

1.000 (reference) 1.111 (0.885–1.396) 1.402 (0.981–2.003) 1.374 (0.678–2.786) 1.080 (0.766–1.523) 0.973 (0.705–1.343) 0.962 (0.684–1.354)

– – – –

– – – –

1.000 (reference) 0.958 (0.747–1.230) 0.920 (0.701–1.207) 1.337 (0.863–2.072)

– – –

– – –

1.000 (reference) 0.867 (0.505–1.488) 0.846 (0.453–1.579)

Social risk factors Below the poverty threshold㛳 Low income Substance abuse Homeless Mental health disorder Social work visit in prior year STI Clinical factors Disease severity APACHE score Charlson Comorbidity Index score NIHSS score 0–2 3–9 ⱖ10 Hypoxia Dysphagia DNR/DNI Disability¶ Functional status Morse Fall Scale score 0–50 (low) ⬎50 (high) Unknown Nonambulatory HBPC Eligible and using HBPC Eligible and not using HBPC Not eligible for HBPC

Continued on following page

780 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

www.annals.org

Hospital Profiling for Stroke Readmission

Original Research

Table 2—Continued Variable

Adjusted OR (95% CI) Model 1†

Utilization** ⱖ1 hospital visit in prior year Outpatient visits in prior year

Model 2‡ – –

Model 3§ – –

1.342 (1.022–1.763) 1.001 (0.995–1.006)

APACHE ⫽ Acute Physiology and Chronic Health Evaluation; CHF ⫽ congestive heart failure; CMS ⫽ Centers for Medicare & Medicaid Services; COPD ⫽ chronic obstructive pulmonary disease; DM ⫽ diabetes mellitus; DNI ⫽ do not intubate; DNR ⫽ do not resuscitate; ESRD ⫽ end-stage renal disease; HBPC ⫽ home-based primary care; NIHSS ⫽ National Institutes of Health Stroke Scale; OR ⫽ odds ratio; STI ⫽ sexually transmitted infection. * Values are adjusted ORs (95% CIs) unless otherwise indicated. Race was not included in the hospital-level analysis. † CMS-based model. ‡ Model 1 plus social risk factors. § Model 2 plus clinical factors. 㛳 According to the U.S. Census Bureau. ¶ Based on priority score. ** Visits in the Veterans Health Administration include those to physicians, the laboratory, radiology, the pharmacy, and prosthetics departments and other services available to veterans.

to be readmitted (OR, 1.01 [95% CI, 1.00 to 1.02]) (Table 2). Patients with metastatic cancer (OR, 2.31 [CI, 1.17 to 4.56]) and skin ulcers (OR, 1.91 [CI, 1.29 to 2.83]) had greater odds of 30-day readmission. The c-statistic for our CMS-based model (model 1) was 0.636, which is similar to that reported in the CMS technical documents (0.623) for patients aged 18 years or older (Table 3) (11). Model 2 (CMS-Based Model Plus Social Risk Factors)

Multivariate analysis showed a significant association between low-income veterans and 30-day readmission. Other social risk factors (for example, homelessness, social work visits, and use of substance abuse services) were not associated with 30-day readmission in patients with stroke. A likelihood ratio test of joint significance can be used to determine whether a group of explanatory variables (such as all social risk factors) significantly affects an outcome by calculating the difference in likelihood ratio test statistics between the 2 models; this test showed that adding the 7 social risk factors to model 1 was not significantly associated with 30-day readmission rates (P ⫽ 0.060). Despite a higher prevalence of social risk factors among patients who were readmitted and the significant differences among these variables at the hospital level (Appendix Table 2), these variables had a limited effect on

improving model performance based on the c-statistic (for model 2, 0.646; P ⫽ 0.11). However, the NRI index (OR, 0.205 [CI, 0.106 to 0.305]; P ⬍ 0.001) more accurately classified risk for 30-day readmission in model 2. Model 3 (CMS-Based Model Plus Social Risk and Clinical Factors)

Model 3 also showed the importance of age and clinical factors on 30-day readmission. Older patients (OR, 1.01 [CI, 1.00 to 1.02]), those with skin ulcers (OR, 1.82 [CI, 1.24 to 2.72]), and those with higher APACHE scores (OR, 1.02 [CI, 1.00 to 1.03]) were more likely to be readmitted within 30 days. Stroke severity was not independently associated with readmission in this model. Patients with at least 1 hospitalization in the past year (OR, 1.34 [CI, 1.02 to 1.76]) had higher odds of 30-day readmission. The addition of social and clinical factors to model 1 had a statistically significant effect on model performance (c-statistic, 0.661; P ⫽ 0.005). A likelihood ratio test of joint significance for the addition of the social and clinical risk factors to model 1 was significant (P ⫽ 0.045). The NRI index (0.261 [CI, 0.161 to 0.360]; P ⬍ 0.001) more accurately classified 30-day readmission when clinical and social risk factors were included in the risk prediction model.

Table 3. Model Performance Variable

Model 1*

Model 2†

Model 3§

c-Statistic Hosmer–Lemeshow GOF statistic Adjusted R2 Predictive ability (lowest decile, highest decile) Model chi-square NRI index (95% CI)

0.636 0.866 0.056 (5.83%, 27.11%) 107.93 –

0.646 0.462 0.063 (4.37%, 27.41%) 120.00 0.205 (0.10–0.30)

0.661 0.856 0.074 (4.08%, 28.28%) 138.37 0.26 (0.16–0.36)

GOF ⫽ goodness-of-fit; NRI ⫽ net reclassification improvement. * Centers for Medicare & Medicaid Services–based model. † Model 1 plus social risk factors. ‡ Model 2 plus clinical factors. www.annals.org

2 December 2014 Annals of Internal Medicine Volume 161 • Number 11 781

Original Research

Hospital Profiling for Stroke Readmission

Table 4. Distribution of Observed and 30-d RSRRs All Patients Aged >18 y (n ⴝ 3436)

Variable

Mean (95% CI) Median (25th, 75th percentile) Range

RSRR, %

Patients Across All 114 VA Hospitals Studied, n

Observed Readmission Rate, %*

Model 1†

Model 2‡

Model 3§

30.1 (21.5㛳) 28.5 (14.0, 39.0) 1–99

11.1 (9.8–12.6) 11.5 (5.7, 15.8) 0.0–37.5

12.8 (8.7–18.4) 12.8 (12.4, 13.2) 11.2–17.0

12.8 (8.7–18.4) 12.8 (12.4, 13.1) 11.5–16.1

12.8 (8.7–18.4) 12.8 (12.5, 13.2) 11.6–15.4

CMS ⫽ Centers for Medicare & Medicaid Services; RSRR ⫽ risk-standardized readmission rate; VA ⫽ Veterans Health Administration. * Analysis limited to facilities with at least 25 patients (per CMS guidelines). † CMS-based model. ‡ Model 1 plus social risk factors. § Model 2 plus clinical factors. 㛳 SD.

Effect of Using a Detailed Model on Comparisons of 30-Day RSRRs Among Hospitals

Table 4 shows the distribution of facility volume, observed readmission rate, and RSRRs. The mean observed hospital-level 30-day readmission rate was 11.1% (CI, Figure. Hospital-level RSRRs. CMS Model

RSRR (95% CI)

0.25 0.20 0.15 0.10

1

6

11

16

21

26

31

36

41

46

51

56

61

Facilities, n CMS Model and Social Risk Factors

RSRR (95% CI)

0.25 0.20

DISCUSSION

0.15 0.10

1

6

11

16

21

26

31

36

41

46

51

56

61

Facilities, n CMS Model and Social Risk and Clinical Factors

0.25

RSRR (95% CI)

9.8% to 12.6%). The RSRR calculated from the 3 models had a similar range. For the 65 facilities with at least 25 patients, we plotted the 30-day RSRRs for each facility estimated with the 3 models side by side to examine whether they differed across hospitals and whether use of a comprehensive model affected these differences. The Figure shows the RSRRs with the associated 95% CIs ordered by hospital by using the 3 models. All 3 models yielded “as-expected” (that is, they did not differ from those of the national VA average) 30day RSRRs for all hospitals; therefore, the addition of detailed clinical information or social risk factors did not alter assessment of facility performance. We also examined the rank-order correlation among the 3 models, which was high (model 1 vs. model 2, r ⫽ 0.988; model 1 vs. model 3, r ⫽ 0.982; and model 2 vs. model 3, r ⫽ 0.994), suggesting that adding clinical and social risk factors to a CMS-based model does not alter facility rankings. In other words, a more comprehensive model improved model performance but did not affect hospital profiling in the VA.

0.20 0.15 0.10

1

6

11

16

21

26

31

36

41

46

51

56

61

Facilities, n

CMS ⫽ Centers for Medicare & Medicaid Services; RSRR ⫽ riskstandardized readmission rate. 782 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

We found that clinical factors, such as severity of general illness and comorbid conditions, are important predictors of 30-day stroke readmission at the patient level but explain little of the variation in 30-day readmission rates. This finding is in line with that of other disease-specific 30-day readmission models (3). The poor overall performance of the stroke readmission model in our study (cstatistic ⬵ 0.6) and research for other conditions indicate that identifying patients at high risk for 30-day stroke readmission for the purpose of intervening in their care remains a challenge (3, 22). We also found that models including social risk and clinical factors performed better than a CMS-based model but did not affect hospital profiling based on 30-day readmission to a VA hospital. This finding may not be too surprising because the modeling methodology used by the CMS significantly affects whether hospitals can be distinguished from each another. The Bayesian approach used by www.annals.org

Hospital Profiling for Stroke Readmission

the CMS “shrinks” the point estimate of smaller hospitals toward the grand mean (23, 24) and thus is less likely to identify a small hospital as an outlier. In our study, the CMS approach essentially rendered all VA hospitals similar in terms of 30-day readmission. Larger changes in model performance or larger samples may be necessary to affect the profiling of a hospital using this methodology. As part of the Patient Protection and Affordable Care Act, the CMS was directed to develop an incentive program designed to penalize hospitals that have readmission rates higher than the national mean (25). The proposed rule has caused concern among some hospitals, especially those caring for more disadvantaged populations, because the models used to compare hospitals do not account for the social risk of the patients served. As a result, hospitals caring for disadvantaged populations might be penalized by current readmission models that do not account for their needs (3, 4, 26, 27). Model 2 included many social risk factors that varied at the hospital level (Appendix Table 2), but only low income was significantly associated with readmission. Perhaps most of the social risk factors that we identified are not useful for improving the prediction of stroke readmission, and other variables—such as “living alone” or “social support,” which were unavailable for this analysis—are more important. Further research repeating this work with an improved set of social risk factors, a focus on other diseases, and in non-VA settings should shed light on the importance of social risk factors for hospital profiling. Our results provide some useful information to VA policymakers. A CMS-based Bayesian approach essentially rendered all VA hospitals similar in terms of 30-day readmission. Overall, the CMS approach may not be a good measure to assess VA hospital quality given the low annual volume of stroke readmissions. On the other hand, it may be suited for quality assessments that include penalties, because outliers are less likely to be identified simply by chance and fewer hospitals would be inappropriately penalized. Our study has limitations. First, our analysis is based on a predominantly male veteran cohort, which limits the generalizability of our findings. Second, we included hospital readmissions for VA and non-VA facilities using VA data, Non-VA Medical Care files (admissions reimbursed to private hospitals by the VA), and Medicare data. Among patients aged 65 years or older who were readmitted, 20.7% were admitted only to a hospital participating in the Medicare program, which shows the importance of including these data in VA readmission modeling. For veterans who are not Medicare beneficiaries, these analyses may have missed hospitalizations to some non-VA facilities because these hospitalizations may have not been captured in the Non-VA Medical Care files. The rate at which veterans aged 18 to 65 years who are initially hospitalized at a VA medical center for their index stroke and then are readmitted at non-VA facilities is not www.annals.org

Original Research

known nationally. However, in a preliminary analysis of readmissions of 10 000 veterans to non-VA hospitals in Indiana, we did not find readmissions to non-VA hospitals in a cohort of patients with minor stroke. In addition, Hospital Compare will report VA hospital-level data on 30-day readmission by using the CMS methodology. We used the publicly available CMS macros to do these analyses. Checking whether the underlying methods used by the CMS were appropriate for VA data was beyond the scope of this article. Finally, the small sample size and limited facility volume, although in line with CMS guidelines on inclusion of facilities with more than 25 patients (8), may have diminished our ability to differentiate hospitals by using a CMS hierarchical approach. In conclusion, we found that adding clinical data and some administrative measures of social risk led to some improvement in model performance compared with a CMS-based model but did not affect hospital profiling in the VA. Further research is needed to understand factors that explain variation in 30-day readmission for patients admitted with stroke. From the San Francisco Veterans Affairs Medical Center and University of California, San Francisco, San Francisco, California; Veterans Health Administration Health Services Research and Development Stroke Quality Enhancement Research Initiative Program and Center of Excellence on Implementing Evidence-Based Practice, Richard L. Roudebush Veterans Affairs Medical Center, Indiana University School of Medicine, and Regenstrief Institute, Indianapolis, Indiana; Veterans Affairs Greater Los Angeles Healthcare System and University of California, Los Angeles, Los Angeles, California; and University of Washington and Veterans Affairs Puget Sound Health Care System, Seattle, Washington. Disclaimer: The views expressed in this article are those of the authors

and do not necessarily represent the views of the U.S. Department of Veterans Affairs. Financial Support: By the Veterans Health Administration Office of Quality and Performance and Health Services Research and Development Service Quality Enhancement Research Initiative of the Department of Veterans Affairs (RRP 12-192) and the National Heart, Lung, and Blood Institute, U.S. Department of Health and Human Services (1R01HL116522-01A1). Disclosures: Disclosures can be viewed at www.acponline.org/authors /icmje/ConflictOfInterestForms.do?msNum⫽M14-0361. Reproducible Research Statement: Study protocol and statistical code:

Available from Dr. Keyhani (e-mail, [email protected]). Data set: Not available. Requests for Single Reprints: Salomeh Keyhani, MD, MPH, San Fran-

cisco Veterans Affairs Medical Center, 4150 Clement (111A1), San Francisco, CA 94121; e-mail, [email protected]. Current author addresses and author contributions are available at www.annals.org. 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11 783

Original Research

Hospital Profiling for Stroke Readmission

References 1. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, et al; American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics—2013 update: a report from the American Heart Association. Circulation. 2013;127:e6-e245. [PMID: 23239837] doi:10.1161/CIR.0b013e31828124ad 2. Keyhani S, Arling G, Williams LS, Ross JS, Ordin DL, Myers J, et al. The use and misuse of thrombolytic therapy within the Veterans Health Administration. Med Care. 2012;50:66-73. [PMID: 22182924] doi:10.1097/MLR .0b013e3182294092 3. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306:1688-98. [PMID: 22009101] doi:10.1001/jama.2011.1515 4. Amarasingham R, Moore BJ, Tabak YP, Drazner MH, Clark CA, Zhang S, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48: 981-8. [PMID: 20940649] doi:10.1097/MLR.0b013e3181ef60d9 5. Lichtman JH, Leifheit-Limson EC, Jones SB, Watanabe E, Bernheim SM, Phipps MS, et al. Predictors of hospital readmission after stroke: a systematic review. Stroke. 2010;41:2525-33. [PMID: 20930150] doi:10.1161 /STROKEAHA.110.599159 6. Bernheim S, Wang C, Wang Y, Bhat K, Savage S, Lichtman J, et al; Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation. Hospital 30-Day Readmission Following Acute Ischemic Stroke Hospitalization Measure Methodology Report. Centers for Medicare & Medicaid Services; 2010. Contract no. HHSM-500-2008-00025I, task order 1. 7. National Quality Forum. Risk Adjustment for Socioeconomic Status or Other Sociodemographic Factors: Draft Report. Washington, DC: National Quality Forum; 2014. 8. Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation. Respecifying the Hospital 30-Day Ischemic Stroke Readmission Measure by Adding a Planned Readmission Algorithm. Centers for Medicare & Medicaid Services; 2012. Contract no. HHSM-500-2008-00025i/ HHSM-500-t0001, modification no. 000007. 9. Arling G, Reeves M, Ross J, Williams LS, Keyhani S, Chumbler N, et al. Estimating and reporting on the quality of inpatient stroke care by Veterans Health Administration Medical Centers. Circ Cardiovasc Qual Outcomes. 2012; 5:44-51. [PMID: 22147888] doi:10.1161/CIRCOUTCOMES.111.961474 10. Reker DM, Reid K, Duncan PW, Marshall C, Cowper D, Stansbury J, et al. Development of an integrated stroke outcomes database within Veterans Health Administration. J Rehabil Res Dev. 2005;42:77-91. [PMID: 15742252] 11. Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation. Testing Hospital-Level Acute Ischemic Stroke 30-Day Mortality & Readmission Measures in California All-Payer Data. Centers for Medicare & Medicaid Services; 2012. Contract no. HHSM-500-2008-00025I/ HHSM-500-T0001 MN.

784 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

12. Arbaje AI, Wolff JL, Yu Q, Powe NR, Anderson GF, Boult C. Postdischarge environmental and socioeconomic factors and the likelihood of early hospital readmission among community-dwelling Medicare beneficiaries. Gerontologist. 2008;48:495-504. [PMID: 18728299] 13. Williams LS, Yilmaz EY, Lopez-Yunez AM. Retrospective assessment of initial stroke severity with the NIH Stroke Scale. Stroke. 2000;31:858-62. [PMID: 10753988] 14. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100:1619-36. [PMID: 1959406] 15. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-83. [PMID: 3558716] 16. Morse JM. Enhancing the safety of hospitalization by reducing patient falls. Am J Infect Control. 2002;30:376-80. [PMID: 12360147] 17. O’Connell B, Myers H. The sensitivity and specificity of the Morse Fall Scale in an acute care setting. J Clin Nurs. 2002;11:134-6. [PMID: 11845750] 18. Morse JM, Black C, Oberle K, Donahue P. A prospective study to identify the fall-prone patient. Soc Sci Med. 1989;28:81-6. [PMID: 2928815] 19. U.S. Census Bureau. American Community Survey. 2014. Accessed at www .census.gov/acs/www on 10 October 2014. 20. Krumholz HM, Normand ST, Galusha DH, Mattera JA, Rich AS, Wang Y; Yale University. Risk-Adjustment Models for AMI and HF 30-Day Mortality. Centers for Medicare & Medicaid Services; 2011. Subcontract no. 8908-03-02. 21. U.S. Department of Health and Human Services. Hospital Compare. 2011. Accessed at www.medicare.gov/hospitalcompare/search.html on 10 October 2014. 22. Krumholz HM, Parent EM, Tu N, Vaccarino V, Wang Y, Radford MJ, et al. Readmission after hospitalization for congestive heart failure among Medicare beneficiaries. Arch Intern Med. 1997;157:99-104. [PMID: 8996046] 23. Austin PC. A comparison of Bayesian methods for profiling hospital performance. Med Decis Making. 2002;22:163-72. [PMID: 11958498] 24. Austin PC, Alter DA, Tu JV. The use of fixed- and random-effects models for classifying hospitals as mortality outliers: a Monte Carlo assessment. Med Decis Making. 2003;23:526-39. [PMID: 14672113] 25. Proposed provisions of the hospital readmission reduction program. Fed Regis. 2011;76:51476. 26. Joynt KE, Jha AK. Who has higher readmission rates for heart failure, and why? Implications for efforts to improve care using financial incentives. Circ Cardiovasc Qual Outcomes. 2011;4:53-9. [PMID: 21156879] doi:10.1161 /CIRCOUTCOMES.110.950964 27. Joynt KE, Orav EJ, Jha AK. Thirty-day readmission rates for Medicare beneficiaries by race and site of care. JAMA. 2011;305:675-81. [PMID: 21325183] doi:10.1001/jama.2011.123

www.annals.org

Annals of Internal Medicine Current Author Addresses: Dr. Keyhani: San Francisco Veterans Affairs

Medical Center, 4150 Clement (111A1), San Francisco, CA 94121. Drs. Myers, Williams, and Bravata: Richard L. Roudebush Veterans Affairs Medical Center, 1481 West 10th Street (11H), Indianapolis, IN 46202. Dr. Cheng: Department of Neurology, ML 127, Veterans Affairs Greater Los Angeles Healthcare System, 11301 Wilshire Boulevard, Los Angeles, CA 90073. Dr. Hebert: Veterans Affairs Puget Sound Health Care System, 1100 Olive Way, Suite 1400, Seattle, WA 98101. Author Contributions: Conception and design: S. Keyhani, P. Hebert,

D.M. Bravata. Analysis and interpretation of the data: S. Keyhani, L.J. Myers, E. Cheng, P. Hebert, L.S. Williams, D.M. Bravata. Drafting of the article: S. Keyhani. Critical revision of the article for important intellectual content: S. Keyhani, L.J. Myers, E. Cheng, L.S. Williams, D.M. Bravata. Final approval of the article: S. Keyhani, L.J. Myers, E. Cheng, P. Hebert, L.S. Williams, D.M. Bravata. Statistical expertise: S. Keyhani, L.J. Myers, P. Hebert. Obtaining of funding: S. Keyhani, L.S. Williams, D.M. Bravata. Administrative, technical, or logistic support: S. Keyhani. Collection and assembly of data: L.J. Myers, L.S. Williams.

APPENDIX: EFFECT OF CLINICAL AND SOCIAL RISK FACTORS ON HOSPITAL PROFILING FOR STROKE READMISSION: SAS DOCUMENTATION This appendix describes the details for calculating the 30day RSRRs. The Yale New Haven Health Services Corporation/ Center for Outcomes Research and Evaluation (YNHHSC/ CORE) provided much of the information and code contained here and in the SAS programs via their “SAS packs,” which included SAS code to identify comorbid conditions, categorize readmissions as planned or unplanned, and calculate the RSRRs. The SAS packs are publicly available by e-mailing the YNHHSC/ CORE at [email protected]. Cohort The initial stroke cohort was identified from the fiscal year 2007 Stroke Special Project data that, in brief, were a sample of 5000 veterans hospitalized in fiscal year 2007 with a primary diagnosis of stroke as identified in VA administrative data. Charts were reviewed to assess inclusion eligibility, resulting in 3965 patients being eligible for final chart abstraction. We then excluded the following patients from the readmission analysis, consistent with CMS criteria for the stroke readmission measures: those transferred from a VA facility to a non-VA facility and ultimately discharged from a non-VA facility (because the intent of the analysis was to examine outcomes for patients discharged with stroke from a VA facility), those transferred and whose final diagnosis was not stroke (because CMS criteria required that stroke be the final discharge diagnosis), those discharged against medical advice (because the opportunity to deliver full care was not available), those who died during the index admission (because they were not eligible for readmissions), those with CMS HMO coverage in the month before or after index admission or discharge (because 30-day outcome data may not have been availwww.annals.org

able in the CMS), and those not discharged to home or the community (because the readmission measure focused on patients discharged to these settings). After exclusions, the final analytic sample included 3426 veterans from 114 VA acute care facilities who were discharged with stroke. Outcome: 30-Day Readmissions The outcome for the analyses was 30-day unplanned readmissions. We used the CMS Planned Readmission Algorithm, version 3.0, for stroke, which employs flow charts to categorize a readmission as planned or unplanned on the basis of diagnoses or procedures. Consistent with the 2014 readmission measures, the first readmission after discharge was used in the outcome measure. Thus, if a planned readmission was followed by an unplanned readmission, the unplanned readmission was not counted in the outcome because those events may have been related to the care provided during the planned readmission. Therefore, only the first readmission after discharge was assessed. Patients with a planned readmission after being discharged were considered not to have had an event (that is, no readmission). We used the 2012 Agency for Healthcare Research and Quality Clinical Classification Software to group procedure and diagnosis codes for the Planned Readmission Algorithm. The crosswalk was provided by the YNHHSC/CORE in their Stroke Readmission SAS Pack and is publicly available by request. Data Sources The SAS programs used chart review data from the Stroke Special Project and preprocessed VA and CMS data for the analysis.

Data Sources Used to Identify Risk Covariates and 30-Day Readmissions

Fiscal Year 2007 Stroke Special Project Data. Data from the Stroke Special Project for fiscal year 2007 were collected through retrospective chart review of medical records. They were used to obtain demographic and clinical covariates. VA Medical SAS Inpatient Data Sets. The medical SAS inpatient data sets included the following 4 main types of care: acute, extended, observation, and non-VA. The acute, extended, and non-VA files contained 4 data sets: main, bed section, procedure, and surgery. The main file contained information for each VA hospitalization, the bed section file contained more detailed information for each VA hospitalization, and the procedure and surgery files contained information on procedures and surgeries done during an inpatient stay. The extended care files contained information for patients residing in community living centers and were not used in these analyses. The observation file contained 3 data sets: main, bed section, and procedure and had the same format as described. The observation files included patients with inpatient stays for monitoring, evaluation, or assessment and were for stays generally lasting fewer than 24 hours. The observation stays were included in the outcome measure if the stay resulted in a patient leaving against medical 2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

advice, dying, or being transferred. The non-VA files contained information on inpatient stays at non-VA facilities that were funded by the VA. VA Medical SAS Outpatient Event Data Sets. The outpatient event file contained information on outpatient encounters. These files were used for identifying risk covariates. VA Medical SAS Inpatient Data Sets. The VA medical SAS inpatient files contained information for patients who have had outpatient procedures and tests while hospitalized. These files were used to identify risk covariates. CMS Medical Provider Analysis and Review Data Set. The Medical Provider Analysis and Review data sets contained information on CMS beneficiaries with inpatient stays. The data were used to identify risk covariates and 30-day readmission events. CMS Part A (Inpatient) Data Set. The inpatient files contained information on inpatient stays and were used to identify risk covariates. CMS Hospital Outpatient Data Sets. Outpatient data sets contained information on Medicare claims for the facility component of the inpatient services. These data were used to identify risk covariates. CMS Part B (Outpatient) Data Sets. Part A (outpatient) data sets refer to Medicare claims for physician services and other outpatient care and services. They were used to identify risk covariates. CMS Medicare Enrollment Data Set. The Medicare enrollment data set contained information on CMS benefits and coverage and was used to identify HMO enrollment and coverage in the month before and after stroke admission. Non-VA Medical Care (Formerly Fee Basis) Data Sets. The Non-VA Medical Care (formerly Fee Basis) data sets contained information on inpatient and outpatient care provided at non-VA facilities that was funded by the VA. Non-VA care was authorized when the VA could not offer needed care, when a non-VA provider might have been more economical, or on an emergency basis when travel to a VA facility was not feasible. The short-term acute inpatient care file was used to identify readmissions. Index Data Set Appendix Table 1 identifies the variables used for analysis and the statistical models that each variable included. We will provide upon request detailed SAS code that shows how they were identified and constructed. We used the 2012 version of the ICD-9-CM to Condition Category crosswalk, which the YNHHSC/CORE provided in the SAS pack, to identify the 25 comorbid conditions used in model 1. SAS Macro for Risk Adjustment The SAS program “Keyhani-Analysis.sas” used the preprocessed cohort data and risk covariates (from the Index Data Set) to estimate hospital-level, 30-day, all-cause RSRRs by using hierarchical logistic regression models. The YNHHSC/CORE provided the SAS code to calculate the RSRRs in their SAS packs. In brief, data were modeled simultaneously at the patient and hospital level to account for variance in outcomes within and be2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

tween hospitals. At the patient level, the analysis modeled the log odds of readmission by using the specified covariates for the statistical model of interest and a random hospital-specific intercept. At the hospital level, the analysis modeled the intercepts—which represent the underlying risk for readmission after patient risk is accounted for—as arising from a normal distribution. The RSRR was then calculated as the number of predicted readmissions to the number of expected readmissions at each hospital, multiplied by the national observed readmission rate. The SAS macro created 4 permanent output SAS data sets for each of the 3 models in which X indicated models 1, 2, and 3. Final Study Sample

The data were at the patient level and included the patient identifier, risk covariates included in the analysis, and dichotomous readmission indicator (Stk_ModelX_radm_analysis). Parameter Estimates

The parameter estimates contained model coefficients, SEs, ORs, and 95% CIs (Stk_ModelX_Radm_Est). Hospital-Level RSRR

The hospital-level RSRR contained the hospital identifier; total number of patients (volume); observed, predicted, and expected readmission rate; risk-standardized readmission ratio; and RSRR. The data were at the hospital level and provided the point estimates of the readmission rate at each facility (Stk_ ModelX_Radm_RSRR). Bootstrap Results

The bootstrap results file was at the hospital-specific bootstrap iteration level. It included all of the variables in the hospital-level RSRR file plus an iteration variable that indicated the specific iteration during which a hospital was resampled. These data were used to classify hospital performance using the 95% CIs (Stk_ModelX_Radm_RSRR_BS). Classification of Facility Performance Per CMS methodology, facility performance was categorized using the facility’s RSRR and the corresponding 95% CI as obtained via the 1000 bootstrapping simulations. A facility’s performance was assigned by comparing the 95% CI with the national observed readmission rate. Facilities with at least 25 patients were categorized as “no different from the VA national rate” if the 95% CI overlapped the national VA observed readmission rate, “worse than the VA national rate” if the entire 95% CI was greater than the national VA observed readmission rate, and “better than the VA national rate” if the entire 95% CI was less than the national VA observed readmission rate. In our analyses, 65 of the 114 VA facilities discharged at least 25 patients with stroke. The other 49 facilities discharged fewer than 25 patients with stroke. Per CMS methodology, those 49 facilities were not included in the plots showing the distribution of RSRRs because the number of cases was too small to www.annals.org

reliably tell how well the hospitals were performing. However, data from patients at those 49 facilities were used to determine the national observed readmission rate and in calculations to report the overall VA 30-day RSRR. SAS Programs The “Keyhani-Analysis.sas” program called a macro included in the SAS pack from the YNHHSC/CORE (that is,

“Readmission_Macros_2013_3yr_SK.sas”). Those macros generated the parameter estimates, facility RSRRs, and bootstrapped simulations (used to calculate the 95% CIs). The program then output variables frequencies, ORs, and 95% CIs for the parameter estimates; calculated 95% CIs for the RSRRs on the basis of 1000 bootstrapped simulations; categorized facilities on the basis of RSRRs; and generated plots of the facility RSRRs.

Appendix Table 1. Variables Used for Analysis Variable Name Used in the Code

Description*

Values

Source of Data

Model 1

Model 2

Model 3

Age CMS01

Age at stroke admission CHF (80)

⻫

⻫

Hypertensive heart disease (90)

CMS03

Cerebral hemorrhage (95)

CMS04

Ischemic or unspecified stroke (96)

CMS05

Cerebrovascular disease (97)

CMS06 CMS07

Hemiplegia, paraplegia, paralysis, or functional disability (100–102) Vascular or circulatory disease (104–106)

CMS08

Metastatic cancer (7)

CMS09

Cancer (8–12)

CMS10

DM and related complications (15–20, 119, 120) Protein–calorie malnutrition (21)

Chart review VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data American Community Survey data‡

⻫ ⻫

CMS02

– 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 0%–100%†

⻫

⻫

CMS11 CMS12 CMS13 CMS14 CMS15 CMS16 CMS17

Fluid, electrolyte, acid, or base disorders (22, 23) Other endocrine, metabolic, or nutritional disorders (24) Severe hematologic disorders (40) Iron-deficiency and other/unspecified anemias and blood disease (47) Dementia and senility (49, 50)

CMS18

Quadriplegia, paraplegia, and functional disability (67–69, 177, 178) Seizure disorders and convulsions (74)

CMS19

COPD (108)

CMS20

Other lung disorders (115)

CMS21

ESRD (130)

CMS22

Renal failure (131)

CMS23

Other urinary tract disorders (136)

CMS24 CMS25

Decubitus or chronic skin ulcer (148, 149) Major symptoms or abnormalities (166)

Percent_Below_ Poverty

Percentage of persons within a ZIP code who are below the poverty threshold

⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫ ⻫

Continued on following page

www.annals.org

2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

Appendix Table 1—Continued Variable Name Used in the Code

Description*

Values

Source of Data

Low_Income

Low income

Substance

Substance abuse disorder in 1 y before admission Homeless in 1 y before admission

1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Yes 0 ⫽ No 1 ⫽ Ambulatory 0 ⫽ Nonambulatory 1 ⫽ Yes 0 ⫽ No 0–2 3–9 ⱖ10 Low ⫽ 0–50 High ⫽ ⬎50 Unknown 1 ⫽ Yes 0 ⫽ No –

Homeless MH Social_Work

Mental health disorder in 1 y before admission Social work visit in 1 y before admission

STD

STI in 2 y before admission

Hypoxia

NIHSS_CAT

Presence of hypoxia in the first 4 d of admission§ Ambulatory status before stroke admission Notes indicate that dysphagia was on the problem list Retrospective NIHSS score

MORSE_CAT

Morse Fall Scale score at admission

DNRDNI

DNR or DNI status on admission

Charlson

Charlson Comorbidity Index score based on history Modified APACHE-III score HBPC eligibility and utilization¶

Prestamb㛳 Dysphagia

APACHE_Score HBPC

Num_Admits Num_Outpt_Visits SC_NoCopay

Number of acute stay hospitalizations in 1 y before admission Number of VA ambulatory care visits in 1 y before admission Indicator to identify whether VA copayment for services was waived because of military service–connected disability

– 1 ⫽ Eligible and using HBPC 2 ⫽ Eligible and not using HBPC 3 ⫽ Not eligible for HBPC 0, ⱖ1 0–high† 1 ⫽ Yes 0 ⫽ No

Model 1

Model 2

Model 3

VA administrative data

⻫

⻫

VA and CMS administrative data VA and CMS administrative data VA and CMS administrative data VA administrative data

⻫

⻫

⻫

⻫

⻫

⻫

⻫

⻫

VA and CMS administrative data Chart review

⻫

⻫ ⻫

Chart review

⻫

Chart review

⻫

Chart review

⻫

Chart review

⻫

Chart review

⻫

Chart review

⻫

Chart review VA administrative data

⻫ ⻫

VA and CMS administrative data VA administrative data

⻫

VA administrative data

⻫

⻫

APACHE ⫽ Acute Physiology and Chronic Health Evaluation; CHF ⫽ congestive heart failure; CMS ⫽ Centers for Medicare & Medicaid Services; COPD ⫽ chronic obstructive pulmonary disease; DM ⫽ diabetes mellitus; DNI ⫽ do not intubate; DNR ⫽ do not resuscitate; ESRD ⫽ end-stage renal disease; HBPC ⫽ home-based primary care; NIHSS ⫽ National Institutes of Health Stroke Scale; STI ⫽ sexually transmitted infection; VA ⫽ Veterans Health Administration. * Data in parentheses indicate International Classification of Diseases, Ninth Revision, Clinical Modification, to Condition Category crosswalk. † Continuous. ‡ From the U.S. Census Bureau. § Hypoxia defined by O2 saturation ⬍90% and/or Pao2 ⬍60 mm Hg. 㛳 Patients with an unknown ambulatory status before stroke were classified as ambulatory in the analysis. ¶ Eligibility is based on a distance of ⬍60 mi from a patient’s home to the nearest VA medical center; utilization is based on VA ambulatory clinic codes for HBPC in the 1 y before admission.

2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

www.annals.org

Appendix Table 2. Hospital-Level Distribution of Demographic and Clinical Characteristics Across Hospitals* Variable

Median (Range)

Age, y Men, % CHF, % Hypertensive heart disease, % Cerebral hemorrhage, % Ischemic or unspecified stroke, % Cerebrovascular disease, % Hemiplegia, paraplegia, paralysis, or functional disability, % Vascular or circulatory disease, % Metastatic cancer or acute leukemia, % Cancer, % DM and related complications, % Protein–calorie malnutrition, % Fluid, electrolyte, acid, or base disorders, % Obesity/disorders of the thyroid or cholesterol or lipid levels, % Severe hematologic disorders, % Iron-deficiency and other/unspecified anemias and blood disease, % Dementia and senility, % Quadriplegia, paraplegia, or functional disability, % Seizure disorders and convulsions, % COPD, % Other lung disorders, % ESRD or dialysis, % Renal failure, % Other urinary tract disorders, % Decubitus or chronic skin ulcer, % Major symptoms or abnormalities, % Below the poverty threshold, %† Low income, % Substance abuse, % Homeless, % Mental health disorder, % Social work visit in prior year, % STI, % APACHE score Charlson Comorbidity Index score NIHSS score Hypoxia, % Dysphagia, % DNR/DNI, % Disability‡ Morse Fall Scale score, % 0–50 (low) ⱖ51 (high) Unknown Nonambulatory, % HBPC in prior year, % Eligible and using HBPC Eligible and not using HBPC Not eligible for HBPC ⱖ1 hospitalization, n Hospital visits, n

67.1 (55.0–86.0) 100.0 (86.4–100.0) 15.5 (0.0–100.0) 0.0 (0.0–11.8) 0.0 (0.0–12.5) 26.9 (0.0–100.0) 15.4 (0.0–100.0) 9.4 (0.0–100.0) 25.0 (0.0–100.0) 0.0 (0.0–11.1) 17.5 (0.0–66.7) 42.0 (0.0–100.0) 0.0 (0.0–25.0) 15.0 (0.0–100.0) 62.3 (0.0–100.0) 0.0 (0.0–8.3) 18.1 (0.0–100.0) 9.9 (0.0–37.5) 2.4 (0.0–12.1) 3.7 (0.0–20.0) 19.3 (0.0–100.0) 10.4 (0.0–100.0) 0.0 (0.0–11.1) 13.7 (0.0–100.0) 11.1 (0.0–100.0) 3.8 (0.0–100.0) 44.6 (0.0–100.0) 18.2 (5.7–43.4) 47.7 (0.0–100.0) 27.5 (0.0–100.0) 5.1 (0.0–50.0) 40.0 (0.0–100.0) 21.7 (0.0–100.0) 4.7 (0.0–22.2) 12.4 (0.0–29.0) 1.5 (0.0–9.0) 3.9 (0.0–23.0) 0 (0–20) 7.6 (0.0–100.0) 7.0 (0.0–75.0) 34.8 (0.0–100.0) 44.6 (0.0–100.0) 35.8 (0.0–100.0) 7.4 (0.0–100.0) 3.4 (0.0–100.0) 0.0 (0.0–16.7) 88.7 (42.9–100.0) 8.1 (0.0–57.1) 29.4 (0.0–100.0) 17.8 (0.0–37.2)

APACHE ⫽ Acute Physiology and Chronic Health Evaluation; CHF ⫽ congestive heart failure; COPD ⫽ chronic obstructive pulmonary disease; DM ⫽ diabetes mellitus; DNI ⫽ do not intubate; DNR ⫽ do not resuscitate; ESRD ⫽ end-stage renal disease; HBPC ⫽ home-based primary care; NIHSS ⫽ National Institutes of Health Stroke Scale; STI ⫽ sexually transmitted infection. * Hospital sample comprised 114 hospitals and 3436 patients. † According to the U.S. Census Bureau. ‡ Based on priority score.

www.annals.org

2 December 2014 Annals of Internal Medicine Volume 161 • Number 11

Copyright © American College of Physicians 2014.