Research
Original Investigation
Risk of Discharge to Postacute Care A Patient-Centered Outcome for the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator Sanjay Mohanty, MD; Yaoming Liu, PhD; Jennifer L. Paruch, MD; Thomas E. Kmiecik, PhD; Mark E. Cohen, PhD; Clifford Y. Ko, MD, MS, MSHS; Karl Y. Bilimoria, MD, MS
IMPORTANCE Individualized risk prediction tools have an important role as decision aids for
use by patients and surgeons before surgery. Patient-centered outcomes should be incorporated into such tools to widen their appeal and improve their usability. OBJECTIVE To develop a patient-centered outcome for the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator, a web-based, individualized risk prediction tool. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study using data from the ACS NSQIP, a national clinical data registry. A total of 973 211 patients from July 2010 to June 2012, encompassing 392 hospitals, were used in this analysis. MAIN OUTCOMES AND MEASURES Risk of discharge to a postacute care setting. RESULTS The overall rate of discharge to postacute care was 8.8%. Significant predictors of discharge to postacute care included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.20-2.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). The final model predicted risk of discharge to postacute care with excellent accuracy (C statistic = 0.924) and calibration (Brier score = 0.05). CONCLUSIONS AND RELEVANCE Individualized risk of discharge to postacute care can be predicted with excellent accuracy. This outcome will be incorporated into the ACS NSQIP Surgical Risk Calculator. JAMA Surg. 2015;150(5):480-484. doi:10.1001/jamasurg.2014.3176 Published online March 25, 2015.
I
ndividualized surgical risk prediction tools have become an important opportunity for shared decision making in surgical patients. These are tools that allow input of the procedure type as well as individual patient factors, such as demographic characteristics and comorbidities, after which an underlying prediction algorithm outputs predicted risk for a number of postoperative outcomes.1 Currently, most of these tools offer traditional outcomes such as postoperative cardiac events, urinary tract infections, or surgical site infections. Although understanding the risk of these outcomes is important for decision making, there has been recent focus on complementary outcomes that represent the concerns and preferences of patients and are perhaps easier to conceptualize. These patient-centered outcomes 480
Author Affiliations: Department of Surgery, Henry Ford Hospital, Detroit, Michigan (Mohanty); Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois (Mohanty, Liu, Paruch, Cohen, Ko, Bilimoria); Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois (Kmiecik, Bilimoria); Department of Surgery, University of California, Los Angeles (Ko); VA Greater Los Angeles Healthcare System, Los Angeles, California (Ko). Corresponding Author: Sanjay Mohanty, MD, Division of Research and Optimal Patient Care, American College of Surgeons, 633 N St Clair, 22nd Floor, Chicago, IL 60611 ([email protected]).
allow more informed decision making, particularly when it comes to choosing one of several treatment or intervention options. Their incorporation into prediction tools is essential for widening their appeal and use by surgeons and patients. Discharge to postacute care settings such as acute rehabilitation facilities is an example of one such outcome. It is patient centered, reflecting in part a functional outcome following surgery, and predicting patients at high risk for such a discharge can be important for discharge planning, health system resource allocation, and cost containment. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator currently allows for entry of 21 preoperative patient factors in ad-
JAMA Surgery May 2015 Volume 150, Number 5 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a Tulane University User on 04/11/2016
jamasurgery.com
Risk of Discharge to Postacute Care
Original Investigation Research
dition to the type of procedure and uses regression models to output the individualized risk of 8 surgical outcomes. This universal risk prediction tool leverages a large amount of clinical data and has been shown to provide predictions that are nearly as accurate as those from specialized calculators.1 Since 2011, patient discharge destination has been collected from all ACS NSQIP hospitals. This encompasses both home and postacute care destinations such as skilled nursing facilities, acute rehabilitation facilities, and unskilled nursing homes. We sought to use these data to develop a new outcome for the Surgical Risk Calculator: risk of discharge to a postacute care facility.
Table 1. Rates of Postacute Care Discharge Overall and by Specialty Specialty Overall (n = 973 211) General surgery (n = 515 404)
24.5
Vascular surgery (n = 84 377)
16.3
Obstetrics/gynecology (n = 65 077) Neurosurgery (n = 56 566) Urology (n = 48 376)
Data were obtained from ACS NSQIP Semiannual Report data sets from July 2010 to June 2012. This program, including its history, purpose, and sampling method, has been detailed extensively in the literature.2-4 Briefly, ACS NSQIP is a nationally validated, risk-adjusted, outcomes-based program designed to measure and improve the quality of surgical care. It has collected data from more than 500 hospitals in a standardized, reliable fashion on a variety of patient factors, including demographic characteristics and comorbidities, operations, and postoperative outcomes. These data are collected by trained and audited surgical clinical reviewers at each participating hospital. Following the index operation, patients are followed up for 30-day postoperative outcomes. In this study, all patients undergoing operations between July 2010 and June 2012 were included. Because this study did not meet the definition of human research, institutional review board approval and informed consent were not required.
1.1 14.6 2.8
Plastic surgery (n = 24 975)
3.0
Otolaryngology (n = 17 420)
1.7
Cardiac surgery (n = 6862)
Data Source and Patients
4.6
Orthopedic surgery (n = 141 409)
Thoracic surgery (n = 12 946)
Methods
Postacute Care Discharge Rate, % 8.8
7.8 20.3
fixed-slope hierarchical models. This accounts for clustering of cases in hospitals and uses an empirical Bayes-type shrinkage adjustment.5 Twenty-one patient factors as well as the operation CPT linear risk score and a work relative value units variable were used for risk prediction.5,6 The final model was validated on the remaining one-third of the cohort. Model performance was evaluated using the C statistic and the Brier score. The C statistic, or area under the receiver operating characteristic curve, is a measure of discrimination, with 0.5 corresponding to discrimination that is no better than chance and 1.0 corresponding to perfect prediction. Finally, the Brier score is both a measure of discrimination and calibration and is defined as the means squared difference between predicted and actual outcomes, with 0.0 indicating perfect model performance.5 After model validation, a final model was developed using the entire cohort for incorporation into the risk calculator. All data manipulation and statistical analysis were performed using SAS version 9.4 statistical software (SAS Institute, Inc).
Preoperative Risk Factors and Outcome Twenty-one preoperative risk factors were used in predicting patient-specific risk of postacute care discharge. These preoperative factors were selected based on predictive value, availability, and face validity. All preoperative factors were made categorical before inclusion into regression models. Risk of postacute care discharge coming purely from the procedure was determined from preliminary models in which Current Procedural Terminology (CPT) code was used as a random effect in a hierarchical model. This generated a continuous variable—a logit-transformed predicted probability of the outcome— unique for each procedure. An additional variable for work relative value units was included to account for procedure complexity.1 The outcome of interest was discharge to postacute care. All nonhome destinations, which included skilled nursing facilities, unskilled nursing facilities, and all acute rehabilitation facilities, were grouped together. Patients who died were excluded, and those who were discharged from the hospital to facilities at which they were living prior to their operation were grouped with patients discharged home.
Statistical Analysis For this study, two-thirds of the total cohort, randomly sampled without replacement, was used to develop random-intercept,
Results There were 973 211 cases from 392 hospitals in the final analysis. These encompassed all surgical specialties except transplant and trauma. The overall event rate of discharge to postacute care was 8.8%. This varied by specialty, with the highest rates in patients undergoing orthopedic (24.5%), cardiac (20.3%), and vascular (16.3%) procedures (Table 1). In aggregate, patients discharged home were younger and healthier, with fewer comorbidities and better functional status. The derivation (n = 643 211) and validation (n = 330 000) cohorts were not significantly different in terms of their demographic characteristics or comorbidities (Table 2). The most significant predictors associated with continuing care discharge, adjusted for procedure type and complexity, included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.202.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). All predictors are shown in Table 3. In addition to these patient factors, the final model included emergent case status, the presence of dyspnea, renal fail-
jamasurgery.com
(Reprinted) JAMA Surgery May 2015 Volume 150, Number 5
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a Tulane University User on 04/11/2016
481
Research Original Investigation
Risk of Discharge to Postacute Care
Table 2. Selected Characteristics of Derivation and Validation Cohorts
Patient Factor
% Characteristic
Validation Cohort (n = 330 000)
P Value
Age >65 y
34.7
34.3
.13
Female
57.3
57.4
.47
Functional status 96.8
96.7
2.7
2.7
Totally
0.6
0.6
1 or 2
55.4
55.4
3
38.7
38.6
5.9
6.0
5.4
5.5
Original Investigation
Risk of Discharge to Postacute Care A Patient-Centered Outcome for the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator Sanjay Mohanty, MD; Yaoming Liu, PhD; Jennifer L. Paruch, MD; Thomas E. Kmiecik, PhD; Mark E. Cohen, PhD; Clifford Y. Ko, MD, MS, MSHS; Karl Y. Bilimoria, MD, MS
IMPORTANCE Individualized risk prediction tools have an important role as decision aids for
use by patients and surgeons before surgery. Patient-centered outcomes should be incorporated into such tools to widen their appeal and improve their usability. OBJECTIVE To develop a patient-centered outcome for the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator, a web-based, individualized risk prediction tool. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study using data from the ACS NSQIP, a national clinical data registry. A total of 973 211 patients from July 2010 to June 2012, encompassing 392 hospitals, were used in this analysis. MAIN OUTCOMES AND MEASURES Risk of discharge to a postacute care setting. RESULTS The overall rate of discharge to postacute care was 8.8%. Significant predictors of discharge to postacute care included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.20-2.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). The final model predicted risk of discharge to postacute care with excellent accuracy (C statistic = 0.924) and calibration (Brier score = 0.05). CONCLUSIONS AND RELEVANCE Individualized risk of discharge to postacute care can be predicted with excellent accuracy. This outcome will be incorporated into the ACS NSQIP Surgical Risk Calculator. JAMA Surg. 2015;150(5):480-484. doi:10.1001/jamasurg.2014.3176 Published online March 25, 2015.
I
ndividualized surgical risk prediction tools have become an important opportunity for shared decision making in surgical patients. These are tools that allow input of the procedure type as well as individual patient factors, such as demographic characteristics and comorbidities, after which an underlying prediction algorithm outputs predicted risk for a number of postoperative outcomes.1 Currently, most of these tools offer traditional outcomes such as postoperative cardiac events, urinary tract infections, or surgical site infections. Although understanding the risk of these outcomes is important for decision making, there has been recent focus on complementary outcomes that represent the concerns and preferences of patients and are perhaps easier to conceptualize. These patient-centered outcomes 480
Author Affiliations: Department of Surgery, Henry Ford Hospital, Detroit, Michigan (Mohanty); Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois (Mohanty, Liu, Paruch, Cohen, Ko, Bilimoria); Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois (Kmiecik, Bilimoria); Department of Surgery, University of California, Los Angeles (Ko); VA Greater Los Angeles Healthcare System, Los Angeles, California (Ko). Corresponding Author: Sanjay Mohanty, MD, Division of Research and Optimal Patient Care, American College of Surgeons, 633 N St Clair, 22nd Floor, Chicago, IL 60611 ([email protected]).
allow more informed decision making, particularly when it comes to choosing one of several treatment or intervention options. Their incorporation into prediction tools is essential for widening their appeal and use by surgeons and patients. Discharge to postacute care settings such as acute rehabilitation facilities is an example of one such outcome. It is patient centered, reflecting in part a functional outcome following surgery, and predicting patients at high risk for such a discharge can be important for discharge planning, health system resource allocation, and cost containment. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator currently allows for entry of 21 preoperative patient factors in ad-
JAMA Surgery May 2015 Volume 150, Number 5 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a Tulane University User on 04/11/2016
jamasurgery.com
Risk of Discharge to Postacute Care
Original Investigation Research
dition to the type of procedure and uses regression models to output the individualized risk of 8 surgical outcomes. This universal risk prediction tool leverages a large amount of clinical data and has been shown to provide predictions that are nearly as accurate as those from specialized calculators.1 Since 2011, patient discharge destination has been collected from all ACS NSQIP hospitals. This encompasses both home and postacute care destinations such as skilled nursing facilities, acute rehabilitation facilities, and unskilled nursing homes. We sought to use these data to develop a new outcome for the Surgical Risk Calculator: risk of discharge to a postacute care facility.
Table 1. Rates of Postacute Care Discharge Overall and by Specialty Specialty Overall (n = 973 211) General surgery (n = 515 404)
24.5
Vascular surgery (n = 84 377)
16.3
Obstetrics/gynecology (n = 65 077) Neurosurgery (n = 56 566) Urology (n = 48 376)
Data were obtained from ACS NSQIP Semiannual Report data sets from July 2010 to June 2012. This program, including its history, purpose, and sampling method, has been detailed extensively in the literature.2-4 Briefly, ACS NSQIP is a nationally validated, risk-adjusted, outcomes-based program designed to measure and improve the quality of surgical care. It has collected data from more than 500 hospitals in a standardized, reliable fashion on a variety of patient factors, including demographic characteristics and comorbidities, operations, and postoperative outcomes. These data are collected by trained and audited surgical clinical reviewers at each participating hospital. Following the index operation, patients are followed up for 30-day postoperative outcomes. In this study, all patients undergoing operations between July 2010 and June 2012 were included. Because this study did not meet the definition of human research, institutional review board approval and informed consent were not required.
1.1 14.6 2.8
Plastic surgery (n = 24 975)
3.0
Otolaryngology (n = 17 420)
1.7
Cardiac surgery (n = 6862)
Data Source and Patients
4.6
Orthopedic surgery (n = 141 409)
Thoracic surgery (n = 12 946)
Methods
Postacute Care Discharge Rate, % 8.8
7.8 20.3
fixed-slope hierarchical models. This accounts for clustering of cases in hospitals and uses an empirical Bayes-type shrinkage adjustment.5 Twenty-one patient factors as well as the operation CPT linear risk score and a work relative value units variable were used for risk prediction.5,6 The final model was validated on the remaining one-third of the cohort. Model performance was evaluated using the C statistic and the Brier score. The C statistic, or area under the receiver operating characteristic curve, is a measure of discrimination, with 0.5 corresponding to discrimination that is no better than chance and 1.0 corresponding to perfect prediction. Finally, the Brier score is both a measure of discrimination and calibration and is defined as the means squared difference between predicted and actual outcomes, with 0.0 indicating perfect model performance.5 After model validation, a final model was developed using the entire cohort for incorporation into the risk calculator. All data manipulation and statistical analysis were performed using SAS version 9.4 statistical software (SAS Institute, Inc).
Preoperative Risk Factors and Outcome Twenty-one preoperative risk factors were used in predicting patient-specific risk of postacute care discharge. These preoperative factors were selected based on predictive value, availability, and face validity. All preoperative factors were made categorical before inclusion into regression models. Risk of postacute care discharge coming purely from the procedure was determined from preliminary models in which Current Procedural Terminology (CPT) code was used as a random effect in a hierarchical model. This generated a continuous variable—a logit-transformed predicted probability of the outcome— unique for each procedure. An additional variable for work relative value units was included to account for procedure complexity.1 The outcome of interest was discharge to postacute care. All nonhome destinations, which included skilled nursing facilities, unskilled nursing facilities, and all acute rehabilitation facilities, were grouped together. Patients who died were excluded, and those who were discharged from the hospital to facilities at which they were living prior to their operation were grouped with patients discharged home.
Statistical Analysis For this study, two-thirds of the total cohort, randomly sampled without replacement, was used to develop random-intercept,
Results There were 973 211 cases from 392 hospitals in the final analysis. These encompassed all surgical specialties except transplant and trauma. The overall event rate of discharge to postacute care was 8.8%. This varied by specialty, with the highest rates in patients undergoing orthopedic (24.5%), cardiac (20.3%), and vascular (16.3%) procedures (Table 1). In aggregate, patients discharged home were younger and healthier, with fewer comorbidities and better functional status. The derivation (n = 643 211) and validation (n = 330 000) cohorts were not significantly different in terms of their demographic characteristics or comorbidities (Table 2). The most significant predictors associated with continuing care discharge, adjusted for procedure type and complexity, included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.202.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). All predictors are shown in Table 3. In addition to these patient factors, the final model included emergent case status, the presence of dyspnea, renal fail-
jamasurgery.com
(Reprinted) JAMA Surgery May 2015 Volume 150, Number 5
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a Tulane University User on 04/11/2016
481
Research Original Investigation
Risk of Discharge to Postacute Care
Table 2. Selected Characteristics of Derivation and Validation Cohorts
Patient Factor
% Characteristic
Validation Cohort (n = 330 000)
P Value
Age >65 y
34.7
34.3
.13
Female
57.3
57.4
.47
Functional status 96.8
96.7
2.7
2.7
Totally
0.6
0.6
1 or 2
55.4
55.4
3
38.7
38.6
5.9
6.0
5.4
5.5
