Letters
Comparing the National Surgical Quality Improvement Program With the Nationwide Inpatient Sample Database
sis was performed using STATA 64-bit Special Edition, version 11.2 (StataCorp). Our study was exempt from review as designated by the University of California, San Diego Human Research Protections Program.
Both raw and risk-adjusted outcomes are increasingly being made publicly available.1-3 The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) is heralded as the most robust database to examine surgical outcomes. However, enrollment in the NSQIP is expensive, and the use of administrative databases may be more cost-effective.2-4 In our study, we compare the receiver operating characteristic curves of the Nationwide Inpatient Sample (NIS) with those of the NSQIP to determine which is superior at performing analyses of risk-adjusted outcomes for several operations.
Results | There were 242 584 patients in the NIS and 73 130 patients in the NSQIP. Unadjusted complication rates were higher in the NIS than in the NSQIP for 7 surgical procedures. Mortality rates were higher for most procedures in the NIS; they were similar for appendectomy, laparoscopic cholecystectomy, and coronary artery bypass graft (Table 1). The C statistic was much higher in most logistic regressions for both mortality and complications in the NSQIP database (Table 2).
Methods | Our study uses 2010 data from both the NIS and the NSQIP. Inpatients older than 18 years of age were included. Patients were identified by International Classification of Diseases, Ninth Revision codes (NIS) and Current Procedural Terminology codes (NSQIP): abdominal aortic aneurysm repair, appendectomy, aortic valve replacement, coronary artery bypass graft, carotid endarterectomy, laparoscopic cholecystectomy, total and partial colectomy, esophagectomy, sleeve gastrectomy, pancreatectomy, and ventral hernia repair. Outcomes included inpatient death and complications. Patients were classified as having a complication if they had one of the following: infection (surgical site, deep incisional, and organ/space), wound disruption, pneumonia, pulmonary embolism, acute renal failure, urinary tract infection, cerebrovascular accident, myocardial infarction, and blood loss requiring transfusion. We performed multivariate logistic regressions predicting inpatient mortality and complication by procedure. In the NSQIP, the models were adjusted for preoperative risk variables in the database. In the NIS, the models were adjusted for age, race, sex, insurance status, and Charlson comorbidity index. The area under the receiver operating characteristic curve (C statistic) was calculated for each model. Statistical analy-
Discussion | Our study shows that the NSQIP is superior to the NIS administrative database as represented by higher C statistic values. Our study also finds that both mortality and complication rates were higher in the NIS than in the NSQIP. It is possible that hospitals participating in the NSQIP have lower mortality rates because they systematically examine their surgical outcomes. However, an alternative explanation is that hospitals in the NSQIP underreport their complications. Nurse abstractors are able to reason and exclude complication rates or mortality rates that are not directly related to a procedure. This is not true of the NIS. As the landscape for postoperative complication reimbursement changes, it will be prudent to repeat our study in several years and note if complication rates decrease. Our study is limited by differences in coding. The NIS relies on automated data extraction from discharge diagnoses, whereas the NSQIP relies on trained nurses to manually extract information. Variations exist between International Classification of Diseases, Ninth Revision and Current Procedural Terminology codes. Also, each database was developed for different purposes and may not correlate.3 The NIS was developed for reimbursement purposes, whereas the NSQIP was
PACIFIC COAST SURGICAL ASSOCIATION
Table 1. Comparison of Unadjusted Outcomes Patients, No. (%) All Patients, No.
Complication(s)a
Procedure
NIS
NIS
Abdominal aortic aneurysm repair
11 304
3651
Appendectomy
45 824
15 649
Aortic valve replacement
13 220
Coronary artery bypass graft Carotid endarterectomy
NSQIP
Death P Value
NSQIP
NIS
NSQIP
P Value
1763 (15.6)
508 (13.9)
.03
494 (4.4)
74 (2.0)
Comparing the National Surgical Quality Improvement Program With the Nationwide Inpatient Sample Database
sis was performed using STATA 64-bit Special Edition, version 11.2 (StataCorp). Our study was exempt from review as designated by the University of California, San Diego Human Research Protections Program.
Both raw and risk-adjusted outcomes are increasingly being made publicly available.1-3 The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) is heralded as the most robust database to examine surgical outcomes. However, enrollment in the NSQIP is expensive, and the use of administrative databases may be more cost-effective.2-4 In our study, we compare the receiver operating characteristic curves of the Nationwide Inpatient Sample (NIS) with those of the NSQIP to determine which is superior at performing analyses of risk-adjusted outcomes for several operations.
Results | There were 242 584 patients in the NIS and 73 130 patients in the NSQIP. Unadjusted complication rates were higher in the NIS than in the NSQIP for 7 surgical procedures. Mortality rates were higher for most procedures in the NIS; they were similar for appendectomy, laparoscopic cholecystectomy, and coronary artery bypass graft (Table 1). The C statistic was much higher in most logistic regressions for both mortality and complications in the NSQIP database (Table 2).
Methods | Our study uses 2010 data from both the NIS and the NSQIP. Inpatients older than 18 years of age were included. Patients were identified by International Classification of Diseases, Ninth Revision codes (NIS) and Current Procedural Terminology codes (NSQIP): abdominal aortic aneurysm repair, appendectomy, aortic valve replacement, coronary artery bypass graft, carotid endarterectomy, laparoscopic cholecystectomy, total and partial colectomy, esophagectomy, sleeve gastrectomy, pancreatectomy, and ventral hernia repair. Outcomes included inpatient death and complications. Patients were classified as having a complication if they had one of the following: infection (surgical site, deep incisional, and organ/space), wound disruption, pneumonia, pulmonary embolism, acute renal failure, urinary tract infection, cerebrovascular accident, myocardial infarction, and blood loss requiring transfusion. We performed multivariate logistic regressions predicting inpatient mortality and complication by procedure. In the NSQIP, the models were adjusted for preoperative risk variables in the database. In the NIS, the models were adjusted for age, race, sex, insurance status, and Charlson comorbidity index. The area under the receiver operating characteristic curve (C statistic) was calculated for each model. Statistical analy-
Discussion | Our study shows that the NSQIP is superior to the NIS administrative database as represented by higher C statistic values. Our study also finds that both mortality and complication rates were higher in the NIS than in the NSQIP. It is possible that hospitals participating in the NSQIP have lower mortality rates because they systematically examine their surgical outcomes. However, an alternative explanation is that hospitals in the NSQIP underreport their complications. Nurse abstractors are able to reason and exclude complication rates or mortality rates that are not directly related to a procedure. This is not true of the NIS. As the landscape for postoperative complication reimbursement changes, it will be prudent to repeat our study in several years and note if complication rates decrease. Our study is limited by differences in coding. The NIS relies on automated data extraction from discharge diagnoses, whereas the NSQIP relies on trained nurses to manually extract information. Variations exist between International Classification of Diseases, Ninth Revision and Current Procedural Terminology codes. Also, each database was developed for different purposes and may not correlate.3 The NIS was developed for reimbursement purposes, whereas the NSQIP was
PACIFIC COAST SURGICAL ASSOCIATION
Table 1. Comparison of Unadjusted Outcomes Patients, No. (%) All Patients, No.
Complication(s)a
Procedure
NIS
NIS
Abdominal aortic aneurysm repair
11 304
3651
Appendectomy
45 824
15 649
Aortic valve replacement
13 220
Coronary artery bypass graft Carotid endarterectomy
NSQIP
Death P Value
NSQIP
NIS
NSQIP
P Value
1763 (15.6)
508 (13.9)
.03
494 (4.4)
74 (2.0)
