Letters
Table. Responses to Survey by OR Physicians and Registered Nurses % of Respondents Who Strongly Agree or Agree Statement Use of the AWS timeout process improves surgical team engagement compared with the traditional timeout process. Use of the AWS improves patient safety compared with the traditional timeout process. Use of the AWS improves compliance with SCIP documentation compared with traditional methods. Use of the AWS improves the opportunities to prevent OR errors /near misses compared with traditional communication methods. Use of the AWS improves the operative team’s communication throughout the surgical procedure compared with traditional communication methods. The AWS is user friendly.
Total 86.96
Registered Nurse 92.86
Physician 73.68
84.78
92.86
73.68
91.11
92.86
84.21
84.44
92.86
66.67
78.26
85.71
63.16
84.44
100.00
78.95
I feel competent using the AWS.
81.82
100.00
72.22
The postoperative debriefing checklist in the AWS ensures that a consistent verification process takes place prior to the patient exiting the OR.a I have received feedback on my compliance with the AWS checklist.b The preprocedure checklist helps improve the preprocedure safety verification process.b
88.00
100.00
72.73
100.00
100.00
NA
92.86
92.86
NA
curve and technical problems with implementation early in use. Surveys show that physicians and staff members believe that the AWS improves communication, team engagement, patient safety, and compliance with Surgical Care Improvement Project measures. Future goals for the AWS at our facility include tracking near misses to improve patient safety and staff education opportunities, developing a structured feedback system for operative team members, and incorporating other risk assessments. Juliet Nissan, MD Valkiria Campos, RN, BSN, CPAN Hector Delgado, RN, MSN, MHA Christina Matadial, MD Seth Spector, MD Author Affiliations: Bruce W. Carter Department of VA Medical Center, Miami VA Healthcare System, University of Miami Miller School of Medicine, Jackson Memorial Hospital, Miami, Florida. Corresponding Author: Juliet Nissan, MD, Bruce W. Carter Department of VA Medical Center, Miami VA Healthcare System, University of Miami Miller School of Medicine, Jackson Memorial Hospital, 1611 NW 12th Ave, East Tower, Ste 2169, Miami, FL 33136 ([email protected]). Published Online: September 24, 2014. doi:10.1001/jamasurg.2014.1825. Author Contributions: Dr Nissan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: All authors. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Nissan, Campos, Spector. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Nissan, Campos, Delgado, Spector. Administrative, technical, or material support: Matadial, Spector. Study supervision: Matadial, Spector. Conflict of Interest Disclosures: None reported. Previous Presentation: The paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 7, 2014; New Haven, Connecticut. Additional Information: LiveData, the company that manufactured the AWS used at our institution (OR-Dashboard), had no role in the design and conduct of the study; analysis and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. 1210
Abbreviations: AWS, automated workflow system; NA, not applicable; OR, operating room; SCIP, Surgical Care Improvement Project. a
Only physicians and registered nurses responded.
b
Only registered nurses responded.
1. Gawande A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books/Henry Holt and Co; 2011. 2. World Alliance for Patient Safety; World Health Organization. Safe surgery saves lives: second global patient safety challenge. http://www.who.int/patientsafety /safesurgery/knowledge_base/SSSL_Brochure_finalJun08.pdf. Accessed March 24, 2014. 3. World Health Organization. Surgical safety checklist (first edition). http://www .who.int/patientsafety/safesurgery/tools_resources/SSSL_Checklist_finalJun08 .pdf. Accessed March 24, 2014. 4. Haynes AB, Weiser TG, Berry WR, et al; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360(5):491-499. 5. Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? a systematic review. Ann Surg. 2013;258(6):856-871. 6. Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370(11):1029-1038.
ASSOCIATION OF VA SURGEONS
Measuring Surgical Quality: Which Measure Should We Trust? The use of surgical quality measures to target quality improvement efforts and evaluate hospital performance is now standard. Surgical quality in Veterans Affairs (VA) hospitals is measured by the VA Surgical Quality Improvement Program (VASQIP),1 the Surgical Care Improvement Program (SCIP),2 and the Patient Safety Indicators (PSIs).3 Each approach has a different perspective on surgical quality and uses a different source of data. For example, the VASQIP evaluates 30-day postoperative morbidity and mortality outcomes among other parameters, the SCIP measures compliance with specific perioperative processes of care, and the PSIs calculate the rates of potentially preventable, inpatient, surgical adverse events using administrative data. We explored the correlation between VASQIP, SCIP, and PSI measures and how consistently they identified high- and low-performing VA hospitals. Methods | We used quality indicator data from fiscal year 2009 (ie, from October 1, 2008, to September 30, 2009) from 67 VA
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
jamasurgery.com
Letters
Table 1. Spearman Correlation Coefficients and Agreement in Performance Rankings of FY09 Surgical Quality Measuresa Quality Measure VASQIP mortality O/E ratio
VASQIP Morbidity 0.127
P Value .31
VASQIP Mortality
SCIP composite
0.047
.71
−0.141
.25
PSI surgery compositeb
0.268
.03
0.054
.66
SCIP Composite
P Value
−0.134
.28
and statistical significance was set at P ⱕ .05.
Abbreviations: FYO9, fiscal year 2009; O/E, observed to expected; PSI, Patient Safety Indicator; SCIP, Surgical Care Improvement Program; VASQIP, Veterans Affairs Surgical Quality Improvement Program. a
P Value
b
We adapted the PSI composite software to develop a PSI surgery composite score calculated for the postoperative PSIs using numerator-based weights.5
Facility rankings were compared using each of the surgical quality measures,
Table 2. Agreement in High, Average, and Low Hospital Surgical Performance Using FY09 Surgical Quality Measuresa % of Hospitals No. of Measures Rated All 4
Top 25% 0
Average 50% 10
Bottom 25% 2
3
8
21
6
2
22
33
19
1
34
27
39
0
36
9
34
100
100
100
Total
hospitals with advanced surgical programs. We obtained the hospitals’ VASQIP morbidity and mortality observed to expected ratios and SCIP compliance scores from the 2010 VA Facility Quality and Safety Report.4 We ran the PSI software on hospital surgical discharge data to generate observed and riskadjusted rates for each of the 7 postoperative PSIs. We then adapted the PSI composite software to develop a PSI surgery composite score calculated for the postoperative PSIs using numerator-based weights.5 Using these 4 quality measures, we ranked hospitals and examined the correlation between ranks. We also identified the top and bottom 25% of hospitals, and calculated the number of hospitals with high or low performance on multiple indicators. Results | Few comparisons yielded significant correlations. Only the hospital VASQIP morbidity observed to expected ratio and the hospital PSI surgery composite score had a significant, albeit weak, association (r = 0.267, P = .03) (Table 1). Agreement on whether hospitals were high, average, or low performers was similarly moderate: the SCIP compliance score and the PSI surgery composite score agreed on performance category for 45% of the hospitals (the highest agreement), whereas the SCIP compliance score and the VASQIP mortality observed to expected ratio agreed for only 37% (lowest agreement). Although none of the hospitals performed well on all 4 measures, 5 of the 67 hospitals (7%) were in the top 25% on 3 of the measures. On all 4 measures, 7 hospitals (10%) were considered average, and 1 hospital (1%) was in the bottom 25% (Table 2). Discussion | High performance on one type of surgical quality measure was not associated with high performance on another. The lack of correlation between the VASQIP measures, the SCIP compliance score, and the PSI surgery composite score suggests that these indicators measure different dimensions of surgical quality. Information from multiple quality mea-
Abbreviations: FYO9, fiscal year 2009; PSI, Patient Safety Indicator; SCIP, Surgical Care Improvement Program; VASQIP, Veterans Affairs Surgical Quality Improvement Program. a
The VASQIP 30-day morbidity and mortality observed to expected ratios, the SCIP composite of compliance, and the PSI surgery composite score developed using numerator weights for the 7 postoperative PSIs.
sures is useful in directing individual facilities toward different quality improvement activities. However, from the perspective of comparing facilities, these differences highlight the importance of examining more than 1 measure. Our findings illustrate the potential confusion that may be associated with multiple, poorly correlated measures that purport to measure quality. However, the confusion arises only when quality is conceptualized as an underlying latent construct that is reflected in the individual indicators. When quality is conceptualized as a construct created by combining individual indicators that reflect different dimensions of quality, the low correlation of individual indicators does not create a problem. In fact, as noted by Feinstein,6 combining uncorrelated dimensions into a composite measure is more consistent with clinical needs than a composite created from multiple dimensions of the same phenomena. We postulate that measures such as the PSIs, VASQIP, and SCIP could be used to develop a single composite measure of quality that encompasses several aspects of surgical quality. In the future, a single composite measure of surgical quality could provide more actionable information for patients, health care professionals, and policy makers as they attempt to differentiate hospital performance. Hillary J. Mull, PhD, MPP Qi Chen, MD, PhD Michael Shwartz, PhD, MBA Kamal M. F. Itani, MD Amy K. Rosen, PhD Author Affiliations: Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, Massachusetts (Mull, Chen, Shwartz, Rosen); Department of Surgery, Boston University School of Medicine, Boston, Massachusetts (Mull, Itani, Rosen); Department of Operations and Technology Management, Boston University School of Management, Boston, Massachusetts (Shwartz); Department of Surgery, VA Boston Healthcare
jamasurgery.com
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
1211
Letters
System, Boston, Massachusetts (Itani); Harvard Medical School, Boston, Massachusetts (Itani). Corresponding Author: Hillary J. Mull, PhD, MPP, Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, 150 S Huntington Ave, 152M, Boston, MA 02130 ([email protected]). Published Online: September 24, 2014. doi:10.1001/jamasurg.2014.373. Author Contributions: Dr Mull had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Mull, Chen, Shwartz, Itani. Acquisition, analysis, or interpretation of data: Mull, Chen, Shwartz, Rosen. Drafting of the manuscript: Mull, Itani. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Mull, Chen, Shwartz. Obtained funding: Rosen Administrative, technical, or material support: Chen, Itani. Study supervision: Rosen.
Table 1. Top 5 Most Common Surgical Procedures Performed for CHAMPIONS Patients and the Average Length of Stay Associated With Each Procedure Surgical Procedure Colon resection (hemicolectomy, low anterior resection, sigmoidectomy)
Part of Total, %
Average Length of Stay, d
25
17.67
Lung resection (lobectomy, wedge resection)
18
6.77
Coronary artery bypass grafting
14
10.00
Open-heart valve replacement or repair
7
12.00
Cholecystectomy
7
4.00
Abbreviation: CHAMPIONS, Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients.
Conflict of Interest Disclosures: None reported. Funding/Support: This research was funded by VA Health Services Research and Development Service grant SDR 07-002 (Dr Rosen, principal investigator). Role of the Funder/Sponsor: The VA Health Services Research and Development Service had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Previous Presentation: This paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 7, 2014; New Haven, Connecticut. 1. Khuri SF, Daley J, Henderson W, et al; National VA Surgical Quality Improvement Program. The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. Ann Surg. 1998;228(4):491-507. 2. Bratzler DW. The Surgical Infection Prevention and Surgical Care Improvement Projects: promises and pitfalls. Am Surg. 2006;72(11):1010-1016. 3. Patient Safety Indicators overview. Agency for Healthcare Research and Quality website. http://www.qualityindicators.ahrq.gov/Modules/psi_overview.aspx. Accessed March 3, 2013. 4. Department of Veterans Affairs; Veterans Health Administration (VHA). 2010 VHA Facility Quality and Safety Report. http://www.va.gov/health/docs /HospitalReportCard2010.pdf. Published October 2010. Accessed August 13, 2014. 5. Agency for Healthcare Research and Quality (AHRQ). AHRQ Quality Indicators: Composite Measures User Guide for the Patient Safety Indicators (PSI), Version 4.2. Rockville, MD: AHRQ; 2010. 6. Feinstein AR. Multi-item “instruments” vs Virginia Apgar’s principles of clinimetrics. Arch Intern Med. 1999;159(2):125-128.
ASSOCIATION OF VA SURGEONS
Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients Half of the people older than 65 years of age will undergo a surgical procedure in their remaining lifetime.1 With more than 1 million veterans currently older than 84 years of age, many compromised individuals will be seeking surgical care in Veterans Affairs (VA) hospitals.2 Postoperatively, discharge needs are often underestimated and complicated to coordinate. In 2010, the Departments of Surgery and Geriatrics at the VA Connecticut Healthcare System received VA Clinical Initiative funding to create the Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients (CHAMPIONS), an individualized program intended to return geriatric patients home with maximum independence after surgery. 1212
Methods | The CHAMPIONS team consisted of a geriatrician, an advanced practice registered nurse, a physical therapist, a surgeon, and a case manager. Geriatric patients undergoing elective major surgery were eligible. Patients gave oral informed consent to participate in CHAMPIONS. There was no financial compensation given. Preoperative cognitive assessments (using the Saint Louis University Mental Status Examination score), functional assessments (using the activities of daily living score and the independent activities of daily living score), and home safety assessments were conducted by the advanced practice registered nurse and physical therapist during home visits to determine in-hospital and postdischarge needs, including dementia prevention tools and home safety equipment. Patients were comanaged during hospitalization, seen at home 48 to 72 hours after hospital discharge, and followed up for 30 days. Emergency department visits and 30-day readmissions were tracked. Using match-paired Wilcoxon tests, we reassessed patients 30 days after hospital discharge for change in cognition or function. Institutional review board approval was obtained before analyzing the CHAMPIONS database. Results | Seventy-two patients entered CHAMPIONS. Eight patients did not complete the program at the patient’s or patient’s family’s request, and only 6 required short-term rehabilitation; 58 patients were discharged home and completed the postdischarge evaluation. The median patient age was 79 years (range, 68-93 years). More than 87% of patients underwent general or cardiothoracic surgery (Table 1). Fifty-four percent had a cognitive deficit, and 35% were frail. There was no significant postoperative change in the Saint Louis University Mental Status Examination scores (for 55 patients; P = .92) or in activities of daily living scores (for 58 patients; P = .83). There was a small decrease in postoperative independent activities of daily living scores (for 58 patients; P = .03; maximum score, 16; preoperative median score, 16 [interquartile range, 14-16]; postoperative median score, 15 [interquartile range, 12-16]). Eighteen patients had emergency department visits within 30 days of hospital discharge, and only 8 of the 72 patients (11.1%) required readmission (Table 2). Discussion | The use of CHAMPIONS has improved geriatric surgical care at the West Haven VA hospital as demonstrated by
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
jamasurgery.com
Table. Responses to Survey by OR Physicians and Registered Nurses % of Respondents Who Strongly Agree or Agree Statement Use of the AWS timeout process improves surgical team engagement compared with the traditional timeout process. Use of the AWS improves patient safety compared with the traditional timeout process. Use of the AWS improves compliance with SCIP documentation compared with traditional methods. Use of the AWS improves the opportunities to prevent OR errors /near misses compared with traditional communication methods. Use of the AWS improves the operative team’s communication throughout the surgical procedure compared with traditional communication methods. The AWS is user friendly.
Total 86.96
Registered Nurse 92.86
Physician 73.68
84.78
92.86
73.68
91.11
92.86
84.21
84.44
92.86
66.67
78.26
85.71
63.16
84.44
100.00
78.95
I feel competent using the AWS.
81.82
100.00
72.22
The postoperative debriefing checklist in the AWS ensures that a consistent verification process takes place prior to the patient exiting the OR.a I have received feedback on my compliance with the AWS checklist.b The preprocedure checklist helps improve the preprocedure safety verification process.b
88.00
100.00
72.73
100.00
100.00
NA
92.86
92.86
NA
curve and technical problems with implementation early in use. Surveys show that physicians and staff members believe that the AWS improves communication, team engagement, patient safety, and compliance with Surgical Care Improvement Project measures. Future goals for the AWS at our facility include tracking near misses to improve patient safety and staff education opportunities, developing a structured feedback system for operative team members, and incorporating other risk assessments. Juliet Nissan, MD Valkiria Campos, RN, BSN, CPAN Hector Delgado, RN, MSN, MHA Christina Matadial, MD Seth Spector, MD Author Affiliations: Bruce W. Carter Department of VA Medical Center, Miami VA Healthcare System, University of Miami Miller School of Medicine, Jackson Memorial Hospital, Miami, Florida. Corresponding Author: Juliet Nissan, MD, Bruce W. Carter Department of VA Medical Center, Miami VA Healthcare System, University of Miami Miller School of Medicine, Jackson Memorial Hospital, 1611 NW 12th Ave, East Tower, Ste 2169, Miami, FL 33136 ([email protected]). Published Online: September 24, 2014. doi:10.1001/jamasurg.2014.1825. Author Contributions: Dr Nissan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: All authors. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Nissan, Campos, Spector. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Nissan, Campos, Delgado, Spector. Administrative, technical, or material support: Matadial, Spector. Study supervision: Matadial, Spector. Conflict of Interest Disclosures: None reported. Previous Presentation: The paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 7, 2014; New Haven, Connecticut. Additional Information: LiveData, the company that manufactured the AWS used at our institution (OR-Dashboard), had no role in the design and conduct of the study; analysis and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. 1210
Abbreviations: AWS, automated workflow system; NA, not applicable; OR, operating room; SCIP, Surgical Care Improvement Project. a
Only physicians and registered nurses responded.
b
Only registered nurses responded.
1. Gawande A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books/Henry Holt and Co; 2011. 2. World Alliance for Patient Safety; World Health Organization. Safe surgery saves lives: second global patient safety challenge. http://www.who.int/patientsafety /safesurgery/knowledge_base/SSSL_Brochure_finalJun08.pdf. Accessed March 24, 2014. 3. World Health Organization. Surgical safety checklist (first edition). http://www .who.int/patientsafety/safesurgery/tools_resources/SSSL_Checklist_finalJun08 .pdf. Accessed March 24, 2014. 4. Haynes AB, Weiser TG, Berry WR, et al; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360(5):491-499. 5. Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? a systematic review. Ann Surg. 2013;258(6):856-871. 6. Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370(11):1029-1038.
ASSOCIATION OF VA SURGEONS
Measuring Surgical Quality: Which Measure Should We Trust? The use of surgical quality measures to target quality improvement efforts and evaluate hospital performance is now standard. Surgical quality in Veterans Affairs (VA) hospitals is measured by the VA Surgical Quality Improvement Program (VASQIP),1 the Surgical Care Improvement Program (SCIP),2 and the Patient Safety Indicators (PSIs).3 Each approach has a different perspective on surgical quality and uses a different source of data. For example, the VASQIP evaluates 30-day postoperative morbidity and mortality outcomes among other parameters, the SCIP measures compliance with specific perioperative processes of care, and the PSIs calculate the rates of potentially preventable, inpatient, surgical adverse events using administrative data. We explored the correlation between VASQIP, SCIP, and PSI measures and how consistently they identified high- and low-performing VA hospitals. Methods | We used quality indicator data from fiscal year 2009 (ie, from October 1, 2008, to September 30, 2009) from 67 VA
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
jamasurgery.com
Letters
Table 1. Spearman Correlation Coefficients and Agreement in Performance Rankings of FY09 Surgical Quality Measuresa Quality Measure VASQIP mortality O/E ratio
VASQIP Morbidity 0.127
P Value .31
VASQIP Mortality
SCIP composite
0.047
.71
−0.141
.25
PSI surgery compositeb
0.268
.03
0.054
.66
SCIP Composite
P Value
−0.134
.28
and statistical significance was set at P ⱕ .05.
Abbreviations: FYO9, fiscal year 2009; O/E, observed to expected; PSI, Patient Safety Indicator; SCIP, Surgical Care Improvement Program; VASQIP, Veterans Affairs Surgical Quality Improvement Program. a
P Value
b
We adapted the PSI composite software to develop a PSI surgery composite score calculated for the postoperative PSIs using numerator-based weights.5
Facility rankings were compared using each of the surgical quality measures,
Table 2. Agreement in High, Average, and Low Hospital Surgical Performance Using FY09 Surgical Quality Measuresa % of Hospitals No. of Measures Rated All 4
Top 25% 0
Average 50% 10
Bottom 25% 2
3
8
21
6
2
22
33
19
1
34
27
39
0
36
9
34
100
100
100
Total
hospitals with advanced surgical programs. We obtained the hospitals’ VASQIP morbidity and mortality observed to expected ratios and SCIP compliance scores from the 2010 VA Facility Quality and Safety Report.4 We ran the PSI software on hospital surgical discharge data to generate observed and riskadjusted rates for each of the 7 postoperative PSIs. We then adapted the PSI composite software to develop a PSI surgery composite score calculated for the postoperative PSIs using numerator-based weights.5 Using these 4 quality measures, we ranked hospitals and examined the correlation between ranks. We also identified the top and bottom 25% of hospitals, and calculated the number of hospitals with high or low performance on multiple indicators. Results | Few comparisons yielded significant correlations. Only the hospital VASQIP morbidity observed to expected ratio and the hospital PSI surgery composite score had a significant, albeit weak, association (r = 0.267, P = .03) (Table 1). Agreement on whether hospitals were high, average, or low performers was similarly moderate: the SCIP compliance score and the PSI surgery composite score agreed on performance category for 45% of the hospitals (the highest agreement), whereas the SCIP compliance score and the VASQIP mortality observed to expected ratio agreed for only 37% (lowest agreement). Although none of the hospitals performed well on all 4 measures, 5 of the 67 hospitals (7%) were in the top 25% on 3 of the measures. On all 4 measures, 7 hospitals (10%) were considered average, and 1 hospital (1%) was in the bottom 25% (Table 2). Discussion | High performance on one type of surgical quality measure was not associated with high performance on another. The lack of correlation between the VASQIP measures, the SCIP compliance score, and the PSI surgery composite score suggests that these indicators measure different dimensions of surgical quality. Information from multiple quality mea-
Abbreviations: FYO9, fiscal year 2009; PSI, Patient Safety Indicator; SCIP, Surgical Care Improvement Program; VASQIP, Veterans Affairs Surgical Quality Improvement Program. a
The VASQIP 30-day morbidity and mortality observed to expected ratios, the SCIP composite of compliance, and the PSI surgery composite score developed using numerator weights for the 7 postoperative PSIs.
sures is useful in directing individual facilities toward different quality improvement activities. However, from the perspective of comparing facilities, these differences highlight the importance of examining more than 1 measure. Our findings illustrate the potential confusion that may be associated with multiple, poorly correlated measures that purport to measure quality. However, the confusion arises only when quality is conceptualized as an underlying latent construct that is reflected in the individual indicators. When quality is conceptualized as a construct created by combining individual indicators that reflect different dimensions of quality, the low correlation of individual indicators does not create a problem. In fact, as noted by Feinstein,6 combining uncorrelated dimensions into a composite measure is more consistent with clinical needs than a composite created from multiple dimensions of the same phenomena. We postulate that measures such as the PSIs, VASQIP, and SCIP could be used to develop a single composite measure of quality that encompasses several aspects of surgical quality. In the future, a single composite measure of surgical quality could provide more actionable information for patients, health care professionals, and policy makers as they attempt to differentiate hospital performance. Hillary J. Mull, PhD, MPP Qi Chen, MD, PhD Michael Shwartz, PhD, MBA Kamal M. F. Itani, MD Amy K. Rosen, PhD Author Affiliations: Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, Massachusetts (Mull, Chen, Shwartz, Rosen); Department of Surgery, Boston University School of Medicine, Boston, Massachusetts (Mull, Itani, Rosen); Department of Operations and Technology Management, Boston University School of Management, Boston, Massachusetts (Shwartz); Department of Surgery, VA Boston Healthcare
jamasurgery.com
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
1211
Letters
System, Boston, Massachusetts (Itani); Harvard Medical School, Boston, Massachusetts (Itani). Corresponding Author: Hillary J. Mull, PhD, MPP, Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, 150 S Huntington Ave, 152M, Boston, MA 02130 ([email protected]). Published Online: September 24, 2014. doi:10.1001/jamasurg.2014.373. Author Contributions: Dr Mull had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Mull, Chen, Shwartz, Itani. Acquisition, analysis, or interpretation of data: Mull, Chen, Shwartz, Rosen. Drafting of the manuscript: Mull, Itani. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Mull, Chen, Shwartz. Obtained funding: Rosen Administrative, technical, or material support: Chen, Itani. Study supervision: Rosen.
Table 1. Top 5 Most Common Surgical Procedures Performed for CHAMPIONS Patients and the Average Length of Stay Associated With Each Procedure Surgical Procedure Colon resection (hemicolectomy, low anterior resection, sigmoidectomy)
Part of Total, %
Average Length of Stay, d
25
17.67
Lung resection (lobectomy, wedge resection)
18
6.77
Coronary artery bypass grafting
14
10.00
Open-heart valve replacement or repair
7
12.00
Cholecystectomy
7
4.00
Abbreviation: CHAMPIONS, Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients.
Conflict of Interest Disclosures: None reported. Funding/Support: This research was funded by VA Health Services Research and Development Service grant SDR 07-002 (Dr Rosen, principal investigator). Role of the Funder/Sponsor: The VA Health Services Research and Development Service had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Previous Presentation: This paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 7, 2014; New Haven, Connecticut. 1. Khuri SF, Daley J, Henderson W, et al; National VA Surgical Quality Improvement Program. The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. Ann Surg. 1998;228(4):491-507. 2. Bratzler DW. The Surgical Infection Prevention and Surgical Care Improvement Projects: promises and pitfalls. Am Surg. 2006;72(11):1010-1016. 3. Patient Safety Indicators overview. Agency for Healthcare Research and Quality website. http://www.qualityindicators.ahrq.gov/Modules/psi_overview.aspx. Accessed March 3, 2013. 4. Department of Veterans Affairs; Veterans Health Administration (VHA). 2010 VHA Facility Quality and Safety Report. http://www.va.gov/health/docs /HospitalReportCard2010.pdf. Published October 2010. Accessed August 13, 2014. 5. Agency for Healthcare Research and Quality (AHRQ). AHRQ Quality Indicators: Composite Measures User Guide for the Patient Safety Indicators (PSI), Version 4.2. Rockville, MD: AHRQ; 2010. 6. Feinstein AR. Multi-item “instruments” vs Virginia Apgar’s principles of clinimetrics. Arch Intern Med. 1999;159(2):125-128.
ASSOCIATION OF VA SURGEONS
Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients Half of the people older than 65 years of age will undergo a surgical procedure in their remaining lifetime.1 With more than 1 million veterans currently older than 84 years of age, many compromised individuals will be seeking surgical care in Veterans Affairs (VA) hospitals.2 Postoperatively, discharge needs are often underestimated and complicated to coordinate. In 2010, the Departments of Surgery and Geriatrics at the VA Connecticut Healthcare System received VA Clinical Initiative funding to create the Comprehensive Home-Based Acute Care Medical Program Initiative for Older Noncritical Surgical Patients (CHAMPIONS), an individualized program intended to return geriatric patients home with maximum independence after surgery. 1212
Methods | The CHAMPIONS team consisted of a geriatrician, an advanced practice registered nurse, a physical therapist, a surgeon, and a case manager. Geriatric patients undergoing elective major surgery were eligible. Patients gave oral informed consent to participate in CHAMPIONS. There was no financial compensation given. Preoperative cognitive assessments (using the Saint Louis University Mental Status Examination score), functional assessments (using the activities of daily living score and the independent activities of daily living score), and home safety assessments were conducted by the advanced practice registered nurse and physical therapist during home visits to determine in-hospital and postdischarge needs, including dementia prevention tools and home safety equipment. Patients were comanaged during hospitalization, seen at home 48 to 72 hours after hospital discharge, and followed up for 30 days. Emergency department visits and 30-day readmissions were tracked. Using match-paired Wilcoxon tests, we reassessed patients 30 days after hospital discharge for change in cognition or function. Institutional review board approval was obtained before analyzing the CHAMPIONS database. Results | Seventy-two patients entered CHAMPIONS. Eight patients did not complete the program at the patient’s or patient’s family’s request, and only 6 required short-term rehabilitation; 58 patients were discharged home and completed the postdischarge evaluation. The median patient age was 79 years (range, 68-93 years). More than 87% of patients underwent general or cardiothoracic surgery (Table 1). Fifty-four percent had a cognitive deficit, and 35% were frail. There was no significant postoperative change in the Saint Louis University Mental Status Examination scores (for 55 patients; P = .92) or in activities of daily living scores (for 58 patients; P = .83). There was a small decrease in postoperative independent activities of daily living scores (for 58 patients; P = .03; maximum score, 16; preoperative median score, 16 [interquartile range, 14-16]; postoperative median score, 15 [interquartile range, 12-16]). Eighteen patients had emergency department visits within 30 days of hospital discharge, and only 8 of the 72 patients (11.1%) required readmission (Table 2). Discussion | The use of CHAMPIONS has improved geriatric surgical care at the West Haven VA hospital as demonstrated by
JAMA Surgery November 2014 Volume 149, Number 11
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Georgia User on 05/26/2015
jamasurgery.com
