Letters COMMENT & RESPONSE
Response to Patient Satisfaction as a Possible Indicator of Quality Surgical Care To the Editor The recent study by Lyu et al1 found little or no association between patient satisfaction on Medicare Hospital Consumer Assessment of Healthcare Providers and Systems Survey (HCAHPS) scores and the quality of care in operating rooms. This finding is in contradistinction to our recent study 2 that observed a significant association between facility-wide patient satisfaction (on HCAHPS scores) and the incidence of central line–associated bloodstream infections in the intensive care unit (hospital staff responsiveness estimated rate ratio, 1.16; P < .001). We came to the opposite conclusion, that quality problems in one part of a facility may reflect a facility-wide problem with the facility’s culture of safety, which, thus, may be reflected in patient satisfaction surveys. T h e r e we r e m a j o r d i f f e r e n c e s b e t we e n t h e s e 2 studies. 1, 2 Lyu et al 1 primarily used process measures and the employee Safety Attitudes Questionnaire in 31 facilities to evaluate quality. In our study,2 we used an outcome measure of central line–associated bloodstream infections with data from 1987 facilities and compared them with regard to 3 HCAHPS questions that involved room cleanliness, staff responsiveness, and communication with the patient. There have been a number of reports that have presented data indicating that process measures may not be the best predictor of quality.3-5 A concern is that a facility may “teach to the test” and may only pay attention to improving 1 link in the chain of events that is needed to produce an adverse event– free outcome.3 Furthermore, the method used to administer the Safety Attitudes Questionnaire is of concern because the last 4 digits of the respondent’s social security number were used as a “personalized token,” which may have created the impression that the survey was not anonymous.1 Despite this and the low number of facilities surveyed, there was a trend for an overall correlation of the Safety Attitudes Questionnaire with patient surveys. A climate of teamwork and a culture of safety were found to be modestly correlated and statistically significant. Both of these measures are sensitive to the nursing staff and their work environment and bolster the observation that these issues are of utmost importance in delivering highquality care.3 Because of the problems with sample size and process measures, we argue that caution should be used when interpreting the patient survey results of the study by Lyu et al.1 We believe that patient satisfaction scores play an integral role in quality assurance and should be an important part of value-based purchasing initiatives. In nearly every other jamasurgery.com
industry, patient satisfaction surveys are taken seriously; they should also be taken seriously in hospitals. Daniel M. Saman, DrPH, MPH, CPH Kevin T. Kavanagh, MD, MS Author Affiliations: Division of Research, Essentia Institute of Rural Health, Duluth, Minnesota (Saman); Health Watch USA, Somerset, Kentucky (Kavanagh). Corresponding Author: Daniel M. Saman, DrPH, MPH, CPH, Division of Research, Essentia Institute of Rural Health, 421 N 6th Ave E, Duluth, MN 55805 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Lyu H, Wick EC, Housman M, Freischlag JA, Makary MA. Patient satisfaction as a possible indicator of quality surgical care. JAMA Surg. 2013;148(4):362-367. 2. Saman DM, Kavanagh KT, Johnson B, Lutfiyya MN. Can inpatient hospital experiences predict central line-associated bloodstream infections? PLoS One. 2013;8(4):e61097. 3. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay for performance on patient outcomes. N Engl J Med. 2012;366(17):1606-1615. 4. Kavanagh KT. Premier pay for performance and patient outcomes. N Engl J Med. 2012;367(4):381-382; author reply 382-383. 5. Kavanagh KT, Cimiotti JP, Abusalem S, Coty MB. Moving healthcare quality forward with nursing-sensitive value-based purchasing. J Nurs Scholarsh. 2012;44(4):385-395.
Valuing Patient Experience as a Unique and Intrinsically Important Aspect of Health Care Quality To the Editor Lyu and colleagues1 could not find significant associations between patients’ overall ratings of hospitals and those hospitals’ performance on surgical process measures. Their study of 31 hospitals had insufficient power to establish moderately strong associations such as those previously shown in a similar but larger study.2 However, even if their findings were confirmed in a larger sample, they should be neither surprising nor concerning. Quality is multidimensional, and individual quality indicators may or may not reflect quality of care in other areas. Hence, some health care providers perform better or worse on measures in the patient experience domain than on measures in the clinical quality domain.3 There is also considerable variation within each quality domain, with some hospitals performing better on cardiac measures than on pneumonia measures, for example.4 Variation in performance within a measure set is, in fact, desirable because it indicates that each measure is contributing unique information to the total quality score. Improving quality in one domain need not be deleterious to others because changes to a system’s infrastructure and processes can influence a broad range of outcomes. Lyu et al conclude that the use of patient survey results as a “comprehensive measure of quality to determine hospital reimbursement must be questioned in the absence of other reliable outcome metrics.”1(p365) We know of no federal initiative that holds health care providers accountable for paJAMA Surgery October 2013 Volume 148, Number 10
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 06/07/2015
985
Response to Patient Satisfaction as a Possible Indicator of Quality Surgical Care To the Editor The recent study by Lyu et al1 found little or no association between patient satisfaction on Medicare Hospital Consumer Assessment of Healthcare Providers and Systems Survey (HCAHPS) scores and the quality of care in operating rooms. This finding is in contradistinction to our recent study 2 that observed a significant association between facility-wide patient satisfaction (on HCAHPS scores) and the incidence of central line–associated bloodstream infections in the intensive care unit (hospital staff responsiveness estimated rate ratio, 1.16; P < .001). We came to the opposite conclusion, that quality problems in one part of a facility may reflect a facility-wide problem with the facility’s culture of safety, which, thus, may be reflected in patient satisfaction surveys. T h e r e we r e m a j o r d i f f e r e n c e s b e t we e n t h e s e 2 studies. 1, 2 Lyu et al 1 primarily used process measures and the employee Safety Attitudes Questionnaire in 31 facilities to evaluate quality. In our study,2 we used an outcome measure of central line–associated bloodstream infections with data from 1987 facilities and compared them with regard to 3 HCAHPS questions that involved room cleanliness, staff responsiveness, and communication with the patient. There have been a number of reports that have presented data indicating that process measures may not be the best predictor of quality.3-5 A concern is that a facility may “teach to the test” and may only pay attention to improving 1 link in the chain of events that is needed to produce an adverse event– free outcome.3 Furthermore, the method used to administer the Safety Attitudes Questionnaire is of concern because the last 4 digits of the respondent’s social security number were used as a “personalized token,” which may have created the impression that the survey was not anonymous.1 Despite this and the low number of facilities surveyed, there was a trend for an overall correlation of the Safety Attitudes Questionnaire with patient surveys. A climate of teamwork and a culture of safety were found to be modestly correlated and statistically significant. Both of these measures are sensitive to the nursing staff and their work environment and bolster the observation that these issues are of utmost importance in delivering highquality care.3 Because of the problems with sample size and process measures, we argue that caution should be used when interpreting the patient survey results of the study by Lyu et al.1 We believe that patient satisfaction scores play an integral role in quality assurance and should be an important part of value-based purchasing initiatives. In nearly every other jamasurgery.com
industry, patient satisfaction surveys are taken seriously; they should also be taken seriously in hospitals. Daniel M. Saman, DrPH, MPH, CPH Kevin T. Kavanagh, MD, MS Author Affiliations: Division of Research, Essentia Institute of Rural Health, Duluth, Minnesota (Saman); Health Watch USA, Somerset, Kentucky (Kavanagh). Corresponding Author: Daniel M. Saman, DrPH, MPH, CPH, Division of Research, Essentia Institute of Rural Health, 421 N 6th Ave E, Duluth, MN 55805 ([email protected]). Conflict of Interest Disclosures: None reported. 1. Lyu H, Wick EC, Housman M, Freischlag JA, Makary MA. Patient satisfaction as a possible indicator of quality surgical care. JAMA Surg. 2013;148(4):362-367. 2. Saman DM, Kavanagh KT, Johnson B, Lutfiyya MN. Can inpatient hospital experiences predict central line-associated bloodstream infections? PLoS One. 2013;8(4):e61097. 3. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay for performance on patient outcomes. N Engl J Med. 2012;366(17):1606-1615. 4. Kavanagh KT. Premier pay for performance and patient outcomes. N Engl J Med. 2012;367(4):381-382; author reply 382-383. 5. Kavanagh KT, Cimiotti JP, Abusalem S, Coty MB. Moving healthcare quality forward with nursing-sensitive value-based purchasing. J Nurs Scholarsh. 2012;44(4):385-395.
Valuing Patient Experience as a Unique and Intrinsically Important Aspect of Health Care Quality To the Editor Lyu and colleagues1 could not find significant associations between patients’ overall ratings of hospitals and those hospitals’ performance on surgical process measures. Their study of 31 hospitals had insufficient power to establish moderately strong associations such as those previously shown in a similar but larger study.2 However, even if their findings were confirmed in a larger sample, they should be neither surprising nor concerning. Quality is multidimensional, and individual quality indicators may or may not reflect quality of care in other areas. Hence, some health care providers perform better or worse on measures in the patient experience domain than on measures in the clinical quality domain.3 There is also considerable variation within each quality domain, with some hospitals performing better on cardiac measures than on pneumonia measures, for example.4 Variation in performance within a measure set is, in fact, desirable because it indicates that each measure is contributing unique information to the total quality score. Improving quality in one domain need not be deleterious to others because changes to a system’s infrastructure and processes can influence a broad range of outcomes. Lyu et al conclude that the use of patient survey results as a “comprehensive measure of quality to determine hospital reimbursement must be questioned in the absence of other reliable outcome metrics.”1(p365) We know of no federal initiative that holds health care providers accountable for paJAMA Surgery October 2013 Volume 148, Number 10
Downloaded From: http://archsurg.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 06/07/2015
985
