Clinical Gastroenterology and Hepatology 2015;-:-–-
Quality Measurement in Gastroenterology: Confessions of a Realist Spencer D. Dorn Division of Gastroenterology and Hepatology, University of North Carolina School of Medicine, Chapel Hill, North Carolina Clinicians are required to report their performance on an ever-increasing number of quality measures. However, it is difficult to measure health care quality and it is unclear whether broadly applying accountability measures effectively improves care. This article considers these challenges and includes recommendations that may help gastroenterologists respond to demands for increased quality measurement. Keywords: Quality; Measurement; Accountability.
Not everything that can be counted counts, and not everything that counts can be counted. William Bruce Cameron, Informal Sociology, A Casual Introduction to Sociological Thinking n today’s digitized, measurement-oriented, datadriven world, payers increasingly are using quality measures to hold clinicians accountable for quality of care. The National Quality Measures Clearinghouse lists more than 2000 such measures and new measures are being developed at unprecedented speed.1 The recently enacted Medicare Access and CHIP Reauthorization Act of 2015 doubles down on quality measurement; by 2022, up to 9% of Medicare physician payments will be linked to performance on quality measures.2 The general idea is that measuring quality and linking it to financial incentives and consumer choice will drive improvement. But does it really work this way? The late comedian George Carlin said, “Scratch any cynic and you will find a disappointed idealist.” Not long ago I was fairly idealistic about health care quality measurement.3 However, with additional experience practicing gastroenterology, serving on local and national quality-related committees, and helping to lead a large academic practice, my thoughts and feelings have shifted. Yes, I recognize that the quality measurement movement has resulted in undeniable good. It has increased attention to quality, safety, and patient experience; driven clinicians, hospitals, and health systems to develop new competencies; helped emphasize the importance of avoiding unnecessary services; and highlighted that systems of care (not individual clinicians alone) determine quality and patient experiences.
I
Still, I now better appreciate how hard it is to measure health care quality—many metrics do not accurately reflect quality—and I question whether broadly doing so effectively improves care.4 Beyond this, I am acutely concerned about how much time, money, and attention widespread quality measurement consumes, as well as the potentially damaging effects it has on clinicians’ spirits. In short, and paraphrasing one famous Institute of Medicine5 report, between the quality measurement ideal and the quality measurement reality lies not just a gap but a chasm. In this article I consider the challenges of measuring health care quality, and then make several pragmatic recommendations that I hope will help gastroenterologists respond to demands for increased quality measurement.
Problems and Challenges Measurement Issues Quantifying health care quality is exceedingly difficult. Measures must be valid (ie, actually reflect the quality of care delivered) and reliable (ie, variation should reflect true differences in performance, rather than measurement error). When care stretches over time it can be difficult to determine the appropriate measurement time frame. Similarly, when care involves multiple clinicians it can be difficult to determine who gets credit for different processes and outcomes. Finally, it is impossible to adjust fully for socioeconomic status, genetic factors, and other patient characteristics that affect outcomes.3
Generalizability Quality measures exist for only a narrow and discrete portion of clinical care. However, how clinicians perform on these small, convenient, unrepresentative sets of measures often influences payment for all of their clinical activities, as well as how they appear on publically available quality reports. My adenoma detection rate is © 2015 by the AGA Institute 1542-3565/$36.00 http://dx.doi.org/10.1016/j.cgh.2015.07.033
2
Spencer D. Dorn
an important indicator of how well I perform screening colonoscopies, but it certainly does not reflect the quality of my overall practice.
Burden Quality reporting programs befuddle clinicians, reduce their efficiency, and consume too much of their time.6 Meanwhile, large health care systems spend millions of dollars each year measuring and reporting performance. Many of these resources would be better deployed elsewhere.
Demotivating Clinicians intrinsically are motivated. We seek to feel in control of our own professional lives (autonomy), we crave the opportunity to become better and better at what we do (mastery), and we yearn to contribute to something larger than our own selves (purpose).7 Tying quality measures to carrots and sticks can undermine motivation and worsen performance, especially for the vast majority of clinical activities that go unmeasured.8 Of course, clinicians also are motivated extrinsically. Ideally, financial incentives would align with medical professionalism (eg, improved patient outcomes) to complement, rather than undermine, clinicians’ intrinsic motivation.9
Performance Perversity When placed under strong external pressure, individuals sometimes will cut corners to achieve unrealistic performance targets. In health care this may include sham improvements that satisfy a measure without improving patient care (eg, distributing smoking cessation booklets to all smokers), cream skimming (ie, avoiding patients who are most likely to suffer poor outcomes), coding trickery, and even outright fraud (eg, the recent Veteran’s Affairs wait list scandal).10
Limited Effectiveness There is shockingly little evidence that measuring and externally reporting quality actually improves care.4,11,12 One reason is that it typically is hard to draw a straight line between health care and health outcomes, especially over short time horizons. Consequently, quality measures often focus on relatively obscure processes (eg, immunization rates) that only weakly are linked with health outcomes.13 A second reason is that measurement only leads to improvement in organizations that have a culture of accountability and a workforce skilled in quality improvement activities.14 Organizations without these capabilities are merely checking boxes without improving care. For instance, measuring and feeding back adenoma detection rates may not, on its own,
Clinical Gastroenterology and Hepatology Vol.
-,
No.
-
meaningfully increase adenoma detection rates.15 Finally, the types of quality measures used for accountability purposes are not useful for improvement. This is because accountability measures are too far downstream to explain how processes might be changed to improve outcomes. In addition, to ensure reliability, accountability measures must be collected over periods of time that are too long to be useful for rapid cycle improvement efforts.16
Barriers to Real Improvement Externally imposed, top-down quality programs actually may get in the way of real improvement. These programs sap clinician enthusiasm and siphon critical resources, leaving scant resources to support the internal, bottom-up programs that seem more effective.17 Furthermore, imposing external solutions discourages organizations from taking their own innovative approaches to improve quality, which, after all, is determined locally.18 Consequently, quality measurement is often an end in itself rather than a means to better care.19
Recommendations Recognize the Need for Quality Measurement If we do not measure quality then our services will become commoditized and payers will focus on costs alone. Therefore, despite the many challenges discussed earlier, gastroenterologists still should embrace and report valid, reliable, and clinically relevant measures. For many independent practices the easiest option is to join a quality registry that interfaces with endoscopy report writers, even if that means reporting some potentially invalid or unreliable measures simply to avoid penalties.
Resist Meaningless Measures and Push for Better Measures Gastroenterology societies should push back against invalid or meaningless measures and push for more relevant ones. For instance, when evidence-based guidelines changed, internists successfully lobbied to replace 3 quality measures that addressed low-density lipoprotein levels with more valid measures that assess whether high-risk patients are prescribed statins.20
Supplement Quantitative Measures With Qualitative Assessments It will never be possible to fully quantify the quality of all the care gastroenterologists provide. Health care is too varied, human judgment is too nuanced, and most medical decisions are not based on clear evidence.21 A limited set of sound quantitative measures therefore should be supplemented with qualitative assessments of
-
2015
care, preferably provided by patients because they are in the best position to evaluate the care they personally receive.22
Quality Measurement Judo Gastroenterology practices should leverage the time, attention, personnel, and other resources demanded by externally mandated quality programs to improve quality, however; they, not payers or regulators, define it. The focus should be on enhancing systems of care in ways that empower clinicians and better meet patient needs. For instance, our practice has developed systems to ensure our patients are scheduled for both clinic visits and open-access endoscopic procedures with the right physician, at the right time, and at the right practice location. We currently are working to better ensure our patients reliably receive their gastrointestinal pathology results, along with clear and appropriate follow-up recommendations. These sorts of things may not affect our performance on externally imposed quality measures, but they matter to our patients and therefore are important to us. Ultimately, is that not what quality is all about?
References 1. Panzer RJ, Gitomer RS, Greene WH, et al. Increasing demands for quality measurement. JAMA 2013;310:1971–1980. 2. HR 2. Medicare Access and CHIP Reauthorization Act. 114th Congress. Available from: https://www.congress.gov/bill/114thcongress/house-bill/2. Accessed: May 3, 2015. 3. Dorn SD. Gastroenterology in a new era of accountability: part 1 - an overview of performance measurement. Clin Gastroenterol Hepatol 2011;9:563–566. 4. Brook RH. The end of the quality improvement movement: long live improving value. JAMA 2010;304:1831–1832. 5. Institute of Medicine. Committee on Quality of Health Care. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001. 6. Medical Group Management Association. MGMA physician practice research: Medicare physician quality reporting programs not improving patient quality, needlessly complex. Available from: http://www.mgma.com/about/mgma-press-room/pressreleases/2007-2012/physician-practice-assessment-medicarequality-reporting-programs-research. Accessed: April 18, 2015. 7. Pink DH. Drive: the surprising truth about what motivates us. New York, NY: Penguin, 2011. 8. Woolhandler S, Ariely D, Himmelstein DU. Why pay for performance may be incompatible with quality improvement. BMJ 2012;345:e5015.
Quality Measurement in Gastroenterology
3
9. Conrad DA, Perry L. Quality-based financial incentives in health care: can we improve quality by paying for it? Annu Rev Public Health 2009;30:357–371. 10. Moynihan D. The problem at the VA: ‘performance perversity’. LA Times June 1, 2014. 11. Houle SK, McAlister FA, Jackevicius CA, et al. Does performance-based remuneration for individual health care practitioners affect patient care?: a systematic review. Ann Intern Med 2012;157:889–899. 12. Eijkenaar F, Emmert M, Scheppach M, et al. Effects of pay for performance in health care: a systematic review of systematic reviews. Health Policy 2013;110:115–130. 13. Krumholz HM, Normand SL, Spertus JA, et al. Measuring performance for treating heart attacks and heart failure: the case for outcomes measurement. Health Aff (Millwood) 2007; 26:75–85. 14. Cassel CK, Conway PH, Delbanco SF, et al. Getting more performance from performance measurement. N Engl J Med 2014; 371:2145–2147. 15. Corley DA, Jensen CD, Marks AR. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc 2011;74:656–665. 16. Solberg LI, Mosser G, McDonald S. The three faces of performance measurement: improvement, accountability, and research. Jt Comm J Qual Improv 1997;23:135–147. 17. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf 2012;21:964–968. 18. Werner RM, McNutt R. A new strategy to improve quality: rewarding actions rather than measures. JAMA 2009; 301:1375–1377. 19. Goitein L. Virtual quality: the failure of public reporting and payfor-performance programs. JAMA Intern Med 2014; 174:1912–1913. 20. Stine N, Chokshi D. Multifactorial risk assessment for atherosclerotic cardiovascular disease–reply. JAMA 2015;313:972. 21. Darst JR, Newburger JW, Resch S, et al. Deciding without data. Congenit Heart Dis 2010;5:339–342. 22. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry 1988;25:25–36.
Reprint requests Address requests for reprints to: Spencer D. Dorn, MD, MPH, MHA, Division of Gastroenterology and Hepatology, University of North Carolina at Chapel Hill, CB 7080, Chapel Hill, North Carolina 27599-7080. e-mail: [email protected]; fax: (919) 966–8929. Acknowledgments The author thanks Robert Sandler, MD, MPH, for reviewing an earlier version of this manuscript. Conflicts of interest The author discloses no conflicts.
Quality Measurement in Gastroenterology: Confessions of a Realist Spencer D. Dorn Division of Gastroenterology and Hepatology, University of North Carolina School of Medicine, Chapel Hill, North Carolina Clinicians are required to report their performance on an ever-increasing number of quality measures. However, it is difficult to measure health care quality and it is unclear whether broadly applying accountability measures effectively improves care. This article considers these challenges and includes recommendations that may help gastroenterologists respond to demands for increased quality measurement. Keywords: Quality; Measurement; Accountability.
Not everything that can be counted counts, and not everything that counts can be counted. William Bruce Cameron, Informal Sociology, A Casual Introduction to Sociological Thinking n today’s digitized, measurement-oriented, datadriven world, payers increasingly are using quality measures to hold clinicians accountable for quality of care. The National Quality Measures Clearinghouse lists more than 2000 such measures and new measures are being developed at unprecedented speed.1 The recently enacted Medicare Access and CHIP Reauthorization Act of 2015 doubles down on quality measurement; by 2022, up to 9% of Medicare physician payments will be linked to performance on quality measures.2 The general idea is that measuring quality and linking it to financial incentives and consumer choice will drive improvement. But does it really work this way? The late comedian George Carlin said, “Scratch any cynic and you will find a disappointed idealist.” Not long ago I was fairly idealistic about health care quality measurement.3 However, with additional experience practicing gastroenterology, serving on local and national quality-related committees, and helping to lead a large academic practice, my thoughts and feelings have shifted. Yes, I recognize that the quality measurement movement has resulted in undeniable good. It has increased attention to quality, safety, and patient experience; driven clinicians, hospitals, and health systems to develop new competencies; helped emphasize the importance of avoiding unnecessary services; and highlighted that systems of care (not individual clinicians alone) determine quality and patient experiences.
I
Still, I now better appreciate how hard it is to measure health care quality—many metrics do not accurately reflect quality—and I question whether broadly doing so effectively improves care.4 Beyond this, I am acutely concerned about how much time, money, and attention widespread quality measurement consumes, as well as the potentially damaging effects it has on clinicians’ spirits. In short, and paraphrasing one famous Institute of Medicine5 report, between the quality measurement ideal and the quality measurement reality lies not just a gap but a chasm. In this article I consider the challenges of measuring health care quality, and then make several pragmatic recommendations that I hope will help gastroenterologists respond to demands for increased quality measurement.
Problems and Challenges Measurement Issues Quantifying health care quality is exceedingly difficult. Measures must be valid (ie, actually reflect the quality of care delivered) and reliable (ie, variation should reflect true differences in performance, rather than measurement error). When care stretches over time it can be difficult to determine the appropriate measurement time frame. Similarly, when care involves multiple clinicians it can be difficult to determine who gets credit for different processes and outcomes. Finally, it is impossible to adjust fully for socioeconomic status, genetic factors, and other patient characteristics that affect outcomes.3
Generalizability Quality measures exist for only a narrow and discrete portion of clinical care. However, how clinicians perform on these small, convenient, unrepresentative sets of measures often influences payment for all of their clinical activities, as well as how they appear on publically available quality reports. My adenoma detection rate is © 2015 by the AGA Institute 1542-3565/$36.00 http://dx.doi.org/10.1016/j.cgh.2015.07.033
2
Spencer D. Dorn
an important indicator of how well I perform screening colonoscopies, but it certainly does not reflect the quality of my overall practice.
Burden Quality reporting programs befuddle clinicians, reduce their efficiency, and consume too much of their time.6 Meanwhile, large health care systems spend millions of dollars each year measuring and reporting performance. Many of these resources would be better deployed elsewhere.
Demotivating Clinicians intrinsically are motivated. We seek to feel in control of our own professional lives (autonomy), we crave the opportunity to become better and better at what we do (mastery), and we yearn to contribute to something larger than our own selves (purpose).7 Tying quality measures to carrots and sticks can undermine motivation and worsen performance, especially for the vast majority of clinical activities that go unmeasured.8 Of course, clinicians also are motivated extrinsically. Ideally, financial incentives would align with medical professionalism (eg, improved patient outcomes) to complement, rather than undermine, clinicians’ intrinsic motivation.9
Performance Perversity When placed under strong external pressure, individuals sometimes will cut corners to achieve unrealistic performance targets. In health care this may include sham improvements that satisfy a measure without improving patient care (eg, distributing smoking cessation booklets to all smokers), cream skimming (ie, avoiding patients who are most likely to suffer poor outcomes), coding trickery, and even outright fraud (eg, the recent Veteran’s Affairs wait list scandal).10
Limited Effectiveness There is shockingly little evidence that measuring and externally reporting quality actually improves care.4,11,12 One reason is that it typically is hard to draw a straight line between health care and health outcomes, especially over short time horizons. Consequently, quality measures often focus on relatively obscure processes (eg, immunization rates) that only weakly are linked with health outcomes.13 A second reason is that measurement only leads to improvement in organizations that have a culture of accountability and a workforce skilled in quality improvement activities.14 Organizations without these capabilities are merely checking boxes without improving care. For instance, measuring and feeding back adenoma detection rates may not, on its own,
Clinical Gastroenterology and Hepatology Vol.
-,
No.
-
meaningfully increase adenoma detection rates.15 Finally, the types of quality measures used for accountability purposes are not useful for improvement. This is because accountability measures are too far downstream to explain how processes might be changed to improve outcomes. In addition, to ensure reliability, accountability measures must be collected over periods of time that are too long to be useful for rapid cycle improvement efforts.16
Barriers to Real Improvement Externally imposed, top-down quality programs actually may get in the way of real improvement. These programs sap clinician enthusiasm and siphon critical resources, leaving scant resources to support the internal, bottom-up programs that seem more effective.17 Furthermore, imposing external solutions discourages organizations from taking their own innovative approaches to improve quality, which, after all, is determined locally.18 Consequently, quality measurement is often an end in itself rather than a means to better care.19
Recommendations Recognize the Need for Quality Measurement If we do not measure quality then our services will become commoditized and payers will focus on costs alone. Therefore, despite the many challenges discussed earlier, gastroenterologists still should embrace and report valid, reliable, and clinically relevant measures. For many independent practices the easiest option is to join a quality registry that interfaces with endoscopy report writers, even if that means reporting some potentially invalid or unreliable measures simply to avoid penalties.
Resist Meaningless Measures and Push for Better Measures Gastroenterology societies should push back against invalid or meaningless measures and push for more relevant ones. For instance, when evidence-based guidelines changed, internists successfully lobbied to replace 3 quality measures that addressed low-density lipoprotein levels with more valid measures that assess whether high-risk patients are prescribed statins.20
Supplement Quantitative Measures With Qualitative Assessments It will never be possible to fully quantify the quality of all the care gastroenterologists provide. Health care is too varied, human judgment is too nuanced, and most medical decisions are not based on clear evidence.21 A limited set of sound quantitative measures therefore should be supplemented with qualitative assessments of
-
2015
care, preferably provided by patients because they are in the best position to evaluate the care they personally receive.22
Quality Measurement Judo Gastroenterology practices should leverage the time, attention, personnel, and other resources demanded by externally mandated quality programs to improve quality, however; they, not payers or regulators, define it. The focus should be on enhancing systems of care in ways that empower clinicians and better meet patient needs. For instance, our practice has developed systems to ensure our patients are scheduled for both clinic visits and open-access endoscopic procedures with the right physician, at the right time, and at the right practice location. We currently are working to better ensure our patients reliably receive their gastrointestinal pathology results, along with clear and appropriate follow-up recommendations. These sorts of things may not affect our performance on externally imposed quality measures, but they matter to our patients and therefore are important to us. Ultimately, is that not what quality is all about?
References 1. Panzer RJ, Gitomer RS, Greene WH, et al. Increasing demands for quality measurement. JAMA 2013;310:1971–1980. 2. HR 2. Medicare Access and CHIP Reauthorization Act. 114th Congress. Available from: https://www.congress.gov/bill/114thcongress/house-bill/2. Accessed: May 3, 2015. 3. Dorn SD. Gastroenterology in a new era of accountability: part 1 - an overview of performance measurement. Clin Gastroenterol Hepatol 2011;9:563–566. 4. Brook RH. The end of the quality improvement movement: long live improving value. JAMA 2010;304:1831–1832. 5. Institute of Medicine. Committee on Quality of Health Care. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001. 6. Medical Group Management Association. MGMA physician practice research: Medicare physician quality reporting programs not improving patient quality, needlessly complex. Available from: http://www.mgma.com/about/mgma-press-room/pressreleases/2007-2012/physician-practice-assessment-medicarequality-reporting-programs-research. Accessed: April 18, 2015. 7. Pink DH. Drive: the surprising truth about what motivates us. New York, NY: Penguin, 2011. 8. Woolhandler S, Ariely D, Himmelstein DU. Why pay for performance may be incompatible with quality improvement. BMJ 2012;345:e5015.
Quality Measurement in Gastroenterology
3
9. Conrad DA, Perry L. Quality-based financial incentives in health care: can we improve quality by paying for it? Annu Rev Public Health 2009;30:357–371. 10. Moynihan D. The problem at the VA: ‘performance perversity’. LA Times June 1, 2014. 11. Houle SK, McAlister FA, Jackevicius CA, et al. Does performance-based remuneration for individual health care practitioners affect patient care?: a systematic review. Ann Intern Med 2012;157:889–899. 12. Eijkenaar F, Emmert M, Scheppach M, et al. Effects of pay for performance in health care: a systematic review of systematic reviews. Health Policy 2013;110:115–130. 13. Krumholz HM, Normand SL, Spertus JA, et al. Measuring performance for treating heart attacks and heart failure: the case for outcomes measurement. Health Aff (Millwood) 2007; 26:75–85. 14. Cassel CK, Conway PH, Delbanco SF, et al. Getting more performance from performance measurement. N Engl J Med 2014; 371:2145–2147. 15. Corley DA, Jensen CD, Marks AR. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc 2011;74:656–665. 16. Solberg LI, Mosser G, McDonald S. The three faces of performance measurement: improvement, accountability, and research. Jt Comm J Qual Improv 1997;23:135–147. 17. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf 2012;21:964–968. 18. Werner RM, McNutt R. A new strategy to improve quality: rewarding actions rather than measures. JAMA 2009; 301:1375–1377. 19. Goitein L. Virtual quality: the failure of public reporting and payfor-performance programs. JAMA Intern Med 2014; 174:1912–1913. 20. Stine N, Chokshi D. Multifactorial risk assessment for atherosclerotic cardiovascular disease–reply. JAMA 2015;313:972. 21. Darst JR, Newburger JW, Resch S, et al. Deciding without data. Congenit Heart Dis 2010;5:339–342. 22. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry 1988;25:25–36.
Reprint requests Address requests for reprints to: Spencer D. Dorn, MD, MPH, MHA, Division of Gastroenterology and Hepatology, University of North Carolina at Chapel Hill, CB 7080, Chapel Hill, North Carolina 27599-7080. e-mail: [email protected]; fax: (919) 966–8929. Acknowledgments The author thanks Robert Sandler, MD, MPH, for reviewing an earlier version of this manuscript. Conflicts of interest The author discloses no conflicts.
