Special Series: Quality Care Symposium
Perspective
The Measure of a Physician, the Measure of Our Practices: What We Measure Reflects What We Believe Vanderbilt-Ingram Cancer Center, Nashville, TN
As Crash Davis said in Bull Durham, “Well, I believe in the soul . . . the hanging curve ball, high fiber, good scotch, that the novels of Susan Sontag are self-indulgent, overrated crap. I believe Lee Harvey Oswald acted alone. I believe there ought to be a constitutional amendment outlawing Astroturf and the designated hitter. I believe in . . . opening your presents Christmas morning rather than Christmas Eve . . ..”1 And what do most medical oncologists practicing in 2014 believe? We believe that measurement promotes quality through the identification of performance gaps, allowing us to focus our energy to promote improvement where it is most needed. We believe that our treatments are evidence based for most of what we do and that by measuring what we do, we can judge the quality of our work. We believe that although variation is generally not random, because small sample sizes can lead to erroneous conclusions, we need to overcome chance with our methods and sample size. We believe that in the evaluation of quality, outcomes count more than process or the intent behind our actions. We believe that we can construct systems of measurement that neither distract us from our patients nor incur significant expense. We believe the discussion should focus on the quantity and nature of the things we do, not on the cost of those things alone. We believe that most of what others ask us to measure is distracting and a so-called unpaid mandate. And we believe that our patients can die without wasting resources on futile treatments, in circumstances that respect both their wishes and those of their families without anyone looking over anyone’s shoulders. But we have lots of questions. Does measurement only matter when it shows an opportunity for improvement? Despite showing a high baseline level of achievement, some measures remain important. The American Society of Clinical Oncology (ASCO) Quality Oncology Practice Initiative (QOPI) includes measures evaluating the use of adjuvant chemotherapy appropriately in subsets of patients with lung, colorectal, or breast cancer. Aggregate performance rates in QOPI-participating practices have never been below 90%.2 Does this mean these measures are not of continuing importance in these high-performing practices? When variation is seen (and it has been seen, because the overall performance on these measures is a threshold requirement for QOPI certification, which must be achieved), lower scores are often associated with lower numbers of patients reported by the practice—sometimes referred to as the small n problem. Although we all know that the effect of random variation is amplified by small numbers,3 how do we Copyright © 2014 by American Society of Clinical Oncology
M A Y 2014
account for this in national quality-reporting systems? If we simply exclude those who report on small numbers, does that put individual patients with rare conditions at risk? Does that discourage small practices from treating unusual conditions? If we are committed to only measuring improvable things, how do we fairly reward and recognize those who have led the way with high achievement and early adoption of measurement? And some questions are even less straightforward. How do we decide what proportion of care should be based on level-one evidence? How should we aggregate measures with differing levels of evidence in the composite quality scores? Although the evolution of chemotherapy in cancer care is often based on prospective randomized clinical trials, surgical evolution, which recently has reflected breakthroughs in technology, is not. Will (and should) there ever be a randomized trial comparing stitches with staples or tissue glue? Can we ever examine the specific choice of anticoagulants or antibiotics used in perioperative hospital quality measures? If the equivalent of imatinib for the BCR-ABL mutation in chronic myelogenous leukemia were invented to target KRAS mutations in other cancers, would we insist on a randomized trial to demonstrate benefit if six of 10 patients treated in phase II trials entered complete remission? How should we change the way we think about phase III trials?4 When we talk about cost, it seems we are talking about quantities or services and not the individual expense of things. However, comparisons of the cost of routine care demonstrate it to be significantly higher in the United States as compared with other industrialized countries, both on an absolute basis and one of relative pricing. For example, the average hospital discharge in 2008 was associated with a cost of $18,142, as compared with the median $6,222 of other Organisation for Economic Co-operation and Development nations and with the cost in Canada of $13,483. This is not restricted to hospital costs, where services included in the hospital stay may vary. Prescription drug expenses, payment for primary care office visits, and surgical professional fees are all similarly higher in the United States.5 In oncology, costs are particularly high, whether we are discussing screening for lung cancer, diagnostic care, or surgical care, and they are particularly high when considering both oral and parenteral drugs. And who would not pay almost anything to hold cancer at bay? How should we determine a fair price for a drug that maintains remission of a deadly disease yet requires daily administration? Futile treatments, particularly those near the end of life, are considered wasteful, but unless we •
jop.ascopubs.org
221
Information downloaded from jop.ascopubs.org and provided by at North Carolina State University Libraries on March 6, 2015 from 152.14.136.77 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
By Michael N. Neuss, MD
Michael N. Neuss
liefs, preferences, and comorbidities. We should start to think about not just what and how much we do but also about whether the price per unit of service can be decreased. We should always remember that the primary goal is not cost containment but rather delivery of appropriate care to each of our patients. We should not expect this to be easy. Author’s Disclosures of Potential Conflicts of Interest Although all authors completed the disclosure declaration, the following author(s) and/or an author’s immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO’s conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors. Employment or Leadership Position: Michael N. Neuss, ASCO Quality of Care Committee and QOPI Steering Committee (U) Consultant or Advisory Role: None Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None Corresponding author: Michael N. Neuss, MD, 694 Preston Research Building, Pierce Ave, Nashville, TN 37232; e-mail: [email protected].
DOI: 10.1200/JOP.2014.001410
References 1. Shelton R: Bull Durham [film]. Orion Pictures, 1988 2. Neuss MN, Malin JL, Chan S, et al: Measuring the improving quality of outpatient care in medical oncology practices in the United States. J Clin Oncol 31: 1471-1477, 2013 3. Hofer TP, Hayward RA, Greenfield S, et al: The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA 281:2098-2105, 1999 4. Roberts TG Jr, Lynch TJ Jr, Chabner BA: The phase III trial in the era of targeted therapy: Unraveling the “go or no go” decision. J Clin Oncol 21:36833695, 2003
222
JOURNAL
OF
ONCOLOGY PRACTICE
•
5. Squires DA: The US health system in perspective: A comparison of twelve industrialized nations. http://www.commonwealthfund.org/Publications/IssueBriefs/2011/Jul/US-Health-System-in-Perspective.aspx 6. Bach PB, Schrag D, Begg CB: Resurrecting treatment histories of dead patients: A study design that should be laid to rest. JAMA 292:2765-2770, 2004 7. Schnipper LE, Smith TJ, Raghavan D, et al: American Society of Clinical Oncology identifies five key opportunities to improve care and reduce costs: The top five list for oncology. J Clin Oncol 30:1715-1724, 2012 8. Schnipper LE, Lyman GH, Blayney DW, et al: American Society of Clinical Oncology 2013 top five list in oncology. J Clin Oncol 34:4362-4370, 2013
V O L . 10, I S S U E 3
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at North Carolina State University Libraries on March 6, 2015 from 152.14.136.77 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
can predict with some high reliability when the end of life will be, how can we know when drug treatments will be wasted?6 Ultimately, is all this measuring really helping us to be better physicians? I know I need to wash my hands when I am going into a patient’s room, but when I see the hand-washing monitor, I am less likely to just peak around the corner and say a quick hello, fearing that will be counted as going into the patient’s room without washing my hands. I have a finite amount of time; when I have to record the details of my discussion about consent for chemotherapy, that is time I lose to spend on the conversation itself. If I carefully, compassionately, and patiently do my best to explain to a patient that hospital care will not help him or her live longer or better than hospice care, and he or she says, “I just don’t feel comfortable in that hospice and won’t go,” is the patient’s decision really a measure of our care? More importantly, should our patients, their families, and we their physicians be worrying about what things cost at this moment, when the focus should be on compassion and communication and not simply the location where all this is happening? Although the Choosing Wisely campaign and the ASCO top five in particular help propel the conversation,7,8 we need to go further and talk about much more than just the finances. These are complicated and important questions filled with subtlety and nuance. There is a hierarchy of importance. We cannot charge forward with measures, incentives, penalties, and public reporting without first understanding how we allow for the appropriate variation of care as determined by patient be-
Perspective
The Measure of a Physician, the Measure of Our Practices: What We Measure Reflects What We Believe Vanderbilt-Ingram Cancer Center, Nashville, TN
As Crash Davis said in Bull Durham, “Well, I believe in the soul . . . the hanging curve ball, high fiber, good scotch, that the novels of Susan Sontag are self-indulgent, overrated crap. I believe Lee Harvey Oswald acted alone. I believe there ought to be a constitutional amendment outlawing Astroturf and the designated hitter. I believe in . . . opening your presents Christmas morning rather than Christmas Eve . . ..”1 And what do most medical oncologists practicing in 2014 believe? We believe that measurement promotes quality through the identification of performance gaps, allowing us to focus our energy to promote improvement where it is most needed. We believe that our treatments are evidence based for most of what we do and that by measuring what we do, we can judge the quality of our work. We believe that although variation is generally not random, because small sample sizes can lead to erroneous conclusions, we need to overcome chance with our methods and sample size. We believe that in the evaluation of quality, outcomes count more than process or the intent behind our actions. We believe that we can construct systems of measurement that neither distract us from our patients nor incur significant expense. We believe the discussion should focus on the quantity and nature of the things we do, not on the cost of those things alone. We believe that most of what others ask us to measure is distracting and a so-called unpaid mandate. And we believe that our patients can die without wasting resources on futile treatments, in circumstances that respect both their wishes and those of their families without anyone looking over anyone’s shoulders. But we have lots of questions. Does measurement only matter when it shows an opportunity for improvement? Despite showing a high baseline level of achievement, some measures remain important. The American Society of Clinical Oncology (ASCO) Quality Oncology Practice Initiative (QOPI) includes measures evaluating the use of adjuvant chemotherapy appropriately in subsets of patients with lung, colorectal, or breast cancer. Aggregate performance rates in QOPI-participating practices have never been below 90%.2 Does this mean these measures are not of continuing importance in these high-performing practices? When variation is seen (and it has been seen, because the overall performance on these measures is a threshold requirement for QOPI certification, which must be achieved), lower scores are often associated with lower numbers of patients reported by the practice—sometimes referred to as the small n problem. Although we all know that the effect of random variation is amplified by small numbers,3 how do we Copyright © 2014 by American Society of Clinical Oncology
M A Y 2014
account for this in national quality-reporting systems? If we simply exclude those who report on small numbers, does that put individual patients with rare conditions at risk? Does that discourage small practices from treating unusual conditions? If we are committed to only measuring improvable things, how do we fairly reward and recognize those who have led the way with high achievement and early adoption of measurement? And some questions are even less straightforward. How do we decide what proportion of care should be based on level-one evidence? How should we aggregate measures with differing levels of evidence in the composite quality scores? Although the evolution of chemotherapy in cancer care is often based on prospective randomized clinical trials, surgical evolution, which recently has reflected breakthroughs in technology, is not. Will (and should) there ever be a randomized trial comparing stitches with staples or tissue glue? Can we ever examine the specific choice of anticoagulants or antibiotics used in perioperative hospital quality measures? If the equivalent of imatinib for the BCR-ABL mutation in chronic myelogenous leukemia were invented to target KRAS mutations in other cancers, would we insist on a randomized trial to demonstrate benefit if six of 10 patients treated in phase II trials entered complete remission? How should we change the way we think about phase III trials?4 When we talk about cost, it seems we are talking about quantities or services and not the individual expense of things. However, comparisons of the cost of routine care demonstrate it to be significantly higher in the United States as compared with other industrialized countries, both on an absolute basis and one of relative pricing. For example, the average hospital discharge in 2008 was associated with a cost of $18,142, as compared with the median $6,222 of other Organisation for Economic Co-operation and Development nations and with the cost in Canada of $13,483. This is not restricted to hospital costs, where services included in the hospital stay may vary. Prescription drug expenses, payment for primary care office visits, and surgical professional fees are all similarly higher in the United States.5 In oncology, costs are particularly high, whether we are discussing screening for lung cancer, diagnostic care, or surgical care, and they are particularly high when considering both oral and parenteral drugs. And who would not pay almost anything to hold cancer at bay? How should we determine a fair price for a drug that maintains remission of a deadly disease yet requires daily administration? Futile treatments, particularly those near the end of life, are considered wasteful, but unless we •
jop.ascopubs.org
221
Information downloaded from jop.ascopubs.org and provided by at North Carolina State University Libraries on March 6, 2015 from 152.14.136.77 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
By Michael N. Neuss, MD
Michael N. Neuss
liefs, preferences, and comorbidities. We should start to think about not just what and how much we do but also about whether the price per unit of service can be decreased. We should always remember that the primary goal is not cost containment but rather delivery of appropriate care to each of our patients. We should not expect this to be easy. Author’s Disclosures of Potential Conflicts of Interest Although all authors completed the disclosure declaration, the following author(s) and/or an author’s immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO’s conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors. Employment or Leadership Position: Michael N. Neuss, ASCO Quality of Care Committee and QOPI Steering Committee (U) Consultant or Advisory Role: None Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None Corresponding author: Michael N. Neuss, MD, 694 Preston Research Building, Pierce Ave, Nashville, TN 37232; e-mail: [email protected].
DOI: 10.1200/JOP.2014.001410
References 1. Shelton R: Bull Durham [film]. Orion Pictures, 1988 2. Neuss MN, Malin JL, Chan S, et al: Measuring the improving quality of outpatient care in medical oncology practices in the United States. J Clin Oncol 31: 1471-1477, 2013 3. Hofer TP, Hayward RA, Greenfield S, et al: The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA 281:2098-2105, 1999 4. Roberts TG Jr, Lynch TJ Jr, Chabner BA: The phase III trial in the era of targeted therapy: Unraveling the “go or no go” decision. J Clin Oncol 21:36833695, 2003
222
JOURNAL
OF
ONCOLOGY PRACTICE
•
5. Squires DA: The US health system in perspective: A comparison of twelve industrialized nations. http://www.commonwealthfund.org/Publications/IssueBriefs/2011/Jul/US-Health-System-in-Perspective.aspx 6. Bach PB, Schrag D, Begg CB: Resurrecting treatment histories of dead patients: A study design that should be laid to rest. JAMA 292:2765-2770, 2004 7. Schnipper LE, Smith TJ, Raghavan D, et al: American Society of Clinical Oncology identifies five key opportunities to improve care and reduce costs: The top five list for oncology. J Clin Oncol 30:1715-1724, 2012 8. Schnipper LE, Lyman GH, Blayney DW, et al: American Society of Clinical Oncology 2013 top five list in oncology. J Clin Oncol 34:4362-4370, 2013
V O L . 10, I S S U E 3
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at North Carolina State University Libraries on March 6, 2015 from 152.14.136.77 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
can predict with some high reliability when the end of life will be, how can we know when drug treatments will be wasted?6 Ultimately, is all this measuring really helping us to be better physicians? I know I need to wash my hands when I am going into a patient’s room, but when I see the hand-washing monitor, I am less likely to just peak around the corner and say a quick hello, fearing that will be counted as going into the patient’s room without washing my hands. I have a finite amount of time; when I have to record the details of my discussion about consent for chemotherapy, that is time I lose to spend on the conversation itself. If I carefully, compassionately, and patiently do my best to explain to a patient that hospital care will not help him or her live longer or better than hospice care, and he or she says, “I just don’t feel comfortable in that hospice and won’t go,” is the patient’s decision really a measure of our care? More importantly, should our patients, their families, and we their physicians be worrying about what things cost at this moment, when the focus should be on compassion and communication and not simply the location where all this is happening? Although the Choosing Wisely campaign and the ASCO top five in particular help propel the conversation,7,8 we need to go further and talk about much more than just the finances. These are complicated and important questions filled with subtlety and nuance. There is a hierarchy of importance. We cannot charge forward with measures, incentives, penalties, and public reporting without first understanding how we allow for the appropriate variation of care as determined by patient be-
