Decision Analysis and Cost-Effectiveness Analysis for Comparative Effectiveness Research—A Primer David J. Sher, MD, MPH,* and Rinaa S. Punglia, MD, MPH† Although the analysis of real-world data is the foundation of comparative effectiveness analysis, not all clinical questions are easily approached with patient-derived information. Decision analysis is a set of modeling and analytic tools that simulate treatment and disease processes, including the incorporation of patient preferences, thus generating optimal treatment strategies for varying patient, disease, and treatment conditions. Although decision analysis is informed by evidence-derived outcomes, its ability to test treatment strategies under different conditions that are realistic but not necessarily reported in the literature makes it a useful and complementary technique to more standard data analysis. Similarly, costeffectiveness analysis is a discipline in which the relative costs and benefits of treatment alternatives are rigorously compared. With the well-recognized increase in highly technical, costly radiation therapy technologies, the cost-effectiveness of these different treatments would come under progressively more scrutiny. In this review, we discuss the theoretical and practical aspects of decision analysis and cost-effectiveness analysis, providing examples that highlight their methodology and utility. Semin Radiat Oncol 24:14-24 C 2014 Elsevier Inc. All rights reserved.
Introduction
T
he Institute of Medicine (IOM) published 6 defining characteristics of methodology for comparative effectiveness research in 2009 (IOM Initial National Priorities for Comparative Effectiveness Research, 2009): (1) Has the objective of directly informing a specific clinical decision from the patient perspective or a health policy decision from the population perspective. (2) Compares at least 2 alternative interventions, each with the potential to be “best practice.” (3) Describes results and the population of subgroup levels. (4) Measures outcomes—both benefits and harms—that are important to patients. (5) Employs methods and data sources appropriate for the decision of interest.
*Department of Radiation Oncology, Rush University Medical Center, Chicago, IL. †Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women's Hospital, Boston, MA. The authors declare no conflict of interest. Address reprint requests to David J. Sher, MD, MPH, Rush University Medical Center, Department of Radiation Oncology, Chicago, IL. E-mail: [email protected]
14
1053-4296/13/$-see front matter & 2014 Elsevier Inc. All rights reserved. doi:http://dx.doi.org/10.1016/j.semradonc.2013.08.002
(6) Is conducted in settings that are similar to those in which the intervention would be used in practice. There are many well-known clinical trial, registry, and large database techniques and tools that classically conform to this definition. Randomized clinical trials—explanatory or pragmatic—serve as the backbone for obtaining level I evidence, and the importance of discerning real-world outcomes has raised the stature of both retrospective data mining and prospective registry studies. Yet, there are inherent limitations to these methodologies, as detailed in the article by Meyer et al.1 Randomized trials help guide decisions but require a long period for follow-up, especially when studying diseases that may have a long clinical history, before treatment differences can be discerned. Therefore, data from such studies may not have matured until the treatment itself is obsolete. Retrospective or even prospective large database analyses often do not account for key patient characteristics, either personal or disease related, that may play a key role in determining the optimal therapy for a given patient. Indeed, 1 defining characteristic of all of these methodologies is their reliance on actual patient outcomes and the ultimate reporting of a single preferred treatment or strategy for the “average person.” These studies consequently have no mechanism of incorporating
Decision and cost-effectiveness analysis toxicity and patient preferences into the determination of the best treatment strategy. For a given patient, the best decision for him or her may hinge on his or her own personal, disease, or treatment characteristics, including treatment efficacy or toxicity probabilities that cannot be specified within a clinical trial or retrospective study. The decision to add radiation therapy for patients is a personal one and dependent on an individual's specific set of clinical factors and personal preferences. Optimal decision making requires careful consideration of the potential risks and expected outcomes of radiation therapy. As implied by the name, decision analysis is a set of mathematical tools whose goal is to determine the optimal outcome (whether it is the best treatment or imaging approach, or surveillance strategy, etc) for a given decision under a widely varying set of conditions. Although randomized trials and retrospective studies use real-patient data and biostatistical tools to determine the efficacy or effectiveness of a treatment, decision analysis wields finely tuned mathematical models to simulate the treatment and disease process and thus determine “winning” strategies based on computation. These models are entirely informed by the literature and thus hinge on prospective and retrospective evidence, yet the fact that they are ultimately mathematical models liberates the analyst to test different hypotheses on the importance of patient, treatment, and disease characteristics not possible in patient-based, realworld data. Thus, decision analysis provides a mechanism with which to integrate the available data to compare the outcomes of different treatment strategies. Indeed, decision analysis is perhaps the perfect means to satisfy each of the IOM's criteria for comparative effectiveness research. The motivation for comparative effectiveness research goes beyond which treatment decision is the right one to whether a decision is right in the context of limited resources. The escalating costs of American health care are inescapable. In 2011, the United States spent $2.6 trillion on health care, which accounts for nearly one-fifth of the economy. This figure amounts to approximately $8000 per person, which is $3000 more than the next country and twice the average for developed countries. Since 1985, health spending has increased 2% per year faster than gross domestic product (GDP), such that in 2010, medical spending amounted to nearly 18% of GDP.2 Cancer care comprises a significant amount of the absolute health care expenditure as well as its growth. In 2010, $124.6 billion were spent in oncology; antineoplastics were the leading class of hospital drug expenditures, and biological agents accounted for 15% of all prescription drug costs.3 Moreover, without any change in costs, population changes alone will increase oncology spending by 27% in the year 2020, and if costs increase by 5% per year, the costs from cancer care alone will top $200 billion.4 It is actually very difficult to know the extent to which radiotherapy (RT) services contribute to accelerating costs. Unfortunately, the most recent data analyzing oncology expenditures lump all treatments together by phase of care (initial, maintenance, and end-of-life), without specifying the treatments rendered. In a interesting study presented at the
15 American Society for Radiation Oncology National Meeting in 2011, Shen et al5 showed that the total Medicare payments for external beam RT increased 322% between 2000 and 2009, from $256 million to $1.08 billion, with most that increase attributable to intensity-modulated radiation therapy (IMRT) reimbursement. Despite the absence of published data on this topic, however, with the advent of multimillion dollar proton facilities and progressively more common implementation of IMRT and stereotactic RT, our community recognition of rising RT-related costs echoes the famous quote by Supreme Court Justice Potter Stewart: “I know it when I see it.”
Decision Analysis Motivation for Decision Analysis Decision analytic techniques have been used to inform decision making in situations of uncertainty. Decision analyses can delineate more clearly the trade-offs for individual clinical scenarios where there are gaps in existing data regarding treatment choice in a number of ways. First, they provide a framework to combine available data from randomized and retrospective trials to study questions for which no direct evidence is currently available. Decision analysis also provides a vehicle to extrapolate results for more recent trials to model outcomes while the decision to use a treatment is still relevant. Decision analyses are especially useful when treatment decisions need to be made for the patient of today, while data from additional studies specific to the patient's clinical characteristics may be forthcoming. Second, decision models can be used to tailor expected outcomes to an individual set of clinical circumstances. Results of decision models have been published using the available data from randomized and retrospective trials to model the life course of clinical subsets of patients diagnosed with breast cancer (for example, those patients with breast cancer with genetic mutations, node-negative disease, or disease responsive to a particular systemic agent) to study longterm outcomes and inform current treatment choice.6-8 Finally, decision analyses can model a number of different outcomes, such as overall survival or time without recurrence, allowing estimates to reflect the outcome most relevant to an individual patient.
Methodology of Decision Analysis Structuring the Decision A decision model or decision tree is created to represent the clinical choice. The model needs to be simple enough to be understood and feasible to do. To do this, a model needs simplifying assumptions, but at the same time, a model needs to be complex enough to describe accurately the features of the decision. Decision nodes are represented by squares in the decision tree, chance nodes by circles, and terminal nodes by triangles. Figure 1 represents a decision in a hypothetical situation where a patient is confronted with the following treatment dilemma regarding whether to undergo radiation therapy or not for illness X. Radiation therapy for illness X is toxic and
D.J. Sher and R.S. Punglia
16
Figure 1 Diagram of a simple decision tree of a patient with illness X. This model represents a decision in a hypothetical situation where a patient is confronted with the following treatment dilemma regarding whether to undergo radiation therapy or not for illness X. Radiation therapy for illness X is toxic, and results in a risk of 10% chance of death. However, if a patient with illness X survives the treatment, they have an 80% chance of remaining alive. If they do not undergo radiation, they have a 70% chance of remaining alive. Tree constructed with TreeAge Pro (Williamstown, MA). Calculating expected result from each option: Starting with the right side of the last chance node for radiation therapy, we find that if a patient survives treatment, they have an 80% chance of living and garnering an outcome of 1, and a 20% chance of dying and garnering an outcome of 0. The expected value at this chance node is derived by multiplying the probability of the outcome with the value associated with it, which in our case gives us a value of 0.80. Moving to the left of the decision tree, we then substitute the value of 0.80 as the value associated with surviving treatment and combine this value and its probability of 0.90 with that of dying from treatment (outcome of 0 with probability of 0.10) to derive an expected value for radiation therapy of 0.72. The expected value for no treatment is obtained from multiplying the likelihood of living (0.70) with its outcome of 1 and adding that to the product of the likelihood of dying (0.30) and its outcome of 0, for an expected value of 0.70. (Color version of figure is available online.)
results in a 10% risk of death. However, if a patient with illness X survives the treatment, they have an 80% chance of remaining alive. If they do not undergo radiation, they have a 70% chance of remaining alive. What treatment should a patient undergo? In the model, the decision of whether to undergo radiation or not is represented by the square. Here the choice is under the control of the decision maker or the patient with illness X. After choosing a treatment, there is a chance node to the right of which all possible events are represented as branches. The chance events are not in the control of the patient or decision maker. Instead, likelihood of events following a chance node is decided by probability or outcomes data from the literature or expert opinion if literature is not available. The probabilities of chance events at each chance node need to add to 1 or 100%. In our example (Fig. 1), if a patient with illness X chooses radiation therapy, he has a 10% chance of dying from the treatment and a 90% chance of surviving. If the patient survives, he then has an 80% chance of being alive and a 20% chance of dying. If a patient chooses no treatment, he has a 70% chance of living and a 30% chance of dying. The terminal nodes shown with triangles in our example of illness X (Fig. 1) represent the final outcomes states associated with each of the possible chance pathways. In our hypothetical example, the only 2 outcomes are living or dying. Along with each of the terminal nodes is a utility or value assigned to that final state. In our simple model, being alive is associated with a value of 1 and dead is 0. However, decision analyses provide
the opportunity to incorporate a number of intermediate values to inform a treatment decision as described next.
Outcome Values Although one outcome can be likelihood of survival, decision analysis can describe the value assigned to outcome states in terms of utilities. This ability to describe utilities for different outcomes states allows modelers to incorporate side effects of treatments into their model, an important consideration given that many cancer treatments affect a patient's quality of life. A utility for a given health state is the “value” ascribed to that health state similar to its quality of life. Stiggelbout and de Haes9 define utility as, The values that subjects attach to . . . possible health states are called the utilities of the health state. A utility is defined as the level of desirability that people associate with a particular outcome. It is a cardinal number that represents the strength of an individual's preference for a particular outcome when faced with uncertainty. Utilities are assigned to each outcome on a scale that is established by assigning a value of 1 to the state of optimal health and a value of 0 to death. Although some authors use utility scores as qualityof-life scores, utility is a concept that in fact is essentially different for quality of life per se. Utilities reflect both the quality of life and the value of that quality of life, relative to death and optimal health.
Decision and cost-effectiveness analysis How does one assess a utility for a health state? These methodologies are beyond the scope of this chapter, but the most standard pathway is to use a “standard gamble” or “time trade-off” techniques to measure the value of the health state when there is uncertainty. The key in either instrument is rooting the answers within the framework of death (utility value 0) and perfect health (value 1), which preserves important mathematical characteristics of the value. An example of the time trade-off technique is asking how much time in perfect health would be the equivalent value to you in this suboptimal health state (eg, “if you are 35 years old now and are going to live until 85 with a mastectomy, how many fewer years would you live if you were able to preserve the breast?”). Whose utilities should go into the model being built? Most often healthy subjects or patients not in the certain state being studied are used to assess utilities. But utilities can vary widely within the group from which they are derived, and there is considerable literature describing the adaptation phenomena of utilities in that patients in a particular state ascribe a higher utility to it than those not in the compromised state. Therefore, the choice of source for utilities should match the purpose of the decision analysis. Is this a model to inform policy decisions? In which case, average population values could be used. Or is this a model describing an individual patient decision? In which case, patient values could be used. In all cases, the outcome measure needs to be consistent across terminal nodes of a given model. The ability to measure utilities allows one to describe time outcomes in terms of quality-adjusted life years (QALYs). A QALY is determined by multiplying the time spent in a certain state with the utility for a health state relative to 1 year in perfect health. For example, 6 months of time in a state whose utility is ascribed to be 0.5 would be the equivalent of 0.25 QALYs. Calculating the Expected Outcome The final step in a decision analysis is evaluating the expected utility or outcome of each treatment strategy. Going back to our hypothetical example of illness X (Fig. 1), we multiply the probabilities to calculate the expected result of each of the treatment choices, radiation therapy or no treatment. In this example, the expected value for radiation therapy is 0.72 and that for no treatment is 0.70. So the strategy that leads to the greatest expected value is radiation therapy. Sensitivity Analysis Probabilities at chance nodes and outcomes values are derived from published literature or expert opinion. Where there are few primary data or controversies in the literature, sensitivity analyses are useful in decision analytic models. Sensitivity analyses vary a certain parameter (eg, probability of developing a local or distant recurrence after having initial treatment) over the entire plausible range of its values, and then study the effect of varying the parameter on the given outcomes. Sensitivity analyses provide a measure of the robustness of a conclusion in a manner similar to confidence intervals and P values in biostatistics. They can confirm the lack of significant dependence of an outcome on a certain transition parameter and
17 increase the level of confidence or alternatively by revealing dependence of an outcome measure on the variable studied, may identify areas where further research may be valuable.10 In our example of illness X (Fig. 1), we may not know the exact likelihood of treatment mortality or it may vary by physician or with patient characteristics. We can conduct a 1-way sensitivity analysis (Fig. 2) to ask “what if” treatment mortality was y, then what would the optimal treatment strategy be. In our example, we find that as treatment mortality increases, the expected value for radiation therapy decreases and that the threshold value for mortality is 0.125. Below this mortality threshold, radiation therapy is associated with the best expected value, but more than this value, no treatment is the optimal strategy. One-way sensitivity analyses provide threshold values. Twoway and 3-way sensitivity analyses systematically vary 2-3 parameters on different axes, respectively. The outcomes for each combination of variables can then be assessed.
Markov Models Elements and Assumptions Markov models are useful when time needs to be incorporated into a decision model. These models study outcomes by describing events as discrete states. They were first applied for medical applications to predict prognosis in 1983 by Beck and Pauker11 who designed a method of describing patient outcomes by dividing them into discrete health states. Hypothetical “patients” start the model in a single health state. As the model cycles, the patients travel through time, and may remain in that health state or transition to a different health state. In the simplified breast cancer model stated later (Fig. 3), patients who have completed initial therapy start in the “No disease” health state and with each cycle may (1) remain in the “No disease” health state, (2) have a finite probability of having a recurrence and transition to the “Recurrence” health state, or (3) have a finite probability of dying of causes unrelated to their cancer and transition to the “Dead” health state. Once in the “Recurrence” health state, patients may remain there or with a certain probability die from their recurrent disease and enter the “Dead” health state. The “Dead” health state is an absorbing health state, in that once entered, patients may not exit. An absorbing model is that which is run until all patients are in the “Dead” health state and can be used to calculate residence times in transient states allowing for estimates such as life expectancy or time spent without recurrence. Markov models are particularly useful in modeling clinical situations where a patient is at risk for a given event over an extended period of time (eg, recurrence from breast cancer).12 Markov processes rely on a series of assumptions: A patient is always in one of a finite number of discrete health states. All events are represented in the model as transitions from one state to another. Time is divided into equal increments referred to as Markov cycles, which may be a finite number of days,
D.J. Sher and R.S. Punglia
18
Figure 2 Sensitivity analysis of mortality from treatment for illness X. This graph represents a 1-way sensitivity analysis of mortality associated with treatment. As mortality from treatment increases from 0-1 on the x-axis, the expected value for radiation therapy decreases. The threshold mortality at which radiation therapy and no treatment yield equivalent expected outcomes is 0.125. At mortality risks below this threshold value, radiation therapy is superior to no treatment; at risks above 0.125, no treatment is the preferred strategy. Tree constructed with TreeAge Pro (Williamstown, MA). (Color version of figure is available online.)
months, or years depending on the clinical scenario modeled. Transitions between health states may only occur once per cycle. There is no “memory” feature in a Markov model, so that all patients in a given health state face the same transition probabilities, irrespective of when they entered that health state.
Figure 3 This representation of a Markov model has 3 health states: no disease, recurrence, and distant metastases. All patients start in the no disease health state from which each cycle they can develop recurrence or distant disease with a specified probability. From the recurrence health state, patients can also enter the distant metastases health state, again with a certain probability. The distant metastases health state is an absorbing health state, in that once entered, a patient cannot exit.
Evaluation Methods Two common methods to evaluate a Markov model include cohort simulation and Monte Carlo simulation. With cohort simulation, a population or entire cohort is subjected to the decision model simultaneously and results are described at a population level (example, average life expectancy for the cohort). Monte Carlo simulation refers to a process by which simulated patients enter the model 1 at a time. The computer then performs a coin toss or randomly chooses a value for a variable in the decision analysis using a prespecified distribution and then tracks the results. The computer does this thousands of times, thus simulating many thousands of “patients,” and the mean and standard deviation for an outcome (eg, mean life expectancy) are calculated. The term Monte Carlo comes from gambling because as in gambling, the outcome depends on a random selection of values. Moreover, if the computer does it enough times or “plays the game” long enough, one can generate a distribution of the possible results. Of course, it is up to the designer of the decision model to not only pick the values but also the distribution of the values (bellshaped, triangular, or discrete cardinal), which require additional assumptions.
Case Example With the aforementioned context, we can evaluate a recent decision analysis designed to measure the quality-of-life effects of prostate-specific antigen (PSA) screening. Although screening can lead to earlier diagnosis of prostate cancer, it can also lead to overdiagnosis and overtreatment in patients who may not have prostate cancer or who have prostate cancer that would have never become clinically detectable. Therefore, the
Decision and cost-effectiveness analysis key question here is what are the trade-offs associated with screening and is it worth it. Heijnsdijk et al13 created a model based on data from the European Randomized Study of Screening for Prostate Cancer. From the screening study, they incorporated a 29% reduction in prostate cancer mortality with PSA screening. Their model was used to simulate individual life histories of men who underwent PSA screening varying the age cohort and screening strategy. They incorporated utilities for individual health states such as bowel dysfunction and urinary incontinence after treatment. Their results revealed that annual PSA screening of men between ages 55 and 69 years results in a 28% reduction of death from prostate cancer, which validates that their model is well calibrated to the European Randomized Study of Screening for Prostate Cancer trial. Annual screening resulted in 73 life years gained per 1000 men screened, but only 56 QALYs which incorporate the downsides to screening including the side effects of treatment. The authors found that their outcome of QALYs gained was very sensitive to the utility estimate used for the posttreatment recovery period. If no loss in utility during this period was used (1 instead of the base case 0.95 for utility), then the resulting outcome per 1000 men screened was 72 QALYs. However, if 0.93 was used for the utility, then only 6 QALYs were gained. This finding highlights the need for further research into the utility of the posttreatment period and also reveals that whether screening should be done may be a function of an individual's posttreatment utility. Such conclusions were only reachable using a decision analysis, which can incorporate the patient's health state evaluation into the analysis.
Cost-Effectiveness Analysis (CEA) Motivation for CEA Our national consciousness is continuously—and not necessarily inappropriately—assaulted with warnings about rising costs. We hear about them in popular media, in political debates, and in reimbursement decisions from insurance companies. What are less commonly discussed are actual solutions to these rising costs. Medicare has generally attempted blanket solutions for cost escalations, such as the sustainable growth rate formula, which simply links payments—not coverage decisions—to the GDP.14 In fact, the Center for Medicare and Medicaid is legally not allowed to consider cost-effectiveness of therapies, so the differential allocation of money (more or less) does not change health outcomes, just how much they cost. Private payers also rarely consider the cost-effectiveness of therapies in their coverage decisions. Cigna, for example, generally covers proton beam therapy (PBT) for prostate cancer because it is clinically equivalent, but not superior, to conventional treatments, and it does not consider cost or cost-effectiveness; however, in some Cigna-administered plans, in which “medical necessity” is mandated, PBT is not covered because it is more expensive and not proven to be more efficacious. This
19 latter ruling implies the consideration of treatment costs in some plans; it does not rely on CEA but rather absolute costs.15 Although the focus on cost minimization is understandable, the real issue is how do we maximize the health outcomes given whatever financial resources we have. That is, how do we get the most health care “bang” for our “buck.” The term “rationing” is taboo in American vernacular, but we do have a finite amount of health care dollars that need to be put into the best use. In other words, we need to ration the resources that we have. So simply blindly cutting down on the reimbursement for given services is far less useful for society than evaluating what interventions are the most cost-effective and focusing resources on those high-yield services. CEA is a robust field whose sole aim is to improve the value of health care interventions; that is, get the most bang for the buck. Although CEA is related to cost control in the sense that it allows us to prioritize treatments given limited resources, it is fundamentally a separate concept. In principle, we can spend less and achieve more, if the right interventions are chosen, or cynically, spend more and achieve less, if the wrong treatments are funded.
Incremental Cost-Effectiveness Ratio (ICER) At the end of any CEA, the result is presented as an ICER, or ICER. There are 2 components of this outcome, the term incremental and the term cost-effectiveness ratio. The term incremental refers to the fact that the cost-effectiveness of any intervention is stated in comparison to another, less effective intervention.16 The less effective treatment may be no treatment or it may be a rudimentary therapy or it may be a floridly expensive but ineffective treatment. In any case, an ICER for a given therapy is always relative to a different therapy, and thus the choice of alternatives is obviously critical. The term cost-effectiveness ratio refers to the form of the end point, which is a simple ratio of cost to effectiveness. The cost can be in any denomination, with dollars and euros as the most common in the medical literature. The effectiveness measure chosen by the analyst may also vary, from QALYs to life years to disability associated life year, though the standard metric is the QALY.17 As a result, the ICER is typically reported in a structure like $55,000 per QALY. It is noteworthy that when a CEA is done with the QALY or disability associated life year as the effectiveness measure, it is also termed a cost-utility analysis, as these 2 units are utilities.18 An intuition about the ICER can be obtained by understanding the “cost-effectiveness space,” which is a graph that outlines the 4 permutations of effectiveness and cost differences (Fig. 4) with a new therapy. The origin of this space is the current treatment or standard, and the new treatment's cost and effectiveness are then plotted relative to the standard treatment's cost and effectiveness. In the northwest space, the treatment is more costly and less effective, and this scenario is called “dominance” (ie, the new treatment is dominated by the standard treatment).16 Conversely, in the southeast space, the new treatment is more effective and less costly, and it dominates the standard treatment. The more common scenarios are seen in the northeast and southwest quadrants, in which
D.J. Sher and R.S. Punglia
20
Figure 4 Cost-effectiveness space. The relative difference between the costs and effectiveness of 2 strategies can be plotted on this graph, with the origin reflecting the standard therapy, and the X- and Y-coordinate as the relative difference between the 2. Dominance occurs in the Northwest and Southeast boxes. In the Northeast and Southwest boxes, the incremental cost-effectiveness ratio (ICER) is determined by the slope of the line. Any intervention to the right of the line has a superior ICER.
the new treatment is more effective and more costly or less effective and cheaper, respectively. In these cases, the slope of the line is equivalent to the ICER, and whether the treatment is considered cost-effective is dependent on the societal willingness to pay (WTP), which can be designated by a line with a slope equivalent to the WTP. Graphically, if the ICER falls to the right of the WTP line, the treatment is cost-effective.
The Magical $50,000 Per QALY Line Consider the following thought experiment. Every health intervention that a society or health plan may implement is assigned an ICER. How would you then maximize health care outcomes? Well, the solution is straightforward. One would start with funding the most cost-effective treatment (ie, lowest ICER or cost per QALY), and then fund the second most costeffective treatment, and continue on until there are no more health care dollars to spend. This approach—dubbed the “shopping-spree problem”—will maximize QALYs.16 Of course, there are fundamental concerns with this protocol. Simply summating QALYs without regard to who is receiving them raises concerns about equity and the potential for increasing health disparities. Similarly, society may want to fund some very cost-ineffective treatments for certain populations, such as children. Yet, on balance, this algorithm does provide a mechanism to optimize the value of health care dollars. In practice, though, no payer is going to order every health care intervention by ICER and then allocate money in that order. Instead, we typically consider interventions as “costeffective” or not by the absolute ICER and whether it passes a certain threshold, which is called societal WTP. In some cases, the specific threshold is not particularly relevant: an ICER of $500,000 per QALY is never going to be cost-effective and an ICER of $5000 per QALY is. Yet, many interventions hover somewhere in between: what defines cost-effectiveness?
For better or for worse, $50,000 per QALY has been established as an accepted line above which interventions are not cost-effective. In fact, the National Institute for Health Clinical Excellence, which determines coverage decisions in the United Kingdom, typically uses £20,00030,000 per QALY (roughly $30,000-$45,000 per QALY) as their threshold for approving new technologies.19 Interestingly, the $50,000 per QALY threshold originates from a 1984 paper on renal replacement therapy, in which the ICER was $50,000 per life year in Canadian dollars. Somewhat amazingly, the ICER for renal replacement therapy (eg, dialysis) has remained remarkably steady over the decades, and although arguments can be made for a different standard in 2013, on balance this “magic number” has been recently shown to be a reasonable WTP.20 More studies are certainly needed to analyze the continued legitimacy of the number.
Methodology of CEA Perspective The first question in any CEA is the perspective of the analysis, which basically refers to who is paying the bill.16 The preferred perspective is societal, because it more broadly considers the implications of the intervention and allows society to prioritize more disparate interventions with different types of outcomes on the population. For example, a societal perspective includes costs of lost productivity as well as caregiver time, which are important to understanding the true implications of a given therapy.16 However, obtaining all of these costs is challenging for the analyst, and so often a payer perspective is used instead. In this setting, only the costs that the insurance company (or Medicare, Medicaid, etc) are considered. There are other potential perspectives as well, such as from the organizational and even patient standpoint. Ultimately, the analysis' perspective must fit with the goal of the study and how the result is going to be used.
Effectiveness Measure As detailed before, there are many potential effectiveness measures, with the life year and QALY serving as the most common metrics. The Gold Commission, which studied the formalism of CEA and issued standard guidelines for reporting (literally, the Gold standard), considers the QALY as the standard effectiveness measure.17 Although there are certainly papers that have used the life year instead, these analyses are considered suboptimal. Within the realm of radiation oncology, a consideration of the differences in quality of life— enshrined within the QALY—are particularly important, as advanced and costly technologies such as IMRT or PBT typically improve the morbidity associated with treatment but not the oncologic outcome. Moreover, an analysis of the cost-effectiveness of organ-preservation treatment paradigms versus more radical (and expensive) surgery warrants the use of utilities to assess the quality-of-life effect of the competing strategies.
Decision and cost-effectiveness analysis Costs As detailed before, the costs that are included in a CEA are in large part a function of the perspective. In the “gold standard” perspective, that is, societal, every cost needs to be considered. Different texts have stratified these costs into different classes. Hunink16 classify costs as “initial” (beginning of treatment), “induced” (resulting from the intervention), and “averted” (costs not incurred because of the intervention). The Gold Commission divided them into “direct costs” (whatever is done with the treatments) and “productivity costs,” which includes anything related to morbidity and mortality from the disease or treatments.17 This latter category is particularly important in the societal perspective, as lost economic productivity (both patient and caregiver) is included. No matter what the nomenclature though, the key concept is that in societal analyses, nearly every cost is included in the study. In contrast, with the payer perspective, only costs paid by the payer are relevant. In this circumstance, costing is much easier, as essentially one only must look at health care reimbursements. One major difference between societal and payer perspectives is the consideration of capital expenses. For example, a CEA investigating the cost-effectiveness of prostate PBT from a payer standpoint does not need to consider the cost of building an expensive facility.21 In contrast, with a societal analysis, that capital investment must be considered, because those dollars could be used for a different purpose. Obviously the costeffectiveness of PBT will be sacrificed if one considers the upfront expense of the proton facility, which emphasizes the need to pay careful attention to the study perspective. It is important to realize that the costs that are used in a CEA are not simply the price or charge of a given good or service, but rather its opportunity cost, which is the value of that resource in its next best use.16 In principle, the opportunity cost of a service equals its market price, but this assumes perfect competition that may not exist in health care. It is therefore common in United States' analyses to use Medicare reimbursements, as the large size of Center for Medicare and Medicaid is thought to best reflect the market price and opportunity cost of a given good; furthermore, in an elegant study looking at different costing methods, Hayman et al22 further showed that Medicare reimbursement nicely reflects the true cost of radiation therapy services. Discounting One key difference between a decision analysis and CEA is the use of discounting, which essentially holds that a dollar today is worth more than a dollar next year; or in reverse, a dollar next year is worth less than a dollar today.16 Financial discounting does make intuitive sense, as a dollar today that would be put into the health care intervention can instead be placed into an investment that yields more money in the future. Thus if the “discount rate” is 3%, $1 today is equal to $1.03 next year. Turning that around, $1 next year is equal to $0.97 today. In 2 years, the “present value” of $1 is 1/(1.03),3 which is $0.94.16 Similarly, as the effectiveness measure is always compared with the cost, QALYs are also discounted. Although slightly counterintuitive, effectiveness discounting holds that a QALY in 5 years is worth less than it is today.
21 One critical consequence of discounting—and its biggest criticism—is that a large cost or effectiveness difference far in the future has significantly less effect than one in the present. This reality actually plays a large role in radiation oncology CEAs. Many advanced technologies such as IMRT or PBT aim to reduce the late effects of treatments, yet these late effects may not manifest for 5-10 years, so this potential benefit is dramatically mitigated by discounting. For example, if cranial proton therapy for medulloblastoma at age 5 years leads to intelligence quotient gains from aged 15 through 80, and each IQ (intelligence quotient) gain is associated with a 0.2 QALY and $5000 productivity gain, the QALY difference at age 25 is only 0.11, and it is only 0.07 at age 40. The comparable productivity gains are similarly only $2800 and $1800 at those ages, respectively. Model-Based CEA There are 2 basic methodologies for CEAs: model- and trialbased studies.18 The model-based analysis is nearly identical to a classic decision analysis, which has previously been described. A Markov model is designed, and QALYs or an alternative effectiveness measure is accrued over the time horizon of the analysis (preferably, lifetime horizon). However, during each stage, costs are also accumulated, so that by the end of the model's lifetime, there is a cost and QALY associated with each strategy. One then lines up the strategies from least effective to most effective, and the ICERs are determined by dividing the difference between costs by the difference in quality-adjusted life expectancies. Just as in a decision analysis, model assumptions are tested in sensitivity analyses, in which a parameter of interest is varied and the resulting change(s) in the ICERs are evaluated. One important additional type of analysis more commonly employed in a CEA is called probabilistic sensitivity analysis.18 Consider the following example in which an important parameter in an analysis is the local control rate of stereotactic body radiation therapy (SBRT). We estimate that parameter in the “base case” model using the literature, but the reality is that our true knowledge of that parameter is imperfect. Different clinical studies have different estimates for the control rate. Ideally, we need a tool to investigate the cost-effectiveness of SBRT over the range of potential efficacy values. Indeed, probabilistic sensitivity analysis accomplishes just this type of analysis. In short, uncertain parameters are first modeled using a mathematical distribution—the local control rate may be a log-normal distribution whose shape is based on an influential meta-analysis—and the model is run thousands of times, with each run choosing a value for the uncertain parameter that is drawn from the mathematical distribution that describes the uncertainty. The ICER for a given strategy would obviously vary with each trial, depending on the parameters that are chosen. These ICERs are then plotted against hypothetical societal WTP values in an “acceptability curve” (Fig. 5). The xaxis of an acceptability curve reflects increasing societal WTP values, and the y-axis is the percentage of trials in which the ICER is less than that WTP. For a given strategy, the higher the curve, the more cost-effective it is irrespective of the uncertain parameters.
D.J. Sher and R.S. Punglia
22
Figure 5 Acceptability curve. Each point on the curve represents the fraction (y-axis) of trials in which the incremental costeffectiveness ratio (ICER) was lower than the willingness to pay (WTP). In this hypothetical example, the strategy reflected by the dotted line is more often cost-effective than the strategy denoted by the solid line.
Trial-Based CEA Although model-based CEAs are more common, trial-based CEA is also an established methodology to compare treatment strategies.18 In these studies, costs and utilities are measured in the patient population in an efficacy study, and they are statistically compared between the 2 (or more) arms to create the ICER and estimate cost-effectiveness under the uncertainty implicit in real-world data. In general, the CEA is a smaller, parallel study nested within a larger clinical question. They are significant undertakings because the cost data are challenging to document and the utility elicitation requires significant effort from the patients to ensure robust data. Pragmatic trials may also be conducted with the strict purpose of determining the cost-effectiveness of the arms, but to our knowledge, such a trial has never been done in radiation oncology.18 Although both designs are very challenging to perform well, trial-based CEA also provide the best “real-world” cost-effectiveness estimates. A further description of this methodology is beyond the scope of this article.
Case Examples As previously described, there are 3 general classes of costeffectiveness questions in radiation oncology23: 1. What is the cost-effectiveness of using RT within different treatment paradigms? In general, this idea refers to comparing primary, radical surgical approaches to organ-preservation therapy. 2. What is the cost-effectiveness of using RT at all? A sample analysis investigating this question would be adjuvant breast RT after lumpectomy or mastectomy. 3. What is the cost-effectiveness of using different modalities (or fractionation regimens) of RT? This category probably houses the most fertile ground for research, as the explosion of expensive —and potentially superior—
technologies such as IMRT, SBRT, and PBT begs for robust analyses of their cost-effectiveness. For the purposes of this review, we highlight 2 costeffectiveness studies—1 based on a Markov model and another based on a randomized trial—both of which investigated the cost-effectiveness of RT for metastatic bone pain. Model-Based Study: Palliative Bone RT for Metastatic Prostate Cancer Bone metastases are a common complication of metastatic prostate cancer, and with such a high prevalence, their optimal management has broad cost-effectiveness implications. Konski24 performed a Markov model-based CEA in which he compared pain medication alone, single-fraction RT, multifraction RT, and chemotherapy for the management of metastatic bone pain in hormone-refractory prostate cancer. He used the payer perspective, which in this study was Medicare. A Markov model was created and consisted of only a few states: pain after treatment, no pain after treatment, recurrent pain after retreatment, no pain after retreatment, and death. The main difference in the model between the fractionation regimens was the retreatment rate (consistent with clinical trials), so he was adequately able to isolate the relative balance between initial upfront cost of multifraction RT with lower downstream costs from fewer retreatments. The study found that chemotherapy was dominated (thus more expensive and less effective) by pain medication, but the ICERs for single-fraction and multifraction RT were $7000 per QALY and $36,000 per QALY, respectively. The author concluded that single-fraction RT was the most cost-effective solution, although if one assumes the societal WTP is $50,000 per QALY, in fact the multiple-fraction regimen is the better strategy. In any case, a Markov model was the only technique that could compare these different approaches—including
Decision and cost-effectiveness analysis both pain medication alone and chemotherapy—highlighting the strength of model-based approaches. Trial-Based Study: Dutch Bone Pain Study The Dutch Bone Pain study was a landmark randomized trial comparing single-fraction RT with multifraction RT for the palliation of bone pain.25 The study showed no significant benefit to multiple-fraction RT except a reduced rate of retreatment. As reirradiation carries an increased cost, the initial more costly 6-fraction regimen may ultimately be more cost-effective if it leads to fewer retreatments and downstream costs. Fortunately, the trial design incorporated a cost-effectiveness component.26 All patients were asked to fill out several qualityof-life questionnaires over time, including the EuroQol classification system (EQ-5D), which provides utility estimates. Treatment costs were estimated from a sampling of the treating hospitals, and patient-specific costs were obtained by questionnaires sent out to a subset of the patient population. The total costs were estimated over the first 12 weeks of treatment, as the authors found that subsequent costs were similar between the groups and estimating later costs would prove challenging to model. The quality-adjusted life expectancy was estimated by multiplying the utility for a given time period by the length of that period, until the patient died; there was a procedure to estimate the QALY for censored patients. The authors found that the single-fraction regimen was associated with more average QALYs and less average cost; in other words, it was a dominant strategy. However, given the small patient numbers and heterogeneity in the individual patient costs and QALYs, these cost and QALY differences were not statistically significant. Thus, they created an acceptability curve, in which they tested the ICER as a function of the statistical variability within the study and varying societal WTP levels. The authors found that the single-fraction was clearly cost-effective (P o 0.05) with a WTP less than $40,000 per QALY, and the P value with a WTP of $100,000 per QALY was borderline, with a P value of 0.09. As these cost and effectiveness estimates were directly taken from a landmark clinical trial, their analysis provides excellent support for single-fraction RT as the most cost-effective treatment. As the authors nicely concluded: “Compared with multiple-fraction RT, single-fraction RT provides equal palliation and quality of life and has lower medical and societal costs, at least in The Netherlands. Therefore, single-fraction RT should be considered as the palliative treatment of choice for cancer patients with painful bone metastases.”26 Interestingly, these 2 studies, using very different methodologies in different health care systems, ultimately reached similar conclusions on the cost-effectiveness of RT for bone metastases. The P value reflecting clear superiority for the single-fraction arm in the Dutch trial turned greater than 0.05 at $40,000 per QALY, which is roughly where Konski's study claimed the multifraction ICER lies. Nevertheless, it is important to remember that any one CEA is not the “final answer” for the cost-effectiveness of a given treatment. There are many implicit assumptions and analytic techniques that may sway the analysis toward 1 treatment over the other, and recognizing
23 and exploring these influential parameters is a critical part of the analysis and subsequent interpretation.
Conclusion With the increasing attention on health care spending and focus on reducing costs, including coverage for radiation oncology services, it is our professional responsibility to determine the treatments that provide the most value to society and to our patient population. CEA is an established discipline whose raison d'etre is to extract the greatest value from each health care dollar. In the future, we hope the tools in the CEA armamentarium would be robustly applied to novel RT techniques and technologies, with the goal of increasing the effectiveness of our services across society, given whatever financial resources are ultimately assigned to our field.
References 1. Meyer AM, Carpenter WR, Abernethy AP, et al: Data for cancer comparative effectiveness research: past, present, and future potential. Cancer 118(21):5186-5197, 2012 2. Spiro T, Lee EO, Emanuel EJ: Price and utilization: Why we must target both to curb health care costs. Ann Intern Med 157:586-590, 2012 3. Meropol NJ, Schrag D, Smith TJ, et al: American Society of Clinical Oncology guidance statement: The cost of cancer care. J Clin Oncol 27:3868-3874, 2009 4. Mariotto AB, Yabroff KR, Shao Y, et al: Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst 103:117-128, 2011 5. Shen XAP, Levin D, Rao VM, et al: Trends in radiation oncology services in the IMRT era: Evolving patterns of usage and payments in Medicare patients from 2000-2009. Int J Radiat Oncol Biol Phys 2011 [Abstract] 6. Schrag D, Kuntz KM, Garber JE, et al: Decision analysis—Effects of prophylactic mastectomy and oophorectomy on life expectancy among women with BRCA1 or BRCA2 mutations. N Engl J Med 336:1465-1471, 1997 7. Elkin EB, Weinstein MC, Winer EP, et al: HER-2 testing and trastuzumab therapy for metastatic breast cancer: A cost-effectiveness analysis. J Clin Oncol 22:854-863, 2004 8. Hillner BE, Smith TJ: Efficacy and cost effectiveness of adjuvant chemotherapy in women with node-negative breast cancer. A decision-analysis model. N Engl J Med 324:160-168, 1991 9. Stiggelbout AM, de Haes JC: Patient preference for cancer therapy: An overview of measurement approaches. J Clin Oncol 19:220-230, 2001 10. Weinstein MC, Stason WB: Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 296:716-721, 1977 11. Beck JR, Pauker SG: The Markov process in medical prognosis. Med Decis Making 3:419-458, 1983 12. Sonnenberg FA, Beck JR: Markov models in medical decision making: A practical guide. Med Decis Making 13:322-338, 1993 13. Heijnsdijk EA, Wever EM, Auvinen A, et al: Quality-of-life effects of prostate-specific antigen screening. N Engl J Med 367:595-605, 2012 14. Barr DA: Bending the Medicare cost curve for physicians' services: Lessons learned from Canada. J Gen Intern Med 27:1555-1559, 2012 15. Cigna Medical Coverage Policy. Proton Beam Therapy for Prostate Cancer. 16. Hunink MGM: Decision Making in Health and Medicine: Integrating Evidence and Values. Cambridge: Cambridge University Press, 2001 17. Gold MR, Siegel JE, Russell LB, et al: Cost-Effectiveness in Health and Medicine. Oxford, USA, Oxford University Press, 1996, pp 1 [online resource 450 p.] 18. Drummond MF, McGuire A: Economic Evaluation in Health Care: Merging Theory With Practice. Oxford: Oxford University Press, 2001 19. McCabe C, Claxton K, Culyer AJ: The NICE cost-effectiveness threshold: What it is and what that means. Pharmacoeconomics 26:733-744, 2008
24 20. Winkelmayer WC, Weinstein MC, Mittleman MA, et al: Health economic evaluations: The special case of end-stage renal disease treatment. Med Decis Making 22:417-430, 2002 21. Konski A, Speier W, Hanlon A, et al: Is proton beam therapy cost effective in the treatment of adenocarcinoma of the prostate? J Clin Oncol 25:3603-3608, 2007 22. Hayman JA, Lash KA, Tao ML, et al: A comparison of two methods for estimating the technical costs of external beam radiation therapy. Int J Radiat Oncol Biol Phys 47:461-467, 2000 23. Sher DJ: Cost-effectiveness studies in radiation therapy. Expert Rev Pharmacoecon Outcomes Res 10:567-582, 2010
D.J. Sher and R.S. Punglia 24. Konski A: Radiotherapy is a cost-effective palliative treatment for patients with bone metastasis from prostate cancer. Int J Radiat Oncol Biol Phys 60:1373-1378, 2004 25. Steenland E, Leer JW, van Houwelingen H, et al: The effect of a single fraction compared to multiple fractions on painful bone metastases: A global analysis of the Dutch Bone Metastasis Study. Radiother Oncol 52:101-109, 1999 26. van den Hout WB, van der Linden YM, Steenland E, et al: Single- versus multiple-fraction radiotherapy in patients with painful bone metastases: Cost-utility analysis based on a randomized trial. J Natl Cancer Inst 95:222-229, 2003
Introduction
T
he Institute of Medicine (IOM) published 6 defining characteristics of methodology for comparative effectiveness research in 2009 (IOM Initial National Priorities for Comparative Effectiveness Research, 2009): (1) Has the objective of directly informing a specific clinical decision from the patient perspective or a health policy decision from the population perspective. (2) Compares at least 2 alternative interventions, each with the potential to be “best practice.” (3) Describes results and the population of subgroup levels. (4) Measures outcomes—both benefits and harms—that are important to patients. (5) Employs methods and data sources appropriate for the decision of interest.
*Department of Radiation Oncology, Rush University Medical Center, Chicago, IL. †Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women's Hospital, Boston, MA. The authors declare no conflict of interest. Address reprint requests to David J. Sher, MD, MPH, Rush University Medical Center, Department of Radiation Oncology, Chicago, IL. E-mail: [email protected]
14
1053-4296/13/$-see front matter & 2014 Elsevier Inc. All rights reserved. doi:http://dx.doi.org/10.1016/j.semradonc.2013.08.002
(6) Is conducted in settings that are similar to those in which the intervention would be used in practice. There are many well-known clinical trial, registry, and large database techniques and tools that classically conform to this definition. Randomized clinical trials—explanatory or pragmatic—serve as the backbone for obtaining level I evidence, and the importance of discerning real-world outcomes has raised the stature of both retrospective data mining and prospective registry studies. Yet, there are inherent limitations to these methodologies, as detailed in the article by Meyer et al.1 Randomized trials help guide decisions but require a long period for follow-up, especially when studying diseases that may have a long clinical history, before treatment differences can be discerned. Therefore, data from such studies may not have matured until the treatment itself is obsolete. Retrospective or even prospective large database analyses often do not account for key patient characteristics, either personal or disease related, that may play a key role in determining the optimal therapy for a given patient. Indeed, 1 defining characteristic of all of these methodologies is their reliance on actual patient outcomes and the ultimate reporting of a single preferred treatment or strategy for the “average person.” These studies consequently have no mechanism of incorporating
Decision and cost-effectiveness analysis toxicity and patient preferences into the determination of the best treatment strategy. For a given patient, the best decision for him or her may hinge on his or her own personal, disease, or treatment characteristics, including treatment efficacy or toxicity probabilities that cannot be specified within a clinical trial or retrospective study. The decision to add radiation therapy for patients is a personal one and dependent on an individual's specific set of clinical factors and personal preferences. Optimal decision making requires careful consideration of the potential risks and expected outcomes of radiation therapy. As implied by the name, decision analysis is a set of mathematical tools whose goal is to determine the optimal outcome (whether it is the best treatment or imaging approach, or surveillance strategy, etc) for a given decision under a widely varying set of conditions. Although randomized trials and retrospective studies use real-patient data and biostatistical tools to determine the efficacy or effectiveness of a treatment, decision analysis wields finely tuned mathematical models to simulate the treatment and disease process and thus determine “winning” strategies based on computation. These models are entirely informed by the literature and thus hinge on prospective and retrospective evidence, yet the fact that they are ultimately mathematical models liberates the analyst to test different hypotheses on the importance of patient, treatment, and disease characteristics not possible in patient-based, realworld data. Thus, decision analysis provides a mechanism with which to integrate the available data to compare the outcomes of different treatment strategies. Indeed, decision analysis is perhaps the perfect means to satisfy each of the IOM's criteria for comparative effectiveness research. The motivation for comparative effectiveness research goes beyond which treatment decision is the right one to whether a decision is right in the context of limited resources. The escalating costs of American health care are inescapable. In 2011, the United States spent $2.6 trillion on health care, which accounts for nearly one-fifth of the economy. This figure amounts to approximately $8000 per person, which is $3000 more than the next country and twice the average for developed countries. Since 1985, health spending has increased 2% per year faster than gross domestic product (GDP), such that in 2010, medical spending amounted to nearly 18% of GDP.2 Cancer care comprises a significant amount of the absolute health care expenditure as well as its growth. In 2010, $124.6 billion were spent in oncology; antineoplastics were the leading class of hospital drug expenditures, and biological agents accounted for 15% of all prescription drug costs.3 Moreover, without any change in costs, population changes alone will increase oncology spending by 27% in the year 2020, and if costs increase by 5% per year, the costs from cancer care alone will top $200 billion.4 It is actually very difficult to know the extent to which radiotherapy (RT) services contribute to accelerating costs. Unfortunately, the most recent data analyzing oncology expenditures lump all treatments together by phase of care (initial, maintenance, and end-of-life), without specifying the treatments rendered. In a interesting study presented at the
15 American Society for Radiation Oncology National Meeting in 2011, Shen et al5 showed that the total Medicare payments for external beam RT increased 322% between 2000 and 2009, from $256 million to $1.08 billion, with most that increase attributable to intensity-modulated radiation therapy (IMRT) reimbursement. Despite the absence of published data on this topic, however, with the advent of multimillion dollar proton facilities and progressively more common implementation of IMRT and stereotactic RT, our community recognition of rising RT-related costs echoes the famous quote by Supreme Court Justice Potter Stewart: “I know it when I see it.”
Decision Analysis Motivation for Decision Analysis Decision analytic techniques have been used to inform decision making in situations of uncertainty. Decision analyses can delineate more clearly the trade-offs for individual clinical scenarios where there are gaps in existing data regarding treatment choice in a number of ways. First, they provide a framework to combine available data from randomized and retrospective trials to study questions for which no direct evidence is currently available. Decision analysis also provides a vehicle to extrapolate results for more recent trials to model outcomes while the decision to use a treatment is still relevant. Decision analyses are especially useful when treatment decisions need to be made for the patient of today, while data from additional studies specific to the patient's clinical characteristics may be forthcoming. Second, decision models can be used to tailor expected outcomes to an individual set of clinical circumstances. Results of decision models have been published using the available data from randomized and retrospective trials to model the life course of clinical subsets of patients diagnosed with breast cancer (for example, those patients with breast cancer with genetic mutations, node-negative disease, or disease responsive to a particular systemic agent) to study longterm outcomes and inform current treatment choice.6-8 Finally, decision analyses can model a number of different outcomes, such as overall survival or time without recurrence, allowing estimates to reflect the outcome most relevant to an individual patient.
Methodology of Decision Analysis Structuring the Decision A decision model or decision tree is created to represent the clinical choice. The model needs to be simple enough to be understood and feasible to do. To do this, a model needs simplifying assumptions, but at the same time, a model needs to be complex enough to describe accurately the features of the decision. Decision nodes are represented by squares in the decision tree, chance nodes by circles, and terminal nodes by triangles. Figure 1 represents a decision in a hypothetical situation where a patient is confronted with the following treatment dilemma regarding whether to undergo radiation therapy or not for illness X. Radiation therapy for illness X is toxic and
D.J. Sher and R.S. Punglia
16
Figure 1 Diagram of a simple decision tree of a patient with illness X. This model represents a decision in a hypothetical situation where a patient is confronted with the following treatment dilemma regarding whether to undergo radiation therapy or not for illness X. Radiation therapy for illness X is toxic, and results in a risk of 10% chance of death. However, if a patient with illness X survives the treatment, they have an 80% chance of remaining alive. If they do not undergo radiation, they have a 70% chance of remaining alive. Tree constructed with TreeAge Pro (Williamstown, MA). Calculating expected result from each option: Starting with the right side of the last chance node for radiation therapy, we find that if a patient survives treatment, they have an 80% chance of living and garnering an outcome of 1, and a 20% chance of dying and garnering an outcome of 0. The expected value at this chance node is derived by multiplying the probability of the outcome with the value associated with it, which in our case gives us a value of 0.80. Moving to the left of the decision tree, we then substitute the value of 0.80 as the value associated with surviving treatment and combine this value and its probability of 0.90 with that of dying from treatment (outcome of 0 with probability of 0.10) to derive an expected value for radiation therapy of 0.72. The expected value for no treatment is obtained from multiplying the likelihood of living (0.70) with its outcome of 1 and adding that to the product of the likelihood of dying (0.30) and its outcome of 0, for an expected value of 0.70. (Color version of figure is available online.)
results in a 10% risk of death. However, if a patient with illness X survives the treatment, they have an 80% chance of remaining alive. If they do not undergo radiation, they have a 70% chance of remaining alive. What treatment should a patient undergo? In the model, the decision of whether to undergo radiation or not is represented by the square. Here the choice is under the control of the decision maker or the patient with illness X. After choosing a treatment, there is a chance node to the right of which all possible events are represented as branches. The chance events are not in the control of the patient or decision maker. Instead, likelihood of events following a chance node is decided by probability or outcomes data from the literature or expert opinion if literature is not available. The probabilities of chance events at each chance node need to add to 1 or 100%. In our example (Fig. 1), if a patient with illness X chooses radiation therapy, he has a 10% chance of dying from the treatment and a 90% chance of surviving. If the patient survives, he then has an 80% chance of being alive and a 20% chance of dying. If a patient chooses no treatment, he has a 70% chance of living and a 30% chance of dying. The terminal nodes shown with triangles in our example of illness X (Fig. 1) represent the final outcomes states associated with each of the possible chance pathways. In our hypothetical example, the only 2 outcomes are living or dying. Along with each of the terminal nodes is a utility or value assigned to that final state. In our simple model, being alive is associated with a value of 1 and dead is 0. However, decision analyses provide
the opportunity to incorporate a number of intermediate values to inform a treatment decision as described next.
Outcome Values Although one outcome can be likelihood of survival, decision analysis can describe the value assigned to outcome states in terms of utilities. This ability to describe utilities for different outcomes states allows modelers to incorporate side effects of treatments into their model, an important consideration given that many cancer treatments affect a patient's quality of life. A utility for a given health state is the “value” ascribed to that health state similar to its quality of life. Stiggelbout and de Haes9 define utility as, The values that subjects attach to . . . possible health states are called the utilities of the health state. A utility is defined as the level of desirability that people associate with a particular outcome. It is a cardinal number that represents the strength of an individual's preference for a particular outcome when faced with uncertainty. Utilities are assigned to each outcome on a scale that is established by assigning a value of 1 to the state of optimal health and a value of 0 to death. Although some authors use utility scores as qualityof-life scores, utility is a concept that in fact is essentially different for quality of life per se. Utilities reflect both the quality of life and the value of that quality of life, relative to death and optimal health.
Decision and cost-effectiveness analysis How does one assess a utility for a health state? These methodologies are beyond the scope of this chapter, but the most standard pathway is to use a “standard gamble” or “time trade-off” techniques to measure the value of the health state when there is uncertainty. The key in either instrument is rooting the answers within the framework of death (utility value 0) and perfect health (value 1), which preserves important mathematical characteristics of the value. An example of the time trade-off technique is asking how much time in perfect health would be the equivalent value to you in this suboptimal health state (eg, “if you are 35 years old now and are going to live until 85 with a mastectomy, how many fewer years would you live if you were able to preserve the breast?”). Whose utilities should go into the model being built? Most often healthy subjects or patients not in the certain state being studied are used to assess utilities. But utilities can vary widely within the group from which they are derived, and there is considerable literature describing the adaptation phenomena of utilities in that patients in a particular state ascribe a higher utility to it than those not in the compromised state. Therefore, the choice of source for utilities should match the purpose of the decision analysis. Is this a model to inform policy decisions? In which case, average population values could be used. Or is this a model describing an individual patient decision? In which case, patient values could be used. In all cases, the outcome measure needs to be consistent across terminal nodes of a given model. The ability to measure utilities allows one to describe time outcomes in terms of quality-adjusted life years (QALYs). A QALY is determined by multiplying the time spent in a certain state with the utility for a health state relative to 1 year in perfect health. For example, 6 months of time in a state whose utility is ascribed to be 0.5 would be the equivalent of 0.25 QALYs. Calculating the Expected Outcome The final step in a decision analysis is evaluating the expected utility or outcome of each treatment strategy. Going back to our hypothetical example of illness X (Fig. 1), we multiply the probabilities to calculate the expected result of each of the treatment choices, radiation therapy or no treatment. In this example, the expected value for radiation therapy is 0.72 and that for no treatment is 0.70. So the strategy that leads to the greatest expected value is radiation therapy. Sensitivity Analysis Probabilities at chance nodes and outcomes values are derived from published literature or expert opinion. Where there are few primary data or controversies in the literature, sensitivity analyses are useful in decision analytic models. Sensitivity analyses vary a certain parameter (eg, probability of developing a local or distant recurrence after having initial treatment) over the entire plausible range of its values, and then study the effect of varying the parameter on the given outcomes. Sensitivity analyses provide a measure of the robustness of a conclusion in a manner similar to confidence intervals and P values in biostatistics. They can confirm the lack of significant dependence of an outcome on a certain transition parameter and
17 increase the level of confidence or alternatively by revealing dependence of an outcome measure on the variable studied, may identify areas where further research may be valuable.10 In our example of illness X (Fig. 1), we may not know the exact likelihood of treatment mortality or it may vary by physician or with patient characteristics. We can conduct a 1-way sensitivity analysis (Fig. 2) to ask “what if” treatment mortality was y, then what would the optimal treatment strategy be. In our example, we find that as treatment mortality increases, the expected value for radiation therapy decreases and that the threshold value for mortality is 0.125. Below this mortality threshold, radiation therapy is associated with the best expected value, but more than this value, no treatment is the optimal strategy. One-way sensitivity analyses provide threshold values. Twoway and 3-way sensitivity analyses systematically vary 2-3 parameters on different axes, respectively. The outcomes for each combination of variables can then be assessed.
Markov Models Elements and Assumptions Markov models are useful when time needs to be incorporated into a decision model. These models study outcomes by describing events as discrete states. They were first applied for medical applications to predict prognosis in 1983 by Beck and Pauker11 who designed a method of describing patient outcomes by dividing them into discrete health states. Hypothetical “patients” start the model in a single health state. As the model cycles, the patients travel through time, and may remain in that health state or transition to a different health state. In the simplified breast cancer model stated later (Fig. 3), patients who have completed initial therapy start in the “No disease” health state and with each cycle may (1) remain in the “No disease” health state, (2) have a finite probability of having a recurrence and transition to the “Recurrence” health state, or (3) have a finite probability of dying of causes unrelated to their cancer and transition to the “Dead” health state. Once in the “Recurrence” health state, patients may remain there or with a certain probability die from their recurrent disease and enter the “Dead” health state. The “Dead” health state is an absorbing health state, in that once entered, patients may not exit. An absorbing model is that which is run until all patients are in the “Dead” health state and can be used to calculate residence times in transient states allowing for estimates such as life expectancy or time spent without recurrence. Markov models are particularly useful in modeling clinical situations where a patient is at risk for a given event over an extended period of time (eg, recurrence from breast cancer).12 Markov processes rely on a series of assumptions: A patient is always in one of a finite number of discrete health states. All events are represented in the model as transitions from one state to another. Time is divided into equal increments referred to as Markov cycles, which may be a finite number of days,
D.J. Sher and R.S. Punglia
18
Figure 2 Sensitivity analysis of mortality from treatment for illness X. This graph represents a 1-way sensitivity analysis of mortality associated with treatment. As mortality from treatment increases from 0-1 on the x-axis, the expected value for radiation therapy decreases. The threshold mortality at which radiation therapy and no treatment yield equivalent expected outcomes is 0.125. At mortality risks below this threshold value, radiation therapy is superior to no treatment; at risks above 0.125, no treatment is the preferred strategy. Tree constructed with TreeAge Pro (Williamstown, MA). (Color version of figure is available online.)
months, or years depending on the clinical scenario modeled. Transitions between health states may only occur once per cycle. There is no “memory” feature in a Markov model, so that all patients in a given health state face the same transition probabilities, irrespective of when they entered that health state.
Figure 3 This representation of a Markov model has 3 health states: no disease, recurrence, and distant metastases. All patients start in the no disease health state from which each cycle they can develop recurrence or distant disease with a specified probability. From the recurrence health state, patients can also enter the distant metastases health state, again with a certain probability. The distant metastases health state is an absorbing health state, in that once entered, a patient cannot exit.
Evaluation Methods Two common methods to evaluate a Markov model include cohort simulation and Monte Carlo simulation. With cohort simulation, a population or entire cohort is subjected to the decision model simultaneously and results are described at a population level (example, average life expectancy for the cohort). Monte Carlo simulation refers to a process by which simulated patients enter the model 1 at a time. The computer then performs a coin toss or randomly chooses a value for a variable in the decision analysis using a prespecified distribution and then tracks the results. The computer does this thousands of times, thus simulating many thousands of “patients,” and the mean and standard deviation for an outcome (eg, mean life expectancy) are calculated. The term Monte Carlo comes from gambling because as in gambling, the outcome depends on a random selection of values. Moreover, if the computer does it enough times or “plays the game” long enough, one can generate a distribution of the possible results. Of course, it is up to the designer of the decision model to not only pick the values but also the distribution of the values (bellshaped, triangular, or discrete cardinal), which require additional assumptions.
Case Example With the aforementioned context, we can evaluate a recent decision analysis designed to measure the quality-of-life effects of prostate-specific antigen (PSA) screening. Although screening can lead to earlier diagnosis of prostate cancer, it can also lead to overdiagnosis and overtreatment in patients who may not have prostate cancer or who have prostate cancer that would have never become clinically detectable. Therefore, the
Decision and cost-effectiveness analysis key question here is what are the trade-offs associated with screening and is it worth it. Heijnsdijk et al13 created a model based on data from the European Randomized Study of Screening for Prostate Cancer. From the screening study, they incorporated a 29% reduction in prostate cancer mortality with PSA screening. Their model was used to simulate individual life histories of men who underwent PSA screening varying the age cohort and screening strategy. They incorporated utilities for individual health states such as bowel dysfunction and urinary incontinence after treatment. Their results revealed that annual PSA screening of men between ages 55 and 69 years results in a 28% reduction of death from prostate cancer, which validates that their model is well calibrated to the European Randomized Study of Screening for Prostate Cancer trial. Annual screening resulted in 73 life years gained per 1000 men screened, but only 56 QALYs which incorporate the downsides to screening including the side effects of treatment. The authors found that their outcome of QALYs gained was very sensitive to the utility estimate used for the posttreatment recovery period. If no loss in utility during this period was used (1 instead of the base case 0.95 for utility), then the resulting outcome per 1000 men screened was 72 QALYs. However, if 0.93 was used for the utility, then only 6 QALYs were gained. This finding highlights the need for further research into the utility of the posttreatment period and also reveals that whether screening should be done may be a function of an individual's posttreatment utility. Such conclusions were only reachable using a decision analysis, which can incorporate the patient's health state evaluation into the analysis.
Cost-Effectiveness Analysis (CEA) Motivation for CEA Our national consciousness is continuously—and not necessarily inappropriately—assaulted with warnings about rising costs. We hear about them in popular media, in political debates, and in reimbursement decisions from insurance companies. What are less commonly discussed are actual solutions to these rising costs. Medicare has generally attempted blanket solutions for cost escalations, such as the sustainable growth rate formula, which simply links payments—not coverage decisions—to the GDP.14 In fact, the Center for Medicare and Medicaid is legally not allowed to consider cost-effectiveness of therapies, so the differential allocation of money (more or less) does not change health outcomes, just how much they cost. Private payers also rarely consider the cost-effectiveness of therapies in their coverage decisions. Cigna, for example, generally covers proton beam therapy (PBT) for prostate cancer because it is clinically equivalent, but not superior, to conventional treatments, and it does not consider cost or cost-effectiveness; however, in some Cigna-administered plans, in which “medical necessity” is mandated, PBT is not covered because it is more expensive and not proven to be more efficacious. This
19 latter ruling implies the consideration of treatment costs in some plans; it does not rely on CEA but rather absolute costs.15 Although the focus on cost minimization is understandable, the real issue is how do we maximize the health outcomes given whatever financial resources we have. That is, how do we get the most health care “bang” for our “buck.” The term “rationing” is taboo in American vernacular, but we do have a finite amount of health care dollars that need to be put into the best use. In other words, we need to ration the resources that we have. So simply blindly cutting down on the reimbursement for given services is far less useful for society than evaluating what interventions are the most cost-effective and focusing resources on those high-yield services. CEA is a robust field whose sole aim is to improve the value of health care interventions; that is, get the most bang for the buck. Although CEA is related to cost control in the sense that it allows us to prioritize treatments given limited resources, it is fundamentally a separate concept. In principle, we can spend less and achieve more, if the right interventions are chosen, or cynically, spend more and achieve less, if the wrong treatments are funded.
Incremental Cost-Effectiveness Ratio (ICER) At the end of any CEA, the result is presented as an ICER, or ICER. There are 2 components of this outcome, the term incremental and the term cost-effectiveness ratio. The term incremental refers to the fact that the cost-effectiveness of any intervention is stated in comparison to another, less effective intervention.16 The less effective treatment may be no treatment or it may be a rudimentary therapy or it may be a floridly expensive but ineffective treatment. In any case, an ICER for a given therapy is always relative to a different therapy, and thus the choice of alternatives is obviously critical. The term cost-effectiveness ratio refers to the form of the end point, which is a simple ratio of cost to effectiveness. The cost can be in any denomination, with dollars and euros as the most common in the medical literature. The effectiveness measure chosen by the analyst may also vary, from QALYs to life years to disability associated life year, though the standard metric is the QALY.17 As a result, the ICER is typically reported in a structure like $55,000 per QALY. It is noteworthy that when a CEA is done with the QALY or disability associated life year as the effectiveness measure, it is also termed a cost-utility analysis, as these 2 units are utilities.18 An intuition about the ICER can be obtained by understanding the “cost-effectiveness space,” which is a graph that outlines the 4 permutations of effectiveness and cost differences (Fig. 4) with a new therapy. The origin of this space is the current treatment or standard, and the new treatment's cost and effectiveness are then plotted relative to the standard treatment's cost and effectiveness. In the northwest space, the treatment is more costly and less effective, and this scenario is called “dominance” (ie, the new treatment is dominated by the standard treatment).16 Conversely, in the southeast space, the new treatment is more effective and less costly, and it dominates the standard treatment. The more common scenarios are seen in the northeast and southwest quadrants, in which
D.J. Sher and R.S. Punglia
20
Figure 4 Cost-effectiveness space. The relative difference between the costs and effectiveness of 2 strategies can be plotted on this graph, with the origin reflecting the standard therapy, and the X- and Y-coordinate as the relative difference between the 2. Dominance occurs in the Northwest and Southeast boxes. In the Northeast and Southwest boxes, the incremental cost-effectiveness ratio (ICER) is determined by the slope of the line. Any intervention to the right of the line has a superior ICER.
the new treatment is more effective and more costly or less effective and cheaper, respectively. In these cases, the slope of the line is equivalent to the ICER, and whether the treatment is considered cost-effective is dependent on the societal willingness to pay (WTP), which can be designated by a line with a slope equivalent to the WTP. Graphically, if the ICER falls to the right of the WTP line, the treatment is cost-effective.
The Magical $50,000 Per QALY Line Consider the following thought experiment. Every health intervention that a society or health plan may implement is assigned an ICER. How would you then maximize health care outcomes? Well, the solution is straightforward. One would start with funding the most cost-effective treatment (ie, lowest ICER or cost per QALY), and then fund the second most costeffective treatment, and continue on until there are no more health care dollars to spend. This approach—dubbed the “shopping-spree problem”—will maximize QALYs.16 Of course, there are fundamental concerns with this protocol. Simply summating QALYs without regard to who is receiving them raises concerns about equity and the potential for increasing health disparities. Similarly, society may want to fund some very cost-ineffective treatments for certain populations, such as children. Yet, on balance, this algorithm does provide a mechanism to optimize the value of health care dollars. In practice, though, no payer is going to order every health care intervention by ICER and then allocate money in that order. Instead, we typically consider interventions as “costeffective” or not by the absolute ICER and whether it passes a certain threshold, which is called societal WTP. In some cases, the specific threshold is not particularly relevant: an ICER of $500,000 per QALY is never going to be cost-effective and an ICER of $5000 per QALY is. Yet, many interventions hover somewhere in between: what defines cost-effectiveness?
For better or for worse, $50,000 per QALY has been established as an accepted line above which interventions are not cost-effective. In fact, the National Institute for Health Clinical Excellence, which determines coverage decisions in the United Kingdom, typically uses £20,00030,000 per QALY (roughly $30,000-$45,000 per QALY) as their threshold for approving new technologies.19 Interestingly, the $50,000 per QALY threshold originates from a 1984 paper on renal replacement therapy, in which the ICER was $50,000 per life year in Canadian dollars. Somewhat amazingly, the ICER for renal replacement therapy (eg, dialysis) has remained remarkably steady over the decades, and although arguments can be made for a different standard in 2013, on balance this “magic number” has been recently shown to be a reasonable WTP.20 More studies are certainly needed to analyze the continued legitimacy of the number.
Methodology of CEA Perspective The first question in any CEA is the perspective of the analysis, which basically refers to who is paying the bill.16 The preferred perspective is societal, because it more broadly considers the implications of the intervention and allows society to prioritize more disparate interventions with different types of outcomes on the population. For example, a societal perspective includes costs of lost productivity as well as caregiver time, which are important to understanding the true implications of a given therapy.16 However, obtaining all of these costs is challenging for the analyst, and so often a payer perspective is used instead. In this setting, only the costs that the insurance company (or Medicare, Medicaid, etc) are considered. There are other potential perspectives as well, such as from the organizational and even patient standpoint. Ultimately, the analysis' perspective must fit with the goal of the study and how the result is going to be used.
Effectiveness Measure As detailed before, there are many potential effectiveness measures, with the life year and QALY serving as the most common metrics. The Gold Commission, which studied the formalism of CEA and issued standard guidelines for reporting (literally, the Gold standard), considers the QALY as the standard effectiveness measure.17 Although there are certainly papers that have used the life year instead, these analyses are considered suboptimal. Within the realm of radiation oncology, a consideration of the differences in quality of life— enshrined within the QALY—are particularly important, as advanced and costly technologies such as IMRT or PBT typically improve the morbidity associated with treatment but not the oncologic outcome. Moreover, an analysis of the cost-effectiveness of organ-preservation treatment paradigms versus more radical (and expensive) surgery warrants the use of utilities to assess the quality-of-life effect of the competing strategies.
Decision and cost-effectiveness analysis Costs As detailed before, the costs that are included in a CEA are in large part a function of the perspective. In the “gold standard” perspective, that is, societal, every cost needs to be considered. Different texts have stratified these costs into different classes. Hunink16 classify costs as “initial” (beginning of treatment), “induced” (resulting from the intervention), and “averted” (costs not incurred because of the intervention). The Gold Commission divided them into “direct costs” (whatever is done with the treatments) and “productivity costs,” which includes anything related to morbidity and mortality from the disease or treatments.17 This latter category is particularly important in the societal perspective, as lost economic productivity (both patient and caregiver) is included. No matter what the nomenclature though, the key concept is that in societal analyses, nearly every cost is included in the study. In contrast, with the payer perspective, only costs paid by the payer are relevant. In this circumstance, costing is much easier, as essentially one only must look at health care reimbursements. One major difference between societal and payer perspectives is the consideration of capital expenses. For example, a CEA investigating the cost-effectiveness of prostate PBT from a payer standpoint does not need to consider the cost of building an expensive facility.21 In contrast, with a societal analysis, that capital investment must be considered, because those dollars could be used for a different purpose. Obviously the costeffectiveness of PBT will be sacrificed if one considers the upfront expense of the proton facility, which emphasizes the need to pay careful attention to the study perspective. It is important to realize that the costs that are used in a CEA are not simply the price or charge of a given good or service, but rather its opportunity cost, which is the value of that resource in its next best use.16 In principle, the opportunity cost of a service equals its market price, but this assumes perfect competition that may not exist in health care. It is therefore common in United States' analyses to use Medicare reimbursements, as the large size of Center for Medicare and Medicaid is thought to best reflect the market price and opportunity cost of a given good; furthermore, in an elegant study looking at different costing methods, Hayman et al22 further showed that Medicare reimbursement nicely reflects the true cost of radiation therapy services. Discounting One key difference between a decision analysis and CEA is the use of discounting, which essentially holds that a dollar today is worth more than a dollar next year; or in reverse, a dollar next year is worth less than a dollar today.16 Financial discounting does make intuitive sense, as a dollar today that would be put into the health care intervention can instead be placed into an investment that yields more money in the future. Thus if the “discount rate” is 3%, $1 today is equal to $1.03 next year. Turning that around, $1 next year is equal to $0.97 today. In 2 years, the “present value” of $1 is 1/(1.03),3 which is $0.94.16 Similarly, as the effectiveness measure is always compared with the cost, QALYs are also discounted. Although slightly counterintuitive, effectiveness discounting holds that a QALY in 5 years is worth less than it is today.
21 One critical consequence of discounting—and its biggest criticism—is that a large cost or effectiveness difference far in the future has significantly less effect than one in the present. This reality actually plays a large role in radiation oncology CEAs. Many advanced technologies such as IMRT or PBT aim to reduce the late effects of treatments, yet these late effects may not manifest for 5-10 years, so this potential benefit is dramatically mitigated by discounting. For example, if cranial proton therapy for medulloblastoma at age 5 years leads to intelligence quotient gains from aged 15 through 80, and each IQ (intelligence quotient) gain is associated with a 0.2 QALY and $5000 productivity gain, the QALY difference at age 25 is only 0.11, and it is only 0.07 at age 40. The comparable productivity gains are similarly only $2800 and $1800 at those ages, respectively. Model-Based CEA There are 2 basic methodologies for CEAs: model- and trialbased studies.18 The model-based analysis is nearly identical to a classic decision analysis, which has previously been described. A Markov model is designed, and QALYs or an alternative effectiveness measure is accrued over the time horizon of the analysis (preferably, lifetime horizon). However, during each stage, costs are also accumulated, so that by the end of the model's lifetime, there is a cost and QALY associated with each strategy. One then lines up the strategies from least effective to most effective, and the ICERs are determined by dividing the difference between costs by the difference in quality-adjusted life expectancies. Just as in a decision analysis, model assumptions are tested in sensitivity analyses, in which a parameter of interest is varied and the resulting change(s) in the ICERs are evaluated. One important additional type of analysis more commonly employed in a CEA is called probabilistic sensitivity analysis.18 Consider the following example in which an important parameter in an analysis is the local control rate of stereotactic body radiation therapy (SBRT). We estimate that parameter in the “base case” model using the literature, but the reality is that our true knowledge of that parameter is imperfect. Different clinical studies have different estimates for the control rate. Ideally, we need a tool to investigate the cost-effectiveness of SBRT over the range of potential efficacy values. Indeed, probabilistic sensitivity analysis accomplishes just this type of analysis. In short, uncertain parameters are first modeled using a mathematical distribution—the local control rate may be a log-normal distribution whose shape is based on an influential meta-analysis—and the model is run thousands of times, with each run choosing a value for the uncertain parameter that is drawn from the mathematical distribution that describes the uncertainty. The ICER for a given strategy would obviously vary with each trial, depending on the parameters that are chosen. These ICERs are then plotted against hypothetical societal WTP values in an “acceptability curve” (Fig. 5). The xaxis of an acceptability curve reflects increasing societal WTP values, and the y-axis is the percentage of trials in which the ICER is less than that WTP. For a given strategy, the higher the curve, the more cost-effective it is irrespective of the uncertain parameters.
D.J. Sher and R.S. Punglia
22
Figure 5 Acceptability curve. Each point on the curve represents the fraction (y-axis) of trials in which the incremental costeffectiveness ratio (ICER) was lower than the willingness to pay (WTP). In this hypothetical example, the strategy reflected by the dotted line is more often cost-effective than the strategy denoted by the solid line.
Trial-Based CEA Although model-based CEAs are more common, trial-based CEA is also an established methodology to compare treatment strategies.18 In these studies, costs and utilities are measured in the patient population in an efficacy study, and they are statistically compared between the 2 (or more) arms to create the ICER and estimate cost-effectiveness under the uncertainty implicit in real-world data. In general, the CEA is a smaller, parallel study nested within a larger clinical question. They are significant undertakings because the cost data are challenging to document and the utility elicitation requires significant effort from the patients to ensure robust data. Pragmatic trials may also be conducted with the strict purpose of determining the cost-effectiveness of the arms, but to our knowledge, such a trial has never been done in radiation oncology.18 Although both designs are very challenging to perform well, trial-based CEA also provide the best “real-world” cost-effectiveness estimates. A further description of this methodology is beyond the scope of this article.
Case Examples As previously described, there are 3 general classes of costeffectiveness questions in radiation oncology23: 1. What is the cost-effectiveness of using RT within different treatment paradigms? In general, this idea refers to comparing primary, radical surgical approaches to organ-preservation therapy. 2. What is the cost-effectiveness of using RT at all? A sample analysis investigating this question would be adjuvant breast RT after lumpectomy or mastectomy. 3. What is the cost-effectiveness of using different modalities (or fractionation regimens) of RT? This category probably houses the most fertile ground for research, as the explosion of expensive —and potentially superior—
technologies such as IMRT, SBRT, and PBT begs for robust analyses of their cost-effectiveness. For the purposes of this review, we highlight 2 costeffectiveness studies—1 based on a Markov model and another based on a randomized trial—both of which investigated the cost-effectiveness of RT for metastatic bone pain. Model-Based Study: Palliative Bone RT for Metastatic Prostate Cancer Bone metastases are a common complication of metastatic prostate cancer, and with such a high prevalence, their optimal management has broad cost-effectiveness implications. Konski24 performed a Markov model-based CEA in which he compared pain medication alone, single-fraction RT, multifraction RT, and chemotherapy for the management of metastatic bone pain in hormone-refractory prostate cancer. He used the payer perspective, which in this study was Medicare. A Markov model was created and consisted of only a few states: pain after treatment, no pain after treatment, recurrent pain after retreatment, no pain after retreatment, and death. The main difference in the model between the fractionation regimens was the retreatment rate (consistent with clinical trials), so he was adequately able to isolate the relative balance between initial upfront cost of multifraction RT with lower downstream costs from fewer retreatments. The study found that chemotherapy was dominated (thus more expensive and less effective) by pain medication, but the ICERs for single-fraction and multifraction RT were $7000 per QALY and $36,000 per QALY, respectively. The author concluded that single-fraction RT was the most cost-effective solution, although if one assumes the societal WTP is $50,000 per QALY, in fact the multiple-fraction regimen is the better strategy. In any case, a Markov model was the only technique that could compare these different approaches—including
Decision and cost-effectiveness analysis both pain medication alone and chemotherapy—highlighting the strength of model-based approaches. Trial-Based Study: Dutch Bone Pain Study The Dutch Bone Pain study was a landmark randomized trial comparing single-fraction RT with multifraction RT for the palliation of bone pain.25 The study showed no significant benefit to multiple-fraction RT except a reduced rate of retreatment. As reirradiation carries an increased cost, the initial more costly 6-fraction regimen may ultimately be more cost-effective if it leads to fewer retreatments and downstream costs. Fortunately, the trial design incorporated a cost-effectiveness component.26 All patients were asked to fill out several qualityof-life questionnaires over time, including the EuroQol classification system (EQ-5D), which provides utility estimates. Treatment costs were estimated from a sampling of the treating hospitals, and patient-specific costs were obtained by questionnaires sent out to a subset of the patient population. The total costs were estimated over the first 12 weeks of treatment, as the authors found that subsequent costs were similar between the groups and estimating later costs would prove challenging to model. The quality-adjusted life expectancy was estimated by multiplying the utility for a given time period by the length of that period, until the patient died; there was a procedure to estimate the QALY for censored patients. The authors found that the single-fraction regimen was associated with more average QALYs and less average cost; in other words, it was a dominant strategy. However, given the small patient numbers and heterogeneity in the individual patient costs and QALYs, these cost and QALY differences were not statistically significant. Thus, they created an acceptability curve, in which they tested the ICER as a function of the statistical variability within the study and varying societal WTP levels. The authors found that the single-fraction was clearly cost-effective (P o 0.05) with a WTP less than $40,000 per QALY, and the P value with a WTP of $100,000 per QALY was borderline, with a P value of 0.09. As these cost and effectiveness estimates were directly taken from a landmark clinical trial, their analysis provides excellent support for single-fraction RT as the most cost-effective treatment. As the authors nicely concluded: “Compared with multiple-fraction RT, single-fraction RT provides equal palliation and quality of life and has lower medical and societal costs, at least in The Netherlands. Therefore, single-fraction RT should be considered as the palliative treatment of choice for cancer patients with painful bone metastases.”26 Interestingly, these 2 studies, using very different methodologies in different health care systems, ultimately reached similar conclusions on the cost-effectiveness of RT for bone metastases. The P value reflecting clear superiority for the single-fraction arm in the Dutch trial turned greater than 0.05 at $40,000 per QALY, which is roughly where Konski's study claimed the multifraction ICER lies. Nevertheless, it is important to remember that any one CEA is not the “final answer” for the cost-effectiveness of a given treatment. There are many implicit assumptions and analytic techniques that may sway the analysis toward 1 treatment over the other, and recognizing
23 and exploring these influential parameters is a critical part of the analysis and subsequent interpretation.
Conclusion With the increasing attention on health care spending and focus on reducing costs, including coverage for radiation oncology services, it is our professional responsibility to determine the treatments that provide the most value to society and to our patient population. CEA is an established discipline whose raison d'etre is to extract the greatest value from each health care dollar. In the future, we hope the tools in the CEA armamentarium would be robustly applied to novel RT techniques and technologies, with the goal of increasing the effectiveness of our services across society, given whatever financial resources are ultimately assigned to our field.
References 1. Meyer AM, Carpenter WR, Abernethy AP, et al: Data for cancer comparative effectiveness research: past, present, and future potential. Cancer 118(21):5186-5197, 2012 2. Spiro T, Lee EO, Emanuel EJ: Price and utilization: Why we must target both to curb health care costs. Ann Intern Med 157:586-590, 2012 3. Meropol NJ, Schrag D, Smith TJ, et al: American Society of Clinical Oncology guidance statement: The cost of cancer care. J Clin Oncol 27:3868-3874, 2009 4. Mariotto AB, Yabroff KR, Shao Y, et al: Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst 103:117-128, 2011 5. Shen XAP, Levin D, Rao VM, et al: Trends in radiation oncology services in the IMRT era: Evolving patterns of usage and payments in Medicare patients from 2000-2009. Int J Radiat Oncol Biol Phys 2011 [Abstract] 6. Schrag D, Kuntz KM, Garber JE, et al: Decision analysis—Effects of prophylactic mastectomy and oophorectomy on life expectancy among women with BRCA1 or BRCA2 mutations. N Engl J Med 336:1465-1471, 1997 7. Elkin EB, Weinstein MC, Winer EP, et al: HER-2 testing and trastuzumab therapy for metastatic breast cancer: A cost-effectiveness analysis. J Clin Oncol 22:854-863, 2004 8. Hillner BE, Smith TJ: Efficacy and cost effectiveness of adjuvant chemotherapy in women with node-negative breast cancer. A decision-analysis model. N Engl J Med 324:160-168, 1991 9. Stiggelbout AM, de Haes JC: Patient preference for cancer therapy: An overview of measurement approaches. J Clin Oncol 19:220-230, 2001 10. Weinstein MC, Stason WB: Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 296:716-721, 1977 11. Beck JR, Pauker SG: The Markov process in medical prognosis. Med Decis Making 3:419-458, 1983 12. Sonnenberg FA, Beck JR: Markov models in medical decision making: A practical guide. Med Decis Making 13:322-338, 1993 13. Heijnsdijk EA, Wever EM, Auvinen A, et al: Quality-of-life effects of prostate-specific antigen screening. N Engl J Med 367:595-605, 2012 14. Barr DA: Bending the Medicare cost curve for physicians' services: Lessons learned from Canada. J Gen Intern Med 27:1555-1559, 2012 15. Cigna Medical Coverage Policy. Proton Beam Therapy for Prostate Cancer. 16. Hunink MGM: Decision Making in Health and Medicine: Integrating Evidence and Values. Cambridge: Cambridge University Press, 2001 17. Gold MR, Siegel JE, Russell LB, et al: Cost-Effectiveness in Health and Medicine. Oxford, USA, Oxford University Press, 1996, pp 1 [online resource 450 p.] 18. Drummond MF, McGuire A: Economic Evaluation in Health Care: Merging Theory With Practice. Oxford: Oxford University Press, 2001 19. McCabe C, Claxton K, Culyer AJ: The NICE cost-effectiveness threshold: What it is and what that means. Pharmacoeconomics 26:733-744, 2008
24 20. Winkelmayer WC, Weinstein MC, Mittleman MA, et al: Health economic evaluations: The special case of end-stage renal disease treatment. Med Decis Making 22:417-430, 2002 21. Konski A, Speier W, Hanlon A, et al: Is proton beam therapy cost effective in the treatment of adenocarcinoma of the prostate? J Clin Oncol 25:3603-3608, 2007 22. Hayman JA, Lash KA, Tao ML, et al: A comparison of two methods for estimating the technical costs of external beam radiation therapy. Int J Radiat Oncol Biol Phys 47:461-467, 2000 23. Sher DJ: Cost-effectiveness studies in radiation therapy. Expert Rev Pharmacoecon Outcomes Res 10:567-582, 2010
D.J. Sher and R.S. Punglia 24. Konski A: Radiotherapy is a cost-effective palliative treatment for patients with bone metastasis from prostate cancer. Int J Radiat Oncol Biol Phys 60:1373-1378, 2004 25. Steenland E, Leer JW, van Houwelingen H, et al: The effect of a single fraction compared to multiple fractions on painful bone metastases: A global analysis of the Dutch Bone Metastasis Study. Radiother Oncol 52:101-109, 1999 26. van den Hout WB, van der Linden YM, Steenland E, et al: Single- versus multiple-fraction radiotherapy in patients with painful bone metastases: Cost-utility analysis based on a randomized trial. J Natl Cancer Inst 95:222-229, 2003
