&RPSDUDWLYH(IIHFWLYHQHVV5HVHDUFK'HFLVLRQ%DVHG (YLGHQFH &KDUOHV-RVHSK.RZDOVNL$GDP-RHO0UGMHQRYLFK Perspectives in Biology and Medicine, Volume 57, Number 2, Spring 2014, pp. 224-248 (Article) 3XEOLVKHGE\7KH-RKQV+RSNLQV8QLYHUVLW\3UHVV DOI: 10.1353/pbm.2014.0017

For additional information about this article http://muse.jhu.edu/journals/pbm/summary/v057/57.2.kowalski.html

Access provided by Ebsco Publishing (29 Dec 2014 05:37 GMT)

Comparative Effectiveness Research decision-based evidence

Charles Joseph Kowalski and Adam Joel Mrdjenovich

ABSTRACT  In the clinical research context, comparative effectiveness research

(CER) refers to the comparison of several health-care interventions administered under real-world conditions to individuals representative of the day-to-day clinical practice target population. We provide a brief history of CER and argue that CER can be used to deliver useful, but currently lacking information. Three study designs that can accomplish this are discussed, and incorporating CER into cost-benefit analyses is examined. The relationships between CER and evidence-based and personalized medicine are also considered, as is the challenge of implementing CER results into routine clinical practice.

S

urvival of the fittest in evolutionary biology has a counterpart in the evolution of research paradigms. It’s called survival of the funded, and there is a sense in which paradigms are even more adaptable than species. Whereas species may become extinct if their fitness declines below a critical threshold, paradigms can rise again, perhaps with a new name, following fiscal collapse, provided only that funding is once again made available. A current example is the born-again concept of comparative effectiveness research (CER), which achieved resurrection status with the passage of the

Health and Behavioral Sciences IRB, University of Michigan, Ann Arbor. Correspondence: Charles Kowalski, Health and Behavioral Sciences IRB, University of Michigan, 540 E. Liberty Street, Suite 202, Ann Arbor, MI 48104. E-mail: [email protected]. Perspectives in Biology and Medicine, volume 57, number 2 (spring 2014): 224–248. © 2014 by Johns Hopkins University Press

224

Comparative Effectiveness Research

Affordable Care Act (ACA) of 2010. The ACA established a private, nonprofit entity to oversee publicly financed CER. According to its “Draft Methodology Report,” the core mission of the Patient-Centered Outcomes Research Institute (PCORI) is to identify priorities for CER, fund studies, support improvements in CER methodology, and “assist patients, clinicians, purchasers and policy-makers in making informed health decisions by advancing the quality and relevance of evidence concerning the manner in which diseases, disorders, and other health conditions can effectively and appropriately be prevented, diagnosed, treated, monitored, and managed through research and evidence synthesis” (PCORI 2014). Importantly, PCORI has formed a number of internal advisory committees to set research priorities—panels that include practicing physicians, patient and consumer representatives, clinical and health services researchers, payers, manufacturers, and others. Involvement of multiple stakeholders is critical if PCORI is to identify those questions important to clinicians and patients.1 In this paper, we begin by providing a brief history of CER, including attempts to promote it through health-care policy. We review definitions of CER from the literature and describe CER as it exists today. We then summarize and counter several arguments against CER: that CER will place excessive demands on industry and have a negative impact on innovation; that CER will delay progress in the development of personalized medicine; that CER is not evidenced based; that CER will be used to ration health care; and that CER is challenging to implement in clinical practice.Along the way, we address issues and questions involving the validity of CER, ways in which findings from CER should be applied, and the degree to which such knowledge can be incorporated into everyday clinical decision-making. Specifically, we argue that CER can, in fact, produce valid information and serve as a useful tool in clinical decision-making, and we take the position that CER can be used to deliver useful, but currently lacking, information. The arguments, issues, and questions identified provide a context for the discussion that follows later in the paper, concerning three alternative classes of clinical study designs that should prove useful in CER. (For a detailed inventory of potential study designs, see chapter 5 of Rogers 2014.) Having argued that one should do CER, it is incumbent upon us to indicate how this can be accomplished. We propose that CER can be accomplished through the use of observational studies, pragmatic trials, and Bayesian/adaptive approaches.

A number of advocacy groups have noted the importance of being represented on the PCORI advisory committees. For example, the American Psychological Association (APA) has called for the inclusion of psychological expertise on the Board of Governors of PCORI: “In order to truly reform health care, any new infrastructure created for commissioning or conducting clinical comparative effectiveness research must adequately reflect the prominent role that psychological and behavioral factors play in disease prevention as well as the promotion and maintenance of wellness” (APA 2010). Since approximately half of the leading causes of death in the United States are attributable to personal health behaviors (CDC 2012; IO 2001; Mokdad et al. 2004), the APA’s position makes sense. 1

spring 2014 • volume 57, number 2

225

Charles Joseph Kowalski and Adam Joel Mrdjenovich

History and Definitions of CER Manchikanti et al. (2011) provide a detailed description of the evolution of the concept of CER, whose roots they traced back to mid-18th-century Scotland and the “arithmetical medicine” practiced by graduates of the Edinburgh Medical School. Concato et al. (2010) reach back even further, to James Lind who compared various treatments for scurvy in the mid-1700s. We focus on more recent developments in the United States, beginning with the Office of Technology Assessment (OTA), which was established in 1972 to provide Congress with objective advice on scientific and technical issues. The OTA lasted until 1995, when it was abolished as part of the downsizing of the federal government following the dictates of the so-called “Contract with America.” Although this action had symbolic currency, showing that Congress would not exempt its own bureaucracy from the necessary cuts (Park 2000), it also showed that political forces can have powerful effects on government-funded research agendas, including CER. Establishment of the OTA was followed by the short-lived National Center for Healthcare Technology (1978), the National Center for Health Services Research (NCHSR), and the Agency for Health Care Policy and Research (AHCPR, 1989– 99). As Gray, Gusamo, and Collins (2003) note, the AHCPR narrowly escaped being eliminated in 1995, only to be reauthorized (with a new mandate and name— the Agency for Healthcare Research and Quality, or AHRQ) with overwhelming support in 1999.2 In 2003, the Medicare Modernization Act funded the AHRQ to focus research on “outcomes, comparative clinical effectiveness, and appropriateness of healthcare items and services” (AHRQ 2014). Then, in early 2008, the Institute of Medicine (IOM) published a report that called for a national initiative that would support better decision-making about interventions in health care (Eden et al. 2008). This initiative centered around the concept of CER, for which the IOM offered an updated definition: “CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition, or to improve the delivery of health care. . . . The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels” (IOM 2009, 13). Other governmental entities offered similar definitions, which are detailed in Saver (2011). (See also Ashton and Wray 2013.) For example, the Federal Coordinating Council for Comparative Effectiveness Research, established as part of the 2009 American Recovery and Reinvestment Act, defined CER as “the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in ‘real world’ settings,” while the Congressional Budget Office defined CER as “a rigorous This period in the evolution of the CER concept is discussed by Wennberg (2010).

2

226

Perspectives in Biology and Medicine

Comparative Effectiveness Research

evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients.” Lauer and Collins (2010) also note that CER is not a new concept to the National Institutes of Health (NIH), “which has long recognized and supported the value of CER for providing evidence-based, well-validated approaches to medical care” (2182). Other sketches of CER are given in Ashton and Wray (2013), Gilbert (2009), and Meek, Renaud-Mutart, and Cosler (2013). et al. In light of these definitions, Marko and Weil (2010) state that CER “seeks to inform clinical decisions between alternate management strategies using data that reflects real patient populations and real-world clinical scenarios for the purpose of improving patient outcomes” (989). Similarly, Sox and Greenfield (2009) note the two key elements of these definitions are the direct comparison of effective interventions and their study in patients who are typical of day-to-day clinical care. These features would ensure that the research would provide information that decision makers need to know, as would a third feature, research designed to identify the clinical characteristics that predict which intervention would be most successful in an individual patient. The same research design would also help policymakers by identifying subpopulations of patients that are more likely to benefit from one intervention than the other. (203)

For easy reference, we list what seem to be the four defining characteristics of CER: 1. Direct comparison of interventions (no placebos) with respect to benefits, harms, and quality of life outcomes; 2. Real-world settings and clinical scenarios in patients likely to be candidates for the interventions; 3. Identification of subgroups of patients who are more likely to benefit from one intervention than another; and 4. Identification of the clinical characteristics that predict which intervention would be most likely successful in an individual patient.

Arguments Against CER Before proceeding to a discussion of alternative classes of clinical study designs that should prove useful in CER, we review and counter several arguments against CER to establish why the search for such study designs is so important. Excessive Demands on Industry and Negative Impact on Innovation

A number of proposals have been made to change FDA regulations so that CER would be incorporated into its decision-making processes—in particular, into the spring 2014 • volume 57, number 2

227

Charles Joseph Kowalski and Adam Joel Mrdjenovich

approval process for newly developed drugs. For example, Alexander and Stafford (2009) recommend that the FDA “generate [comparative effectiveness] data prior to the widespread adoption of a drug or treatment,” and “incorporate the principles of comparative effectiveness research throughout the process of approval and regulation” (2488). Stafford, Wagner and Lavori (2009) further suggest that what is known about the comparative effectiveness of new treatments be included in the labeling and marketing materials. Garattini and Bertele (2007) think that the FDA should require that new drugs prove superior to existing medicines before they are approved. The contrast between requiring CER for drug approval and the current practice of showing the new drug to be better than a placebo is striking, and it would have ramifications on a number of fronts. Most reactions from the pharmaceutical industry were entirely predictable, arguing that these requirements would add a major new hurdle to the development and approval of drugs, adding significant time and cost, costs that would impact on industry innovation (see, for example, Gottlieb 2009, 2011;Vernon, Golec, and Stevens 2010). Gottlieb (2011) argues that “Market forces are requiring drug companies to prove that their drugs are better than existing medicines. . . . More companies are doing comparator trials voluntarily” (3), but he fails to cite any of the many examples of industry-sponsored trials with designed-in biases that all but guarantee a favorable outcome (Kowalski and Mrdjenovich 2013b; see also Goldacre 2009). On the other hand, some in the industry are beginning to see the writing on the wall, and even supporting the movement. For example, Berger and Grainger (2010), representing Eli Lilly, state that “the companies that will survive and thrive in this new environment will be those that embrace comparative effectiveness research (CER) as the next logical step in the progression of requiring more rigorous evidence and recognize it as a necessary input for a value-driven healthcare system” (916), and they continue: “We believe that when well executed, CER will not exacerbate any discordance in the perceived value of innovation; rather, it will facilitate the alignment of biopharmaceutical innovators with the needs of patients, providers and payers” (919).There is no gainsaying that CER will be expensive. Still, the associated costs might be worth being able to answer the question “Is the new drug better than what is currently available?” instead of the question “Is the new drug better than nothing?” Note that, in the above, we have separated financial issues from legislative agendas. Of course, this separation cannot be so easily realized in the real world, where those with financial incentives lobby to influence political decisions. Avorn (2009) notes that “opposition to CER found its voice in commentators who claimed that these studies will inevitably lead to government domination of the doctor-patient relationship,‘cookbook medicine,’ and rationing” (1928), and Garber and Tunis (2009) observe that “The deepest concern about CER is that it will be misused, which is why some legislators seek to prohibit information on comparative

228

Perspectives in Biology and Medicine

Comparative Effectiveness Research

effectiveness from influencing coverage policy and payment decisions” (1927). One such prohibition was included in the Patient Protection and Affordable Care Act of 2010 and is critiqued by Neumann and Weinstein (2010). However, Garber and Tunis urge that “decisions will not be improved by discouraging the use of the most relevant and valid information about what works and in whom” (1927). Here, as in many walks of life, knowledge is power, and power is not always used fairly and wisely.The answer lies not in discouraging learning, but in fostering justice and wisdom. The knowledge about what works best in whom is needed to guide rational clinical decision-making, and this knowledge will not be provided by the pharmaceutical industry which has little incentive to go beyond the placebocontrolled trials currently needed to secure marketing approval. Delayed Progress in Personalized Medicine

Not all concerns about CER are attributable solely to an industry with what Fleck refers to as “an obsessive-compulsive profit disorder.”3 Advocates of personalized medicine come from many walks of life, and some have raised concerns that concentrating on CER “might stymie progress in personalized medicine” (Garber and Tunis 2009, 1925, quoting from the Partnership to Improve Patient Care, a coalition of 36 industry, patient-advocacy, and clinician organizations). If it were true that CER was limited to the use of RCTs to acquire the information it needs to inform clinical decision-making, there would be cause for concern. However, Luce et al. (2009) argue that CER can and should employ other study designs to satisfy the evidentiary needs of CER. They show how Bayesian/ adaptive approaches and the use of pragmatic (practical) clinical trials (PCTs) can fulfill these needs. Similarly, Epstein and Teagarden (2010) note: “A tension between personalized medicine and CER is created when pressure is placed on CER to conform to the prevailing RCT model and where that pressure imposes severe constraints to its usefulness to personalized medicine. . . .This tension does not have to exist. CER studies can, and are typically designed to, include a wide range of patient populations common to all healthcare provider environments” (906). They go on to say that the use of epidemiological research methods “can help nonrandomized studies address personalized medicine concepts,” taking advantage of “the huge and ever-expanding claims databases, registries and natural cohorts that are emerging in this increasingly wired healthcare delivery system” (906).They also cite the approaches previously mentioned by Luce and colleagues. Garber and Tunis (2009) also recognize that “CER may well require innovative approaches to clinical trials—such as adaptive, pragmatic, or other novel trial designs. . . . CER is not a panacea, but it is a key to individualized care and innovation, not a threat” (1927). Use of one or another of these alternative study designs can produce evidence that will help guide individualized care. 3 This descriptive quote is from a back-cover review of Arnold (2009) by Leonard M. Fleck. Another good one is the “apothecary industrial complex,” mentioned by Goldacre (2009, 155).

spring 2014 • volume 57, number 2

229

Charles Joseph Kowalski and Adam Joel Mrdjenovich

Chen (2009) gives an example of an empathetic physician, wanting to provide individualized care but lacking the evidence to do so, who was motivated to try extreme measures to try to reverse what seemed surely to be a terminal cancer. She concludes: “Without evidence, the decision to do something, anything, can seem more humane than the decision to do nothing . . . but it is not always best for our patients” (16). The choice between aggressive surgery and palliation will never be easy, but relevant data may actually serve to “strengthen the doctor-patient relationship” (16). Two chapters in Evans et al. (2011)—chapter 3,“More Is Not Necessarily Better,” and chapter 4, “Earlier Is Not Necessarily Better”—are replete with examples of where CER might inform individual decision-making about choices affecting outcomes that are important to the patients who make them. The need may be especially acute in decisions about mental health care, in which “information on absolute and especially relative effectiveness [of treatments] under ‘real world’ practice conditions is often lacking” (Wang, Ulbricht, and Schoenbaum 2009, 784). Mental illness is not only highly prevalent, it is the leading cause of disability in the United States for persons between the ages of 15 and 44 (NIMH 2013a). In fact, the burden of disease from mental illness exceeds that of any other health condition (Whiteford et al. 2013).Yet, even as spending on mental health care has increased in the United States, due in large part to the adoption of newer psychotropic medications, the need for effective treatment among individuals with mental illness commonly goes unmet. Despite years of implementation and experience with various treatments for mental health conditions, and a 65% increase in the utilization of mental health services over the past decade, half of Americans who experience a mental illness receive no treatment at all, and one-quarter receive treatment that is not therapeutic or indicated for their condition (Susser et al. 2006;Wang, Ulbricht, and Schoenbaum 2009). Clearly, knowing which treatments are most effective could enhance the capacity to prevent, diagnose, treat, and manage mental health conditions. Not Evidence-Based

Detractors of CER sometimes claim that CER is not evidence-based. In this section, we address the validity of CER and its relationship to evidence-based medicine (EBM). It will be seen that the definitions of these two activities are such that it is hard to find fault with either; it is only in their implementation or application that any questions have arisen. CER is in many respects similar to the concept of EBM, which is defined by Sackett et al. (1997) as “The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (2). It is hard to imagine anyone disagreeing with the general tenor of this statement. Of course one wants to base decisions concerning medical care on the best available evidence. But when one asks the obvious question of just what constitutes the current best evidence, not all will agree.

230

Perspectives in Biology and Medicine

Comparative Effectiveness Research

One sticking point is the so-called “hierarchy of evidence,” which accords RCTs and meta-analyses of RCTs special prominence. Many have questioned the special, gold standard status of the RCT, as evidenced by two special issues of this journal in 2005 (48: 475–584) and 2009 (52: 161–331). (See Schechter and Perlman 2009, who summarize the 2005 papers before introducing the 2009 contributions.) But despite the existence of these concerns, there are many who believe, often passionately, that the only way to study treatment effectiveness is with an RCT. As seen above, CER is a deliberate departure from this notion; CER is often best accomplished using alternative study designs. Indeed, the RCT is not the gold standard for CER, even in the case of pragmatic trials.4 It seems likely that this is the basis for some to claim that CER is not evidence-based (Manchikanti et al. 2011), but reliance on RCTs hardly qualifies as a necessary condition for reaching evidence-based conclusions (Kowalski 2013). To the extent that EBM relies on the RCT, relevance to individual cases will be limited: RCTs, by design, strive to ensure that the only difference between the groups receiving different interventions are the interventions themselves, which generally results in restrictive inclusion criteria (such as no comorbidities) so that the results will not apply to many (most) individuals. Ironically, although proponents of EBM claim that their approach is focused on individualized care, EBM is often critiqued for being so invested in clinical epidemiology that it loses sight of individual patients (Tonelli 2001). As put by Payne (2009):“A definition of ‘high quality’ medical care is too narrow when it relies only on empirical evidence gathered by randomized controlled clinical trials” (14). The National Institutes of Health (NIH), widely acknowledged to be rigorous and evidence-based, has long viewed CER as “providing evidencebased, well-validated approaches to medical care” (Lauer and Collins 2010, 2182). Rogers (2014) thinks that CER “can be considered an offshoot of a global movement in evidence-based medicine and interdisciplinary science,” emphasizing the harmonization of the two movements, certainly not antagonism (8). The devil is in the details. Let’s start with the IOM definition of CER: “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care” (IOM 2009, 13). As with EBM, it is hard to imagine anyone disagreeing with the notion that evidence should be an important factor in deciding between available health-care interventions or delivery mechanisms. Just about everyone agrees that CER has the potential to deliver useful, but currently Pragmatic RCTs, as explained in Kowalski (2010), will have more relevance to clinical practice than their explanatory counterparts. Although pragmatic RCTs will prove valuable in addressing many CER questions, we stop short of suggesting that they should be viewed as the gold standard of CER generally. Other approaches may be better suited for some questions—for example, observational studies will often provide product safety information that cannot be obtained by any RCT (Kowalski and Mrdjenovich n.d.). Bayesian methods can incorporate relevant information about comparative effectiveness from sources other than the study under consideration, including information accruing simultaneously with that being captured by the study. 4

spring 2014 • volume 57, number 2

231

Charles Joseph Kowalski and Adam Joel Mrdjenovich

unavailable, information. However, “it is when the discussion shifts to how CER data should be used to make payment decisions that opinions begin to quickly diverge among practitioners, patients, payers, and manufacturers of medical therapies including pharmaceutical companies” (Meek, Renaud-Mutart, and Cosler 2013, 196).

Should CER Be Used In Financial Decision-Making? Once the validity of CER has been established, the question of how such knowledge can be applied remains. Some detractors argue that when CER becomes cost-effectiveness research, the result will be health-care rationing, or even the establishment of “death panels.” Often, detractors of CER do not explicitly mention payment decisions when voicing their reservations. For example, Manchikanti et al. (2010b), while admitting that EBM and CER “share many similarities and goals,” somehow find some differences they believe important enough to share: “EBM is essentially focused upon the use of the right (types and extent of) knowledge to guide the right and good intentions and actions of medical practice, which is fundamental to prudential clinic decision-making. In contrast, CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels” (E56). EBM is seen by them to guide only the clinical encounter; CER, on the other hand, will be used—read “abused”—by consumers, policy makers and purchasers. In a later article, Manchikanti et al. (2011) quote the mission of PCORI as being “to promote comparative effectiveness research (CER) to assist patients, clinicians, purchasers, and policy-makers in making informed health decisions by advancing the quality and relevance of evidence concerning the manner in which diseases, disorders, and other health conditions can effectively and appropriately be prevented, diagnosed, treated, monitored, and managed through research and evidence synthesis,” but they conclude from this that “PCORI is operating in an ad hoc manner that is incompatible with the principles of evidence-based practice” (E249). They also note that, “PCORI and CER have been described as government-driven solutions without following the principles of EBM with an extensive focus on costs rather than quality” (E249). This surely is not in evidence from the content of the definitions cited previously, nor from the stated aim of PCORI (2014), which conscientiously, explicitly, and judicially includes an evidenced-based approach: “The Patient Centered Outcomes research Institute (PCORI) was created to fund research that will provide patients, their caregivers and clinicians with the evidence-based information needed to make better informed health care decisions.” Nor is it clear where the claim that PCORI has “an extensive focus on costs” comes from. The Affordable Care Act, which created PCORI, states that “the findings of PCORI-sponsored research cannot be construed as mandates for practice guidelines, coverage recommendations, payment,

232

Perspectives in Biology and Medicine

Comparative Effectiveness Research

or policy recommendations” (Neumann and Weinstein 2010, 1495). In addition, the Affordable Care Act includes several explicit limitations on Medicare’s payment formulation and the use of CER (Pearson and Bach 2010). Despite these mandated safeguards on the uses to which CER may be put, Manchikanti et al. (2010a, 2010b, 2011) vehemently oppose CER at its roots. It appears that they are protecting the pain physicians’ financial turf, much in the same way that North American Spine Society attack on the Agency for Health Care Policy and Research (AHCPR) in 1996 was aimed at discrediting agency guidelines indicating that nonsurgical approaches were generally preferred for managing acute back pain problems (Deyo et al. 1997). Such turf protection efforts are perhaps understandable, but not all professional societies have taken such a narrow stance. For example, the American Psychological Association (APA) recognizes both that CER is evidence-based, consistent with the IOM definition, and of value in guiding practice. Consistent with years of work in EBM, the evidence-based practice movement has become a key feature of mental health care systems and policy. A number of state-level initiatives encourage or mandate the use of specific mental health treatments within Medicaid programs (Tanenbaum 2005). At the federal level, a major initiative of National Institute of Mental Health (NIMH) and Substance Abuse and Mental Health Services Administration (SAMHSA) focused on the promotion, implementation, and evaluation of evidence based practices within state mental health systems (NIH 2004). The APA has been addressing issues of how best to conceptualize and examine bases for practice for decades, which has led to the development of “Evidence Based Practice in Psychology” (EBPP), defined as the “integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences. . . .The purpose of EBPP is to promote effective psychological practice and enhance public health by applying empirically supported principles of psychological assessment, case formulation, therapeutic relationship, and intervention” (APA Presidential Task Force 2006, 284).Analogous to the EBM/CER relationship, the APA provides clarification on the relation between “empirically supported treatments” (ESTs) and EBPP. ESTs consist of specific mental health treatment interventions that have been shown to be efficacious in controlled clinical trials. ESTs start with treatments and ask whether the treatment works for certain disorders under specific circumstances (Chambless 2007). EBPP, on the other hand, involves a broader range of activities (assessment, case formulation, and so forth). EBPP begins with the patient and asks what research evidence (including, but not limited to, results from RCTs) will assist clinicians in achieving the best outcomes for patients.Thus, EBPP is really a process of decision-making that incorporates multiple types of research evidence drawn from a variety of designs and methods, not just RCTs (APA Presidential Task Force 2006). Congress has funded CER, but it has strictly limited the uses to which CER can be put, particularly with regard to making payment decisions.This is, at least in part, due to some of the compromises needed to accommodate the differing viewpoints

spring 2014 • volume 57, number 2

233

Charles Joseph Kowalski and Adam Joel Mrdjenovich

of various stakeholders, but it may be wise to keep those producing CER separate from those applying it. There is a distinction between clinical-effectiveness-analysis and cost-effective-analysis, and the latter is much more politically contentious than the former. Wilensky (2008) argues for such a separation, noting: “Because clinical effectiveness is the most basic and costly step in learning how to spend smarter, it should proceed first and in as politically protected a manner as possible” (967). Writing before the establishment of PCORI, she asserted that “the sustainability of such a center will only become clear after it survives the first clinical comparative effectiveness information that contradicts conventional wisdom or endangers the latest therapy du jour” (967).There would seem to be much wisdom behind keeping the development of CER separate from the application of CER results.The former is scientifically oriented, with the focus being on producing valid, evidence-based research.The latter is necessarily political: different stakeholders will see CER results through different lenses, which may well result in conflicting viewpoints. With regard to using CER to make payment decisions, Pearson and Bach (2010) suggest that “Politically, the straightforward idea of paying equally for comparable results would make sense to most Americans. Powerful advocates— including purchasers, patients, and producers of cost-effective services—could be expected to form a natural coalition in support of an evidence-based approach to reimbursement” (1802).They acknowledge that the change toward “evidence-based reimbursement would appear threatening to many manufacturers whose current business models are based on the existing payment structure,” but they argue that “Ultimately, however, paying more for better results is the best way to spur the kind of innovation desired by most patients, clinicians, and payers” (1802). Pearson and Bach also acknowledge that Just mentioning Medicare and comparative effectiveness research in the same sentence is enough to raise temperatures in Washington health policy circles. Those who see this research as a threat to patient choice or provider profits do not want it applied to Medicare. Those who see it as a remedy for the nation’s health care ills do not want a politically explosive link to Medicare that might bring down the whole comparative effectiveness initiative. (1796)

Despite the political resistance to using CER in setting Medicare costs, Pearson and Bach have developed a conceptual and practical tool—one that we believe could be implemented—to link coverage decisions with evidence-based reimbursement levels. Payment levels would depend on whether the proposed service had evidence of superior, or comparable, or yet-to-be determined comparative clinical effectiveness. They give an example of how this dynamic pricing approach would work in practice, and they conclude that “using comparative effectiveness research to set reimbursement rates at the time of coverage is a promising option that we would argue the nation cannot afford to ignore” (1803).

234

Perspectives in Biology and Medicine

Comparative Effectiveness Research

However, while noting that CER can be used to inform payment decisions, it would be unrealistic to expect that resistance to doing so will easily fade. On the side of the notion that CER should be included among the factors considered in making payment decisions is the belief that such usage lies at the very conceptual base of CER. For example, Gilbert (2009) notes that “to ignore the cost of an intervention would defeat one of the primary purposes of comparative effectiveness research—to help control skyrocketing health care costs by showing whether costly new medications and medical technologies are more effective than less expensive interventions” (12). Many think that considerations of bang-for-the-buck are necessary, since budgets for health care are limited. For example, Neumann and Weinstein (2010) suggest that the notion that the country can avoid the difficult trade-offs that cost-utility analysis helps to illuminate was “magical thinking . . . another example of our country’s avoidance of unpleasant truths about our resource constraints” (1496). Spending smarter can in fact reduce overall spending in some situations, and this has been recognized by some payers. For example, Doyle (2011) points out that “in May 2010, WellPoint, Inc. became the first health benefits company to develop standardized CER guidelines that it will use in formulary decision making. . . . The company’s guidelines explicitly note that a more expensive medication can be less expensive overall if the member’s health is improved, resulting in the use of fewer healthcare resources” (30, emphasis added). On the other hand, not everyone agrees with these arguments based on supply-side or “lifeboat ethics.” Koch (2012) observes that to say resources are not infinite is not to say they are necessarily scarce and must be rationed. We may choose to limit service to the elderly, the immigrants, or the poor on the basis of scarcity, but what we’re really saying is that we’d rather spend our resources on other things, or people. In embracing scarcity as a natural condition, we permit the empowerment of the wealthy through lower tax rates rather than insisting upon tax programs that assure monies for care of the elderly and the poor. (74)

Our take on the question of whether CER should be used in making payment decisions is that CER should be one of the factors considered in arriving at such decisions, but not the only factor. Even if we were to decide, following Koch, that all are entitled to the treatment thought best, and we are willing to pay for that treatment, CER is useful in identifying what is best in this context.When comparative effectiveness is not known, cost-effectiveness judgments frequently come down to cost alone, and this, too, is too narrow a view for a prudent decision-maker to take. Simply choosing the cheapest treatment may represent lost opportunity costs if this cheaper treatment results in fewer overall health benefits. Nor is the most expensive treatment always the best treatment. For example, Evans et al. (2011) note that human albumin solution, given as an intravenous drip, was used for some 50 years to resuscitate burned and other critically ill patients before a proper trial showed

spring 2014 • volume 57, number 2

235

Charles Joseph Kowalski and Adam Joel Mrdjenovich

that it was no more effective than salt water. Since albumin is some 20 times more expensive than saline, the amount of money wasted was substantial. Even with the commitment to treat all needing resuscitation, costs be damned, it would be difficult to argue that the theory suggesting that albumin would reduce mortality should not be tested. It is important to emphasize that CER is an element of decision-making, not a complete, stand-alone procedure for making resource allocation decisions in health care. CER cannot incorporate all of the values relevant to such decisions, and all of the values are needed to shape fair choices. PCORI (2014) has recognized this, and its working definition of patient-centered outcomes research states that such research “accesses the benefits and harms of preventive, diagnostic, therapeutic, palliative, or health delivery system interventions to inform decision making, highlighting comparisons and outcomes that matter to people … inclusive of an individual’s preferences, autonomy and needs, focusing on outcomes that people notice and care about such as survival, function, symptoms, and health-related quality of life.” The experiences of other countries also may be helpful in guiding policy development. Schmidt and Kreis (2009) report on two reviews that assessed CER use in Australia, France, Germany, the Netherlands, Sweden, and the United Kingdom, and they conclude that “health technology assessment [CER] systems have played central, if not transformative, roles in contributing to evidence-based decision-making and in identifying interventions that provide the most value for the money” (20).While it is true the United States has a unique sociopolitical structure, the ideal of justice for all motivates any strategy for a country’s provision of health care to its citizens. A Challenge to Implement in Clinical Practice

Even if CER is evidence based and applicable to questions regarding the most cost-effective ways to combat disease, it can still be asked whether this information can be implemented—incorporated into everyday clinical decision-making. One of the biggest problems with CER may be the inability to incorporate its usage into clinical practice in a timely fashion. For example, one approach to implementing CER into clinical practice in the mental health arena has been through the development of guidelines for best practice. The process has been gradual, due to the range of evidence that must be weighed in order to develop such guidelines, not only in terms of efficacy, but also in terms of clinical utility (generalizability, feasibility, patient acceptability, costs and benefits). (For a historical and contextual discussion, see APA Presidential Task Force 2006.) At an even more basic level, Giffin and Woodcock (2010) worry that it would be difficult just to do all the clinical trials required to support CER. They argue that “the clinical trial system is already at capacity and will not be able to absorb large amounts of comparative effectiveness research without diverting resources from other needs,” and they propose “a federally funded national clinical research

236

Perspectives in Biology and Medicine

Comparative Effectiveness Research

infrastructure . . . encouraging community-based clinicians and their patients to participate in the trials” (2075). This has the upside of ensuring the trials be run in the real world, but it is relevant only in those situations in which the RCT is the best medium for producing CER information. The engagement of communitybased clinicians and their patients in a variety of research contexts is surely welcome, and a blueprint for how this may be accomplished may already be available. The Department of Veterans Affairs (VA) has been conducting CER for many years (see Concato et al. 2010).They have also taken some steps to smooth the transition from research to practice. For example, the Quality Enhancement Research Initiative (QUERI) disseminates research findings to VA policymakers, providers, and patients, while the Center for Implementation Practice and Research Support (CIPRS) aims to improve the quality and performance of VA health care through implementation researchand practice. Despite this infrastructure, Naik and Petersen (2009) note that “the translation of this investment into practice, enabling new laboratory discoveries to reach patients’ bedsides, is frustratingly slow” (1929). They conclude that “the primary goal of CER is to enhance the translation of new medical discoveries into safe and high-quality health care for all Americans. This goal can be achieved only if our renewed investment in CER includes a commitment to implementation research” (1931). Wennberg (2010), while encouraged by the Affordable Care Act’s provisions for CER and the potential for CER to aid in clinical decision-making, thinks that there were some important omissions—that the definition of CER in the Act was too narrow to right all of the wrongs in the current health-care system. For example, he thinks that not enough attention is given to situations in which patient preferences are powerful factors in determining treatment choice, and that more emphasis should have been given to the development of the science of health-care delivery. However, while the ACA may have failed to explicitly address these concerns, PCORI has undertaken a thoughtful expansion of Congress’s basic blueprint. PCORI has identified five priorities for its research agenda: 1. Assessment of options for prevention, diagnosis and treatment; 2. Improving health-care systems; 3. Communication and dissemination research; 4. Addressing disparities; and 5. Accelerating PCOR and methodological research.

There is plenty of room under items 2 and 3 to conduct meaningful implementation science research. During its first round of proposal funding, 12 of the total 25 projects funded were in these categories, and all of these included bench-to-bedside components. While it remains to be seen if PCORI will survive the rigors of the stakeholder-inspired politics that it surely faces, but PCORI’s evolutionary development has benefitted from the experiences of its predecessors, retaining those traits contributing to survival potential and jettisoning those that spring 2014 • volume 57, number 2

237

Charles Joseph Kowalski and Adam Joel Mrdjenovich

would diminish its fitness. An encouraging sign is that CER and PCORI are being portrayed positively in the popular press. A recent article in Time magazine, while recognizing that the “comparative-effectiveness debate” is apt to continue, estimated that we could save some $28 billion if we “allow and fund comparative-effectiveness evaluations in decisions to prescribe drugs, tests, and medical devices” (Brill 2013, 31).

How Can CER Be Accomplished? Having argued that one should do CER, it is incumbent upon us to indicate how this can be accomplished. The classical randomized, placebo-controlled, doubleblind clinical trial has played a prominent role in testing the efficacy of new healthcare interventions, especially drugs, but it is not designed to answer the questions posed by CER. Luce et al. (2009) note that as currently designed and conducted, “many RCTs are ill suited to meet the evidentiary needs implicit in the IOM definition of CER: comparison of effective interventions among patients in typical patient care settings, with decisions tailored to individual patient needs” (206). The typical placebo-controlled RCT does not directly compare effective interventions; it is not carried out in typical patient care settings; individual patient needs are subordinated to standardized treatment protocols; and randomization does not allow for patient choice or for treatments tailored to individuals. Many RCTs employ simple outcome measures (or surrogates for these), ignoring quality-of-life outcomes that may be more important to patients and more relevant for policymakers (Kowalski, Pennell, and Vinokur 2008; Kowalski et al. 2012). Even the staunchest proponents of RCTs readily admit that they do not allow the identification of subpopulations of patients that are more likely to benefit from one intervention than the other. In the introduction to his 2007 book, Rothwell cites two sources that illustrate this: “One should look for treatment-covariate interactions, but . . . one should look very cautiously in the spirit of exploratory data analysis rather than that of formal hypothesis testing (David Byar, 1985),” and, more (melo)dramatically,“Subgroup analysis kills people (Richard Peto, 1995)” (x). He also cites a source relevant to the RCT’s ability to provide information that will allow decisions to be tailored to individual patient needs: “It is right for each physician to want to know about the behaviour to be expected from the intervention or therapy applied to his individual patient . . . it is not right, however, for a physician to expect to know this (John W. Tukey, 1986).” We turn now to a consideration of several study designs that promise to satisfy the evidentiary demands of CER: (1) observational studies, (2) pragmatic clinical trials, and (3) Bayesian/adaptive approaches. Although they have certain points of contact, each study design has its own strengths and limitations, and each holds much promise for contributing to CER.

238

Perspectives in Biology and Medicine

Comparative Effectiveness Research Observational Studies

A valuable overview of the role of observational investigations in CER was given by Marko and Weil (2010). Their adaptation of the IOM definition of CER is that it “seeks to inform clinical decisions between alternate treatment strategies using data that reflects real patient populations and real-world clinical scenarios for the purpose of improving patient outcomes” and “to combine treatment efficacy data with quality of life, outcomes, and other forms of effectiveness data to guide selection of optimal patient management strategies” (989). The first part of this statement clearly underscores the potential utility of observational studies, in that they are, by definition, based in the real world where patients have preferences, help make treatment decisions, are variably compliant, have other conditions and may be taking other drugs to treat them (Kowalski and Mrdjenovich 2013a). The second part admits to the role that RCTs have in establishing efficacy, but implicitly notes that while efficacy may be necessary for a treatment to be considered a viable option, it is not sufficient to cement a decision. We may be happy that a drug lowers our blood pressure or cholesterol level, but not if it does so at the expense of unwanted, bothersome side effects, or if it is priced beyond what we can afford. We need to go beyond efficacy, and to study effectiveness (Kowalski 2010). Marco and Weil (2010) note Effectiveness studies, which are central to CER, have the more complex goal of assessing the beneficial effects of the particular treatment in the context of every­day practice. Here, the patient population is more heterogeneous, conditions are less controlled, and multiple variables may impact the outcomes. Studying the beneficial effects of therapy therefore frequently requires larger sample populations that are followed over longer periods of time, a situation well suited for observational research … effectiveness research requires that the effects of a number of potential variables and confounders be analyzed rather than elimi­ nated. (991, original emphasis)

There are a number of different kinds of observational studies. Von Elm et al. (2007) describe many of these and give guidelines for reporting their results. The ones that should prove most useful in CER are prospective cohort studies and patient databases, but others may also find a niche. Dreyer et al. (2010) have written a paper providing reasons “why observational studies should be among the tools used in comparative effectiveness research,” the gist of which is that observational studies are best used to evaluate real-world applicability of evidence derived largely through randomized trials; to study patients and conditions not typically included or studied in randomized trials; to better understand current treatment practices and how patients are assessed in order to design an appropriate clinical trial; and to provide information that can be derived only through larger studies or long-term follow-up. (1820)

spring 2014 • volume 57, number 2

239

Charles Joseph Kowalski and Adam Joel Mrdjenovich

In addition, they provide examples where observational studies can contribute useful information in situations where RCTs cannot be performed, including when RCTs are precluded by ethical considerations, when larger studies are required, when treatment adherence differs, when providers have different training, and when treatments are off-label (Kowalski 2013). In order to assure that observational studies are performed in accordance with best practices, they point to several sets of guidelines that can be used to guide observational research efforts.The first of these focuses specifically on observational studies in CER. The GRACE (Good ReseArch for Comparative Effectiveness) principles comprise a series of three questions to guide evaluation of observational studies of comparative effectiveness (Dreyer et al. 2010): 1. Were the study plans (including research questions, main comparisons, outcomes, etc.) specified before conducting the study? 2. Was the study conducted and analyzed in a manner consistent with good practice and reported in sufficient detail for evaluation and replication? 3. How valid is the interpretation of CE for the population of interest, assuming sound methods and appropriate follow-up?

Elaborations on these questions, as well as links to publications that cite use of them, are available on the GRACE website (http://www.graceprinciples.org). A checklist is under development; this will undoubtedly have some overlap with the STROBE (Strengthening The Reporting of Observational studies in Epidemiology) checklist (von Elm et al. 2007; http://www.strobe-statement.org). STROBE will be of interest to those doing observational research of any kind; GRACE focuses on the use of observational studies in CER. The GRACE principles stress the importance of a prospective approach to database construction where all the data are collected in accordance with a protocol. However, some contributions to CER can be expected from retrospective analyses of databases collected using less specific criteria. Berger et al. (2009) spell out good research practices for CER using secondary data sources, including claims databases, patient registries, electronic medical record databases, and other routinely collected health-care data. In particular, they modify the CONSORT and STROBE guidelines to these special data sources and provide a list of five recommendations to guide CER in these contexts. Finally, all CER needs to evaluate the risks, as well as the benefits of the interventions being compared, and observational studies are especially valuable in this regard. RCTs are virtually never large or long enough to be able to discover rare or late-developing side effects, and observational study designs are almost always required to derive accurate safety profiles.

240

Perspectives in Biology and Medicine

Comparative Effectiveness Research Pragmatic Trials

We have already noted that RCTs, as usually designed, are ill-suited to provide the information required by CER. In addition, the R part of the RCT is not always necessary or possible. Marko and Weil (2010) describe four situations where randomization cannot or need not be applied: randomization may not be necessary (when the effect under study is dramatic and possible confounding effects negligible); it may be impossible (clinicians or patients refuse to participate); it may be unethical (equipoise may not exist); or it may be self-defeating (when the effectiveness of the intervention depends on the subjects’ active participation in or knowledge of the treatment (Kowalski 2013; Kowalski and Mrdjenovich 2013a).5 Observational studies can often prove useful in these cases, and, when appropriate, the methods described in the preceding section can be employed. However, when randomization is not problematic, the classical RCT for efficacy may be restructured more along pragmatic lines so that the real-world demands of the research setting are maintained. We have previously described the differences between explanatory and pragmatic clinical trials, noting that they differ with respect to the definitions of treatments, the assessment of the results, the selection of subjects, and the ways in which the treatments are compared (Kowalski 2010; Marco and Weil 2010). Along each of these dimensions the pragmatic approach mirrors a real-world, not a rigidly controlled, stance: treatment definitions are generally complex and flexible; assessments are made along a number of dimensions, including things that are important to patients, like cost, side effects, and ease of administration; subjects are recruited so as to be representative of those who will be treated in the future, including those with co-morbidities and those who may comply imperfectly. The intent of a pragmatic trial is to identify the treatment that will perform the best in typical patient care settings, and so is better suited than its explanatory counterpart to contribute to CER. Borgerson (2013) has argued that all clinical trials are more ethically justifiable the fewer the idealizing elements included in the trial design—in other words, the more pragmatic the approach. A clever way to visualize where a proposed study is located along the explanatory—pragmatic design continuum is given by Thorpe et al. (2009), and the closer it is to the pragmatic extreme, the more likely it is to fit into the CER agenda.What to look for in a (general) pragmatic trial is discussed by Zwarenstein et al. (2008). Tunis, Stryer, and Clancy (2003) focus on PCTs and CER and the contribution PCTs could make to decision-making in clinical and health policy.They emphasize that PCTs are trials in which the hypothesis and study design are developed specifically to answer the questions faced by decision makers, and they note that the current supply of PCT results is inadequate to answer many important clinical and 5 A similar list was given by Black (1996), who points to instances when experimentation was unnecessary, inappropriate, impossible, or inadequate. Included in the “inappropriate” category were studies to detect rare or long-developing adverse drug reactions.

spring 2014 • volume 57, number 2

241

Charles Joseph Kowalski and Adam Joel Mrdjenovich

policy questions. They suggest that the reasons for this include the fact that PCTs are typically costly and many potential sponsors lack the motivation to assume these costs. One way to reduce costs is to adopt “large simple trials” (Peto and Baigent 1998), trials that are much like the PCTs described above, except that they employ simple outcome measures. There is no doubt that this does in fact reduce costs, but it does so at the expense of ignoring certain outcomes (such as quality of life) that are particularly important to patients and policy makers (Kowalski, Pennell, and Vinokur 2008; Kowalski et al. 2012). These will need to be balanced on a case-bycase basis; the design of a study should in every case be driven by the question to be answered and by any extant practical, political, or ethical constraints (Kowalski and Mrdjenovich 2013a). When PCTs are appropriate, the associated increased costs may in fact be costeffective. Wang, Ulbricht, and Schoenbaum (2009) cited several PCTs to illustrate the value of CER for improving practice, policy, and cost effectiveness in mental health care. For example, although efficacy trials had suggested that atypical antipsychotic medications were better than conventional antipsychotics for the treatment of schizophrenia, subsequent findings from a series of PCTs, the NIMH Clinical Antipsychotic Trials of Intervention Effectiveness, called prevailing patterns of antipsychotic use into question and sparked a debate among policy researchers, policy makers, and patient advocates concerning the degree to which results can or should be used to inform prescription drug policies (Rosenheck et al. 2006). In a related study of atypical antipsychotics used to treat dementia, cost effectiveness analysis showed that effectiveness was comparable between active treatments and placebo, but health care costs were significantly lower for patients assigned to placebo (Rosenheck et al. 2006). The STEP-BD trial examined treatment options for bipolar disorder and demonstrated that the common practice of adding antidepressants to a mood stabilizer did not increase the risk of mania as widely believed (NIMH 2013b). A final example of the potential contribution of PCTs in the mental health context is the Sequenced Treatment Alternatives to Relieve Depression trial, which focused on options for patients who do not respond to initial treatment with antidepressant medication. This trial highlighted the need for CER to identify strategies that can speed recovery from depression and provided empirical support for practice guidelines consistent with the large body of research that has established the effectiveness of strategies to improve quality of care for depression (Rush 2007). Another way in which costs might be reduced, while maintaining at least some of the advantages of randomization, is through the use of cluster randomized trials (CRTs). In a CRT, the unit of randomization may be a community, worksite, school, family, or a health plan. CRTs are often used when evaluating health promotional campaigns (in communities) or educational strategies (in schools), where randomization at an individual level would be difficult or impossible. They have been used less often on CER involving drugs, but there is no reason that they

242

Perspectives in Biology and Medicine

Comparative Effectiveness Research

cannot be usefully employed in this venue. The feasibility and ethics of the use of these trials in the comparison of drug effectiveness at health plans is considered by Sabin et al. (2008) and Kowalski and Mrdjenovich (2013b). Bayesian/Adaptive Approaches

Luce et al. (2009) think that Bayesian/adaptive approaches to clinical studies could be particularly valuable in meeting CER evidence challenges, including “the need to compare multiple active treatment strategies in real-world settings, to focus experimental resources on the most promising approaches, to identify patient subgroups in which treatments are more (or less) effective, to introduce new treatments into the evaluation process as quickly as possible, and to make optimal use of all existing experimental information when a study is designed and as it is conducted” (208). The classical RCT can be useful in guiding one-time yes/ no decisions like marketing approval for a drug, but RCTs are inflexible and not well suited for comparing interventions as evidence accumulates over time from all relevant sources of information, both within and exterior to the confines of the trial. However, these desiderata are within the reach of the Bayesian. Berry (2006) notes: Bayesian approach is ideally suited to adapting to information that accrues during a trial. . . . Accumulating results can be assessed at any time, including continually, with the possibility of modifying the design of the trial, for example, by slowing (or stopping) or expanding accrual, imbalancing randomization to favour better-performing therapies, dropping or adding treatment arms, and changing the trial population to focus on patient subsets that are responding better to the experimental therapies. Bayesian analyses use available patient-outcome information, including biomarkers that accumulating data indicate might be related to clinical outcome. They also allow for the use of historical information and for synthesizing the results of relevant trials. (27)

Berry (1993, 2006), Lesaffre and Lawson (2012), and Spiegelhalter et al. (1994) show how Bayesian methods can be implemented, removing one of the more troubling impediments (computational difficulty) to their routine use. Spiegelhalter et al. (1994) also present the practical case for the use of Bayesian methods in clinical trials.While many of the traditional/frequentist versus Bayesian approach arguments have centered on ideological issues (such as the use of personal probability and prior distributions), they show that answers to many relevant, clinically important questions are available only if one takes the Bayesian approach.A detailed description of the Bayesian approach is beyond the scope of the present paper, but we believe that approach to be in every way superior not only in the context of CER, but in all aspects of statistical inference.We should also mention that the ethical dimension of the Bayesian approach has been well developed (see, for example, Berry 2004 and Kadane 1996).

spring 2014 • volume 57, number 2

243

Charles Joseph Kowalski and Adam Joel Mrdjenovich

In terms of how Bayesian approaches might work in practice, if interventions A and B are legitimate candidates for CER, considerable background knowledge concerning the efficacy of A and B is already available. For example, A and B may be FDA-approved drugs, each of which has surpassed the placebo threshold for efficacy; the question now shifts to comparative effectiveness—which of A or B should be used in day-to-day clinical practice? This information is not already available from the preliminary efficacy trials, but the (efficacy) information that is available should not be ignored in the effectiveness comparison. The Bayesian approach allows formal incorporation of this prior information whereas traditional approaches either ignore this background altogether, or let it influence design choices in unspecified, arbitrary ways.

Conclusion The study designs promoted in this paper as being most appropriate for CER, especially the observational and Bayesian approaches, do not fare well when considered relative the EBM hierarchy of evidence that confers gold standard status to the RCT and meta-analyses of several RCTs. This is not surprising, in that RCTs are aimed at efficacy (internal validity), while CER focuses on effectiveness (external validity). Study designs properly mirror study objectives (Kowalski and Mrdjenovich 2013a). Deciding which of two (admittedly) efficacious interventions should be used in everyday clinical practice requires a different approach than establishing efficacy in the first place.

References Agency for Healthcare Research and Quality (AHRQ). 2014. Section 1013: Priority Topics for Research: Medicare Prescription Drug, Improvement, and Modernization Act of 2003. October 2006. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/policymakers/section-1013.html. Alexander, G. Caleb, and Randall S. Stafford. 2009. “Does Comparative Effectiveness Have a Comparative Edge?” JAMA 301: 2488–90. American Psychological Association (APA). 2010. “APA Calls for Psychological Expertise in Oversight of Comparative Effectiveness Research.” http://www.apa.org/about/gr/ science/spin/2010/04/expertise.aspx. American Psychological Association (APA) Presidential Task Force. 2006. “Evidence-Based Practice in Psychology.” Am Psychol 61 (4): 271–85. Arnold, Denis G., ed. 2009. Ethics and the Business of Biomedicine. New York: Cambridge University Press. Ashton, Carol M., and Nelda R.Wray. 2013. Comparative Effectiveness Research: Evidence, Medicine and Policy. New York: Oxford University Press. Avorn, Jerry. 2009. “Debate About Funding Comparative-Effectiveness Research.” N Engl J Med 360 (19): 1927–29.

244

Perspectives in Biology and Medicine

Comparative Effectiveness Research Berger, Marc L., and David Grainger. 2010. “Comparative Effectiveness Research:The View from a Pharmaceutical Company.” Pharmacoeconomics 28 (10): 915–22. Berger, Marc L., et al. 2009. “Good Research Practices for Comparative Effectiveness Research: Defining, Reporting and Interpreting Nonrandomized Studies of Treatment Effects Using Secondary Data Sources. The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part I.” Value Health 12 (8): 1044–52. Berry, Donald A. 1993. “The Case for Bayesianism in Clinical Trials.” Stat Med 12 (15–16): 1377–93. Berry, Donald A. 2004. “Bayesian Statistics and the Efficiency and Ethics of Clinical Trials.” Stat Sci 19 (1): 175–87. Berry, Donald A. 2006. “Bayesian Clinical Trials.” Nat Rev Drug Discov 5 (1): 27–36. Black, Nick. 1996. “Why We Need Observational Studies to Evaluate the Effectiveness of Health Care.” BMJ 312 (7040): 1215–18. Borgerson, Kirstin. 2013. “Are Explanatory Trials Ethical? Shifting the Burden of Justification in Clinical Trial Design.” Theor Med Bioeth 34 (4): 293–308. Brill, Steven. 2013.“Bitter Pill: How Outrageous Pricing and Egregious Profits Are Destroying Our Health Care.” Time, March 4. Centers for Disease Control and Prevention (CDC). 2012. “Leading Causes of Death.” http://www.cdc.gov/nchs/fastats/lcod.htm. Chambless, Dianne L., et al. 2007. “An Update on Empirically Validated Therapies.” http:// www.apa.org/divisions/div12/est/newrpt.pdf. Chen, Pauline W. 2009. “A Tool to Strengthen the Doctor-Patient Relationship.” Hastings Cent Rep 39 (16): 15–17. Concato, John, et al. 2010. “Comparative Effectiveness Research: What Kind of Studies Do We Need?” J Investig Med 58 (16): 764–79. Deyo, Richard A., et al. 1997. “The Messenger Under Attack: Intimidation of Researchers by Special Interest Groups.” N Engl J Med 336 (16): 1176–80. Doyle, John J. 2011. “The Effect of Comparative Effectiveness Research on Drug Development Innovation: A 360o Value Appraisal.” Comparative Effectiveness Res 23: 27–34. Dreyer, Nancy A., et al. 2010. “GRACE Principles: Recognizing High-Quality Observational Studies of Comparative Effectiveness.” Am J Manag Care 16 (16): 467–71. Eden, Jill, et al., eds. 2008. Knowing What Works in Health Care: A Roadmap for the Nation. Washington, DC: National Academies Press. Epstein, Robert S., and J. Russell Teagarden. 2010. “Comparative Effectiveness Research and Personalized Medicine: Catalyzing or Colliding?” Pharmacoeconomics 28 (10): 906–13. Evans, Imogen, et al. 2011. Testing Treatments: Better Research for Better Healthcare. 2nd ed. London: Pinter & Martin. Garattini, Silvio, and Vittorio Bertele. 2007. How can we regulate medicines better? BMJ 335 (7624): 803–5. Garber, Ian M., and Sean R.Tunis. 2009.“Does Comparative-Effectiveness Research Threaten Personalized Medicine?” N Engl J Med 360: 1925–27. Giffin, Robert B., and Janet Woodcock. 2010. “Comparative Effectiveness Research: Who Will Do the Studies?” Health Aff 29 (11): 2075–81. Gilbert, Susan. 2009. “The Nesting-Egg Problem:Why Comparative Effectiveness Research Is Trickier Than It Looks.” Hastings Cent Rep 39 (6): 11–14. Goldacre, Ben. 2009. Bad Science. London: Harper Perennial.

spring 2014 • volume 57, number 2

245

Charles Joseph Kowalski and Adam Joel Mrdjenovich Gottlieb, Scott. 2009. “Promoting and Using Comparative Research:What Are the Promises and Pitfalls of a New Federal Effort?” Am Enterprise Inst Public Policy Res 2: 1–8. Gottlieb, Scott. 2011. “The FDA Should Not Mandate Comparative Effectiveness Trials.” Am Enterprise Inst Public Policy Res 5: 1–9. Gray, Bradford H., Michael K. Gusamo, and Sara R. Collins. 2003. “AHCPR and the Changing Politics of Health Services Research.” Health Aff (Suppl.Web Exclusives):W3– 283–307. Institute of Medicine (IOM). 2001. Health and Behavior:The Interplay of Biological, Behavioral, and Societal Influences. Washington, DC: National Academies Press. Institute of Medicine (IOM). 2009. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: National Academies Press. Kadane, Joseph B., ed. 1996. Bayesian Methods and Ethics in a Controlled Clinical Trial. New York: Wiley. Kowalski, Charles J. 2010. “Pragmatic Problems with Clinical Equipoise.” Perspect Biol Med 53 (2): 161–73. Kowalski, Charles J. 2013. “When Ethics Precludes Randomization: Put Prospective, Matched-Pair, Observational Studies to Work.” Perspect Biol Med 56 (2): 18–35. Kowalski, Charles J., and Adam J. Mrdjenovich. 2013a. “Patient Preference Clinical Trials: Why and When They Will Sometimes Be Preferred.” Perspect Biol Med 56 (1): 18–35. Kowalski, Charles J., and Adam J. Mrdjenovich. 2013b. “Studying Group Behavior: Cluster Randomized Trials.” Am J Clin Exp Med 1 (1): 5–15. Kowalski, Charles J., and Adam J. Mrdjenovich. n.d. “Pharmacovigilance Observed: Why Watchful Waiting Will Work.” J Clin Diagn Res, in press. Kowalski, Charles J., Steven Pennell, and Amiram Vinokur. 2008. “Felicitometry: Measuring the ‘Quality’ in Quality of Life.” Bioethics 22 (6): 307–13. Kowalski, Charles J., et al. 2012. “Felicitometric Hermeneutics: Interpreting Quality of Life Measurements.” Theor Med Bioethics 33: 207–20. Lauer, Michael S., and Francis S. Collins. 2010. “Using Science to Improve the Nation’s Health System: NIH’s Commitment to Comparative Effectiveness Research.” JAMA 303 (21): 2182–83. Lesaffre, Emmanuel, and Andrew B. Lawson. 2012. Bayesian Biostatistics. New York: Wiley. Luce, Bryan R., et al. 2009.“Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change.” Ann Intern Med 151: 206–9. Manchikanti, Laxmaiah, et al. 2010a. “Facts, Fallacies, and Politics of Comparative Effectiveness Research. Part 1: Basic Considerations.” Pain Physician 13 (1): E23–E54. Manchikanti, Laxmaiah, et al. 2010b. “Facts, Fallacies, and Politics of Comparative Effectiveness Research. Part 2: Implications for Interventional Pain Management.” Pain Physician 13 (1): E55–E79. Manchikanti, Laxmaiah, et al. 2011. “The Impact of Comparative Effectiveness Research on Interventional Pain Management: Evolution from Medicare Modernization Act to Patient Protection and Affordable Care Act and the Patient Centered Outcomes Research Institute.” Pain Physician 14 (3): E249–E282. Marko, Nicholas F., and Robert J. Weil. 2010. “The Role of Observational Investigations in Comparative Effectiveness Research.” Value Health 13 (8): 989–97. Meek, Patrick D., Amy C. Renaud-Mutart, and Leon E. Cosler. 2013. “Comparative Effectiveness.” In Understanding Health Outcomes and Pharmacoeconomics, ed. G. E. MacKinnon III, 195–206. Burlington, MA: Jones & Bartlett Learning. 246

Perspectives in Biology and Medicine

Comparative Effectiveness Research Mokdad, Ali H., et al. 2004. “Actual Causes of Death in the United States, 2000.” JAMA 291 (10): 1238–45. Naik, Aanand D., and Laura A. Petersen. 2009. “The Neglected Purpose of Comparative-Effectiveness Research.” N Engl J Med 360 (19): 1929–31. National Institutes of Health (NIH). 2004. “State Implementation of Evidence Based Practices: Bridging Science And Service.” http://grants.nih.gov/grants/guide/rfa-files/RFAMH-03-007.html. National Institute of Mental Health (NIMH). 2013a. “Statistics.” http://www.nimh.nih. gov/health/publications/the-numbers-count-mental-disorders-in-america/index.shtml. National Institute of Mental Health (NIMH). 2013b. “Systematic Treatment Enhancement Program for Bipolar Disorder.” http://www.nimh.nih.gov/funding/clinical-trials-for-researchers/practical/step-bd/index.shtml. Nellesen, Dave, Howard G. Birnbaum, and Paul E. Greenberg. 2010. “Perspectives on Comparative Effectiveness Research: Views from Diverse Constituencies.” Pharmacoeconomics 28 (10): 789–98. Neumann, Peter J., and Milton C.Weinstein. 2010. “Legislating Against the Use of Cost-Effectiveness Information.” N Engl J Med 363 (16): 1495–97. Park, Robert L. 2000. Voodoo Science: The Road from Foolishness to Fraud. New York: Oxford University Press. Patient-Centered Outcomes Research Institute (PCORI). 2014. http://www.pcori.org. Payne, Richard. 2009. “The Quality Mantra: Proceed Carefully.” Hastings Cent Rep 39 (6): 14–15. Pearson, Steven D., and Peter B. Bach. 2010. “How Medicare Could Use Comparative Effectiveness Research in Deciding on New Coverage and Reimbursement.” Health Aff 29 (10): 1796–1804. Peto, Richard, and Colin Baigent. 1998. “Trials:The Next 50 Years. Large Scale Randomised Evidence of Moderate Benefits.” BMJ 317 (7167): 1170–71. Rogers, Mary A. M. 2014. Comparative Effectiveness Research. New York: Oxford University Press. Rosenheck, Robert A., et al. 2006. “Cost-Effectiveness of Second-Generation Antipsychotics and Perphenazine in a Randomized Trial of Treatment for Chronic Schizophrenia.” Am J Psychiatry 163 (12): 2080–89. Rothwell, Peter M. 2007. Treating Individuals: From Randomized Trials to Personalized Medicine. New York: Elsevier. Rush, A. 2007. “STAR*D: What Have We Learned?” Am J Psychiatry 164 (2): 201–4. Sabin, James E., et al. 2008. “Comparing Drug Effectiveness at Health Plans: The Ethics of Cluster Randomized Trials.” Hastings Cent Rep 38 (5): 39–48. Sackett, David L., et al. 1997. Evidence-Based Medicine. New York: Churchill Livingstone. Saver, Richard S. 2011. “Health Care Reform’s Wild Card: The Uncertain Effectiveness of Comparative Effectiveness Research.” U Penn Law Rev 159: 2147–2207. Schechter, Alan N., and Robert L. Perlman. 2009. “Evidence-Based Medicine Again.” Perspect Biol Med 52 (2): 161–63. Schmidt, Harald, and Julia Kreis. 2009. “Lessons from Abroad.” Hastings Cent Rep 39: 20–22. Sox, Harold C., and Sheldon Greenfield. 2009. “Comparative Effectiveness Research: A Report from the Institute of Medicine.” Ann Intern Med 151 (3): 203–5.

spring 2014 • volume 57, number 2

247

Charles Joseph Kowalski and Adam Joel Mrdjenovich Spiegelhalter, David J., Laurence S. Freedman, and Mahesh K. Parmar. 1994. “Bayesian Approaches to Randomized Trials.” JR Stat Soc A 157: 357–416. Stafford Randall S., Todd H. Wagner, and Philip W. Lavori. 2009. “New, but Not Improved? Incorporating Comparative-Effectiveness Information into FDA Labeling.” N Engl J Med 361 : 1230-33. Susser, Ezra, et al. 2006, Psychiatric Epidemiology: Searching for the Causes of Mental Disorders. New York: Oxford University Press. Tanenbaum, Sandra J. 2005. “Evidence Based Practices Mental Health Policy.” Health Aff 24 (1): 163–73. Tonelli, Mark R. 2001. “The Limits of Evidence-Based Medicine.” Respir Care 46 (12): 1435–40. Thorpe, Kevin E., et al. 2009. “A Pragmatic-Explanatory Continuum Indicator Summary (PRECIS): A Tool to Help Trial Designers.” CMAJ 180 (10): E47–E57. Tunis, Sean R., David B. Stryer, and Carolyn M. Clancy. 2003. “Practical Clinical Trials: Increasing the value of Clinical Research for Decision Making in Clinical and Health Policy.” JAMA 290 (12): 1624–32. Vernon, John A., Joseph H. Golec, and J.Stedman Stevens. 2010. “Comparative Effectiveness Regulations and Pharmaceutical Innovation.” Pharmacoeconomics 28 (10): 877–87. von Elm, Erik, et al. 2007. “The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies.” Ann Intern Med 147 (8): 573–77. Wang, Philip S., Christine M. Ulbricht, and Michael Schoenbaum. 2009. “Improving Mental Health Treatments Through Comparative Effectiveness Research.” Heath Aff 28 (3): 783–91. Wennberg, John E. 2010. Tracking Medicine: A Researcher’s Quest to Understand Health Care. New York: Oxford University Press. Whiteford, Harvey A., et al. 2013. “Global Burden of Disease Attributable to Mental and Substance Use Disorders: Findings from the Global Burden of Disease Study.” Lancet 382 (9904): 1575–86. Wilensky, Gail R. 2008. “Cost-Effectiveness Information: Yes, It’s Important, but Keep It Separate, Please!” Ann Intern Med 148 (12): 967–68. Zwarenstein, Merrick, et al. 2008. “Improving the Reporting of Pragmatic Trials: An Extension of the CONSORT Statement.” BMJ 337: a2390.

248

Perspectives in Biology and Medicine

Copyright of Perspectives in Biology & Medicine is the property of Johns Hopkins University Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Comparative effectiveness research: decision-based evidence.

In the clinical research context, comparative effectiveness research (CER) refers to the comparison of several health-care interventions administered ...
548KB Sizes 2 Downloads 8 Views