EDITORIAL For reprint orders, please contact: [email protected]
Comparative effectiveness research and the regulation of drugs, biologics and devices There is a general agreement that medical care will achieve the best results for patients when informed by a strong evidence base. Too often, intuition, anecdote and traditional approaches have been proven wrong when put to the test. The need for more reliable information to guide clinical decision-making is the basis of the current drive for comparative effectiveness research (CER) – that is, evaluation of the real-world outcomes of clinical interventions. The establishment of the Patient-Centered Outcomes Research Institute, as part of the Affordable Care Act, embodied these objectives in legislation [101] . It is hoped that a robust program of CER will be undertaken in the USA. This naturally raises the question: how will the new CER enterprise intersect with the development processes and regulatory framework that currently exist for medical products? In the following discussion, pharmaceuticals will be used as the prototypical medical product, but the concepts will also pertain to biological products and medical devices. Ironically, although the evidentiary base supporting the use of drugs, biologics and many devices is much more robust than for most other medical activities, there is a great deal of criticism regarding the ability of the medical product development process to generate adequate information. This results, in part, from the fact that medical product development is a distinct, highly regulated, observable process, carried out by commercial entities, with a clear set of deliverables at the end. By contrast, no formal process exists through which other types of medical interventions are introduced and evaluated before widespread use. Additionally, regulators typically focus on the quality or robustness of the evidence, rather than the number of questions that are answered. Although keenly aware of residual uncertainties, regulators aim for a very low probability of making a wrong decision, since their actions frequently have draconian medical and financial consequences, and are subject to legal and other challenges. The tolerance for (and recognition of the probability of) error is probably the greatest divide separating the CER enterprise and the current framework for medical product regulation [1] . Faced with a nearly infinite universe of unanswered clinical questions, researchers will naturally pursue the most expedient ways of getting answers. The tradeoffs between expediency and accuracy must be further defined. So how will the drug development enterprise fit into the evolving CER enterprise (or vice versa)? First, it may be useful to describe the current drug development process, for which many misconceptions persist. Clinical drug development can be defined as the progressive reduction of uncertainty about the medical properties of a candidate drug (i.e., an investigational drug not yet approved). In progressive steps, more is learned about human pharmacology, side effects and drug efficacy. In the early stages of clinical development, when these properties are being explored for the first time in patients, the issue of comparative performance is not ordinarily of greatest interest. As efficacy is more formally explored in randomized trials, choice of comparator is dictated by specific clinical circumstances, as discussed below.
10.2217/CER.13.9
2(2), 95–97 (2013)
Janet Woodcock*
“Comparative effectiveness research is a logical and necessary consequence of the need for continuous improvement in medicine.”
*Center for Drug Evaluation & Research, US FDA, 10903 New Hampshire Avenue, Silver Spring, MD 20993, USA Tel.: +1 301 796 5400 [email protected]
part of
ISSN 2042-6305
95
EDITORIAL Woodcock In serious or life-threatening diseases, new drugs are generally studied against placebo if no therapy exists, or as add-on-to, or against, an existing standard-of-care. Therefore, many of these development programs have an active comparator arm as part of the program. If the intent is to equal, rather than improve upon, standard-of-care (as may be the case in antibiotic drug development), trials are statistically designed as ‘noninferiority’ trials in which the loss of a certain percentage of the effect of standard-ofcare is ruled out. Such trials generally require larger numbers of subjects and outcome events to be interpretable, compared with placebocontrolled or superiority trials. As a result of these practices, many drugs intended to treat cancer, acute heart disease, serious infections and other life-threatening illnesses are studied in comparative trials before marketing, usually against what was felt to be the optimal therapy at the time the trial was designed. The situation is different for less serious diseases, particularly acute symptomatic disorders (e.g., migraine, dysmenorrhea and motion sickness). Regardless of the alternatives, investigational drugs to treat such conditions are often studied against placebo, to best define their effects. Often, numerous other interventions are available for the condition, some of which may have been studied in ways other than randomized trials (or never studied). Investigational drugs intended to treat chronic diseases are usually evaluated in trials that are larger and longer than those used for short-term symptomatic diseases. Such trials are usually comparative, unless no alternate therapy is available. Sometimes, for example, in rheumatoid arthritis, short-term, placebo-controlled trials may be performed in conjunction with longer term active-controlled trials to gain as much information as possible about the new agent. In many diseases (e.g., diabetes), information about efficacy can be learned fairly quickly, but information on longer term safety and durability of response requires much longer trials to be carried out. These trials are often comparative. At the end of these various development programs, if a drug is approved for marketing, there is firm evidence that the drug is effective (as defined by the various trial end points, often including comparisons with one or more approved drugs) – enough is known about the drug’s safety profile to make a rational benefit–risk assessment, and there is substantial
96
J. Compar. Effect. Res. (2013) 2(2)
knowledge about the drug’s pharmacology, including dosing and drug interaction potential. Nevertheless, clinicians are often frustrated by the significant questions that remain unanswered about how to use a new drug in clinical practice [2] . Questions often arise about initial or sequential choice of therapy versus other drugs or interventions for the condition (a common subject of inquiry for CER); which patients are best candidates for the drug; longterm outcomes, both safety and efficacy; how long to treat or when to stop use in patients with multiple other chronic conditions; use in combination with other agents approved for the condition, and so forth. Additionally, as other new medical interventions become available, questions about comparisons or combinations with these modalities arise. Finally, practitioners want to know how the new agents perform in the real world of clinical practice. Clearly, generation of the above information must be an ongoing project, one that is under taken in settings where the various interventions are actually employed. The clinical research methods used in medical product development are too costly for use in most CER, and their results may lack adequate generalizability to the practice setting. Thus, new methods for obtaining reliable information must be brought to bear. These methods may not need to be as robust as those used for regulatory purposes; however, they must yield results that are significantly more reliable than the anecdote, intuition and traditional approaches that have served us poorly in the past. A tradeoff between reliability and feasibility is the likely evolving point of friction between the CER and product development enterprises [3,4] . There is great hope that the emerging availability of electronic health records will provide a way out of this dilemma. Although these records represent unprecedented sources of patient data, they will not enable all questions to be easily and definitively answered. The US FDA has created, under contract, a drug safety evaluation system using claims data and electronic health records, known as the MiniSentinel System [5] . The Mini-Sentinel contains records on approximately 159 million lives, and the FDA is currently using the system to help evaluate drug safety signals. A typical question would be: ‘Is exposure to drug × associated with an increased incidence of safety event y?’ In addressing this question, it must be asked:
future science group
Comparative effectiveness research & the regulation of drugs, biologics & devices
‘Increased relative to what?’ This often involves comparisons with other treatment modalities. Early experience with this system is promising; rapid results can be obtained to supplement other sources of information. However, owing to the nonrandomized nature of the data, the answers thus obtained are by no means definitive and often serve to raise or lower concerns, rather than settle an issue. To be able to fully exploit the promise of electronic health data, clinical research and medical practice must become more unified. The community of practice must be invited to participate in the study of medical care. Since CER by definition will study real-world settings, it provides a natural opportunity to engage practitioners. CER should not simply be research done to healthcare; it should be performed as a part of healthcare by medical practitioners. CER is a logical and necessary consequence of the need for continuous improvement in medicine. Integration of research into medical practice could have beneficial consequences for both CER and medical product development. Practice-centered research could easily allow for randomization among various acceptable References 1
2
3
Temple R. A regulator’s view of comparative effectiveness research. Clin. Trials 9, 56–65 (2012). Tricoci P, Allen JM, Kramer JM et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 301(8) 831–841 (2009). Woodcock J. Evidence vs. access: can twenty-first-century drug regulation refine the
future science group
interventions, with outcomes captured in electronic records. Such randomized outcome studies could be robust enough for regulatory use and could (with appropriate support by product sponsors) help fill the gaps left by the current development processes. For example, such studies could provide highly relevant data on longer-term safety, use in heterogeneous populations, or duration of use. Real-world trials could also answer questions about the optimal sequence or combination of therapies or other interventions. Clearly, the integration of research and practice offers the best opportunity for generating the data needed for evidence-based medicine and informing medical product development. Financial & competing interests disclosure The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. No writing assistance was utilized in the production of this manuscript. tradeoffs? Clin. Pharmacol. Ther. 91(3), 378–380 (2012).
4
EDITORIAL
Institute of Medicine, Forum on Drug Discovery, Development and Translation. Envisioning a Transformed Clinical Trials Enterprise in the United States; Establishing an Agenda for 2012. Workshop Summary. The National Academies Press, Washington, DC, USA (2012).
www.futuremedicine.com
5
Behrman RE, Benner JS, Brown JS et al. Developing the Sentinel System – a national resource for evidence development. N. Engl. J. Med. 364, 498–499 (2011).
■■ Website 101 Patient Protection and Affordable Care Act,
Public Law 111–148. www.gpo.gov/fdsys/pkg/BILLS111hr3590enr/pdf/BILLS-111hr3590enr.pdf (Accessed 9 January 2013)
97
Comparative effectiveness research and the regulation of drugs, biologics and devices There is a general agreement that medical care will achieve the best results for patients when informed by a strong evidence base. Too often, intuition, anecdote and traditional approaches have been proven wrong when put to the test. The need for more reliable information to guide clinical decision-making is the basis of the current drive for comparative effectiveness research (CER) – that is, evaluation of the real-world outcomes of clinical interventions. The establishment of the Patient-Centered Outcomes Research Institute, as part of the Affordable Care Act, embodied these objectives in legislation [101] . It is hoped that a robust program of CER will be undertaken in the USA. This naturally raises the question: how will the new CER enterprise intersect with the development processes and regulatory framework that currently exist for medical products? In the following discussion, pharmaceuticals will be used as the prototypical medical product, but the concepts will also pertain to biological products and medical devices. Ironically, although the evidentiary base supporting the use of drugs, biologics and many devices is much more robust than for most other medical activities, there is a great deal of criticism regarding the ability of the medical product development process to generate adequate information. This results, in part, from the fact that medical product development is a distinct, highly regulated, observable process, carried out by commercial entities, with a clear set of deliverables at the end. By contrast, no formal process exists through which other types of medical interventions are introduced and evaluated before widespread use. Additionally, regulators typically focus on the quality or robustness of the evidence, rather than the number of questions that are answered. Although keenly aware of residual uncertainties, regulators aim for a very low probability of making a wrong decision, since their actions frequently have draconian medical and financial consequences, and are subject to legal and other challenges. The tolerance for (and recognition of the probability of) error is probably the greatest divide separating the CER enterprise and the current framework for medical product regulation [1] . Faced with a nearly infinite universe of unanswered clinical questions, researchers will naturally pursue the most expedient ways of getting answers. The tradeoffs between expediency and accuracy must be further defined. So how will the drug development enterprise fit into the evolving CER enterprise (or vice versa)? First, it may be useful to describe the current drug development process, for which many misconceptions persist. Clinical drug development can be defined as the progressive reduction of uncertainty about the medical properties of a candidate drug (i.e., an investigational drug not yet approved). In progressive steps, more is learned about human pharmacology, side effects and drug efficacy. In the early stages of clinical development, when these properties are being explored for the first time in patients, the issue of comparative performance is not ordinarily of greatest interest. As efficacy is more formally explored in randomized trials, choice of comparator is dictated by specific clinical circumstances, as discussed below.
10.2217/CER.13.9
2(2), 95–97 (2013)
Janet Woodcock*
“Comparative effectiveness research is a logical and necessary consequence of the need for continuous improvement in medicine.”
*Center for Drug Evaluation & Research, US FDA, 10903 New Hampshire Avenue, Silver Spring, MD 20993, USA Tel.: +1 301 796 5400 [email protected]
part of
ISSN 2042-6305
95
EDITORIAL Woodcock In serious or life-threatening diseases, new drugs are generally studied against placebo if no therapy exists, or as add-on-to, or against, an existing standard-of-care. Therefore, many of these development programs have an active comparator arm as part of the program. If the intent is to equal, rather than improve upon, standard-of-care (as may be the case in antibiotic drug development), trials are statistically designed as ‘noninferiority’ trials in which the loss of a certain percentage of the effect of standard-ofcare is ruled out. Such trials generally require larger numbers of subjects and outcome events to be interpretable, compared with placebocontrolled or superiority trials. As a result of these practices, many drugs intended to treat cancer, acute heart disease, serious infections and other life-threatening illnesses are studied in comparative trials before marketing, usually against what was felt to be the optimal therapy at the time the trial was designed. The situation is different for less serious diseases, particularly acute symptomatic disorders (e.g., migraine, dysmenorrhea and motion sickness). Regardless of the alternatives, investigational drugs to treat such conditions are often studied against placebo, to best define their effects. Often, numerous other interventions are available for the condition, some of which may have been studied in ways other than randomized trials (or never studied). Investigational drugs intended to treat chronic diseases are usually evaluated in trials that are larger and longer than those used for short-term symptomatic diseases. Such trials are usually comparative, unless no alternate therapy is available. Sometimes, for example, in rheumatoid arthritis, short-term, placebo-controlled trials may be performed in conjunction with longer term active-controlled trials to gain as much information as possible about the new agent. In many diseases (e.g., diabetes), information about efficacy can be learned fairly quickly, but information on longer term safety and durability of response requires much longer trials to be carried out. These trials are often comparative. At the end of these various development programs, if a drug is approved for marketing, there is firm evidence that the drug is effective (as defined by the various trial end points, often including comparisons with one or more approved drugs) – enough is known about the drug’s safety profile to make a rational benefit–risk assessment, and there is substantial
96
J. Compar. Effect. Res. (2013) 2(2)
knowledge about the drug’s pharmacology, including dosing and drug interaction potential. Nevertheless, clinicians are often frustrated by the significant questions that remain unanswered about how to use a new drug in clinical practice [2] . Questions often arise about initial or sequential choice of therapy versus other drugs or interventions for the condition (a common subject of inquiry for CER); which patients are best candidates for the drug; longterm outcomes, both safety and efficacy; how long to treat or when to stop use in patients with multiple other chronic conditions; use in combination with other agents approved for the condition, and so forth. Additionally, as other new medical interventions become available, questions about comparisons or combinations with these modalities arise. Finally, practitioners want to know how the new agents perform in the real world of clinical practice. Clearly, generation of the above information must be an ongoing project, one that is under taken in settings where the various interventions are actually employed. The clinical research methods used in medical product development are too costly for use in most CER, and their results may lack adequate generalizability to the practice setting. Thus, new methods for obtaining reliable information must be brought to bear. These methods may not need to be as robust as those used for regulatory purposes; however, they must yield results that are significantly more reliable than the anecdote, intuition and traditional approaches that have served us poorly in the past. A tradeoff between reliability and feasibility is the likely evolving point of friction between the CER and product development enterprises [3,4] . There is great hope that the emerging availability of electronic health records will provide a way out of this dilemma. Although these records represent unprecedented sources of patient data, they will not enable all questions to be easily and definitively answered. The US FDA has created, under contract, a drug safety evaluation system using claims data and electronic health records, known as the MiniSentinel System [5] . The Mini-Sentinel contains records on approximately 159 million lives, and the FDA is currently using the system to help evaluate drug safety signals. A typical question would be: ‘Is exposure to drug × associated with an increased incidence of safety event y?’ In addressing this question, it must be asked:
future science group
Comparative effectiveness research & the regulation of drugs, biologics & devices
‘Increased relative to what?’ This often involves comparisons with other treatment modalities. Early experience with this system is promising; rapid results can be obtained to supplement other sources of information. However, owing to the nonrandomized nature of the data, the answers thus obtained are by no means definitive and often serve to raise or lower concerns, rather than settle an issue. To be able to fully exploit the promise of electronic health data, clinical research and medical practice must become more unified. The community of practice must be invited to participate in the study of medical care. Since CER by definition will study real-world settings, it provides a natural opportunity to engage practitioners. CER should not simply be research done to healthcare; it should be performed as a part of healthcare by medical practitioners. CER is a logical and necessary consequence of the need for continuous improvement in medicine. Integration of research into medical practice could have beneficial consequences for both CER and medical product development. Practice-centered research could easily allow for randomization among various acceptable References 1
2
3
Temple R. A regulator’s view of comparative effectiveness research. Clin. Trials 9, 56–65 (2012). Tricoci P, Allen JM, Kramer JM et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 301(8) 831–841 (2009). Woodcock J. Evidence vs. access: can twenty-first-century drug regulation refine the
future science group
interventions, with outcomes captured in electronic records. Such randomized outcome studies could be robust enough for regulatory use and could (with appropriate support by product sponsors) help fill the gaps left by the current development processes. For example, such studies could provide highly relevant data on longer-term safety, use in heterogeneous populations, or duration of use. Real-world trials could also answer questions about the optimal sequence or combination of therapies or other interventions. Clearly, the integration of research and practice offers the best opportunity for generating the data needed for evidence-based medicine and informing medical product development. Financial & competing interests disclosure The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. No writing assistance was utilized in the production of this manuscript. tradeoffs? Clin. Pharmacol. Ther. 91(3), 378–380 (2012).
4
EDITORIAL
Institute of Medicine, Forum on Drug Discovery, Development and Translation. Envisioning a Transformed Clinical Trials Enterprise in the United States; Establishing an Agenda for 2012. Workshop Summary. The National Academies Press, Washington, DC, USA (2012).
www.futuremedicine.com
5
Behrman RE, Benner JS, Brown JS et al. Developing the Sentinel System – a national resource for evidence development. N. Engl. J. Med. 364, 498–499 (2011).
■■ Website 101 Patient Protection and Affordable Care Act,
Public Law 111–148. www.gpo.gov/fdsys/pkg/BILLS111hr3590enr/pdf/BILLS-111hr3590enr.pdf (Accessed 9 January 2013)
97
